Stress Testing

Stress testing deals with the quality of the application in the environment. The idea is to create an environment more demanding of the application than the application would experience under normal work loads. This is the hardest and most complex category of testing to accomplish and it requires a joint effort from all teams.

A test environment is established with many testing stations. At each station, a script is exercising the system. These scripts are usually based on the regression suite. More and more stations are added, all simultaneous hammering on the system, until the system breaks. The system is repaired and the stress test is repeated until a level of stress is reached that is higher than expected to be present at a customer site.

Race conditions and memory leaks are often found under stress testing. A race condition is a conflict between at least two tests. Each test works correctly when done in isolation. When the two tests are run in parallel, one or both of the tests fail. This is usually due to an incorrectly managed lock.

A memory leak happens when a test leaves allocated memory behind and does not correctly return the memory to the memory allocation scheme. The test seems to run correctly, but after being exercised several times, available memory is reduced until the system fails.

IT Industry

In software testing, stress testing often refers to tests that put a greater emphasis on robustness, availability, and error handling under a heavy load, rather than on what would be considered correct behavior under normal circumstances. In particular, the goals of such tests may be to ensure the software doesn't crash in conditions of insufficient computational resources (such as memory or disk space), unusually high concurrency, or denial of service attacks.

Examples:

• A web server may be stress tested using scripts, bots, and various denial of service tools to observe the performance of a web site during peak loads.

Financial sector

• Instead of doing financial projection on a "best estimate" basis, a company may do stress testing where they look at how robust a financial instrument is in certain crashes. They may test the instrument under, for example, the following stresses:
  • What happens if the market crashes by more than x% this year?
  • What happens if interest rates go up by at least y%?
  • What if half the instruments in the portfolio terminate their contacts in the 5th year?
  • What happens if oil prices rise by 200%?

This type of analysis has become increasingly widespread, and has been taken up by various governmental bodies (such as the FSA in the UK) as a regulatory requirement on certain financial institutions to ensure adequate capital allocation levels to cover potential losses incurred during extreme, but plausible, events. This emphasis on adequate, risk adjusted determination of capital has been further enhanced by modifications to banking regulations such as Basel II. Stress testing models typically allow not only the testing of individual stressors, but also combinations of different events. There is also usually the ability to test the current exposure to a known historical scenario (such as the Russian debt default in 1998 or 9/11 terrorist attacks) to ensure the liquidity of the institution.