This section explains the methodology you use when you design and record a test case.
Each test case you record should perform its own setup in stage 1, and should undo this setup in stage 3, so that the test case can be executed independently of every other test case. In other words, the test case should not rely on the successful or unsuccessful completion of another test case, and the order in which it is executed should have no bearing on its outcome.
If a test case relies on a prior test case to perform some setup actions, and an error causes the setup to fail or, worse yet, the application to crash, all subsequent test cases will fail because they cannot achieve the state where the test is designed to begin.
Each test case you record should verify a single aspect of the application in stage 2. When a test case designed in this manner passes or fails, it’s easy to determine specifically what aspect of the target application is either working or not working.
If a test case contains more than one objective, many outcomes are possible. Therefore, an exception may not point specifically to a single failure in the software under test but rather to several related function points. This makes debugging more difficult and time-consuming and leads to confusion in interpreting and quantifying results. The net result is an overall lack of confidence in any statistics that might be generated.
There are techniques you can use to do more than one verification in a test case.
In order for a test case to be able to function properly, the application must be in a stable state when the test case begins to execute. This stable state is called the base state. The recovery system is responsible for maintaining the base state in the event the application fails or crashes, either during a test case’s execution or between test cases.
To restore the application to the base state, the recovery system contains a routine called DefaultBaseState that makes sure that:
If these conditions are not sufficient for your application, you can customize the recovery system.
When defining test requirements, the goal is to rigorously test each application feature. To do so, you need to decide which set of inputs to a feature will provide the most meaningful test results.