In a continuous integration environment tests are ideally executed at least once per day for testing the daily build and ensuring the quality of your application under test. To understand how reliable your test set is for measuring the quality of your AUT it is inevitable to sometimes have a look at how the results change over time. For example you might have tests in your test set that frequently change status, therefore being no real measure for quality.
Use the data mart view RV_TestStatusExtended to create a report that lists the results for a specific test in the context of a specific execution plan. This allows you to see how this test’s results have changed over time. For convenience, we will narrow the list of results down to those related to tagged builds, thus looking at specific milestone builds of the application under test only. This report collects test result data for tests in the context of execution plans and builds. In the following query we:
SELECT TestName, ExecutionPlanName, VersionName, BuildName, TestRunID, PassedCount, FailedCount, NotExecutedCount FROM RV_TestStatusExtended WHERE TestID = ${TESTID|1|Test ID} AND ExecutionPlanID = ${EXECUTIONPLANID|1|Execution Plan ID} AND BuildIsTagged = 1 ORDER BY BuildOrderNumber
TestName | Execution PlanName | Version Name | BuildName | TestRunID | Passed Count | FailedCount | NotExecutedCount |
---|---|---|---|---|---|---|---|
UI Tests | EN|SQL2012|IE9|IIS | 3.0 | 579_Drop02 | 7741797 | 59 | 5 | 0 |
UI Tests | EN|SQL2012|IE9|IIS | 3.0 | 579_Drop02 | 7745078 | 63 | 1 | 0 |
UI Tests | EN|SQL2012|IE9|IIS | 3.0 | 593_Drop03 | 7787437 | 63 | 1 | 0 |
UI Tests | EN|SQL2012|IE9|IIS | 3.0 | 605_Drop04 | 7848720 | 63 | 1 | 0 |