Configuration Suite Status

Problem

Configuration suites allow you to execute the same set of tests against multiple configurations, for example multiple browsers or operating systems. To be able to make reasonable statements related to quality and reliability of your application under test you will want to keep track of the results for each individual configuration.

Solution

Use the data mart view RV_ConfigurationSuiteStatus to create a report that returns the passed, failed, and not executed counts for each configuration per build.

SELECT BuildName, ConfigurationName, PassedCount, FailedCount, NotExecutedCount
FROM RV_ConfigurationSuiteStatus 
WHERE ConfigurationSuiteID = ${configSuiteID|97|Configuration Suite ID}
ORDER BY BuildOrderNumber, ConfigurationName
The query does the following:
  • Retrieves the status counts per build of test runs from the RV_ConfigurationSuiteStatus view.
  • Narrows the results down to the configuration suite (ConfigurationSuiteID).
The result of the SQL query shows the status of your test runs for each configuration.
BuildName ConfigurationName PassedCount FailedCount NotExecutedCount
350 Chrome 0 1 0
350 Firefox 0 1 0
350 Internet Explorer 0 1 0
351 Chrome 1 0 0
351 Firefox 1 0 0
351 Internet Explorer 0 1 0
352 Chrome 1 0 0
352 Firefox 1 0 0
352 Internet Explorer 1 0 0
In this example, we use the ID of the configuration suite to get all configurations. It is also possible to restrict the result to specific builds, in which case you would have to include BuildID, BuildName, or BuildOrderNumber in the where clause.
Note: The view RV_ConfigurationSuiteStatus only contains aggregated status counts without any test-specific data. To retrieve additional test-specific data, you can use, for example, the view RV_LatestTestStatus.