|
Description:
|
Refer to AlmTestSetsRunner.cs file located here: https://github.com/jenkinsci/hp-application-automation-tools-plugin/blob/master/HpToolsLauncher/Runners/AlmTestSetsRunner.cs
The issue is in the RunTestSet function.
1. Before running the test set, a stopwatch timer variable testSw is initialized to null. Line 529.
2. A While loop then runs through each test instance in the test set. The While loop is on line 548.
a. Before each test instance is executed, the stopwatch is started, and that testSw variable is set to “Stopwatch.StartNew()”. This occurs on line 595.
3. Unfortunately, as each test instance is completed, the current value of the stopwatch is not saved/stored.
a. The stopwatch is reset, the next test instance starts, and the previous stopwatch value is lost.
4. When the last test instance in the test set is completed, the code exits the While loop.
a. At that point, the testSw variable contains the duration of the last test instance.
b. For our test run, the last test instance took 4 minutes 20 seconds.
5. Next, on line 655, the code loops through all test instances to consolidate information for each test instance.
a. The Runtime for each test instance is set to testSw.Elapsed on line 672.
b. Remember, that the testSw value is currently set to the duration of the last test instance … 4 minutes 20 seconds for our example.
c. Also, remember that the stopwatch is still running. A stop command was never executed.
d. So, the duration for the first test instance is actually getting set to the duration of the last test instance.
e. Each subsequent test instance duration is getting set to the duration of the last test instance plus some small additional time because the stopwatch is still running and updating testSw.
f. That explains why our durations start at 4 minutes 20 seconds, and gradually increase to 4 minutes 48 seconds. It took 28 seconds to loop through the test instances in this FOR loop.
6. The incorrect Total Runtime in the upper right corner of the All Tests page is just the sum of these incorrect test instance durations.
a. Fix the test instance problem, and the Total Runtime problem will automatically be fixed.
This code needs an update to fix this test duration issue.
Thanks,
Keith
|
