|
There is a serious issue with the Gerrit trigger plugin and its "retrigger" functionality.
This issue causes the plugin to get confused about how many and which builds it started for a particular changeset/patchset combination, and decide to post the Verification or Code Review update even if not all test have passed yet.
This behaviour leads to inconsistent verification results in Gerrit and can be easily replicated as follows:
1.) Start a blank, vanilla Jenkins server of an arbitrary version on your local host (the jobs below assume that a BASH shell is available).
2.) Install the gerrit-trigger plugin (arbitrary version, as all are affected)
3.) Increase the executor count of the "master" node to at least 6.
4.) Restart the server and create a connection to a Gerrit service.
5.) Shut down the server and extract the two jobs (Test_1 and Test_2) into the "jobs" directory of the Jenkins server.
5.a ) Test_1 will always fail, Test_2 will always succeed. Both will create a lock-file named "LOCKFILE-${JOB_NAME}-${BUILD_NUMBER}" in "${JENKINS_HOME}/workspace/Central" and wait for it to be deleted, before succeeding or failing.
6.) (Optional) Alter the Test_1 and Test_2 parameters to listen to a specific repository. By default repo and branch are set to "**".
After this set-up is done, upload a patchset to the Gerrit service in question.
This will cause a build of "Test_1" and "Test_2" to be started. Now, while they are running (they are wait until you delete their lock-file), go into the "manual trigger" interface:
http://[jenkins]/gerrit_manual_trigger/?
Search for the changeset and issue a retrigger. This will cause two additional builds to spawn, one for "Test_1" and one for "Test_2".
Now that 4 tests are running, the correct behaviour of the Gerrit trigger plugin is to ignore the first two runs that have been started by the "Patchset Created" event.
Instead, it should wait until BOTH of the retriggered builds have finished and then upload a "Verified -1" message to Gerrit.
To test this, first delete the two original lock files:
${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_1-1
${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_2-1
This tests the first half of the problem. This works fine. After both have finished, the Gerrit Trigger plugin will still wait for the others to finish.
Now, to expose the bug, ONLY delete the lockfile for the retriggered TEST_2 job:
${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_2-2
This will let Test_2 finish successfully (as Test_2 always succeeds). The bug that happens now is, that the trigger plugin uploads a "Verified +1" message, DESPITE "Test_1 #2" not yet being finished.
If you then also let "Test_1" finish, it uploads yet another verification, this time "Verified -1".
This behaviour is absolutely wrong, as it sends a verification state without waiting for all jobs to complete, thus leading to inconsistent results in Gerrit.
Here's an excerpt of how this behaviour looks like in Gerrit:
##############################################################################
Jenkins
Patch Set 1: -Verified
Build Started Test_2/1/ (1/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_1/1/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_2/2/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_1/2/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1: Verified+1
Build Successful
Test_2/2/ : SUCCESS
-------------------------------------------------------
Jenkins
Patch Set 1: Verified-1
Build Failed
Test_1/2/ : FAILURE
##############################################################################
You can see two bugs at work here:
1.) The plugin gets confused when counting the number of jobs started
2.) The plugin does not wait for (Test_1 #2) after (Test_2 #2) has finished.
|
