Here is a case where the process worked as desired. The job which failed
[0] was retried [1] after 5 seconds and passed upon retry. It did not
disrupt the voting for the patch [2] :)
Hopefully this will always be the case. The job failure did not show up
in the gerrit log which I think is different from past behavior.
However, based on previous naginator induced voting irregularities, I
suspect that it may cause voting anomalies if the timing is unlucky.
I will continue to monitor for connection resets and voting anomalies,
but so far so good.
Thanks,
-daw-
[0] https://jenkins.fd.io/job/vpp-verify-master-centos8-x86_64/3335/
[1] https://jenkins.fd.io/job/vpp-verify-master-centos8-x86_64/3336/
[2] https://gerrit.fd.io/r/c/vpp/+/32206
On 5/4/2021 1:26 PM, Dave Wallace via lists.fd.io wrote:
Folks,
As a temporary measure to help alleviate the burden of rechecking
gerrit changes and wasting cycles re-running jobs which have already
passed, I have deployed the Naginator Jenkins plugin configuration to
retry VPP jobs which fail with the error signature "Error cloning
remote repo" [0].
You may recall there is a potential for jobs which are restarted after
a failed job to override a -1 vote by a job which failed prior to the
restart.
Please look for this when reviewing the status of gerrit changes prior
to merge as this could break the CI pipeline.
Vexxhost is monitoring the network segments using tcpdump to determine
what device is causing the TCP connection resets and once the issue is
resolved I will remove the Naginator configuration from the VPP job
configuration.
I will continue to closely monitoring job status for connection resets
and also look for any Naginator induced issues with job retries.
Thanks,
-daw-
[0] https://gerrit.fd.io/r/c/ci-management/+/32197
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19327): https://lists.fd.io/g/vpp-dev/message/19327
Mute This Topic: https://lists.fd.io/mt/82583361/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
