On May 8, 2019, at 8:36 AM, Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>> wrote:
OK, ninja build failures are intermittent? ok..havent splinted way back in build history intermittent is not the word I would choose. Other changes are not hitting this issue at all.. This specific change https://gerrit.fd.io/r/#/c/19350/ seems to have a 100% hit rate. I am trying to understand how to isolate the problem. Can it be triggered by doing retest while a current build is in progress? not clear to me what you mean here since there is no make test running for centos here… Ed --Tom On 05/08/2019 10:25 AM, Ed Kern via Lists.Fd.Io wrote: starting new thread because it really has nothing to do with previous thread… So I am not going to write: a. its related to your code b. its not the build infra But I will say there have been good centos verify runs… BEFORE your patches BETWEEN various revisions of your changes BETWEEN retries of your changes AFTER you stopped trying to retry your changes With multiple different patches… So not really having anything to track on the build infra side (nor is this a voting issue) Im starting a new thread and including tom to see if he has any centos guesses on why we are only seeing problems on centos. Ed On May 8, 2019, at 2:46 AM, Yu, Ping <ping...@intel.com><mailto:ping...@intel.com> wrote: Recently centOS build always fail and patch review is blocked even retrying several times. 16:17:11 -- Configuring done 16:17:12 -- Generating done 16:17:12 -- Build files have been written to: /w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root/build-vpp-native/vpp 16:17:12 ninja: error: manifest 'build.ninja' still dirty after 100 tries 16:17:12 16:17:12 make[4]: *** [Makefile:695: vpp-build] Error 1 16:17:12 make[4]: Leaving directory '/w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root' 16:17:12 make[3]: *** [Makefile:931: install-packages] Error 1 16:17:12 make[3]: Leaving directory '/w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root' 16:17:12 error: Bad exit status from /var/tmp/rpm-tmp.8TtZeQ (%build) 16:17:12 16:17:12 16:17:12 RPM build errors: 16:17:12 Bad exit status from /var/tmp/rpm-tmp.8TtZeQ (%build) 16:17:12 make[2]: *** [RPM] Error 1 16:17:12 make[2]: Leaving directory `/w/workspace/vpp-verify-master-centos7/extras/rpm' 16:17:12 make[1]: *** [pkg-rpm] Error 2 16:17:12 make[1]: Leaving directory `/w/workspace/vpp-verify-master-centos7' 16:17:12 make: *** [verify] Error 2 16:17:12 Build step 'Execute shell' marked build as failure -----Original Message----- From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> [mailto:vpp-dev@lists.fd.io] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io Sent: Tuesday, May 7, 2019 4:21 PM To: Ed Kern (ejk) <e...@cisco.com><mailto:e...@cisco.com>; Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) <ksek...@cisco.com><mailto:ksek...@cisco.com> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] false verified + 1 for gerrit jobs reduce the number of retries (or possibly completely) +1 for completely. Vratko. -----Original Message----- From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> <vpp-dev@lists.fd.io><mailto:vpp-dev@lists.fd.io> On Behalf Of Ed Kern via Lists.Fd.Io Sent: Tuesday, 2019-May-07 00:34 To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) <ksek...@cisco.com><mailto:ksek...@cisco.com> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] false verified + 1 for gerrit jobs This is a known issue with how the retry mechanic interacts (badly) with gerrit occasionally. This odds of this happening were a bit higher over the last couple of days specific to centos retries. This is tied to the JNLP changes made to csit and how the initial connection is made. While the change has improved JNLP issues across the board there are a couple new nits that ill have to debug. At this point though I’m not going to touch/debug further until the csit jobs/reports are completed. Im actually hopeful that ill be able to reduce the number of retries (or possibly completely) in the long run with the HAProxy bypass in place. Ed On May 6, 2019, at 11:18 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) <ksek...@cisco.com><mailto:ksek...@cisco.com> wrote: Hi all, I noticed a job getting a +1 even though some of the builds failed ... https://gerrit.fd.io/r/#/c/18444/ please note patch set 9 .... fd.io<http://fd.io> JJB Patch Set 9: Verified-1 Build Failed https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1804/2459/ : FAILURE No problems were identified. If you know why this problem occurred, please add a suitable Cause for it. ( https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1804/2459/ ) Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-arm-verify-mas ter-ubuntu1804/2459 https://jenkins.fd.io/job/vpp-beta-verify-master-ubuntu1804/6891/ : FAILURE No problems were identified. If you know why this problem occurred, please add a suitable Cause for it. ( https://jenkins.fd.io/job/vpp-beta-verify-master-ubuntu1804/6891/ ) Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-beta-verify-ma ster-ubuntu1804/6891 https://jenkins.fd.io/job/vpp-csit-verify-device-master-1n-skx/788/ : SUCCESS (skipped) Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-de vice-master-1n-skx/788 https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/13041/ : SUCCESS Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-make-test-docs -verify-master/13041 https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/19027/ : SUCCESS Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master- ubuntu1604/19027 https://jenkins.fd.io/job/vpp-verify-master-clang/6731/ : SUCCESS Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master- clang/6731 https://jenkins.fd.io/job/vpp-docs-verify-master/15343/ : SUCCESS Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-docs-verify-ma ster/15343 https://jenkins.fd.io/job/vpp-verify-master-centos7/18764/ : NOT_BUILT Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master- centos7/18764 7:11 PM fd.io<http://fd.io> JJB Patch Set 9: Verified+1 Build Successful https://jenkins.fd.io/job/vpp-verify-master-centos7/18764/ : SUCCESS Logs: https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master- centos7/18764 7:15 PM Thanks, Klement -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12956): https://lists.fd.io/g/vpp-dev/message/12956 Mute This Topic: https://lists.fd.io/mt/31543758/675058 Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [therb...@redhat.com<mailto:therb...@redhat.com>] -=-=-=-=-=-=-=-=-=-=-=- -- Thomas F Herbert NFV and Fast Data Planes Networking Group Office of the CTO Red Hat
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12958): https://lists.fd.io/g/vpp-dev/message/12958 Mute This Topic: https://lists.fd.io/mt/31543758/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
