OK,  ninja build failures are intermittent?

I am trying to understand how to isolate the problem.

Can it be triggered by doing retest while a current build is in progress?

--Tom

On 05/08/2019 10:25 AM, Ed Kern via Lists.Fd.Io wrote:
starting new thread because it really has nothing to do with previous thread…

So I am not going to write:
a.  its related to your code
b. its not the build infra


But I will say there have been good centos verify runs…

BEFORE your patches
BETWEEN various revisions of your changes
BETWEEN retries of your changes
AFTER you stopped trying to retry your changes

With multiple different patches…

So not really having anything to track on the build infra side (nor is this a 
voting issue)  Im starting
a new thread and including tom to see if he has any centos guesses on why we 
are only seeing
problems on centos.

Ed



On May 8, 2019, at 2:46 AM, Yu, Ping <ping...@intel.com> wrote:

Recently centOS build always fail and patch review is blocked even retrying 
several times.


16:17:11 -- Configuring done
16:17:12 -- Generating done
16:17:12 -- Build files have been written to: 
/w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root/build-vpp-native/vpp
16:17:12 ninja: error: manifest 'build.ninja' still dirty after 100 tries
16:17:12
16:17:12 make[4]: *** [Makefile:695: vpp-build] Error 1
16:17:12 make[4]: Leaving directory 
'/w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root'
16:17:12 make[3]: *** [Makefile:931: install-packages] Error 1
16:17:12 make[3]: Leaving directory 
'/w/workspace/vpp-verify-master-centos7/build-root/rpmbuild/vpp-19.08/build-root'
16:17:12 error: Bad exit status from /var/tmp/rpm-tmp.8TtZeQ (%build)
16:17:12
16:17:12
16:17:12 RPM build errors:
16:17:12     Bad exit status from /var/tmp/rpm-tmp.8TtZeQ (%build)
16:17:12 make[2]: *** [RPM] Error 1
16:17:12 make[2]: Leaving directory 
`/w/workspace/vpp-verify-master-centos7/extras/rpm'
16:17:12 make[1]: *** [pkg-rpm] Error 2
16:17:12 make[1]: Leaving directory `/w/workspace/vpp-verify-master-centos7'
16:17:12 make: *** [verify] Error 2
16:17:12 Build step 'Execute shell' marked build as failure


-----Original Message-----
From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Vratko 
Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Sent: Tuesday, May 7, 2019 4:21 PM
To: Ed Kern (ejk) <e...@cisco.com>; Klement Sekera -X (ksekera - PANTHEON 
TECHNOLOGIES at Cisco) <ksek...@cisco.com>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] false verified + 1 for gerrit jobs

reduce the number of retries (or possibly completely)
+1 for completely.

Vratko.

-----Original Message-----
From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Ed Kern via 
Lists.Fd.Io
Sent: Tuesday, 2019-May-07 00:34
To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) 
<ksek...@cisco.com>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] false verified + 1 for gerrit jobs

This is a known issue with how the retry mechanic interacts (badly) with gerrit 
occasionally.  This odds of this happening were a bit higher over the last
couple of days specific to centos retries.   This is tied to the JNLP changes
made to csit and how the initial connection is made.   While the change
has improved JNLP issues across the board there are a couple new nits
that ill have to debug.

At this point though I’m not going to touch/debug further until the csit 
jobs/reports are completed.

Im actually hopeful that ill be able to reduce the number of retries (or 
possibly completely) in the long run with the HAProxy bypass in place.

Ed




On May 6, 2019, at 11:18 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at 
Cisco) <ksek...@cisco.com> wrote:

Hi all,

I noticed a job getting a +1 even though some of the builds failed ...

https://gerrit.fd.io/r/#/c/18444/

please note patch set 9 ....

fd.io JJB
Patch Set 9: Verified-1 Build Failed
https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1804/2459/ :
FAILURE No problems were identified. If you know why this problem
occurred, please add a suitable Cause for it. (
https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1804/2459/ )
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-arm-verify-mas
ter-ubuntu1804/2459
https://jenkins.fd.io/job/vpp-beta-verify-master-ubuntu1804/6891/ :
FAILURE No problems were identified. If you know why this problem
occurred, please add a suitable Cause for it. (
https://jenkins.fd.io/job/vpp-beta-verify-master-ubuntu1804/6891/ )
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-beta-verify-ma
ster-ubuntu1804/6891
https://jenkins.fd.io/job/vpp-csit-verify-device-master-1n-skx/788/ :
SUCCESS (skipped) Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-de
vice-master-1n-skx/788
https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/13041/ :
SUCCESS Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-make-test-docs
-verify-master/13041
https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/19027/ :
SUCCESS Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-
ubuntu1604/19027
https://jenkins.fd.io/job/vpp-verify-master-clang/6731/ : SUCCESS
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-
clang/6731 https://jenkins.fd.io/job/vpp-docs-verify-master/15343/ :
SUCCESS
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-docs-verify-ma
ster/15343 https://jenkins.fd.io/job/vpp-verify-master-centos7/18764/
: NOT_BUILT
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-
centos7/18764
7:11 PM
fd.io JJB
Patch Set 9: Verified+1 Build Successful
https://jenkins.fd.io/job/vpp-verify-master-centos7/18764/ : SUCCESS
Logs:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-
centos7/18764
7:15 PM

Thanks,
Klement


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12956): https://lists.fd.io/g/vpp-dev/message/12956
Mute This Topic: https://lists.fd.io/mt/31543758/675058
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [therb...@redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-

--
*Thomas F Herbert*
NFV and Fast Data Planes
Networking Group Office of the CTO
*Red Hat*
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12957): https://lists.fd.io/g/vpp-dev/message/12957
Mute This Topic: https://lists.fd.io/mt/31543758/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
  • ... Klement Sekera via Lists.Fd.Io
    • ... Ed Kern via Lists.Fd.Io
      • ... Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
        • ... Yu, Ping
          • ... Ed Kern via Lists.Fd.Io
            • ... Thomas F Herbert
              • ... Ed Kern via Lists.Fd.Io

Reply via email to