On Nov 29, 2017, at 3:09 AM, Marco Varlese <mvarl...@suse.de<mailto:mvarl...@suse.de>> wrote:
Hi Ed, On Wed, 2017-11-29 at 03:24 +0000, Ed Kern (ejk) wrote: All the jobs that ive looked at vpp verify’s are still set to ‘single use slave’ So there is no re-use.. if the job is run by jenkins and is either ubuntu or centos it will attempt to pull and install ubuntu: vpp-dpdk-dev vpp-dpdk-dkms I suppose that's exactly my point of "not-clean-state”. Just to be clear… Are you unhappy that it is doing those package installs before running the make verify (or build.sh in the case of opensuse) Or just because you think those package installs are breaking the build. It will attempt to install but since it finds those packages already installed then obviously it doesn't keep going. well no…well I certainly should have kept going (you trimmed the log you sent to not include the failure or a link to a specific job. I was trying to ask and understand how is it possible that on a "fresh booted" VM there're already packages installed. Well when either the openstack or container image are ‘booted’ neither dpdk or vpp are installed. 32k other prereqs and base packages but (tmk with openstack clone) not dpdk. I suppose that what the message "Up-to-date DPDK package already installed" points out, am I correct? again i want to be careful we are on the same page.. DPDK is not installed in the base template image. DPDK IS now installed as a package (to ubuntu and centos): AFTER check style BEFORE make verify or build.sh as part of the jenkins build scripts. Similarly, the DPDK tarball is already downloaded when the 'curl' command runs since the source tarball can already be found in /dpdk So from the base template we go from no directory template base image is spun up git clone/fetch/etc is run check style is run there should be no dpdk.xxx.tar.xz anywhere at this point make verify or build.sh (im only speaking about verify builds ) is run as part of that ( i myself trip the download by doing make config in the dpdk directory, never bothered to track down how it is tripped ‘normally’) the tar.xz is pulled. if you actually see an order different then this id be curious to see it. thanks, Ed centos: vpp-dpdk-devel if running the build of opensuse or any build outside of jenkins it will attempt to build it from scratch.. Right, I believe that should always be the approach tho... (thats one of the issues you were seeing the other day) https://gerrit.fd.io/r/#/c/9606/ both that dpdk-17.08 rev’d up and also the fact that they added stable to the directory name.. and to make matters worse (only because their mirrors are hosed) the makefile was pointed at fast.dpdk.org<http://fast.dpdk.org/> which points to three servers that return at least two different cksums…(So a total of 3 different a. pre 11/27 b. two post 11/27) in my patch above i just changed it to static.dpdk.org<http://static.dpdk.org/> which is slower but consistent. Great... we could have modified my patch since yours looks pretty similar... anyway, I abonded mine with the open yours gets merged sooner. Note: ignore the cpoc failures…im still bumping into oom condition waiting on vanessa to come back and bump me up https://rt.linuxfoundation.org/Ticket/Display.html?id=48884 On Nov 28, 2017, at 7:58 AM, Marco Varlese <mvarl...@suse.de<mailto:mvarl...@suse.de>> wrote: To add on the reported issue below, similarly, (I think that) the DPDK tarball can be also (always) found in the workspace hence never "freshly" downloaded either. there is no tarball as part of vpp so its always getting freshly pulled…from one location or another.. Snipped of the logs below: --- 11:00:48 @@@@ Finding source for dpdk @@@@ 11:00:48 @@@@ Makefile fragment found in /w/workspace/vpp-verify-master- ubuntu1604/build-data/packages/dpdk.mk @@@@ 11:00:48 @@@@ Source found in /w/workspace/vpp-verify-master-ubuntu1604/dpdk @@@@ --- On Tue, 2017-11-28 at 15:49 +0100, Marco Varlese wrote: All, While looking into an issue which Dave raised yesterday, I bumped into something which does not look right to me. The Jenkins jobs executing the VPP builds do not start from a clean-state. I would expect - for instance - not to find the DPDK package already installed on the target host which builds a GERRIT patch. well…it isn’t…its just installed post check style but before any of the other primary build script. I would basically like to see the VM boot-strapping (as per the package installed by the ci-management scripts) and nothing else. Anything else would be installed because of the scripts we execute to build/install VPP (as per the Jenkins jobs). Would you agree/disagree? So this is how its already happening with the open stack slaves unless something has changed. But i will say that I disagree and its not how im doing the container base images and thats around make install-dep In that every three days ill ‘bake into’ the basic build container a run of make install-dep. Even though it still does run make install-dep again at build time the actual number of packages that it has to pull in (and number of external services that have to be available) is trimmed down to as small a number as possible. This is still a problem in the ‘test’ area that likes to pip install python dependencies but are not linked to main install-dep but i wont rant about that here... In the Jenkins logs I can see the below all the time: ---- 11:00:48 make -C dpdk install-deb 11:00:48 make[1]: Entering directory '/w/workspace/vpp-verify-master- ubuntu1604/dpdk' 11:00:48 Building IPSec-MB 0.46 library 11:00:48 ========================================================== 11:00:48 Up-to-date DPDK package already installed 11:00:48 ========================================================== 11:00:48 make[1]: Leaving directory '/w/workspace/vpp-verify-master- ubuntu1604/dpdk' 11:00:48 ---- Also, I would like to ask if by any chance the tarball which VPP build-system requires (e.g. DPDK / IPSecMB / etc.) are somehow cached somewhere (either on the Jenkins-slave or Proxied) since I think the DPDK tarball (currently pulled) may not be coming from the real location… Wont speak for the openstack build side but I actually think it may be worth just doing something similar to the install-dep above with the dpdk… installing the binaries from nexus every few days so while it will check the nexus server for an update it normally wouldn’t have to actually pull or install. Will give that some thought… and from previous thread Since the MD5 checksum on the DPDK tarball fails; to answer your question, no, it has never happened to me to see this specific issue before. I don't think there's anything specific to the openSUSE setup and/or scripts being executed. I'd rather feel is - as I said earlier - something to do with an hiccup on the infrastructure side. The fact that a 'recheck' made it passing I suppose it confirms my current theory. well opensuse still has issues ‘passing’ or not…namely no artifacts are ever generated/pushed 01:43:40 [ssh-agent] Stopped. 01:43:40 Archiving artifacts 01:43:40 WARN: No artifacts found that match the file pattern "build-root/*.rpm,build-root/*.deb,dpdk/*.rpm,dpdk/*.deb". Configuration error? 01:43:40 WARN: ?build-root/*.rpm? doesn?t match anything: ?build-root? exists but not ?build-root/*.rpm? 01:43:40 [PostBuildScript] - Execution post build scripts. 01:43:40 [vpp-verify-master-opensuse] $ /bin/bash /tmp/jenkins8435214415951107237.sh 01:43:40 Build logs: <a href="https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-opensuse/501";>https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-opensuse/501</a> 01:43:40 uname -a: So for me opensuse ‘passing’ is something akin to a banner reading ‘Mission Accomplished’ Ed Cheers, -- Marco V SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev -- Marco V SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
