Hi Maciej, On Fri, Sep 1, 2017 at 4:08 PM, Maciej Borzęcki <maciej.borze...@rndity.com> wrote:
> On Fri, Sep 1, 2017 at 3:54 PM, Andrea Galbusera <giz...@gmail.com> wrote: > > Hi! > > > > I was trying to share sstate between different hosts, but the consumer > build > > system seems to be unable to use re-use any sstate object. My scenario is > > setup as follows: > > > > * The cache was populated by a pristine qemux86 core-image-minimal build > of > > morty. This was done in a crops/poky container (running in docker on Mac) > > * The cache was then served via HTTP > > Make sure that you use a decent HTTP server. Simple `python3 -m > http.server` will quickly choke when the mirror is being checked. Also > running bitbake -DDD -v makes investigating this much easier. > To be honest, the current server was indeed setup with python's SimpleHTTPServer... As you suggest, I checked the verbose debug log and noticed what's happening behind the apparently happy "Checking sstate mirror object availability" step. After a first "SState: Successful fetch test for" that I see correctly served with 200 on the server side, tests for any other sstate object suddenly and systematically fail with logs like this: DEBUG: SState: Attempting to fetch file://7d/sstate:libxml2:i586- poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz DEBUG: Searching for 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3: 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz in paths: /home/vagrant/koan/morty/build/sstate-cache DEBUG: Defaulting to /home/vagrant/koan/morty/build/sstate-cache/7d/sstate: libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz for 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23b db86ac7ab32_package_qa.tgz DEBUG: Testing URL file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3: 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz DEBUG: For url ['file', '', '7d/sstate:libxml2:i586-poky- linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz', '', '', OrderedDict()] comparing ['file', '', '.*', '', '', OrderedDict()] to ['http', '192.168.33.1:8000', ' /sstate-cache/PATH', '', '', OrderedDict([('downloadfilename', 'PATH')])] DEBUG: For url file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3: 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz returning http://192.168.33.1:8000/sstate-cache/7d/sstate%3Alibxml2%3Ai586-poky-linux% 3A2.9.4%3Ar0%3Ai586%3A3%3A7da8fc 3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz;downloadfilename=7d/sstate: libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab 32_package_qa.tgz DEBUG: checkstatus: trying again DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad file descriptor> DEBUG: SState: Unsuccessful fetch test for file://7d/sstate:libxml2:i586- poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz Nothing is reported server-side for any of these failures... As you recommend, I'll try to setup something more "decent" for the HTTP server and see if it helps. > > * The second host is a VM running Ubuntu 16.04 where I set > SSTATE_MIRRORS to > > point to the hosted sstate cache like this: > > > > SSTATE_MIRRORS ?= "\ > > file://.* http://192.168.33.1:8000/sstate-cache/PATH;downloadfilename= > PATH" > > > > * I checked with curl that the VM can successfully get sstate objects > from > > the server. > > * Then I start a new build (same metadata revisions, default > configuration > > for core-image-minimal) and each and every task run from scratch with no > > sstate cache re-use. > > > > Here are the two configurations from bitbake and /etc/lsb-release files: > > > > On the container used to seed sstate cache: > > > > Build Configuration: > > BB_VERSION = "1.32.0" > > BUILD_SYS = "x86_64-linux" > > NATIVELSBSTRING = "universal" > > TARGET_SYS = "i586-poky-linux" > > MACHINE = "qemux86" > > DISTRO = "poky" > > DISTRO_VERSION = "2.2.2" > > TUNE_FEATURES = "m32 i586" > > TARGET_FPU = "" > > meta > > meta-poky > > meta-yocto-bsp = "morty:2a70e84643381eca0e7bf7928d4a3d56f9651128" > > > > $ cat /etc/lsb-release > > DISTRIB_ID=Ubuntu > > DISTRIB_RELEASE=16.04 > > DISTRIB_CODENAME=xenial > > DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS" > > > > On the VM that should consume the cache: > > > > Build Configuration: > > BB_VERSION = "1.32.0" > > BUILD_SYS = "x86_64-linux" > > NATIVELSBSTRING = "Ubuntu-16.04" > > TARGET_SYS = "i586-poky-linux" > > MACHINE = "qemux86" > > DISTRO = "poky" > > DISTRO_VERSION = "2.2.2" > > TUNE_FEATURES = "m32 i586" > > TARGET_FPU = "" > > meta > > meta-poky > > meta-yocto-bsp = "morty:2a70e84643381eca0e7bf7928d4a3d56f9651128" > > > > $ cat /etc/lsb-release > > DISTRIB_ID=Ubuntu > > DISTRIB_RELEASE=16.04 > > DISTRIB_CODENAME=xenial > > DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS" > > > > > > To me, the only differing bit that in my understanding can lead to sstate > > cache objects invalidation is the value of NATIVELSBSTRING which is > > "universal" inside the container and "Ubuntu-16.04". This sounds strange > to > > me, since both underlying systems are Ubuntu 16.04 (although not exactly > the > > same dot release) as confirmed by /etc/lsb-release contents. > > > > Is the different NATIVELSBSTRING the root cause for everything being > > re-built? If so, what's causing them being different in the end and what > > does "universal" exactly mean (to me it looks like a more generic and > > incluse term than any distro label, so I'm confused...)? > > > > > > -- > > _______________________________________________ > > yocto mailing list > > yocto@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/yocto > > > > > > -- > Maciej Borzecki > RnDity >
-- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto