The documentation isn’t perfect. Sometimes it becomes outdated and no one notices, or it may be inaccurate to begin with. Personally I would never tamper with content of downloads and preserve it exactly as it is. Doesn’t have to be git lfs, plain artifact storage options work too.
Alex On Fri 25. Aug 2023 at 11.20, Stephan M <stephan.muehlstras...@gmail.com> wrote: > Hi, > > I'm trying to follow the steps to replicate a build offline: > https://docs.yoctoproject.org/4.2.2/dev-manual/building.html?highlight=bb_no_network#replicating-a-build-offline > > This is not working as expected. Originally I ran into this with NXP's > mickledore build, but I can replicate that with a plain mickledore poky > environment. > > In the documentation it says: > > *Optionally Remove Any Git or other SCM Subdirectories From the Downloads > Directory:* If you want, you can clean up your downloads directory by > removing any Git or other Source Control Management (SCM) subdirectories > such as ${DL_DIR}/git2/*. ... > > When I do that, the offline build breaks. Without that step it succeeds. > > I did the following: > > ~/src$ git clone -b mickledore-4.2.2 https://git.yoctoproject.org/poky > Cloning into 'poky'... > ... > ~/src$ cd poky/ > ~/src/poky$ . oe-init-build-env > > You had no conf/local.conf file. This configuration file has therefore been > > created for you from > /home/stm/src/poky/meta-poky/conf/templates/default/local.conf.sample > > You may wish to edit it to, for example, select a different MACHINE (target > > hardware). > ... > > # Now edited conf/local.conf and added: BB_GENERATE_MIRROR_TARBALLS = "1" > > ~/src/poky/build$ bitbake core-image-minimal --runonly=fetch > > # All the packages are downloaded and tar files are created in > ${TOPDIR}/downloads > # Now I remove the Git data from the "downloads" directory: > > ~/src/poky/build$ rm -rf downloads/git2/* > > # Now I edit conf/local.conf again and add the following according to the > documentation > # SOURCE_MIRROR_URL ?= "file://${TOPDIR}/downloads/" > > # INHERIT += "own-mirrors" > > # BB_NO_NETWORK = "1" > > ~/src/poky/build$ tail -10 conf/local.conf > > # CONF_VERSION is increased each time build/conf/ changes incompatibly and > is used to > > # track the version of this file when it was generated. This can safely be > ignored if > > # this doesn't mean anything to you. > > CONF_VERSION = "2" > > > > BB_GENERATE_MIRROR_TARBALLS = "1" > > SOURCE_MIRROR_URL ?= "file://${TOPDIR}/downloads/" > > INHERIT += "own-mirrors" > > BB_NO_NETWORK = "1" > > > # An attempt to run the build fails early with these error messages: > > ~/src/poky/build$ bitbake core-image-minimal > > Loading cache: 100% | > | ETA: --:--:-- > > Loaded 0 entries from dependency cache. > > Parsing recipes: 100% > |##################################################################################| > Time: 0:00:07 > > Parsing of 899 .bb files complete (0 cached, 899 parsed). 1799 targets, 49 > skipped, 0 masked, 0 errors. > > NOTE: Resolving any missing task queue dependencies > > > > Build Configuration: > > BB_VERSION = "2.4.0" > > BUILD_SYS = "x86_64-linux" > > NATIVELSBSTRING = "universal" > > TARGET_SYS = "x86_64-poky-linux" > > MACHINE = "qemux86-64" > > DISTRO = "poky" > > DISTRO_VERSION = "4.2.2" > > TUNE_FEATURES = "m64 core2" > > TARGET_FPU = "" > > meta > > meta-poky > > meta-yocto-bsp = "HEAD:6e17b3e644ca15b8b4afd071ccaa6f172a0e681a" > > > > Initialising tasks: 100% > |###############################################################################| > Time: 0:00:01 > > Sstate summary: Wanted 1074 Local 0 Mirrors 0 Missed 1074 Current 0 (0% > match, 0% complete) > > NOTE: Executing Tasks > > ERROR: gnu-config-native-20230216+gitAUTOINC+63acb96f92-r0 do_unpack: > Bitbake Fetcher Error: UnpackError('No up to date source found: clone > directory not available or not up to date: > /home/stm/src/poky/build/downloads/git2/git.savannah.gnu.org.git.config.git; > shallow clone not enabled', 'git:// > git.savannah.gnu.org/git/config.git;protocol=https;branch=master') > > ERROR: Logfile of failure stored in: > /home/stm/src/poky/build/tmp/work/x86_64-linux/gnu-config-native/20230216+gitAUTOINC+63acb96f92-r0/temp/log.do_unpack.19635 > > ERROR: Task > (virtual:native:/home/stm/src/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_unpack) > failed with exit code '1' > > ERROR: pkgconfig-native-0.29.2+gitAUTOINC+d97db4fae4-r0 do_unpack: Bitbake > Fetcher Error: UnpackError('No up to date source found: clone directory not > available or not up to date: > /home/stm/src/poky/build/downloads/git2/gitlab.freedesktop.org.pkg-config.pkg-config.git; > shallow clone not enabled', 'git:// > gitlab.freedesktop.org/pkg-config/pkg-config.git;branch=master;protocol=https > ') > > ERROR: Logfile of failure stored in: > /home/stm/src/poky/build/tmp/work/x86_64-linux/pkgconfig-native/0.29.2+gitAUTOINC+d97db4fae4-r0/temp/log.do_unpack.19647 > > ERROR: Task > (virtual:native:/home/stm/src/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_unpack) > failed with exit code '1' > > NOTE: Tasks Summary: Attempted 44 tasks of which 22 didn't need to be > rerun and 2 failed. > > > > Summary: 2 tasks failed: > > > virtual:native:/home/stm/src/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb: > do_unpack > > > virtual:native:/home/stm/src/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb: > do_unpack > > Summary: There were 2 ERROR messages, returning a non-zero exit code. > > > > Am I doing something wrong, or is this a bug? > > The background for these experiments is the question how to guarantee > long-term reproducibility of our Yocto project builds. My current idea is > to preserve the "downloads" directory as a separate Git repository, maybe > using Git LFS. Is this a good or bad idea? > > I read of course > https://docs.yoctoproject.org/4.2.2/test-manual/reproducible-builds.html?highlight=reproducibility#reproducible-builds > , > but my concern are the downloaded packages. If I try to reproduce a certain > build 5 or 10 years from now, it might no longer be possible to download > all the relevant packages if I want to create a build environment from > scratch. > > Thanks > Stephan > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60890): https://lists.yoctoproject.org/g/yocto/message/60890 Mute This Topic: https://lists.yoctoproject.org/mt/100952497/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
