On Mon, Jan 30, 2023 at 2:57 PM Jose Quaresma <quaresma.j...@gmail.com> wrote: > Denys Dmytriyenko <de...@denix.org> escreveu no dia segunda, 30/01/2023 à(s) > 17:18: >> >> On Mon, Jan 30, 2023 at 11:02:25AM -0600, Ryan Eatmon via >> lists.yoctoproject.org wrote: >> > >> > >> > On 1/30/2023 9:20, Jose Quaresma wrote: >> > >This patch adds the possibility to change some bitbake variable >> > >that cannot be changed anywhere else, one of then is the TMPDIR. >> > > >> > >Using the same TMPDIR it is very sensitive and prone to several errors >> > >when used in more complex situations. >> > >This configuration forces that all native packages have to be the same >> > >between all >> > >machines and this requirement is very easy to break. >> > >Suppose you use a macinhe override somewhere on a native recipe >> >> Even w/o multiconfig, this will break for multi-machine builds. Native >> packages are for the build host and have no knowledge about the target. >> You should probably look into cross packages instead of native ones for >> this use case. >> >> Altering native packages between different targets will cause them to >> rebuild, which they shouldn't. Moreover, it will mess up your sstate >> and lead to weird issues down the road when trying to re-use your >> sstate mirror/cache. > > Is that the issue of rebuilding some native packages that I am trying to > avoid. > But it is very hard to fix this in OE-core and everything else layers what > happens. > > For your information I have some other issues with multiconfig and native > packages > that have the source files provided on the layer. this two recipes don't work > with the > rm_work bbclass because the in one of the machines > texinfo-dummy-native gettext-minimal-native > > Another issue is that sometimes one of the machines uses the > SOURCE_DATE_EPOCH_FALLBACK > and the other uses SOURCE_DATE_EPOCH and this also triggers a rebuild of one > of the machines. > > Recently a new issue with the rm_work and the spdx > comes in https://lists.openembedded.org/g/openembedded-core/message/174276 > where I have the steps to reproduce. > > All of these issues can be solved easily with TMDIR for each machine, > but because I understand it can be invasive I submit this patch that doesn't > change anything > in meta-ti but can add the possibility to use a different TMPDIR for each > machine. > > A different TMPDIR for each machine is also referenced in the official > documentation > "Minimally, each configuration file must define the machine and the temporary > directory BitBake uses for the build. Suggested practice dictates that you do > not overlap the temporary directories used during the builds." > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > >> > >and this requirement is no longer met. >> > >Many of these anomalies can be verified with the use of the rm_work and >> > >create-spdx bitbake classes. >> > > >> > >A previous attempt [1] had already been made but was refused >> > >but this way it can be done without side effects for other users. >> > > >> > >[1] https://lists.yoctoproject.org/g/meta-ti/message/14767 >> > >> > >> > So the difference between this patch and the previous one, is that >> > you are now included a file that is not there normally so that you >> > can put the contents of the previous patch in the new include file >> > to get around the issue in the archiver. > > The archiver issue is fixed but it will be very easy to break again. > Myself recently broke it again so I added a test to help there > https://git.yoctoproject.org/poky/commit/?id=b5baa7dc8bd1c0d2d78c532d97a44120346b5edd > >> > >> > I'm not saying I'm opposed to this approach (yet), I'm just trying >> > to understand the entire story. >> > >> > Did you find anything in looking at fixing the archiver to better >> > support the multi-config issue? > > The issue now is different and is related with spdx and rm_work > https://lists.openembedded.org/g/openembedded-core/message/174276 > > With this patch I can set a new TMPDIR folder for each machine and > can build successfully.
If you don't have spdx enabled then these sort of issues happen less often, but after the above patch was merged in oe-core (and also backported to kirkstone), we cannot build multiconfig targets from meta-ti anymore with spdx enabled. This is special to spdx because of the way the task dependencies are done, which requires it to be executed before rm_work. This is OK when a machine specific TMPDIR is used and explodes quite quickly (due races) when sharing TMPDIR because of the parallel rm_work tasks that end up removing stuff that is still required by other parallel tasks. Cheers, -- Ricardo Salveti
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15704): https://lists.yoctoproject.org/g/meta-ti/message/15704 Mute This Topic: https://lists.yoctoproject.org/mt/96629864/21656 Group Owner: meta-ti+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/meta-ti/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
