Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Tue, Feb 20, 2018 at 10:28:47AM -0800, Khem Raj wrote: > On Tue, Feb 20, 2018 at 8:17 AM, Denys Dmytriyenko wrote: > > On Tue, Feb 20, 2018 at 04:08:06PM +, Khem Raj wrote: > >> On Tue, Feb 20, 2018 at 7:41 AM Martin Jansa > >> wrote: > >> > >> > I'm OK with adding bison-native for all versions. > >> > >> As long we know that there are no side effects of this dep in kernel builds > > > > You do realize that when you build with oe-core toolchain, every package > > gets > > bison-native dependency automatically? Regardless whether it was requested > > or > > not. If there were side effects, we would have seen it long time ago... > > Yes I see it seems to be pulled in by toolchain regardless, so for this > patch we might just be adding to determinism. but I think it raises question > on why toolchain is adding this everywhere, I think we should remove it > from toolchain default deps. It will make metadata more consistent > across internal/external toolchains There were several similar patches I submitted last month for different components to add flex-native and/or bison-native. Most of them got accepted and we had a follow up discussion with Richard in gcc thread: http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146295.html He also submitted this patch to prevent indirect flex/bison-native dependencies, but I don't believe it got merged yet: http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146324.html Richard, do you have any comments? -- Denys -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Tue, Feb 20, 2018 at 8:17 AM, Denys Dmytriyenko wrote: > On Tue, Feb 20, 2018 at 04:08:06PM +, Khem Raj wrote: >> On Tue, Feb 20, 2018 at 7:41 AM Martin Jansa wrote: >> >> > I'm OK with adding bison-native for all versions. >> >> As long we know that there are no side effects of this dep in kernel builds > > You do realize that when you build with oe-core toolchain, every package gets > bison-native dependency automatically? Regardless whether it was requested or > not. If there were side effects, we would have seen it long time ago... Yes I see it seems to be pulled in by toolchain regardless, so for this patch we might just be adding to determinism. but I think it raises question on why toolchain is adding this everywhere, I think we should remove it from toolchain default deps. It will make metadata more consistent across internal/external toolchains -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Tue, Feb 20, 2018 at 04:08:06PM +, Khem Raj wrote: > On Tue, Feb 20, 2018 at 7:41 AM Martin Jansa wrote: > > > I'm OK with adding bison-native for all versions. > > As long we know that there are no side effects of this dep in kernel builds You do realize that when you build with oe-core toolchain, every package gets bison-native dependency automatically? Regardless whether it was requested or not. If there were side effects, we would have seen it long time ago... > > When we added bc-native 2+ years ago: > > commit c067e52cffe002de3b39aa1bced308dd532859c1 > > Author: Alejandro Hernandez > > Date: Thu Oct 2 15:33:23 2014 -0500 > > > > kernel: Added bc-native as DEPENDS > > > > The makefile checks for bc during for compilation > > > > it was also needed only for the latest kernel and we didn't make it > > conditional on the kernel version. Now with RSS it's even less dangerous as > > the bison-native will appear only in kernel's RSS and as Denys said it's > > quite common dependency so many components pull it into RSS anyway. > > > > On Tue, Feb 20, 2018 at 3:53 PM, Otavio Salvador < > > otavio.salva...@ossystems.com.br> wrote: > > > >> On Tue, Feb 20, 2018 at 7:10 AM, Khem Raj wrote: > >> > On 2/19/18 5:20 PM, Denys Dmytriyenko wrote: > >> >> > >> >> On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: > >> >>> > >> >>> On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko > >> >>> wrote: > >> > >> From: Denys Dmytriyenko > >> > >> Explicitly depend on bison-native for deterministic builds, as it is > >> required > >> for the build: > >> > >> | HOSTCC scripts/basic/fixdep > >> | GEN ./Makefile > >> | HOSTCC scripts/kconfig/conf.o > >> | YACCscripts/kconfig/zconf.tab.c > >> | /bin/sh: bison: command not found > >> | scripts/Makefile.lib:217: recipe for target > >> 'scripts/kconfig/zconf.tab.c' failed > >> > >> In most cases, this dependency comes indirectly via toolchain > >> dependencies, > >> specifically binutils-cross, which pulls bison-native. Different > >> setups, > >> such as with external toolchain, would expose this problem, since > >> correct > >> dependency is not marked explicitly. > >> > >> Signed-off-by: Denys Dmytriyenko > >> --- > >> I'm seeing this now on 4.16-rc2 with external toolchain. > >> >>> > >> >>> > >> >>> I am not against the addition but in fact, it is because 4.16 stopped > >> >>> of using pre-generated files and now requires bison to be available. > >> >> > >> >> > >> >> Ok, good, I was wondering why I didn't see it before 4.16... > >> >> > >> >> > >> >>> I am unsure it ought to be on kernel.bbclass at this moment. > >> >> > >> >> > >> >> Why not? Because it's 4.16+ specific? > >> >> > >> >> By default bison-native dependency gets added implicitly when built > >> with > >> >> the > >> >> toolchain from oe-core, so this doesn't add any extra dependencies, > >> only > >> >> makes > >> >> it explicit. > >> >> > >> > it will be overcompensating for < 4.16 kernel recipes where this > >> dependency > >> > is not needed. it might be a legit dependency for another component, > >> that > >> > doesn't stage this package in kernel recipe-specific sysroot but this > >> change > >> > will do that. So its not completely innocent change. > >> > >> I'd be in favor of matching the version and adding it if it is a 4.16 > >> or newer kernel. Adding for prior versions seems wrong for me. > >> > >> -- > >> Otavio Salvador O.S. Systems > >> http://www.ossystems.com.brhttp://code.ossystems.com.br > >> Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750 > >> > > -- > >> ___ > >> Openembedded-core mailing list > >> Openembedded-core@lists.openembedded.org > >> http://lists.openembedded.org/mailman/listinfo/openembedded-core > >> > > > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Tue, Feb 20, 2018 at 7:41 AM Martin Jansa wrote: > I'm OK with adding bison-native for all versions. > As long we know that there are no side effects of this dep in kernel builds > > When we added bc-native 2+ years ago: > commit c067e52cffe002de3b39aa1bced308dd532859c1 > Author: Alejandro Hernandez > Date: Thu Oct 2 15:33:23 2014 -0500 > > kernel: Added bc-native as DEPENDS > > The makefile checks for bc during for compilation > > it was also needed only for the latest kernel and we didn't make it > conditional on the kernel version. Now with RSS it's even less dangerous as > the bison-native will appear only in kernel's RSS and as Denys said it's > quite common dependency so many components pull it into RSS anyway. > > On Tue, Feb 20, 2018 at 3:53 PM, Otavio Salvador < > otavio.salva...@ossystems.com.br> wrote: > >> On Tue, Feb 20, 2018 at 7:10 AM, Khem Raj wrote: >> > On 2/19/18 5:20 PM, Denys Dmytriyenko wrote: >> >> >> >> On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: >> >>> >> >>> On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko >> >>> wrote: >> >> From: Denys Dmytriyenko >> >> Explicitly depend on bison-native for deterministic builds, as it is >> required >> for the build: >> >> | HOSTCC scripts/basic/fixdep >> | GEN ./Makefile >> | HOSTCC scripts/kconfig/conf.o >> | YACCscripts/kconfig/zconf.tab.c >> | /bin/sh: bison: command not found >> | scripts/Makefile.lib:217: recipe for target >> 'scripts/kconfig/zconf.tab.c' failed >> >> In most cases, this dependency comes indirectly via toolchain >> dependencies, >> specifically binutils-cross, which pulls bison-native. Different >> setups, >> such as with external toolchain, would expose this problem, since >> correct >> dependency is not marked explicitly. >> >> Signed-off-by: Denys Dmytriyenko >> --- >> I'm seeing this now on 4.16-rc2 with external toolchain. >> >>> >> >>> >> >>> I am not against the addition but in fact, it is because 4.16 stopped >> >>> of using pre-generated files and now requires bison to be available. >> >> >> >> >> >> Ok, good, I was wondering why I didn't see it before 4.16... >> >> >> >> >> >>> I am unsure it ought to be on kernel.bbclass at this moment. >> >> >> >> >> >> Why not? Because it's 4.16+ specific? >> >> >> >> By default bison-native dependency gets added implicitly when built >> with >> >> the >> >> toolchain from oe-core, so this doesn't add any extra dependencies, >> only >> >> makes >> >> it explicit. >> >> >> > it will be overcompensating for < 4.16 kernel recipes where this >> dependency >> > is not needed. it might be a legit dependency for another component, >> that >> > doesn't stage this package in kernel recipe-specific sysroot but this >> change >> > will do that. So its not completely innocent change. >> >> I'd be in favor of matching the version and adding it if it is a 4.16 >> or newer kernel. Adding for prior versions seems wrong for me. >> >> -- >> Otavio Salvador O.S. Systems >> http://www.ossystems.com.brhttp://code.ossystems.com.br >> Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750 >> > -- >> ___ >> Openembedded-core mailing list >> Openembedded-core@lists.openembedded.org >> http://lists.openembedded.org/mailman/listinfo/openembedded-core >> > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
I'm OK with adding bison-native for all versions. When we added bc-native 2+ years ago: commit c067e52cffe002de3b39aa1bced308dd532859c1 Author: Alejandro Hernandez Date: Thu Oct 2 15:33:23 2014 -0500 kernel: Added bc-native as DEPENDS The makefile checks for bc during for compilation it was also needed only for the latest kernel and we didn't make it conditional on the kernel version. Now with RSS it's even less dangerous as the bison-native will appear only in kernel's RSS and as Denys said it's quite common dependency so many components pull it into RSS anyway. On Tue, Feb 20, 2018 at 3:53 PM, Otavio Salvador < otavio.salva...@ossystems.com.br> wrote: > On Tue, Feb 20, 2018 at 7:10 AM, Khem Raj wrote: > > On 2/19/18 5:20 PM, Denys Dmytriyenko wrote: > >> > >> On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: > >>> > >>> On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko > >>> wrote: > > From: Denys Dmytriyenko > > Explicitly depend on bison-native for deterministic builds, as it is > required > for the build: > > | HOSTCC scripts/basic/fixdep > | GEN ./Makefile > | HOSTCC scripts/kconfig/conf.o > | YACCscripts/kconfig/zconf.tab.c > | /bin/sh: bison: command not found > | scripts/Makefile.lib:217: recipe for target > 'scripts/kconfig/zconf.tab.c' failed > > In most cases, this dependency comes indirectly via toolchain > dependencies, > specifically binutils-cross, which pulls bison-native. Different > setups, > such as with external toolchain, would expose this problem, since > correct > dependency is not marked explicitly. > > Signed-off-by: Denys Dmytriyenko > --- > I'm seeing this now on 4.16-rc2 with external toolchain. > >>> > >>> > >>> I am not against the addition but in fact, it is because 4.16 stopped > >>> of using pre-generated files and now requires bison to be available. > >> > >> > >> Ok, good, I was wondering why I didn't see it before 4.16... > >> > >> > >>> I am unsure it ought to be on kernel.bbclass at this moment. > >> > >> > >> Why not? Because it's 4.16+ specific? > >> > >> By default bison-native dependency gets added implicitly when built with > >> the > >> toolchain from oe-core, so this doesn't add any extra dependencies, only > >> makes > >> it explicit. > >> > > it will be overcompensating for < 4.16 kernel recipes where this > dependency > > is not needed. it might be a legit dependency for another component, that > > doesn't stage this package in kernel recipe-specific sysroot but this > change > > will do that. So its not completely innocent change. > > I'd be in favor of matching the version and adding it if it is a 4.16 > or newer kernel. Adding for prior versions seems wrong for me. > > -- > Otavio Salvador O.S. Systems > http://www.ossystems.com.brhttp://code.ossystems.com.br > Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750 > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Tue, Feb 20, 2018 at 7:10 AM, Khem Raj wrote: > On 2/19/18 5:20 PM, Denys Dmytriyenko wrote: >> >> On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: >>> >>> On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko >>> wrote: From: Denys Dmytriyenko Explicitly depend on bison-native for deterministic builds, as it is required for the build: | HOSTCC scripts/basic/fixdep | GEN ./Makefile | HOSTCC scripts/kconfig/conf.o | YACCscripts/kconfig/zconf.tab.c | /bin/sh: bison: command not found | scripts/Makefile.lib:217: recipe for target 'scripts/kconfig/zconf.tab.c' failed In most cases, this dependency comes indirectly via toolchain dependencies, specifically binutils-cross, which pulls bison-native. Different setups, such as with external toolchain, would expose this problem, since correct dependency is not marked explicitly. Signed-off-by: Denys Dmytriyenko --- I'm seeing this now on 4.16-rc2 with external toolchain. >>> >>> >>> I am not against the addition but in fact, it is because 4.16 stopped >>> of using pre-generated files and now requires bison to be available. >> >> >> Ok, good, I was wondering why I didn't see it before 4.16... >> >> >>> I am unsure it ought to be on kernel.bbclass at this moment. >> >> >> Why not? Because it's 4.16+ specific? >> >> By default bison-native dependency gets added implicitly when built with >> the >> toolchain from oe-core, so this doesn't add any extra dependencies, only >> makes >> it explicit. >> > it will be overcompensating for < 4.16 kernel recipes where this dependency > is not needed. it might be a legit dependency for another component, that > doesn't stage this package in kernel recipe-specific sysroot but this change > will do that. So its not completely innocent change. I'd be in favor of matching the version and adding it if it is a 4.16 or newer kernel. Adding for prior versions seems wrong for me. -- Otavio Salvador O.S. Systems http://www.ossystems.com.brhttp://code.ossystems.com.br Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On 2/19/18 5:20 PM, Denys Dmytriyenko wrote: On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko wrote: From: Denys Dmytriyenko Explicitly depend on bison-native for deterministic builds, as it is required for the build: | HOSTCC scripts/basic/fixdep | GEN ./Makefile | HOSTCC scripts/kconfig/conf.o | YACCscripts/kconfig/zconf.tab.c | /bin/sh: bison: command not found | scripts/Makefile.lib:217: recipe for target 'scripts/kconfig/zconf.tab.c' failed In most cases, this dependency comes indirectly via toolchain dependencies, specifically binutils-cross, which pulls bison-native. Different setups, such as with external toolchain, would expose this problem, since correct dependency is not marked explicitly. Signed-off-by: Denys Dmytriyenko --- I'm seeing this now on 4.16-rc2 with external toolchain. I am not against the addition but in fact, it is because 4.16 stopped of using pre-generated files and now requires bison to be available. Ok, good, I was wondering why I didn't see it before 4.16... I am unsure it ought to be on kernel.bbclass at this moment. Why not? Because it's 4.16+ specific? By default bison-native dependency gets added implicitly when built with the toolchain from oe-core, so this doesn't add any extra dependencies, only makes it explicit. it will be overcompensating for < 4.16 kernel recipes where this dependency is not needed. it might be a legit dependency for another component, that doesn't stage this package in kernel recipe-specific sysroot but this change will do that. So its not completely innocent change. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Mon, Feb 19, 2018 at 09:36:22PM -0300, Otavio Salvador wrote: > On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko wrote: > > From: Denys Dmytriyenko > > > > Explicitly depend on bison-native for deterministic builds, as it is > > required > > for the build: > > > > | HOSTCC scripts/basic/fixdep > > | GEN ./Makefile > > | HOSTCC scripts/kconfig/conf.o > > | YACCscripts/kconfig/zconf.tab.c > > | /bin/sh: bison: command not found > > | scripts/Makefile.lib:217: recipe for target 'scripts/kconfig/zconf.tab.c' > > failed > > > > In most cases, this dependency comes indirectly via toolchain dependencies, > > specifically binutils-cross, which pulls bison-native. Different setups, > > such as with external toolchain, would expose this problem, since correct > > dependency is not marked explicitly. > > > > Signed-off-by: Denys Dmytriyenko > > --- > > I'm seeing this now on 4.16-rc2 with external toolchain. > > I am not against the addition but in fact, it is because 4.16 stopped > of using pre-generated files and now requires bison to be available. Ok, good, I was wondering why I didn't see it before 4.16... > I am unsure it ought to be on kernel.bbclass at this moment. Why not? Because it's 4.16+ specific? By default bison-native dependency gets added implicitly when built with the toolchain from oe-core, so this doesn't add any extra dependencies, only makes it explicit. -- Denys -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
On Mon, Feb 19, 2018 at 8:54 PM, Denys Dmytriyenko wrote: > From: Denys Dmytriyenko > > Explicitly depend on bison-native for deterministic builds, as it is required > for the build: > > | HOSTCC scripts/basic/fixdep > | GEN ./Makefile > | HOSTCC scripts/kconfig/conf.o > | YACCscripts/kconfig/zconf.tab.c > | /bin/sh: bison: command not found > | scripts/Makefile.lib:217: recipe for target 'scripts/kconfig/zconf.tab.c' > failed > > In most cases, this dependency comes indirectly via toolchain dependencies, > specifically binutils-cross, which pulls bison-native. Different setups, > such as with external toolchain, would expose this problem, since correct > dependency is not marked explicitly. > > Signed-off-by: Denys Dmytriyenko > --- > I'm seeing this now on 4.16-rc2 with external toolchain. I am not against the addition but in fact, it is because 4.16 stopped of using pre-generated files and now requires bison to be available. I am unsure it ought to be on kernel.bbclass at this moment. -- Otavio Salvador O.S. Systems http://www.ossystems.com.brhttp://code.ossystems.com.br Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] kernel.bbclass: explicitly depend on bison-native for deterministic builds
From: Denys Dmytriyenko Explicitly depend on bison-native for deterministic builds, as it is required for the build: | HOSTCC scripts/basic/fixdep | GEN ./Makefile | HOSTCC scripts/kconfig/conf.o | YACCscripts/kconfig/zconf.tab.c | /bin/sh: bison: command not found | scripts/Makefile.lib:217: recipe for target 'scripts/kconfig/zconf.tab.c' failed In most cases, this dependency comes indirectly via toolchain dependencies, specifically binutils-cross, which pulls bison-native. Different setups, such as with external toolchain, would expose this problem, since correct dependency is not marked explicitly. Signed-off-by: Denys Dmytriyenko --- I'm seeing this now on 4.16-rc2 with external toolchain. meta/classes/kernel.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index 943938d..321c0a4 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -4,7 +4,7 @@ KERNEL_PACKAGE_NAME ??= "kernel" KERNEL_DEPLOYSUBDIR ??= "${@ "" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else d.getVar("KERNEL_PACKAGE_NAME") }" PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }" -DEPENDS += "virtual/${TARGET_PREFIX}binutils virtual/${TARGET_PREFIX}gcc kmod-native bc-native lzop-native" +DEPENDS += "virtual/${TARGET_PREFIX}binutils virtual/${TARGET_PREFIX}gcc kmod-native bc-native lzop-native bison-native" PACKAGE_WRITE_DEPS += "depmodwrapper-cross" do_deploy[depends] += "depmodwrapper-cross:do_populate_sysroot" -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core