Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Richard Purdie]

On Tue, 2018-04-03 at 10:11 -0400, Bruce Ashfield wrote:
> On Tue, Apr 3, 2018 at 9:59 AM, Richard Purdie
>  wrote:
> > 
> > On Tue, 2018-04-03 at 08:58 -0400, Bruce Ashfield wrote:
> > > 
> > > On Tue, Apr 3, 2018 at 12:23 AM, Jonathan Liu 
> > > wrote:
> > > > 
> > > > 
> > > > Set KBUILD_BUILD_VERSION=1 to avoid build version being
> > > > incremented
> > > > on
> > > > every build. It is visible in the output of "cat /proc/version"
> > > > after
> > > > the hash character.
> > > It is possible that someone might actually want this feature, for
> > > debug or
> > > different build variants.
> > > 
> > > So this would be better as a variable that can be overridden.
> > Surely users can override it in this form (like the USER/HOST
> > variants
> > already there)?
> I've never used the user/host variant, so I don't know.
> 
> How can you override an exported variable in a shell function in a
> bbclass ? I've never tried.
> 
> I know for a fact, that I wouldn't want this on by default, hence my
> comment.

KBUILD_BUILD_VERSION[unexport] = "1"

will override the export flag...

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Bruce Ashfield]

On Tue, Apr 3, 2018 at 9:59 AM, Richard Purdie
 wrote:
> On Tue, 2018-04-03 at 08:58 -0400, Bruce Ashfield wrote:
>> On Tue, Apr 3, 2018 at 12:23 AM, Jonathan Liu 
>> wrote:
>> >
>> > Set KBUILD_BUILD_VERSION=1 to avoid build version being incremented
>> > on
>> > every build. It is visible in the output of "cat /proc/version"
>> > after
>> > the hash character.
>> It is possible that someone might actually want this feature, for
>> debug or
>> different build variants.
>>
>> So this would be better as a variable that can be overridden.
>
> Surely users can override it in this form (like the USER/HOST variants
> already there)?

I've never used the user/host variant, so I don't know.

How can you override an exported variable in a shell function in a
bbclass ? I've never tried.

I know for a fact, that I wouldn't want this on by default, hence my comment.

Bruce

>
> Cheers,
>
> Richard



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Richard Purdie]

On Tue, 2018-04-03 at 08:58 -0400, Bruce Ashfield wrote:
> On Tue, Apr 3, 2018 at 12:23 AM, Jonathan Liu 
> wrote:
> > 
> > Set KBUILD_BUILD_VERSION=1 to avoid build version being incremented
> > on
> > every build. It is visible in the output of "cat /proc/version"
> > after
> > the hash character.
> It is possible that someone might actually want this feature, for
> debug or
> different build variants.
> 
> So this would be better as a variable that can be overridden.

Surely users can override it in this form (like the USER/HOST variants
already there)?

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Bruce Ashfield]

On Tue, Apr 3, 2018 at 12:23 AM, Jonathan Liu  wrote:
> Set KBUILD_BUILD_VERSION=1 to avoid build version being incremented on
> every build. It is visible in the output of "cat /proc/version" after
> the hash character.

It is possible that someone might actually want this feature, for debug or
different build variants.

So this would be better as a variable that can be overridden.

Bruce

>
> Signed-off-by: Jonathan Liu 
> ---
>  meta/classes/kernel.bbclass | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 4877965af0..78d6c30b07 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -151,6 +151,7 @@ PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-firmware-.*"
>
>  export OS = "${TARGET_OS}"
>  export CROSS_COMPILE = "${TARGET_PREFIX}"
> +export KBUILD_BUILD_VERSION = "1"
>  export KBUILD_BUILD_USER = "oe-user"
>  export KBUILD_BUILD_HOST = "oe-host"
>
> --
> 2.16.3
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Jonathan Liu]

Set KBUILD_BUILD_VERSION=1 to avoid build version being incremented on
every build. It is visible in the output of "cat /proc/version" after
the hash character.

Signed-off-by: Jonathan Liu 
---
 meta/classes/kernel.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4877965af0..78d6c30b07 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -151,6 +151,7 @@ PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-firmware-.*"
 
 export OS = "${TARGET_OS}"
 export CROSS_COMPILE = "${TARGET_PREFIX}"
+export KBUILD_BUILD_VERSION = "1"
 export KBUILD_BUILD_USER = "oe-user"
 export KBUILD_BUILD_HOST = "oe-host"
 
-- 
2.16.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Trevor Woerner]

Should this be configurable?

Is there an assumption everyone who uses oe-core wants reproducible builds?

Could this affect one's ability to perform debugging?
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Bystricky, Juro]

Quite possibly. However, the biggest abuser of __FILE__ seems to be kernel and 
kernel_modules.


From: Burton, Ross [ross.bur...@intel.com]
Sent: Thursday, May 18, 2017 2:10 PM
To: Bystricky, Juro
Cc: OE-core; Purdie, Richard; Joshua G Lock; Juro Bystricky
Subject: Re: [PATCH] kernel.bbclass: improve reproducibility


On 18 May 2017 at 22:05, Juro Bystricky 
> wrote:
Kernel and kernel modules contain hard coded paths referencing the host
build system. This is usually because the source code contains __FILE__
at some place. This prevents binary reproducibility. However, some compilers
allow remapping of the __FILE__ value. If we detect the compiler is capable
of doing this, we replace the source path $(S) part of __FILE__ by a string 
"/kernel-source".
For example:

Is this not something we should do globally?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Burton, Ross]

On 18 May 2017 at 22:05, Juro Bystricky  wrote:

> Kernel and kernel modules contain hard coded paths referencing the host
> build system. This is usually because the source code contains __FILE__
> at some place. This prevents binary reproducibility. However, some
> compilers
> allow remapping of the __FILE__ value. If we detect the compiler is capable
> of doing this, we replace the source path $(S) part of __FILE__ by a
> string "/kernel-source".
> For example:
>

Is this not something we should do globally?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel.bbclass: improve reproducibility

[Juro Bystricky]

Kernel and kernel modules contain hard coded paths referencing the host
build system. This is usually because the source code contains __FILE__
at some place. This prevents binary reproducibility. However, some compilers
allow remapping of the __FILE__ value. If we detect the compiler is capable
of doing this, we replace the source path $(S) part of __FILE__ by a string 
"/kernel-source".
For example:

​/​data/​master/​build-​repro/​tmp/​work-​shared/​qemux86/​kernel-​source/​drivers/​media/​v4l2-​core/​videobuf2-​core.​c

will be replaced by a reproducible value:

/kernel-source/drivers/media/v4l2-core/videobuf2-core.c.

Signed-off-by: Juro Bystricky 
---
 meta/classes/kernel.bbclass | 18 --
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 8954b28..0a2cd3d 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -249,6 +249,18 @@ python do_devshell_prepend () {
 
 addtask bundle_initramfs after do_install before do_deploy
 
+get_cc_option () {
+   # Check if KERNEL_CC supports the option "file-prefix-map".
+   # This option allows us to build images with __FILE__ values 
that do not
+   # contain the host build path.
+   cc_option_supported=`${KERNEL_CC} -Q --help=joined | grep 
ffile-prefix-map`
+   cc_extra=""
+   if [ $cc_option_supported =