On 2022-03-13 04:37, Bruce Ashfield wrote:
On Sat, Mar 12, 2022 at 2:12 PM Jimmy Assarsson <jimmyassars...@gmail.com
<mailto:jimmyassars...@gmail.com>> wrote:
On 2022-03-09 16:47, Bruce Ashfield wrote:
> On Sat, Feb 26, 2022 at 2:17 AM Jimmy Assarsson
> <jimmyassars...@gmail.com <mailto:jimmyassars...@gmail.com>> wrote:
>> On 2022-02-24 16:41, Bruce Ashfield wrote:
>>> On Tue, Feb 22, 2022 at 3:42 AM Jimmy Assarsson
>>> <jimmyassars...@gmail.com <mailto:jimmyassars...@gmail.com>> wrote:
>>>> On 2022-02-21 05:13, Bruce Ashfield wrote:
>>>>> on 19/02/2022 Jimmy Assarsson wrote:
>>>>>> On 2022-02-19 20:42, Bruce Ashfield wrote:
>>>>>>> On Sat, Feb 19, 2022 at 5:28 AM Jimmy Assarsson
>>>>>>> <jimmyassars...@gmail.com <mailto:jimmyassars...@gmail.com>> wrote:
>>>>>>>>
>>>>>>>> When patching a kernel git repository, in a recipe inheriting
kernel-yocto.bbclass,
>>>>>>>> the resulting commit hash will become different every time the
source is unpacked and patched.
>>>>>>>>
>>>>>>>> This is a problem that causes non-reproducible builds, when the
commit hash is built
>>>>>>>> into the kernel (CONFIG_LOCALVERSION_AUTO=y).
>>>>>>>>
>>>>>>>>
>>>>>>>> Currently it is not a problem in linux-yocto, since an empty
.scmversion is
>>>>>>>> created in kernel_do_configure [1]. This is preventing the kernel
build from
>>>>>>>> generating its own .scmversion.
>>>>>>>>
>>>>>>>> If removing this commit, setting CONFIG_LOCALVERSION_AUTO=y and
applying any
>>>>>>>> patch in the linux-yocto recipe, this will result in a
non-reproducible build.
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>>> diff --git a/tools/kgit-s2q b/tools/kgit-s2q
>>>>>>>> index 706783e..b46a138 100755
>>>>>>>> --- a/tools/kgit-s2q
>>>>>>>> +++ b/tools/kgit-s2q
>>>>>>>> @@ -558,7 +558,7 @@ do
>>>>>>>> echo "($APPLIED/$COUNT) `basename $i`"
>>>>>>>> fi
>>>>>>>>
>>>>>>>> - git am --keep-non-patch $DIR/$i > /dev/null 2>&1
>>>>>>>> + git am --keep-non-patch --committer-date-is-author-date $DIR/$i
> /dev/null 2>&1
>>>>>>>> if [ $? != 0 ];then
>>>>>>>> git am --abort > /dev/null 2>&1
>>>>>>>> echo "[INFO]: check of $DIR/$i with \"git am\"
did not pass, trying reduced context."
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm not sure if this is a proper solution to fix the problem or
what side effects it may introduce?
>>>>>>>
>>>>>>> There's nothing fundamentally wrong with that solution, but there
are
>>>>>>> modes for the kernel builds where we absolutely do not want the
>>>>>>> reproducible build (and older dates) to be in effect (see my
comments
>>>>>>> on the OE core mailing list when the ability to disable
reproducible
>>>>>>> builds was almost removed).
>>>>>>
>>>>>> Thanks, now I've looked it up.
>>>>>>
>>>>>>> We could add an option to the patch queue management that was
enabled
>>>>>>> when reproducible builds are in play, so git am could operate like
>>>>>>> that.
>>>>>>>
>>>>>>> Alternatively, you could bbappend the other kernel's do_configure
and
>>>>>>> do the same .scmversion trick to make sure that everything is
>>>>>>> consistent.
>>>>>>
>>>>>> I don't follow, can you be more specific?
>>>>>> We encountered this problem when building linux-imx (from
meta-freescale).
>>>>>> Preferably the .scmversion workaround in [1] should be dropped, so
that
>>>>>> there is no need to reinvent/mimic the output of
>>>>>> linux/scripts/setlocalversion, in a bitbake recipe.
>>>>>
>>>>> I see no compelling reason to drop the .scmversion in the main
kernel.bbclass
>>>>> (at least not in the short term). It has been there for a
significant amount
>>>>> of time, and removing it now would very likely cause some ripple
effects
>>>>> througout the ecosystem (it is there for a few different issues).
While not
>>>>> elegant, it is an available and workable solution to the problem it
is
>>>>> solving.
>>>>
>>>> I agree. And obviously this is not a big issue for others.
>>>> There have been at least two failed attempts on removing this
[4][5][6].
>>>>
>>>>> I was suggesting that anyone can bbappend any kernel recipe to do
the similar
>>>>> .scmversion set (assuming they aren't using the core kernel bbclass
>>>>> configure()).
>>>>>
>>>>> But from your next statement, that isn't going to work as the sole
solution,
>>>>> since yes, that will only restore where we keep the git rev out of
the
>>>>> localversion.
>>>>>
>>>>>>
>>>>>> To be clear, we want the result of CONFIG_LOCALVERSION_AUTO=y, but
>>>>>> with a reproducible output.
>>>>>
>>>>> And that's actually part of the reason the .scmversion is set, the
>>>>> setlocalversion script was appending values in some configurations
and
>>>>> combinations of patching, etc, such that issues were popping up with
version
>>>>> matching.
>>>>>
>>>>> Part of getting that to work, definitely could be to apply patches
with the
>>>>> author date, but that has to be triggered on reproducible builds
being
>>>>> enabled (which is of course the default) and also preserving the use
of the
>>>>> .scmversion for the baseline / basic kernel configuration routines.
>>>>>
>>>>> .. so one config to say "don't set scmversion" and another to say
"apply
>>>>> patches with author date" (this one toggling based on reproducible
builds).
>>>>> And then both scenarios can be supported.
>>>>
>>>> Currently I'm only interested in fixing the "apply patches with
>>>> author date", to get reproducible builds.
>>>>
>>>> So we will end up with a new config in kernel-yocto.bbclass, since
this
>>>> is the only location where kgit-s2q is used.
>>>> And we will get a new option in kgit-s2q, that will result in
>>>> "git am --committer-date-is-author-date" being used.
>>>>
>>>> Any name suggestions for the config and option?
>>>> KERNEL_PATCH_WITH_AUTHOR_DATE
>>>> KERNEL_PATCH_REPRODUCIBLE_HASH
>>>> KERNEL_PATCH_REPRODUCIBLE_COMMIT
>>>
>>> It is better to abstract it behind the reproducible feature
>>> description, since there are other kernel reproducibility components
>>> that exist, or may appear in the future.
>>
>> Sure.
>>
>>> If you want, I can have a go at the patch, which will allow me to run
>>> my local tests and put it in my next consolidated pull request.
>>
>> That would be great :)
>> Thanks!
>
> It took me a bit longer than expected, but the lightly tested v1 is
> here:
https://git.yoctoproject.org/poky-contrib/commit/?h=zedd/kernel&id=cfbc3bf788ba1fc7d5f4de1a95c72a66243562e6
<https://git.yoctoproject.org/poky-contrib/commit/?h=zedd/kernel&id=cfbc3bf788ba1fc7d5f4de1a95c72a66243562e6>
>
> And by lightly tested, I mean I can still patch the kernel and things
> didn't explode in flames. i haven't checked if the commit dates, etc,
> do what you were looking for.
Thanks Bruce!
Now I've tested it.
The changes in kernel-yocto.bbclass looks good to me.
In kgit-s2q, the check of $commit_sha_type is missing square brackets:
- if "$commit_sha_type" == "author"; then
+ if [ "$commit_sha_type" == "author" ]; then
Once fixed, the commit dates are as expected and test_reproducible_builds
passes.
oops. Yes, I had fixed that as well, but didn't push it, as I was running out
the door for a week of vacation.
The commit-sha option and arguments looks good.
However the help description of the 'date' option argument, is not
completely clear, at least not to me:
--commit-sha type of commit sha's to create: 'author date' or 'commit
date' (author or date)
default is 'commit date'
If it wasn't for 'author date', I would have assumed 'commit date', gives
the same result. 'applied date' or similar, makes more sense to me.
I'll clarify the help a bit when I push the final version and send it to OE
core next week.
Thanks for the testing!
Bruce
Gentle ping :)
Have you had time to look into this?
Best regards,
jimmy
Best regards,
jimmy
> I'm heading out on vacation for the next week, so I won't send this
> for merging into OE core yet, but feel free to try it out, send
> feedback and I'll do a v2 when I return (after March 19th).
>
> Bruce
>
>>
>> Best regards,
>> jimmy
>>
>>> Bruce
>>>
>>>> --patch-with-author-date
>>>> --apply-patch-with-author-date
>>>> --committer-date-is-author-date
>>>>
>>>>
>>>> [4] https://patchwork.openembedded.org/patch/138333
<https://patchwork.openembedded.org/patch/138333>
>>>> [5] https://patchwork.openembedded.org/patch/171479
<https://patchwork.openembedded.org/patch/171479>
>>>> [6] https://patchwork.openembedded.org/patch/171513
<https://patchwork.openembedded.org/patch/171513>
>>>>
>>>>
>>>> Best regards,
>>>> jimmy
>>>>
>>>>
>>>>> Bruce
>>>>>
>>>>>>
>>>>>> Best regards,
>>>>>> jimmy
>>>>>>
>>>>>>
>>>>>>> Bruce
>>>>>>>
>>>>>>>>
>>>>>>>> [1]
https://git.yoctoproject.org/poky/tree/meta/classes/kernel.bbclass?h=honister#n589
<https://git.yoctoproject.org/poky/tree/meta/classes/kernel.bbclass?h=honister#n589>
>>>>>>>> [2]
https://git.yoctoproject.org/meta-freescale/tree/classes/fsl-kernel-localversion.bbclass?h=honister
<https://git.yoctoproject.org/meta-freescale/tree/classes/fsl-kernel-localversion.bbclass?h=honister>
>>>>>>>> [3]
https://github.com/OE4T/meta-tegra/blob/honister/recipes-kernel/linux/linux-tegra_4.9.bb
<https://github.com/OE4T/meta-tegra/blob/honister/recipes-kernel/linux/linux-tegra_4.9.bb>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at
its end
- "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11213):
https://lists.yoctoproject.org/g/linux-yocto/message/11213
Mute This Topic: https://lists.yoctoproject.org/mt/89251386/21656
Group Owner: linux-yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
