Alexander Kanavin <alex.kana...@gmail.com> escreveu no dia quarta,
18/10/2023 à(s) 09:48:
> I sent a v2 of this patch where this issue is corrected.
>
Thanks
>
> Alex
>
> On Tue, 17 Oct 2023 at 16:15, Alexander Kanavin via
> lists.openembedded.org <alex.kanavin=gmail....@lists.openembedded.org>
> wrote:
> >
> > Thanks, I was completely sure in my head I had fixed this, but either
> > I haven't, or it has been somehow lost.
> >
> > Alex
> >
> > On Tue, 17 Oct 2023 at 16:06, Jose Quaresma <quaresma.j...@gmail.com>
> wrote:
> > >
> > >
> > >
> > > Alexander Kanavin <alex.kana...@gmail.com> escreveu no dia terça,
> 17/10/2023 à(s) 14:31:
> > >>
> > >> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
> > >> why sstate is not being reused, but until now it had no tests that
> would
> > >> ensure it works. This commit adds three basic scenarios:
> > >>
> > >> 1. make a change in a really basic, common recipe that is at the very
> root
> > >> of dependency trees (quilt-native), and ensure that change is
> correctly discovered when
> > >> building an image.
> > >>
> > >> 2. make a change in gcc-source recipe, which is somewhat special
> > >> (operates in work-shared), and ensure that gcc-runtime builds track
> > >> that down as well.
> > >>
> > >> 3. make a change in base_do_configure() definition from base.bbclass,
> > >> which is not recipe-specific, but affects many basic recipes, and
> ensure that
> > >> is correctly reported as well.
> > >>
> > >> The test itself actually runs twice:
> > >> - first against a fully populated build directory, where
> > >> the printdiff code is guaranteed to find the correct previous
> > >> stamp that can be compared with in a predictable manner.
> > >>
> > >> - then in an empty build directory where the printdiff code
> > >> goes to look in the sstate cache, and so the existence of the
> > >> previous signature can be tested, but not the difference with it
> > >> (what the exact difference would be is unpredictable as the
> > >> sstate cache is indeed shared between many builds).
> > >>
> > >> Signed-off-by: Alexander Kanavin <a...@linutronix.de>
> > >> ---
> > >> .../base-do-configure-modified.bbclass | 3 +
> > >> .../gcc-source/gcc-source_%.bbappend | 2 +
> > >> .../quilt-native/quilt-native_%.bbappend | 2 +
> > >> meta/lib/oeqa/selftest/cases/sstatetests.py | 104
> ++++++++++++++++++
> > >> 4 files changed, 111 insertions(+)
> > >> create mode 100644
> meta-selftest/classes/base-do-configure-modified.bbclass
> > >> create mode 100644
> meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >> create mode 100644
> meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >>
> > >> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass
> b/meta-selftest/classes/base-do-configure-modified.bbclass
> > >> new file mode 100644
> > >> index 00000000000..3f96827a428
> > >> --- /dev/null
> > >> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
> > >> @@ -0,0 +1,3 @@
> > >> +base_do_configure:append () {
> > >> + echo "this changes base_do_configure() definiton"
> > >> +}
> > >> diff --git
> a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >> new file mode 100644
> > >> index 00000000000..205720982cb
> > >> --- /dev/null
> > >> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >> @@ -0,0 +1,2 @@
> > >> +# This bbappend is used to alter the recipe using the
> test_recipe.inc file created by tests.
> > >> +include test_recipe.inc
> > >> diff --git
> a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >> new file mode 100644
> > >> index 00000000000..205720982cb
> > >> --- /dev/null
> > >> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >> @@ -0,0 +1,2 @@
> > >> +# This bbappend is used to alter the recipe using the
> test_recipe.inc file created by tests.
> > >> +include test_recipe.inc
> > >> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py
> b/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> index bdad9088d37..29e31f2ef9f 100644
> > >> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
> > >> latestfiles = sorted(filedates.keys(), key=lambda f:
> filedates[f])[-2:]
> > >> bb.siggen.compare_sigfiles(latestfiles[-2],
> latestfiles[-1], recursecb)
> > >> self.assertEqual(recursecb_count,1)
> > >> +
> > >> +class SStatePrintdiff(SStateBase):
> > >> + def run_test_printdiff_changerecipe(self, target, change_recipe,
> change_bbtask, change_content, expected_sametmp_output,
> expected_difftmp_output):
> > >> + self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> > >> +""")
> > >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> > >> + bitbake(target)
> > >> + bitbake("-S none {}".format(target))
> > >> + bitbake(change_bbtask)
> > >> + self.write_recipeinc(change_recipe, change_content)
> > >> + result_sametmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> + self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> > >> +""")
> > >> + self.track_for_cleanup(self.topdir +
> "/tmp-sstateprintdiff-2")
> > >> + result_difftmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> + self.delete_recipeinc(change_recipe)
> > >> + for item in expected_sametmp_output:
> > >> + self.assertIn(item, result_sametmp.output)
> > >> + for item in expected_difftmp_output:
> > >> + self.assertIn(item, result_difftmp.output)
> > >> +
> > >> + def run_test_printdiff_changeconfig(self, target,
> change_content, expected_sametmp_output, expected_difftmp_output):
> > >> + self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> > >> +""")
> > >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> > >> + bitbake(target)
> > >> + bitbake("-S none {}".format(target))
> > >> + self.append_config(change_content)
> > >> + result_sametmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> + self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> > >> +""")
> > >> + self.append_config(change_content)
> > >> + self.track_for_cleanup(self.topdir +
> "/tmp-sstateprintdiff-2")
> > >> + result_difftmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> + for item in expected_sametmp_output:
> > >> + self.assertIn(item, result_sametmp.output)
> > >> + for item in expected_difftmp_output:
> > >> + self.assertIn(item, result_difftmp.output)
> > >> +
> > >> +
> > >> + # Check if printdiff walks the full dependency chain from the
> image target to where the change is in a specific recipe
> > >> + def test_image_minimal_vs_quilt(self):
> > >> + expected_output = ("Task quilt-native:do_install couldn't be
> used from the cache because:",
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> + expected_sametmp_output = expected_output + ("Variable
> do_install value changed",'+ echo "this changes the task signature"')
> > >> + expected_difftmp_output = expected_output
> > >> +
> > >> + self.run_test_printdiff_changerecipe("core-image-minimal",
> "quilt-native", "-c do_install quilt-native",
> > >> +"""
> > >> +do_install:append() {
> > >> + echo "this changes the task signature"
> > >> +}
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> +
> > >> + # Check if changes to gcc-source (which uses tmp/work-shared)
> are correctly discovered
> > >> + def test_gcc_runtime_vs_gcc_source(self):
> > >> + expected_output = ("Task gcc-source-13.2.0:do_preconfigure
> couldn't be used from the cache because:",
> > >
> > >
> > > We should avoid using the PV=13.2.0 explicitly imo otherwise it will
> need to be updated with the recipe forever.
> > >
> > > The archiver selftest have a gcc-source-PV example
> > >
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141
> > >
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> + expected_sametmp_output = expected_output + ("Variable
> do_preconfigure value changed",'+ print("this changes the task
> signature")')
> > >> + #FIXME: printdiff is supposed to find at least one
> preconfigure task signature in the sstate cache, but isn't able to
> > >> + #expected_difftmp_output = expected_output
> > >> + expected_difftmp_output = ()
> > >> +
> > >> + self.run_test_printdiff_changerecipe("gcc-runtime",
> "gcc-source", "-c do_preconfigure gcc-source-13.2.0",
> > >
> > >
> > > Same here
> > >
> > > Jose
> > >
> > >>
> > >> +"""
> > >> +python do_preconfigure:append() {
> > >> + print("this changes the task signature")
> > >> +}
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> +
> > >> + # Check if changing a really base task definiton is reported
> against multiple core recipes using it
> > >> + def test_image_minimal_vs_base_do_configure(self):
> > >> + expected_output = ("Task zstd-native:do_configure couldn't
> be used from the cache because:",
> > >> +"Task texinfo-dummy-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task ldconfig-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task gettext-minimal-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task tzcode-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task makedevs-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task pigz-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task update-rc.d-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task unzip-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task gnu-config-native:do_configure couldn't be used from the cache
> because:",
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> + expected_sametmp_output = expected_output + ("Variable
> base_do_configure value changed",'+ echo "this changes
> base_do_configure() definiton"')
> > >> + expected_difftmp_output = expected_output
> > >> +
> > >> + self.run_test_printdiff_changeconfig("core-image-minimal",
> > >> +"""
> > >> +INHERIT += "base-do-configure-modified"
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> --
> > >> 2.39.2
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > José Quaresma
> >
> >
> >
>
--
Best regards,
José Quaresma
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189373):
https://lists.openembedded.org/g/openembedded-core/message/189373
Mute This Topic: https://lists.openembedded.org/mt/102017475/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
