I sent a v2 of this patch where this issue is corrected.
Alex
On Tue, 17 Oct 2023 at 16:15, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail....@lists.openembedded.org>
wrote:
>
> Thanks, I was completely sure in my head I had fixed this, but either
> I haven't, or it has been somehow lost.
>
> Alex
>
> On Tue, 17 Oct 2023 at 16:06, Jose Quaresma <quaresma.j...@gmail.com> wrote:
> >
> >
> >
> > Alexander Kanavin <alex.kana...@gmail.com> escreveu no dia terça,
> > 17/10/2023 à(s) 14:31:
> >>
> >> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
> >> why sstate is not being reused, but until now it had no tests that would
> >> ensure it works. This commit adds three basic scenarios:
> >>
> >> 1. make a change in a really basic, common recipe that is at the very root
> >> of dependency trees (quilt-native), and ensure that change is correctly
> >> discovered when
> >> building an image.
> >>
> >> 2. make a change in gcc-source recipe, which is somewhat special
> >> (operates in work-shared), and ensure that gcc-runtime builds track
> >> that down as well.
> >>
> >> 3. make a change in base_do_configure() definition from base.bbclass,
> >> which is not recipe-specific, but affects many basic recipes, and ensure
> >> that
> >> is correctly reported as well.
> >>
> >> The test itself actually runs twice:
> >> - first against a fully populated build directory, where
> >> the printdiff code is guaranteed to find the correct previous
> >> stamp that can be compared with in a predictable manner.
> >>
> >> - then in an empty build directory where the printdiff code
> >> goes to look in the sstate cache, and so the existence of the
> >> previous signature can be tested, but not the difference with it
> >> (what the exact difference would be is unpredictable as the
> >> sstate cache is indeed shared between many builds).
> >>
> >> Signed-off-by: Alexander Kanavin <a...@linutronix.de>
> >> ---
> >> .../base-do-configure-modified.bbclass | 3 +
> >> .../gcc-source/gcc-source_%.bbappend | 2 +
> >> .../quilt-native/quilt-native_%.bbappend | 2 +
> >> meta/lib/oeqa/selftest/cases/sstatetests.py | 104 ++++++++++++++++++
> >> 4 files changed, 111 insertions(+)
> >> create mode 100644
> >> meta-selftest/classes/base-do-configure-modified.bbclass
> >> create mode 100644
> >> meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> create mode 100644
> >> meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >>
> >> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass
> >> b/meta-selftest/classes/base-do-configure-modified.bbclass
> >> new file mode 100644
> >> index 00000000000..3f96827a428
> >> --- /dev/null
> >> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
> >> @@ -0,0 +1,3 @@
> >> +base_do_configure:append () {
> >> + echo "this changes base_do_configure() definiton"
> >> +}
> >> diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> new file mode 100644
> >> index 00000000000..205720982cb
> >> --- /dev/null
> >> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> @@ -0,0 +1,2 @@
> >> +# This bbappend is used to alter the recipe using the test_recipe.inc
> >> file created by tests.
> >> +include test_recipe.inc
> >> diff --git
> >> a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >> b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >> new file mode 100644
> >> index 00000000000..205720982cb
> >> --- /dev/null
> >> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >> @@ -0,0 +1,2 @@
> >> +# This bbappend is used to alter the recipe using the test_recipe.inc
> >> file created by tests.
> >> +include test_recipe.inc
> >> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> b/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> index bdad9088d37..29e31f2ef9f 100644
> >> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
> >> latestfiles = sorted(filedates.keys(), key=lambda f:
> >> filedates[f])[-2:]
> >> bb.siggen.compare_sigfiles(latestfiles[-2],
> >> latestfiles[-1], recursecb)
> >> self.assertEqual(recursecb_count,1)
> >> +
> >> +class SStatePrintdiff(SStateBase):
> >> + def run_test_printdiff_changerecipe(self, target, change_recipe,
> >> change_bbtask, change_content, expected_sametmp_output,
> >> expected_difftmp_output):
> >> + self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> >> +""")
> >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> >> + bitbake(target)
> >> + bitbake("-S none {}".format(target))
> >> + bitbake(change_bbtask)
> >> + self.write_recipeinc(change_recipe, change_content)
> >> + result_sametmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> + self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> >> +""")
> >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> >> + result_difftmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> + self.delete_recipeinc(change_recipe)
> >> + for item in expected_sametmp_output:
> >> + self.assertIn(item, result_sametmp.output)
> >> + for item in expected_difftmp_output:
> >> + self.assertIn(item, result_difftmp.output)
> >> +
> >> + def run_test_printdiff_changeconfig(self, target, change_content,
> >> expected_sametmp_output, expected_difftmp_output):
> >> + self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> >> +""")
> >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> >> + bitbake(target)
> >> + bitbake("-S none {}".format(target))
> >> + self.append_config(change_content)
> >> + result_sametmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> + self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> >> +""")
> >> + self.append_config(change_content)
> >> + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> >> + result_difftmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> + for item in expected_sametmp_output:
> >> + self.assertIn(item, result_sametmp.output)
> >> + for item in expected_difftmp_output:
> >> + self.assertIn(item, result_difftmp.output)
> >> +
> >> +
> >> + # Check if printdiff walks the full dependency chain from the image
> >> target to where the change is in a specific recipe
> >> + def test_image_minimal_vs_quilt(self):
> >> + expected_output = ("Task quilt-native:do_install couldn't be used
> >> from the cache because:",
> >> +"We need hash",
> >> +"most recent matching task was")
> >> + expected_sametmp_output = expected_output + ("Variable do_install
> >> value changed",'+ echo "this changes the task signature"')
> >> + expected_difftmp_output = expected_output
> >> +
> >> + self.run_test_printdiff_changerecipe("core-image-minimal",
> >> "quilt-native", "-c do_install quilt-native",
> >> +"""
> >> +do_install:append() {
> >> + echo "this changes the task signature"
> >> +}
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> +
> >> + # Check if changes to gcc-source (which uses tmp/work-shared) are
> >> correctly discovered
> >> + def test_gcc_runtime_vs_gcc_source(self):
> >> + expected_output = ("Task gcc-source-13.2.0:do_preconfigure
> >> couldn't be used from the cache because:",
> >
> >
> > We should avoid using the PV=13.2.0 explicitly imo otherwise it will need
> > to be updated with the recipe forever.
> >
> > The archiver selftest have a gcc-source-PV example
> > https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141
> >
> >> +"We need hash",
> >> +"most recent matching task was")
> >> + expected_sametmp_output = expected_output + ("Variable
> >> do_preconfigure value changed",'+ print("this changes the task
> >> signature")')
> >> + #FIXME: printdiff is supposed to find at least one preconfigure
> >> task signature in the sstate cache, but isn't able to
> >> + #expected_difftmp_output = expected_output
> >> + expected_difftmp_output = ()
> >> +
> >> + self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source",
> >> "-c do_preconfigure gcc-source-13.2.0",
> >
> >
> > Same here
> >
> > Jose
> >
> >>
> >> +"""
> >> +python do_preconfigure:append() {
> >> + print("this changes the task signature")
> >> +}
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> +
> >> + # Check if changing a really base task definiton is reported against
> >> multiple core recipes using it
> >> + def test_image_minimal_vs_base_do_configure(self):
> >> + expected_output = ("Task zstd-native:do_configure couldn't be
> >> used from the cache because:",
> >> +"Task texinfo-dummy-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task ldconfig-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task gettext-minimal-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task tzcode-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task makedevs-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task pigz-native:do_configure couldn't be used from the cache because:",
> >> +"Task update-rc.d-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"Task unzip-native:do_configure couldn't be used from the cache because:",
> >> +"Task gnu-config-native:do_configure couldn't be used from the cache
> >> because:",
> >> +"We need hash",
> >> +"most recent matching task was")
> >> + expected_sametmp_output = expected_output + ("Variable
> >> base_do_configure value changed",'+ echo "this changes
> >> base_do_configure() definiton"')
> >> + expected_difftmp_output = expected_output
> >> +
> >> + self.run_test_printdiff_changeconfig("core-image-minimal",
> >> +"""
> >> +INHERIT += "base-do-configure-modified"
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> --
> >> 2.39.2
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > José Quaresma
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189371):
https://lists.openembedded.org/g/openembedded-core/message/189371
Mute This Topic: https://lists.openembedded.org/mt/102017475/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
