Re: [OE-core] [bitbake-devel] [PATCH 2/2] fetch2: Fix race condition in latest_revision
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I believe the behaviour we observed was actually caused by a bug in one of our earlier patch sets. I spent a couple hours today trying to recreate the issue, without any luck. I'd therefore skip this particular patch. - -- With best regards Jasper Orschulko DevOps Engineer Tel. +49 30 58 58 14 265 Fax +49 30 58 58 14 999 jasper.orschu...@iris-sensing.com • • • • • • • • • • • • • • • • • • • • • • • • • • iris-GmbH infrared & intelligent sensors Schnellerstraße 1-5 | 12439 Berlin https://iris-sensing.com/ On Mon, 2021-11-08 at 11:28 +0100, Alexander Kanavin wrote: > It helps if you can provide a specific example/test case for this. > > Alex > > On Mon, 8 Nov 2021 at 11:26, Martin Koppehel wrote: > > -- snip > > > I'm afraid I don't understand why this is a race condition? Where > > is the timeout > > > that stops one being set? > > > > The commit message is misleading here, will change that. It is > > actually > > not a race condition as we originally thought, but more a problem > > of > > recursive calls to _latest_revision. > > > > The git fetcher does something similar in _lsremote, where the git > > lsremote would be called recursively to populate the environment, > > and > > there's code in place to prevent that. > > > > In our case, since we persist the revs, we have to filter out empty > > revs > > because otherwise the next repo fetcher invocation would also try > > to > > populate the environment but then revs[key] would already be set to > > an > > empty string, causing a metadata mismatch later. > > > > Therefore we only populate the revision cache if _latest_revision > > actually returns a valid value. > > We did reproduce this with multiple recipes that used up the same > > SRC_URI and therefore influencing each other, this case works fine > > now. > > > > Cheers, > > Martin > > > > > > > > -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE4WyPMIC5Ap4+Ooo1Ygqew07VMNUFAmGL6P8ACgkQYgqew07V MNVKBQf/Z/PE68CI1L3F1jeHm9jX+DNs1dyA7IB6fOAHdeqYvKNKpbedpHXVHFlp oalj3jIEDJMKE8b8LmH/VrRhLTr68PSEshVIbHOKQQfJmy6FnbQw5VXTMZON92AS phiP4H4jMvr8sx942d+YoVwtk1iMO6+oZwUEx2k5uGiyVeayqMvs0XyrHQY65GhT 7RG9ZSOXutnkF1NCHto4ln5kb8G+MUcS3D57FRbsiGV4XvIaMTqFbn3ub6NP2x5H qMjQwWXLrF1eIQc6hVBXxPnloyPvZESp98/rTDHdBQsJB9DbS2akQ8MZBM+EIiff wtVJRNJa0jG/uDvF3wLCoO+ycHF8SQ== =vnW2 -END PGP SIGNATURE- -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158078): https://lists.openembedded.org/g/openembedded-core/message/158078 Mute This Topic: https://lists.openembedded.org/mt/86878868/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [bitbake-devel] [PATCH 2/2] fetch2: Fix race condition in latest_revision
It helps if you can provide a specific example/test case for this. Alex On Mon, 8 Nov 2021 at 11:26, Martin Koppehel wrote: > -- snip > > I'm afraid I don't understand why this is a race condition? Where is the > timeout > > that stops one being set? > > The commit message is misleading here, will change that. It is actually > not a race condition as we originally thought, but more a problem of > recursive calls to _latest_revision. > > The git fetcher does something similar in _lsremote, where the git > lsremote would be called recursively to populate the environment, and > there's code in place to prevent that. > > In our case, since we persist the revs, we have to filter out empty revs > because otherwise the next repo fetcher invocation would also try to > populate the environment but then revs[key] would already be set to an > empty string, causing a metadata mismatch later. > > Therefore we only populate the revision cache if _latest_revision > actually returns a valid value. > We did reproduce this with multiple recipes that used up the same > SRC_URI and therefore influencing each other, this case works fine now. > > Cheers, > Martin > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#157967): https://lists.openembedded.org/g/openembedded-core/message/157967 Mute This Topic: https://lists.openembedded.org/mt/86878868/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [bitbake-devel] [PATCH 2/2] fetch2: Fix race condition in latest_revision
On Fri, 2021-11-05 at 14:30 +0100, Jasper Orschulko via lists.openembedded.org wrote: > From: Martin Koppehel > > Setting latest_revision contained a race condition, where it would be > set to an empty string, if the hash calculation function would take to > long. > > Signed-off-by: Jasper Orschulko > --- > lib/bb/fetch2/__init__.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py > index 6a38cb09..9dc23d05 100644 > --- a/lib/bb/fetch2/__init__.py > +++ b/lib/bb/fetch2/__init__.py > @@ -1602,7 +1602,9 @@ class FetchMethod(object): > try: > return revs[key] > except KeyError: > -revs[key] = rev = self._latest_revision(ud, d, name) > +rev = self._latest_revision(ud, d, name) > +if rev != '': > +revs[key] = rev > return rev > > def sortable_revision(self, ud, d, name): I'm afraid I don't understand why this is a race condition? Where is the timeout that stops one being set? Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#157947): https://lists.openembedded.org/g/openembedded-core/message/157947 Mute This Topic: https://lists.openembedded.org/mt/86878868/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
