Bug#1010660: britney: Crash with RecursionError: maximum recursion depth exceeded
Hi, tl;dr: we need to cook up a similar patch as bd375fdd85b4e00b89fb087760369bb6f3df0a20 for the migration phase of britney. We recently experienced this problem in Debian too. Traceback (most recent call last): File "/srv/release.debian.org/britney/code/b2/britney.py", line 1586, in Britney().main() File "/srv/release.debian.org/britney/code/b2/britney.py", line 1573, in main self.upgrade_testing() File "/srv/release.debian.org/britney/code/b2/britney.py", line 1244, in upgrade_testing self.run_auto_hinter() File "/srv/release.debian.org/britney/code/b2/britney.py", line 1518, in run_auto_hinter for lst in self.get_auto_hinter_hints(self.upgrade_me): File "/srv/release.debian.org/britney/code/b2/britney.py", line 1482, in get_auto_hinter_hints hint = find_related(e, set(), True) File "/srv/release.debian.org/britney/code/b2/britney.py", line 1471, in find_related if not find_related(p, hint): File "/srv/release.debian.org/britney/code/b2/britney.py", line 1471, in find_related if not find_related(p, hint): File "/srv/release.debian.org/britney/code/b2/britney.py", line 1471, in find_related if not find_related(p, hint): [Previous line repeated 989 more times] File "/srv/release.debian.org/britney/code/b2/britney.py", line 1465, in find_related hint.add(excuse.item) File "/srv/release.debian.org/britney/code/b2/britney2/migrationitem.py", line 79, in __hash__ if not self.version: RecursionError: maximum recursion depth exceeded We discussed it on IRC and Sebastian worked around it... [2022-08-28 UTC+2] [16:20:20] elbrus: With the blocks on nginx and openimageio/opencolorio removed, britney still crashes [16:30:11] s/nginx and//g [20:34:39] Sebastinas: so you added hints to "unbreak" britney? [20:34:48] yes [20:35:09] we should save the current state of the archive for britneys test suite [20:35:22] but I forgot how that worked [20:35:55] -*- elbrus will try to figure it out, hopefully in time [20:36:55] elbrus: The circular dependency of openimageio/opencolorio triggers an endless recursion in the auto-hinter [20:37:31] aha, now that's something that sounds like an actionaly bug report for britney [20:37:57] if we can reproduce the failure in a test Thankfully Niels joined the discussion as he experienced the issue before in the policy phase of britney: [20:38:12] I am surprised, I thought the auto-hinter had infinite-recursion detection already [20:38:48] nthykier: if it's not in a test, it can be broken ;) [20:39:33] I think I've seen these kind of errors a couple of times [20:39:41] not so much in Debian [20:48:20] https://bpa.st/YSQA [20:56:05] not sure that is infinite to be honest [20:56:49] the other solver had to be rewritten to be iterative because it would stack overflow (because some clusters involved more then 1000 packages in a grouping) [20:57:51] guess that time has come for the legay auto-hinter as well now [21:00:47] (the net effect is the same - it is just more whether you will find a logical bug in the code or the code is "corect" and we just surpassed the limit ) [21:00:53] correct* [21:01:56] nthykier: thanks for the information [21:03:42] nthykier: but how does making a solver iterative prevent recursion limits (it sounds like it's normally the cause of recursion) [21:09:36] elbrus: the iterative solution replaces the recursion (call stack) with a for-loop an a regular stack (python list). The latter "only" limited by the amount of memory you have, where the former is much more limited [21:09:52] (s/for-loop/loop/) [21:11:14] nthykier: do you happen to have a commit ID that was involved in that rewrite (to get inpiration from if I need it) [21:11:38] bd375fdd85b4e00b89fb087760369bb6f3df0a20 [21:12:40] thanks [21:13:11] 5 years ago Paul OpenPGP_signature Description: OpenPGP digital signature
Re: Bug#1010660: britney: Crash with RecursionError: maximum recursion depth exceeded
On 09 juin 2022 10:30, Paul Gevers wrote: > Hi Christian, Hi paul, [...] >>> RecursionError: maximum recursion depth exceeded > > Seems like Ubuntu also ran into the issue and (temporarily) applied > https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/commit/?id=83e6102613b37f3e464ecd269b9dd9c621075021 Thanks I'll try this patch is this bug reappears. Christian
Re: Bug#1010660: britney: Crash with RecursionError: maximum recursion depth exceeded
Hi Christian, On Fri, 2022-05-06 at 12:41 +0200, Christian Marillat wrote: I'm using britney (last commit normally) 2ccce826090ebc3f2cbdb26df3c5b0817f7a7cc2 You can download data used for this crash here : https://www.deb-multimedia.org/tests/britney-2022-05-06.tar.xz Since may 2, 2022 I see : Traceback (most recent call last): File "/usr/bin/britney.py", line 1583, in Britney().main() File "/usr/bin/britney.py", line 1570, in main self.upgrade_testing() File "/usr/bin/britney.py", line 1241, in upgrade_testing self.run_auto_hinter() File "/usr/bin/britney.py", line 1515, in run_auto_hinter for lst in self.get_auto_hinter_hints(self.upgrade_me): File "/usr/bin/britney.py", line 1479, in get_auto_hinter_hints hint = find_related(e, set(), True) File "/usr/bin/britney.py", line 1468, in find_related if not find_related(p, hint): File "/usr/bin/britney.py", line 1468, in find_related if not find_related(p, hint): File "/usr/bin/britney.py", line 1468, in find_related if not find_related(p, hint): [Previous line repeated 989 more times] File "/usr/bin/britney.py", line 1462, in find_related hint.add(excuse.item) File "/usr/lib/python3/dist-packages/britney2/migrationitem.py", line 79, in __hash__ if not self.version: RecursionError: maximum recursion depth exceeded Seems like Ubuntu also ran into the issue and (temporarily) applied https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/commit/?id=83e6102613b37f3e464ecd269b9dd9c621075021 Paul OpenPGP_signature Description: OpenPGP digital signature
Processed: Re: Bug#1010660: britney: Crash with RecursionError: maximum recursion depth exceeded
Processing control commands:
> reassign -1 release.debian.org
Bug #1010660 {Done: Christian Marillat }
[release.debian.org] britney: Crash with RecursionError:
Ignoring request to reassign bug #1010660 to the same package
> retitle -1 britney: Crash with RecursionError: maximum recursion depth
> exceeded
Bug #1010660 {Done: Christian Marillat }
[release.debian.org] britney: Crash with RecursionError:
Changed Bug title to 'britney: Crash with RecursionError: maximum recursion
depth exceeded' from 'britney: Crash with RecursionError:'.
--
1010660: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010660
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Re: Bug#1010660: britney: Crash with RecursionError: maximum recursion depth exceeded
Control: reassign -1 release.debian.org Control: retitle -1 britney: Crash with RecursionError: maximum recursion depth exceeded On Fri, 2022-05-06 at 12:41 +0200, Christian Marillat wrote: > I'm using britney (last commit normally) > 2ccce826090ebc3f2cbdb26df3c5b0817f7a7cc2 britney is not part of dak and is not maintained by the FTP team; reassigned to the correct pseudo-package. > You can download data used for this crash here : > > https://www.deb-multimedia.org/tests/britney-2022-05-06.tar.xz > > Since may 2, 2022 I see : > > Traceback (most recent call last): > File "/usr/bin/britney.py", line 1583, in > Britney().main() > File "/usr/bin/britney.py", line 1570, in main > self.upgrade_testing() > File "/usr/bin/britney.py", line 1241, in upgrade_testing > self.run_auto_hinter() > File "/usr/bin/britney.py", line 1515, in run_auto_hinter > for lst in self.get_auto_hinter_hints(self.upgrade_me): > File "/usr/bin/britney.py", line 1479, in get_auto_hinter_hints > hint = find_related(e, set(), True) > File "/usr/bin/britney.py", line 1468, in find_related > if not find_related(p, hint): > File "/usr/bin/britney.py", line 1468, in find_related > if not find_related(p, hint): > File "/usr/bin/britney.py", line 1468, in find_related > if not find_related(p, hint): > [Previous line repeated 989 more times] > File "/usr/bin/britney.py", line 1462, in find_related > hint.add(excuse.item) > File "/usr/lib/python3/dist-packages/britney2/migrationitem.py", > line 79, in __hash__ > if not self.version: > RecursionError: maximum recursion depth exceeded
