Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Sean Whitton]

control: severity -1 important

Hello,

On Tue 11 Oct 2022 at 05:05PM -07, Sean Whitton wrote:

> control: reassign -1 cl-ironclad
> control: retitle -1 cl-ironclad: fails to compile against recent SBCL on 
> 32-bit ARM architectures
>
> I intend to resolve this bug by disabling the failing compilation.
> SBCL upstream do not think there is evidence of a serious SBCL bug.

This reduces the severity of this bug, rather than resolving it.
SBCL is not the only CL implementation in Debian.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Processed: Re: Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Debian Bug Tracking System]

Processing control commands:

> severity -1 important
Bug #1016750 [cl-ironclad] cl-ironclad: fails to compile against recent SBCL on 
32-bit ARM architectures
Severity set to 'important' from 'serious'

-- 
1016750: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016750
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Processed: Re: Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Debian Bug Tracking System]

Processing control commands:

> reassign -1 cl-ironclad
Bug #1016750 [src:sbcl, src:cl-ironclad] sbcl breaks cl-ironclad autopkgtest on 
armhf: Heap exhausted, game over.
Bug reassigned from package 'src:sbcl, src:cl-ironclad' to 'cl-ironclad'.
No longer marked as found in versions sbcl/2:2.2.6-2 and cl-ironclad/0.57-1.
Ignoring request to alter fixed versions of bug #1016750 to the same values 
previously set
> retitle -1 cl-ironclad: fails to compile against recent SBCL on 32-bit ARM 
> architectures
Bug #1016750 [cl-ironclad] sbcl breaks cl-ironclad autopkgtest on armhf: Heap 
exhausted, game over.
Changed Bug title to 'cl-ironclad: fails to compile against recent SBCL on 
32-bit ARM architectures' from 'sbcl breaks cl-ironclad autopkgtest on armhf: 
Heap exhausted, game over.'.

-- 
1016750: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016750
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Sean Whitton]

control: reassign -1 cl-ironclad
control: retitle -1 cl-ironclad: fails to compile against recent SBCL on 32-bit 
ARM architectures

I intend to resolve this bug by disabling the failing compilation.
SBCL upstream do not think there is evidence of a serious SBCL bug.

-- 
Sean Whitton

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Paul Gevers]

Hi Sean,

On 09-08-2022 18:17, Sean Whitton wrote:

I first looked at  and
the failure doesn't show up there -- do you know what that would be?


Yes, it's the difficulty of making good UI. In the past we reported the 
last overall result, but for testing that includes all tests with 
${random} package from unstable so that could be changing all the time, 
so we changed that to only report "pure" runs. It's on our wishlist to 
change it again and report both flavors on the same page, but we need a 
reasonable design to do that without causing too much confusion.


Paul


OpenPGP_signature
Description: OpenPGP digital signature

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Sean Whitton]

Hello,

On Tue 09 Aug 2022 at 07:46AM +02, Paul Gevers wrote:

> Hi Sean,
>
> On 09-08-2022 05:08, Sean Whitton wrote:
>> It looks like Lisp just ran out of memory.
>
> Yes, but it does so systematically.
>
>> Indeed, I can't see this
>> failure on debci.debian.org at present,
>
> Huh? Did you check https://ci.debian.net/packages/c/cl-ironclad/testing/armhf/
> or https://ci.debian.net/packages/c/cl-ironclad/testing/armel/ You'll see
> there that cl-ironclad was retried with sbcl about 11 + 10 times and every
> time it failed (and never succeeded).
>
>> which makes sense if it's a
>> random OOM problem on weaker architectures like armhf -- so, not the
>> fault of the new sbcl upload.
>
> This isn't random. And, our armhf box has 255 GB of RAM (and armel VM has 26
> GB), so running out of memory isn't likely. What can happen is that threads
> use too much resources for the address space, but that's something under
> control of the packages in question if I'm correct (but please correct me if I
> misunderstand). I'm not saying it's (easily) fixable, just that it
> systematically runs out of reachable memory during the particular test.

Right, it's not random.  I was looking at the page for unstable.

I first looked at  and
the failure doesn't show up there -- do you know what that would be?
Then I clicked on unstable but not testing, it would seem.

I'll write to upstream about this.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Paul Gevers]

Hi Sean,

On 09-08-2022 05:08, Sean Whitton wrote:

It looks like Lisp just ran out of memory.


Yes, but it does so systematically.


Indeed, I can't see this
failure on debci.debian.org at present,


Huh? Did you check 
https://ci.debian.net/packages/c/cl-ironclad/testing/armhf/ or 
https://ci.debian.net/packages/c/cl-ironclad/testing/armel/ You'll see 
there that cl-ironclad was retried with sbcl about 11 + 10 times and 
every time it failed (and never succeeded).



which makes sense if it's a
random OOM problem on weaker architectures like armhf -- so, not the
fault of the new sbcl upload.


This isn't random. And, our armhf box has 255 GB of RAM (and armel VM 
has 26 GB), so running out of memory isn't likely. What can happen is 
that threads use too much resources for the address space, but that's 
something under control of the packages in question if I'm correct (but 
please correct me if I misunderstand). I'm not saying it's (easily) 
fixable, just that it systematically runs out of reachable memory during 
the particular test.



Is it possible to label the tests as
flaky on only a single architecture?


Yes [1], but I'm not seeing flaky behavior.

Paul

[1] add the skippable restriction to the test you want to be able to 
ignore and exit with exit code 77 when the failure happens on the 
architecture you want to ignore. If you want to skip the test on armhf 
and armel altogether, I rather suggest you use an "Architecture: !armel 
!armhf" line in the d/t/control section of the test.


OpenPGP_signature
Description: OpenPGP digital signature

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Sean Whitton]

Hello Paul,

On Sat 06 Aug 2022 at 04:19PM +02, Paul Gevers wrote:

> With a recent upload of sbcl the autopkgtest of cl-ironclad fails in testing
> when that autopkgtest is run with the binary packages of sbcl from
> unstable. It passes when run with only packages from testing.
> 
> ; wrote
>   
> /tmp/autopkgtest-lxc.d2c27h4h/downtmp/autopkgtest_tmp/usr/share/common-lisp/source/ironclad/src/ciphers/tea-tmp93OFNPHA.fasl
> ; compilation finished in 0:00:00.160
> ; compiling file
>   "/usr/share/common-lisp/source/ironclad/src/ciphers/threefish.lisp" (written
>  18 FEB 2022 01:39:10 PM):
> Heap exhausted during garbage collection: 16 bytes available, 48 requested.
> Immobile Object Counts
>  Gen   GF type  fdefn symbol   code  Boxed   ConsRaw   Code  SmMix   Mixed
>  LgRaw LgCode  LgMix Waste%   AllocTrig   Dirty GCs Mem-age
>   1 00  0  0  0   7121939   9668  5273 827
>   0  0 17   19.96186748032499253   18850   1   1.1503
>   2 00  0  0  0  19865   2766  12586 848361681
>   10 34127   11.7   137430448 5368709   23089   0   0.9496
>   3 00  0  0  0  45116   7190   7447604   35521673
>   2394673311.0   270091288 2007806   0   0.4559
>   4 00  0  0  0  0  0  0  0  0   0
>   0  0  00.0   0 200   0   0   0.
>   5 00  0  0  0  0  0  0  0  0   0
>   0  0  00.0   0 200   0   0   0.
>   6 00  0  0  0   1370471   1242   3507314  40
>   2551342811.930584464 200 251   0   0.
> Tot 00  0  0  0  73472  11366  30943   4200   4975   4221
> 5046357566.9   499973680 [93.1% of 536870912 max]
> GC control variables:
>*GC-INHIBIT* = true
>*GC-PENDING* = true
> fatal error encountered in SBCL pid 1357:
> Heap exhausted, game over.

It looks like Lisp just ran out of memory.  Indeed, I can't see this
failure on debci.debian.org at present, which makes sense if it's a
random OOM problem on weaker architectures like armhf -- so, not the
fault of the new sbcl upload.  Is it possible to label the tests as
flaky on only a single architecture?

-- 
Sean Whitton

Bug#1016750: sbcl breaks cl-ironclad autopkgtest on armhf: Heap exhausted, game over.

[Paul Gevers]

Source: sbcl, cl-ironclad
Control: found -1 sbcl/2:2.2.6-2
Control: found -1 cl-ironclad/0.57-1
Severity: serious
Tags: sid bookworm
User: debian...@lists.debian.org
Usertags: breaks needs-update

Dear maintainer(s),

With a recent upload of sbcl the autopkgtest of cl-ironclad fails in 
testing when that autopkgtest is run with the binary packages of sbcl 
from unstable. It passes when run with only packages from testing. In 
tabular form:


   passfail
sbcl   from testing2:2.2.6-2
cl-ironcladfrom testing0.57-1
all others from testingfrom testing

I copied some of the output at the bottom of this report.

Currently this regression is blocking the migration of sbcl to testing 
[1]. Due to the nature of this issue, I filed this bug report against 
both packages. Can you please investigate the situation and reassign the 
bug to the right package?


More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation

Paul

[1] https://qa.debian.org/excuses.php?package=sbcl

https://ci.debian.net/data/autopkgtest/testing/armhf/c/cl-ironclad/24381695/log.gz

; wrote 
/tmp/autopkgtest-lxc.d2c27h4h/downtmp/autopkgtest_tmp/usr/share/common-lisp/source/ironclad/src/ciphers/tea-tmp93OFNPHA.fasl

; compilation finished in 0:00:00.160
; compiling file 
"/usr/share/common-lisp/source/ironclad/src/ciphers/threefish.lisp" 
(written 18 FEB 2022 01:39:10 PM):

Heap exhausted during garbage collection: 16 bytes available, 48 requested.
Immobile Object Counts
 Gen   GF type  fdefn symbol   code  Boxed   ConsRaw   Code  SmMix 
 Mixed  LgRaw LgCode  LgMix Waste%   AllocTrig   Dirty GCs 
Mem-age
  1 00  0  0  0   7121939   9668  5273 
   827  0  0 17   19.96186748032499253   18850   1 
 1.1503
  2 00  0  0  0  19865   2766  12586 84836 
  1681 10 34127   11.7   137430448 5368709   23089   0 
 0.9496
  3 00  0  0  0  45116   7190   7447604   3552 
  16732394673311.0   270091288 2007806   0 
 0.4559
  4 00  0  0  0  0  0  0  0  0 
 0  0  0  00.0   0 200   0   0 
 0.
  5 00  0  0  0  0  0  0  0  0 
 0  0  0  00.0   0 200   0   0 
 0.
  6 00  0  0  0   1370471   1242   3507314 
402551342811.930584464 200 251   0 
 0.
Tot 00  0  0  0  73472  11366  30943   4200   4975 
 42215046357566.9   499973680 [93.1% of 536870912 max]

GC control variables:
   *GC-INHIBIT* = true
   *GC-PENDING* = true
fatal error encountered in SBCL pid 1357:
Heap exhausted, game over.

   0: 0xf746478c pc=0x98 {0x4f0380b8+b0fc7fe0} {code_serialno=3901}
   1: 0xf7464730 pc=0x4f005760 {0x4f005000+0760} (FLET 
"WITHOUT-GCING-BODY-5" :IN SB-KERNEL::SUB-GC)
   2: 0xf74646d4 pc=0x4f0053f8 {0x4f005000+03f8} (FLET 
"WITHOUT-INTERRUPTS-BODY-1" :IN SB-KERNEL::SUB-GC)

   3: 0xf7464678 pc=0x4f005108 {0x4f005000+0108} SB-KERNEL::SUB-GC
   4: 0xf746466c pc=(nil) LRA=0x3e2e75: [I]0xf7464654 
pc=0x3e4d8 {0x4fb3f240+b04ff298} {code_serialno=f5f}

   6: 0xf7464634 pc=0x5028c858 {0x5028c4b0+03a8} SB-C::MAKE-WIRED-TN
   7: 0xf7464604 pc=0x4fd9a348 {0x4fd9a030+0318} 
SB-C::IR2-CONVERT-FULL-CALL-ARGS
   8: 0xf74645c4 pc=0x4f8506f8 {0x4f8505c8+0130} 
SB-C::IR2-CONVERT-FIXED-FULL-CALL
   9: 0xf74645b4 pc=0x4f3d8b18 {0x4f3d8990+0188} 
SB-C::IR2-CONVERT-FULL-CALL
  10: 0xf7464598 pc=0x4f6fe720 {0x4f6fda90+0c90} 
SB-C::IR2-CONVERT-BLOCK

  11: 0xf7464584 pc=0x4f2c8f58 {0x4f2c8dc8+0190} SB-C::IR2-CONVERT
  12: 0xf7464550 pc=0x504637a8 {0x50463198+0610} 
SB-C::%COMPILE-COMPONENT
  13: 0xf7464538 pc=0x5041c820 {0x5041c258+05c8} 
SB-C::COMPILE-COMPONENT

  14: 0xf7464514 pc=0x5037ca10 {0x5037c7b8+0258} SB-C::COMPILE-TOPLEVEL
  15: 0xf7464500 pc=0x503f0eb8 {0x503f0c28+0290} 
SB-C::CONVERT-AND-MAYBE-COMPILE
  16: 0xf74644b4 pc=0x50393270 {0x50392a10+0860} 
SB-C::PROCESS-TOPLEVEL-FORM
  17: 0xf746449c pc=0x50457490 {0x504573a0+00f0} 
SB-C::PROCESS-TOPLEVEL-PROGN
  18: 0xf7464450 pc=0x50393188 {0x50392a10+0778} 
SB-C::PROCESS-TOPLEVEL-FORM
  19: 0xf7464404 pc=0x50393408 {0x50392a10+09f8} 
SB-C::PROCESS-TOPLEVEL-FORM
  20: 0xf74643b8 pc=0x50393408 {0x50392a10+09f8} 
SB-C::PROCESS-TOPLEVEL-FORM
  21: 0xf7464334 pc=0x502f60e0 {0x502f3b20+25c0} (LAMBDA 
(SB-KERNEL::FORM  :CURRENT-INDEX ) :IN 
SB-C::SUB-COMPILE-FILE)
  22: 0xf74642c0 pc=0x4f0c5038 {0x4f0c49a8+0690} 
SB-C::%DO-FORMS-FROM-INFO
  23: 0xf7464244 pc=0x502f3ea0 {0x502f3b20+0380} (FLET "LAMBDA0" 
:IN "SYS:SRC;COMPILER;MAIN.LISP")
  24: