bug#54557: “fakechroot” execution engine doesn’t work for Bash

2022-03-28 Thread Ludovic Courtès 
Hi,

Maxim Cournoyer  skribis:

> I'd be OK with --without-bash-malloc; it seems we'll pay a bit in terms
> of Bash performance in exchange for better memory usage.  It also brings
> benefits such as solving this issue and may benefit from
> advances/bugfixes to glibc's malloc in the future, if there are any.

We might actually benefit from the improvements made in glibc’s malloc
over the years (it’s more actively developed than that of Bash), who
knows.

Anyway, pushed to ‘core-updates’ as
c6b5161e97ed1010d61331874b09c3231af3b1f9.

Thanks,
Ludo’.

bug#54557: “fakechroot” execution engine doesn’t work for Bash

2022-03-28 Thread Ludovic Courtès 
Hi,

Philippe SWARTVAGHER  skribis:

> FTR, the --without-bash-malloc is used in the Debian bash package:
>
> apt source bash
>
> cd bash-5.1/debian
>
> grep -Irn without-bash-malloc
> changelog:145:  * Configure the normal build --without-bash-malloc as well.
> changelog:1125:  * Configure the static build --without-bash-malloc.
> changelog:1462:  * Disable the GNU/kFreeBSD kludge
> (--without-bash-malloc). Closes: #234137.
> changelog:1546:  * Configure --without-bash-malloc on GNU/FreeBSD
> (closes: #194182).
> changelog:1739:  * Configure --without-bash-malloc. At least on hppa,
> this fixes an error,
> rules:79:    --without-bash-malloc
>
>
> This option is also advised in Linux From Scratch:
> https://www.linuxfromscratch.org/lfs/view/stable/chapter08/bash.html

Good to know, thanks for sharing.

I just realized that ‘bash-minimal’ in Guix already uses it, so that’s
another way to work around the ‘guix pack -RR’ issue at hand.

Ludo’.

bug#54557: “fakechroot” execution engine doesn’t work for Bash

2022-03-25 Thread Maxim Cournoyer 
Hi Ludovic,

Ludovic Courtès  writes:

[...]

> The message comes from ld.so.  My guess is that the problem comes from
> Bash’s terrible ‘malloc’ replacement:
>

[...]

> [Time passes…]  I confirmed this hypothesis by running:
>
>   guix pack -RR -S /bin=bin -m manifest.scm
>
> on this manifest:
>
> (use-modules (guix) (guix utils)
>  (guix profiles)
>  (gnu packages bash))
>
> (define bash-sans-malloc
>   (package/inherit bash
> (name "bash-sans-malloc")
> (arguments
>  (substitute-keyword-arguments (package-arguments bash)
>((#:configure-flags flags ''())
> `(cons "--without-bash-malloc" ,flags))
>
> (packages->manifest (list bash-sans-malloc))
>
>
> Works just fine:
>
> $ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; 
> GUIX_EXECUTION_ENGINE=fakechroot /tmp/pack/bin/bash --version'
> GNU bash, version 5.1.8(1)-release (x86_64-unknown-linux-gnu)
> Copyright (C) 2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 

[...]

> Perhaps we should build Bash ‘--without-bash-malloc’ unconditionally in
> ‘core-updates’?  The ‘INSTALL’ file reads:
>
> '--with-bash-malloc'
>  Use the Bash version of 'malloc' in the directory 'lib/malloc'.
>  This is not the same 'malloc' that appears in GNU libc, but an
>  older version originally derived from the 4.2 BSD 'malloc'.  This
>  'malloc' is very fast, but wastes some space on each allocation.
>  This option is enabled by default.  The 'NOTES' file contains a
>  list of systems for which this should be turned off, and
>  'configure' disables this option automatically for a number of
>  systems.
>
> There might be other options if we want to keep it, such as changing the
> ELF visibility of those symbols, but I wonder if it’s worth the effort.
>
> Thoughts?

I'd be OK with --without-bash-malloc; it seems we'll pay a bit in terms
of Bash performance in exchange for better memory usage.  It also brings
benefits such as solving this issue and may benefit from
advances/bugfixes to glibc's malloc in the future, if there are any.

Well done investigating!

Maxim

bug#54557: “fakechroot” execution engine doesn’t work for Bash

2022-03-25 Thread Philippe SWARTVAGHER 



Le 24/03/2022 à 22:02, Ludovic Courtès a écrit :

Hello!

The “fakechroot” engine fails when applied to ‘bash’:

--8<---cut here---start->8---
$ guix pack -RR bash -S /bin=bin
/gnu/store/mnbmklbvd1pk7yby0k8h26msk6z11m1m-bash-tarball-pack.tar.gz
$ (cd /tmp/pack; tar xf 
/gnu/store/mnbmklbvd1pk7yby0k8h26msk6z11m1m-bash-tarball-pack.tar.gz)
$ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; 
GUIX_EXECUTION_ENGINE=fakechroot /tmp/pack/bin/bash --version'
/tmp/pack/gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin//bash: out 
of memory to store relocation results for /tmp/pack/bin/bash
--8<---cut here---end--->8---

The message comes from ld.so.  My guess is that the problem comes from
Bash’s terrible ‘malloc’ replacement:

--8<---cut here---start->8---
$ objdump -T /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/bash | 
grep malloc
004ae6f0 gDF .text  003b  Basemalloc_usable_size
004ae850 gDF .text  0009  Basemalloc
00484e70 gDF .text  005b  Basexmalloc
004ee020 gDO .bss   0004  Base
malloc_trace_at_exit
004e8c24 gDO .data  0004  Base
malloc_mmap_threshold
00484f70 gDF .text  00dd  Basesh_xmalloc
004f7a04 gDO .bss   0004  Basemalloc_trace
004f7a08 gDO .bss   0004  Basemalloc_flags
004ae730 gDF .text  0005  Basesh_malloc
004f7a00 gDO .bss   0004  Basemalloc_register
004ae660 gDF .text  000c  Base
_malloc_unblock_signals
004ae630 gDF .text  002e  Base
_malloc_block_signals
--8<---cut here---end--->8---

[Time passes…]  I confirmed this hypothesis by running:

   guix pack -RR -S /bin=bin -m manifest.scm

on this manifest:

--8<---cut here---start->8---
(use-modules (guix) (guix utils)
  (guix profiles)
  (gnu packages bash))

(define bash-sans-malloc
   (package/inherit bash
 (name "bash-sans-malloc")
 (arguments
  (substitute-keyword-arguments (package-arguments bash)
((#:configure-flags flags ''())
 `(cons "--without-bash-malloc" ,flags))

(packages->manifest (list bash-sans-malloc))
--8<---cut here---end--->8---

Works just fine:

--8<---cut here---start->8---
$ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; 
GUIX_EXECUTION_ENGINE=fakechroot /tmp/pack/bin/bash --version'
GNU bash, version 5.1.8(1)-release (x86_64-unknown-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
--8<---cut here---end--->8---

So in the end, it’s a bug report that’s kinda closed already.

Perhaps we should build Bash ‘--without-bash-malloc’ unconditionally in
‘core-updates’?  The ‘INSTALL’ file reads:

--8<---cut here---start->8---
'--with-bash-malloc'
  Use the Bash version of 'malloc' in the directory 'lib/malloc'.
  This is not the same 'malloc' that appears in GNU libc, but an
  older version originally derived from the 4.2 BSD 'malloc'.  This
  'malloc' is very fast, but wastes some space on each allocation.
  This option is enabled by default.  The 'NOTES' file contains a
  list of systems for which this should be turned off, and
  'configure' disables this option automatically for a number of
  systems.
--8<---cut here---end--->8---

There might be other options if we want to keep it, such as changing the
ELF visibility of those symbols, but I wonder if it’s worth the effort.

Thoughts?

Ludo’.


Hello,

FTR, the --without-bash-malloc is used in the Debian bash package:

apt source bash

cd bash-5.1/debian

grep -Irn without-bash-malloc
changelog:145:  * Configure the normal build --without-bash-malloc as well.
changelog:1125:  * Configure the static build --without-bash-malloc.
changelog:1462:  * Disable the GNU/kFreeBSD kludge 
(--without-bash-malloc). Closes: #234137.
changelog:1546:  * Configure --without-bash-malloc on GNU/FreeBSD 
(closes: #194182).
changelog:1739:  * Configure --without-bash-malloc. At least on hppa, 
this fixes an error,

rules:79:    --without-bash-malloc


This option is also advised in Linux From Scratch: 
https://www.linuxfromscratch.org/lfs/view/stable/chapter08/bash.html


--
Philippe

bug#54557: “fakechroot” execution engine doesn’t work for Bash

2022-03-24 Thread Ludovic Courtès 
Hello!

The “fakechroot” engine fails when applied to ‘bash’:

--8<---cut here---start->8---
$ guix pack -RR bash -S /bin=bin 
/gnu/store/mnbmklbvd1pk7yby0k8h26msk6z11m1m-bash-tarball-pack.tar.gz
$ (cd /tmp/pack; tar xf 
/gnu/store/mnbmklbvd1pk7yby0k8h26msk6z11m1m-bash-tarball-pack.tar.gz)
$ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; 
GUIX_EXECUTION_ENGINE=fakechroot /tmp/pack/bin/bash --version'
/tmp/pack/gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin//bash: out 
of memory to store relocation results for /tmp/pack/bin/bash
--8<---cut here---end--->8---

The message comes from ld.so.  My guess is that the problem comes from
Bash’s terrible ‘malloc’ replacement:

--8<---cut here---start->8---
$ objdump -T /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/bash | 
grep malloc
004ae6f0 gDF .text  003b  Basemalloc_usable_size
004ae850 gDF .text  0009  Basemalloc
00484e70 gDF .text  005b  Basexmalloc
004ee020 gDO .bss   0004  Base
malloc_trace_at_exit
004e8c24 gDO .data  0004  Base
malloc_mmap_threshold
00484f70 gDF .text  00dd  Basesh_xmalloc
004f7a04 gDO .bss   0004  Basemalloc_trace
004f7a08 gDO .bss   0004  Basemalloc_flags
004ae730 gDF .text  0005  Basesh_malloc
004f7a00 gDO .bss   0004  Basemalloc_register
004ae660 gDF .text  000c  Base
_malloc_unblock_signals
004ae630 gDF .text  002e  Base
_malloc_block_signals
--8<---cut here---end--->8---

[Time passes…]  I confirmed this hypothesis by running:

  guix pack -RR -S /bin=bin -m manifest.scm

on this manifest:

--8<---cut here---start->8---
(use-modules (guix) (guix utils)
 (guix profiles)
 (gnu packages bash))

(define bash-sans-malloc
  (package/inherit bash
(name "bash-sans-malloc")
(arguments
 (substitute-keyword-arguments (package-arguments bash)
   ((#:configure-flags flags ''())
`(cons "--without-bash-malloc" ,flags))

(packages->manifest (list bash-sans-malloc))
--8<---cut here---end--->8---

Works just fine:

--8<---cut here---start->8---
$ unshare -mrf sh -c 'mount -t tmpfs -o ro none /gnu/store; 
GUIX_EXECUTION_ENGINE=fakechroot /tmp/pack/bin/bash --version'
GNU bash, version 5.1.8(1)-release (x86_64-unknown-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
--8<---cut here---end--->8---

So in the end, it’s a bug report that’s kinda closed already.

Perhaps we should build Bash ‘--without-bash-malloc’ unconditionally in
‘core-updates’?  The ‘INSTALL’ file reads:

--8<---cut here---start->8---
'--with-bash-malloc'
 Use the Bash version of 'malloc' in the directory 'lib/malloc'.
 This is not the same 'malloc' that appears in GNU libc, but an
 older version originally derived from the 4.2 BSD 'malloc'.  This
 'malloc' is very fast, but wastes some space on each allocation.
 This option is enabled by default.  The 'NOTES' file contains a
 list of systems for which this should be turned off, and
 'configure' disables this option automatically for a number of
 systems.
--8<---cut here---end--->8---

There might be other options if we want to keep it, such as changing the
ELF visibility of those symbols, but I wonder if it’s worth the effort.

Thoughts?

Ludo’.