Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Zhu Zihao]

We can make GNU build system tries autoreconf or autogen.sh first if we
encounter some build issues. But I don't think it's a good idea to
remove configure script in the snippet of origin. Because you can't
track all files that generated by Autotools. It is also burden for
packager to pick the build-time dependencies like libtool,
autoconf-archive, gnulib etc. if we force all autotools baseed packages
use autoreconf

For software developer, they prefer to bundle a generated configure
script because it will be a bootstrap problem sometimes.

A simple examples,the m4 macro GUILE_PKG defined in guile.m4 helps
autotools to check guile installation. If there's no generated configure
script. Software user should have guile.m4 to generate configure script.
A macro that detects guile installation requires a guile installation to
work. Catch 22 :)


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao


signature.asc
Description: PGP signature

Re: The Shepherd on Fibers

[adriano]

Il giorno mar, 29/03/2022 alle 15.29 +, Attila Lendvai ha scritto:
> > > Which socket file?
> > > 
> > > (In practice ‘exec-command’ would only fail if the program cannot
> > > be
> > > found.)
> > 
> > Attila Lendvai has encountered the socket file to be deleted
> > before,
> > this seemed to have been the cause and I have seen a few bug
> > reports
> > (about tor?) that might have had the same cause. I don't have a bug
> > number at hand.
> 
> my first ever service has non-trivial work to do in the start GEXP,
> and unsurprisingly, it has often ended up throwing an exception while
> i subled along my learning curve. it's not caught by shepherd right
> around the call to the start GEXP, and reaches a handler up higher
> that ends up deleting the socket (and IIRC skipping the start of rest
> of the services).
> 
> it's on my TODO to harden the error handling in shepherd once the
> shepherd-for-guix patch is in, or something equivalent that shortens
> the edit-compile-test cycle of shepherd.

What's the shepherd-for-guix patch ? What does it bring to Shepherd ?

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Mittwoch, dem 30.03.2022 um 21:24 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op wo 30-03-2022 om 20:55 [+0200]:
> > Note that many autotools-based packages already require the
> > addition of autoconf and friends due to being pulled from git. 
> > That being said, it's somewhat hard to argue for completely
> > dropping them, because
> > a. simply matching files via ".in" suffix would be error-prone
> > b. autoreconf should regenerate these files regardless
> > Therefore, my counter-proposal would be to just simply always run
> > the bootstrap script or autoreconf, even if the respective files
> > are tarballed, as well as adding autoconf and automake to the
> > implicit native inputs of gnu build system.
> 
> It should be possible to look for ‘# Generated by GNU Autoconf’ and
> ‘# Makefile.in generated by automake’ lines in some 'find-generated-
> autotools-fies’or something.
I don't think that will work correctly.  Case in point: pre-inst-env in
the Guix source tree.

> Adding autoconf & automake seems a bit much to me (gnu-build-system
> is not necessarily autotools-build-system, the exact required version
> of autoconf & automake can vary), but otherwise your proposal seems
> good to me.
We could drop both from the deriving system or simply define a constant
%implicit-gnu-inputs-no-autotools (name subject to bikeshedding).

Re: gcc-4.7 and gcc-4.8 compilation failure in x64

[Ekaitz Zarraga]

Good idea, I will!


--- Original Message ---

On Wednesday, March 30th, 2022 at 9:40 PM, Reza Housseini 
 wrote:

> Did you try to use guix time-machine and compile 4.7 from there? Perhaps
>
> the issue is the glibc version?
>
> On 3/30/22 21:32, Ekaitz Zarraga wrote:
>
> > Hi Reza,
> >
> > On Wednesday, March 30th, 2022 at 9:27 PM, Reza Housseini 
> > reza.housse...@gmail.com wrote:
> >
> > > Looks like a bug in gcc: 
> > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81712
> > >
> > > You probably have to patch gcc 4.* as it was only backported until gcc 5.5
> > >
> > > On 3/29/22 21:01, Ekaitz Zarraga wrote:
> > >
> > > > > And you try to compile that using `guix build -m manifest.scm` the 
> > > > > compilation fails. The same happens when choosing 4.8.
> > > > >
> > > > > More specifically I mean the output of that command is:
> > > >
> > > > ./md-unwind-support.h:60:47: error: dereferencing pointer to incomplete 
> > > > type
> > > >
> > > > ...
> > > >
> > > > error: in phase 'build': uncaught exception:
> > > >
> > > > %exception #< program: "make" arguments: ("-j" "8" 
> > > > "LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> > > >  -Wl,-dynamic-linker 
> > > > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> > > >  
> > > > "LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> > > >  -Wl,-dynamic-linker 
> > > > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> > > >  "BOOT_CFLAGS=-O2 -g0") exit-status: 2 term-signal: #f stop-signal: #f>
> > > >
> > > > phase `build' failed after 256.1 seconds command "make" "-j" "8" 
> > > > "LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> > > >  -Wl,-dynamic-linker 
> > > > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> > > >  
> > > > "LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> > > >  -Wl,-dynamic-linker 
> > > > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> > > >  "BOOT_CFLAGS=-O2 -g0" failed with status 2 builder for` 
> > > > /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed with 
> > > > exit code 1
> > > >
> > > > build of /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv 
> > > > failed
> > > >
> > > > View build log at 
> > > > '/var/log/guix/drvs/bx/2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv.gz'.
> > > >
> > > > guix build: error: build of 
> > > > `/gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed
> >
> > So the gcc-4.7 and gcc-4.8 packages never worked? Or we need an older libc?

Re: gcc-4.7 and gcc-4.8 compilation failure in x64

[Reza Housseini]

Did you try to use guix time-machine and compile 4.7 from there? Perhaps 
the issue is the glibc version?


On 3/30/22 21:32, Ekaitz Zarraga wrote:

Hi Reza,



On Wednesday, March 30th, 2022 at 9:27 PM, Reza Housseini 
 wrote:


Looks like a bug in gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81712

You probably have to patch gcc 4.* as it was only backported until gcc 5.5

On 3/29/22 21:01, Ekaitz Zarraga wrote:


And you try to compile that using `guix build -m manifest.scm` the compilation 
fails. The same happens when choosing 4.8.

More specifically I mean the output of that command is:

./md-unwind-support.h:60:47: error: dereferencing pointer to incomplete type

...

error: in phase 'build': uncaught exception:

%exception #< program: "make" arguments: ("-j" "8" 
"LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" 
"LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" "BOOT_CFLAGS=-O2 -g0") exit-status: 2 
term-signal: #f stop-signal: #f>

phase `build' failed after 256.1 seconds command "make" "-j" "8" 
"LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" 
"LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" "BOOT_CFLAGS=-O2 -g0" failed with 
status 2 builder for` /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed with exit code 1

build of /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv failed

View build log at 
'/var/log/guix/drvs/bx/2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv.gz'.

guix build: error: build of 
`/gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed


So the gcc-4.7 and gcc-4.8 packages never worked? Or we need an older libc?



OpenPGP_0xC375C6AF05125C52.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: gcc-4.7 and gcc-4.8 compilation failure in x64

[Ekaitz Zarraga]

Hi Reza,



On Wednesday, March 30th, 2022 at 9:27 PM, Reza Housseini 
 wrote:

> Looks like a bug in gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81712
>
> You probably have to patch gcc 4.* as it was only backported until gcc 5.5
>
> On 3/29/22 21:01, Ekaitz Zarraga wrote:
>
> > > And you try to compile that using `guix build -m manifest.scm` the 
> > > compilation fails. The same happens when choosing 4.8.
> >
> > More specifically I mean the output of that command is:
> >
> > ./md-unwind-support.h:60:47: error: dereferencing pointer to incomplete type
> >
> > ...
> >
> > error: in phase 'build': uncaught exception:
> >
> > %exception #< program: "make" arguments: ("-j" "8" 
> > "LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> >  -Wl,-dynamic-linker 
> > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> >  
> > "LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> >  -Wl,-dynamic-linker 
> > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> >  "BOOT_CFLAGS=-O2 -g0") exit-status: 2 term-signal: #f stop-signal: #f>
> >
> > phase `build' failed after 256.1 seconds command "make" "-j" "8" 
> > "LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> >  -Wl,-dynamic-linker 
> > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> >  
> > "LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib
> >  -Wl,-dynamic-linker 
> > -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2"
> >  "BOOT_CFLAGS=-O2 -g0" failed with status 2 builder for` 
> > /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed with exit 
> > code 1
> >
> > build of /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv failed
> >
> > View build log at 
> > '/var/log/guix/drvs/bx/2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv.gz'.
> >
> > guix build: error: build of 
> > `/gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed


So the gcc-4.7 and gcc-4.8 packages never worked? Or we need an older libc?

Re: gcc-4.7 and gcc-4.8 compilation failure in x64

[Reza Housseini]

Looks like a bug in gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81712

You probably have to patch gcc 4.* as it was only backported until gcc 5.5

On 3/29/22 21:01, Ekaitz Zarraga wrote:

And you try to compile that using `guix build -m manifest.scm` the compilation 
fails. The same happens when choosing 4.8.


More specifically I mean the output of that command is:

./md-unwind-support.h:60:47: error: dereferencing pointer to incomplete type
...
error: in phase 'build': uncaught exception:
%exception #< program: "make" arguments: ("-j" "8" 
"LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" 
"LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" "BOOT_CFLAGS=-O2 -g0") exit-status: 2 
term-signal: #f stop-signal: #f>
phase `build' failed after 256.1 seconds
command "make" "-j" "8" "LDFLAGS_FOR_TARGET=-B/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib 
-Wl,-dynamic-linker -Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" 
"LDFLAGS=-Wl,-rpath=/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib -Wl,-dynamic-linker 
-Wl,/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/ld-linux-x86-64.so.2" "BOOT_CFLAGS=-O2 -g0" failed with 
status 2
builder for `/gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed 
with exit code 1
build of /gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv failed
View build log at 
'/var/log/guix/drvs/bx/2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv.gz'.
guix build: error: build of 
`/gnu/store/bx2zldsx3by529hri5pfx15k12f67gzb-gcc-4.7.4.drv' failed


OpenPGP_0xC375C6AF05125C52.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

Liliana Marie Prikler schreef op wo 30-03-2022 om 20:55 [+0200]:
> Note that many autotools-based packages already require the addition of
> autoconf and friends due to being pulled from git.  That being said,
> it's somewhat hard to argue for completely dropping them, because
> a. simply matching files via ".in" suffix would be error-prone
> b. autoreconf should regenerate these files regardless
> Therefore, my counter-proposal would be to just simply always run the
> bootstrap script or autoreconf, even if the respective files are
> tarballed, as well as adding autoconf and automake to the implicit
> native inputs of gnu build system.

It should be possible to look for ‘# Generated by GNU Autoconf’ and ‘#
Makefile.in generated by automake’ lines in some 'find-generated-
autotools-fies’or something.

Adding autoconf & automake seems a bit much to me (gnu-build-system is
not necessarily autotools-build-system, the exact required version of
autoconf & automake can vary), but otherwise your proposal seems good
to me.

> > For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
> > circularity problem
> The obvious solution to which would be to implement m4 in mes :)
> 
> > [B]uilding 'configure' and 'Makefile.in' from source might not always
> > be possible, but WDYT of building 'configure' & 'Makefile.in' from
> > source for packages where it does not result in bootstrapping
> > problems?
> See above, but to reiterate, I'm generally in favor.
> 
> Regarding tooling support, I think autotools should have an option to
> build a non-bootstrapped dist tarball.  If more upstreams produced such
> stripped tarballs, we wouldn't even be having that debate.

Yes, would be nice if upstreams could choose to opt-out.  Or maybe opt-
in instead.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Liliana Marie Prikler]

Am Mittwoch, dem 30.03.2022 um 14:04 +0200 schrieb Maxime Devos:
> Hi guix,
> 
> Quite some packages in Guix use the Autotools system.  In this
> system, a 'configure.ac' and 'Makefile.am' script / makefile is
> written, from which 'autoconf' & 'automake' generate a very long bash
> script and a Makefile.in.  Depending on the maintainer of the
> upstream package, this 'configure' and 'Makefile.in' are sometimes
> included in release tarballs.
> 
> This seems in conflict with:
> 
>   * (guix)Submitting patches: ‘Make sure the package does not use
>     bundled copies of software already available as separate
>     packages.’
> 
>     Autotools packages have a 'config.guess' and 'config.sub' script
>     that need to be updated whenever there's a new architecture.  As
>     such, for some packages, these need to be replaced for aarch64,
>     powerpc or risv64.  There are also some packages with very old
>     configure scripts that don't support --build/--host/--target,
>     which could gain --build/--host/--target support by just
>     regenerating them.
> 
>     This also makes ensuring a package does not contain any malware
>     much harder, because the configure script (and related files)
>     needs to be read in their entirity.
> 
>  * When an upstream tarball contains .so, .dll, .a, etc. binaries,
>    they are removed downstream in a snippet.  Why would the Autotools
>    be an exception?
Note that many autotools-based packages already require the addition of
autoconf and friends due to being pulled from git.  That being said,
it's somewhat hard to argue for completely dropping them, because
a. simply matching files via ".in" suffix would be error-prone
b. autoreconf should regenerate these files regardless
Therefore, my counter-proposal would be to just simply always run the
bootstrap script or autoreconf, even if the respective files are
tarballed, as well as adding autoconf and automake to the implicit
native inputs of gnu build system.

> For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
> circularity problem
The obvious solution to which would be to implement m4 in mes :)

> [B]uilding 'configure' and 'Makefile.in' from source might not always
> be possible, but WDYT of building 'configure' & 'Makefile.in' from
> source for packages where it does not result in bootstrapping
> problems?
See above, but to reiterate, I'm generally in favor.

Regarding tooling support, I think autotools should have an option to
build a non-bootstrapped dist tarball.  If more upstreams produced such
stripped tarballs, we wouldn't even be having that debate.

Cheers

Re: Deprecating legacy build phase style when cross-compiling vs. native

[Maxime Devos]

Josselin Poiret schreef op wo 30-03-2022 om 17:17 [+0200]:
> This means that packages that were not updated to fit the new style
> should all fail to cross-compile.  This simple bug could be resolved by
> adding %output to gnu-cross-build, however as was argued on IRC this is
> now undocumented behaviour and we'd rather switch all packages to the
> new style instead.  While I 100% agree with this, I think we should have
> a uniform deprecation policy for this matter, and that there shouldn't
> be such a disparity between cross and native builds.
> 
> What do you all think?

Not sure how this deprecation would look like (a NEWS entry, a blog
post, a (lowercase) news entry, a linter detecting %output, ...), but I
agree with (eventually) removing %output / %outputs whether cross-
compilation or not and for all build systems.  More generally, I agree
that disparity between cross and native builds would ideally be
minimised (I consider native builds to be a special case of cross
builds where one can run the compiled binaries and run tests).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Deprecating legacy build phase style when cross-compiling vs. native

[Maxime Devos]

Josselin Poiret schreef op wo 30-03-2022 om 17:17 [+0200]:
> Following a report on IRC [1] that nss-certs wouldn't cross-build, I
> found out that the old syntax of using %output instead of the shinier
> $output was still supported only when native-compiling, and not when
> cross-compiling, at least for build-systems inheriting from gnu.

This is a bit of a distraction from the main point of this e-mail, but
nss-certs is architecture-independent, so you could add '#:target
#false' to 'arguments'.  Then the native compilation code is run
instead of the cross-compilation code, and the same derivation as for
native compilation is reused, saving a little build time (and disk
space, in case nars are generated or deduplication is disabled).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: The Shepherd on Fibers

[Maxime Devos]

Ludovic Courtès schreef op wo 30-03-2022 om 17:14 [+0200]:
> >  (format #t "Accepted connection on ~0@*~a from ~1@*~a." "foo"
> > "bar"))
> 
> Hmm that doesn’t seem to work:
> 
> --8<---cut here---start->8---
> scheme@(guile-user)> ,use(ice-9 format)
> scheme@(guile-user)> (format #f "Accepted connection on ~@0*~a from
> ~@1*~a." 1 2)

You are writing ~@0*~a. However, the number must be before the @:
~0@*~a. If the number is moved between the ~ and @, then it works for
me.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

The GNU Shepherd 0.9.0rc1 available for testing!

[Ludovic Courtès]

Hello Guix!

A release candidate of the Shepherd 0.9.0 is available for testing!

  https://alpha.gnu.org/gnu/shepherd/shepherd-0.9.0rc1.tar.gz
  https://alpha.gnu.org/gnu/shepherd/shepherd-0.9.0rc1.tar.gz.sig

  SHA256: 6b7cdbb8d2509fca0c4d08b855031ea72c887a65828ae6493c2e5a25130c3c37

You can test it on Guix System with:

  guix time-machine --branch=wip-shepherd-upgrade -- \
system reconfigure …

and on Guix Home:

  guix time-machine --branch=wip-shepherd-upgrade -- \
home reconfigure …

Bug reports are most welcome.

The branch changes the ‘ssh-daemon’ service to an inetd-style service:

  https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-shepherd-upgrade

You’re welcome to propose patches to change other services to systemd or
inetd style.

The ‘make-forkexec-constructor/container’ procedure that we have doesn’t
compose well (we’d have to make
‘make-inetd-forkexec-constructor/container’, etc.).  But I think we
could achieve the same thing differently, with wrappers, so that we can
have inetd and systemd services running in containers too.  Future work…

There’s also more work we can do in future Shepherd versions, such as
adding ‘herd journal SERVICE’, moving state out of , adding a
REPL :-), etc., but what we have today is a reasonable milestone IMO.

Besides, a new POT file should be available soon if you’d like to help
translate the new release in your language:

  https://translationproject.org/domain/shepherd.html

Ludo’.


signature.asc
Description: PGP signature

Re: Specify substitute url in GUI installer

[ZeroBeyond]

Hi,

 I don't think substitute server authorization is a problem. In 
most cases, the purpose of using a substitute server in the installation 
process is mainly for speeding up package downloads. The content of the 
substitute server mirror remains the same as the official one. As I 
know, in that case, the authorization is not necessary. So that adding a 
substitute server option in the graphical installer is reasonable. It 
would improve the convenience radically, especially for people in some 
countries or regions where a fast internet connection to the official 
server is not available.


ZeroBeyond

Re: Specify substitute url in GUI installer

[ZeroBeyond]

Hi,

  I don't think substitute server authorization is a problem. In 
most cases, the purpose of using a substitute server in the installation 
process is mainly for speeding up package downloads. The content of the 
substitute server mirror remains the same as the official one. As I 
know, in that case, the authorization is not necessary. So that adding a 
substitute server option in the graphical installer is reasonable. It 
would improve the convenience radically, especially for people in some 
countries or regions where a fast internet connection to the official 
server is not available.

ZeroBeyond

Re: Specify substitute url in GUI installer

[ZeroBeyond]

Hi,

 I don't think substitute server authorization is a problem. In 
most cases, the purpose of using a substitute server in the installation 
process is mainly for speeding up package downloads. The content of the 
substitute server mirror remains the same as the official one. As I 
know, in that case, the authorization is not necessary. So that adding a 
substitute server option in the graphical installer is reasonable. It 
would improve the convenience radically, especially for people in some 
countries or regions where a fast internet connection to the official 
server is not available.


ZeroBeyond

Re: Specify substitute url in GUI installer

[ZeroBeyond]

Hi,

 I don't think substitute server authorization is a problem. In
most cases, the purpose of using a substitute server in the installation
process is mainly for speeding up package downloads. The content of the
substitute server mirror remains the same as the official one. As I
know, in that case, the authorization is not necessary. So that adding a
substitute server option in the graphical installer is reasonable. It
would improve the convenience radically, especially for people in some
countries or regions where a fast internet connection to the official
server is not available.

ZeroBeyond

Re: Specify substitute url in GUI installer

[hahawang]

Hi,

 I don't think substitute server authorization is a problem. In 
most cases, the purpose of using a substitute server in the installation 
process is mainly for speeding up package downloads. The content of the 
substitute server mirror remains the same as the official one. As I 
know, in that case, the authorization is not necessary. So that adding a 
substitute server option in the graphical installer is reasonable. It 
would improve the convenience radically, especially for people in some 
countries or regions where a fast internet connection to the official 
server is not available.


ZeroBeyond

Re: Profiling of man-db database generation with zlib vs zstd

[Jonathan McHugh]

Hi Maxim,

Out of interest, will a zstd dictionary be (eventually) utilised as a strategy 
for further reducing compression and decompression speeds?

```
The compression library Zstandard (also known as "Zstd") has the ability to 
create an external "dictionary" from a set of training files which can be used 
to more efficiently (in terms of compression and decompression speed and also 
in terms of compression ratio) compress files of the same type as the training 
files. For example, if a dictionary is "trained" on an example set of email 
messages, anyone with access to the dictionary will be able to more efficiently 
compress another email file. The trick is that the commonalities are kept in 
the dictionary file, and, therefore, anyone wishing to decompress the email 
must have already had that same dictionary sent to them.[2] 
```
http://fileformats.archiveteam.org/wiki/Zstandard_dictionary

I appreciate it may confuse your piecemeal benchmarking (certainly at this 
stage) but I would assume that creating a dictionary (or dictionaries, say 
covering each Guix package category for linguistic overlaps) for manpages would 
further improve zstd speeds.

HTH,


Jonathan McHugh
indieterminacy@libre.brussels

March 30, 2022 4:49 PM, "Maxim Cournoyer"  wrote:

> Hi Ludovic,
> 
> Ludovic Courtès  writes:
> 
> [...]
> 
>> To isolate the problem, you could allocate the 4 MiB buffer outside of
>> the loop and use ‘get-bytevector-n!’, and also remove code that writes
>> to ‘output’.
> 
> I've adjusted the benchmark like so:
> 
> --8<---cut here---start->8---
> (use-modules (ice-9 binary-ports)
> (ice-9 match)
> (rnrs bytevectors)
> (zstd))
> 
> (define MiB (expt 2 20))
> (define block-size (* 4 MiB))
> (define bv (make-bytevector block-size))
> (define input-file "/tmp/chromium-98.0.4758.102.tar.zst")
> 
> (define (run)
> (call-with-input-file input-file
> (lambda (port)
> (call-with-zstd-input-port port
> (lambda (input)
> (while (not (eof-object?
> (get-bytevector-n! input bv 0 block-size)
> 
> (run)
> --8<---cut here---end--->8---
> 
> It now runs much faster:
> 
> --8<---cut here---start->8---
> $ time+ zstd -cdk /tmp/chromium-98.0.4758.102.tar.zst > /dev/null
> cpu: 98%, mem: 10560 KiB, wall: 0:09.56, sys: 0.37, usr: 9.06
> --8<---cut here---end--->8---
> 
> --8<---cut here---start->8---
> $ time+ guile ~/src/guile-zstd/benchmark.scm
> cpu: 100%, mem: 25152 KiB, wall: 0:11.69, sys: 0.38, usr: 11.30
> --8<---cut here---end--->8---
> 
> So guile-zstd was about 20% slower, not too far.
> 
> For completeness, here's the same benchmark adjusted for guile-zlib:
> 
> --8<---cut here---start->8---
> (use-modules (ice-9 binary-ports)
> (ice-9 match)
> (rnrs bytevectors)
> (zlib))
> 
> (define MiB (expt 2 20))
> (define block-size (* 4 MiB))
> (define bv (make-bytevector block-size))
> (define input-file "/tmp/chromium-98.0.4758.102.tar.gz")
> 
> (define (run)
> (call-with-input-file input-file
> (lambda (port)
> (call-with-gzip-input-port port
> (lambda (input)
> (while (not (eof-object?
> (get-bytevector-n! input bv 0 block-size)
> 
> (run)
> --8<---cut here---end--->8---
> 
> --8<---cut here---start->8---
> $ time+ guile ~/src/guile-zstd/benchmark-zlib.scm
> cpu: 86%, mem: 14552 KiB, wall: 0:23.50, sys: 1.09, usr: 19.15
> --8<---cut here---end--->8---
> 
> --8<---cut here---start->8---
> $ time+ gunzip -ck /tmp/chromium-98.0.4758.102.tar.gz > /dev/null
> cpu: 98%, mem: 2304 KiB, wall: 0:35.99, sys: 0.60, usr: 34.99
> --8<---cut here---end--->8---
> 
> Surprisingly, here guile-zlib appears to be faster than the 'gunzip'
> command; guile-zstd is about twice as fast to decompress this 4 GiB
> something archive (compressed with zstd at level 19).
> 
> So, it seems the foundation we're building on is sane after all. This
> suggests that compression is not the bottleneck when generating the man
> pages database, probably because it only needs to read the first few
> bytes of each compressed manpage to gather the information it needs, and
> that the rest is more expensive compared to that (such as
> string-tokenize'ing the lines read to parse the data).
> 
> To be continued...
> 
> Thanks!
> 
> Maxim

Deprecating legacy build phase style when cross-compiling vs. native

[Josselin Poiret]

Hello everyone,

Following a report on IRC [1] that nss-certs wouldn't cross-build, I
found out that the old syntax of using %output instead of the shinier
$output was still supported only when native-compiling, and not when
cross-compiling, at least for build-systems inheriting from gnu.

Looking at guix/build-system/gnu.scm [2], we have gnu-build
vs. gnu-cross-build, which are responsible for native vs. cross-builds.
The former uses with-build-variables to set the legacy build variables
such as %output and %outputs (see the def in guix/gexp.scm), whereas the
latter does not, only doing so manually and so omitting %output.

This means that packages that were not updated to fit the new style
should all fail to cross-compile.  This simple bug could be resolved by
adding %output to gnu-cross-build, however as was argued on IRC this is
now undocumented behaviour and we'd rather switch all packages to the
new style instead.  While I 100% agree with this, I think we should have
a uniform deprecation policy for this matter, and that there shouldn't
be such a disparity between cross and native builds.

What do you all think?

[1] https://logs.guix.gnu.org/guix/2022-03-30.log#150521
[2] 
https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build-system/gnu.scm?id=1c2da6603565bafba58b81742ce705dc8becb2f7

Best,
-- 
Josselin Poiret

Re: The Shepherd on Fibers

[Ludovic Courtès]

Maxime Devos  skribis:

> (format #t "Accepted connection on ~0@*~a from ~1@*~a." "foo" "bar"))

Hmm that doesn’t seem to work:

--8<---cut here---start->8---
scheme@(guile-user)> ,use(ice-9 format)
scheme@(guile-user)> (format #f "Accepted connection on ~@0*~a from ~@1*~a." 1 
2)

FORMAT: error with call: (format #f "Accepted connection on ~@0<===*~a from 
~@1*~a." ===>1 2 )
misplaced modifier
FORMAT: INTERNAL ERROR IN FORMAT-ERROR!
destination: #f
format string: "Accepted connection on ~@0*~a from ~@1*~a."
format args: (1 2)
error args:  (#f "error in format" () #f)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
error in format
--8<---cut here---end--->8---

Keeping it as future work…

Ludo’.

Re: Profiling of man-db database generation with zlib vs zstd

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

[...]

> To isolate the problem, you could allocate the 4 MiB buffer outside of
> the loop and use ‘get-bytevector-n!’, and also remove code that writes
> to ‘output’.

I've adjusted the benchmark like so:

--8<---cut here---start->8---
(use-modules (ice-9 binary-ports)
 (ice-9 match)
 (rnrs bytevectors)
 (zstd))

(define MiB (expt 2 20))
(define block-size (* 4 MiB))
(define bv (make-bytevector block-size))
(define input-file "/tmp/chromium-98.0.4758.102.tar.zst")

(define (run)
  (call-with-input-file input-file
(lambda (port)
  (call-with-zstd-input-port port
(lambda (input)
  (while (not (eof-object?
   (get-bytevector-n! input bv 0 block-size)

(run)
--8<---cut here---end--->8---

It now runs much faster:

--8<---cut here---start->8---
$ time+ zstd -cdk /tmp/chromium-98.0.4758.102.tar.zst > /dev/null
cpu: 98%, mem: 10560 KiB, wall: 0:09.56, sys: 0.37, usr: 9.06
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ time+ guile ~/src/guile-zstd/benchmark.scm
cpu: 100%, mem: 25152 KiB, wall: 0:11.69, sys: 0.38, usr: 11.30
--8<---cut here---end--->8---

So guile-zstd was about 20% slower, not too far.

For completeness, here's the same benchmark adjusted for guile-zlib:

--8<---cut here---start->8---
(use-modules (ice-9 binary-ports)
 (ice-9 match)
 (rnrs bytevectors)
 (zlib))

(define MiB (expt 2 20))
(define block-size (* 4 MiB))
(define bv (make-bytevector block-size))
(define input-file "/tmp/chromium-98.0.4758.102.tar.gz")

(define (run)
  (call-with-input-file input-file
(lambda (port)
  (call-with-gzip-input-port port
(lambda (input)
  (while (not (eof-object?
   (get-bytevector-n! input bv 0 block-size)

(run)
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ time+ guile ~/src/guile-zstd/benchmark-zlib.scm
cpu: 86%, mem: 14552 KiB, wall: 0:23.50, sys: 1.09, usr: 19.15
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ time+ gunzip -ck /tmp/chromium-98.0.4758.102.tar.gz > /dev/null
cpu: 98%, mem: 2304 KiB, wall: 0:35.99, sys: 0.60, usr: 34.99
--8<---cut here---end--->8---

Surprisingly, here guile-zlib appears to be faster than the 'gunzip'
command; guile-zstd is about twice as fast to decompress this 4 GiB
something archive (compressed with zstd at level 19).

So, it seems the foundation we're building on is sane after all.  This
suggests that compression is not the bottleneck when generating the man
pages database, probably because it only needs to read the first few
bytes of each compressed manpage to gather the information it needs, and
that the rest is more expensive compared to that (such as
string-tokenize'ing the lines read to parse the data).

To be continued...

Thanks!

Maxim

Re: lxc and subuid

[Maxime Devos]

Antonio Carlos Padoan Junior schreef op wo 30-03-2022 om 15:13 [+0200]:
> Thanks Maxime,
> 
> 
> Maxime Devos  writes:
> 
> > Antonio Carlos Padoan Junior schreef op wo 30-03-2022 om 08:51 [+0200]:
> > > Hello,
> > > 
> > > I'm trying to figure out how to set a unprivileged container using lxc
> > > in guix. I do not know either how to allocate subuid/gid space in guix,
> > 
> > subuid/gid are _not_ unprivileged.  They are an userspace feature by
> > the (privileged) setuid binary 'newuidmap', see
> > .
> > 
> > I don't think there's currently a mechanism for that in Guix System,
> > except manually creating and modifying /etc/subuid appropriately and
> > installing the setuid binaries.  However, I suppose that the 'user-
> > account' record could be extended to support subuid/subgid and
> > automatically create /etc/subuid.
> 
> I created them manually as you suggested. But now I'm in trouble with
> the creation of virtual network interfaces for the container. It is not
> possible to follow the standard lxc documentation and apply it for guix 
> directly.
> The same problem if I use lxd. 
> 
> I'm looking the "Singularity service" as an alternative for lxc but it seem 
> it does
> not install the daemon (as per guix documentation). I have no idea
> how to properly proceed and set a viable singularity deamon in my machine.
> 
> I will try docker service instead, but this is not exactly what I'm
> looking for (but I hope at least it will work).
> 
> I have the feeling people create guix packages and services for
> personal use but without minimal documentation on how to use properly on
> guix. Please consider that as a critic from someone that has goodwill
> but who is a little bit frustrated today.

I'm not familiar with lxc, lxd, Docker or Singularity so I'm afraid I
cannot help here.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: better error messages through assertions

[Andy Wingo]

On Wed 30 Mar 2022 11:37, Ludovic Courtès  writes:

>> scheme@(guile-user)> (container-contents '())
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> In procedure struct-vtable: Wrong type argument in position 1
> scheme@(guile-user)> ,use(srfi srfi-9)
> scheme@(guile-user)> (define-record-type 
>  (make-foo x)
>  foo?
>  (x foo-x))
> scheme@(guile-user)> ,optimize (foo-x '())
> $9 = (if (eq? (struct-vtable '()) )
>   (struct-ref '() 0)
>   (throw 'wrong-type-arg
>  'foo-x
>  "Wrong type argument: ~S"
>  (list '())
>  (list '(
>
> With Guile 3, it might be that adding an extra ‘struct?’ test would have
> little effect on performance; we’d need to check.

Would have no effect.

Incidentally, you might want to use ,optimize-cps;

  scheme@(guile-user)> ,optimize (foo-x '())
  $9 = (if (eq? (struct-vtable '()) )
(struct-ref '() 0)
(throw 'wrong-type-arg
   'foo-x
   "Wrong type argument: ~S"
   (list '())
   (list '(
  scheme@(guile-user)> ,optimize-cps (foo-x '())
  L0:   ;   at 
:15:14
v0 := self
L1(...)
  L1:
receive()
v1 := const ()  ; arg   at 
:15:21
throw throw/value+data[#(wrong-type-arg "struct-vtable" "Wrong type 
argument in position 1 (expecting struct): ~S")](v1) ;  at :15:14

L1(...) means, pass all values to L1.  In this case because there are
varargs on the stack from the procedure call.  L1 parses them with the
receive().  Anyway, here we see that with respect to the immediate '(),
that all the tests folded.  If we instead lift to a procedure:

  scheme@(guile-user)> ,optimize-cps (lambda (x) (foo-x x))
  L0:   ;   at 
:16:14
v0 := self
L1(...)
  L1:
receive()
v1 := current-module()  ; moduleat 
:16:14
cache-set![0](v1)   ;   at 
:16:14
v2 := const-fun L7  ; _ 
return v2   ;   at 
:16:14

  L7:   ;   at 
:16:14
v3 := self
L8(...)
  L8:
v4 := receive(x); x 
heap-object?(v4) ? L10() : L38();   at 
:16:26
  L10():
struct?(v4) ? L11() : L38() ;   at 
:16:26
  L38():
throw throw/value+data[#(wrong-type-arg "struct-vtable" "Wrong type 
argument in position 1 (expecting struct): ~S")](v4) ;  at :16:26
  L11():
v5 := scm-ref/tag[struct](v4)   ; vtableat 
:16:26
v6 := cache-ref[(0 . )]()  ; cachedat 
:10:20
heap-object?(v6) ? L19() : L14();   at 
:10:20
  L19():
L20(v6) ;   at 
:10:20
  L14():
v7 := cache-ref[0](); mod   at 
:10:20
v8 := const; name  at 
:10:20
v9 := lookup-bound(v7, v8)  ; var   at 
:10:20
cache-set![(0 . )](v9) ;   at 
:10:20
L20(v9) ;   at 
:10:20
  L20(v10): ; box 
v11 := scm-ref/immediate[(box . 1)](v10); arg   at 
:10:20
eq?(v5, v11) ? L22() : L37();   at 
:16:26
  L37():
throw throw/value+data[#(wrong-type-arg foo-x "Wrong type argument: 
~S")](v4) ;  at :16:26
  L22():
v12 := word-ref/immediate[(struct . 6)](v5) ; rfields   at 
:16:26
v13 := v12  ; nfields   at 
:16:26
imm-u64-<[0](v13) ? L25() : L35()   ;   at 
:16:26
  L35():
v21 := const 0  ; _ at 
:16:26
throw throw/value+data[#(out-of-range "struct-ref/immediate" "Argument 2 
out of range: ~S")](v21) ;  at :16:26
  L25():
v14 := pointer-ref/immediate[(struct . 7)](v5) ; ptrat 
:16:26
v15 := load-u64[0](); word  at 
:16:26
v16 := u32-ref[bitmask](v5, v14, v15)   ; bits  at 
:16:26
v17 := load-u64[1](); mask  at 
:16:26
v18 := ulogand(v17, v16); res   at 
:16:26
u64-imm-=[0](v18) ? L31() : L33()   ;   at 
:16:26
  L33():
v20 := const 0  ; _ at 
:16:26
throw throw/value+data[#(wrong-type-arg "struct-ref/immediate" "Wrong type 
argument in position 2 (expecting boxed field): ~S")](v20) ;  at :16:26
  L31():
v19 := scm-ref/immediate[(struct . 1)](v4)  ; val   at 
:16:26
return v19  ;   at 
:16:26

Here we see the first procedure which 

Re: lxc and subuid

[Antonio Carlos Padoan Junior]

Thanks Maxime,


Maxime Devos  writes:

> Antonio Carlos Padoan Junior schreef op wo 30-03-2022 om 08:51 [+0200]:
>> Hello,
>> 
>> I'm trying to figure out how to set a unprivileged container using lxc
>> in guix. I do not know either how to allocate subuid/gid space in guix,
>
> subuid/gid are _not_ unprivileged.  They are an userspace feature by
> the (privileged) setuid binary 'newuidmap', see
> .
>
> I don't think there's currently a mechanism for that in Guix System,
> except manually creating and modifying /etc/subuid appropriately and
> installing the setuid binaries.  However, I suppose that the 'user-
> account' record could be extended to support subuid/subgid and
> automatically create /etc/subuid.

I created them manually as you suggested. But now I'm in trouble with
the creation of virtual network interfaces for the container. It is not
possible to follow the standard lxc documentation and apply it for guix 
directly.
The same problem if I use lxd. 

I'm looking the "Singularity service" as an alternative for lxc but it seem it 
does
not install the daemon (as per guix documentation). I have no idea
how to properly proceed and set a viable singularity deamon in my machine.

I will try docker service instead, but this is not exactly what I'm
looking for (but I hope at least it will work).

I have the feeling people create guix packages and services for
personal use but without minimal documentation on how to use properly on
guix. Please consider that as a critic from someone that has goodwill
but who is a little bit frustrated today.   


>
> Greetings,
> Maxime
>

Best regards,
-- 
Antonio Carlos PADOAN JUNIOR
GPG fingerprint:
243F 237F 2DD3 4DCA 4EA3  1341 2481 90F9 B421 A6C9

Re: using srfi-189 in (gnu services configuration)

[Attila Lendvai]

> > - the current code uses the symbol 'DISABLED
>
> It's a bit of a distraction to the discusses issue, but in Guile
> Scheme, symbols are case-sensitive, so (not (eq? 'disabled 'DISABLED)).


to clarify: i'm using uppercase here only to discriminate scheme
symbols from a free-flowing english text. it's common practice in the
CL world, but i've also seen it in the Guix docstrings.


> This does not appear to be true, at least for (srfi srfi-9) records:
>
> the following code can put unspecified in Guile records without any
> errors:
>
> (use-modules (srfi srfi-9))
> (define-record-type 
> (make-foobar foo) foo? (foo foobar-foo))
> (pk 'foobar (make-foobar unspecified))
> ;;; (foobar #< foo: #>)


my apologies for stating something with confidence that is not true!

i have vivid memory of having tried to use *unspecified*, and getting
errors from record accessors, but i cannot reproduce it now. maybe i
did something with UNDEFINED?, but i don't even see now how to get
hold of that value.

anyway, i'll try to patch up (gnu services configuration) to use
*unspecified* instead of 'DISABLED, and i'll report back with the end
result it it's worthy of that.


> Anyway, even if unspecified causes problems, this can be resolved by
> introducing a new constant like unspecified or the symbol 'disabled',
> but without the potential confusion with a symbol. E.g.:
>
> (define-values (unset-configuration-value unset-configuration-value?)
> (let ()
> (define-record-type 
> (make-unset-configuration-value) unset-configuration-value?
> unset-configuration-value?)
> (values (make-unset-configuration-value)
> unset-configuration-value?)))


it's not really relevant now, but this is pretty much what srfi-189
does, but as a documented standard.


> srfi-189 is also an option, but it seems to me that Haskell-style
> Maybe/Just/None that would require lots of wrapping and unwrapping
> which seems a bit tedious to me -- doable and definitely an option, but
> potentially tedious.


i'm afraid about that, too, but i cannot say before i start
implementing it.

and i think the config code will be equally littered with (if
(unspecified?  ...)  ...) forms, as opposed to (maybe-ref ...) forms
when using srfi-189.


> Additionally, for your Swarm example, would something like the
> following work:


this is an excellent idea! (namely, to capture the settings of various
swarms into instances, and then predefine the two well-known swarms)

i'll implement this first, and only move on to the config stuff
afterwards.

thanks again Maxime,

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“He alone is great and happy who fills his own station of independence, and has 
neither to command nor to obey.”
— Johann Wolfgang von Goethe (1749–1832), 'With the Iron Hand' (1773), 
Act I

Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

Hi guix,

Quite some packages in Guix use the Autotools system.  In this system,
a 'configure.ac' and 'Makefile.am' script / makefile is written, from
which 'autoconf' & 'automake' generate a very long bash script and a
Makefile.in.  Depending on the maintainer of the upstream package, this
'configure' and 'Makefile.in' are sometimes included in release
tarballs.

This seems in conflict with:

  * (guix)Submitting patches: ‘Make sure the package does not use
bundled copies of software already available as separate
packages.’

Autotools packages have a 'config.guess' and 'config.sub' script
that need to be updated whenever there's a new architecture.  As
such, for some packages, these need to be replaced for aarch64,
powerpc or risv64.  There are also some packages with very old
configure scripts that don't support --build/--host/--target,
which could gain --build/--host/--target support by just
regenerating them.

This also makes ensuring a package does not contain any malware
much harder, because the configure script (and related files)
needs to be read in their entirity.

 * When an upstream tarball contains .so, .dll, .a, etc. binaries,
   they are removed downstream in a snippet.  Why would the Autotools
   be an exception?

For some ‘early’ packages (gcc, glibc, binutils, ...), there's a
circularity problem, so building 'configure' and 'Makefile.in' from
source might not always be possible, but WDYT of building 'configure' &
'Makefile.in' from source for packages where it does not result in
bootstrapping problems?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: lxc and subuid

[Maxime Devos]

Antonio Carlos Padoan Junior schreef op wo 30-03-2022 om 08:51 [+0200]:
> Hello,
> 
> I'm trying to figure out how to set a unprivileged container using lxc
> in guix. I do not know either how to allocate subuid/gid space in guix,

subuid/gid are _not_ unprivileged.  They are an userspace feature by
the (privileged) setuid binary 'newuidmap', see
.

I don't think there's currently a mechanism for that in Guix System,
except manually creating and modifying /etc/subuid appropriately and
installing the setuid binaries.  However, I suppose that the 'user-
account' record could be extended to support subuid/subgid and
automatically create /etc/subuid.

Greetings,
Maxime


signature.asc
Description: This is a digitally signed message part

Re: The Shepherd on Fibers

[Ludovic Courtès]

Hi,

Attila Lendvai  skribis:

> my first ever service has non-trivial work to do in the start GEXP,
> and unsurprisingly, it has often ended up throwing an exception while
> i subled along my learning curve. it's not caught by shepherd right
> around the call to the start GEXP, and reaches a handler up higher
> that ends up deleting the socket (and IIRC skipping the start of rest
> of the services).

Ah ha!  I guess that can happen if the service is started right from the
config file; it cannot happen if the service is started via ‘herd start’
(because all the ‘start’ exceptions are caught and reported early), or
if it’s started from the config file but in a separate fiber with
‘start-in-the-background’.

Ludo’.

Re: The Shepherd on Fibers

[Ludovic Courtès]

Philip McGrath  skribis:

> But, as the above paper says, this means that Chez's `call/cc`,
> `dynamic-wind`, etc. are *unsafe* from the perspective of Racket's
> control primitives. From the docs for Racket's `ffi/unsafe/vm` library [6]:

I think that’s the crux of the problem and widely recognized:
‘dynamic-wind’ and ‘call/cc’ should be avoided.  This was also mentioned
in the context of Rees’ 1996 “Security Kernel” paper¹.

Ludo’.

¹ http://mumble.net/~jar/pubs/secureos/

Re: better error messages through assertions

[Ludovic Courtès]

Hi Philip,

Philip McGrath  skribis:

> I'm thinking that a reasonable place to start might be to implement a
> `contract->sanitizer` form that would allow using contracts to create 
> sanitizers, ideally with no changes to `(guix records)`.

OK.  I’d prefer if people who define record types could directly write:

  (field getter (contract integer/c))

rather than:

  (field getter (sanitizer (contract->sanitizer integer/c)))

But that’s more of a detail.

> What is the preferred mechanism for exceptions?

For Guix code, SRFI-34/35.

> Likewise, what record system should I use?

SRFI-9.

(Perhaps we should put answers to these questions in the “Coding Style”
section of the manual.)

> Also, I don't know much about how the "abi" aspect of (guix records)
> works and what types of changes there would trigger rebuilds. (Though, 
> again, I hope no changes would be needed for the proof-of-concept phase.)

I don’t think you need to worry about that.

> Another problem here seems to be the fault of (srfi srfi-9). For example:

[...]

> scheme@(guile-user)> (container-contents '())
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure struct-vtable: Wrong type argument in position 1
> (expecting struct): ()
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> ,bt
> In current input:
>   3:0  1 (_)
> In ice-9/boot-9.scm:
>   1685:16  0 (raise-exception _ #:continuable? _)
> ```
>
> It seems like `container-contents` and other field accessors ought to
> check their arguments with `container?` (or the applicable predicate) 
> and not leave error reporting to `struct-vtable`.

SRFI-9 generates the smallest amount of code for the job:

--8<---cut here---start->8---
scheme@(guile-user)> ,use(srfi srfi-9)
scheme@(guile-user)> (define-record-type 
   (make-foo x)
   foo?
   (x foo-x))
scheme@(guile-user)> ,optimize (foo-x '())
$9 = (if (eq? (struct-vtable '()) )
  (struct-ref '() 0)
  (throw 'wrong-type-arg
 'foo-x
 "Wrong type argument: ~S"
 (list '())
 (list '(
--8<---cut here---end--->8---

With Guile 3, it might be that adding an extra ‘struct?’ test would have
little effect on performance; we’d need to check.

> Perhaps this could be fixed in the (guix records) layer?

Could be, yes.

Thanks for looking into this!

Ludo’.

Re: The Shepherd on Fibers

[Ludovic Courtès]

Hi!

Philip McGrath  skribis:

>> Fibers is used in a single-threaded fashion, which is the main
>> constraint for shepherd since it forks.  That also means that fibers
>> cannot be preempted, so it’s fully cooperative scheduling.
>> 
>
> Would it be feasible for shepherd *not* to fork?

Sure, but that would make it a daemon supervisor that doesn’t launch any
daemon, which would make it less useful.  :-)

> Or only to fork in a way that cooperates with fibers?

A process that has multiple POSIX threads cannot fork reliably; that’s
the core of the problem (it’s a problem of POSIX and ‘fork’, not
specific to Guile and Fibers).

> Obviously forking is pretty ubiquitous, but I the 2019 paper "A fork()
> in the road"[1] fairly convincing in its argument that

Yes, I read it before.  We could have bindings for ‘posix_spawn’ but
it’s kind of ugly and limited.

> More concretely, preemption is a big benefit of fibers.

I’m not arguing against preemption, I like it.  :-)

However, as currently implemented in Fibers, we cannot rely on it,
though Maxime mentioned that it could use signals instead of a separate
thread, which would work.  We can always revisit that later.  I making a
first step here, and being cautious, but we can and should unleash our
creativity once we have the basics in place.

> Racket has a cross-platform C library, "rktio"[3], for accessing
> os-specific functionality. It was refactored into its current form in 
> 2017 as an early step toward Racket-on-Chez: while it happens to
> provide exactly the functionality Racket needs, it no longer is
> specific to Racket or any particular implementation thereof. That
> includes everything needed to implement the Concurrent ML system and
> nonblocking IO on a variety of OSes and kernels.
>
> In particular, it implements—IIUC primarily in
> "rktio_process.c"—abstractions (over `fork` or alternatives) to start
> a new process running something, with environment, stdin, stdout, and 
> stderror wired up ports in the sense of
> `current-{input,output,error}-port`, and use the Concurrent ML system
> to monitor its exit status, send it signals, etc. The Racket-level API
> is documented at [4].

Interesting.  I’m not sure this is directly translatable to Guile.
However, libevent support for Fibers should address this specific point,
so I’m not really concerned.

> Second, I'm a little uneasy about `unwind-protect`:

Your criticism is valid, though here it has a single user and it’s “good
enough” for that case I think.

Thanks for your feedback!

Ludo’.

lxc and subuid

[Antonio Carlos Padoan Junior]

Hello,

I'm trying to figure out how to set a unprivileged container using lxc
in guix. I do not know either how to allocate subuid/gid space in guix, is it
possible? Any advices?

Best regards,
-- 
Antonio Carlos PADOAN JUNIOR
GPG fingerprint:
243F 237F 2DD3 4DCA 4EA3  1341 2481 90F9 B421 A6C9