bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

The build errors out with:

--8<---cut here---start->8---
  SNARF  weak-set.doc
  SNARF  weak-table.doc
  SNARF  weak-vector.doc
  SNARF  posix.doc
  SNARF  net_db.doc
  SNARF  socket.doc
  SNARF  regex-posix.doc
  CCLD libguile-2.2.la
.libs/libguile_2.2_la-posix.o: In function `scm_tmpnam':
/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile/posix.c:1574: warning: 
the use of `tmpnam' is dangerous, better use `mkstemp'
  CCLD guile
  GEN  guile-procedures.texi
make[3]: Leaving directory 
'/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile'
make[2]: Leaving directory 
'/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile'
Making all in bootstrap
make[2]: Entering directory 
'/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/bootstrap'
  BOOTSTRAP GUILEC ice-9/eval.go
wrote `ice-9/eval.go'
  BOOTSTRAP GUILEC ice-9/psyntax-pp.go
  BOOTSTRAP GUILEC language/cps/intmap.go
building of `/gnu/store/i2p0gqxm378ydlh58qpy6jas7rdk79jg-guile-2.2.2.drv' timed 
out after 3600 seconds of silence
cannot build derivation
  `/gnu/store/c34w733549bjvcdixb0yj306ydhwv8c3-guile2.2-gnutls-3.5.9.drv':
  1 dependencies couldn't be built
--8<---cut here---end--->8---

I'm not sure where this timeout value can be configured, but clearly it
shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
manage to build it under 1 hour.

Maxim

bug#27152: deprecation warnings with Guile 2.2.2

[Ricardo Wurmus]

I get a couple of deprecation warnings with Guile 2.2.2, for example

Import (ice-9 threads) to have access to `current-processor-count'.
`_IOFBF' is deprecated.  Use the symbol 'block instead.

I only see them after “export GUILE_WARN_DEPRECATED=detailed”.  Without
that variable I get a block of text that informs me that deprecated
features have been used.

This happens especially when running “make” or when not all files have
been compiled.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27067: Feature request: please allow for either automatic logging of all output of every guix commands or add an option to each guix command to allow it for all to be logged

[Ludovic Courtès]

Hi Brenton,

Brenton Horne  skribis:

> As I understand it only the guix build command has an option to log output to 
> a file.

Every derivation that is built, regardless of the command, has its log
stored in /var/log/guix/drvs.  What ‘guix build’ has is a simple way to
get the file name or URL of a build log:

  guix build --log-file coreutils

See:

  
https://www.gnu.org/software/guix/manual/html_node/Additional-Build-Options.html
  
https://www.gnu.org/software/guix/manual/html_node/Invoking-guix_002ddaemon.html
(specifically --lose-logs and --disable-log-compression)

Can we say your feature request has already been fulfilled?  :-)

Thanks,
Ludo’.

bug#27137: make check 'FAIL: tests/guix-package-net'

[myglc2]

On 05/30/2017 at 17:45 Ludovic Courtès writes:

> Hi myglc2,
>
> myglc2  skribis:
>
>> + test t-profile-21734-2-link = t-profile-21734-2-link
>> + guix package -p t-profile-21734 --switch-generation=-1
>> accepted connection from pid 30492, user g1
>> switched from generation 2 to 1
>> guix package: warning: Your Guix installation is 13 days old.
>> guix package: warning: Consider running 'guix pull' followed by
>> 'guix package -u' to get up-to-date packages and security updates.
>>
>> ++ readlink_base t-profile-21734
>> +++ readlink t-profile-21734
>> ++ basename t-profile-21734-1-link
>> + test t-profile-21734-1-link = t-profile-21734-1-link
>> ++ seq 1 3
>> + for i in `seq 1 3`
>> + guix package --bootstrap --roll-back -p t-profile-21734
>> accepted connection from pid 30827, user g1
>> guix package: error: profile 't-profile-21734' does not exist
>
> Could it be that a concurrent process removed ‘t-profile-21734’ behind
> our back?

I have no idea! This time I did ...

make -j check

... but I often do ...

make -j 10 check

... so I repeated ...

make -j 10 check

... a few times thinking maybe another test could delete the link, but
could not reproduce the error. BTW, from each 'make check' I do end up
with a dangling link like ...

t-profile-alt-10987-1-link ->
/home/g1/src/guix/test-tmp/store/hwr4xiqd60wrh033wvjljly50j6d39dg-profile

... is this normal?

> As you can see, that symlink was available just above.
> Or is it 100% reproducible when running
>
>   make check TESTS=tests/guix-package-net.sh
>
> ?

It is not reproducible ... except by deleting the profile while the test is
running ;-)

HTH - George

bug#27135: /root is world readable by default

[Marius Bakke]

Ludovic Courtès  writes:

> Hi Alex,
>
> Alex Griffin  skribis:
>
>> After a default install of GuixSD, anybody can read root's home
>> directory. I think /root should have permissions 700 instead of 755.
>
> Fixed in 41db5a756369f5b14d1e67a523ee0940cad56744.
>
> For the other user accounts, useradd(8) does its thing, and apparently
> it defaults to world-readable accounts (it defaults to a umask of 022 as
> written in the man page).
>
> Thoughts?

I'm in favor of overriding that default. I usually chmod /home/* to 0700
anyway. 0750 would be okay too and probably covers more use cases.


signature.asc
Description: PGP signature

bug#27135: /root is world readable by default

[Ludovic Courtès]

Hi Alex,

Alex Griffin  skribis:

> After a default install of GuixSD, anybody can read root's home
> directory. I think /root should have permissions 700 instead of 755.

Fixed in 41db5a756369f5b14d1e67a523ee0940cad56744.

For the other user accounts, useradd(8) does its thing, and apparently
it defaults to world-readable accounts (it defaults to a umask of 022 as
written in the man page).

Thoughts?

Thanks,
Ludo’.

bug#27137: make check 'FAIL: tests/guix-package-net'

[Ludovic Courtès]

Hi myglc2,

myglc2  skribis:

> + test t-profile-21734-2-link = t-profile-21734-2-link
> + guix package -p t-profile-21734 --switch-generation=-1
> accepted connection from pid 30492, user g1
> switched from generation 2 to 1
> guix package: warning: Your Guix installation is 13 days old.
> guix package: warning: Consider running 'guix pull' followed by
> 'guix package -u' to get up-to-date packages and security updates.
>
> ++ readlink_base t-profile-21734
> +++ readlink t-profile-21734
> ++ basename t-profile-21734-1-link
> + test t-profile-21734-1-link = t-profile-21734-1-link
> ++ seq 1 3
> + for i in `seq 1 3`
> + guix package --bootstrap --roll-back -p t-profile-21734
> accepted connection from pid 30827, user g1
> guix package: error: profile 't-profile-21734' does not exist

Could it be that a concurrent process removed ‘t-profile-21734’ behind
our back?  As you can see, that symlink was available just above.

Or is it 100% reproducible when running

  make check TESTS=tests/guix-package-net.sh

?

Thanks,
Ludo’.

bug#27069: LUKS partition ruined by guix init

[Marius Bakke]

Mark H Weaver  writes:

> Marius Bakke  writes:
>
>> some...@selfhosted.xyz writes:
>>
>>> Hi,
>>>
>>> I have several times over tried to install to a LUKS-partition with
>>> btrfs and failed but last time I almost had success. guix init command
>>> complained that the bios_grub flag wasn't set after many hours of
>>> compiling packages and therefore didn't install grub at first. I
>>> checked the partition with "cryptsetup luksUUID /dev/sda1" before
>>> rerunning guix init and it gives me the luksUUID so the partition
>>> seemed intact up to that point. However, after setting the bios_grub
>>> flag and rerunning guix init, the installation is successful but
>>> afterwards the cryptsetup luksUUID command says that the partition is
>>> not a valid luks device, so it seems like the GRUB installation part
>>> is what ruins the partition.
>>
>> What is your partition layout? The "bios_grub" partition must be a
>> separate (typically tiny) partition with no other data on it.
>
> FWIW, my current development system is GuixSD running with a
> LUKS-encrypted Btrfs root partition with GUID partition table.
> I don't remember encountering any difficulties during the install.

Interesting. Did you reserve some sectors before the first partition? Or
do you have the root partition marked as "bios_grub", which is what
seemingly wiped the LUKS headers here?


signature.asc
Description: PGP signature

bug#27003: [PATCH 0/3] Generalized wrap phase for perl, python.

[Marius Bakke]

Arun Isaac  writes:

>> I cannot currently make guarantees on when I'll be able to wrap my head
>> around the entire problem, so if you come to conclusions for next steps
>> on the basis of your testing, we should go with those.
>>
>> Else I will revisit this issue when I have time (which might be a
>> while).
>
> I don't have any ideas on how to fix this. And, I don't understand Guix
> internals very well. So, I'll wait for you or someone else to come up
> with a solution. But, I can help with testing patches.

I'm not sure if #25235 should block this issue though, since it's a
useful change regardless. But, not handling it adds "technical debt",
and a (minor) problem that is not currently present in perl modules.

We do have until the next 'core-updates' to think about it though ;-)


signature.asc
Description: PGP signature

bug#27120: [PATCH] gnu: graphicsmagick: Remove bundled libraries from source checkout.

[Ludovic Courtès]

Leo Famulari  skribis:

> Fixes .
>
> * gnu/packages/imagemagick.scm (graphicsmagick)[source]: Add a snippet
> to delete bundled libraries.
> [version]: Bump the package revision counter to 3.

That was fast!

> +(snippet
> + '(begin
> +;; Remove bundled software. This reduces the size of the
> +;; build source checkout from 177 MiB to 49 MiB. This 
> should
> +;; not be necessary when using the GraphicsMagick release
> +;; tarball, because these files are not distributed 
> there.
> +(for-each delete-file-recursively '("bzlib" "dcraw" 
> "hp2xx"
> +"jbig" "jp2" "jpeg"
> +"lcms" "libxml" "png"
> +"ralcgm" "tiff" "ttf"
> +"webp" "wmf" "xlib"
> +"zlib"))

You can even remove ‘begin’.

LGTM, thank you!

Ludo’.

bug#27149: ‘perl-build-system’ does not support cross-compilation

[Ricardo Wurmus]

When trying to cross-build GCompris for armhf the build fails because
Perl dependencies cannot be cross-built.

The “perl-build-system” needs to be augmented to do the right thing when
cross-compilation is requested.  But what is the right thing to do?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#26897: ‘cmake-build-system’ does not support cross-compilation

[Ricardo Wurmus]

Implemented in commit d2ac5e297578dea1c872f77a26ef4d481d5dc7bd.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27142: classpath-0.99-1.e7c13ee0c fails to build

[Ricardo Wurmus]

Fixed with commit 93c103ab594fae918743f50ec8346854f0a85f1a.
-- 
Ricardo

bug#27003: [PATCH 0/3] Generalized wrap phase for perl, python.

[Arun Isaac]

> I cannot currently make guarantees on when I'll be able to wrap my head
> around the entire problem, so if you come to conclusions for next steps
> on the basis of your testing, we should go with those.
>
> Else I will revisit this issue when I have time (which might be a
> while).

I don't have any ideas on how to fix this. And, I don't understand Guix
internals very well. So, I'll wait for you or someone else to come up
with a solution. But, I can help with testing patches.

bug#27146: system init: if /etc doesn't exist, create it

[ng0]

I had the unfortunate experience of remote fixing a system,
and one way I tried to fix my original problem was to
remove the /etc directory (this included the content of
debian 8 with some alternative changes to the minimal
one I did not set up myself).
When I rebooted, this is still at activation of the first
system generation, I get the repl because the symlink
target does not exist (obviously).

We should think of such situations, however uncommon they are,
and check if /etc exists. If it doesn't exist create it
otherwise it is assumed that it exists and for example
this symlink creation will work: ssl -> /run/current-system/profile/etc/ssl

-- 
ng0
OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588


signature.asc
Description: PGP signature

bug#27145: document GuixSD system recovery process + document bournish and recovery/guile repl

[ng0]

Currently neither bournish nor the guile recovery repl are documented
in a way which is helpful for a person which isn't technical
knowledged running into the unhappy case of a system which doesn't
boot. In fact both are not even mentioned in our documentation
and the built in help is not sufficient.
-- 
ng0
OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588


signature.asc
Description: PGP signature

bug#26948: ‘write-file’ output should not be locale-dependent

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer  skribis:

> l...@gnu.org (Ludovic Courtès) writes:

[...]

>> But wait!  “guix build nss-certs --check -K” fails, and the diff is:
>>
>> $ LANGUAGE= diff -ur 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2{,-check}
>> Only in 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs:
>>  
>> AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
>> Only in 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs: 
>> AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem
>> diff -ur 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs/ae8153b9.0
>>  
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs/ae8153b9.0
>> --- 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs/ae8153b9.0
>> 1970-01-01 01:00:01.0 +0100
>> +++ 
>> /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs/ae8153b9.0
>>   1970-01-01 01:00:01.0 +0100
>> @@ -3,10 +3,10 @@
>>  # distrust=
>>  # openssl-trust=codeSigning emailProtection serverAuth
>>  -BEGIN CERTIFICATE-
>> -MIIHyTCCBbGgAwIBAgIBATANBgkqhkiG9w0BAQUFADB9MQswCQYDVQQGEwJJTDEW
>> +MIIHhzCCBW+gAwIBAgIBLTANBgkqhkiG9w0BAQsFADB9MQswCQYDVQQGEwJJTDEW
>
> Can this be explained by locale alone? That is troubling.

Yes it’s troubling, it deserves more investigation.

>> There are two ways to create nars.  One is via the ‘export-paths’ RPC
>> (implemented in the daemon in C++), which does not interpret file names
>> and thus leaves them untouched.  The other one is via ‘write-file’ from
>> (guix serialization), which is written in Scheme and thus converts file
>> names from locale encoding (specifically, ‘scandir’ does that.)
>>
>> ‘guix publish’ uses the latter, so ‘guix publish’ is sensitive to locale
>> settings, which is pretty bad.
>>
>> Guile currently does not allow us to specify whether/how file names
>> should be decoded, but possible solutions have been discussed for 2.2.
>>
>> In the meantime, solutions are:
>>
>>   1. To run ‘guix publish’ in a UTF-8 locale, which apparently was not
>>  the case.
>
> I'm surprised by that. Wouldn't a utf8 locale be the default?

Users are free to choose their favorite locale.  Also, on a foreign
distro where locales are not properly set up, you end up in the C locale
with US-ASCII encoding (as was the case here).

>>   2. Add to (guix build syscalls) a separate locale-independent
>>  ‘scandir’ implementation and use that.
>
> If the general solution is to fix it in Guile, the workaround proposed
> in 1. seems preferable.

I implemented ‘scandir/utf-8’ and used that in ‘write-file’ (patches
attached).  Unfortunately that’s not enough since libguile procedures
like ‘open-file’ still do locale-dependent conversion, so we’d need to
duplicate those as well, which is not great.

But on second thought, I think the problem is not in the ‘write-file’
call that ‘guix publish’ makes: if it were, ‘scandir’ would return bogus
file names (with question marks), but then trying to open them would
fail, so ‘write-file’ wouldn’t produce a bogus nar.

So I think the culprit is ‘restore-file-set’ (used in ‘guix offload’
when retrieving store items from a build machine): this one reads file
names as UTF-8, via ‘read-store-path’, but then when it tries to create
those files using Guile’s primitives, their names can be be converted,
possibly with those question marks popping up.  Here ‘restore-file-set’
can’t notice that Guile changed the file names behind its back.

The short-term fix is to ensure guix-daemon itself runs in a UTF-8
locale.  :-/

I’ve restarted guix-daemon and ‘guix publish’ in a UTF-8 locale on
hydra.gnu.org.

Thanks,
Ludo’.


>From e7f464bac58e1f09de5ceb194c4a30f6d899b29a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= 
Date: Tue, 30 May 2017 12:03:54 +0200
Subject: [PATCH] syscalls: Add 'scandir/utf-8'.

* guix/build/syscalls.scm (%struct-dirent-header): New C struct.
(opendir/utf-8, closedir/utf-8, readdir/utf-8, scandir/utf-8): New
procedures.
* tests/syscalls.scm ("scandir/utf-8, ENOENT")
("scandir/utf-8, ASCII file names")
("scandir/utf8, UTF-8 file names"): New tests.
---
 guix/build/syscalls.scm | 73 +
 tests/syscalls.scm  | 39 ++
 2 files changed, 112 insertions(+)

diff --git a/guix/build/syscalls.scm b/guix/build/syscalls.scm
index 52439afd4..cfb43e93b 100644
--- a/guix/build/syscalls.scm
+++ b/guix/build/syscalls.scm
@@ -67,6 +67,7 @@
 mkdtemp!
 fdatasync
 pivot-root
+scandir/utf-8
 fcntl-flock
 
 set-thread-name
@@ -812,6 +813,78 @@ system to PUT-OLD."
 

 ;;;
+;;; Opendir & co.
+;;;
+
+(define-c-struct %struct-dirent-header
+  sizeof-dirent-header
+  list
+  read-dir

bug#27144: guix publish: “mutex already locked by thread”

[Ludovic Courtès]

Seen on hydra.gnu.org:

--8<---cut here---start->8---
GET /jkpcipgxfyfb60pr4b5n4x4j9k5mcxpp.narinfo
In ice-9/boot-9.scm:
 160: 5 [catch #t # ...]
In unknown file:
   ?: 4 [apply-smob/1 #]
In guix/workers.scm:
  84: 3 [#]
  72: 2 [loop]
  76: 1 [# misc-error 
...]
In unknown file:
   ?: 0 [make-stack #t]
ERROR: In procedure make-stack:
ERROR: Throw to key `misc-error' with args `("mutex already locked by thread")'.
--8<---cut here---end--->8---

This is with
/gnu/store/cxmj38x6rh0ykq3d5dlqbxr5h1zgiiaf-guile2.0-guix-0.13.0-1.a6d728b.

Ludo'.

bug#26948: gnutls errors on multiple guix commands

[Ludovic Courtès]

Hi Mark,

Mark H Weaver  skribis:

>> In the meantime we can work around it this way:
>>
>> diff --git a/guix/build/download.scm b/guix/build/download.scm
>> index ce4708a87..6ef623334 100644
>> --- a/guix/build/download.scm
>> +++ b/guix/build/download.scm
>> @@ -296,6 +296,13 @@ session record port using PORT as its underlying 
>> communication port."
>>(make-parameter (or (getenv "GUIX_TLS_CERTIFICATE_DIRECTORY")
>>(getenv "SSL_CERT_DIR"  ;like OpenSSL
>>  
>> +(define (set-certificate-credentials-x509-trust-file!* cred file format)
>> +  "Like 'set-certificate-credentials-x509-trust-file!', but without the file
>> +name decoding bug described at
>> +."
>> +  (let ((data (call-with-input-file file get-bytevector-all)))
>> +(set-certificate-credentials-x509-trust-data! cred data format)))
>> +
>>  (define (make-credendials-with-ca-trust-files directory)
>>"Return certificate credentials with X.509 authority certificates read 
>> from
>>  DIRECTORY.  Those authority certificates are checked when
>> @@ -309,7 +316,7 @@ DIRECTORY.  Those authority certificates are checked when
>>  (let ((file (string-append directory "/" file)))
>>;; Protect against dangling symlinks.
>>(when (file-exists? file)
>> -(set-certificate-credentials-x509-trust-file!
>> +(set-certificate-credentials-x509-trust-file!*
>>   cred file
>>   x509-certificate-format/pem
>>(or files '()))
>>
>>
>> WDYT?  I’ll commit it if that’s fine with you.
>
> I'm not sufficiently familiar with GnuTLS to properly review this, but I
> trust your judgement.

Pushed as 27fd13c3c2701204f48fe0012438edbb91957dfc.

Thanks,
Ludo’.

bug#27142: classpath-0.99-1.e7c13ee0c fails to build

[Ricardo Wurmus]

On a machine with 62G memory classpath fails to build from source.

--8<---cut here---start->8---
$ free -h
  totalusedfree  shared  buff/cache   available
Mem:62G1.1G 35G 32M 25G 61G
Swap:  5.0G  0B5.0G
--8<---cut here---end--->8---

The problem appears to be that by default the JVM tries to allocate as
much memory as it can, so the initial size of the heap may be larger
than the set maximum.

Here’s the error:

--8<---cut here---start->8---
/gnu/store/ncvpxqykv672wjfygmki7phc7v8kz8iz-ecj-javac-on-jamvm-wrapper-3.2.2/bin/javac
  -nowarn -source 1.6 -target 1.6 -bootclasspath '' -classpath 
../vm/reference:..:../external/w3c_dom:../external/sax:../external/relaxngDatatype:../external/jsr166:.::
 -J-Xmx768M -d . @classes
Minimum heap size greater than max!
touch compile-classes
touch resources
/gnu/store/rw6zb4l1dqg1zx252ri9722ffb1r20jm-fastjar-0.98/bin/fastjar cf 
glibj.zip gnu java javax org sun META-INF && 
/gnu/store/rw6zb4l1dqg1zx252ri9722ffb1r20jm-fastjar-0.98/bin/fastjar i glibj.zip
sun: No such file or directory
Error adding sun to jar archive!
make[1]: *** [Makefile:674: glibj.zip] Error 1
make[1]: Leaving directory 
'/tmp/guix-build-classpath-0.99-1.e7c13ee0c.drv-0/source/lib'
make: *** [Makefile:523: all-recursive] Error 1
phase `build' failed after 3.7 seconds
--8<---cut here---end--->8---

I think we may be able to fix this by ensuring that we also pass
-J-Xms768M to the compiler (or -Xms768M to the invocation of “java”) to
set the initial heap size no greater than the maximum.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net