Re: GNU Guix & GuixSD 0.16.0 released

[Pjotr Prins]

Yesterday I gave a passionate 'I love Guix' lightning talk about Guix
packages at the biohackathon in Japan. Many people after approached me
and said that the website is confusing them because it looks like it
is promoting a Linux distribution... I know this is an old topic, but
it would be nice if we found a way to promote Guix packaging as it
works so well outside GuixSD.

Pj.

Re: Guix & IPFS

[Pjotr Prins]

On Sun, Dec 09, 2018 at 09:19:20PM +0100, Pierre Neidhardt wrote:
> I found a solution to package IPFS without gx, so go-ipfs has just hit master!

That is excellent news. That means we have a way of distributing
materials in a robust fashion.

Pj.

Re: Long term plan for GuixSD security: microkernels, ocap, RISC-V support

[Adonay Felipe Nogueira]

Em 24/09/2018 11:14, Ludovic Courtès escreveu:
> Christopher Lemmer Webber  skribis:
>>- There's also Google's recent work with Magenta/Fuschia.  From what
>>  I've read, architecturally this looks right.  I think the reason
>>  for worry here is the same difficulty the community has had to
>>  build actual community and libre distributions on top of the
>>  Android ecosystem could apply here.
> 
> Indeed.
> 
> We could also mention MINIX, which many of us are already using daily.
> :-)
> 
> Putting aside Fuschia, I think the Hurd and MINIX are by far the
> solutions that require the less work to be in a state where people with
> “regular needs” like the rest of us to switch (MINIX is probably in that
> state already.)
> 
> The Hurd already has a very advanced POSIX C library, which is not
> negligible, especially compared to the other OSes.  Much progress has
> been made in recent years wrt. drivers (using the Rump kernel in
> particular.)  There are of course serious shortcomings, in particular
> lack of 64-bit and SMP support.  But fixing these is relatively “little
> work” in the grand scheme of things.
> 
> To put this in perspective, consider Linux namespaces: they have already
> seen years of evolution, and the story of user namespaces shows that
> it’s far from complete.

I don't know if what I'll say will be off-topic here given that this
list is about Guix development, not on general free/libre software
activism, but please forgive me anyways.

So, my worry is that if we somehow were to support Fuchsia and if it
were to be not strong auto-upgradable copyleft with community-oriented
enforcement, then we could actually loose the freedoms of the software
for the end user. This thought was initially presented by Eben Moglen
during one of his talks[1], but I just tried to bring the issue to Guix.

[1]
https://media.libreplanet.org/u/libreplanet/m/the-free-software-movement-in-the-age-of-trump/



signature.asc
Description: OpenPGP digital signature

Re: bug#33676: GuixSD on eoma68-a20?

[Andreas Enge]

Hello Danny,

On Sat, Dec 08, 2018 at 06:12:14PM +0100, Danny Milosavljevic wrote:
> Guix received one but we have so far be unable to get the GuixSD flash
> image because something always breaks on Hydra before it's done

although the problem seems to lie somewhere else, I would just like to
point out that you can use your account on bayfront to build things for
arm as well - it offloads to the redhill machine. Notice, however, that
this is a single machine building everything from source without using
the hydra or berlin build farms for substitutes.

I can also give you a direct login on redhill if you need it.

Andreas

Re: GuixSD on eoma68-a20?

[Mark H Weaver]

Hi Danny and Ludovic,

Danny Milosavljevic  writes:

> After the change, I get the following on Hydra 
> :
>
> ---
> @ build-started /gnu/store/scnqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv - 
> x86_64-linux 
> /var/log/guix/drvs/sc//nqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv.bz2
> environment variable `PATH' set to 
> `/gnu/store/5ka9gmcwcj2919q0cj0wkjng058lwrgq-qemu-minimal-3.0.0/bin:/gnu/store/5s2nib1lrd2101bbrivcl17kjx1mspw6-coreutils-8.30/bin'
> creating raw image of 1024.00 MiB...
> Formatting '/gnu/store/4yp6s4jlq2frb2jqzd7q3kp99vzmnpm5-disk-image', fmt=raw 
> size=1073741824
> Could not access KVM kernel module: Permission denied
> qemu-system-x86_64: failed to initialize KVM: Permission denied

This indicates that /dev/kvm on hydra.gnunet.org has permissions that
prevent the guix build user from accessing it.

I raised this issue long ago (2015) on the guix-sysadmin mailing list.
In that message, I noted that on hydra.gnunet.org, /dev/kvm has mode
0600 and is owned by root, which caused our disk image derivations to
fail.

At the time, Ludovic chmod'd /dev/kvm, and mentioned that he had tried
to make this persistent via /etc/udev/rules.d/70-persistent-net.rules,
but that for some reason that didn't seem to work, possibly because it
was being overridden by another rule or startup script.  As far as I
know, we never implemented a proper fix.

Ludovic, for now, can you chmod it again?

 Thanks,
   Mark

Re: Guix & IPFS

[Pierre Neidhardt]

I found a solution to package IPFS without gx, so go-ipfs has just hit master!

I'll put the "gx-download" on hold since it's not used for anything else at the 
moment.

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: 01/01: hydra: Increase image sizes for USB image and Flash image.

[Leo Famulari]

On Sun, Dec 09, 2018 at 05:16:58PM +0100, Ricardo Wurmus wrote:
> 
> Ludovic Courtès  writes:
> 
> > One problem with the installation OS is that it’s pulling ALSA and all
> > sorts of sound-related libraries (libsamplerate, etc.), which clearly is
> > unnecessary in the installation image.  That comes from the alsa-utils
> > udev rules.  We could remove those udev rules, but since they’re in
> > %base-services, I chose not to do that to avoid breaking everyone’s
> > config.
> 
> I’m in favour of moving them elsewhere, such as %desktop-services.

Or we could just remove the sound services from the installation image.


signature.asc
Description: PGP signature

Re: Golang programs keeping references [gnu: go: Update default to 1.11.]

[Pierre Neidhardt]

I've investigated the possible solutions to avoid including the paths into the
binaries.

I've found this:

https://github.com/golang/go/issues/16860

It's still unresolved and only planned for Go 1.13.

In the meantime, I've played with the -trimpath option to see if I could get
something out of it.

I've added this to Go's (build) function:

--8<---cut here---start->8---
  "-asmflags=all=-trimpath=/gnu/store"
  "-gcflags=all=-trimpath=/gnu/store"
--8<---cut here---end--->8---

The resulting binary is indeed trimmed, but that's not enough: it seems that
Guix detects the remaining part of the path as a store item and includes it in
the list of requisites.  Is this really how Guix works?  It does not need the
full path?

Go supports only one call to -trimpath, so we can't even set this to the Go
inputs.

Regarding Boyer-Moore over the binary, we would have to apply the changes for
all recursive Go libraries.  Now is there a reliable way to detect what's a Go
library and what is not?  We don't want to patch non-Go libraries, right?
(Let's not forget that there is CGo...)

If we can't think of a way to detect a Go library, Boyer-Moore does not seem to
be a solution either.  And we might have to wait for Go 1.13...

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

Hi!

> I’ve just uploaded these to
> :
>
>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig

Great!

> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
> bootstrap.scm refers to a different version of
> linux-libre-headers-stripped so it should be the only one whose hash
> needs to be changed.

I don't see that...and the hash matches.

> Thanks, and let me know if we need anything else!

See attached patch.  Push to my gitlab core-updates-next, not to
savannah yet.

janneke

>From 3cc2f27208de71f9dd0b37593db974b7c1a305f3 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen 
Date: Sun, 9 Dec 2018 19:05:45 +0100
Subject: [PATCH] bootstrap: Switch to official bootstrap urls.

* gnu/packages/bootstrap.scm (%bootstrap-linux-libre-headers): Switch to
official bootstrap urls.
(%bootstrap-mescc-tools): Likewise.
(%bootstrap-mes): Likewise.
---
 gnu/packages/bootstrap.scm | 41 +-
 1 file changed, 14 insertions(+), 27 deletions(-)

diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm
index a835cbc206..eac729f785 100644
--- a/gnu/packages/bootstrap.scm
+++ b/gnu/packages/bootstrap.scm
@@ -405,17 +405,10 @@ $out/bin/guile --version~%"
(lambda (system)
  (origin
(method url-fetch)
-   (uri (match system
-  ("i686-linux"
-   (string-append
-"http://lilypond.org/janneke/guix/20181124/";
-"linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz"))
-  ("x86_64-linux"
-   (string-append
-"http://lilypond.org/janneke/guix/20181124/";
-"linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz"))
-  (_ (error "linux-libre-headers-bootstrap: system not supported:"
-(%current-system)
+   (uri (map (cute string-append <>
+   "/i686-linux/20181020/"
+   "linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz")
+ %bootstrap-base-urls))
(sha256
 (base32
  "0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s"
@@ -656,15 +649,11 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
 ,(bootstrap-origin
   (origin
 (method url-fetch)
-(uri (string-append
-  "http://lilypond.org/janneke/guix/20181124/";
-  (match (%current-system)
-("i686-linux"
- "mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz")
-("x86_64-linux"
- "mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz")
-(_ (error "bootstrap-mescc-tools: system not supported:"
-  (%current-system))
+(uri (map
+  (cute string-append <>
+"/i686-linux/20181020/"
+"mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz")
+  %bootstrap-base-urls))
 (sha256
  (base32
   "11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz")))
@@ -708,13 +697,11 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
 ,(bootstrap-origin
   (origin
 (method url-fetch)
-(uri (string-append
-  "http://lilypond.org/janneke/guix/20181124/";
-  (match (%current-system)
-("i686-linux"
- "mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz")
-("x86_64-linux"
- "mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz"
+(uri (map
+  (cute string-append <>
+"/i686-linux/20181020/"
+"mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz")
+ %bootstrap-base-urls))
 (sha256
  (match (%current-system)
((or "i686-linux" "x86_64-linux")
-- 
2.19.2

Re: 01/01: hydra: Increase image sizes for USB image and Flash image.

[Ricardo Wurmus]

Ludovic Courtès  writes:

> One problem with the installation OS is that it’s pulling ALSA and all
> sorts of sound-related libraries (libsamplerate, etc.), which clearly is
> unnecessary in the installation image.  That comes from the alsa-utils
> udev rules.  We could remove those udev rules, but since they’re in
> %base-services, I chose not to do that to avoid breaking everyone’s
> config.

I’m in favour of moving them elsewhere, such as %desktop-services.

-- 
Ricardo

CDN performance

[Ludovic Courtès]

Hello Chris,

Chris Marusich  skribis:

> Regarding DNS, it would be nice if we could use an official GNU
> subdomain.  If we can't use a GNU subdomain, we should at least make
> sure we have some kind of DNS auto-renewal set up so that nobody can
> poach our domain names.  And the operators should take appropriate
> precautions when sharing any credentials used for managing it all.

Agreed.

Regarding the GNU sub-domain, as I replied to Meiyo, I’m in favor of it,
all we need is someone to champion setting it up.

> Regarding CDNs, I definitely think it's worth a try!  Even Debian is
> using CloudFront (cloudfront.debian.net).  In fact, email correspondence
> suggests that as of 2013, Amazon may even have been paying for it!
>
> https://lists.debian.org/debian-cloud/2013/05/msg00071.html

(Note that debian.net is not Debian, and “there’s no cloud, only other
people’s computer” as the FSFE puts it.)

> Although I don't doubt that a CDN will perform better than what we have
> now, I do think it would be good to measure the performance so that we
> know for sure the money spent is actually providing a benefit.  It would
> be nice to have some data before and after to measure how availability
> and performance have changed.  Apart from anecdotes, what data do we
> have to determine whether performance has improved after introducing a
> CDN?  For example, the following information could be useful:
>
>   * Network load on the origin server(s)
>   * Clients' latency to (the addresses pointed to by) ci.guix.info
>   * Clients' throughput while downloading substitutes from ci.guix.info

Note that performance is one aspect; another one is availability (we’ve
seen that with the recent hydra.gnu.org outage!).  We know we’ll always
win in terms of availability by having a CDN in front of our servers.

That said, measuring performance is very useful and it’s great that we
can benefit from your expertise here!

> Here, it took 0.459667 - 0.254210 = 0.205457 seconds (about 205 ms) to
> establish the TCP connection after the DNS lookup.  The average
> throughput was 1924285 bytes per second (about 40 megabits per second,
> where 1 megabit = 10^6 bits).  It seems my connection to berlin is
> already pretty good!

Indeed.  The bandwidth problem on berlin is when you’re the first to
download a nar and it’s not been cached by nginx yet.  In that case, you
get very low bandwidth (like 10 times less than when the item is cached
by nginx.)  I’ve looked into it, went as far as strace’ing nginx, but
couldn’t find the reason of this.

Do you any idea?

> Establishing the TCP connection took about 21 ms (which matches the mtr
> output), and the throughput was about 79 megabits per second.  (On this
> machine, 100 Mbps is the current link speed, according to dmesg output.)
> This means that in my case, when using CloudFront the latency is 10x
> lower, and the throughput (for a cache hit) is 2x higher, than using
> berlin.guixsd.org directly!

Impressive.

> It would be interesting to see what the performance is for others.

I’ve tried this from home (in France, with FTTH):

--8<---cut here---start->8---
$ measure_get 
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
  2>/dev/null
url_effective: 
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip: 141.80.181.40
remote_port: 443
size_download: 69899433 B
speed_download: 1522001.000 B/s
time_appconnect: 0.178892 s
time_connect: 0.049649 s
time_namelookup: 0.000422 s
time_pretransfer: 0.178934 s
time_redirect: 0.00 s
time_starttransfer: 0.278312 s
time_total: 45.926021 s
$ measure_get 
https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
 2>/dev/null
url_effective: 
https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip: 2600:9000:2116:6e00:c:49d4:5a80:93a1
remote_port: 443
size_download: 69899433 B
speed_download: 20803402.000 B/s
time_appconnect: 0.552008 s
time_connect: 0.482477 s
time_namelookup: 0.467598 s
time_pretransfer: 0.552157 s
time_redirect: 0.00 s
time_starttransfer: 0.735758 s
time_total: 3.360500 s
--8<---cut here---end--->8---

Wall-clock time is less than a tenth; woow.

Thanks for sharing your insight and scripts!

Ludo’.

Re: GuixSD on librem phone?

[Divan Santana]

swedebu...@riseup.net writes:

> I would like to know if there is any interest in this?

I'd be interested. I bought the librem 5.

Hope it turns out well.

Re: Building Bash with Geesh

[Timothy Sample]

Hi Jan,

Jan Nieuwenhuizen  writes:

> Timothy Sample writes:
>
> Hello,
>
> Attached are two small patches that allow me to run the Gash test suite
> with Geesh, like so
>
> PATH=$PATH:bin SHELL='../geesh/pre-inst-env geesh' ./check.sh 
>
> assuming that both projects live in the same parent directory.
>
> Geesh does much better than Gash, it only fails 11/126 tests.
>
> Moreover, all POSIX tests pass!  The ones that fail involve `echo -e'
> and Bash-isms, notably substitutions like ${var/foo} (and all %, %%, #,
> ##, ...etc).
>
> What a great job!

Thanks!  I had a lot of help from the Oil shell test suite that we
talked about before.  Right now I am working on sed, but integrating the
Oil tests into Gash is something that I want to do soon.

With respect to Bash-isms, I’ve been deliberately avoiding them.  POSIX
seemed complicated enough for a first go!  The substring operations
(“%%”, etc.) are all POSIX, but Geesh raises a “not implemented” error
whenever it sees them.  :)

> Interestingly, Gash passes all tests that Geesh fails, and Gash fails
> about 20 other tests.

What an interesting coincidence!

> It looks like that once we manage to combine Geesh and Gash, stealing
> eachothers code or otherwise, we have a shell that can replace bash and
> coreutils&co in the initial bootstrap.

I am still of the opinion that Geesh’s parser is its strongest asset
(hint, hint).  The back-end code is all pretty ad-hoc.

> Greetings,
> janneke
>
> Also put up on: https://gitlab.com/janneke/geesh
>
> From 0aca7e71517b63fc3f67f3a72757f69d7a91158a Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen 
> Date: Sun, 9 Dec 2018 07:35:18 +0100
> Subject: [PATCH 1/2] guix: Use getcwd instead of hard-coded directory.
>
> * guix.scm (make-select): Use getcwd instead of hard-coded directory.
> ---
>  guix.scm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/guix.scm b/guix.scm
> index 04973a9..3b647dd 100644
> --- a/guix.scm
> +++ b/guix.scm
> @@ -24,7 +24,7 @@
>version)))
>  
>  (define (make-select)
> -  (let* ((directory (repository-discover "/home/samplet/code/geesh"))
> +  (let* ((directory (repository-discover (getcwd)))

Definitely didn’t mean to commit that!  :p

>   (repository (repository-open directory))
>   (oid (reference-target (repository-head repository)))
>   (commit (commit-lookup repository oid))
> -- 
> 2.19.1
>
> From 52d3b6ea3c47a84f58e06e0d09db5ce8f9cf383e Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen 
> Date: Sun, 9 Dec 2018 07:37:22 +0100
> Subject: [PATCH 2/2] Support -e, -x in geesh script.
>
> * scripts/geesh.in (options-spec): Support -e, -x.
> ---
>  scripts/geesh.in | 11 ++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/geesh.in b/scripts/geesh.in
> index fa8001f..e7a0051 100644
> --- a/scripts/geesh.in
> +++ b/scripts/geesh.in
> @@ -24,17 +24,26 @@
>  (set! %load-compiled-path (cons "@GODIR@" %load-compiled-path))
>  
>  (use-modules (geesh repl)
> + (geesh environment)
>   (ice-9 getopt-long))
>  
>  (define options-spec
>'((command (single-char #\c) (value #t))
> -(stdin (single-char #\s
> +(errexit (single-char #\e))
> +(stdin (single-char #\s))
> +(xtrace (single-char #\x
>  
>  (let* ((options (getopt-long (program-arguments) options-spec
>   #:stop-at-first-non-option #t))
> (command (option-ref options 'command #f))
> +   (errexit? (option-ref options 'errexit #f))
> (stdin (option-ref options 'stdin #f))
> +   (xtrace? (option-ref options 'xtrace #f))
> (args (option-ref options '() '(
> +  (when errexit?
> +(setopt! 'errexit #t))
> +  (when xtrace?
> +(setopt! 'xtrace #t))
>(cond
> ((and command stdin)
>  (format (current-error-port)


-- Tim

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hello!

l...@gnu.org (Ludovic Courtès) skribis:

> So, on x86_64-linux, here’s what I got:
>
> ludo@ribbon ~/src/guix/+core-updates-next$ ./pre-inst-env guix build 
> bootstrap-tarballs
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0
> ludo@ribbon ~/src/guix/+core-updates-next$ (cd 
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0; for i in * 
> ; do echo $i && guix hash $i ; done )
> guile-static-stripped-2.2.4-x86_64-linux.tar.xz
> 06ygv0b3dpvy93jji5yl4hdz17w4hiqm64m7ypf6g2vi4nfn9xl9
> linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz
> 0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
> mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz
> 11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
> mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz
> 0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
> static-binaries-0-x86_64-linux.tar.xz
> 0bqyjdbas67wgfwv4rcmr5a5b31l8kk8z6gsidxay7liih5rp5hn
> ludo@ribbon ~/src/guix/+core-updates-next$ git describe
> v0.15.0-3121-g4ae7dc7b9a

I’ve just uploaded these to
:

  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
  mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
  mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
  mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
  mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig

Could you adjust bootstrap.scm to refer to this URL?  Currently I see
bootstrap.scm refers to a different version of
linux-libre-headers-stripped so it should be the only one whose hash
needs to be changed.

Thanks, and let me know if we need anything else!

Ludo’.


signature.asc
Description: PGP signature

Re: Installer and luks support.

[Mathieu Othacehe]

Hi,

> Good point.  Perhaps a simple hack would be to detect that /mnt is
> already an overlayfs mount and in that case skip the “herd start
> cow-store /mnt” step?  Would that be enough?

Nope sadly, it won't work because keeping the overlayfs prevent
from closing the luks mapping which will conflict with an eventual
different partitioning.

So unless I'm wrong, that leaves two options:

1. Find a way to close/umount everything but that sounds complex.
2. Like evoked with Gábor, allow the user to resume the installer but
remove the partitioning step and the other steps that will conflict with
this intermediate, wobbly state.

Uppon consideration, I'm not fond of option 2. The ideal would really
be to find a way to umount the overlay, close the luks mapping and allow
the user to resume the installer from anywhere.

Mathieu

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen  skribis:

> Jan Nieuwenhuizen writes:
>
>>>   
>>> 
>>>
>>> Could you share yours so we can compare?
>>
>> Mine is here
>>
>> 
>> http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>>
>> ...so it looks like we're walking a pretty big round , for --rounds=2 ;-)
>
> Diffoscope output is huge, here is the file diff list (manually edited)

Here’s an excerpt of the differences in the .go files:

--8<---cut here---start->8---
$ diffoscope --exclude-directory-metadata janneke ludo
--- janneke
+++ ludo
├── lib
│ ├── guile
│ │ ├── 2.2
│ │ │ ├── ccache
│ │ │ │ ├── language
│ │ │ │ │ ├── tree-il
│ │ │ │ │ │ ├── effects.go
│ │ │ │ │ │ │ ├── 
/gnu/store/02iklp4swqs0ipxhg5x9b2shmj6b30h1-binutils-2.31.1/bin/readelf --wide 
--decompress --hex-dump=.rodata {}
│ │ │ │ │ │ │ │ @@ -473,38 +473,38 @@
│ │ │ │ │ │ │ │0x00016d78 6c2d3137 63356138 34616166 34633833 
l-17c5a84aaf4c83
│ │ │ │ │ │ │ │0x00016d88 30612d31 3862 2700  
0a-18b..'...
│ │ │ │ │ │ │ │0x00016d98 0900  70726f63 2d6e616d 
proc-nam
│ │ │ │ │ │ │ │0x00016da8 6500  2700  
e...'...
│ │ │ │ │ │ │ │0x00016db8 0400  61726773  
args
│ │ │ │ │ │ │ │0x00016dc8 2700  1600  
'...
│ │ │ │ │ │ │ │0x00016dd8 6c2d3638 30623737 35666233 37613436 
l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016de8 332d3136 6263 2700  
3-16bc..'...
│ │ │ │ │ │ │ │ +  0x00016de8 332d3166 6530 2700  
3-1fe0..'...
│ │ │ │ │ │ │ │0x00016df8 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e08 35666233 37613436 332d3136 6264 
5fb37a463-16bd..
│ │ │ │ │ │ │ │ +  0x00016e08 35666233 37613436 332d3166 6531 
5fb37a463-1fe1..
│ │ │ │ │ │ │ │0x00016e18 2700  0700  
'...
│ │ │ │ │ │ │ │0x00016e28 666f726d 616c7300 2700  
formals.'...
│ │ │ │ │ │ │ │0x00016e38 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e48 35666233 37613436 332d3136 6233 
5fb37a463-16b3..
│ │ │ │ │ │ │ │ +  0x00016e48 35666233 37613436 332d3166 6437 
5fb37a463-1fd7..
│ │ │ │ │ │ │ │0x00016e58 2700  1600  
'...
│ │ │ │ │ │ │ │0x00016e68 6c2d3638 30623737 35666233 37613436 
l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016e78 332d3136 6234 2700  
3-16b4..'...
│ │ │ │ │ │ │ │ +  0x00016e78 332d3166 6438 2700  
3-1fd8..'...
│ │ │ │ │ │ │ │0x00016e88 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e98 35666233 37613436 332d3136 6235 
5fb37a463-16b5..
│ │ │ │ │ │ │ │ +  0x00016e98 35666233 37613436 332d3166 6439 
5fb37a463-1fd9..
--8<---cut here---end--->8---

It clearly looks like the psyntax non-deterministic gensym bug described
at .  I guess we’ll have to give it another
go.

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen  skribis:

> Jan Nieuwenhuizen writes:
>
>>>   
>>> 
>>>
>>> Could you share yours so we can compare?
>>
>> Mine is here
>>
>> 
>> http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>>
>> ...so it looks like we're walking a pretty big round , for --rounds=2 ;-)
>
> Diffoscope output is huge, here is the file diff list (manually edited)

Here’s an excerpt of the differences in the .go files:

--8<---cut here---start->8---
$ diffoscope --exclude-directory-metadata janneke ludo
--- janneke
+++ ludo
├── lib
│ ├── guile
│ │ ├── 2.2
│ │ │ ├── ccache
│ │ │ │ ├── language
│ │ │ │ │ ├── tree-il
│ │ │ │ │ │ ├── effects.go
│ │ │ │ │ │ │ ├── 
/gnu/store/02iklp4swqs0ipxhg5x9b2shmj6b30h1-binutils-2.31.1/bin/readelf --wide 
--decompress --hex-dump=.rodata {}
│ │ │ │ │ │ │ │ @@ -473,38 +473,38 @@
│ │ │ │ │ │ │ │0x00016d78 6c2d3137 63356138 34616166 34633833 
l-17c5a84aaf4c83
│ │ │ │ │ │ │ │0x00016d88 30612d31 3862 2700  
0a-18b..'...
│ │ │ │ │ │ │ │0x00016d98 0900  70726f63 2d6e616d 
proc-nam
│ │ │ │ │ │ │ │0x00016da8 6500  2700  
e...'...
│ │ │ │ │ │ │ │0x00016db8 0400  61726773  
args
│ │ │ │ │ │ │ │0x00016dc8 2700  1600  
'...
│ │ │ │ │ │ │ │0x00016dd8 6c2d3638 30623737 35666233 37613436 
l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016de8 332d3136 6263 2700  
3-16bc..'...
│ │ │ │ │ │ │ │ +  0x00016de8 332d3166 6530 2700  
3-1fe0..'...
│ │ │ │ │ │ │ │0x00016df8 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e08 35666233 37613436 332d3136 6264 
5fb37a463-16bd..
│ │ │ │ │ │ │ │ +  0x00016e08 35666233 37613436 332d3166 6531 
5fb37a463-1fe1..
│ │ │ │ │ │ │ │0x00016e18 2700  0700  
'...
│ │ │ │ │ │ │ │0x00016e28 666f726d 616c7300 2700  
formals.'...
│ │ │ │ │ │ │ │0x00016e38 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e48 35666233 37613436 332d3136 6233 
5fb37a463-16b3..
│ │ │ │ │ │ │ │ +  0x00016e48 35666233 37613436 332d3166 6437 
5fb37a463-1fd7..
│ │ │ │ │ │ │ │0x00016e58 2700  1600  
'...
│ │ │ │ │ │ │ │0x00016e68 6c2d3638 30623737 35666233 37613436 
l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016e78 332d3136 6234 2700  
3-16b4..'...
│ │ │ │ │ │ │ │ +  0x00016e78 332d3166 6438 2700  
3-1fd8..'...
│ │ │ │ │ │ │ │0x00016e88 1600  6c2d3638 30623737 
l-680b77
│ │ │ │ │ │ │ │ -  0x00016e98 35666233 37613436 332d3136 6235 
5fb37a463-16b5..
│ │ │ │ │ │ │ │ +  0x00016e98 35666233 37613436 332d3166 6439 
5fb37a463-1fd9..
--8<---cut here---end--->8---

It clearly looks like the psyntax non-deterministic gensym bug described
at .  I guess we’ll have to give it another
go.

Ludo’.

Re: Using a CDN or some other mirror?

[Ludovic Courtès]

Hi Hartmut,

Hartmut Goebel  skribis:

> Am 09.12.2018 um 04:33 schrieb Chris Marusich:
>> Instead, we would be using a CDN as a performance optimization that is
>> transparent to a Guix user.  You seem unsettled by the idea of
>> entrusting any part of substitute delivery to a third party, but
>> concretely what risks do you foresee?
>
> I have serious privacy concerns.
>
> TL;DR: A CDN is a centralized infrastructure, allowing to collect
> information about valuable vulnerability information of almost all
> Guix-users and -systems. This is might become a thread to freedom of
> speech, human rights, democracy and economics. Guix should build on a
> decentralized infrastructure.

Heck it would be ironic to find myself arguing in favor of centralized
commercial services.  So I won’t do that.  :-)

Clearly, I do understand the concerns you list.  As a maintainer, I’m
looking for solutions that can address real problems (availability of
substitutes and bandwidth) while not being a threat to our user’s
privacy and security.

The operator of a substitute server (or caching proxy), in general,
knows which IPs downloaded vulnerable software.  This is the main
threat.

This can be mitigated by talking to nearby mirrors and not just
ci.guix.info, a feature we implemented a year ago (see
),
or by using several substitute servers, or by not using (or not always
using) substitutes.  Few distros have all these options.

We might also be able to somehow balance requests between several CDNs
or mirrors.

But again, medium- to long-term, the goal is to move towards IPFS or
GNUnet/Bittorrent.  IPFS is attractive because it would probably require
no modifications to ‘guix substitutes’ and only minor changes to ‘guix
publish’ since the IPFS daemon has an HTTP interface.

>> Regarding your suggestion to ask universities to host mirrors (really,
>> caching proxies), I think it could be a good idea.  As Leo mentioned,
>> the configuration to set up an NGINX caching proxy of Hydra (or berlin)
>> is freely available in maintenance.git.  Do you think we could convince
>> some universities to host caching proxies that just run an NGINX web
>> server using those configurations?
>
> The difference is: For a traditional "ftp"-mirror, an organization just
> needs to add another source to its existing configuration and administer
> to the save way as all other mirrors. Whereas for a caching proxy they
> need to change the setup of the web-server and learn how to administer
> the cache. This difference might make it difficult to convince
> organizations to mirror.
>
> I could try and ask a few organizations in my area, but I would need
> figures for this.

What would you need to know?  ‘guix weather’ can provide info about
storage size.

Thanks,
Ludo’.

Re: bootstrapping SED-4.5, circular dependency?

[Ludovic Courtès]

Hi!

Jan Nieuwenhuizen  skribis:

> Ludovic Courtès writes:
>
> Hello,
>
>> Jan, could it be that the ‘basename’ and ‘dirname’ implementations in
>> Gash fail this test?  It might be that sed is not needed if you get
>> these right.
>
> Ah, yes!  I have added some test to Gash and this
>
> basename -- /
>
> failed, producing the empty string, where coreutils' basename returns
> `/'.
>
> Gash "simply" (there is som if'ing involved) calls Guile's basename,
> which disagrees here and returns "" for "/".
>
> I'm assuming that this is expected behaviour for Guile?... so I
> special-cased this for Gash' basename.

Looking at
,
I think it’s a bug of Guile’s ‘basename’ (which is not implemented in
terms of libc’s ‘basename’.)

The fix might be as simple as the patch below, WDYT?

In the meantime Gash should work around the bug.

Thanks,
Ludo’.

diff --git a/libguile/filesys.c b/libguile/filesys.c
index e1aeeed1b..c1cd50e21 100644
--- a/libguile/filesys.c
+++ b/libguile/filesys.c
@@ -1602,11 +1602,20 @@ SCM_DEFINE (scm_basename, "basename", 1, 1, 0,
   c_filename = scm_to_utf8_string (filename);
   scm_dynwind_free (c_filename);
 
+  if (strcmp (c_filename, "/") == 0
+  || strcmp (c_filename, "//") == 0)
+/* As per
+   ,
+   "/" and "//" are treated specially.  */
+res = scm_from_utf8_string ("/");
+  else
+{
   c_last_component = last_component (c_filename);
   if (!c_last_component)
 res = filename;
   else
 res = scm_from_utf8_string (c_last_component);
+}
   scm_dynwind_end ();
 
   if (!SCM_UNBNDP (suffix) &&

Re: Installer and luks support.

[Ludovic Courtès]

Hi!

Mathieu Othacehe  skribis:

>> I suppose that if you run “halt” or “reboot”, everything is terminated
>> properly, right?  I’m not sure if you should worry beyond that; in
>> general, it’s reasonable to assume that people will reboot when the
>> installation is over, no?
>
> Yes it is. My concern is when the guix system init command fails because
> let's say the network went down, in that case forcing the user to reboot
> and restart the install process seems painful, don't you think?

Good point.  Perhaps a simple hack would be to detect that /mnt is
already an overlayfs mount and in that case skip the “herd start
cow-store /mnt” step?  Would that be enough?

Ludo’.

Re: 01/01: hydra: Increase image sizes for USB image and Flash image.

[Ludovic Courtès]

guix-comm...@gnu.org skribis:

> commit 07c791c1104db3530eb12c918043fc3b30c093be
> Author: Danny Milosavljevic 
> Date:   Sun Dec 9 00:49:54 2018 +0100
>
> hydra: Increase image sizes for USB image and Flash image.
> 
> * build-aux/hydra/gnu-system.scm (qemu-jobs) : Increase from
> 1024 MiB to 1500 MiB.
> : Increase from 1024 MiB to 1500 MiB.

As discussed earlier on the list, it would be nice™ to see why these
images are so big and what can be done about it.  Really, 1 GiB is
already a lot for such a small image.

In commit 040ae1881952c90dae9478e5cfff6aad0ce950da, I ended up
increasing the image size for the tests, which is not great IMO.  I
looked at ‘guix size’ and couldn’t find any obvious way to improve
things.

One problem with the installation OS is that it’s pulling ALSA and all
sorts of sound-related libraries (libsamplerate, etc.), which clearly is
unnecessary in the installation image.  That comes from the alsa-utils
udev rules.  We could remove those udev rules, but since they’re in
%base-services, I chose not to do that to avoid breaking everyone’s
config.

Food for thought!

Ludo’.

Re: Using a CDN or some other mirror?

[Hartmut Goebel]

Am 09.12.2018 um 04:33 schrieb Chris Marusich:
> Instead, we would be using a CDN as a performance optimization that is
> transparent to a Guix user.  You seem unsettled by the idea of
> entrusting any part of substitute delivery to a third party, but
> concretely what risks do you foresee?

I have serious privacy concerns.

TL;DR: A CDN is a centralized infrastructure, allowing to collect
information about valuable vulnerability information of almost all
Guix-users and -systems. This is might become a thread to freedom of
speech, human rights, democracy and economics. Guix should build on a
decentralized infrastructure.

A distribution provider gets a notion which system is running which
software in which version. In case of guix, the provider even gets the
exact version of the software and all its dependencies. Combining this
with the rise of IPv6, which per default uses the MAC address as part of
the IP address, actually allows identifying a single system.

This information is extremely valuable for all kinds of attackers as it
eases attacking a system a lot. This becomes a thread to

  * to opposition members, dissidents and human rights activists as the
intelligent agencies can target these persons much more precisely,
  * to companies all over the world as many countries do industrial
espionage.

This becomes even worst when using a CDN, since the CDN is a centralized
system: A single CDN provider gains knowledge for almost all systems all
over the world. Which means: this valuable vulnerability information is
collected at a single place. Intelligence agencies might be keen on
getting access to this information and a centralized system makes it
easy for them. And there is evidence they actually collect this
information [*].

This gets even worse when the CDN belongs to one of these companies
compiling personal profiles, like Google, Facebook or Tencent. Amazon
belongs to this group.

I have the strong opinion that Guix should build on a decentralized
infrastructure to support keeping the freedom of speech, democracy and
human rights.

[*] Actually it is known the US-American intelligence agencies have
equipment placed at Verizon to collect all kind of data [1]. One can
reason the same is true for other big providers in the US. The USA has
the FISA act AFAIU enforcing US companies to collaborate in industrial
espionage. In Germany it is known that the BND is extracting high-volume
data at the central internet exchange (DE-CIX) [2]. One can reason such
also happens in other countries, esp. members of the five-eyes, France,
Russia, China, Israel, Saudi Arabia, Iran, Irak, etc.

> Regarding your suggestion to ask universities to host mirrors (really,
> caching proxies), I think it could be a good idea.  As Leo mentioned,
> the configuration to set up an NGINX caching proxy of Hydra (or berlin)
> is freely available in maintenance.git.  Do you think we could convince
> some universities to host caching proxies that just run an NGINX web
> server using those configurations?

The difference is: For a traditional "ftp"-mirror, an organization just
needs to add another source to its existing configuration and administer
to the save way as all other mirrors. Whereas for a caching proxy they
need to change the setup of the web-server and learn how to administer
the cache. This difference might make it difficult to convince
organizations to mirror.

I could try and ask a few organizations in my area, but I would need
figures for this.


[1] https://www.bbc.com/news/world-us-canada-23123964 or search the
internet for e.g. "cia verizon espionage"
[2]
https://www.heise.de/newsticker/meldung/Gerichtsurteil-BND-darf-weiterhin-Internet-Knoten-De-CIX-anzapfen-4061494.html
[3] https://en.wikipedia.org/wiki/Foreign_Intelligence_Surveillance_Act
[4]

-- 
+++hartmut

| Hartmut Goebel|   |
| hart...@goebel-consult.de | www.goebel-consult.de |

Re: GuixSD on eoma68-a20?

[Danny Milosavljevic]

After the change, I get the following on Hydra 
:

---
@ build-started /gnu/store/scnqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv - 
x86_64-linux 
/var/log/guix/drvs/sc//nqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv.bz2
environment variable `PATH' set to 
`/gnu/store/5ka9gmcwcj2919q0cj0wkjng058lwrgq-qemu-minimal-3.0.0/bin:/gnu/store/5s2nib1lrd2101bbrivcl17kjx1mspw6-coreutils-8.30/bin'
creating raw image of 1024.00 MiB...
Formatting '/gnu/store/4yp6s4jlq2frb2jqzd7q3kp99vzmnpm5-disk-image', fmt=raw 
size=1073741824
Could not access KVM kernel module: Permission denied
qemu-system-x86_64: failed to initialize KVM: Permission denied
Backtrace:
   2 (primitive-load "/gnu/store/c2phacd8p7x3g8ysnzrx09jmnxp?")
In ./gnu/build/vm.scm:
152:2  1 (load-in-linux-vm _ #:output _ #:qemu _ #:memory-size _ ?)
In ./guix/build/utils.scm:
616:6  0 (invoke _ . _)

./guix/build/utils.scm:616:6: In procedure invoke:
Throw to key `srfi-34' with args `(#)'.
builder for `/gnu/store/scnqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv' failed 
with exit code 1
@ build-failed /gnu/store/scnqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv - 1 
builder for `/gnu/store/scnqgfc3k4434h3gch22hnh0z8qdbvdb-disk-image.drv' failed 
with exit code 1
---

What does it mean?


pgpDzIgJ5Ck3h.pgp
Description: OpenPGP digital signature

Re: Installer and luks support.

[Gábor Boskovits]

Mathieu Othacehe  ezt írta (időpont: 2018. dec.
9., V, 12:15):
>
>
> Hi Gábor,
>
> > Can't we offer here the option to retry the failed step? i.e. guix system 
> > init
> > in this case? I mean we have everything set up to simply retry. Does
> > that make sense?
>
> Yes that could be a viable option. We could allow the user to launch
> again the install command, but we cannot allow him to go back to the
> installer menu and resume the process wherever.
>

I believe this might be a good choice first. This seems also easier to do.

> Or, we could keep the menu active but remove some critical steps such as
> "partitioning" that require umounting/closing.
>

On the long run it would be nicer to get the dependencies and the resources
figured out, so the we can go back to whatever step we want. I mean if a step
needs a resource available, for example the mount then we can make it available
when entering that step. And when for example we repartiton the drive, then all
later steps depends on that, so we need to redo all the later steps. WDYT?

> What do you think?
>
> Thanks,
>
> Mathieu

Re: Installer and luks support.

[Mathieu Othacehe]

Hi Gábor,

> Can't we offer here the option to retry the failed step? i.e. guix system init
> in this case? I mean we have everything set up to simply retry. Does
> that make sense?

Yes that could be a viable option. We could allow the user to launch
again the install command, but we cannot allow him to go back to the
installer menu and resume the process wherever.

Or, we could keep the menu active but remove some critical steps such as
"partitioning" that require umounting/closing.

What do you think?

Thanks,

Mathieu

Re: Installer and luks support.

[Gábor Boskovits]

Hello Mathieu,

Mathieu Othacehe  ezt írta (időpont: 2018. dec.
9., V, 2:15):
>
>
> Hey Ludo,
>
> > I suppose that if you run “halt” or “reboot”, everything is terminated
> > properly, right?  I’m not sure if you should worry beyond that; in
> > general, it’s reasonable to assume that people will reboot when the
> > installation is over, no?
>
> Yes it is. My concern is when the guix system init command fails because
> let's say the network went down, in that case forcing the user to reboot
> and restart the install process seems painful, don't you think?

Can't we offer here the option to retry the failed step? i.e. guix system init
in this case? I mean we have everything set up to simply retry. Does
that make sense?

>
> Thanks,
>
> Mathieu
>

g_bor

Re: Using a CDN or some other mirror?

[Hartmut Goebel]

Hi Ludo,

Am 07.12.2018 um 15:05 schrieb Ludovic Courtès:
> However, Guix is very different from these: on the build farm, we build
> several new store items per minute, and we aim to distribute them to our
> users. 
> […]  
> For Guix I think a caching proxy […] is a better 
> fit:

In this constellation an old-fashined "ftp" mirror does not work (since
we would need to convince the admins to change the setup, not just to
add another source) and the idea is void.

OTOH, maybe it's worth rethinking the premises. if some region has low
bandwidth if would still be better if one could fetch 90% of the
substitutes from nearby. Thus some daily or twice-a-day rsync would
help. It would also help if some day GuixSD gets some kind of "stable"
(and some "testing") branch and users might subscribe to this, which
would not have so many changes a day.

I'm now finishing this discussion, as it is up to you (the core team) to
decide. I just wanted to share the idea and arguments.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |