Re: Building Bash with Geesh

[Jan Nieuwenhuizen]

Timothy Sample writes:

Hello,

Attached are two small patches that allow me to run the Gash test suite
with Geesh, like so

--8<---cut here---start->8---
PATH=$PATH:bin SHELL='../geesh/pre-inst-env geesh' ./check.sh 
--8<---cut here---end--->8---

assuming that both projects live in the same parent directory.

Geesh does much better than Gash, it only fails 11/126 tests.

Moreover, all POSIX tests pass!  The ones that fail involve `echo -e'
and Bash-isms, notably substitutions like ${var/foo} (and all %, %%, #,
##, ...etc).

What a great job!

Interestingly, Gash passes all tests that Geesh fails, and Gash fails
about 20 other tests.

It looks like that once we manage to combine Geesh and Gash, stealing
eachothers code or otherwise, we have a shell that can replace bash and
coreutils&co in the initial bootstrap.

Greetings,
janneke

Also put up on: https://gitlab.com/janneke/geesh

>From 0aca7e71517b63fc3f67f3a72757f69d7a91158a Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen 
Date: Sun, 9 Dec 2018 07:35:18 +0100
Subject: [PATCH 1/2] guix: Use getcwd instead of hard-coded directory.

* guix.scm (make-select): Use getcwd instead of hard-coded directory.
---
 guix.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guix.scm b/guix.scm
index 04973a9..3b647dd 100644
--- a/guix.scm
+++ b/guix.scm
@@ -24,7 +24,7 @@
   version)))
 
 (define (make-select)
-  (let* ((directory (repository-discover "/home/samplet/code/geesh"))
+  (let* ((directory (repository-discover (getcwd)))
  (repository (repository-open directory))
  (oid (reference-target (repository-head repository)))
  (commit (commit-lookup repository oid))
-- 
2.19.1

>From 52d3b6ea3c47a84f58e06e0d09db5ce8f9cf383e Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen 
Date: Sun, 9 Dec 2018 07:37:22 +0100
Subject: [PATCH 2/2] Support -e, -x in geesh script.

* scripts/geesh.in (options-spec): Support -e, -x.
---
 scripts/geesh.in | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/geesh.in b/scripts/geesh.in
index fa8001f..e7a0051 100644
--- a/scripts/geesh.in
+++ b/scripts/geesh.in
@@ -24,17 +24,26 @@
 (set! %load-compiled-path (cons "@GODIR@" %load-compiled-path))
 
 (use-modules (geesh repl)
+ (geesh environment)
  (ice-9 getopt-long))
 
 (define options-spec
   '((command (single-char #\c) (value #t))
-(stdin (single-char #\s
+(errexit (single-char #\e))
+(stdin (single-char #\s))
+(xtrace (single-char #\x
 
 (let* ((options (getopt-long (program-arguments) options-spec
  #:stop-at-first-non-option #t))
(command (option-ref options 'command #f))
+   (errexit? (option-ref options 'errexit #f))
(stdin (option-ref options 'stdin #f))
+   (xtrace? (option-ref options 'xtrace #f))
(args (option-ref options '() '(
+  (when errexit?
+(setopt! 'errexit #t))
+  (when xtrace?
+(setopt! 'xtrace #t))
   (cond
((and command stdin)
 (format (current-error-port)
-- 
2.19.1

Re: Building Bash with Geesh

[Chris Marusich]

Timothy Sample  writes:

> Over the last year, I built Geesh ,
> which is a simple Shell interpreter written in Guile Scheme.  As of the
> other day, it is capable enough to build Bash!

That's awesome!  I always enjoy reading the emails on this list that are
related to bootstrapping.  It's super important work, and I'm glad you
and the others are making progress!

Thank you,

-- 
Chris


signature.asc
Description: PGP signature

Re: GNU Guix Days before FOSDEM

[Chris Marusich]

Hi Ludo, Pjotr, and others,

l...@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> Pjotr Prins  skribis:
>
>> If you intend to come and/or want to speak please add your name and
>> proposed title to the new page at
>>
>>   https://libreplanet.org/wiki/Group:Guix/FOSDEM2018
>>
>> Alternatively, reply here or mail me privately so we can keep a tally.
>
> The real URL is:
>
>   https://libreplanet.org/wiki/Group:Guix/FOSDEM2019#GNU_Guix_Days
>
> I copied and adjusted the text from last year’s page.
>
> Anyway, feel free to add your name if you would come, and ideas of talks
> or hacking sessions we may have, so we can start planning!

I'll be there - I expect it'll be even more awesome than last year!

-- 
Chris


signature.asc
Description: PGP signature

Re: bootstrapping SED-4.5, circular dependency?

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

Hello,

> Jan, could it be that the ‘basename’ and ‘dirname’ implementations in
> Gash fail this test?  It might be that sed is not needed if you get
> these right.

Ah, yes!  I have added some test to Gash and this

basename -- /

failed, producing the empty string, where coreutils' basename returns
`/'.

Gash "simply" (there is som if'ing involved) calls Guile's basename,
which disagrees here and returns "" for "/".

I'm assuming that this is expected behaviour for Guile?... so I
special-cased this for Gash' basename.

Greetings,
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: bootstrapping SED-4.5, circular dependency?

[Jan Nieuwenhuizen]

Assaf Gordon writes:

Hello Assaf,

> Technically speaking, these kind of dependencies (e.g. needing SED
> to build a package) arise from autotools, not directly from sed's
> build commands.

Ah yes, course.  That is a big difference.

>> Here is what I get:
> [...]
>> ERROR: In procedure scm-error:
>> SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t
>> s//\\1/\n\tq\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\ts//\\1/\n\t
>> q\n\t  }\n\t  /^X\\/\\(\\/\\).*/{\n\ts//\\1/\n\tq\n\t  }\n\t  
>> s/.*/./; q"
>
> The failed sed program is (transcribed from the above error message)
> 
> /^.*\/\([^/][^/]*\)\/*$/{
>   s//\1/
>   q
> }
>   /^X\/\(\/\/\)$/{
>   s//\1/
>   q
>   }
>   /^X\/\(\/\).*/{
>   s//\1/
>   q
>   }
>   s/.*/./; q
> 
>
> It is a very standard sed program, which should (and does) work on every
> existing SED implementation (posix-compliant of course).

Right, now I see.  I really only catered for plain `-e s,,,' commands.

> I would recommend adjusting your sed implementation to support
> these commands, because you'll encounter them in many packages
> that use autotools.

That's sounds like the right thing to do, thanks!

> Digging a bit further, I see this specific sed program originates
> from autoconf, from the M4 code used to implement basename and dirname,
> e.g. here:
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970
>
> So it means every autotools package that uses these macros (and there
> are many) will cause the same issue.
>
> The best solution is likely to implement these sed commands in your
> sed implementation (if you need help in testing I'm happy to try,
> though I know very little about guile).

That is even more helpful!  While I still think we should support the
simple SED script above, there could be a problem with basename/dirname.
Solving that might even sidestep this SED feature.  Many options now!

> As a last resort, I can think of a really ugly hack:
> create a "fake sed" wrapper script in guile,
> that checks if the given SED script is one that you don't implement
> (e.g. the one above). If it is, since we know all it does is
> basename/dirname - just call a real dirname/basename instead of
> using sed's regexes. Otherwise, pass the parameters to the real
> guile-sed.
> Put this guile-sed-wrapper in $PATH, and it might just work...
>
>
> In a completely different approach, since you are trying to bootstrap:
> building OpenBSD's sed binary is very easy (needs just a C compiler
> and works fine on GNU/Linux) - perhaps building it
> would be faster as a first step to get a working sed, then use it
> to build other packages (I can help with that if you want, send me an
> email).

Many thanks, also for these two last resort options!  I'm sure we'll
manage to bootstrap SED.  I'll let you know how it goes.

Greetings,
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Video of Talk: "Everyday Use of GNU Guix"

[Chris Marusich]

Hi Ludo,

Ludovic Courtès  writes:

> Chris, if you want you can push a .md file to guix-artwork.git with a
> link to the video (for example similar to
> )
> and I can do the CVS dance to upload it if you want.  :-)

Sounds good!  I'll let you know once I've done that.

-- 
Chris


signature.asc
Description: PGP signature

Re: Using a CDN or some other mirror?

[Chris Marusich]

Hi everyone,

l...@gnu.org (Ludovic Courtès) writes:

> Ludovic Courtès  skribis:
>
> [...] I’m thinking about using a similar setup, but hosting the mirror
> on some Big Corp CDN or similar.  Chris Marusich came up with a setup
> along these lines a while back:
>
>   https://lists.gnu.org/archive/html/guix-devel/2016-03/msg00312.html
>
> Compared to Chris’s setup, given that ‘guix publish’ now provides
> ‘Cache-Control’ headers (that was not the case back then, see
> ),
> caching in the proxy should Just Work.
>
> I would like us to set up such a mirror for berlin and then have
> ci.guix.info point to that.  The project should be able to pay the
> hosting fees.
>
> Thoughts?

Regarding DNS, it would be nice if we could use an official GNU
subdomain.  If we can't use a GNU subdomain, we should at least make
sure we have some kind of DNS auto-renewal set up so that nobody can
poach our domain names.  And the operators should take appropriate
precautions when sharing any credentials used for managing it all.

Regarding CDNs, I definitely think it's worth a try!  Even Debian is
using CloudFront (cloudfront.debian.net).  In fact, email correspondence
suggests that as of 2013, Amazon may even have been paying for it!

https://lists.debian.org/debian-cloud/2013/05/msg00071.html

I wonder if Amazon would be willing to pay for our CloudFront
distribution if we asked them nicely?

In any case, before deciding to use Amazon CloudFront for ci.guix.info,
it would be prudent to estimate the cost.  CloudFront, like most Amazon
AWS services, is a "pay for what you use" model.  The pricing is here:

https://aws.amazon.com/cloudfront/pricing

To accurately estimate the cost, we need to know how many requests we
expect to receive, and how many bytes we expect to transfer out, during
a single month.  Do we have information like this for berlin today?

Although I don't doubt that a CDN will perform better than what we have
now, I do think it would be good to measure the performance so that we
know for sure the money spent is actually providing a benefit.  It would
be nice to have some data before and after to measure how availability
and performance have changed.  Apart from anecdotes, what data do we
have to determine whether performance has improved after introducing a
CDN?  For example, the following information could be useful:

  * Network load on the origin server(s)
  * Clients' latency to (the addresses pointed to by) ci.guix.info
  * Clients' throughput while downloading substitutes from ci.guix.info

We don't log or collect client metrics, and that's fine.  It could be
useful to add code to Guix to measure things like this when the user
asks to do so, but perhaps it isn't necessary.  It may be good enough if
people just volunteer to manually gather some information and share it.
For example, you can define a shell function like this:

--8<---cut here---start->8---
measure_get () {
curl -L \
 -o /dev/null \
 -w "url_effective: %{url_effective}\\n\
http_code: %{http_code}\\n\
num_connects: %{num_connects}\\n\
num_redirects: %{num_redirects}\\n\
remote_ip: %{remote_ip}\\n\
remote_port: %{remote_port}\\n\
size_download: %{size_download} B\\n\
speed_download: %{speed_download} B/s\\n\
time_appconnect: %{time_appconnect} s\\n\
time_connect: %{time_connect} s\\n\
time_namelookup: %{time_namelookup} s\\n\
time_pretransfer: %{time_pretransfer} s\\n\
time_redirect: %{time_redirect} s\\n\
time_starttransfer: %{time_starttransfer} s\\n\
time_total: %{time_total} s\\n" \
"$1"
}
--8<---cut here---end--->8---

See "man curl" for the meaning of each metric.

You can then use this function to measure a substitute download.  Here's
an example in which I download a large substitute (linux-libre) from one
of my machines in Seattle:

--8<---cut here---start->8---
$ measure_get 
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
 2>/dev/null
url_effective: 
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip: 141.80.181.40
remote_port: 443
size_download: 69899433 B
speed_download: 4945831.000 B/s
time_appconnect: 0.885277 s
time_connect: 0.459667 s
time_namelookup: 0.254210 s
time_pretransfer: 0.885478 s
time_redirect: 0.00 s
time_starttransfer: 1.273994 s
time_total: 14.133584 s
$ 
--8<---cut here---end--->8---

Here, it took 0.459667 - 0.254210 = 0.205457 seconds (about 205 ms) to
establish the TCP connection after the DNS lookup.  The average
throughput was 1924285 bytes per second (about 40 megabits per second,
where 1 megabit = 10^6 bits).  It seems my connection to berlin is
already pretty good!

We can get more information about latency by using a tool like mtr:

--8<---cut h

Re: Installer and luks support.

[Mathieu Othacehe]

Hey Ludo,

> I suppose that if you run “halt” or “reboot”, everything is terminated
> properly, right?  I’m not sure if you should worry beyond that; in
> general, it’s reasonable to assume that people will reboot when the
> installation is over, no?

Yes it is. My concern is when the guix system init command fails because
let's say the network went down, in that case forcing the user to reboot
and restart the install process seems painful, don't you think?

Thanks,

Mathieu

Re: GuixSD on eoma68-a20?

[Danny Milosavljevic]

Hi Mark,

On Sat, 08 Dec 2018 15:59:58 -0500
Mark H Weaver  wrote:

> This sounds like two distinct bugs:
> 
> * Regarding the lack of disk space: if I'm not mistaken, as things are
>   currently implemented, we must specify the size of the disk image
>   manually.  I guess we need to increase the size.

Ah, you are right!  https://hydra.gnu.org/build/3194622/log/raw (the usb-image
for i686) also fails - but nobody noticed because we only distribute the iso
image (which determines its size dynamically).

I didn't make the connection that this is a fixed-size guest filesystem before.

> * This error that occurs within QEMU is apparently not being propagated
>   to the outer build script.  That should be fixed.

Yes.

> > I'm now setting up my own build server and trying to get the flash image
> > that way - which is ridiculous.  
> 
> You seem to be assuming that the problem building the disk image is a
> Hydra-specific problem.  Do you have reason to believe so?  I would
> guess that these bugs are in the disk image builder.

Probably!

According to the progress bar it's missing an additional 40% ?  That's... large.

I've increased the size for the time being, but according to the logs
it was at 800 MiB in 2015.


pgpgEFx__y_kh.pgp
Description: OpenPGP digital signature

Re: bug#33676: GuixSD on eoma68-a20?

[Ludovic Courtès]

Hi,

Danny Milosavljevic  skribis:

> On Sat, 08 Dec 2018 18:34:04 +0100
> Ricardo Wurmus  wrote:
>
>> > Guix received one but we have so far be unable to get the GuixSD flash
>> > image because something always breaks on Hydra before it's done (for
>> > example lack of disk space - see 
>> > https://hydra.gnu.org/build/3198097/log/raw .
>> > Note: The latter thing counts as "successful" build.  WTF?).  
>> 
>> Is this now better with ci.guix.info (berlin.guixsd.org)?
>
> No idea.  How to I get the flash-image using ci.guix.info ?
>
> I tried "guix system disk-image --system=armhf-linux gnu/system/install.scm" 
> but
> that builds a lot of stuff from source.  Let's see whether it will be done 
> before
> the sun exploded :)

I’d suggest trying again, and possibly retrying a bit later so that
substitutes have had a chance to be “baked”.

hydra.gnu.org and now ci.guix.info both provide armhf-linux substitutes.
They have much less armv7 CPU power than x86 but still, it shouldn’t be
that bad.

Ludo’.

Re: bootstrapping SED-4.5, circular dependency?

[Ludovic Courtès]

Hello,

Assaf Gordon  skribis:

> Digging a bit further, I see this specific sed program originates
> from autoconf, from the M4 code used to implement basename and dirname,
> e.g. here:
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970

IIUC, ‘configure’ scripts first try to determine whether they have a
working ‘basename’ and ‘dirname’ with:

--8<---cut here---start->8---
if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then
  as_basename=basename
else
  as_basename=false
fi

if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then
  as_dirname=dirname
else
  as_dirname=false
fi
--8<---cut here---end--->8---

Jan, could it be that the ‘basename’ and ‘dirname’ implementations in
Gash fail this test?  It might be that sed is not needed if you get
these right.

Thanks,
Ludo’.

Re: 01/03: build: file-systems: Add luks to %partition-uuid-readers.

[Ludovic Courtès]

Hello Mathieu!

Mathieu Othacehe  skribis:

>> Thus I think it’s probably best to keep LUKS UUID lookup separate from
>> file system lookup.
>>
>> WDYT?  Are there use cases in the installer where this is problematic?
>
> It makes sense and there are no particular need, so I reverted this
> commit and replaced it by 488a0a8d6120ead3499ba7742c5d630ddf936476 which
> exports a new read-luks-partition-uuid procedure.
>
> What do you think of this new approach?

It LGTM.  Thank you!

Ludo’.

Re: Renice the daemon in config.scm

[Ludovic Courtès]

Hi!

swedebu...@riseup.net skribis:

> I would like to renice the daemon and all its children in config.scm.
>
> Is there a way to do this? 

You could get the PID of the main guix-daemon process (not shown by
‘guix processes’):

  sudo herd status guix-daemon

and then “renice -n 19 -p PID”.

Perhaps we could add an option for this directly in
.

Thanks,
Ludo’.

Re: Installer and luks support.

[Ludovic Courtès]

Hello,

Mathieu Othacehe  skribis:

> I discovered an issue that also exist in other partitioning modes, and I
> don't know how to deal with it. The last step of the installer consists
> in:
>
> * Mount partitions on /mnt (depending on the result of partitioning
>   step).
> * Write the system configuration to /mnt/etc/config.scm
> * Start cow-store service
> * Run guix system init /mnt/etc/config.scm /mnt
> * Unmount /mnt
> * Close luks mappings ---> fail, device is busy.
>
> It fails because at this step, the overlayfs created by cow-store is
> still present. To be able to close luks, I have to:
>
> mkdir /remove-store-overlay
> mount --move /gnu/store /remove-store-overlay
> herd stop cow-store
> herd stop guix-daemon
> kill -9 
> umount /remove-store-overlay
>
> Doing so, everything is cleaned-up, and it is possible to restart the
> installer and run it again (in case the previous install failed or so).
>
> Does anyone have a better idea on how to clean-up the overlayfs of
> cow-store so that everything can be closed properly?

I suppose that if you run “halt” or “reboot”, everything is terminated
properly, right?  I’m not sure if you should worry beyond that; in
general, it’s reasonable to assume that people will reboot when the
installation is over, no?

Not really answering the question, but rather trying to see if we can
sidestep it altogether.  ;-)

Ludo’.

Re: Ruby 2.5, pushing to staging?

[Marius Bakke]

Christopher Baines  writes:

> Christopher Baines  writes:
>
>> Christopher Baines  writes:
>>
>>> A new minor version of Ruby has been out for a while, and it would be
>>> good to get Ruby 2.5 in to Guix.
>>>
>>> I've put up a patch here [1], and tried it locally. I've pushed some
>>> fixes to master to make some packages compatible [2], and while I do get
>>> some failures when building the 958 (according to guix refresh -l)
>>> dependant packages, I'm unsure how many of these are down to the Ruby
>>> upgrade.
>>>
>>> 1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32871
>>> 2: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32870
>>>
>>> Anyway, looking at the contributing guide this is possibly too big of a
>>> change to push directly to master, so should I push this to the staging
>>> branch?
>>
>> I've now gone ahead and pushed to staging. Not really sure what happens
>> now will berlin and hydra pick this up automatically, and start
>> building packages?
>
> So, I'm still interested in pushing Ruby 2.5 forward. I've now pushed an
> upgrade from 2.5.1 to 2.5.3 to the staging branch, as that seemed
> sensible.

Thank you for taking care of Ruby :-)

> I can see that ci.guix.info (berlin) has picked this up (I think here
> [1]), but I'm not sure what to do next...?

I can start this branch on Hydra tomorrow or Monday (it's busy
currently).  If there are no regressions, it should be safe to merge.

We need to synchronize the build farms until users have migrated to
Berlin, since many users probably only have Hydra still.

PS: Staging is currently at ~3500 rebuilds[*], which is way more than
the usual ~1200 rebuilds.  If there are other heavy-impact patches
pending that does not significantly increase this number, we might be
able to squeeze them in.

[*] According to "guix refresh -l -e '(@@ (gnu packages build-tools)
meson-for-build)' boost ruby eudev mesa wayland imagemagick openblas nss"


signature.asc
Description: PGP signature

Re: Video of Talk: "Everyday Use of GNU Guix"

[Ludovic Courtès]

Hi!

Chris Marusich  skribis:

> Mark H Weaver  writes:
>
>> I copied the video to audio-video.gnu.org.  It's now available here:
>>
>> https://audio-video.gnu.org/video/misc/2018-11__Everyday_use_of_GNU_Guix__Chris_Marusich__SeaGL.webm
>
> Thank you, Mark.

Thanks indeed, that was fast!

Chris, if you want you can push a .md file to guix-artwork.git with a
link to the video (for example similar to
)
and I can do the CVS dance to upload it if you want.  :-)

> Too true!  The stickers were nice, but actually the reference cards were
> even more popular.  I printed a few dozen copies on paper vellum, and
> most of them were gone when I left the room.  Thank you, Ludo, for
> creating the reference card just in time for my talk!  ;-)

Ah ah, fun!  We should bring some of these at FOSDEM.

Thanks,
Ludo’.

Re: Paris gathering: Sunday dinner, etc.

[Ludovic Courtès]

Hello Guix!

For those who can make it this Sunday in Paris, I propose that we meet
at 7:45PM at this restaurant:

  Le Petit Cambodge
  20 rue Alibert
  75010 Paris

  https://lepetitcambodge.fr/

They have vegetarian options and it looks good to me.  :-)

See you there!

Ludo’.


signature.asc
Description: PGP signature

Re: Paris gathering: Sunday dinner, etc.

[Ludovic Courtès]

Hi!

Ricardo Wurmus  skribis:

> Ludovic Courtès  writes:
>
>> If anyone is around in Paris on the evening of Sunday 9th, I’d be happy
>> to have dinner with them in town!  Please let me know if you’re
>> interested.
>
> I’m sadly unavailable.  On Sunday I’ll still be on the bus.  Will we go
> for dinner on Monday?  (Or should we join the reproducible builds folks
> in the proposed cafe?)

We’ll go for dinner on Monday too, either with the R-B people or
separately, we’ll have to decide (I suspect only some of the folks
coming on Monday are going to the R-B summit, so it might be wise to
have a separate dinner; we’ll see!).

Have a good trip and see you on Monday!

Ludo’.

Re: Video of Talk: "Everyday Use of GNU Guix"

[Chris Marusich]

Hi everyone,

Joshua Branson  writes:

> Thanks for sharing!

My pleasure!  Hopefully it will help more people to learn about Guix.

Ludovic Courtès  writes:

> Nice talk!  I like the example you gave with GNU Hello, which
> illustrates very well how we can put software into the hands of users
> and make software freedom more practical.  Also great that you managed
> to cover all these topics!

Jan Nieuwenhuizen  writes:

> Yes, I second that.  I enjoyed the recorded talk a lot and especially
> liked your Hello world/Hey there edit; showing how free software and
> guix go hand in hand to empower you.

My hope was that by demonstrating various scenarios, or use cases, it
would show viewers more concretely and more convincingly how Guix can
help them.  A good demonstration can be better than an explanation.

On a related note, I'm excited about Laura Lazzati's Outreachy project
to create video documentation, since I'm sure there are many, many more
scenarios that can be effectively demonstrated through short videos!
Laura, I'm cheering you on!  :-)

Mark H Weaver  writes:

> I copied the video to audio-video.gnu.org.  It's now available here:
>
> https://audio-video.gnu.org/video/misc/2018-11__Everyday_use_of_GNU_Guix__Chris_Marusich__SeaGL.webm

Thank you, Mark.  Actually, I submitted a request to have the video
added, so I suppose we can resolve that now:

https://savannah.gnu.org/task/index.php?15102

However, I don't actually know how to resolve it.  Do you?

"Thompson, David"  writes:

> Nice job!  Also, good job bringing swag with you. That's the element I
> always forget about when I give a talk.  People always want stickers!

Too true!  The stickers were nice, but actually the reference cards were
even more popular.  I printed a few dozen copies on paper vellum, and
most of them were gone when I left the room.  Thank you, Ludo, for
creating the reference card just in time for my talk!  ;-)

-- 
Chris


signature.asc
Description: PGP signature

Re: Patchwork + automated checking and testing of patches

[Chris Marusich]

Hi Chris,

Christopher Baines  writes:

> Chris Marusich  writes:
>
>> [...] Why do you think using Jenkins wouldn't be maintainable? [...]
>
> Sure, I've used Jenkins in different contexts, and indeed currently use
> Jenkins, so I've got some experience here.
>
> I'm obviously ignoring all the good parts here, but one source of data
> is Debian. It used to have a package for Jenkins, and you can see some
> of the work/issues here [1] and the thread about it's removal here [2].
>
> One issue in particular to call out is some potential bootstrapping
> issues [3] that seem to have been encountered.
>
> It seems sensible to me to use Guix when doing things related to the
> Guix project, like automated testing of patches (dogfooding, if you know
> the term). Some of the things I've mentioned here lead me to doubt that
> Jenkins will at some point be available through Guix.
>
> So that's something about the maintainability of a Guix package, but
> there's an operational component to this as well. The attack surface it
> offers is maybe larger than sometimes necessary, if you only need a
> read-only web interface for example.
>
> 1:
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;dist=unstable;ordering=normal;repeatmerged=0;src=jenkins
> 2: https://lists.debian.org/debian-java/2016/01/msg00019.html
> 3: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=714596

Wow!  I had no idea Debian had encountered such difficulties with
Jenkins.  And your point about attack surface is also valid - if we can
get by with a read-only web interface, it's much less to worry about.
After reading the links you've provided and hearing your argument, I'm
inclined to agree that avoiding Jenkins is the right call at this time.

Thank you for explaining things!

-- 
Chris


signature.asc
Description: PGP signature

Re: GuixSD on eoma68-a20?

[Mark H Weaver]

Hi Danny,

Danny Milosavljevic  writes:

> On Sat, 8 Dec 2018 17:39:01 +0100
> swedebugia  wrote:
>
>> Could we pre-order some of these owned by the foundation to
>> be used to to hack on this?
>> 
>> See https://www.crowdsupply.com/eoma68/micro-desktop
>
> Guix received one but we have so far be unable to get the GuixSD flash
> image because something always breaks on Hydra before it's done (for
> example lack of disk space - see https://hydra.gnu.org/build/3198097/log/raw .
> Note: The latter thing counts as "successful" build.  WTF?).

This sounds like two distinct bugs:

* Regarding the lack of disk space: if I'm not mistaken, as things are
  currently implemented, we must specify the size of the disk image
  manually.  I guess we need to increase the size.

* This error that occurs within QEMU is apparently not being propagated
  to the outer build script.  That should be fixed.

> I'm now setting up my own build server and trying to get the flash image
> that way - which is ridiculous.

You seem to be assuming that the problem building the disk image is a
Hydra-specific problem.  Do you have reason to believe so?  I would
guess that these bugs are in the disk image builder.

What do you think?

Regards,
  Mark

Re: bug#33676: GuixSD on eoma68-a20?

[Danny Milosavljevic]

Hi Ricardo,

On Sat, 08 Dec 2018 18:34:04 +0100
Ricardo Wurmus  wrote:

> > Guix received one but we have so far be unable to get the GuixSD flash
> > image because something always breaks on Hydra before it's done (for
> > example lack of disk space - see 
> > https://hydra.gnu.org/build/3198097/log/raw .
> > Note: The latter thing counts as "successful" build.  WTF?).  
> 
> Is this now better with ci.guix.info (berlin.guixsd.org)?

No idea.  How to I get the flash-image using ci.guix.info ?

I tried "guix system disk-image --system=armhf-linux gnu/system/install.scm" but
that builds a lot of stuff from source.  Let's see whether it will be done 
before
the sun exploded :)

> > I have the device here in my hand (I tested it using u-boot only - works 
> > fine).  
> 
> Oh, so it works out of the box with GuixSD already?

As I said, I've tested it with u-boot only so far.


pgpNorLwX6iCV.pgp
Description: OpenPGP digital signature

Re: bootstrapping SED-4.5, circular dependency?

[Assaf Gordon]

Hello,

On 2018-12-08 6:46 a.m., Jan Nieuwenhuizen wrote:

Now that we have successfully bootstrapped GuixSD without using Gcc,
Binutils, GNU C lib, I am working to remove other binary bootstrap
seeds; such as SED and GZIP.

I have bootstrapped GNU make and Bash and was looking into building
GZIP, only to find out that it depends on a SED feature that our
bootstrap-gash does not provide.

So, I figured that SED should be built before GZIP, but I found that
SED-4.5 depends on the same SED feature that GZIP needs...apparently
a circular dependency?


Technically speaking, these kind of dependencies (e.g. needing SED
to build a package) arise from autotools, not directly from sed's build 
commands.




Here is what I get:

[...]

ERROR: In procedure scm-error:
SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\ts//\\1/\n\t
q\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\ts//\\1/\n\tq\n\t  }\n\t  
/^X\\/\\(\\/\\).*/{\n\ts//\\1/\n\tq\n\t  }\n\t  s/.*/./; q"


The failed sed program is (transcribed from the above error message)

/^.*\/\([^/][^/]*\)\/*$/{
s//\1/
q
  }
  /^X\/\(\/\/\)$/{
s//\1/
q
  }
  /^X\/\(\/\).*/{
s//\1/
q
  }
  s/.*/./; q


It is a very standard sed program, which should (and does) work on every
existing SED implementation (posix-compliant of course).


The SED throwing this error is the minimal SED from Gash, that I
implemented in GNU Guile.


I would recommend adjusting your sed implementation to support
these commands, because you'll encounter them in many packages
that use autotools.

Do you know whether bootstrapping of GZIP and/or SED is supported? 


It is supported in the sense that almost any old sed implementation
can be used (e.g. you can build GNU sed if you only have BSD sed
or AIX sed or busybox sed).


Can
you advise me on my next step (looking to bootstrap an earlier SED --
which version? or an earlier GZIP -- which version?)


Digging a bit further, I see this specific sed program originates
from autoconf, from the M4 code used to implement basename and dirname,
e.g. here:
https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n922
https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/m4sugar/m4sh.m4#n970

So it means every autotools package that uses these macros (and there
are many) will cause the same issue.

The best solution is likely to implement these sed commands in your
sed implementation (if you need help in testing I'm happy to try,
though I know very little about guile).

Second best would likely be to investigate with autotools developers
how to avoid this. Perhaps have a specific flag to autoconf to skip
generation of these portability m4 macros, since you know guix's 
basename/dirname are good enough ?



As a last resort, I can think of a really ugly hack:
create a "fake sed" wrapper script in guile,
that checks if the given SED script is one that you don't implement
(e.g. the one above). If it is, since we know all it does is 
basename/dirname - just call a real dirname/basename instead of

using sed's regexes. Otherwise, pass the parameters to the real
guile-sed.
Put this guile-sed-wrapper in $PATH, and it might just work...


In a completely different approach, since you are trying to bootstrap:
building OpenBSD's sed binary is very easy (needs just a C compiler and 
works fine on GNU/Linux) - perhaps building it

would be faster as a first step to get a working sed, then use it
to build other packages (I can help with that if you want, send me an 
email).



regards,
 - assaf

Re: Better support for single-user systems

[Taylan Kammer]

Clément Lassieur  writes:

> Hi Taylan,
>
> You don't need to use the root account at all. 
>
> Taylan Kammer  writes:
>
>> Most desktop users have single unix account and are also in control of
>> root.  These users might not want to differentiate between the current
>> guix version of root and their normal user.  They might also not want
>> to differentiate between the packages available to root and the normal
>> user.  As such I would propose the following two improvements:
>>
>> - Allow a system-wide guix installation that's updated with a special
>>   variant of 'guix pull' executed by root
>
> You can use you current user's guix installation for all commands that
> need root's permissions with 'sudo -E', so you can consider that your
> current user's guix account is the system-wide guix account.
>
> For example, 'sudo -E guix system reconfigure config.scm' updates the
> system with your user's guix.

Aha, thanks, I didn't know about this possibility.

>> - Allow direct addition of packages to the system profile to obviate
>>   the need of running a full 'guix system reconfigure' after adding
>>   packages to the system configuration
>
> You don't need this if you use your user's guix installation only.

Hmm, sometimes I prefer to just log in as root with 'sudo -i' when doing
work where I'll need root privileges repeatedly.  When I do that, my
normal user's packages won't be available.  But I guess 'sudo -E bash'
might be a solution.  I'll try that for now. :-)


Taylan

Re: Octave & QtOctave

[Kei Kebreau]

Kei Kebreau  writes:

> Alex Vong  writes:
>
>> Kei Kebreau  writes:
>>
>>> Alex Vong  writes:
>>>
 Kei Kebreau  writes:

> Alex Vong  writes:
>
>> Hello Kei,
>>
>> Kei Kebreau  writes:
>>
>> [...]
>>>
>>> Here are two tentative patches that make the changes we've discussed.
>>> Also, should we make a deprecated-package definition for qtoctave?
>>
>> I think some additional changes related to "(assoc-ref inputs ..."
>> needed to be made. Otherwise, looks good to me! Here is a patch I made
>> earlier but it was not tested, feel free to cherry-pick what is needed:
>>
>> From 2b04caa66c17da257dfb4f4ccb94e8d629b95e53 Mon Sep 17 00:00:00 2001
>> From: Alex Vong 
>> Date: Mon, 3 Dec 2018 03:39:40 +0800
>> Subject: [PATCH] gnu: Rename "octave" to "octave-cli" and "qtoctave" to
>>  "octave".
>>
>> * gnu/packages/maths.scm (octave): Rename to octave-cli.
>> [name]: Change to "octave-cli".
>> (qtoctave): Rename to octave.
>> [name]: Change to "octave".
>> [inherit]: Inherit from octave-cli.
>> [source]: Likewise.
>> [inputs]: Likewise.
>> [native-inputs]: Likewise.
>> [arguments]: Likewise.
>> (flann): Update accordingly.
>> * gnu/packages/engineering.scm (qucs): Likewise.
>> (qucs-s): Likewise.
>> * gnu/packages/machine-learning.scm (shogun): Likewise.
>
> ...
>
>> - ("octave" ,octave)
>> + ("octave-cli" ,octave-cli)
>
> I see the main difference is that you've replace the package's
> associated string to "octave-cli" as well as the name, whereas I've only
> replaced the package name. Should I replace the associated package
> string, too?

 According to the manual "6.7.2 Package Naming", the associated string is
 used for package management commands such as 'guix package' and 'guix
 build'. Therefore, I think we should change them as well, so that the
 users can install the packages using the command
 "guix package -i octave-cli" and "guix package -i octave"
 respectively. What do you think?
>>>
>>> Maybe this is true when manipulating the package definition, but that
>>> doesn't seem to be the case in general. When I run
>>> "./pre-inst-env guix package --show=shogun", for example,
>>> "octave-cli@4.4.1" is listed as a dependency, even though "octave" is
>>> the associated name in shogun's input list.
>>>
>>> To be clear, I've changed the string for octave's and octave-cli's
>>> package name in their respective package definitions, but I haven't
>>> changed the string in the input lists of octave-cli's dependent
>>> packages.
>>>
>>> I'm inclined to follow convention when it comes to this, and other
>>> packages in input lists seem to omit extensions to the base name of the
>>> package in their assoc-lists. For example, ("gettext", gettext-minimal)
>>> and ("python", python-minimal-wrapper) are common inputs for packages.
>>
>> I think you are right! This was a misunderstanding on my part. I thought
>> the association string in the input lists must be the same as the
>> package name, but appraently this is not the case.
>>
>> Take gettext-minimal as an example,
>> its variable name is 'gettext-minimal',
>> its package name is "gettext-minimal",
>> but its input name is "gettext".
>
> Precisely! Unless anyone else has concerns that should be brought to
> light, I'll be committing this within the next 2 days.
>
> Thank you to all involved so far.

I've pushed this to master. Thanks again!


signature.asc
Description: PGP signature

Re: Ruby 2.5, pushing to staging?

[Christopher Baines]

Christopher Baines  writes:

> Christopher Baines  writes:
>
>> A new minor version of Ruby has been out for a while, and it would be
>> good to get Ruby 2.5 in to Guix.
>>
>> I've put up a patch here [1], and tried it locally. I've pushed some
>> fixes to master to make some packages compatible [2], and while I do get
>> some failures when building the 958 (according to guix refresh -l)
>> dependant packages, I'm unsure how many of these are down to the Ruby
>> upgrade.
>>
>> 1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32871
>> 2: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32870
>>
>> Anyway, looking at the contributing guide this is possibly too big of a
>> change to push directly to master, so should I push this to the staging
>> branch?
>
> I've now gone ahead and pushed to staging. Not really sure what happens
> now will berlin and hydra pick this up automatically, and start
> building packages?

So, I'm still interested in pushing Ruby 2.5 forward. I've now pushed an
upgrade from 2.5.1 to 2.5.3 to the staging branch, as that seemed
sensible.

I can see that ci.guix.info (berlin) has picked this up (I think here
[1]), but I'm not sure what to do next...?

Thanks,

Chris

1: http://ci.guix.info/eval/2010


signature.asc
Description: PGP signature

Re: bug#33676: GuixSD on eoma68-a20?

[Ricardo Wurmus]

Hi Danny,

> Guix received one but we have so far be unable to get the GuixSD flash
> image because something always breaks on Hydra before it's done (for
> example lack of disk space - see https://hydra.gnu.org/build/3198097/log/raw .
> Note: The latter thing counts as "successful" build.  WTF?).

Is this now better with ci.guix.info (berlin.guixsd.org)?

> I have the device here in my hand (I tested it using u-boot only - works 
> fine).

Oh, so it works out of the box with GuixSD already?

-- 
Ricardo

Re: GuixSD on eoma68-a20?

[Danny Milosavljevic]

Hi swedebugia,

On Sat, 8 Dec 2018 17:39:01 +0100
swedebugia  wrote:

> Could we pre-order some of these owned by the foundation to
> be used to to hack on this?
> 
> See https://www.crowdsupply.com/eoma68/micro-desktop

Guix received one but we have so far be unable to get the GuixSD flash
image because something always breaks on Hydra before it's done (for
example lack of disk space - see https://hydra.gnu.org/build/3198097/log/raw .
Note: The latter thing counts as "successful" build.  WTF?).

I have the device here in my hand (I tested it using u-boot only - works fine).

I'm now setting up my own build server and trying to get the flash image
that way - which is ridiculous.

Also, be aware that the Allwinner A20 is really really slow.  There are
ARM computers worth using as a normal PC, but this one isn't.  I tried.

(For example the Allwinner R40 is quite okay to use as a PC - see
Sinovoip Banana Pi M2 Ultra)

I like the concept of easily upgradeable computers, though!


pgpG0Sm5ciLWY.pgp
Description: OpenPGP digital signature

GuixSD on eoma68-a20?

[swedebugia]

Hi

I would like to know if there is any interest in this?

It seems to be well on the way to deliver as promised (though the time 
schedule has been updated a couple of times ;-) )


https://www.crowdsupply.com/eoma68/micro-desktop/updates/what-do-1-000-eoma68-a20-pcbs-look-like

Cost 65$
Est. shipping jan 2019.

Could we pre-order some of these owned by the foundation to
be used to to hack on this?

See https://www.crowdsupply.com/eoma68/micro-desktop

--
Cheers
Swedebugia

Re: 02/03: installer: partionment: Add encryption support.

[Mathieu Othacehe]

Hey!

> Woow, really cool!

Thanks for all the kind words ;)

> Nitpicking on words: s/partitionment/partitioning/,
> s/crypted/encrypted/, s/path/file name/ (this one is just a GNU
> convention.)

Ok I'll do a massive renaming.

>
> Really awesome to have a UI to deal with this.  I didn’t expect it to
> land this quickly, thumbs up!

Best to strike while the iron is hot! However, this encryption part is
still not completely stable, in particular during umounting/closing. I
sent a mail earlier today about that.

I also plan to add a new partitioning scheme: "Use the largest
continuous free space". This way, we will have the same partitioning
schemes as debian installer (which I find quite handy).

Thanks for the feedback!

Mathieu

Re: 01/03: build: file-systems: Add luks to %partition-uuid-readers.

[Mathieu Othacehe]

Hey Ludo!

> Thus I think it’s probably best to keep LUKS UUID lookup separate from
> file system lookup.
>
> WDYT?  Are there use cases in the installer where this is problematic?

It makes sense and there are no particular need, so I reverted this
commit and replaced it by 488a0a8d6120ead3499ba7742c5d630ddf936476 which
exports a new read-luks-partition-uuid procedure.

What do you think of this new approach?

Thanks,

Mathieu

bootstrapping SED-4.5, circular dependency?

[Jan Nieuwenhuizen]

Hi!

Now that we have successfully bootstrapped GuixSD without using Gcc,
Binutils, GNU C lib, I am working to remove other binary bootstrap
seeds; such as SED and GZIP.

I have bootstrapped GNU make and Bash and was looking into building
GZIP, only to find out that it depends on a SED feature that our
bootstrap-gash does not provide.

So, I figured that SED should be built before GZIP, but I found that
SED-4.5 depends on the same SED feature that GZIP needs...apparently
a circular dependency?

Here is what I get:
--8<---cut here---start->8---
configure flags: 
("CONFIG_SHELL=/gnu/store/md6pc5frcqnmndgwmnwk3xkmh4bx71jc-bash-mesboot-4.4.23/bin/bash"
 
"SHELL=/gnu/store/md6pc5frcqnmndgwmnwk3xkmh4bx71jc-bash-mesboot-4.4.23/bin/bash"
 "--prefix=/gnu/store/hy9dq7jhmnjsgbaq9rkg59h9ssgqqzhw-sed-mesboot-4.5" 
"--enable-fast-install" "--build=x86_64-unknown-linux-gnu")
Backtrace:
   5 (apply-smob/1 #)
In ice-9/boot-9.scm:
705:2  4 (call-with-prompt _ _ #)
In ice-9/eval.scm:
619:8  3 (_ #(#(#)))
In gash/commands/sed.scm:
   132:30  2 (sed . _)
In ice-9/boot-9.scm:
   222:17  1 (map1 ("/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\ts//\\1/\n?"))
In unknown file:
   0 (scm-error misc-error #f "~A" ("SED: command not supp?") ?)

ERROR: In procedure scm-error:
SED: command not supported: "/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\ts//\\1/\n\t   
 q\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\ts//\\1/\n\tq\n\t  }\n\t  
/^X\\/\\(\\/\\).*/{\n\ts//\\1/\n\tq\n\t  }\n\t  s/.*/./; q"
--8<---cut here---end--->8---

The SED throwing this error is the minimal SED from Gash, that I
implemented in GNU Guile.

Do you know whether bootstrapping of GZIP and/or SED is supported?  Can
you advise me on my next step (looking to bootstrap an earlier SED --
which version? or an earlier GZIP -- which version?)

Thanks!
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 02/03: installer: partionment: Add encryption support.

[Ludovic Courtès]

Hi again!  :-)

guix-comm...@gnu.org skribis:

> commit 14826f39e3965ddc09c0e7e6e29654e832395af4
> Author: Mathieu Othacehe 
> Date:   Fri Dec 7 14:04:25 2018 +0900
>
> installer: partionment: Add encryption support.
> 
> * gnu/installer.scm (set-installer-path): Add cryptsetup.
> * gnu/installer/newt/partition.scm (prompt-luks-passwords): New procedure,
> (run-partioning-page): Add the possibility to set encryption to "On" on a
> partition and choose a label, add a new partition scheme: "Guided - using 
> the
> entire disk with encryption", prompt for encryption passwords before
> proceeding to formating.
> * gnu/installer/parted.scm ()[crypt-label],
> [crypt-password]: New fields,
> (partition-description): add the encryption label,
> (user-partition-description): add an encryption field,
> (auto-partition): add two partitioning schemes: entire-crypted-root and
> entire-crypted-root-home,
> (call-with-luks-key-file): new procedure,
> (user-partition-upper-path): new procedure,
> (luks-format-and-open): new procedure,
> (luks-close): new procedure,
> (format-user-partitions): format and open luks partitions before creating
> file-system.
> (mount-user-partitions): use the path returned by 
> user-partition-upper-path,
> (umount-user-partitions): close the luks partitions,
> (user-partition->file-system): set device field to label for luks 
> partitions
> and to uuid for the rest,
> (user-partition->mapped-device): new procedure,
> (user-partitions->configuration): add mapped-devices field.

Woow, really cool!

Nitpicking on words: s/partitionment/partitioning/,
s/crypted/encrypted/, s/path/file name/ (this one is just a GNU
convention.)

Really awesome to have a UI to deal with this.  I didn’t expect it to
land this quickly, thumbs up!

Ludo’.

Re: 01/03: build: file-systems: Add luks to %partition-uuid-readers.

[Ludovic Courtès]

Hello!

guix-comm...@gnu.org skribis:

> commit fa358c557f130f393d8c7400ffa954f75112d09e
> Author: Mathieu Othacehe 
> Date:   Fri Dec 7 14:03:06 2018 +0900
>
> build: file-systems: Add luks to %partition-uuid-readers.
> 
> * gnu/build/file-systems.scm (%partition-uuid-readers): Add luks reader.

Currently ‘find-partition-by-luks-uuid’ is separate from
‘find-partition-by-uuid’.  This is because when looking for a LUKS
mapped device, we know we’re looking specifically for that and nothing
else (this is used in the implementation of ‘luks-device-mapping’);
likewise, when looking for a “real partition”, we know we’re not looking
for a LUKS mapped device.

Thus I think it’s probably best to keep LUKS UUID lookup separate from
file system lookup.

WDYT?  Are there use cases in the installer where this is problematic?

Ludo’.