Re: GNU Guix 1.2.0rc2 available for testing!

[Maxim Cournoyer]

Hey,

Pierre Neidhardt  writes:

> Ryan Prior  writes:
>
>> I'm glad for the "without-tests" option because when I'm working on
>> packages with a test suite that takes more than a few seconds, I like to
>> make sure that the rest of everything is working before I start running
>> the tests.
>
> Exactly, that was my original thought.
>
>> Another thing I'd like is an option to build a package reusing the state
>> from a previous build. If a package I'm working on takes a minute or
>> longer to build and I'm having some sort of difficulty, it's obnoxious
>> to wait for that to complete again after every cycle. It could be near
>> instantaneous if I could re-use a cached build, which is doable in
>> Docker, Earthly, and other containerized build systems.
>
> You can kind of do that manually now.
> Build with the -K option and when it fails, drop to the temporary
> directly, source the `environment-variables` file, fix the code or calls
> then proceed.
>
> Obviously reusing cached build would be nicer.
>
> Maybe even better would be debugger access to the daemon.  On build
> error we would be dropped in the debugger in the stack frame where it
> errored.  We would fix it, then continue, in the most traditional Lisp
> fashion!

I was thinking about such a thing recently; it would indeed be nice to
be able to debug in the "real" build environment.  Sometimes a test fail
just in the "real" build environment; so being able to debug it in situ
could be useful.

Maxim

Re: GNU Guix 1.2.0rc2 available for testing!

[Ludovic Courtès]

Hi,

zimoun  skribis:

> On Thu, 19 Nov 2020 at 03:24, zimoun  wrote:
>
>>> You can also check the ‘NEWS’ file and the draft blog post at
>>> 
>>> to make sure your favorite feature is mentioned and/or to discuss the
>>> highlights in the blog post.
>>
>> Attached the first rough draft blog post for GuixHPC blog; I forgot my
>> password to push to the INRIA forge… and too late to wait the reset. :-)
>
> Draft post pushed to GuixHPC website.

Awesome.  I’ll take a closer look before we publish but it LGTM.

Thanks a lot!

Ludo’.

Re: GNU Guix 1.2.0rc2 available for testing!

[Pierre Neidhardt]

Ryan Prior  writes:

> I'm glad for the "without-tests" option because when I'm working on
> packages with a test suite that takes more than a few seconds, I like to
> make sure that the rest of everything is working before I start running
> the tests.

Exactly, that was my original thought.

> Another thing I'd like is an option to build a package reusing the state
> from a previous build. If a package I'm working on takes a minute or
> longer to build and I'm having some sort of difficulty, it's obnoxious
> to wait for that to complete again after every cycle. It could be near
> instantaneous if I could re-use a cached build, which is doable in
> Docker, Earthly, and other containerized build systems.

You can kind of do that manually now.
Build with the -K option and when it fails, drop to the temporary
directly, source the `environment-variables` file, fix the code or calls
then proceed.

Obviously reusing cached build would be nicer.

Maybe even better would be debugger access to the daemon.  On build
error we would be dropped in the debugger in the stack frame where it
errored.  We would fix it, then continue, in the most traditional Lisp
fashion!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: GNU Guix 1.2.0rc2 available for testing!

[zimoun]

Hi Ludo,

On Thu, 19 Nov 2020 at 03:24, zimoun  wrote:

>> You can also check the ‘NEWS’ file and the draft blog post at
>> 
>> to make sure your favorite feature is mentioned and/or to discuss the
>> highlights in the blog post.
>
> Attached the first rough draft blog post for GuixHPC blog; I forgot my
> password to push to the INRIA forge… and too late to wait the reset. :-)

Draft post pushed to GuixHPC website.


Cheers,
simon

Re: GNU Guix 1.2.0rc2 available for testing!

[Ryan Prior]

On November 19, 2020, Pierre Neidhardt  wrote:
> [--without-tests] would encourage untested builds if I
> recall correctly

That may have been a concern (I wasn't part of the conversation) but I
don't see how the current implementation could do that. If a normal
"guix build foo" fails, then it's a busted package period.

I'm glad for the "without-tests" option because when I'm working on
packages with a test suite that takes more than a few seconds, I like to
make sure that the rest of everything is working before I start running
the tests.

Another thing I'd like is an option to build a package reusing the state
from a previous build. If a package I'm working on takes a minute or
longer to build and I'm having some sort of difficulty, it's obnoxious
to wait for that to complete again after every cycle. It could be near
instantaneous if I could re-use a cached build, which is doable in
Docker, Earthly, and other containerized build systems.

Re: GNU Guix 1.2.0rc2 available for testing!

[Pierre Neidhardt]

Hi!

zimoun  writes:

> +`--with-c-toolchain`, and `--without-tests`.  Consider this example:

Oh, I didn't notice we had a --without-tests now, good to know!

I remember asking about such an option a while back, but this was
refused on the ground that it would encourage untested builds if I
recall correctly (I can look up the email again if you insist ;p).

So what's the rationale behind this change of position?

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: GNU Guix 1.2.0rc2 available for testing!

[zimoun]

Hi Ludo,

On Wed, 18 Nov 2020 at 19:53, Ludovic Courtès  wrote:

> If everything goes well, let’s release on Monday 23rd.

Birthday release? :-)


> You can also check the ‘NEWS’ file and the draft blog post at
> 
> to make sure your favorite feature is mentioned and/or to discuss the
> highlights in the blog post.

Attached the first rough draft blog post for GuixHPC blog; I forgot my
password to push to the INRIA forge… and too late to wait the reset. :-)


All the best,
simon
--
>From 2823439d49d83e35ae30df51dcd38d6fb86cd5b7 Mon Sep 17 00:00:00 2001
From: Simon Tournier 
Date: Thu, 19 Nov 2020 03:10:55 +0100
Subject: [PATCH] Add draft notes for 1.2.0.

* drafts/hpc-reproducible-research-in-guix-1.2.0.md: New file.
---
 ...hpc-reproducible-research-in-guix-1.2.0.md | 132 ++
 1 file changed, 132 insertions(+)
 create mode 100644 drafts/hpc-reproducible-research-in-guix-1.2.0.md

diff --git a/drafts/hpc-reproducible-research-in-guix-1.2.0.md b/drafts/hpc-reproducible-research-in-guix-1.2.0.md
new file mode 100644
index 000..e3c997a
--- /dev/null
+++ b/drafts/hpc-reproducible-research-in-guix-1.2.0.md
@@ -0,0 +1,132 @@
+title: DRAFT HPC & reproducible research in Guix 1.2.0
+date: 2020-11-19 02:00:00
+author: Simon Tournier
+slug: hpc-reproducible-research-in-guix-1.2.0
+tags: packages, releases
+---
+
+Version 1.2.0 of Guix was [announced
+yesterday](https://guix.gnu.org/blog/2020/gnu-guix-1.2.0-released/).  As
+the announcement points out, some 200 people contributed more than
+10,000 commits since the previous release.  This post focuses on
+important changes for HPC users, admins, and scientists made since
+version 1.1.0 was released in April 2020.
+
+# Reproducible science workflows
+
+We’re giving users more flexibility on the command line, with the
+addition of three [*package transformation
+options*](https://guix.gnu.org/manual/en/html_node/Package-Transformation-Options.html):
+`--with-debug-info` ([always debug in good
+conditions](https://guix.gnu.org/manual/devel/en/html_node/Rebuilding-Debug-Info.html)!),
+`--with-c-toolchain`, and `--without-tests`.  Consider this example:
+
+```
+guix build octave-cli \
+  --with-c-toolchain=fftw=gcc-toolchain@10 \
+  --with-c-toolchain=fftwf=gcc-toolchain@10
+```
+
+The command above builds a variant of the fftw and fftwf packages using
+version 10 of gcc-toolchain instead of the default tool chain, and then builds
+a variant of the GNU Octave command-line interface using them. GNU Octave
+itself is also built with gcc-toolchain@10.
+
+This other example builds the Hardware Locality (hwloc) library and its
+dependents up to intel-mpi-benchmarks with the Clang C compiler:
+
+```
+guix build --with-c-toolchain=hwloc=clang-toolchain \
+   intel-mpi-benchmarks
+```
+
+On the side of long-term archival of all the software Guix packages refer to,
+Guix now serves the file [`sources.json`](http://guix.gnu.org/sources.json)
+that is ingested by [Software Heritage](https://softwareheritage.org) via the
+[nixguix
+loader](https://docs.softwareheritage.org/devel/_modules/swh/loader/package/nixguix.html).
+In addition to the “archival” check of `guix lint` which sends a “save”
+request to Software Heritage for the specified packages.  More packages are
+continuously archived.
+
+The new option `--path` of [`guix
+graph`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-graph.html),
+shows the shortest path between two nodes.  The example below shows the
+shortest path between the packages gmsh and cunit:
+
+```
+guix graph --path gmsh cunit
+gmsh@4.6.0
+metis@5.1.0
+openblas@0.3.9
+cunit@2.1-3
+
+```
+
+Moreover, the command [`guix
+repl`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-repl.html)
+can now be passed a script which ease [package exploratinon in
+Guile](https://hpc.guix.info/blog/2020/01/reproducible-computations-with-guix/)
+especially when dealing with the new Scheme `(guix transformation)` module for
+package transformations.  And the section [“Programming
+Interface”](https://guix.gnu.org/manual/devel/en/html_node/Programming-Interface.html)
+of the *reference manual* has been greatly expounded.
+
+The [`guix
+pack`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-pack.html#Invoking-guix-pack)
+command creates “application bundles” that can be used to deploy software on
+machines that do not run Guix (yet!), such as HPC clusters. Since its
+[inception in
+2017](https://guix.gnu.org/blog/2017/creating-bundles-with-guix-pack/), it has
+seen a number of improvements.  The relocatable packs can be now [faster by
+the addition of the Fakechroot
+engine](https://hpc.guix.info/blog/2020/05/faster-relocatable-packs-with-fakechroot/).
+
+
+# Packages
+
+Here are highlights among the 2,179 packages added and 4,487 packages
+upgraded since the previous release:
+
+ -