Supported architectures

[Efraim Flashner]

On Thu, Oct 06, 2022 at 04:50:22PM +0200, Ludovic Courtès wrote:
> Hello Guix!
> 
> Will Guix’s 10th year be a release year?  I hope so!
> 
> We need to plan and coordinate.  Releases have to be a group effort;
> some of the most important work won’t be coding but coordination.
> Coordination is key.  I don’t think I should be spearheading that
> effort, but I’m happy to be part of it.
> 
> Who’s ready to commit time towards that goal for the coming weeks?
> 
> Here’s a list of things to do to get there:
> 
>   • Get base binaries on all supported architectures in a timely
> fashion, or drop some of the architectures.
> 
> Namely, ‘make assert-binaries-available’ is currently failing.  It
> uses a manifest that encodes what we consider to be the basic
> requirements for each architecture; it’s not demanding for
> aarch64-linux, even less for armhf-linux and i586-gnu—yet we’re not
> meeting these criteria yet.
> 
> We need to look at missing substitutes, address build issues and
> build farm issues that cause them until we get to zero failures.  If
> after some effort we fail to get to zero, then we should consider
> dropping architectures (I’m looking at armhf-linux and i586-gnu
> specifically).
> 
> So, who’s in?  Let’s get our act together!
> 
> Ludo’.

Firstly, I'd like to mention that we, in general, have a minimum system
requirement of 2GB of RAM, and IIRC there aren't a lot of armhf boards
out there which have that much. We do have a difference between building
natively and cross building / building with '--target'.

I'd like to comment on armhf for a moment. My memory is a but rusty, but
I'm pretty sure that in December of 2021 mesa was bumped from 21.2.x to
21.3.x, and at that time it stopped building on/for armhf. I noticed in
May of 2022 (5 months later) and got the build working again. That we
went 5 months without anyone saying anything in bug reports that mesa
wasn't building shows that either everyone who is using it is using
software that doesn't use mesa, or we really don't have any armhf-linux
users. I'm not advocating dropping the architecture, but it does feel
like we're already at a best-effort level with it. As far as the pieces
needed for bootstrapping aarch64 software (go and probably others),
those get built anyway as needed by aarch64, so there's no worry about
losing support for those software bits.

i586-gnu: Do we have a mini guide on how to setup a build environment?
Something like "add the childhurd service and the secrets service, with
these bits and you're all set to go"? I don't mind poking builds from
time to time, but I'm not sure about how to set it up.

aarch64-linux: I tried a while ago to fix a bunch of the failed builds
on ci.guix.gnu.org and I think I made it worse. Right now there are many
build failures and pending builds. I might see about canceling some of
them and then restarting individual builds to try to increase coverage
again.



-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: system/package testing?

[Maxime Devos]

On 06-10-2022 17:51, Simon Josefsson via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi

Is there any infrastructure to run a system-wide test suite, or at least
a per-package test suite?


There's "make check-system", for system services (and implicitly, for 
the software behind those services), and "make check", for individual 
packages.



 I'm not thinking of 'make check' that is run
during build, but of something similar to Debian's debci/autopkgtest to
test that an installed package works as expected (which may include
running the same self-checks as 'make check' would, but on the installed
package).


I do not understand the difference -- what's installed, is just what's 
built previously, on which "make check" is run, no?


If there are some additional tests beyond "make check", they can be run 
from the check phase (and if they require the package to be installed, 
the phase can be moved after 'install', as is done sometimes already for 
some Python packages).



When designed for Guix, it could be even better than debci/autopkgtest,
it could be something the Guix user could invoke on an installed system
to verify that all the installed packages passes self-tests


This is done automatically, during the build.  If there is a concern it 
depends on the kernel version or CPU model, "guix build --check" can be 
used to redo the build.


 -- I miss

that functionality from debci/autopkgtest.

I'm considering packaging libgssglue for Guix and do some Kerberos
related work on other related packages, but having confidence of any
changes without proper testing of the resulting installed package is
fragile.


If you have additional tests for libgssglue, you can submit them 
upstream, or if they are somehow inappropriate for upstream, do them in 
a check phase.



I'm sure other more complex parts of Guix could benefit from
similar testing too.  A system-wide testing of the published QEMU image
would be useful, to gain confidence that we don't introduce regressions
into a future release of the QEMU image.


See: "make check-system".


Maybe this already exists, and if so my question/request should be
converted into a request for publicity around the feature and a pointer
to its documentation. :)


Info: (guix)Running the Test Suite

Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Planning for a release, for real

[Maxime Devos]

On 06-10-2022 16:50, Ludovic Courtès wrote:

Hello Guix!

Will Guix’s 10th year be a release year?  I hope so!

We need to plan and coordinate.  Releases have to be a group effort;
some of the most important work won’t be coding but coordination.
Coordination is key.  I don’t think I should be spearheading that
effort, but I’m happy to be part of it.

Who’s ready to commit time towards that goal for the coming weeks?

Here’s a list of things to do to get there: [...]


I have some unapplied security patches (from before the latest release 
(1.3.0) (!)) (more precisely, some patches that prepare for actually 
being able to write the security patches, once the preparation patches 
are merged, the actual security fixes should be relatively simple); I 
think it would be rather bad for known security fixes to not be applied 
for a whole release, especially given how long release cycles are in Guix.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Planning for a release, for real

[Julien Lepiller]

I'll take care of the cranslations (notifying translators, ensuring string 
freeze is respected, …)

We need to be careful not to start the stsing freeze step too early. Last time 
(or previous?) we started a week before the scheduled release date, but the 
schedule slipped by a few weeks and we had some pressure in the pipeline 
because some patches could not be applied because of string changes.

Let's try to have a better vision on the planning this time :)

Le 6 octobre 2022 16:50:22 GMT+02:00, "Ludovic Courtès"  a écrit :
>Hello Guix!
>
>Will Guix’s 10th year be a release year?  I hope so!
>
>We need to plan and coordinate.  Releases have to be a group effort;
>some of the most important work won’t be coding but coordination.
>Coordination is key.  I don’t think I should be spearheading that
>effort, but I’m happy to be part of it.
>
>Who’s ready to commit time towards that goal for the coming weeks?
>
>Here’s a list of things to do to get there:
>
>  • Merge ‘staging’ (?).  What’s the status of that one, it seemed ready
>a couple of weeks ago, but then I lost track of it.  Marius?
>
>We need a ‘staging’ champion to keep track of what’s left to be
>done, reports progress, pings people, etc.  That person does not
>have to be hacking like crazy, on the contrary!
>
>  • Get base binaries on all supported architectures in a timely
>fashion, or drop some of the architectures.
>
>Namely, ‘make assert-binaries-available’ is currently failing.  It
>uses a manifest that encodes what we consider to be the basic
>requirements for each architecture; it’s not demanding for
>aarch64-linux, even less for armhf-linux and i586-gnu—yet we’re not
>meeting these criteria yet.
>
>We need to look at missing substitutes, address build issues and
>build farm issues that cause them until we get to zero failures.  If
>after some effort we fail to get to zero, then we should consider
>dropping architectures (I’m looking at armhf-linux and i586-gnu
>specifically).
>
>Again we need a champion to keep track of this and ping people so we
>make progress!
>
>  • Address the blockers of , most of
>which are issues in the installer.
>
>  • Freeze strings: enter a period where translatable strings in the
>code and in the manual must not be changed so translators have a
>chance to keep up.  Julien, how would you like to do that?  Weblate
>has given us more flexibility it seems.
>
>  • Publish a release candidate and call for testing of the installer in
>particular.  Fix bugs, loop.
>
>  • Update NEWS (mostly done already!), prepare a blog post listing the
>highlights and linking to the relevant material.  (See
> for inspiration.)
>
>I’d like us to do this with an eye of getting better organized, which
>involves defining roles such as that of “release managers”.
>
>The NixOS folks handle this in a way that I find inspiring, with
>rotating release manager responsibilities, a schedule announced upfront,
>and a detailed description of the process:
>
>  https://github.com/NixOS/nixpkgs/issues/193585
>  https://nixos.github.io/release-wiki/Home.html
>
>We have
>
>but it’s low-level and dates back to a time where release were a
>one-person activity.  Time for a change.
>
>So, who’s in?  Let’s get our act together!
>
>Ludo’.

system/package testing?

[Development of GNU Guix and the GNU System distribution.]

Hi

Is there any infrastructure to run a system-wide test suite, or at least
a per-package test suite?  I'm not thinking of 'make check' that is run
during build, but of something similar to Debian's debci/autopkgtest to
test that an installed package works as expected (which may include
running the same self-checks as 'make check' would, but on the installed
package).

When designed for Guix, it could be even better than debci/autopkgtest,
it could be something the Guix user could invoke on an installed system
to verify that all the installed packages passes self-tests -- I miss
that functionality from debci/autopkgtest.

I'm considering packaging libgssglue for Guix and do some Kerberos
related work on other related packages, but having confidence of any
changes without proper testing of the resulting installed package is
fragile.  I'm sure other more complex parts of Guix could benefit from
similar testing too.  A system-wide testing of the published QEMU image
would be useful, to gain confidence that we don't introduce regressions
into a future release of the QEMU image.

Maybe this already exists, and if so my question/request should be
converted into a request for publicity around the feature and a pointer
to its documentation. :)

/Simon


signature.asc
Description: PGP signature

Re: Progress with automating testing of patches

[Joshua Branson]

Ludovic Courtès  writes:

> Hi,
>
> jbra...@dismail.de skribis:
>
>> I just created a debbugs-guix.el file in debbugs.  The update should be
>> available on elpa by now.
>>
>> https://issues.guix.gnu.org/56987
>
> Nice!  So that’ll be part of the next ‘emacs-debbugs’ package, right?
>

Yes sir!  It provides debbugs-gnu-guix-search 
debbugs-gnu-my-open-bugs functions.  

>
> Ludo’.

Planning for a release, for real

[Ludovic Courtès]

Hello Guix!

Will Guix’s 10th year be a release year?  I hope so!

We need to plan and coordinate.  Releases have to be a group effort;
some of the most important work won’t be coding but coordination.
Coordination is key.  I don’t think I should be spearheading that
effort, but I’m happy to be part of it.

Who’s ready to commit time towards that goal for the coming weeks?

Here’s a list of things to do to get there:

  • Merge ‘staging’ (?).  What’s the status of that one, it seemed ready
a couple of weeks ago, but then I lost track of it.  Marius?

We need a ‘staging’ champion to keep track of what’s left to be
done, reports progress, pings people, etc.  That person does not
have to be hacking like crazy, on the contrary!

  • Get base binaries on all supported architectures in a timely
fashion, or drop some of the architectures.

Namely, ‘make assert-binaries-available’ is currently failing.  It
uses a manifest that encodes what we consider to be the basic
requirements for each architecture; it’s not demanding for
aarch64-linux, even less for armhf-linux and i586-gnu—yet we’re not
meeting these criteria yet.

We need to look at missing substitutes, address build issues and
build farm issues that cause them until we get to zero failures.  If
after some effort we fail to get to zero, then we should consider
dropping architectures (I’m looking at armhf-linux and i586-gnu
specifically).

Again we need a champion to keep track of this and ping people so we
make progress!

  • Address the blockers of , most of
which are issues in the installer.

  • Freeze strings: enter a period where translatable strings in the
code and in the manual must not be changed so translators have a
chance to keep up.  Julien, how would you like to do that?  Weblate
has given us more flexibility it seems.

  • Publish a release candidate and call for testing of the installer in
particular.  Fix bugs, loop.

  • Update NEWS (mostly done already!), prepare a blog post listing the
highlights and linking to the relevant material.  (See
 for inspiration.)

I’d like us to do this with an eye of getting better organized, which
involves defining roles such as that of “release managers”.

The NixOS folks handle this in a way that I find inspiring, with
rotating release manager responsibilities, a schedule announced upfront,
and a detailed description of the process:

  https://github.com/NixOS/nixpkgs/issues/193585
  https://nixos.github.io/release-wiki/Home.html

We have

but it’s low-level and dates back to a time where release were a
one-person activity.  Time for a change.

So, who’s in?  Let’s get our act together!

Ludo’.


signature.asc
Description: PGP signature

Re: Advanced network configuration

[Julien Lepiller]

I guess using debbugs would give other people a chance to have a look at your 
patches and comment, but I'm the only one who can push anyway. If you decide to 
use debbugs, make sure to CC me too.

Le 6 octobre 2022 15:11:30 GMT+02:00, Alexey Abramov  a 
écrit :
>Hi Ludo, Julien
>
>Ludovic Courtès  writes:
>
>> Hi Alexey,
>>
>> (Cc: Julien, author of Guile-Netlink.)
>>
>
>[...]
>
>>
>> I’m sure your improvements to Guile-Netlink would be welcome.
>
>> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
>> expressive as the underlying netlink interface, but clearly we focused
>> on the most common use cases.
>>
>> If you can think of how you’d like to represent these setups in
>> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
>> snippet you showed?), we (or you :-)) can try and implement it.
>
>Yeah, that was my intention =). @Julien Could you tell me how can I
>collaborate? Shall I send patches to you directly or maybe debbugs, or
>guix-patches?
>
>>> 2. Having a router with Guix at home. I have to run multiple services
>>> that provision 'networking' which is not allowed right now. The DHCP
>>> client service is greedy right now and binds to all available
>>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>>> dhcp-client and static configuration at the same time anyway.
>>
>> OK, we could allow users to change the Shepherd service name used by the
>> DHCP client then.
>
>That would indeed help for now. I can prepare yet another patch for
>that.
>
>[...]
>
>> I’m not sure.  IIUC, a “networking target” here could translate to a
>> Shepherd service that depends on all the relevant DHCP and static
>> networking services.  The question the becomes how to express that
>> grouping conveniently.
>
>Yes, I also would like to point out that their must be a way to
>establish a firewall, for example, *before* any network interface is up
>(After=network-pre.target in systemd [1]). And the same thing during the
>shutdown procedure (Before=network-pre.target in systemd).  Applications
>have to be able to gracefully shutdown their network connections.  Is it
>the case right now, I don't know?
>
>I am checking (shepherd services) where `shutdown-services' defined, and
>seems like it just walks across %services hash table. Am I missing
>something?
>
>Footnotes:
>[1]  
>https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target
>
>-- 
>Alexey

Re: emacs-guix (upstream) needs more love: a survey of repositories, homepage and issues

[Ludovic Courtès]

Hi Simon, Théo, Kaelyn,

Simon Streit  skribis:

> Théo Maxime Tyburn  writes:
>> It is here https://github.com/theottm/emacs-guix-clone. There is only
>> one branch.
>>
>>> if you succeed in building the new merged branch please notify me and
>>> I'll try to upstream it to
>>> https://git.savannah.gnu.org/cgit/guix/emacs-guix.git
>>
>> I just sucessfuly built it. You can probably use
>
> I could successfully build your branch.  Unfortunately with geiser's
> recent upgrade from 0.23.2 to 0.26 emacs-geiser broke on my side.
>
> Geiser has since [1] removed all references to company.  The end result
> is, that emacs-geiser will fail to load a REPL while it tries to
> initiate geiser-company.
>
> Here is my modification that fixes the REPL again: [2].
>
> It'd be nice to see this pushed too and merge all the branches that now
> exist with emacs-guix.

[...]

> [1] 
> https://gitlab.com/emacs-geiser/geiser/-/commit/18faa0ba32c9ce751c16960b2a39b3880b523272
> [2] https://git.steel-is-real.com/emacs-guix-clone/log/?h=simons-hack

I’ve now merged your ‘simons-hack’ branch on Savannah.

I’m sorry for dropping the ball here.  I had told Kaelyn that they could
get an account on Savannah and become responsible for the Emacs-Guix
repo.  The offer still holds, and it can even be extended to you Simon
and to Théo!

As you can see I sometimes fail to follow up on issues, so I’d suggest
that you ping me on IRC (where I go by civodul) so we can proceed.

Thank you!

Ludo’.

Re: Advanced network configuration

[Alexey Abramov]

Hi Ludo, Julien

Ludovic Courtès  writes:

> Hi Alexey,
>
> (Cc: Julien, author of Guile-Netlink.)
>

[...]

>
> I’m sure your improvements to Guile-Netlink would be welcome.

> Regarding ‘static-networking’ in Guix, the goal was to allow it to be as
> expressive as the underlying netlink interface, but clearly we focused
> on the most common use cases.
>
> If you can think of how you’d like to represent these setups in
> ‘static-networking’ (perhaps a ‘bonds’ field similar to the netplan YAML
> snippet you showed?), we (or you :-)) can try and implement it.

Yeah, that was my intention =). @Julien Could you tell me how can I
collaborate? Shall I send patches to you directly or maybe debbugs, or
guix-patches?

>> 2. Having a router with Guix at home. I have to run multiple services
>> that provision 'networking' which is not allowed right now. The DHCP
>> client service is greedy right now and binds to all available
>> interfaces. I sent a [1] patch to solve this. However, I cannot define
>> dhcp-client and static configuration at the same time anyway.
>
> OK, we could allow users to change the Shepherd service name used by the
> DHCP client then.

That would indeed help for now. I can prepare yet another patch for
that.

[...]

> I’m not sure.  IIUC, a “networking target” here could translate to a
> Shepherd service that depends on all the relevant DHCP and static
> networking services.  The question the becomes how to express that
> grouping conveniently.

Yes, I also would like to point out that their must be a way to
establish a firewall, for example, *before* any network interface is up
(After=network-pre.target in systemd [1]). And the same thing during the
shutdown procedure (Before=network-pre.target in systemd).  Applications
have to be able to gracefully shutdown their network connections.  Is it
the case right now, I don't know?

I am checking (shepherd services) where `shutdown-services' defined, and
seems like it just walks across %services hash table. Am I missing
something?

Footnotes:
[1]  
https://www.freedesktop.org/software/systemd/man/systemd.special.html#network-pre.target

-- 
Alexey

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hi,

jbra...@dismail.de skribis:

> I just created a debbugs-guix.el file in debbugs.  The update should be
> available on elpa by now.
>
> https://issues.guix.gnu.org/56987

Nice!  So that’ll be part of the next ‘emacs-debbugs’ package, right?

Ludo’.

Re: Progress with automating testing of patches

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> data.guix.gnu.org just tracks the master branch, and doesn't delete any
> data.
>
> data.qa.guix.gnu.org tracks all branches in the main Guix repository,
> plus branches for issues, and regularly deletes revisions where they're
> either old or the branch no longer exists.
>
> There's nothing technically preventing just using one instance for both
> purposes, it's just operationally I thought it would be easier to split
> the concerns across two instances.

Got it, that makes a lot of sense to me.  Thanks!

Ludo’.

Re: Maintainers meeting notes.

[zimoun]

Hi,

Really cool!  Thank you for the initiative and being transparent about
the under-the-hood tasks of the maintainers’ collective.

On mer., 05 oct. 2022 at 22:04, Mathieu Othacehe  wrote:

> Nothing super fancy, but we will try to post a small note here every
> month in case some people might be interested.

I am interested! :-)


Cheers,
simon

debbugs-guix.el helper function

[zimoun]

Hi,

On mer., 05 oct. 2022 at 22:49, jbra...@dismail.de wrote:
> October 1, 2022 1:08 PM, "Ludovic Courtès"  wrote:

>> --8<---cut here---start->8---
>> (defun ludo-jump-to-guix-qa-url ()
>> "Jump to the QA page of the Debbugs issue at point."
>> (interactive)
>> (let ((url (concat "https://qa.guix.gnu.org/issue";
>> (number-to-string (debbugs-gnu-current-id)
>> (browse-url url)))
>> 
>> (define-key debbugs-gnu-mode-map (kbd "C-M-j") 'ludo-jump-to-guix-qa-url)
>> --8<---cut here---end--->8---
>
> Would it make sense to add something like this to debbugs?

In the same spirit, I have:

--8<---cut here---start->8---
(defmacro defun-bug->url (name url &optional docstring)
  "Macro returning yankage #bug URL.

The `interactive' function that the macro returns is then referred by NAME.

Please provide a DOCSTRING."
  (let ((fun (intern (symbol-name name)))
(doc (concat docstring "\n\n"
   (format "Yankable result: `%sNUMBER'." url
`(defun ,fun (number)
   ,doc
(interactive
 (list
  (progn
(when (not (boundp 'debbugs-gnu-bug-number))
  (setq debbugs-gnu-bug-number -2))
(read-string
 (format "Bug number (%s): " debbugs-gnu-bug-number)
 nil nil debbugs-gnu-bug-number
  (let ((str (format "%s%s" ,url number)))
(kill-new str)
(when current-prefix-arg
  (browse-url str))
(message (format "%s killed." str))

(defun-bug->url my/guix-issues "http://issues.guix.gnu.org/issue/";
  "Add URL of bug NUMBER to `kill-ring'.")
(defun-bug->url my/guix-debbugs "https://debbugs.gnu.org/cgi/bugreport.cgi?bug=";
  "Add (old) URL of bug NUMBER to `kill-ring'.")

(defun my/guix-data (package)
  "Add URL of PACKAGE to `kill-ring'.

Yankable result:
`https://data.guix.gnu.org/repository/1/branch/master/package/PACKAGE/output-history'.

With `universal-argument', load URL using `browse-url'."
  (interactive "sPackage: ")
  (let ((url
 (format
  
"https://data.guix.gnu.org/repository/1/branch/master/package/%s/output-history";
 package)))
(kill-new url)
(when current-prefix-arg
  (browse-url url))
(message (format "%s killed." url
--8<---cut here---end--->8---


And because I find Message-ID and public-inbox nice interface, I also
have:

--8<---cut here---start->8---
  (defun my/public-inbox-insert (number)
"TODO"
(interactive "nBug number: ")
(let* ((meta  (car (debbugs-get-status number)))
   (inbox (car (debbugs-get-attribute meta 'package))) ;Probably 
inaccurate for the general case
   (raw   (debbugs-get-attribute meta 'msgid))
   (msgid (replace-regexp-in-string "<\\|>" "" raw)))
  (message "Message-ID: %s from %s." msgid inbox)
  (my/piem-inject-thread-into-maildir msgid inbox)
  (notmuch-command-to-string "new" "--no-hooks")))
--8<---cut here---end--->8---

which can be adapted (by removing the part about emacs-piem, great
package BTW! and the part about emacs-notmuch).

Well, I have in my TODO list to implement the extraction of Message-ID
from Emacs-Debbugs.  But it is not about debbugs.el and instead about
Gnus.  It could be very helpful to have a way to stash to the kill-ring
the Message-ID of one specific message in the thread; and not only the
Message-ID of the first message in that thread.


Cheers,
simon