Confusing overview of mailing lists

[Erik Grun]

Hello, Guix,

For some time now I've been using Guix on a few test machines. I have to 
say, that I really like it. So much so, that I've now finally wanted to 
get involved a bit closer. So I went to this page[1] with the overview 
of all the mailing lists.


Personally I find the links to the mailing list archives on the 
"Contact" page a bit confusing. Because on the first look it appears, as 
if those list are "archived". It looks even more so since not every list 
has the "(Archive)" behind its name.


My suggestion would be, to remove the archive links altogether from that 
page. I mean, one can still get to the archives, when one goes to the 
respective mailing list.


If this topic doesn't fit here, please move it to the correct list (and 
keep me in the CC).


Best regards,

Erik (egnun)

[1] https://guix.gnu.org/en/contact
--
pub   rsa4096/C02662E2370D6F27 2016-05-27 [SCA] [verfällt: 2024-05-01]
  Schl.-Fingerabdruck = 7E0E 8A9E EAEE 075D E689  F32F C026 62E2 370D 6F27
uid  [ ultimativ ] Erik Grun 
uid  [ ultimativ ] Erik Grun 
sub   rsa4096/1DF94895FA78155E 2016-05-27 [E] [verfällt: 2024-05-01]

You are wondering, what this signature means?
Visit *emailselfdefense.fsf.org/en/infographic.html*
to find out how YOU can securely encrypt your e-mails.


OpenPGP_0xC02662E2370D6F27.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Add anchors in HTML documentation

[Christian Miller]

Hello,

it would be a nice feature to have actual anchors in the HTML
documentation.  I mean with that, that there is an anchor symbol which
upon clicking, copies the exact URL to this section in the HTML
documentation.

This is useful, since currently I always need to go to the top of the
documentation (Table of Contents) to grab the URL that jumps to the
corresponding section, which I paste in the #Guix IRC channel as
pointer on how to do specifc things.

Can we maybe even do this not only for whole sections, but also for
things like "console-font-service-type".  For example if someone asks
how to install TTY fonts.

Best Regards
Christian Miller

Ensuring daemon interop, maybe also store layer standardization?

[John Ericson]

On Thu, Nov 16, 2023, at 10:14 AM, Ludovic Courtès wrote:
> My take is that the daemon rewrite in Guix will aim for 100%
> compatibility at the protocol level (in fact part of what’s needed is
> already available as (guix …) modules), probably with just the same
> feature set.  Christopher Baines may have clearer ideas.

Excellent!

I am glad that Florian already emailed about this, I had similar thoughts.

I have long been a proponent of ensuring interoperability in the "store layer" 
between Nix, Guix, Tvix, and anyone else that wants to have an implementation. 
(To me, the daemon is just one possible implementation, an RPC bridge to 
another implementation.) There are some good developments on the Nix side in 
this space I've been meaning to email this mailing list about. Glad the recent 
guix-daemon work and ensuing discussion are finally getting me to do so! Here 
they are:
 • RFC 134 
https://github.com/NixOS/rfcs/blob/master/rfcs/0134-nix-store-layer.md: this is 
the linchpin holding a few things together on the Nix side. Nix gone back and 
forth over the years emphasizing the degree to which the store layer beneath 
the Nix language is self-contained and can be used/understood on its own. With 
that RFC we're finally committing to "show off the store layer" as official 
policy.
 • https://nixos.org/manual/nix/unstable/store/ there is a new top-level 
section of the Nix manual that is commentating the store-layer. Expect it to 
grow rapidly in the coming months as new information is written, and existing 
information that belongs here but is tagging along elsewhere is moved here.
 • Unit tests with external data "golden masters" for serializes for the daemon 
protocol 
https://github.com/NixOS/nix/tree/master/unit-test-data/libstore/worker-protocol,
 and derivation ATerm format 
https://github.com/NixOS/nix/tree/master/unit-test-data/libstore/derivation. 
(Since the Guix fork I've tried to systematize the daemon protocol with 
bidirectional serializers for explicit data types, which the RPC layer uses.) 
The test data is separate files with the explicit intention of making them easy 
to reuse in other implementations :), as opposed to just in-code string/binary 
literals which would be very inconvenient for them.
 • 
https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-derivation-add 
making sure the JSON derivation format is round-tripable with a new command, 
for ease of use and in hopes that we might someday ditch "ATerm" for the 
on-disk format for everyone's benefit.
 • https://github.com/NixOS/nix/pull/8699 C bindings for the Nix store. Perhaps 
this isn't so important now that you all are going for the full rewrite, but I 
saw Maxim's early email about creating guile bindings for a gradual rewrite. 
The C bindings are explicitly made with the idea of making other langauge 
bindings easier (e.g. https://github.com/NixOS/nix/pull/7735, 
https://hackage.haskell.org/package/hercules-ci-cnix-store). Still mentioning 
this in case it is useful as a "Plan B".
Furthermore, I hope that beyond merely ensuring some interop with what we have 
already, we work to further keep things in sync and hopefully converging (where 
was have diverging new features like Blake hashing in Guix and 
content-addressed derivations in Nix) going forward. To that end, I would love 
to set up an implementation-agnostic standard, maybe even an IETF standard (!).

I think while we all have different ideas (and should continue to have 
different ideas!) on what the user-facing package language looks like, we do 
essential agree on what this store interface looks like. It is a "narrow waist" 
which can be implemented numerous different ways, and used in numerous 
different high level packaging paradigms. It is this quality that makes it so 
worthy of standardization.

> I’m interested in hearing how you view content-addressing and its use.
> In Guix there’s one proposal based on ERIS:
> 
>   https://issues.guix.gnu.org/52555#8
> 
> Previously there was an experiment to implement what you described at
> “partial substitution”:
> 
>   https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00080.html

I have been talking a lot to Florian about these things too. Long ago I emailed 
Ludo and some others about the IPFS & Nix work. Since then RFC 133 was 
accepted, https://github.com/NixOS/rfcs/blob/master/rfcs/0133-git-hashing.md, 
which prepares the way for a lot of that, and more recently 
https://github.com/NixOS/nix/pull/9294 (merged) has begun the implementation.

It sounds like this ERIS plan and Tvix's content addressing are fairly similar 
--- use improved content addressing "underneath the hood". The thing I have 
been working on is trying to expose content-addressing all the way into the 
store path for end-to-end trustlessless.

I think for build artifacts, either way is fine. But for source code the 
end-to-end trustlessness is nice for treating things like Software Heritage as 
a substituter of last resort. 

Re: Ensuring daemon interop, maybe also store layer standardization?

[Florian Klink]

On 23-11-16 14:23:29, John Ericson wrote:

On Thu, Nov 16, 2023, at 10:14 AM, Ludovic Courtès wrote:
I have been talking a lot to Florian about these things too. Long ago I
emailed Ludo and some others about the IPFS & Nix work. Since then RFC
133 was accepted,
https://github.com/NixOS/rfcs/blob/master/rfcs/0133-git-hashing.md,
which prepares the way for a lot of that, and more recently
https://github.com/NixOS/nix/pull/9294 (merged) has begun the
implementation.

It sounds like this ERIS plan and Tvix's content addressing are fairly
similar --- use improved content addressing "underneath the hood". The
thing I have been working on is trying to expose content-addressing all
the way into the store path for end-to-end trustlessless.


tvix-castore uses /only/ the BLAKE3 digest, and thanks to its verified
streaming properties, chunking parameters etc don't "bleed" into the
to-be-signed datastructure (and its content hash), but can be delivered
(and verified) out-of-band, without having to fetch the entire data.

That's also one of the main reasons as to why Iroh (The IPFS Rust
"Rewrite") entirely ditched IPLD and all other hash functions in favor
of raw blake3-hashed blobs [1]), and why they, S5 [2] and Tvix uses them
to address blobs.

Among these projects, there's different areas of priority and focus,
but strictly speaking, they're all just different transport protocols
for the same identifier (blake3 digests of raw contents) and could
interop with each other.

For file system structures, tvix-castore also defines an encoding
similar to git trees, but using the blake3 digest of (a slightly more
sane) serialization as an identifier.

ERIS defines its own addressing scheme, using "ERIS capability URNs",
and in addition to the contents, chunksize (two modes) and
convergence secret make up the identifier.
Internally, it constructs its own Merkle Tree using ChaCha20 and
Blake2b-256.
https://issues.guix.gnu.org/52555 uses it with the larger chunk size and
a null convergence secret.


I think for build artifacts, either way is fine. But for source code
the end-to-end trustlessness is nice for treating things like Software
Heritage as a substituter of last resort. (I made
https://docs.softwareheritage.org/devel/swh-web/uri-scheme-api.html#get--api-1-raw-(swhid)-
forto spit out raw git objects for
https://blog.obsidian.systems/software-heritage-bridge/, but the
pipelining latency issues mean something like SWH's "Vault API" is
probably better.)


I hope some day SWH will also support looking up blobs not just by their
SWHID (which is just using a lot of lipstick to describe how git encodes
blobs), but also by these nice new hash functions of the raw contents,
which would allow SWH to be plugged as a "last resort" source directly-
without having to keep carrying along and calculating additional
git-based hashes / SWHIDs to prepare for the eventuality of having to
reach out to there.


That's a big wall of text, but glad it's all out there. Stuff is really
cooking in all our ecosystems these days, and I'm very excited for
where things are going!


Definitely!

--
flokli

[1]: https://github.com/n0-computer/iroh/discussions/707
[2]: https://github.com/n0-computer/iroh/discussions/709

Re: Better support remote deployment

[Development of GNU Guix and the GNU System distribution.]

On Thu, Nov 16 2023, Ludovic Courtès wrote:

> I wonder if this would really be more convenient than SSH’ing
> into the target machine and running the commands right there.

Just from using 'guix deploy' I'll note that 'guix pull' is an expensive
operation, especially when running a custom version---but it would be
more appropriate for the affected parties to speak up here.

Kind regards
Felix

Re: $EDITOR and “guix edit”

[Liliana Marie Prikler]

Am Donnerstag, dem 16.11.2023 um 16:25 +0100 schrieb Ludovic Courtès:
> Liliana Marie Prikler  skribis:
> 
> > Am Donnerstag, dem 02.11.2023 um 10:43 +0100 schrieb Simon
> > Tournier:
> > > Hi,
> > > 
> > > The command “guix edit” returns “+N path/to/file” that is then
> > > passed to $EDITOR.  Therefore $EDITOR needs the command line:
> > > 
> > >     $ $EDITOR +N /path/to/file
> > > 
> > > Well, that is accepted by many $EDITOR, to my knowledge.  At
> > > least, Emacs, Vi or less are fine with it.
> > This appears to be a somewhat archaic convention.
> 
> “Archaic” is one way to describe it; I’d have said “standard”.  :-)
> (Implemented by Vi(m), Emacs, Nano, less, more, etc.)
I said “archaic”, because POSIX specifically says “all options should
be preceded by the '-' character”, with special exceptions for more,
etc. for backwards compatibility.  It's a bit of a pain, because the
argument that used to be standard among all those tools has (in POSIX
at least) been replaced by several dashed ones.

> > > However, some other $EDITOR does not.  I have in mind “kate” or
> > > “VSCode“,
> > > 
> > >     $ kate -l N path/to/file
> > >     $ code --goto path/to/file:N
> 
> It’d be nice to support these as well.  However, how do we know we’re
> dealing with kate or VSCode?  By checking the basename of $EDITOR?
> Kinda ugly and brittle, but probably better than nothing.
Maybe we can check for a guix_editor shell function and invoke that
rather than EDITOR if defined?

Cheers 

Re: $EDITOR and “guix edit”

[Ludovic Courtès]

Liliana Marie Prikler  skribis:

> Am Donnerstag, dem 02.11.2023 um 10:43 +0100 schrieb Simon Tournier:
>> Hi,
>> 
>> The command “guix edit” returns “+N path/to/file” that is then passed
>> to $EDITOR.  Therefore $EDITOR needs the command line:
>> 
>>     $ $EDITOR +N /path/to/file
>> 
>> Well, that is accepted by many $EDITOR, to my knowledge.  At least,
>> Emacs, Vi or less are fine with it.
> This appears to be a somewhat archaic convention.

“Archaic” is one way to describe it; I’d have said “standard”.  :-)
(Implemented by Vi(m), Emacs, Nano, less, more, etc.)

>> However, some other $EDITOR does not.  I have in mind “kate” or
>> “VSCode“,
>> 
>>     $ kate -l N path/to/file
>>     $ code --goto path/to/file:N

It’d be nice to support these as well.  However, how do we know we’re
dealing with kate or VSCode?  By checking the basename of $EDITOR?
Kinda ugly and brittle, but probably better than nothing.

Thanks,
Ludo’.

Re: Better support remote deployment

[Ludovic Courtès]

Hi,

Maxim Cournoyer  skribis:

> Felix Lechner  writes:

[...]

>> How about a 'remote-user-profile'?
>
> 'guix package' is already the command we use to create profiles; maybe
> it could accept a '--remote' argument to operate on a remote machine?
> Or is this not what 'GUIX_DAEMON_SOCKET=ssh://your-target guix package
> -m manifest.scm' can already accomplish? :-)

Doing that confuses ‘guix package’ because it’d try to perform the
effectful part (switching symlinks) locally, even though everything was
built on another machine.  (It’s also slow due to RPC round trips, as
Efraim wrote.)

But yeah, I think we could do something along these lines by isolating
the effectful bits and evaluating them remotely when needed, like we did
to achieve code sharing between ‘guix deploy’ and ‘guix system
reconfigure’.

That said, I wonder if this would really be more convenient than SSH’ing
into the target machine and running the commands right there.  Perhaps
I’m missing something about the use case?

Ludo’.

Re: Upgrading Guix's security team

[Andreas Enge]

Hello,

Am Thu, Nov 16, 2023 at 03:22:42PM +0100 schrieb Ludovic Courtès:
> Yes, we definitely need a rotation here!  I for one have my name there
> but regardless of my interest, I have to admit that I’ve been unable to
> be sufficiently responsive.  It’s time to let new folks take
> responsibility.
> I think we should make this a fixed-term position, to make it easier for
> people to commit to actually being active when needed, with the
> understanding that it’s not a commitment for life.

all this sounds good. Maybe we should also clean up the mailing list.
I am on the list, but not mentioned on the security team site, and will
be happy to be removed. (My being here probably comes from a mismatch
between being interested in "security" and knowing things about "crypto-
graphy", and my inability to act upon concrete situations of security
problems in packages.)

Andreas

Re: Nix Daemon protocol post / Tvix

[Ludovic Courtès]

Hi Florian,

Florian Klink  skribis:

> I stumbled across your post
> https://guix.gnu.org/blog/2023/a-build-daemon-in-guile/.
>
> I'm working on Tvix (https://tvix.dev/), a reimplementation of Nix in
> Rust.

Neat, thanks for reaching out to us!

[...]

> For example, tvix-store is using a content-addressed merkle storage DAG
> (tvix-castore) under the hood, allowing partial substitution and store
> path subtree sharing.
>
> However we can still provide a Nix-compatible view into all this, so can
> synthesize NAR Archives and NARInfo files for a given store path on
> demand, if we want to. We currently use the HTTP Binary cache protocol
> as a store interface for Nix (via `nar-bridge`, which spins up a
> webserver).
>
> At some point, we now also want to implement the daemon protocol - both
> a client and server, to allow talking to Nix more directly - be it a
> "remote store", or just querying the local Nix store for certain
> information. This is so far mostly oriented towards store operations (as
> we didn't do too much work on the Builder interface yet)

All this sounds nice.

> Nevertheless, I think we should collaborate.
>
> Be that:
>  - just a simple exchange of notes about the behaviour of the protocol
>and certain operations
>  - discussions about designing new protocols, ensuring interop between
>tvix-store and guix stores (there's some ideas for P2P substitution)
>  - or even collaboration and work on getting tvix-store (and tvix-build,
>once it's there) to work with a Guile frontend :-)
>
> I think we're sharing a lot of common interest and would like to start
> having these conversations :-)

That’s a good idea!

My take is that the daemon rewrite in Guix will aim for 100%
compatibility at the protocol level (in fact part of what’s needed is
already available as (guix …) modules), probably with just the same
feature set.  Christopher Baines may have clearer ideas.

I’m interested in hearing how you view content-addressing and its use.
In Guix there’s one proposal based on ERIS:

  https://issues.guix.gnu.org/52555#8

Previously there was an experiment to implement what you described at
“partial substitution”:

  https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00080.html

Hot topics! :-)

Thanks,
Ludo’.

Re: Request-For-Comment process: concrete implementation

[Ludovic Courtès]

Hello,

Simon Tournier  skribis:

> This is a proposal for implementing Request-For-Comment process.
> Comment are welcome in #66844 [1]:
>
> 1: https://issues.guix.gnu.org/issue/66844

Thanks for starting the discussion!  I think that getting such a process
in place is key to sustain friction-less development of Guix, giving
everyone a chance to have their voice heard.

Ludo’.

Re: Performance of computing cross derivations

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> When asked by the data service, it seems to take Guix around 3 minutes
> to compute cross derivations for all packages (to a single
> target). Here's a simple script that replicates this:

To understand the cost of computing a package’s derivation, I generally
start looking at caches and memoization:

--8<---cut here---start->8---
$ GUIX_PROFILING="object-cache" guix build gcc-toolchain -d --no-grafts
/gnu/store/iwn6frqqcyw808sgsnjv26dn6rq7mijd-gcc-toolchain-13.2.0.drv
Object Cache:
  fresh caches:19
  lookups:   3667
  hits:  3342 (91.1%)
  cache size: 323 entries
$ GUIX_PROFILING="object-cache" guix build sed -d --no-grafts 
--target=aarch64-linux-gnu
/gnu/store/yxakl87wizwzcqapx4sdkp56652cxb4m-sed-4.8.drv
Object Cache:
  fresh caches:20
  lookups:   5420
  hits:  4919 (90.8%)
  cache size: 500 entries
--8<---cut here---end--->8---

Caches are critical: since we’re dealing with huge package graphs, we
need to make sure we don’t end up computing the same thing several
times.  (You can also add “memoization” to the ‘GUIX_PROFILING’ variable
above.)

One idiom that defeats caching is:

  (define (make-me-a-package x y z)
(package
  …))

Such a procedure returns a fresh package every time it’s called,
preventing caching from happening (because cache entries are compared
with ‘eq?’).  That typically leads to lower hit rates.

Anyway, lots of words to say that I don’t see anything immediately
obvious with cross-compilation, yet I wouldn’t be surprised if some of
these cache-defeating idioms were used because we’ve payed less
attention to this.

An even better thing to start with: compare the timing of ‘guix build -d
--no-grafts $PKG --target=aarch64-linux-gnu’ for all valid values of
$PKG, and investigate those that take the most time.

HTH!

Ludo’.

Re: Divvying up service definitions

[Ludovic Courtès]

Hello!

Maxim Cournoyer  skribis:

>> * Splitting this as gnu/services/dovecot.scm.
>>   We keep it compatible with 'use-service-modules' at the cost of having
>>   a multitude of files under gnu/services, without any logical grouping
>>   (messy).
>
> That's a great initiative!  I agree that multiple 'define-configuration'
> services per file can be a bit messy, having to use prefixes everywhere,
> making the definitions more verbose.
>
> I don't have a strong preference of the caterogization of services, but
> would perhaps prefer the first one (gnu/services/mail/dovecot.scm),
> which could then make it easy to offer some interface as
> gnu/services/mail.scm that'd re-export all that is needed (would that
> work, or reintroduce the same top-level clashes?).

I’m all for “cleanups” as proposed, and I don’t have strong preferences
on how to do that.

I’d like us to make sure, though, that this is made in a
backward-compatible way: these module names and exported bindings are
part of the API that users refer to from the OS config file.  Renaming
things typically breaks user config, and updating it to use the new
names can be tedious if there are no messages explaining what to do.
‘define-deprecated’ helps, but perhaps we need something a little bit
more fancy now.

(It would be great if we could reach a level of backward-compatibility
close to what Emacs does: nice deprecation messages, recording when a
particular binding was introduced, and generally changing user-visible
things rather slowly.)

Ludo’.

Re: [maintenance] Compressed JSON files and served file extension?

[Ludovic Courtès]

Simon Tournier  skribis:

> However, now I get:
>
> $ wget https://guix.gnu.org/sources.json

If you open it in a browser though, it’s fine, because browsers and in
fact many HTTP clients other than wget, honor ‘Content-Encoding’:

--8<---cut here---start->8---
$ wget -O/dev/null --debug https://guix.gnu.org/sources.json

[...]

---response begin---
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Nov 2023 14:38:04 GMT
Content-Type: application/json
Content-Length: 2670848
Connection: keep-alive
Content-Encoding: gzip
Expires: Thu, 16 Nov 2023 17:38:04 GMT
Cache-Control: max-age=10800
Content-Security-Policy: frame-ancestors 'none'

--8<---cut here---end--->8---

[...]

> Well, it will break some consumers of packages.json and sources.json.
> To my knowledge, for sources.json at least, the only consumer is SWH and
> it appears to me easy to keep them in touch. :-)

It shouldn’t break consumers, their client will just transparently
decompress the stream (I checked with #swh-devel back then just in
case).

That said, if you become aware of actual breakage, we can revisit this!

Thanks,
Ludo’.

Re: Proposal: Differentiate products more clearly (Cycle 01)

[Ludovic Courtès]

Hi Luis,

I’m big fan of what you propose, both graphically in terms of how we
present things!

I love the “cubes” that represent the different levels of Guix
(reminiscent of the hexagons Ricardo had in some of his presentations);
we could add a “systems” (plural) cube that I didn’t see here, but
that’s a detail.  I love that the mockups insist on “User Control” and
“User Autonomy” too.

Like Ricardo, I’m less enthusiastic about the term “package manager”,
but we should be able to work around it.

We should also make sure IMO that Guix System is not perceived as a
completely independent/separate beast, and instead insist on that notion
of a continuum.

Luis Felipe  skribis:

> I can help implement several of the proposed changes.
>
> Subsequent changes (hinted in the mockups) with the same goal of product 
> differentiation could be:
>
> 5. Guix Features section in the manual: Improve it.
>+ Separate in sections to make it easier to browse
>+ List forgotten functional features (if any)
>+ List non-functional features
>+ Compare with similar products
> 6. Guix System Features section in the manual: Add it.
> 7. Guix System Configurations page: A collection of configurations, ideally 
> of real systems currently used in production environments.

Makes sense to me.  This something those of us less into design and more
into coding can help with.

Thank you!!

Ludo’.

Re: Upgrading Guix's security team

[Ludovic Courtès]

Hi John,

Looks like this message was left unanswered for more than a month, which
proves you have a point!

John Kehayias  skribis:

> - current security email/people can be found here, which is nicely
> visible  yet probably in need of a
> hand and new faces for an important but often thankless job; no fault
> to them or Guix as a whole, merely a good time to see how we can keep
> improving

Yes, we definitely need a rotation here!  I for one have my name there
but regardless of my interest, I have to admit that I’ve been unable to
be sufficiently responsive.  It’s time to let new folks take
responsibility.

I think we should make this a fixed-term position, to make it easier for
people to commit to actually being active when needed, with the
understanding that it’s not a commitment for life.

> - currently we are not on the OS security distribution contact list:
> ; this
> had been discussed before but we will need commitment from people
>
> - clear roles will be helpful; to me this includes at least a couple
> of people to coordinate (the majority of security issues will be
> handled through package upgrades/grafts) and people to help review
> and/or contact needed experts, like for Guix internal issues; we
> should make this more precise

We could distinguish security issues in packages provided by Guix from
security issues in Guix itself.

That said, the security team could redirect things to members of the
“core” team for security issues in Guix itself; maybe we don’t need to
formally separate the two.

> - likewise, a clear fixed timeframe for who is on this team; keeping
> people fresh and engaged for what can suddenly be a time sensitive and
> critical job; I think this will also help spread institutional
> knowledge for better security practices in general

+1!

> - members need not be experts but should be active in the community as
> committers (already a round of vetting), familiar with what issues and
> processes may arise, and willing to learn; perhaps we need a list of
> experts to consult though the current teams are a good starting point

+1

> - what are your thoughts? what are the goals and outcomes we as a
> distro want in security?
>
> - finally, I think an internal discussion with maintainers and long
> time active committers would be helpful to get the improvements
> started and moving, in addition to this wider discussion here
>
> And to get things started, I'm happy to volunteer myself to help
> coordinate on security, if deemed okay by our current security team,
> maintainers, and anyone else that's been helping to handle security. A
> coordinating role with a term of say 6 months to a year? Happy to
> provide more information and discuss here or privately; in short I'm
> not a security expert but have time and bandwidth to keep things
> moving and want to learn.

Thank you for getting the ball moving!

I’m all for having you on board and, to set an example, to leave as you
join.

If maintainers agree (Cc’d), I invite you to add your name and a
termination date to the security page, remove my name, and subscribe to
guix-security.  We should add a term for other people on the team too.

How does that sound?

Ludo’.

collection of “guix pull“ bug reports

[Simon Tournier]

Hi,

>>> https://issues.guix.gnu.org/issue/62830
>>> https://issues.guix.gnu.org/issue/63451
>>> https://issues.guix.gnu.org/issue/63830
>>> https://issues.guix.gnu.org/issue/64489
>>> https://issues.guix.gnu.org/issue/64659
>>> https://issues.guix.gnu.org/issue/64753
>>> https://issues.guix.gnu.org/issue/64963

More are reported…

>>> Any idea about what could be unexpected or what needs some love?
>>
>> I haven't checked the above links, but I think something that would help
>> in this regard is better handling of network issues.  E.g, don't print a
>> backtrace on the first connection failure; retry maybe 3 times then
>> print a helpful error mentioning the network appears unreliable.
>
> IMHO, this collection raises two questions:
>
>   1. Why does it happen?  To say it explicitly, I am almost sure that
>   something is not smoothly working as expected on server side.  For
>   instance, I had ‘guix pull’ troubles with a machine and I am doubtful
>   that this machine has network issue (this machine is using the fast
>   network link of my Univ. employer :-))
>
>   2. What could be done on client side for reducing the annoyance?  I
>   agree that substitute failures should be better handled.  For example,
>   retry 3 times then display an error message.  Etc.

Well, most of the time, the error is transient so it is hard to debug.
On client side (#2), well I have ideas how to fake a faulty network on
my end.  What could be done on server side?

Cheers,
simon

Re: Turning off tests leads to a different store item

[Simon Tournier]

Hi Maxim,

On Thu, 09 Nov 2023 at 10:04, Maxim Cournoyer  wrote:

>> I agree.  On a side note, one of the issue is the time of some tests.
>> Sometimes, packaging is frustrating: build takes ages, then you fix some
>> tests, think it will be good, re-launch “guix build”, another test
>> failing, repeat.  It could nice to be able to cache the result of the
>> build phase.
>
> I usually end up working in the /tmp/guix-build-* failed build of a
> package for these situations; then I don't need to rebuild the whole
> thing for each test suite run, and can test changes directly without
> proper patches while working toward a fix.

Do you know all the command lines equivalent for each phase run by all
the build systems? ;-)

Well, this workflow seems appealing but it never works for me on
concrete situations.  Most of the time, my issue does not come from
“what to do from command line” but “how to do it inside the Guix
recipe”.

For example, this test suite adjustment:

  (add-before 'install 'disable-network-tests
(lambda _
  (substitute* "test/runtests.jl"
(("\"async.jl") "# \"async.jl")
(("\"client.jl") "# \"client.jl"))
  (substitute* "test/aws4.jl"
(("@testset.*HTTP.request with AWS authentication.*" all)
 (string-append all "return\n")))
  (substitute* "test/insert_layers.jl"
(("@testset.*Inserted final layer runs handler.*" all)
 (string-append all "return\n")))
  (substitute* "test/multipart.jl"
(("@testset \"Setting of Content-Type.*" all)
 (string-append all "return\n"))
(("@testset \"Deprecation of .*" all)
 (string-append all "return\n")))
  (substitute* "test/websockets.jl"
(("@testset.*External Host.*" all)
 (string-append all "return\n")))
  (substitute* "test/messages.jl"
(("@testset.*Read methods.*" all)
 (string-append all "return\n"))
(("@testset.*Body - .*" all)
 (string-append all "return\n"))
(("@testset.*Write to file.*" all)
 (string-append all "return\n")))
  (substitute* "test/cookies.jl"
(("@testset.*Set-Cookie casing.*" all)
 (string-append all "return\n"

To have the correct replacement rules, I almost never get it right at
the very first try, and the feedback loop is poor.  In some case, the
replacement is done after 'unpack, so I can just kill the build process
and check inside /tmp/guix-build-* if the output is the expected one.

However, depending on the test suite, sometimes, it errors at the first
failing test and does not report all the failing cases.  For example, in
the previous snippet, I could have to build then check, fail and repeat
something like 11 times, adding some try-error attempts for the correct
replacement rule.

Cheers,
simon