Re: Faster "guix pull" by incremental compilation and non-circular modules?

[Gábor Boskovits]

Hello Maxime,

Maxime Devos  ezt írta (időpont: 2022. febr. 28.,
H, 19:51):

> Ludovic Courtès schreef op ma 28-02-2022 om 14:17 [+0100]:
> > Hi,
> >
> > Maxime Devos  skribis:
> >
> > >   2. Instead of building all of Guix as a single derivation,
> > >  create a DAG of derivations.  More concretely:
> > >
> > >  First read the *.scm files to determine which module imports
> > >  which modules. Then to compile, say, (gnu packages acl),
> > >  a derivation taking gnu/packages/acl.scm and its dependencies
> > >  gnu/packages/attr.go, gnu/packages/base.go, ... is made
> > >  compiling gnu/packages/acl.scm to a gnu/packages/acl.go.
> > >
> > >  Then to build all of Guix, 'union-build' or 'file-union' is used.
> >
> > This is what (guix self), used by ‘guix pull’, is already doing.
> >
> > However, currently, package modules are split in just two groups: the
> > “base” group is the closure of (guix packages base), and the second
> > group has all the rest:
> >
> > [...]
>
> Looking at (guix self), it also has a few groups for non-package
> modules (system tests, scripts, ...).
>
> > At its core though, the situation pretty much reflects the free software
> > situation: there are low-level packages (glibc, GCC, GTK, etc.) that
> > might depend on high-level packages (Python, Pandoc, Rust, etc.).
> >
> > It’s not easy to split this spaghetti ball in smaller groups.
>
> It's not easy to manually split the spaghetti, but we don't have
> to, we could let the computer split the spaghetti for us (at least
> partially, because of the circular imports), by computing the graph of
> strongly-connected components and considering each SCC to be a ‘group’,
> some of which depend on other groups, forming a DAG.
>

I was thinking about a bit of a different structure that can also be
automated. My original idea was to use the already existing tree structure
of the derivations, and split it based on depth. I think that gives a bit
more structure, but might require splitting things that now are together
(for example iirc sometimes we are defining bootstrap packages inheriting
from the fully fledged ones, which introduces a syntactic dependency on
something that is  higher up the tree). Wdyt?

Regards,
g_bor


> I believe Ricardo Wurmus has some script for computing the SCC?
>
> Splitting large SCC in smaller parts can be left as an exercise
> for later, it's a somewhat orthogonal concern.
>
> > Thoughts?
>
> I think it would be nice to let (guix self) automatically determine the
> DAG of groups.  It would reduce the ad-hocness of the *...-modules*
> variables (some care required for patches, guix/man-db.scm, .js ...).
>
> It would also make the node tree wide, which could reduce memory usage
> (which might help with the ‘guix pull segfaults on i686-linux’
> reports).  In case of a crash (*), "guix pull" does not have to start
> over from scratch, which would also help with those reports.
>
> (*) This does not help with "failed to compute the derivation of Guix".
>
> Some work would be required, but I think it will be worth it, and it
> only has to be done once.
>
> TBC, this was just an idea I wanted to share, I won't be working on it
> in the forseeable future.
>
> Greetings,
> Maxime.
>

Re: Arun Isaac Presentation on guix-forge this Saturday

[jgart]

On Mon, 30 May 2022 14:31:12 -0500 jgart  wrote:
> On Mon, 30 May 2022 17:55:27 +0200 Ludovic Courtès  wrote:
> Hi Guixers!
> 
> Below is an updated link that actually shows Arun's screenshare:
> 
> https://archive.org/details/arun-isaac-guix-forge-may-28

Other link had no audio ;()

Here's the latest one:

https://archive.org/details/arun-isaac-on-guix-forge

Re: A corner case of broken reproducibility

[Maxime Devos]

Ludovic Courtès schreef op ma 30-05-2022 om 17:58 [+0200]:
> Perhaps it should forcefully “chown -R” home directories at boot time,
> so they have the right UID?  This has been discussed a few times,

I haven't seen the "chown -R" suggestion yet in the context of homes
(only for system accounts, and at activation time, not only boot time).

> especially in the context of system accounts; systemd-homed appears to
> be doing exactly that.

I don't think the problem is that the uid of /home/... was wrong,
rather I think the problem is that Guix has forgotten the uid and hence
invents a new one to put in /etc/passwd instead of keeping the old one.

A pitfall (noticed in the context of system accounts): the user could
have created files outside /home (e.g. in /tmp).  IIUC, this would also
require a reboot to keep name<->uid consistent after "guix system
reconfigure".

A 'chown -R' doesn't seem great to me from a security perspective
(seems very easy to get something wrong, and the TOCTTOU-free chownat
hasn't been merged yet in Guile), a performance perspective (what if
you have a huge $HOME).  Also extra io -> slower boot + disk wear.
It also destroys some information, it's possible to intentionally have
files owned by other users inside $HOME.

Things that seem missing here to me:

  * a mechanism for remembering that an uid is still in use even though
the user has been removed (previously mentioned solutions: keep the
uid in /etc/passwd even though it is ‘removed’, or keep a separate
/etc/passwd-graveyard or such, etc.).  For system accounts and user
accounts.  Won't help in this particular case but would make more
general adding/removing user accounts less fragile (avoid 
accidental reuse).

  * a mechanism for telling Guix ‘I'm renaming the user account, not
creating and removing a new one, so keep the uid’

  * some heuristics for detecting mistakes (e.g.: if Guix thinks it
should create a directory /home/foo for uid 1234, but it notices
there is already a directory /home/bar with that uid 1234, then
that's super suspicious.  Likewise, if Guix thinks the home
/home/foo should be owned by uid 1234, but it notices it's already
owned by 1235!=1234, that's also suspicious).

IIUC, Blake Shaw only changed their user name, not the home
directory, so this would have detected the problem

  * some mechanism for resolving mistakes 

Or maybe something else entirely, whatever works without causing new
problems I guess.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Arun Isaac Presentation on guix-forge this Saturday

[jgart]

On Mon, 30 May 2022 17:55:27 +0200 Ludovic Courtès  wrote:
Hi Guixers!

Below is an updated link that actually shows Arun's screenshare:

https://archive.org/details/arun-isaac-guix-forge-may-28

Let me know if you run into any issues. 

I recommend downloading it from archive.org for local viewing as
archive.org tends to struggle with streaming it.

all best,

jgart

https://whereis.みんな/

Re: Cuirass and SQL

[Arun Isaac]

Hi Ludo,

> Years before, Hydra (https://nixos.org/hydra) also dropped its SQLite
> backend in favor of PostgreSQL only.
>
> Like you, not being a database person, I liked that SQLite was easy to
> deploy and had a clear model: it just touches this one file and that’s
> it.

Exactly! :-)

If we use guile-dbi, it should be possible to support both sqlite and
postgresql. Popular projects like Nextcloud do allow the user to choose
their preferred database system. But, then again, Cuirass is built for
very large scale. So, perhaps it is best to not try and also cover the
small scale end.

> SQLite may be good enough at a small scale; the problem is that you
> never know how long it’ll be before the project you’re hosting is no
> longer small-scale.

Scale may not be an issue at least for the CI. Unlike Guix which needs
to rebuild the entire world of software, most other software are only
going to have a handful of jobs---easily less than 5 or less than 10 in
the worst case. So, even if we trigger all these jobs on every commit,
the total number of runs will easily be manageable with sqlite.

Regards,
Arun

Re: Arun Isaac Presentation on guix-forge this Saturday

[Arun Isaac]

Hi Ludo,

> This is really nice.  I like the idea of having easy deployment through
> a Guix service, and of composing existing tools.  (It’d even be worth a
> blog post, hint hint ;-))

He he, I'm getting the hint! ;-) Maybe, I'll write a blog post when
guix-forge is a little bit more featureful! :-)

> I wonder if there could be a more Guiley flavor of guix-forge, for
> instance with Gitile instead of Cgit

I have added gitile, klaus and other git web viewers to my agenda. I'm
sure everyone will have their favorite git web viewer. So, it'll be good
to cater to a wider range of tastes. Besides, it's really easy to
support multiple read-only git web viewers. Patches welcome! ;-)

Also, I'm thinking parts of guix-forge can be gradually merged into Guix
itself. The webhook-service-type in guix-forge is already a good
candidate with wider use cases than just guix-forge.

> mumi instead of public-inbox (though these two are not quite
> equivalent).

Mumi is tricky to deploy since it is not standalone and requires being
on a special IP allowlist with the Debbugs GNU servers. If mumi could be
augmented to not need a Debbugs server at all, then it would become much
more attractive.

> Thanks for sharing!

Thanks for inventing Guix and making guix-forge inevitable! :-)

Regards,
Arun

Re: wishlist: “repack” generations history of profile

[zimoun]

Hi,

On lun., 30 mai 2022 at 17:40, Ludovic Courtès  wrote:

> Yes, that ‘manifest’ file would have copied elsewhere (we can’t just
> remove part of what’s in a /gnu/store directory).

[...]

>> Instead of this external tracking, I would like to allow this workflow:
>>
>> guix package -p project --list-generations
>> guix package -p project --switch-generation=12
>>
>> whatever the sysadmin collect about the old generations.
>
> Do you expect ‘--list-generations’ to look at older revisions of your
> version-controlled ‘manifest.scm’?

I do not expect that Guix uses my version-controlled ’manifest.scm’ but
instead that Guix uses its own internal one.

If the sysadmin of my cluster does as root “guix gc
--delete-generations=3m”, then this GC is out of my control and
unexpected by me, which somehow breaks the rootless argument.

Other said, because “guix gc” can be run periodically (for good
reasons!), as a user, it is hard to predict what I could loose.

Well, consider the situation:

 1. User install foo bar the profile my-project on January
 2. User update foo bar on February
 3. User works on another project
 4. Months later, user works again on my-project

The generation #1 can be lost.  For sure it depends on the cluster
policy but, as a sysadmin, I do not tell all the users that a GC will be
run – and even if I am doing, I am sure that some user will miss to save
the channels.scm and manifest.scm for each generation.

That’s why, something like “repack” is missing.  As a user, I should be
able to do

guix package --switch-generation=1

whatever the sysadmin collects about the old generations and whatever I
saved using some external tools.

At GC time, enough information of the old generations should be kept
allowing “guix package --switch-generation” or --export-manifest or
else.

We could imagine an intermediary mode between the two current ones:

 + full generation
 + repack (only keep some text files)
 + purge (remove these few text file)



Cheers,
simon

Java $CLASSPATH variable questions

[Christopher Rodriguez]

Hello all,

I noticed the other day that neither `icedtea` nor `openjdk` set up a
`CLASSPATH` variable for the user. Is this intended behavior, or an oversight I
could help to fix in those packages?

And as a corollary, is there any reason (for right now, if the above should set
that variable) I shouldn't set `CLASSPATH` to `$GUIX_PROFILE/share/java` in my
guix home configuration?

Thanks for Your time.

--

Christopher Rodriguez


signature.asc
Description: PGP signature

Re: Merging the purge-python2-packages branch

[zimoun]

Hi Maxim,

On lun., 30 mai 2022 at 11:32, Maxim Cournoyer  
wrote:
> zimoun  writes:
>> On lun., 30 mai 2022 at 09:25, Maxim Cournoyer  
>> wrote:
>>
>>> Most of the removal were automated using a script [0], but each package
>>> removed had their upstream status considered (using last commit or
>>> existing patches for Python 3 compatibility) and quite a few were saved
>>> that way.
>>
>> These packages perfectly build.
>
> On Python 2, that is :-).

Yeah. :-)  Another point of view:

Well, me, personally, I continue to do most of my research using
Python 2 because I cannot afford to port everything to Python
3. And since I do only number crunching, meaning nothing with
security implications, I am not particularly worried.

[...]

Once Python 2 lives in a largely isolated package sub-universe,
I don't see much harm keeping it in Guix for now. If security
issues become apparent, we might have to do something more
drastic.




And if you filter by ’python2-’ from the dashboard,



many are broken but many are still fine.

BTW, ’python2-biopython’ is broken in the branch
’purge-python2-packages’ and still there [1].

1: 


> This effort is an attempt to reduce our dependencies on Python 2 as much
> as possible, so that we can hopefully remove Python 2 from our tree
> before 2030 comes ;-).

I agree and I am advocating since 2019-10-31 [2] for an explicit
plan. :-) However, my understanding of this plan after several
discussions is: remove the python2- variants once they are broken.

It appears to me surprising: we do not provide a schedule for the
removals, then bang purge.

2: 



Well, as a hobbyist, I am fine with such purge.  As a scientific
practitioner using Guix at work, it is more annoying…

>> bamm /gnu/store/17hs9c60isdk8sfkpi0280xzc0gkyxn1-bamm-1.7.3

[...]

>> Therefore, I would not remove them; I have not checked which python2-
>> dependencies they are using.
>
> If something is valuable up there, I'd suggest interested parties
> contribute them to the Guix-Past channel.

…for instance, I use time to time the package ’bamm’.  Because inferiors
and time-machine, the removal of ’bamm’ is not a big deal per se but I
need some time to adapt some scripts, etc.

Other said, I am fine with the purge and I volunteer to help in
transferring from master to guix-past but we need a schedule,
communicate on the purge and more importantly say when it will
happen. :-) (maybe not a purge 2-3 days after an announcement on
guix-devel following 2 weeks in a branch ;-))


Cheers,
simon

PS: Great script, BTW. :-)

Re: auth-tarball-from-git

[Ludovic Courtès]

Hi!

kias...@disroot.org skribis:

> Authenticate a tarball through a signed tag in a git repository (with 
> reproducible builds).
>
> Blog post: https://vulns.xyz/2022/05/auth-tarball-from-git/
>
> Source code: https://github.com/kpcyrd/auth-tarball-from-git
>
> Pretty interesting, could be useful for guix.

The example here is about downloading github.com-generated tarballs,
which we don’t do in Guix for other reasons.

The kind of tarballs that Guix packages refer to are not autogenerated;
they contain more than what the VCS has.  Thus, they’re also more
difficult to authenticate if they’re not signed.

Thanks for sharing!

Ludo’.

Re: Breaking change: Make 'description' of mandatory

[Ludovic Courtès]

Hello Reily,

Apologies for the breakage cause by this change.

Reily Siegel  skribis:

> The problem arises when a certain feature needs to extend two services
> to be useful: take configuration of an emacs package. It must first
> extend (in the case of Guix home) home-emacs-service (from RDE channel)
> with the emacs configuration to be inserted to init.el, and
> home-profile-service-type, to add the emacs package to the profile. It
> seems like simple-service /would/ be a good option here, except as best
> I can figure out it can only extend one service. So instead, I create a
> new service-type, perhaps named my-emacs-feature-configuration-service,
> which takes no value and has no extension mechanism, but only serves to
> extend multiple other "real" services.

OK, that sounds reasonable to me.

> This change (and the discussion at https://issues.guix.gnu.org/55404)
> indicates to me that all service-types, no matter where they are
> implemented, are meant to be consumed by a generic user, not used in a
> one-off way like my configuration does.

I’m not sure what you mean by “generic user”.  The focus in the
discussion above was on all the service types defined within the Guix
repo, but that doesn’t mean one cannot define service types elsewhere.

If you’re defining one-off service types, perhaps adding a ‘description’
feels overkill.

> So, to sum up, I have a few questions:
>
> 1. Is service-type meant for use in individual user configurations?

Sure, if it’s useful, why not: it’s part of the public API.

> 2. Is there an equivalent function to simple-service that takes multiple
>service/value pairs that I have missed?
>(e.g., (simple-service-like service-a val-a service-b val-b ...)
> or (simple-service-like (list service-a val-a service-b val-b)))

No, but we could define one, or perhaps just extend ‘simple-service’ to
three or more arguments instead of just three?

> 3. If the answer to 2 is no, does it make sense to extend simple-service
>to work with multiple service extensions, or is there some reason for
>only extending one service at a time?

The only reason ‘simple-service’ extends a single service type is that
it seemed to be a common use case back then.

Thanks for your feedback!

Ludo’.

Re: A corner case of broken reproducibility

[Ludovic Courtès]

Hi,

Felix Lechner  skribis:

> On Tue, May 24, 2022 at 11:42 PM Blake Shaw  wrote:
>>
>> I changed my user name in my config without adding a new user
>
> I did that once in a new installation. In the second generation of my
> configuration I changed the example user 'bob' to my own username. It
> did not work well.
>
> Due to precautions I was able to go back in as the root user and
> adjust the permissions. The only unexpected part was that the symbolic
> links for per-user profiles encode the relationship between user name
> and uid.

Yes, this part is inherently stateful: Guix will not meddle with your
home directories and such.

Perhaps it should forcefully “chown -R” home directories at boot time,
so they have the right UID?  This has been discussed a few times,
especially in the context of system accounts; systemd-homed appears to
be doing exactly that.

Thoughts?

Ludo’.

Re: Arun Isaac Presentation on guix-forge this Saturday

[Ludovic Courtès]

Hello Arun!

Too bad I missed the event on Saturday.

Arun Isaac  skribis:

> guix-forge is a Guix service that, ambitiously, tries to reproduce a
> sourcehut, GitHub or GitLab like code forge, but using only
> off-the-shelf components like cgit, laminar, public-inbox, etc. The idea
> is to enable users to write a few lines of Guix operating-system
> configuration code, and have an entire code forge deployed and
> ready. guix-forge is similar to projects like FreedomBox, YunoHost and
> Mail-in-a-Box, but for code forges and built on the rock-solid
> foundation that is Guix.

This is really nice.  I like the idea of having easy deployment through
a Guix service, and of composing existing tools.  (It’d even be worth a
blog post, hint hint ;-))

> For now, only the laminar CI feature is in place. In the presentation, I
> will show a setup of laminar CI with guix-forge. On every git commit to
> a project repo, the CI will
> - automatically run tests for the project
> - build and deploy a static project website
>
> If you would like to take a sneak peek of a guix-forge configuration,
> there is a simple example in the Tutorial section of the manual at
> https://guix-forge.systemreboot.net/manual/dev/en/
>
> For a more complex real world configuration that, among other things,
> does continuous deployment of a web service, you can look at the
> guix-forge configuration for genenetwork.org at
> https://git.genenetwork.org/arunisaac/genenetwork-machines . The Laminar
> CI deployed by this configuration is at https://ci.genenetwork.org/

Neat.  I wonder if there could be a more Guiley flavor of guix-forge,
for instance with Gitile instead of Cgit and mumi instead of
public-inbox (though these two are not quite equivalent).

Thanks for sharing!

Ludo’.

Re: Cuirass and SQL

[Ludovic Courtès]

Hi,

Arun Isaac  skribis:

>> Initially, Cuirass was using SQLite but then switched [1] to
>> PostgreSQL.  The main reason is scalability.
>
> Ah, I see. I didn't know that.

Years before, Hydra (https://nixos.org/hydra) also dropped its SQLite
backend in favor of PostgreSQL only.

Like you, not being a database person, I liked that SQLite was easy to
deploy and had a clear model: it just touches this one file and that’s it.

Yet, I’m starting to see a pattern.  :-)

SQLite may be good enough at a small scale; the problem is that you
never know how long it’ll be before the project you’re hosting is no
longer small-scale.

Ludo’.

Re: wishlist: “repack” generations history of profile

[Ludovic Courtès]

Hi,

zimoun  skribis:

> On Mon, 23 May 2022 at 17:42, Ludovic Courtès  wrote:
>
>> Exactly!  ‘guix pull’ profiles are entirely reproducible: we can rebuild
>> them from the output of ‘guix describe’.
>>
>> So ‘guix gc’ (or something) could automatically remove old generation
>> symlinks and instead store the output of ‘guix describe’.  That way,
>> ‘--list-generations’ or ‘--switch-generations’ could transparently
>> display the info or rebuild the generation.
>
> I have in mind to remove all except the manifest file.

[...]

> Or maybe, it is easier to remove as today but save the relevant
> information elsewhere.  WDYT?

Yes, that ‘manifest’ file would have copied elsewhere (we can’t just
remove part of what’s in a /gnu/store directory).

> Today, I manually store the output of “guix describe” before I
> manipulate a profile.  This channels.scm output is tracked by Git as
> part of my project.  Then I run “git log” to find the previous state of
> interest and go back using,
>
> guix time-machine -C channels.scm -- shell -m manifest.scm
>
> which is fine and works well.  Probably the good practise. :-)
>
> Instead of this external tracking, I would like to allow this workflow:
>
> guix package -p project --list-generations
> guix package -p project --switch-generation=12
>
> whatever the sysadmin collect about the old generations.

Do you expect ‘--list-generations’ to look at older revisions of your
version-controlled ‘manifest.scm’?

Ludo’.

Re: Finding a “good” OpenPGP key server

[Ludovic Courtès]

Maxime Devos  skribis:

> Ludovic Courtès schreef op ma 18-04-2022 om 22:24 [+0200]:
>> [... guix refresh -u stuff failing due to not finding the key ...]
>> I’m not sure what a good solution is (other than looking for the key
>> manually on Savannah or on some random key server).
>
> Alternatively, why use key servers at all?  WDYT of something like
>
> (package
>   (name "gnurl")
>   [...]
>   (properties
> ;; Keys that are considered ‘trustworthy’ for signing releases
> ;; of gnurl.
> `((permitted-pgp-signing-keys "CABB A99E ..." "DEAD BEEF ...")
>   ;; Locations of PGP key (possibly with some of them pointing to
>   ;; the same key)
>   (pgp-key-locations
> ,(savannah-pgp-key USER-ID) ... ; most signers are on savannah.gnu.org
> ,(local-file "[...]/someone.pub") ; not easily available from the Web 
>   
> "https://rando/key.pub;
> "ipfs://.../..." "gnunet://..." ; download key via P2P networks
>
> The first part (permitted-pgp-signing-keys) has been suggested previously and
> seems mostly orthogonal, but the second part is new.  It would reduce
> the dependency on central infrastructure.  We could consider key servers
> to be ‘merely’ another fallback.

We could also have our own key server.  Just like ‘guix lint -c
archival’ triggers SWH archival, we could have a tool that triggers key
download on the server so that crypto material never vanishes.

Food for thought…

Ludo’.

Re: emacs-guix (upstream) needs more love: a survey of repositories, homepage and issues

[Ludovic Courtès]

Hello Kaelyn,

Kaelyn  skribis:

>> First, we need to cherry-pick relevant commits from gitlab.com. Any
>> takers? If you Giovanni or anyone else is willing to help, we can grant
>> commit access so we share the work. Another way to help is by listing
>> commits that should be applied.
>>
>> Volunteers?
>
> I'd be happy to help with the efforts!

Yay!

> I just took a few minutes and checked both repos out into a single
> working tree, and there aren't many commits unique to each
> repository. The official savannah repo has 5 commits since they
> diverged, with the 3 oldest looking like variations of the 6 oldest in
> the gitlab repo. Likewise, not counting the 6 just mentioned, there
> are 4 unique commits in the gitlab repo. Those 4 commits are:
>
> * c9aef52 - (gitlab/master, gitlab/HEAD) elisp/ui-package: Add 
> 'guix-package-use-name-at-point' variable (12 months ago)
> * e5ff0e5 - elisp/ui-package: Fix an error on package name read (12 months 
> ago)
> * 8ce6d21 - Rename 'guix-search-…' to 'guix-packages-…' commands (1 year, 3 
> months ago)
> * fbc2bbc - elisp/ui-package: Use thing at point for 'guix-packages-by-name' 
> command (1 year, 3 months ago)

Awesome.

Would you be willing to coordinate work on Emacs-Guix for some time?
If so, I’m in favor of granting you commit access so you can first push
these four commits, and eventually apply patches that are submitted or
fix bugs here and there.

If Giovanni or Théo wants to do that, that’s fine too.  What we need is
to make sure one of us/you can commit some time going forward to at
least protect Emacs-Guix from bitrot, and ideally help improve it, as
time permits.

WDYT?

Bug reports would still go to , which you
can access from the comfort of your Emacs with M-x debbugs-gnu.  :-)

Thanks,
Ludo’.

Re: Merging the purge-python2-packages branch

[Maxim Cournoyer]

Hi Simon,

zimoun  writes:

> Hi Maxim,
>
> Cool!  That’s a nice removal. :-)
>
> On lun., 30 mai 2022 at 09:25, Maxim Cournoyer  
> wrote:
>
>> Most of the removal were automated using a script [0], but each package
>> removed had their upstream status considered (using last commit or
>> existing patches for Python 3 compatibility) and quite a few were saved
>> that way.
>
> These packages perfectly build.

On Python 2, that is :-).

This effort is an attempt to reduce our dependencies on Python 2 as much
as possible, so that we can hopefully remove Python 2 from our tree
before 2030 comes ;-).

> bamm /gnu/store/17hs9c60isdk8sfkpi0280xzc0gkyxn1-bamm-1.7.3
> childsplay   /gnu/store/1gjbfx4s30r074fl1nhr5m8srsxvmavf-childsplay-3.4
> chirp/gnu/store/piwqmj7y6b1b3p0sk9vxwzd3fn68nz00-chirp-20220118
> djvusmooth   /gnu/store/y1djnpg169mm51381ylrjxdrxa8la3xa-djvusmooth-0.3
> find-circ
> /gnu/store/i1vbr24h858irfwlnldjb2brsrl7j168-find-circ-1.2-1.8655dca
> fraggenescan /gnu/store/h211d2xqsf1dcqhm9b9mp4dl463hslxn-fraggenescan-1.30
> gnome-doc-utils  
> /gnu/store/z6kwlmqyrfrpmvh75sz109krlk5jk9i3-gnome-doc-utils-0.20.10
> graphios /gnu/store/38fk6qkqvcjl7q63j8iasqb4igr715jf-graphios-2.0.3
> gtklick  /gnu/store/zdh104pc78camqk96dy23yp61xq0695h-gtklick-0.6.4
> h-client 
> /gnu/store/z8cg927pfl318ds0aifx7s9q4mxkvzgl-h-client-0.0a0-138
> ingen
> /gnu/store/xhxg7q3g5zffx8qpkcnz8h4wp6jwqg9r-ingen-0.0.0-2.cc4a4db33
> key-mon  /gnu/store/y6fq6plq1v2myl8nszhp06c9xr8bcfli-key-mon-1.17
> lekha/gnu/store/akcbq03r9n6ap6p781c18n5xllid5y8s-lekha-0.2.1
> mloop
> /gnu/store/v94mbqhji8ag0v12v00zq9nd5b6lgirq-mloop-0.0.1-0.adebff9
> non-mixer
> /gnu/store/cmm8drl9isnmcgrzlsv9mrqrvknwchfm-non-mixer-1.9.5-4.5ae43bb
> non-timeline 
> /gnu/store/kazn7qihs8q3b4qmi6md7y0dk5ivhv7s-non-timeline-1.9.5-4.5ae43bb
> omnitux  /gnu/store/qbpsz7zaxdpaicmdxvp6s086yqplh4ks-omnitux-1.2.1
> patches  
> /gnu/store/5qvzzri85wiph03kcpknvg0kpnwj528g-patches-0.0-1.ef1b8a7
> pyicoteo /gnu/store/vcrdrgzvgfbv28k9qj291jx8r31m7l1j-pyicoteo-2.0.7
> raul 
> /gnu/store/6l5960xcmdggfkrsnnll5pb72knj2dga-raul-0.8.9-1.4db870b2b
> rawdog   /gnu/store/vbw20xafk84p34y56bgjcbhnz2fvshnm-rawdog-2.23
> sala /gnu/store/c5s2ig87xq4ld7fvx0z1adk8hjgvqkds-sala-1.3
> slingshot/gnu/store/ck765xq1izagvb14bvj6vkaarlam88c0-slingshot-0.9
> wicd /gnu/store/rmg0f4qx7ac3w148cxigmj9xkdzzwa6d-wicd-1.7.4
> youtube-dl-gui   
> /gnu/store/lgf9xbfi7rp91kbajbxirbndssfgfwvw-youtube-dl-gui-0.4
>
> Therefore, I would not remove them; I have not checked which python2-
> dependencies they are using.

If something is valuable up there, I'd suggest interested parties
contribute them to the Guix-Past channel.

What do you think?

Maxim

Re: antioxidant-build-system can be tested as a channel, + GTK app 'castor' builds

[Maxime Devos]

Ludovic Courtès schreef op ma 30-05-2022 om 17:26 [+0200]:
> (Unfortunately notabug.org is down since a few days ago.)

For now, the fallback location
https://github.com/emixa-d/antioxidant-fallback/
can be used.  (rest of the response later)


signature.asc
Description: This is a digitally signed message part

Re: antioxidant-build-system can be tested as a channel, + GTK app 'castor' builds

[Ludovic Courtès]

Hello,

Maxime Devos  skribis:

> I've set up the channel things for the antioxidant repo,
> so now you can put the following in channels.scm:
>
> (use-modules (guix ci))
>
> (cons
>  (channel
>   (name 'antioxidated-packages)
>   (url "https://notabug.org/maximed/cargoless-rust-experiments;)
>   (introduction
>(make-channel-introduction
> "020851ad649480ee4769b77a947642e993ea5956"
> (openpgp-fingerprint
>  "C1F3 3EE2 0C52 8FDB 7DD7  011F 49E3 EE22 1917 25EE"
>  (list (channel-with-substitutes-available
> %default-guix-channel
> "https://ci.guix.gnu.org;)))
>
>
> do "guix pull" and "guix shell antioxidated-castor -- castor" to
> run a 'castor' build (*) with antioxidant-build-system.  (Warning:
> some crates were updated or added without checking the source code
> diff for malware!)

Nice!  (Unfortunately notabug.org is down since a few days ago.)

To make sure share a common understanding, could you post a summary of:

  1. the goals;
  2. the status;
  3. pros and cons over the status quo and other options (if any!);
  4. the next steps.

I think you already explained #1 but it can’t hurt to rehash.  ;-)

It’d be great to address the major shortcoming of the current solution,
which is its isolation from the rest of Guix.

Thanks,
Ludo’.

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

Efraim Flashner schreef op ma 30-05-2022 om 11:23 [+0300]:
> Something that might help with that would be to also include the
> source of the crate somewhere in the output. Then at the worst we
> could just put the rust inputs of librsvg as regular inputs and let
> librsvg do its own special build thing. We'd then still keep the
> build dependency tree you've got working.

Maybe, but that kind of defeats the point of

 * no compiling dependencies again -- old artifacts are reused

and librsvg's build system seems to use Cargo, so then we again have
the problem that Cargo wants Windows crates etc. that are useless in
Guix and its overly-pickyness in versioning, we lose the ability to let
crates be shared libraries (not yet implemented, but could be
implemented in antioxidant) ..., so I'd rather just implement cdylib
and patch the build system appropriately.

(It's build system is more complicated than ordinary crates, but the
additional Makefile.am isn't too long).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Merging the purge-python2-packages branch

[zimoun]

Hi Maxim,

Cool!  That’s a nice removal. :-)

On lun., 30 mai 2022 at 09:25, Maxim Cournoyer  
wrote:

> Most of the removal were automated using a script [0], but each package
> removed had their upstream status considered (using last commit or
> existing patches for Python 3 compatibility) and quite a few were saved
> that way.

These packages perfectly build.

--8<---cut here---start->8---
bamm /gnu/store/17hs9c60isdk8sfkpi0280xzc0gkyxn1-bamm-1.7.3
childsplay   /gnu/store/1gjbfx4s30r074fl1nhr5m8srsxvmavf-childsplay-3.4
chirp/gnu/store/piwqmj7y6b1b3p0sk9vxwzd3fn68nz00-chirp-20220118
djvusmooth   /gnu/store/y1djnpg169mm51381ylrjxdrxa8la3xa-djvusmooth-0.3
find-circ
/gnu/store/i1vbr24h858irfwlnldjb2brsrl7j168-find-circ-1.2-1.8655dca
fraggenescan /gnu/store/h211d2xqsf1dcqhm9b9mp4dl463hslxn-fraggenescan-1.30
gnome-doc-utils  
/gnu/store/z6kwlmqyrfrpmvh75sz109krlk5jk9i3-gnome-doc-utils-0.20.10
graphios /gnu/store/38fk6qkqvcjl7q63j8iasqb4igr715jf-graphios-2.0.3
gtklick  /gnu/store/zdh104pc78camqk96dy23yp61xq0695h-gtklick-0.6.4
h-client /gnu/store/z8cg927pfl318ds0aifx7s9q4mxkvzgl-h-client-0.0a0-138
ingen
/gnu/store/xhxg7q3g5zffx8qpkcnz8h4wp6jwqg9r-ingen-0.0.0-2.cc4a4db33
key-mon  /gnu/store/y6fq6plq1v2myl8nszhp06c9xr8bcfli-key-mon-1.17
lekha/gnu/store/akcbq03r9n6ap6p781c18n5xllid5y8s-lekha-0.2.1
mloop
/gnu/store/v94mbqhji8ag0v12v00zq9nd5b6lgirq-mloop-0.0.1-0.adebff9
non-mixer
/gnu/store/cmm8drl9isnmcgrzlsv9mrqrvknwchfm-non-mixer-1.9.5-4.5ae43bb
non-timeline 
/gnu/store/kazn7qihs8q3b4qmi6md7y0dk5ivhv7s-non-timeline-1.9.5-4.5ae43bb
omnitux  /gnu/store/qbpsz7zaxdpaicmdxvp6s086yqplh4ks-omnitux-1.2.1
patches  
/gnu/store/5qvzzri85wiph03kcpknvg0kpnwj528g-patches-0.0-1.ef1b8a7
pyicoteo /gnu/store/vcrdrgzvgfbv28k9qj291jx8r31m7l1j-pyicoteo-2.0.7
raul 
/gnu/store/6l5960xcmdggfkrsnnll5pb72knj2dga-raul-0.8.9-1.4db870b2b
rawdog   /gnu/store/vbw20xafk84p34y56bgjcbhnz2fvshnm-rawdog-2.23
sala /gnu/store/c5s2ig87xq4ld7fvx0z1adk8hjgvqkds-sala-1.3
slingshot/gnu/store/ck765xq1izagvb14bvj6vkaarlam88c0-slingshot-0.9
wicd /gnu/store/rmg0f4qx7ac3w148cxigmj9xkdzzwa6d-wicd-1.7.4
youtube-dl-gui   /gnu/store/lgf9xbfi7rp91kbajbxirbndssfgfwvw-youtube-dl-gui-0.4
--8<---cut here---end--->8---

Therefore, I would not remove them; I have not checked which python2-
dependencies they are using.


Cheers,
simon

Re: Why does sh in the build environment ignore SIGINT and SIGQUIT?

[Ludovic Courtès]

Hi,

Foo Chuan Wei  skribis:

> The shell in the environment where packages are built ignores SIGINT and
> SIGQUIT. If I add `(invoke "sh" "-c" "trap")` to a custom build phase,
> this output is produced during the build:
>
> trap -- '' INT
> trap -- '' QUIT
>
> Why does the shell in the build environment need to ignore these two
> signals?

That’s because it’s running as PID 1, which does not have default signal
handlers, unlike other processes (see the discussion at
.)

> For context, this affects the build of the smlnj package
> (gnu/packages/sml.scm). The resulting executable seems to inherit the
> signal dispositions of the shell where the executable is built, with the
> result that CTRL-C is ignored at the sml REPL.

Lack of proper signal handler may affect the build process of smlnj, but
I don’t think it can affect the resulting executable.  Which signal
handlers a process running ‘smlnj’ depends on run-time factors, not on
its build environment, unless the build process explicitly records the
set of available signal handlers, but that would be a strange thing to
do.

Ludo’.

Re: Test US mirror for bordeaux.guix.gnu.org and slow downloading of substitutes

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> Now, performance is probably a bit more complicated than just download
> speed. The US mirror holds the narinfo information locally, which should
> enable it to respond to requests more quickly, reducing latency, but
> this is a little harder to test.

For those of us who have a fast connection to the server, download time
is CPU-bound; in that case, offering zstd substitutes can make a real
difference:

  https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/

Ludo’.

Re: Cuirass and SQL

[Development of GNU Guix and the GNU System distribution.]

On Fri, 27 May 2022, zimoun  wrote:

> Initially, Cuirass was using SQLite but then switched [1] to
> PostgreSQL.  The main reason is scalability.

Make sens for the main CI used by Guix or for an organization.

> I do not know if it is a technically doable to have two SQL backends and
> let the user pick the one they prefer.  For sure, it is not doable from
> a maintenance point of view.

With guile-dbi, the maintenance is trivial.

> About the complexity of PostgreSQL, I think the Guix services [2,3]
> help here.

This is what I did.  I have a local CI as a service that build my
projects at every new commits on a specific branch.  It works, but I
feel thing would have been easier to setup with SQLite.  It could had
been a guix-home service for example instead of a system service and
testing the configuration is much easier with a scratch DB.

-- 
Olivier Dion
oldiob.dev

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[raingloom]

On Mon, 30 May 2022 11:23:21 +0300
Efraim Flashner  wrote:

> On Sat, Apr 02, 2022 at 05:18:55PM +0200, Maxime Devos wrote:
> > Hi,
> > 
> > antioxidant-build-system can now be used for some ‘real’ software
> > -- it compiles 'hexyl'.  To test, download
> >  (commit:
> > d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
> > "guix build -L . -f guix.scm" and then
> > 
> > $ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl  
> > > lots of coloured hex output  
> > 
> > Some features of antioxidant-build-system:
> > 
> >   * no copying source code of dependencies
> >   * no compiling dependencies again -- old artifacts are reused
> >   * all dependencies use the usual package input system
> > (native-inputs, inputs, propagated-inputs)
> > 
> > Limitations:
> > 
> >   * no support for linking to arbitrary shared libraries yet
> > (only rust deps)
> >   * makes a few assumptions on the source layout (can be fixed
> > by using more info from Cargo.toml)
> >   * no tests
> >   * no cross-compilation yet
> >   * no shared libraries (just replacing 'rlib' by 'dylib' causes
> > problems)
> >   * code is a bit messy
> >   * no cdylib yet (probably needed for librsvg)  
> 
> Something that might help with that would be to also include the
> source of the crate somewhere in the output. Then at the worst we
> could just put the rust inputs of librsvg as regular inputs and let
> librsvg do its own special build thing. We'd then still keep the
> build dependency tree you've got working.
> 
> 

Maybe put the source in an extra output, not the main "out" one.
Splitting them up later for size reduction will be much more difficult
than doing this from the get-go.

Merging the purge-python2-packages branch

[Maxim Cournoyer]

Hi everyone,

Just so you know, there's this 'purge-python2-packages' branch that does
away with quite a few Python 2 packages.  It greatly reduces the Python
2 dependency graph of Python 2 packages, without totally eliminating it
yet.

Here's the list of removed packages, obtained with

"git log origin/purge-python2-packages...origin/master --oneline --grep='gnu: 
Remove'"

--8<---cut here---start->8---
bcb9b1689f gnu: Remove python2-pytest-cov.
426e945a3e gnu: Remove python2-called-python.
4963a61644 gnu: Remove python-prompt-toolkit-2.
39ca4b0895 gnu: Remove python2-setuptools.
42b923915b gnu: Remove python2-checkm-genome.
2b25c71b1c gnu: Remove python2-backports-csv.
ddde4dedff gnu: Remove python2-fonttools.
63445163fa gnu: Remove python2-py.
ee4f035c2c gnu: Remove python2-pyparsing.
633b0846dd gnu: Remove python2-setuptools-scm.
799022b040 gnu: Remove python2-six-bootstrap.
473c00f17e gnu: Remove python2-pyxdg.
c652b8ce97 gnu: Remove python2-wcwidth.
f528943a13 gnu: Remove python2-pyyaml.
b68c90e347 gnu: Remove python2-nose.
43f57f04ff gnu: Remove python2-attrs-bootstrap.
8102195b97 gnu: Remove python2-libxml2.
90b63ad7a6 gnu: Remove 4store.
2c51888c32 gnu: Remove python2-pyfakefs-bootstrap.
ea7af59a5e gnu: Remove python2-cython.
6b3c3c383d gnu: Remove python2-lirc.
c7abbbf24c gnu: Remove python2-tlsh.
c4fd72dc2f gnu: Remove python2-more-itertools.
e7dafd7fdb gnu: Remove gnome-doc-utils.
ae2cd32bf4 gnu: Remove python2-libmpsse.
d256f68b1a gnu: Remove python2-lxml.
f90acf5982 gnu: Remove vtk-6.
021daaae36 gnu: Remove python2-appdirs.
87382d2c97 gnu: Remove python2-enum34.
cd97147785 gnu: Remove boost-with-python2.
82655b3665 gnu: Remove libpng-1.2.
9c83be6d09 gnu: Remove rapicorn.
f7f51c8aa5 gnu: Remove beast.
64d7097f72 gnu: Remove python2-linecache2.
403c9a8bd6 gnu: Remove python2-traceback2.
f26e596977 gnu: Remove python2-unittest2.
8edc2ca792 gnu: Remove python2-sortedcontainers.
280160e6fe gnu: Remove python2-funcsigs.
9864dd74e6 gnu: Remove python2-functools32.
8bea025529 gnu: Remove python2-hypothesis.
2500d57fe8 gnu: Remove python2-mock.
8dcbf523e8 gnu: Remove python2-pysqlite.
ad11abe73a gnu: Remove fraggenescan.
001dcfe351 gnu: Remove python2-olefile.
f313590b9c gnu: Remove python2-pillow.
46c19049ce gnu: Remove python2-seaborn.
c94cc00653 gnu: Remove python2-iso8601.
cec9ad5412 gnu: Remove python2-pytz.
9923879424 gnu: Remove python2-pretend.
0f3106740f gnu: Remove python2-idna.
60999d1cec gnu: Remove python2-backport-ssl-match-hostname.
f90fcfa33d gnu: Remove python2-scandir.
067338674a gnu: Remove python2-ipaddress.
ecde6fb0f3 gnu: Remove python2-asn1crypto.
acc9331f64 gnu: Remove python2-cryptography-vectors.
d6e92d91e4 gnu: Remove python2-cffi.
80cd6bd92b gnu: Remove python2-coverage.
34632df3ad gnu: Remove python2-dateutil.
4288a209a0 gnu: Remove python2-contextlib2.
a9352dfd8c gnu: Remove python2-typing.
4553ce8c44 gnu: Remove python2-pathlib2.
f31fe6b464 gnu: Remove python2-cryptography.
8c271b7b36 gnu: Remove python2-flaky.
1b11565908 gnu: Remove python2-backports-functools-lru-cache.
806dee732c gnu: Remove python2-pysocks.
15f7284ca1 gnu: Remove python2-importlib-resources.
37e6c0cce1 gnu: Remove python2-configparser.
ff7906505e gnu: Remove python2-unidecode.
2e1f0db2ef gnu: Remove python2-pyopenssl.
5a553edb4b gnu: Remove python2-certifi.
45bb3af633 gnu: Remove python2-zipp.
1762727626 gnu: Remove python2-packaging-bootstrap.
e0d0dda357 gnu: Remove python2-freezegun.
01fbe51039 gnu: Remove python2-wxpython.
c4eecc10bb gnu: Remove python2-sphinxcontrib-websupport.
3c2fce40af gnu: Remove python2-sphinx-alabaster-theme.
80e4e36445 gnu: Remove python2-pyicu.
d63d7f941a gnu: Remove python2-importlib-metadata.
7736e3b727 gnu: Remove python2-atomicwrites.
1fa434063f gnu: Remove python2-soupsieve.
7e6555826e gnu: Remove python2-pylev.
4ea894581c gnu: Remove python2-simplejson.
c2c6b933bc gnu: Remove python2-pyaml.
803f6252e4 gnu: Remove python2-levenshtein.
2f5ca3eb6f gnu: Remove python2-babel.
fbe609158c gnu: Remove python2-imagesize.
3624fd07d4 gnu: Remove python2-chardet.
1fe6e8fe45 gnu: Remove python2-markupsafe.
1484188321 gnu: Remove python2-urllib3.
e4acbdffcb gnu: Remove python2-webencodings.
315b565c1e gnu: Remove python2-pastel.
8ce7494431 gnu: Remove python2-vobject.
0dc76a6463 gnu: Remove python2-twodict.
1f82af14b5 gnu: Remove python2-jinja2.
df863c18f5 gnu: Remove python2-diff-match-patch.
64c7814b47 gnu: Remove python2-urwid.
a37c26a173 gnu: Remove python2-backpack.
bc30e08839 gnu: Remove python2-packaging.
08216338a6 gnu: Remove python2-snowballstemmer.
eb77c6aa62 gnu: Remove python2-funcsigs-bootstrap.
56e84700e2 gnu: Remove python2-clikit.
32b1f9f190 gnu: Remove python2-beautifulsoup4.
fa9c9305a2 gnu: Remove python2-decorator.
1b34fff0e4 gnu: Remove python2-docopt.
b9a8dd6a5e gnu: Remove python2-bottle.
6870c2c22c gnu: Remove python2-requests.
7387002a45 gnu: Remove python2-zinnia.
76ed5d930b gnu: Remove python2-pycairo.
4dbde1c79e 

Re: auth-tarball-from-git

[zimoun]

Hi,

On lun., 30 mai 2022 at 01:45, kias...@disroot.org wrote:

> Authenticate a tarball through a signed tag in a git repository (with 
> reproducible builds).
>
> Blog post: https://vulns.xyz/2022/05/auth-tarball-from-git/

In this post, it reads:

Personally - if I had to decide between these two - I’d prefer
the later because I can always try to authenticate the pinned
tarball later on, 

which is the case for Guix.  Even, Guix is somehow already implementing
this third way because each commit modifying a package is signed.  From
the post, the file ’chrisduerr.pgp’ and ’kchibisov.pgp’ are the Guix
committer keys [1].

(Hum, I do not understand what this means:

but it’s impossible to know for sure which source code has been
used if all I know is “something that had a valid signature on
it”.

but that’s another story. :-))


Well, in this frame about security, the question is: who trusts who?  I
do not think that the addition of an automatic signature check of source
code’s author key enforces more security for Guix users.  It adds more
complexity and does not fix the current bottleneck of trust: the Guix
packager and Guix committer.

Other said, if you do not trust the Guix packagers and Guix committers,
then you have to personally check the authenticity of the source code.
Using an automatic process with data from Guix packages or Guix
committers contradicts the assumption «you do not trust them».
Therefore, it does not fix the current weakness.

However, such ’auth-tarball-from-git’ can be of high interest when
Submitting Patching [2]:

2. If the authors of the packaged software provide a
cryptographic signature for the release tarball, make an effort
to verify the authenticity of the archive. For a detached GPG
signature file this would be done with the gpg --verify command.


Cheers,
simon


1: 
2: 

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Efraim Flashner]

On Sat, Apr 02, 2022 at 05:18:55PM +0200, Maxime Devos wrote:
> Hi,
> 
> antioxidant-build-system can now be used for some ‘real’ software -- it
> compiles 'hexyl'.  To test, download
>  (commit:
> d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
> "guix build -L . -f guix.scm" and then
> 
> $ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl
> > lots of coloured hex output
> 
> Some features of antioxidant-build-system:
> 
>   * no copying source code of dependencies
>   * no compiling dependencies again -- old artifacts are reused
>   * all dependencies use the usual package input system
> (native-inputs, inputs, propagated-inputs)
> 
> Limitations:
> 
>   * no support for linking to arbitrary shared libraries yet
> (only rust deps)
>   * makes a few assumptions on the source layout (can be fixed
> by using more info from Cargo.toml)
>   * no tests
>   * no cross-compilation yet
>   * no shared libraries (just replacing 'rlib' by 'dylib' causes problems)
>   * code is a bit messy
>   * no cdylib yet (probably needed for librsvg)

Something that might help with that would be to also include the source
of the crate somewhere in the output. Then at the worst we could just
put the rust inputs of librsvg as regular inputs and let librsvg do its
own special build thing. We'd then still keep the build dependency tree
you've got working.


-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature