Re: Merging wip-guix-home to master

[Andrew Tropin]

On 2021-09-29 13:39, Xinglu Chen wrote:

> On Tue, Sep 28 2021, Andrew Tropin wrote:
>
>> On 2021-09-24 15:38, Xinglu Chen wrote:
>>
>>> On Thu, Sep 23 2021, Andrew Tropin wrote:
>>>
 The core part of Guix Home project has been moved from rde
 repository[fn:1] to wip-guix-home branch of guix repository.

 I'm about a week on wip-guix-home branch completely and Guix Home works
 fine.  There are no any major issues on rde-devel and guix-devel mailing
 lists and it seems that branch is ready to be merged.

 My guix describe looks like:
 --8<---cut here---start->8---
 Generation 114 Sep 17 2021 13:33:55(current)
   rde 31f8003
 repository URL: https://git.sr.ht/~abcdw/rde
 branch: without-guix-home
 commit: 31f800353a781cef25fc80c05ad824a068a049c8
   guix a2324d8
 repository URL: https://git.savannah.gnu.org/git/guix.git
 branch: wip-guix-home
 commit: a2324d8b56eabf8117bca220a507cc791edffd2e
 --8<---cut here---end--->8---


 There is a discussion[fn:2] on moving home services to (gnu services
 ...)  modules, which is likely to happen, but it's possible to do the
 migration relatively painless by re-exporting necessary symbols in
 (gnu home-services ...) at first and removing them completely later.

 Another important part of the work related to Guix Home project is
 covering related modules and cli with tests, but it can be done in
 parallel and is not a blocker for merging.
>>>
>>> I noticed that the ‘guix home import’ subcommand is included, but I
>>> think it needs more thought and feedback from people before it makes its
>>> way into ‘master’; it also seems to lack documentation.
>>>
>>> I just realized that it generates the following service declaration
>>>
>>> --8<---cut here---start->8---
>>> (service
>>>  home-bash-service-type
>>>  (home-bash-configuration
>>>   (bashrc
>>>(list (slurp-file-gexp
>>>   (local-file "/home/yoctocell/.bashrc"))
>>> --8<---cut here---end--->8---
>>>
>>> but when running ‘guix home reconfigure’, the ~/.bashrc file will be
>>> moved, so when running ‘guix home reconfigure’ for the second time, it
>>> would read the ~/.bashrc which is itself a symlink to a file the store.
>>> ‘guix home import’ clearly isn’t in a usable state as of right now…
>>
>> Yep, I remember that it is not documented.  I think it's ok for
>> generating a simple sample configuration,

It was two independent sentences.

>
> I disagree, I think it’s OK for things like (guix git), which are mainly
> used by developers, to not be documented in the manual.  Ideally, the
> all the modules would be documented.  :-)
>
> However, ‘guix home import’ will be used by developers and regular
> users, and documenting the command should be a requirement.
>
>> but I agree that it's not yet complete, if you wish I'll disable it in
>> cli.
>
> I have sent a few patches to fix it and document it[1], so it should
> probably left as-is for now.
>
>> Just an idea for the future: it's probably better to copy .bashrc to the
>> current directory and do (local-file "./bashrc").
>
> This wouldn’t work if the user doesn’t have write-access to the current
> directory, nor if the current directory is $HOME.  I think it’s better
> for the user to specify the directory themselves.
>
> [1]: 


signature.asc
Description: PGP signature

Re: Go importer and packages with version flags

[Sarah Morgensen]

Katherine Cox-Buday  writes:

> Sarah Morgensen  writes:
>
>> Hi Katherine, Jack,
>>
>> Katherine Cox-Buday  writes:
>>
>>> Jack Hill  writes:
>>>
 Hi Guix,
>>>
>>> Hey, Jack, a few thoughts.
>>>
 While I was was working with the go importer today, it suggested I package
 go-github-com-russross-blackfriday-v2. Fair enough, except we already have 
 a
 package for go-github-com-russross-blackfriday.
>>>
>>> I was poking around a rust code-base the other day and I noticed our crate
>>> importer, and thus a lot of crate packages, have major-version suffixes. I
>>> think one of the unique benefits of Guix is that it can simultaneously have
>>> multiple versions of libraries installed, and I think we should allow this
>>> for library packages.
>>>
>>> I know that leads to dependency graph explosion, but perhaps we only commit
>>> to substitutes for the latest version, and thus any packages using old
>>> versions. It should converge over time unless packages go unmaintained.
>>>
>>> I thought our current stance was to only allow one version at a time, but
>>> the crate packages made me question this. I'd like clarity too.
>>
>> I think there's a bit of a difference between (our packages for) the Rust and
>> Go ecosystems here.
>>
>> In the Go ecosystem, a module is uniquely identified by its module path,
>> e.g. "github.com/russross/blackfriday/v2".  According to Go's major version
>> suffix rules [0], "[s]tarting with major version 2, module paths must have a
>> major version suffix like /v2 that matches the major version."  Therefore,
>> each major version is logically a different module according to Go, and so I
>> think we should treat them as separate packages as well.  (Note that in many
>> cases we can use 'inherit' for brevity.)
>
> That's a great point! I hadn't considered that we could leverage this to
> consider major versioned Go modules as separate packages. That's great!
>
>> Additionally, the major version suffix rules dictate that "[i]f an old 
>> package
>> and a new package have the same import path, the new package must be 
>> backwards
>> compatible with the old package."  Assuming upstream sources follow the 
>> rules,
>> we should be able to update each Go package within each major version without
>> breaking dependencies.
>>
>> (A corollary to that is that packages often break if you try to use a v2 when
>> it is expecting a v1.)
>>
>> I think this differs from Rust, where we have, for example, package-0.1 and
>> package-0.2 coexisting.  This difference should prevent dependency graph
>> explosion for Go.
>
> It's nice that our Rust packages are enjoying the same stance, but I'm still
> not clear on why? Does Rust have the same guarantees?

I'll leave this for someone who actually knows the Rust ecosystem to answer :)

>
>> There are some caveats with "major version suffixes":
>>
>> * Major versions 0 and 1 don't get a version suffix (so no /v1)...
>>
>> * ...except for module paths starting with "gopkg.in/", which always have
>>   a major version suffix, but theirs is styled ".v1" rather than "/v1".
>>
>> * Modules may either be located in the repository root, or in a "/v2"
>>   subdirectory (for major version 2).  This makes things difficult for our
>>   importer, because we can't know whether the unpack path should include 
>> "/v2"
>>   without looking at the repository contents.
>>
>> This is why Jack had to manually add "/v2" to the unpack path.  I recently
>> made some changes to the importer to better handle, for example,
>> "github.com/example/repository/subproject", but it doesn't yet discriminate
>> between "/subproject" and "/v2", so it treated "/v2" like a subdirectory of
>> the repository.  (Until we fix this properly, the importer should probably 
>> not
>> count major version suffixes when calculating the unpack path, since most
>> projects don't use a "/v2" subdirectory.)
>
> As an aside, when I started writing the importer, I didn't know it was a
> possibility to just use the Go toolchain to help us generate packages. I
> thought "the Guix way" was to do it all in scheme. It's nice that it's in
> scheme, but I would love to leverage the Go toolchain instead.
>
> IMO, module resolution and graph dependencies in Go are complicated enough
> that I'd much rather take the effort we put in trying to replicate and keep
> up with the Go toolchain's behavior, and spend that effort elsewhere.
>
> Does anyone have opinions on this?

Hmmm.  Setting aside whether or not we want to use external tools in
general...

If we use the Go tool, we shift the problem domain into "translating between
`go' and Guix."

For example: when Go downloads a module, it only saves the module, not the
whole repository*, so we can't use that to generate the hash.  (* Except it
does if you used GOPROXY=direct, but that's in the module cache, and only a
bare repository.)

Or, the fact that we import the latest available version of packages (unless
--pin-versions is used), but Go use

Re: core-updates-frozen: Planning for the last world rebuild

[Thiago Jung Bauermann]

Hello Ludo,

Em terça-feira, 28 de setembro de 2021, às 08:57:45 -03, Ludovic Courtès 
escreveu:
> Hello Guix!
> 
> Wouldn’t it be nice to get ‘core-updates-frozen’ merged?  :-)
> 
> What about merging the patches blocked by
> ?  We could even have them on a
> separate branch, have Cuirass build it, and merge it 24–36h later, at
> which point substitute coverage should be reasonably good.
> 
> I haven’t followed the new ports, POWER9 and RISC-V.  If there’s
> anything needed in this area, like localized toolchain changes, please
> send a heads-up!

There are two things I’m aware of for POWER9:

• diffutils test fix: https://issues.guix.gnu.org/50239
• gcc-5 build with GCC 10 fix: https://issues.guix.gnu.org/49896

In the case of diffutils, IMHO ideally we would apply the v2 patch there for 
the world rebuild, but as Ludo suggested it’s also possible to make the 
change only for powerpc64le-linux and avoid affecting other platforms.

In the case of gcc-5, it’s the same situation actually. But in addition to 
that, `guix refresh --list-dependent` says that modifying gcc-5 would only 
cause 32 dependent packages to be rebuilt so it’s not necessary to address 
it for the world rebuild.

> As I say every time someone proposes a new port, the main issue is not
> the port itself but maintaining it longer-term.  And here it’s not even
> clear to me what the status of POWER9 is on ‘core-updates-frozen’, so
> it’d be great to get feedback on this, ensure Cuirass builds it, and so
> on.

Yeah, the status of POWER9 on ‘core-updates-frozen’ is obfuscated by some 
build failures early in the dependency chain that were caused by QEMU bugs.
My impression is that the same is also true for aarch64-linux. For 
instance, QEMU can’t run libsigsegv’s testsuite (among other things), and 
that blocks a ton of stuff. Ditto for util-linux’s testsuite.

I was waiting for this world rebuild to happen to see the results in 
Cuirass now that powerpc64le-linux builds are only done on guixp9, but in 
hindsight it would have been better if I requested that those builds were 
restarted instead. Restarting this build should remove the main roadblock 
though:

https://ci.guix.gnu.org/build/703194/details

When that build is restarted and succeeds, does Cuirass also restart the 
builds that failed because they depended on it, or do we need to explicitly 
restart those as well?

What I can say is that IMHO POWER9 support isn’t too bad at least for the 
base packages given that I can build e.g., emacs (assuming that the patch 
for issue 50521 fixing a problem in gtk+-2 is applied), which depends on 
many base packages including X11 and GTK+.

-- 
Thanks,
Thiago

Getting GEM_PATH right for ruby scripts

[Stephen Paul Weber]

Hello everyone!

I've finally been diving deep into the Guix and doing a lot of packaging.  One
thing that has now come to my attention is the way ruby-build-system set
GEM_PATH for wrapped scripts.  It takes the whole GEM_PATH that is present at
build time.  This means that all those packages end up keeping store references
to build-only (indeed, unit-test-only!) dependencies such as rake and minitest.

Of course the problem is that the package specification does not have any way to
know which inputs are going to be needed past build time.  In the ruby packages,
convention is to use propogated-inputs for this, but even so the GEM_PATH we
need would be for all the implicitly propogated inputs.  I guess there should be
some way to get this, since it's what GEM_PATH will be in a profile that the
package is installed to?

I'm sure topics like this have been beaten to death, but I'd like to understand
how things are and if they can be improved as much as will fit in my head.

Thanks,


signature.asc
Description: PGP signature

Re: core-updates-frozen: Planning for the last world rebuild

[Mathieu Othacehe]

Hey!

> have more system tests passing. Currently 16 out of 87 are
> broken. Except for the nfs-root-fs that has always been broken, the
> other tests probably cover user configurations out there.

Made some progress here but there are still a bit more work needed:

* childhurd (https://ci.guix.gnu.org/build/901219/details)
 => gnumach build appears to be broken.

* dovecot (https://ci.guix.gnu.org/build/901189/details)
 => dovecot build appears to be broken.

* patchwork (https://ci.guix.gnu.org/build/901227/details)
 => python-django-pipeline build (dependency of patchwork) appears to be
  broken.

* ganeti-kvm (https://ci.guix.gnu.org/build/901158/details)
 => ganeti build appears to be broken.

* ganeti-lxc (https://ci.guix.gnu.org/build/901157/details)
 => same reason.

* getmail (https://ci.guix.gnu.org/build/901187/details)
 => dovecot build appears to be broken.

* gui-installed-desktop-os-encrypted
  (https://ci.guix.gnu.org/build/923078/details)
 => emacs-exwm build is broken presumably because of
  https://issues.guix.gnu.org/50066.

* jami-provisioning (https://ci.guix.gnu.org/build/901214/details) and
 jami (https://ci.guix.gnu.org/build/901213/details)
 => sobjectizer build appears to be broken.

* ldap (https://ci.guix.gnu.org/build/901179/details)
 => 389-ds-base build appears to be broken.

The overall status can be seen here:
https://ci.guix.gnu.org/eval/24933/dashboard. Let's get those tests fixed
so that we can start this branch on good grounds!

Thanks,

Mathieu

Re: Mailman packaging (was: Re: Python package naming: Dots vs hyphens)

[Christine Lemmer-Webber]

Efraim Flashner  writes:

> [[PGP Signed Part:Undecided]]
> On Wed, Nov 06, 2019 at 02:18:33PM +0100, pelzflorian (Florian Pelz) wrote:
>> Long e-mail / review follows.  Feel free to disagree.
>> 
>> On Wed, Nov 06, 2019 at 09:19:17AM +0200, Efraim Flashner wrote:
>> > On Wed, Nov 06, 2019 at 07:49:56AM +0100, pelzflorian (Florian Pelz) wrote:
>> > > I am in the process of packaging mailman 3 which according to the pypi
>> > > importer needs packages like
>> > > 
>> > > python-flufl.bounce
>> > > python-flufl.i18n
>> > > python-flufl.lock
>> > > python-lazr.config
>> > > python-lazr.delegates
>> > > python-zope.component
>> > > python-zope.configuration
>> > > python-zope.configuration
>> > > python-zope.event
>> > > python-zope.interface
>> > > 
>> > 
>> > I've actually last week started working on that. I've only done the
>> > python part, haven't searched for any javascript or font-awesome which
>> > I'm pretty sure I saw. I've also started working on a service for it
>> > which I haven't committed yet, but it looks like it's going to be
>> > complex.
>> > 
>> > https://gitlab.com/genenetwork/guix-bioinformatics/blob/master/gn/packages/mailman.scm
>> > 
>> 
>> Wow.  That is good.  You have the better setup and have packaged more
>> of the Mailman Suite so please continue with yours.  I too started
>> last week.  I have only looked at the python part of mailman core
>> myself.  Let me compare and complain about some of what I have done
>> differently.  Some things are clearly better your way. ;)  You have
>> come further.  I will watch your repo now and try and test other
>> mailman things tomorrow.

(A couple of years later...)

Has anyone worked on this or used it in recent times?  The above link no
longer works.  However I see:

  https://gitlab.com/genenetwork/guix-bioinformatics/-/tree/wip-mailman

... which hasn't seen activity for about a year, but seems promising?

Re: Merging wip-guix-home to master

[Xinglu Chen]

On Wed, Sep 29 2021, Katherine Cox-Buday wrote:

> Xinglu Chen  writes:
>
>> I disagree, I think it’s OK for things like (guix git), which are mainly
>> used by developers, to not be documented in the manual.
>
> I strongly disagree with this. As a long-time developer, I have used
> documentation both as a user and as a developer many times.
>
> Documentation is for everyone.

I said “to not be documented in the manual” (the majority of (guix …)
modules aren’t documented in the manual); that doesn’t mean that there
aren’t any docstrings.  The manual says that all top-level procedures
should have docstrings (see “16.5.4 Formatting Code”)

We require all top-level procedures to carry a docstring.  This
  requirement can be relaxed for simple private procedures in the ‘(guix
  build ...)’ name space, though.



signature.asc
Description: PGP signature

Re: Merging wip-guix-home to master

[Katherine Cox-Buday]

Xinglu Chen  writes:

> I disagree, I think it’s OK for things like (guix git), which are mainly
> used by developers, to not be documented in the manual.

I strongly disagree with this. As a long-time developer, I have used 
documentation both as a user and as a developer many times.

Documentation is for everyone.

-- 
Katherine

Re: Code sharing between system and home services (was Re: On the naming of System and Home services modules.)

[Maxime Devos]

Ludovic Courtès schreef op di 28-09-2021 om 14:21 [+0200]:
> Hi,
> 
> Joshua Branson  skribis:
> 
> > Apologies if I'm speaking for something I know very little
> > about...Wouldn't it be nice if guix home services would accept a user
> > and a group field?  For the syncthing service, perhaps the user wants to
> > limit Syncthing's runtime permissions.  So instead of running as the
> > user, the user would run synthing as a different user with less permissions?
> 
> That’s not possible unless the calling user is root, since you’d need
> the ability to switch users somehow.

On Debian, a user has a list of ‘subordinate user IDs’ which can be switched
to without root: 
.

Maybe "guix home" could use that mechanism, and this mechanism could be 
implemented
on Guix System as well?

Greetings,
Maxime


signature.asc
Description: This is a digitally signed message part

Re: Merging wip-guix-home to master

[Xinglu Chen]

On Tue, Sep 28 2021, Andrew Tropin wrote:

> On 2021-09-24 15:38, Xinglu Chen wrote:
>
>> On Thu, Sep 23 2021, Andrew Tropin wrote:
>>
>>> The core part of Guix Home project has been moved from rde
>>> repository[fn:1] to wip-guix-home branch of guix repository.
>>>
>>> I'm about a week on wip-guix-home branch completely and Guix Home works
>>> fine.  There are no any major issues on rde-devel and guix-devel mailing
>>> lists and it seems that branch is ready to be merged.
>>>
>>> My guix describe looks like:
>>> --8<---cut here---start->8---
>>> Generation 114  Sep 17 2021 13:33:55(current)
>>>   rde 31f8003
>>> repository URL: https://git.sr.ht/~abcdw/rde
>>> branch: without-guix-home
>>> commit: 31f800353a781cef25fc80c05ad824a068a049c8
>>>   guix a2324d8
>>> repository URL: https://git.savannah.gnu.org/git/guix.git
>>> branch: wip-guix-home
>>> commit: a2324d8b56eabf8117bca220a507cc791edffd2e
>>> --8<---cut here---end--->8---
>>>
>>>
>>> There is a discussion[fn:2] on moving home services to (gnu services
>>> ...)  modules, which is likely to happen, but it's possible to do the
>>> migration relatively painless by re-exporting necessary symbols in
>>> (gnu home-services ...) at first and removing them completely later.
>>>
>>> Another important part of the work related to Guix Home project is
>>> covering related modules and cli with tests, but it can be done in
>>> parallel and is not a blocker for merging.
>>
>> I noticed that the ‘guix home import’ subcommand is included, but I
>> think it needs more thought and feedback from people before it makes its
>> way into ‘master’; it also seems to lack documentation.
>>
>> I just realized that it generates the following service declaration
>>
>> --8<---cut here---start->8---
>> (service
>>  home-bash-service-type
>>  (home-bash-configuration
>>   (bashrc
>>(list (slurp-file-gexp
>>   (local-file "/home/yoctocell/.bashrc"))
>> --8<---cut here---end--->8---
>>
>> but when running ‘guix home reconfigure’, the ~/.bashrc file will be
>> moved, so when running ‘guix home reconfigure’ for the second time, it
>> would read the ~/.bashrc which is itself a symlink to a file the store.
>> ‘guix home import’ clearly isn’t in a usable state as of right now…
>
> Yep, I remember that it is not documented.  I think it's ok for
> generating a simple sample configuration,

I disagree, I think it’s OK for things like (guix git), which are mainly
used by developers, to not be documented in the manual.  Ideally, the
all the modules would be documented.  :-)

However, ‘guix home import’ will be used by developers and regular
users, and documenting the command should be a requirement.

> but I agree that it's not yet complete, if you wish I'll disable it in
> cli.

I have sent a few patches to fix it and document it[1], so it should
probably left as-is for now.

> Just an idea for the future: it's probably better to copy .bashrc to the
> current directory and do (local-file "./bashrc").

This wouldn’t work if the user doesn’t have write-access to the current
directory, nor if the current directory is $HOME.  I think it’s better
for the user to specify the directory themselves.

[1]: 


signature.asc
Description: PGP signature

Re: Linux-libre source code will be taken offline

[zimoun]

Hi,

On Tue, 28 Sept 2021 at 19:52, Jason Self  wrote:

> Yes, git://linux-libre.fsfla.org/releases.git which carries tagged
> releases, scripts, and logs. As you can see it goes all the way back to
> 2.6.21.

Thanks.

Does it make sense to switch from 'url-fetch' to 'git-fetch' in
linux-libre-deblob-scripts?  Modulo what the Vagrant's remark.

--8<---cut here---start->8---
(define (linux-libre-deblob-scripts version gnu-revision
deblob-hash
deblob-check-hash)
  (list (version-major+minor version)
(origin
  (method url-fetch)
  (uri (string-append "https://linux-libre.fsfla.org";
  "/pub/linux-libre/releases/" version "-"
gnu-revision "/"
  "deblob-" (version-major+minor version)))
  (file-name (string-append "linux-libre-deblob-"
version "-" gnu-revision))
  (sha256 deblob-hash))
(origin
  (method url-fetch)
  (uri (string-append "https://linux-libre.fsfla.org";
  "/pub/linux-libre/releases/" version "-"
gnu-revision "/"
  "deblob-check"))
  (file-name (string-append "linux-libre-deblob-check-"
version "-" gnu-revision))
  (sha256 deblob-check-hash
--8<---cut here---end--->8---


Cheers,
simon