Re: core-updates-frozen: Planning for the last world rebuild

[Ricardo Wurmus]

Mathieu Othacehe  writes:


* ldap (https://ci.guix.gnu.org/build/901179/details)
 => 389-ds-base build appears to be broken.


I fixed the build.  Also had to upgrade 389-ds-base to get around 
a linker error.


--
Ricardo

Could the Go importer use the Go toolchain? (was Re: Go importer and packages with version flags)

[Katherine Cox-Buday]

Sarah Morgensen  writes:

>> As an aside, when I started writing the importer, I didn't know it was a
>> possibility to just use the Go toolchain to help us generate packages. I
>> thought "the Guix way" was to do it all in scheme. It's nice that it's in
>> scheme, but I would love to leverage the Go toolchain instead.
>>
>> IMO, module resolution and graph dependencies in Go are complicated enough
>> that I'd much rather take the effort we put in trying to replicate and keep
>> up with the Go toolchain's behavior, and spend that effort elsewhere.
>>
>> Does anyone have opinions on this?
>
> Hmmm.  Setting aside whether or not we want to use external tools in
> general...
>
> If we use the Go tool, we shift the problem domain into "translating between
> `go' and Guix."
>
> For example: when Go downloads a module, it only saves the module, not the
> whole repository*, so we can't use that to generate the hash.  (* Except it
> does if you used GOPROXY=direct, but that's in the module cache, and only a
> bare repository.)

We could use the ~GOMODCACHE~ environment variable[0] to specify where these 
files are placed.

> Or, the fact that we import the latest available version of packages (unless
> --pin-versions is used), but Go uses exact versions selected with MVS [0].

Won't the importer import the import path you specify? The Go toolchain can 
also specify a ~@latest~ "version" to fetch the latest.

> You might also be interested in taking a look at Gentoo dealing with this
> translation problem [1].
>
> I'm not saying that this would necessarily be a bad tradeoff either, but it's
> definitely a tradeoff.

Agreed. I think the problem space of managing the translation between the 
toolchain and Guix is probably both smaller and less volatile.

I.e., the toolchain operations that dump dependency graph type stuff tend to 
support JSON output, and the download bits have options for where stuff gets 
stored on disk. As you know, the steps to go from an import path to a 
repository on-disk is complicated. If we could keep that complexity 
encapsulated upstream, which also happens to be the reference implementation, 
it could probably make the importer resilient to change and maybe even faster.

> Did you have something particular in mind as far as leveraging the Go tooling?

I haven't though too much about it, but ~GOPROXY=direct~ was my guess too. 
Thinking about this, would a shallow fetch into a bare repository be so bad?

It made me start wondering why our ~git-download~ type /doesn't/ have this 
behavior. Are we unnecessarily pulling down the entire commit history with all 
tags and branches of a large repository?

I don't even know what the philosophy is being doing this or not. Isn't the 
only commitment a Guix package makes for software at a particular 
version/commit that the store contains that immutable version of the source, 
not the complete repository?

Also, Guix supports all kinds of download types, one of which is zip files. The 
rust importer generates packages that fetch ~.tar.gz~'s from https://create.io. 
When I started writing the importer, I specifically kept the packages it 
generated decoupled from Google's module cache, but to make the comparison: the 
Go importer could do the same thing as the Rust importer: download zips from 
https://proxy.golang.org.

If this line of reasoning is wrong, it also looks like the toolchain respects 
standard VCS configuration. I wonder if there's a way to manipulate the fetches 
so that they're not shallow or bare.

TLDR: A shallow fetch into a bare repository might actually be /ideal/! It's 
small, and still directly connected to the package's actual source.

> [0] 
>
> [1]  cmd/go: allow extraction of
> urls used to download dependencies
>
> --
> Sarah

[0] - https://golang.org/ref/mod#environment-variables
[1] - https://golang.org/ref/mod#private-module-repo-auth

-- 
Katherine

Re: core-updates-frozen: Planning for the last world rebuild

[Thiago Jung Bauermann]

Hello Mathieu,

Em quinta-feira, 30 de setembro de 2021, às 06:03:52 -03, Mathieu Othacehe 
escreveu:
> Hey Thiago,
> 
> > I was waiting for this world rebuild to happen to see the results in
> > Cuirass now that powerpc64le-linux builds are only done on guixp9, but
> > in hindsight it would have been better if I requested that those
> > builds were restarted instead. Restarting this build should remove the
> > main roadblock though:
> > 
> > https://ci.guix.gnu.org/build/703194/details
> 
> Now restarted!

Thank you! Unfortunately that build is now affected by an issue that I only 
noticed yesterday: powerpc64le-linux builds for core-updates-frozen get 
permanently stuck in the ‘Scheduled’ state, even though guixp9 is 
completely idle (as it is right now).

For example, this evaluation from Sep 17 has thousands of scheduled 
evaluations:

https://ci.guix.gnu.org/eval/19271?status=pending

This is also happening with at least one other platform as well. This 
evaluation from Sep 25 has a lot of scheduled evaluations for i586-gnu (as 
well as powerpc64le-linux):

https://ci.guix.gnu.org/eval/23187?status=pending

> > When that build is restarted and succeeds, does Cuirass also restart
> > the
> > builds that failed because they depended on it, or do we need to
> > explicitly restart those as well?
> 
> Yes, on successful build completion Cuirass looks if some builds that
> were marked as "failed-dependency" (failing because some of the
> dependencies are also failing) can be resumed.

That is awesome!

> That being said, Cuirass is not performing nicely these days because of:
> this issue https://issues.guix.gnu.org/48468.

Ah, ok. Thank you for the heads up.

-- 
Thanks,
Thiago

Re: core-updates-frozen: Planning for the last world rebuild

[Mathieu Othacehe]

Hey Thiago,

> I was waiting for this world rebuild to happen to see the results in 
> Cuirass now that powerpc64le-linux builds are only done on guixp9, but in 
> hindsight it would have been better if I requested that those builds were 
> restarted instead. Restarting this build should remove the main roadblock 
> though:
>
> https://ci.guix.gnu.org/build/703194/details

Now restarted!

>
> When that build is restarted and succeeds, does Cuirass also restart the 
> builds that failed because they depended on it, or do we need to explicitly 
> restart those as well?

Yes, on successful build completion Cuirass looks if some builds that
were marked as "failed-dependency" (failing because some of the
dependencies are also failing) can be resumed.

That being said, Cuirass is not performing nicely these days because of:
this issue https://issues.guix.gnu.org/48468.

Thanks,

Mathieu

Re: core-updates-frozen: Planning for the last world rebuild

[Efraim Flashner]

On Tue, Sep 28, 2021 at 01:57:45PM +0200, Ludovic Courtès wrote:
> Hello Guix!
> 
> Wouldn’t it be nice to get ‘core-updates-frozen’ merged?  :-)
> 
> What about merging the patches blocked by
> ?  We could even have them on a
> separate branch, have Cuirass build it, and merge it 24–36h later, at
> which point substitute coverage should be reasonably good.
> 
> I haven’t followed the new ports, POWER9 and RISC-V.  If there’s
> anything needed in this area, like localized toolchain changes, please
> send a heads-up!
> 
> As I say every time someone proposes a new port, the main issue is not
> the port itself but maintaining it longer-term.  And here it’s not even
> clear to me what the status of POWER9 is on ‘core-updates-frozen’, so
> it’d be great to get feedback on this, ensure Cuirass builds it, and so
> on.

I'm still building away on riscv64-linux. I have some minor changes,
namely renaming target-riscv? to target-riscv64? As far as cuirass
support I have it officially unsupported in %hydra-supported-systems (or
whatever that variable is called). I've tried to keep the patches so
that they can be applied without interfering with the other
architectures. At this point as far as I know it's waiting for another
pair of eyes or so to say it looks OK to merge.

-- 
Efraim Flashner  רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Mailman packaging (was: Re: Python package naming: Dots vs hyphens)

[pelzflorian (Florian Pelz)]

Hello Christine,

On Wed, Sep 29, 2021 at 12:34:06PM -0400, Christine Lemmer-Webber wrote:
> Has anyone worked on this or used it in recent times?

Sorry no, it was too much for me.

> The above link no
> longer works.  However I see:
> 
>   https://gitlab.com/genenetwork/guix-bioinformatics/-/tree/wip-mailman
> 
> ... which hasn't seen activity for about a year, but seems promising?

Note that Mailman-core is not enough, you will need hyperkitty and an
MTA and such things.

There are other Django services in Guix by now (Patchwork); perhaps it
will be similar, I have not tried.

Regards,
Florian

Re: Merging wip-guix-home to master

[Andrew Tropin]

On 2021-09-29 13:39, Xinglu Chen wrote:

> On Tue, Sep 28 2021, Andrew Tropin wrote:
>
>> On 2021-09-24 15:38, Xinglu Chen wrote:
>>
>>> On Thu, Sep 23 2021, Andrew Tropin wrote:
>>>
 The core part of Guix Home project has been moved from rde
 repository[fn:1] to wip-guix-home branch of guix repository.

 I'm about a week on wip-guix-home branch completely and Guix Home works
 fine.  There are no any major issues on rde-devel and guix-devel mailing
 lists and it seems that branch is ready to be merged.

 My guix describe looks like:
 --8<---cut here---start->8---
 Generation 114 Sep 17 2021 13:33:55(current)
   rde 31f8003
 repository URL: https://git.sr.ht/~abcdw/rde
 branch: without-guix-home
 commit: 31f800353a781cef25fc80c05ad824a068a049c8
   guix a2324d8
 repository URL: https://git.savannah.gnu.org/git/guix.git
 branch: wip-guix-home
 commit: a2324d8b56eabf8117bca220a507cc791edffd2e
 --8<---cut here---end--->8---


 There is a discussion[fn:2] on moving home services to (gnu services
 ...)  modules, which is likely to happen, but it's possible to do the
 migration relatively painless by re-exporting necessary symbols in
 (gnu home-services ...) at first and removing them completely later.

 Another important part of the work related to Guix Home project is
 covering related modules and cli with tests, but it can be done in
 parallel and is not a blocker for merging.
>>>
>>> I noticed that the ‘guix home import’ subcommand is included, but I
>>> think it needs more thought and feedback from people before it makes its
>>> way into ‘master’; it also seems to lack documentation.
>>>
>>> I just realized that it generates the following service declaration
>>>
>>> --8<---cut here---start->8---
>>> (service
>>>  home-bash-service-type
>>>  (home-bash-configuration
>>>   (bashrc
>>>(list (slurp-file-gexp
>>>   (local-file "/home/yoctocell/.bashrc"))
>>> --8<---cut here---end--->8---
>>>
>>> but when running ‘guix home reconfigure’, the ~/.bashrc file will be
>>> moved, so when running ‘guix home reconfigure’ for the second time, it
>>> would read the ~/.bashrc which is itself a symlink to a file the store.
>>> ‘guix home import’ clearly isn’t in a usable state as of right now…
>>
>> Yep, I remember that it is not documented.  I think it's ok for
>> generating a simple sample configuration,

It was two independent sentences.

>
> I disagree, I think it’s OK for things like (guix git), which are mainly
> used by developers, to not be documented in the manual.  Ideally, the
> all the modules would be documented.  :-)
>
> However, ‘guix home import’ will be used by developers and regular
> users, and documenting the command should be a requirement.
>
>> but I agree that it's not yet complete, if you wish I'll disable it in
>> cli.
>
> I have sent a few patches to fix it and document it[1], so it should
> probably left as-is for now.
>
>> Just an idea for the future: it's probably better to copy .bashrc to the
>> current directory and do (local-file "./bashrc").
>
> This wouldn’t work if the user doesn’t have write-access to the current
> directory, nor if the current directory is $HOME.  I think it’s better
> for the user to specify the directory themselves.
>
> [1]: 


signature.asc
Description: PGP signature