Re: Building more of ‘core-updates’ on ci.guix

[Simon Tournier]

Hi,

On sam., 08 avril 2023 at 12:28, Josselin Poiret  wrote:

> Should we organize a code sprint soon to bring the community together
> and try and get this finally merged?  I can take care of sending a mail
> to guix-devel, as long as we have enough helping hands for the fateful
> day(s).

Nice idea!

I would suggest to send an email to guix-devel explaining how to
reconfigure using ’core-updates’ for testing and then report the
potential issues.

As an example:

--8<---cut here---start->8---
$ guix time-machine --branch=core-updates -- weather
computing 22 758 package derivations for x86_64-linux...
looking for 23 843 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org ☀
  85.2% substitutes available (20 310 out of 23 843)
  at least 99 872,4 MiB of nars (compressed)
  161 269,2 MiB on disk (uncompressed)
  0,005 seconds per request (121,3 seconds in total)
  196,6 requests per second

  27.6% (976 out of 3 533) of the missing items are queued
  at least 1 000 queued builds
  aarch64-linux: 32 (3.2%)
  powerpc64le-linux: 13 (1.3%)
  x86_64-linux: 955 (95.5%)
  build rate: 508.47 builds per hour
  x86_64-linux: 485.98 builds per hour
  i686-linux: 62.03 builds per hour
looking for 23 843 store items on https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org ⛈
  35.5% substitutes available (8 467 out of 23 843)
  at least 17 312,7 MiB of nars (compressed)
  62 000,0 MiB on disk (uncompressed)
  0,010 seconds per request (239,3 seconds in total)
  99,7 requests per second
  (continuous integration information unavailable)
--8<---cut here---end--->8---

Then, the missing from my manifest are:

--8<---cut here---start->8---
$ guix time-machine --branch=core-updates \
   -- weather -m ~/.config/guix/manifests/emacs.scm --display-missing

[...]

Substitutes are missing for the following items:
  /gnu/store/qwm340cgp45w231jzhgs131vv2y4a8d6-ocaml-yojson-2.0.2
x86_64-linux
  /gnu/store/3d4n5l41vvcqby9jsrwf582n5mqss3di-ocaml-merlin-4.7-414  
x86_64-linux
  /gnu/store/wc2srqbis7nyn0kbx5z3lv17xncr7wvc-ocaml-uuseg-15.0.0
x86_64-linux
  /gnu/store/8ij1pgrjrnzs8nzhq7c8xy1fvlyh1fkk-ocaml-uucp-15.0.0 
x86_64-linux
  /gnu/store/rm0c052hajwypds3dfmz3dbhgxsx7bwi-ocaml-uutf-1.0.3  
x86_64-linux
  /gnu/store/92sq9lr2y5k1zmfxj2z1ab2apzmhz1cf-ocaml-zed-3.2.1   
x86_64-linux
  /gnu/store/h8v335ykhqkmcdv329w2wr0m3fpqny26-ocaml-odoc-2.2.0  
x86_64-linux
  /gnu/store/srnhncz8lr55q677bqpnhfsla59cbh42-ocaml-mew-0.1.0   
x86_64-linux
  /gnu/store/mrz4hihvdmf67rdpph821881fw7lz3rl-ocaml-mew-vi-0.5.0
x86_64-linux
  /gnu/store/dlsqm5yw7l8a33flr9r11k58ywk57s8x-ocaml-react-1.2.2 
x86_64-linux
  /gnu/store/fk3hyh3lr145z6hglbpy3hph13lqw0hb-ocaml-lwt-react-1.2.0 
x86_64-linux
  /gnu/store/km6lqkbrxssmf3h7ryy01ab72z5a30ry-ocaml-topkg-1.0.6 
x86_64-linux
  /gnu/store/ddff0c1zb3nkfns72c1xg87hs6kyqcag-ocaml-mtime-2.0.0 
x86_64-linux
  /gnu/store/9vz8xard511cz04cpfnvgvia3r5avx2s-ocaml-fmt-0.9.0   
x86_64-linux
  /gnu/store/lbfazpx5fgk6jabas4pyvibk3wp4y1qv-ocaml-logs-0.7.0  
x86_64-linux
  /gnu/store/h899y7ah0a1rxls005xsjz7zzynrxj2h-ocaml-lambda-term-3.3.1   
x86_64-linux
  /gnu/store/l2mg99br25hsmk2w5h6hxlcch95py3hp-ocaml-utop-2.10.0 
x86_64-linux
--8<---cut here---end--->8---

Well, from https://ci.guix.gnu.org/ let click on the “screen”
(dashboard) under Action and then in the search bar, let type “ocaml-”
thus it display all the red.  Let pick one red: ocaml-angstrom-0.15.0,
then “Show more“ then the red cross and we are at ocaml-ppxlib-0.25.1.
Let click to ‘raw’, bottom, the error seems:

Re: core-updates sprint (was: Building more of ‘core-updates’ on ci.guix)

[Andreas Enge]

Am Sat, Apr 08, 2023 at 12:28:07PM +0200 schrieb Josselin Poiret:
> Should we organize a code sprint soon to bring the community together
> and try and get this finally merged?  I can take care of sending a mail
> to guix-devel, as long as we have enough helping hands for the fateful
> day(s).
> If so, would next week-end (15-16 april) work?

Excellent idea! I will be available one of the days (tentatively Sunday,
but it will depend on the weather as I would like to go to a gardening
faire ;-)) I suppose we would then coordinate on the #guix channel on IRC?

The powerpc bug has apparently been fixed by one of the latest commits,
so it makes sense to advance. aarch64 is in very bad shape (maybe due to
a lack of build power), and i686 could also do better. But I think we have
a basis for substantial progress.

Andreas

Re: Building more of ‘core-updates’ on ci.guix

[Josselin Poiret]

Hi Andreas and everyone,

Andreas Enge  writes:

> Ludovic and Chris have corrected a problem in dune.scm in commit
>01d9859821c5df6cc76c59e48fddd3c8cfa88ff0
> Now all of core-updates is being built, and the dashboard is back.
> Thanks a lot!
>
> This means that people can now work on repairing their favourite missing
> package, while having substitutes available from berlin (there are also
> some on bordeaux).

Should we organize a code sprint soon to bring the community together
and try and get this finally merged?  I can take care of sending a mail
to guix-devel, as long as we have enough helping hands for the fateful
day(s).

If so, would next week-end (15-16 april) work?

Best,
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Building more of ‘core-updates’ on ci.guix

[Andreas Enge]

Hello all,

Am Fri, Mar 31, 2023 at 10:54:23AM +0200 schrieb Andreas Enge:
> > Well, it does not seem to work.
> >https://ci.guix.gnu.org/
> > shows "all" under build, but
> >https://ci.guix.gnu.org/jobset/core-updates
> > has a few evaluations made after the change that do not take it into
> > account. For instance, 341091 on commit 35b208c shows no new builds.
> > Help welcome, I have no idea what is happening!
> It looks like since my change to "all" of core-updates, the dashboard
> has also disappeared; for instance I see no dots in
>https://ci.guix.gnu.org/eval/341091/dashboard

Ludovic and Chris have corrected a problem in dune.scm in commit
   01d9859821c5df6cc76c59e48fddd3c8cfa88ff0
Now all of core-updates is being built, and the dashboard is back.
Thanks a lot!

This means that people can now work on repairing their favourite missing
package, while having substitutes available from berlin (there are also
some on bordeaux).

The situation looks quite good for x86_64, and a bit less so on i686.
The aarch64 dashboard is completely red, due to an early build failure
by timeout of gcc.

powerpc is also completely red due to an early build failure in libstdc++,
see the last few lines of
   https://ci.guix.gnu.org/build/707624/log/raw
and this bug report:
   http://issues.guix.trop.in/61879

Andreas

Re: Building more of ‘core-updates’ on ci.guix

[Andreas Enge]

Am Thu, Mar 30, 2023 at 01:21:22PM +0200 schrieb Andreas Enge:
> Well, it does not seem to work.
>https://ci.guix.gnu.org/
> shows "all" under build, but
>https://ci.guix.gnu.org/jobset/core-updates
> has a few evaluations made after the change that do not take it into
> account. For instance, 341091 on commit 35b208c shows no new builds.
> Help welcome, I have no idea what is happening!

Ping! Without substitutes now further down the build graph, it becomes
difficult to work on missing pieces (if there are any left).

It looks like since my change to "all" of core-updates, the dashboard
has also disappeared; for instance I see no dots in
   https://ci.guix.gnu.org/eval/341091/dashboard

Andreas

Re: Building more of ‘core-updates’ on ci.guix

[Andreas Enge]

Am Thu, Mar 30, 2023 at 11:20:14AM +0200 schrieb Andreas Enge:
> With staging built and the build farm being quite idle, I have come to the
> same conclusion, so I have modified the specification to build everything,
> and (hopefully) restarted the evaluation.

Well, it does not seem to work.
   https://ci.guix.gnu.org/
shows "all" under build, but
   https://ci.guix.gnu.org/jobset/core-updates
has a few evaluations made after the change that do not take it into
account. For instance, 341091 on commit 35b208c shows no new builds.

Help welcome, I have no idea what is happening!

Andreas

Re: Building more of ‘core-updates’ on ci.guix

[Andreas Enge]

Am Wed, Mar 22, 2023 at 03:32:07PM +0100 schrieb Ludovic Courtès:
> Yes, it looks like we can safely uncomment icecat in ‘%system-packages’.

Done!

> Now, maybe we’ve reached the point where we can get ci.guix to build all
> of ‘core-updates’?

With staging built and the build farm being quite idle, I have come to the
same conclusion, so I have modified the specification to build everything,
and (hopefully) restarted the evaluation.

Andreas

Re: Building more of ‘core-updates’ on ci.guix

[Ludovic Courtès]

Hi,

Andreas Enge  skribis:

> Would it be okay to add the following packages to etc/release-manifests.scm:
> openjdk
> unison (exercises ocaml)
> ghc-pandoc (exercises ghc)
> calibre (exercises Qt5 and python-pyqt)

I would maybe not add them to ‘etc/release-manifest.scm’ because that
goes beyond the original goal of that manifest, which is to ensure that
essential substitutes are available (see “make
assert-binaries-available”).

Now, maybe we’ve reached the point where we can get ci.guix to build all
of ‘core-updates’?

> icecat is supposed to be added as soon as rust is available on i686,
> but I suppose we could add it conditionally only on x86_64, no?
>
> Actually there is already a filter step in %system-manifest.
> The same holds for ungoogled-chromium. It should be enough to just
> add them.

Yes, it looks like we can safely uncomment icecat in ‘%system-packages’.

Thanks,
Ludo’.

Re: Building more of ‘core-updates’ on ci.guix

[Andreas Enge]

Hello!

Am Wed, Mar 15, 2023 at 05:47:08PM +0100 schrieb Ludovic Courtès:
> I’ve just changed it the ‘core-updates’ job to build
> ‘etc/release-manifests.scm’ (you can check what’s in there).  So
> everything goes well (a big “if” :-)), we’ll soon have substitutes for
> Emacs, GTK, and whatnot.

thanks a lot, also for the explanation on what to do over ssh.

Would it be okay to add the following packages to etc/release-manifests.scm:
openjdk
unison (exercises ocaml)
ghc-pandoc (exercises ghc)
calibre (exercises Qt5 and python-pyqt)

icecat is supposed to be added as soon as rust is available on i686,
but I suppose we could add it conditionally only on x86_64, no?
Actually there is already a filter step in %system-manifest.
The same holds for ungoogled-chromium. It should be enough to just
add them.

Andreas

Building more of ‘core-updates’ on ci.guix

[Ludovic Courtès]

Hello!

Andreas Enge  skribis:

> So it would be nice if someone could set up a more complete job for
> core-updates on cuirass or QA, and maybe write up a how-to to see which
> packages work and which ones need more love, preferably by architecture.

I’ve just changed it the ‘core-updates’ job to build
‘etc/release-manifests.scm’ (you can check what’s in there).  So
everything goes well (a big “if” :-)), we’ll soon have substitutes for
Emacs, GTK, and whatnot.

For the record, anyone with (1) SSH access to berlin, or (2) a “TLS user
certificate” for use by Cuirass¹ can do it.  For method #1, set up a
tunnel to the Cuirass web server, like so:

  ssh -L 8081:localhost:8081 berlin.guix.gnu.org

Then visit , click on “Edit” in the vegan-burger
menu on the ‘core-updates’ line, adjust accordingly, and save.  (You can
see that form at
, you can can’t
submit changes.)

HTH!

Ludo’.

¹ 
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/release.org#n205