Re: [rb-general] Quick reproducible test for GNU Guix

[Holger Levsen]

Hi Vagrant,

On Fri, Feb 07, 2020 at 03:08:59PM -0800, Vagrant Cascadian wrote:
> I did some quick reproducibility testing running GNU Guix, and so far
> got pretty good results:
> 
> Using guix (and packages) built from commit:
>   f83d07f7778b699d46741a5667113342f5f0a737
> 
> $ guix challenge --verbose --diff=diffoscope ...
> 2,463 store items were analyzed:
>   - 2,016 (81.9%) were identical
>   - 37 (1.5%) differed
>   - 410 (16.6%) were inconclusive

thanks for sharing these results, very nice! I've added this to the February
report and would be happy to receive future updates on this as well as setup
some continuous testing!


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: [rb-general] Quick reproducible test for GNU Guix

[Christopher Baines]

Ludovic Courtès  writes:

>> Eventually, I'd like to do more systematic test of guix packages, with
>> published logs per-package, rather than whatever I happened to build on
>> the system so far, but this was a quick start to help flesh out ideas
>> for feature requests to "guix challenge" to make this all easier... more
>> on that soon!
>
> That’d be great!
>
> Related to that, Christopher Baines developed a nice feature for the
> Guix Data Service during and after the summit: for each Guix revision,
> there’s a page showing the overall package reproducibility status based
> on the info obtained from our two independent build farms.  The URL is
> something like
> 
> (right now it’s empty for some reason, but normally it shows
> identical/different/inconclusive percentages.)

Hey,

This should be back working now. I changed how cross derivations are
stored and handled, and that broke this page. It's now back showing at
least something, but there's still some work to do before it's showing
representative data.

Chris


signature.asc
Description: PGP signature

Re: [rb-general] Quick reproducible test for GNU Guix

[Ludovic Courtès]

Hi Vagrant!

Vagrant Cascadian  skribis:

> I did some quick reproducibility testing running GNU Guix, and so far
> got pretty good results:
>
> Using guix (and packages) built from commit:
>
>   f83d07f7778b699d46741a5667113342f5f0a737
>
> $ guix challenge --verbose --diff=diffoscope ...
> 2,463 store items were analyzed:
>   - 2,016 (81.9%) were identical
>   - 37 (1.5%) differed
>   - 410 (16.6%) were inconclusive

Woow, nice!  There are 12+K packages now, but at least that gives us a
good overview of the story for core packages.

> The log file produced was 951MB compressed with lz4, including
> diffoscope output.

Still reading it?  ;-)

> At ~82% identical, that's not too bad. The ~17% inconclusive are likely
> substitutes that haven't yet built or failed to build on the substitute
> server, or some local builds that are not yet in guix (e.g. WIP builds
> of reprotest!!). I wonder if "guix challenge" may compare some things
> that are normally only built locally (e.g. grub.cfg or similar ?).

Yes, “guix challenge” without any arguments compares all the items in
your store, which includes file specific to your machine, like
‘grub.cfg’, the initrd, etc.

If you want to focus on packages, you can run something like:

  guix challenge $(guix package -A | cut -f1)

> I did notice that when doing multiple builds in parallel, it uses a
> different user for each build, though I don't know if that's normalized
> within the build environment container. If not, that could fix quite a
> few issues! See issues mentioning "user" at:
>
>   
> https://salsa.debian.org/reproducible-builds/reproducible-notes/blob/master/issues.yml

The user name in the build environment is canonicalized:

  https://git.savannah.gnu.org/cgit/guix.git/tree/nix/libstore/build.cc#n1858

> Eventually, I'd like to do more systematic test of guix packages, with
> published logs per-package, rather than whatever I happened to build on
> the system so far, but this was a quick start to help flesh out ideas
> for feature requests to "guix challenge" to make this all easier... more
> on that soon!

That’d be great!

Related to that, Christopher Baines developed a nice feature for the
Guix Data Service during and after the summit: for each Guix revision,
there’s a page showing the overall package reproducibility status based
on the info obtained from our two independent build farms.  The URL is
something like

(right now it’s empty for some reason, but normally it shows
identical/different/inconclusive percentages.)

Thanks,
Ludo’.