Re: Reproducible Build Summit, Oct. 31st, Berlin

[Ricardo Wurmus]

Hey,

L> So, who’s in?  :-)

J> I am planning to go, sad you won't be there.

L> Awesome, really glad that you can attend—let the bootstrapper’s voice to
L> be heard!

I’m also planning to attend; already contacted Holger.  I’m looking
forward to discussing questions of bootstrapping and user-facing
features relating to reproducibility!

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Reproducible Build Summit, Oct. 31st, Berlin

[Ludovic Courtès]

Howdy!

Jan Nieuwenhuizen  skribis:

> Ludovic Courtès writes:
>
>> Sorry for the late notice: the Reproducible Build Summit will take place
>> from Oct. 31st to Nov. 2nd in Berlin this year:
>>
>>   https://reproducible-builds.org/events/berlin2017/
>
>> In the two previous editions, there were always 2 or 3 Guix there:
>> Manolis, John, Ricardo, and myself participated.  I won’t be able to
>> make it this year, but it would be sad if Guix were missing, especially
>> since there’s been quite a few people working on reproducibility in
>> Guix.  Please let us know if you’d like to come!
>
>> So, who’s in?  :-)
>
> I am planning to go, sad you won't be there.

Awesome, really glad that you can attend—let the bootstrapper’s voice to
be heard!

Please get in touch with Holger to let them know.

Cheers,
Ludo’.

Re: Reproducible Build Summit, Oct. 31st, Berlin

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> Sorry for the late notice: the Reproducible Build Summit will take place
> from Oct. 31st to Nov. 2nd in Berlin this year:
>
>   https://reproducible-builds.org/events/berlin2017/

> In the two previous editions, there were always 2 or 3 Guix there:
> Manolis, John, Ricardo, and myself participated.  I won’t be able to
> make it this year, but it would be sad if Guix were missing, especially
> since there’s been quite a few people working on reproducibility in
> Guix.  Please let us know if you’d like to come!

> So, who’s in?  :-)

I am planning to go, sad you won't be there.

janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Reproducible Build Summit, Oct. 31st, Berlin

[Ludovic Courtès]

Hello Guix!

Sorry for the late notice: the Reproducible Build Summit will take place
from Oct. 31st to Nov. 2nd in Berlin this year:

  https://reproducible-builds.org/events/berlin2017/

In the two previous editions, there were always 2 or 3 Guix there:
Manolis, John, Ricardo, and myself participated.  I won’t be able to
make it this year, but it would be sad if Guix were missing, especially
since there’s been quite a few people working on reproducibility in
Guix.  Please let us know if you’d like to come!

I’ve really enjoyed the previous summits.  It’s a great way to discuss
reproducibility with other projects, share each project’s view, and
discuss the way forward.

Among the forward-looking sessions we had last year was how we could
make reproducible builds directly “visible” and “useful” to users (I
think <https://bugs.gnu.org/28324> and ‘guix challenge’ are examples of
that), and how we can address the problem of opaque bootstrap binaries
(which led to Ricardo’s work on Java and Haskell bootstrapping, the
bootstrappable.org web site, and meshes well with the
independently-started effort around Mes/stage0!).

So, who’s in?  :-)

Ludo’.


signature.asc
Description: PGP signature

Re: Back from the Reproducible Build Summit, 2nd edition

[Ludovic Courtès]

Hi Chris!

Chris Marusich  skribis:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Here’s a report from the 2nd Reproducible Build Summit that took place
>> this week!
>>
>>   
>> https://gnu.org/software/guix/news/reproducible-build-summit-2nd-edition.html
>
> Thank you for sharing these notes.  I see you mentioned some follow-up
> tasks, like hopefully supporting the "k out of n" substitute policy by
> the next summit.  Are we tracking tasks like that somewhere?

I’m afraid we aren’t, but we should probably email them to
bug-g...@gnu.org and set the “wishlist” flag on them, for instance.

Would you like to do that?  :-)

Cheers,
Ludo’.

Re: Back from the Reproducible Build Summit, 2nd edition

[Chris Marusich]

Hi Ludo!

l...@gnu.org (Ludovic Courtès) writes:

> Here’s a report from the 2nd Reproducible Build Summit that took place
> this week!
>
>   
> https://gnu.org/software/guix/news/reproducible-build-summit-2nd-edition.html

Thank you for sharing these notes.  I see you mentioned some follow-up
tasks, like hopefully supporting the "k out of n" substitute policy by
the next summit.  Are we tracking tasks like that somewhere?

-- 
Chris


signature.asc
Description: PGP signature

Back from the Reproducible Build Summit, 2nd edition

[Ludovic Courtès]

Hello GNU folks!

Here’s a report from the 2nd Reproducible Build Summit that took place
this week!

  https://gnu.org/software/guix/news/reproducible-build-summit-2nd-edition.html

Ludo’.


signature.asc
Description: PGP signature

Re: Reproducible Build Summit

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

> Manolis and I had the pleasure to attend the first “Reproducible Build
> Summit” this week, wonderfully well organized by Debian hackers Lunar
> and Holger, along with other brilliant people, and with the support of
> the Linux Foundation, the Open Technology Fund, and Google.
>
>   https://reproducible-builds.org/events/athens2015/

Minutes are now available at this URL.

Ludo’.

Re: Reproducible Build Summit

[Ludovic Courtès]

l...@gnu.org (Ludovic Courtès) skribis:

>Eelco, Manolis, and I sat together for the hacking sessions.  I
>focused on shamelessly stealing the Nix daemon’s ability to rebuild a
>derivation and error out if the result differs (commits 07e70f4 and
>708d907.)

Commit a8d6564 adds the --check option for ‘guix build’:

  ‘--check’
   Rebuild PACKAGE-OR-DERIVATION, which are already available in the
   store, and raise an error if the build results are not bit-for-bit
   identical.

   This mechanism allows you to check whether previously-installed
   substitutes are genuine (*note Substitutes::), or whether a
   package’s build result is deterministic.  *Note Invoking guix
   challenge::, for more background information and tools.

and commit 5b74fe0 adds --rounds:

  ‘--rounds=N’
   Build each derivation N times in a row, and raise an error if
   consecutive build results are not bit-for-bit identical.

   This is a useful way to detect non-deterministic builds processes.
   Non-deterministic build processes are a problem because they make
   it practically impossible for users to _verify_ whether third-party
   binaries are genuine.  *Note Invoking guix challenge::, for more.

   Note that, currently, the differing build results are not kept
   around, so you will have to manually investigate in case of an
   error—e.g., by stashing one of the build results with ‘guix archive
   --export’, then rebuilding, and finally comparing the two results.

I encourage you to use the latter when adding new packages and to
investigate any reproducibility issues!

>I also discussed with Eelco the fact that the daemon was leaking the
>real name of the build directory, meaning that if a build machine
>runs:
>
>  TMPDIR=/foo/bar guix-daemon
>
>and the other runs:
>
>  TMPDIR=/tmp guix-daemon
>
>then the first build process will see a directory called
>/foo/bar/nix-build-xxx.  If it captures the build directory name,
>then we get a discrepancy.  Eelco quickly changed that, such that the
>build process always sees /tmp/nix-build-xxx:
>
> .
>I’ll merge this commit soon.

Done in cb96010.

Ludo’.

Re: Reproducible Build Summit

[Manolis Ragkousis]

Hello Guix!

It was a great, productive, week and I would really like to thank
everyone that made this possible.

Now some things I want to add:

* Signatures challenges

  We had a session on how each project signs the binaries it
distributes and ways to achieve build system compromise detection. Georg
from the Tor project told us about this paper
 which
describes how we can secure a system against such attacks.

I discussed with Ludovic the need to have an automated system that will
continuously verify the binary outputs from multiple build sources so we
can find any possible malicious compromises.
But in order to do that we need to increase our build servers and/or
implement the peer to peer binary distribution (Remi?). More machines -
more builds to compare.

The above will also help in testing Guix reproducibility. Finally I will
help Holger install Guix on the ProfitBricks-sponsored machines.

* Authenticating code from a Git repo

  Here I agree with Ludovic that we should find a way to do something
similar with Qubes, so we can at least be able to pinpoint a future
compromise if it happens.

That's what I wanted to add for now.

This was a great week, I learned a lot of new things from a great bunch
of smart and friendly people and I believe we should get a lot more
involved in this initiative.

Manolis.


0x8DF54C32.asc
Description: application/pgp-keys

Reproducible Build Summit

[Ludovic Courtès]

Hello, Guix!

Manolis and I had the pleasure to attend the first “Reproducible Build
Summit” this week, wonderfully well organized by Debian hackers Lunar
and Holger, along with other brilliant people, and with the support of
the Linux Foundation, the Open Technology Fund, and Google.

  https://reproducible-builds.org/events/athens2015/

There were around 40 people, including contributors to a variety of free
operating systems and distros, which led to many very insightful
sessions and discussions.

Here are the highlights of the sessions I attended and discussions I
had, off the top of my head; Manolis can complement it:

 * Guix & Nix session

   Eelco Dolstra of Nix and myself held a “geek dating” session where we
   presented Nix/Guix to everyone by groups of 8 people.  That was a fun
   exercise, but also difficult to do without any demo/slides.

   I was glad to learn that Lamby, a Debian developer, had come up with
   <https://github.com/lamby/apt-challenge>.  :-)  A later session was
   dedicated to specifying a tool similar to ‘guix challenge’ and dubbed
   ‘reprotest’.

 * Shared reproducibility issue database

   Currently we have a bunch of patches and open reproducibility issues
   at <http://bugs.gnu.org/guix>, but they feel lonely!  The Debian
   folks have a Yaml database (recutils would be nice, but hey!) that is
   used to generate the pages at <https://reproducible.debian.org/>.
   They are kindly offering to open it to other distros, so that we can
   all work together instead of duplicating work.  Expect updates on the
   practical details soonish.

 * Distro bootstrapping

   We had a short session on distro bootstrapping, where we discussed
   how each project addresses it, and what we could do to improve the
   situation.  Most distros, including Debian, do not have a clearly
   specified set of bootstrap binaries like we have, but are willing to
   work towards having a more formal set of requirements, to allow the
   distro to be bootstrapped in different contexts.  There were
   discussions about diverse double compilation and whether/how it could
   help, as well as how we could reduce the bootstrap set (I suggested
   something like keeping TinyC, not GCC, as a bootstrap binary, used to
   build GCC 4.6-, used to build a more recent GCC that requires C++.)

 * Hacking sessions

   Eelco, Manolis, and I sat together for the hacking sessions.  I
   focused on shamelessly stealing the Nix daemon’s ability to rebuild a
   derivation and error out if the result differs (commits 07e70f4 and
   708d907.)

   I also discussed with Eelco the fact that the daemon was leaking the
   real name of the build directory, meaning that if a build machine
   runs:

 TMPDIR=/foo/bar guix-daemon

   and the other runs:

 TMPDIR=/tmp guix-daemon

   then the first build process will see a directory called
   /foo/bar/nix-build-xxx.  If it captures the build directory name,
   then we get a discrepancy.  Eelco quickly changed that, such that the
   build process always sees /tmp/nix-build-xxx:
   
<https://github.com/NixOS/nix/commit/8063fc497ab78fa72962b93874fe25dcca2b55ed>.
   I’ll merge this commit soon.

   Of course the better fix is for packages to not capture the build
   directory name, and even to error out when that happens.  But that
   seems like a longer-term goal, potentially can-of-wormy (think of
   debugging symbols…)

   Besides, I packaged ‘findnewest’ (commit df7393b) by Thomas Klausner
   of NetBSD minutes after he had tagged it.  ;-) This tool is meant to
   help provide useful *and* reproducible timestamps for packages that
   want it.

 * Testing Guix reproducibility

   Holger of Debian said he would be happy to perform independent builds
   of Guix packages on the ProfitBricks-sponsored machines that are used
   for Debian and other distros.  Manolis will discuss with him to get
   that up and running.

 * Authenticating code from a Git repo

   The Qubes people have a sophisticated process to sign their code in
   Git.  Since they want signatures from both the author and the
   reviewers of a given patch series, they use signed tags (signed
   commits can only have one signature.)  Thus they end up automatically
   creating signed tags for each series of patches that is pushed (a bit
   heavyweight, but it does the job.)

 https://www.qubes-os.org/doc/verifying-signatures/

   I’d like to suggest that we do something similar.

I think that’s a lot of food for thought already!  There’ll be other
such meetings in the future, so I hope we can keep making progress
together.  I would also like to get the broader GNU involved in
addressing the issue.

Anyway, that was a great experience, with lots of friendly people to
chat with and have a walk in Athens by night!

Ludo’.


signature.asc
Description: PGP signature