Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-07-23 Thread Holger Levsen
Hi Steven,

On Montag, 20. Juli 2015, Steven Chamberlain wrote:
 `mktemp freebsd-` on FreeBSD would result in random characters
 being appended, resulting in freebsd-.v1adN6Qo as above.
 
 `mktemp -d -t freebsd-` should replace the X's with random
 characters, same as GNU mktemp.  But it doesn't seem to have done that.

this doesnt happen when trying this manually on freebsd:

[jenkins@freebsd-jenkins ~]$ TMPDIR=/srv/workspace/chroots/ mktemp -d -t 
freebsd-
/srv/workspace/chroots//freebsd-.Qnc7a204
[jenkins@freebsd-jenkins ~]$ TMPDIR=/srv/workspace/chroots/ mktemp -d -t 
freebsd 
/srv/workspace/chroots//freebsd.xmBuKFoO

So I've changed the code to use the 2nd command now…
 
 Are you sure that your RSSH command is sending switches -d and -t
 correctly, or do you need a -- or extra quotes?
 
 Take a look in /srv/workspace/chroots/ and see if mktemp has perhaps
 created a file instead of a directory?

there are directories as expected…

So I've disabled the cleanup after build and fired up another, the result can 
be seen at
https://jenkins.debian.net/view/reproducible/job/reproducible_freebsd/9/console
and again ends with 

--
 stage 2.1: cleaning up the object tree
--
cd /srv/workspace/chroots/freebsd.YUCtKJvs/freebsd; MAKEOBJDIRPREFIX=/usr/obj  
MACHINE_ARCH=amd64  MACHINE=amd64  CPUTYPE= 
GROFF_BIN_PATH=/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/usr/bin
  
GROFF_FONT_PATH=/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/usr/share/groff_font
  
GROFF_TMAC_PATH=/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/usr/share/tmac
  
_LDSCRIPTROOT=  VERSION=FreeBSD 11.0-CURRENT amd64 1100077  INSTALL=sh 
/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tools/install.sh  
PATH=/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/usr/sbin:/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/usr/bin:/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/legacy/bin:/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/usr/sbin:/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
CC=cc  CXX=c++DEPFLAGS=  CPP=cpp   AS=as AR=ar LD=ld NM=nm  
OBJDUMP=objdump OBJCOPY=objcopy  RANLIB=ranlib STRINGS=  SIZE=size make  -
f Makefile.inc1 
DESTDIR=/usr/obj/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/tmp par-
cleandir
=== lib (cleandir)
=== lib/csu (cleandir)
=== lib/csu/amd64 (cleandir)
=== lib/libcompiler_rt (cleandir)
=== lib/libc (cleandir)
=== lib/libc/tests (cleandir)
cd: /srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/libc/tests: No such 
file or directory
*** Error code 2

and indeed, /srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/libc/ does not 
exist, while /srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/ exists and 
is populated:

[jenkins@freebsd-jenkins ~]$ ls 
/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/libc
ls: /srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/libc: No such file or 
directory
[jenkins@freebsd-jenkins ~]$ ls 
/srv/workspace/chroots/freebsd.YUCtKJvs/freebsd/lib/libc
libc++/ libcalendar/libcapsicum/libclang_rt/libcompat/  
libcuse/
libc_nonshared/ libcam/ libcasper/  libcom_err/ libcrypt/   
libcxxrt/   
[jenkins@freebsd-jenkins ~]$ 


Any ideas how to proceed now?


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-07-20 Thread Steven Chamberlain
Hi Holger,

Holger Levsen wrote:
 With this, 
 http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproducible_freebsd.sh
  
 gets as far as 
 https://jenkins.debian.net/view/reproducible/job/reproducible_freebsd/7/console
  
 where stage 2.1: cleaning up the object tree fails on make buildworld, 
 because /srv/workspace/chroots/freebsd-
 .v1adN6Qo/freebsd/lib/libc/tests does not exist.

`mktemp freebsd-` on FreeBSD would result in random characters
being appended, resulting in freebsd-.v1adN6Qo as above.

`mktemp -d -t freebsd-` should replace the X's with random
characters, same as GNU mktemp.  But it doesn't seem to have done that.

Are you sure that your RSSH command is sending switches -d and -t
correctly, or do you need a -- or extra quotes?

Take a look in /srv/workspace/chroots/ and see if mktemp has perhaps
created a file instead of a directory?

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-07-18 Thread Holger Levsen
Hi,

so I made some progress on this: a.) there is a build host running freebsd 
10.1 (called freebsd-jenkins.debian.net) now, on which the jenkins user from 
jenkins.debian.net can login via ssh as jenkins user b.) besides the base 
system it has screen git vim sudo denyhosts installed and c.) the 
directories /srv/workspace/chroots/ and /srv/reproducible-results have been 
created (and are owned by the jenkins user) and d.) /usr/obj/srv is a link to 
/srv.

With this, 
http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproducible_freebsd.sh
 
gets as far as 
https://jenkins.debian.net/view/reproducible/job/reproducible_freebsd/7/console 
where stage 2.1: cleaning up the object tree fails on make buildworld, 
because /srv/workspace/chroots/freebsd-
.v1adN6Qo/freebsd/lib/libc/tests does not exist.

And at this point I'm stuck as to why this happens. Any hint much welcome!

(Please note that reproducible_freebsd.sh is just a work-in-progress now and 
there are still some bits from it's source, reproducible_netbsd.sh visible. 
This need to be cleaned up, but shouldn't be too confusing know that this is 
clear.)

On Mittwoch, 17. Juni 2015, Ed Maste wrote:
  https://wiki.freebsd.org/ReproducibleBuilds claims there are 3 known
  issues (for make world AIUI) for HEAD, I would like to build twice and
  verify myself.
 I'm interested in fixing the remaining kernel / world issues, with the
 kernel being my higher priority.

cool!
 
 For the kernel we have the username, hostname, and build timestamp.
 The path is included too, but I don't anticipate trying to address it
 at first; release builds are done in a consistent location anyhow
 (/usr/src).

/me nods - that's what we are doing in (reproducible builds for) Debian too, 
the path has to be the same on rebuilds (as it is included in too many build 
artifacts to deeply.)

 These are used only as user-facing strings for the kern.version sysctl
 and reported by uname. An example kern.version string:
 FreeBSD 10.1-STABLE #28 r280427+86df2de(stable-10): Thu Mar 26 16:07:47 EDT
 2015
 emaste@feynman:/tank/emaste/obj/tank/emaste/src/git-stable-10/sys/GENERIC
 
 From a technical perspective they're trivially eliminated. There may
 be some 3rd party ports expect the precise format, but probably not
 very many (and they should be fixed, anyhow).  There's a much larger
 social issue in convincing the FreeBSD developer community to accept
 their removal, though :-)

If any build (of the same sources) results in the exact same bits, the build 
time becomes meaningless and thus a.) can be dropped or b.) replaced with the 
date of the last modification of the sources - which is meaningful information 
again!

While this is/was a new thought for most everyone (me included...) in my 
experience it also has been convincing logic for most everyone. The technical 
details to achieve this are sometimes a bit harder to achieve, but not 
impossible. (eg they differ whether git, svn or tarballs are the means to get 
access to sources.)

In Debian we want 100% bit identical packages (=.deb files) as this allows us 
to only require a checksum comparison to see whether two builds created 
reproducible results.

  https://wiki.freebsd.org/PortsReproducibleBuilds says Of the 23599
  packages which were built in both runs, 15164 have the same checksum
  when using the previously mentioned patch, giving 64.25% reproducible
  packages. - I'm also curious to re-confirm this - and set up a test
  bed, which can be triggered regularily and easily. Our jenkins set up
  allows this and I'm interested to do this.
 
 I'm pleasantly surprised by the ports results -- 64.25% seems quite
 good for such a straightforward change. The test there is on the same
 host though, and so avoids any non-reproducibility from host/user/path
 leaks.

ah

  My interest is to help FreeBSD with reproducible builds as I want to see
  reproducible builds become the norm in the free software world and as I
  believe FreeBSD is an important part of this world. And also because I'm
  curious. :)
 
 Great! Hopefully we can help lend some weight in convincing upstream
 projects to accept reproducibility patches (once we get further along
 in our ports effort).

I'm looking forward to see this happen! ;-)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-07-18 Thread John-Mark Gurney
Ed Maste wrote this message on Wed, Jun 17, 2015 at 16:48 -0400:
 These are used only as user-facing strings for the kern.version sysctl
 and reported by uname. An example kern.version string:
 FreeBSD 10.1-STABLE #28 r280427+86df2de(stable-10): Thu Mar 26 16:07:47 EDT 
 2015
 emaste@feynman:/tank/emaste/obj/tank/emaste/src/git-stable-10/sys/GENERIC
 
 From a technical perspective they're trivially eliminated. There may
 be some 3rd party ports expect the precise format, but probably not
 very many (and they should be fixed, anyhow).  There's a much larger
 social issue in convincing the FreeBSD developer community to accept
 their removal, though :-)

I don't know about others, but IMO, the only useful information there
is the path it was built from... The machine isn't too useful and even
less useful is probably the build user...  Maybe on larger installs,
the user/machine makes a difference, but that could be a config option
to include those...

So my vote is to eliminate user/machine and just leave the path... And
we could just use user@machine to keep the format compatible, but
constant...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-06-17 Thread Ed Maste
On 16 June 2015 at 17:50, Holger Levsen hol...@layer-acht.org wrote:

 So in a while, I expect to have set up
 https://reproducible.debian.net/freebsd/ as well as
 https://reproducible.debian.net/netbsd/ - but no promises (yet), but these are
 my plans ;-)

Great, looking forward to it!

 https://wiki.freebsd.org/ReproducibleBuilds claims there are 3 known issues
 (for make world AIUI) for HEAD, I would like to build twice and verify
 myself.

I'm interested in fixing the remaining kernel / world issues, with the
kernel being my higher priority.

For the kernel we have the username, hostname, and build timestamp.
The path is included too, but I don't anticipate trying to address it
at first; release builds are done in a consistent location anyhow
(/usr/src).

These are used only as user-facing strings for the kern.version sysctl
and reported by uname. An example kern.version string:
FreeBSD 10.1-STABLE #28 r280427+86df2de(stable-10): Thu Mar 26 16:07:47 EDT 2015
emaste@feynman:/tank/emaste/obj/tank/emaste/src/git-stable-10/sys/GENERIC

From a technical perspective they're trivially eliminated. There may
be some 3rd party ports expect the precise format, but probably not
very many (and they should be fixed, anyhow).  There's a much larger
social issue in convincing the FreeBSD developer community to accept
their removal, though :-)

 https://wiki.freebsd.org/PortsReproducibleBuilds says Of the 23599 packages
 which were built in both runs, 15164 have the same checksum when using the
 previously mentioned patch, giving 64.25% reproducible packages. - I'm also
 curious to re-confirm this - and set up a test bed, which can be triggered
 regularily and easily. Our jenkins set up allows this and I'm interested to do
 this.

I'm pleasantly surprised by the ports results -- 64.25% seems quite
good for such a straightforward change. The test there is on the same
host though, and so avoids any non-reproducibility from host/user/path
leaks.

 My interest is to help FreeBSD with reproducible builds as I want to see
 reproducible builds become the norm in the free software world and as I
 believe FreeBSD is an important part of this world. And also because I'm
 curious. :)

Great! Hopefully we can help lend some weight in convincing upstream
projects to accept reproducibility patches (once we get further along
in our ports effort).

-Ed

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-06-17 Thread Erik Cederstrand

 Den 16/06/2015 kl. 23.50 skrev Holger Levsen hol...@layer-acht.org:
 
 Reproducible builds enable anyone to reproduce bit by bit identical binary 
 packages from a given source, so that anyone can verify that a given binary 
 derived from the source it was said to be derived.  - right now you have to 
 *believe* someone that the binary really comes from said source. And you need 
 to *believe* the system building it wasn't compromised...

The build should be immune to the time of the build, of course. That's fairly 
easy (e.g. use 'ar -D' consistently and leave DEBUG_FLAGS empty).

But what about the user who started the build? This leaks to at least sendmail 
config files.

Being agnostic to the path to the src root (e.g. /usr/src or 
/home/erik/freebsd/HEAD/src) requires rewriting the compiler __FILE__ macro to 
insert a relative path, and make debuggers understand relative paths. This is 
hard.

The FreeBSD subversion revision is also leaked several places.

I think reproduce builds are a noble goal and would enable all sorts of smart 
analysis, e.g. which binaries are affected by a certain commit. Just remember 
to define the requirements that need to be satisfied to get reproduce builds.

Erik
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds


Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-06-17 Thread Holger Levsen
Hi Erik,

On Mittwoch, 17. Juni 2015, Erik Cederstrand wrote:
 The build should be immune to the time of the build, of course. That's
 fairly easy (e.g. use 'ar -D' consistently and leave DEBUG_FLAGS empty).

yup, easy, but this can mean some work. (Which usually can be shared among the 
upstream software projects.)
 
 But what about the user who started the build? This leaks to at least
 sendmail config files.

yup, those are bugs which need to be fixed. (it's also a privacy issue.)

 Being agnostic to the path to the src root (e.g. /usr/src or
 /home/erik/freebsd/HEAD/src) requires rewriting the compiler __FILE__
 macro to insert a relative path, and make debuggers understand relative
 paths. This is hard.

while doing this for Debian we haven't found a way to prevent this (leaking of 
the build path into build products), so our solution now is to use a 
definited path or record the path and build in the same path again.

that is clearly not optimal but currently the only thing we require to be some 
specific way.

 The FreeBSD subversion revision is also leaked several places.

That should not matter, as it's part of the source, so it will be the same 
revision on rebuilds. 

 I think reproduce builds are a noble goal and would enable all sorts of
 smart analysis, e.g. which binaries are affected by a certain commit. Just
 remember to define the requirements that need to be satisfied to get
 reproduce builds.

sure. *I* also don't plan to fix or even work on FreeBSD, I'm merely 
investigating it and sharing the results. If the FreeBSD community wants 
reproducible builds, you will need to work on them ;-)

(I'll be happy to help but thats it.)


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] reproducible builds of FreeBSD in a chroot on Linux

2015-06-16 Thread Holger Levsen
Hi,

sorry for replying so late... on the plus side, I've got a much clearer 
picture now and I've implemented something similar, eg see

https://reproducible.debian.net/openwrt/
and/or
https://reproducible.debian.net/coreboot/

On the original subject of my mail: I have given up on this and will build 
FreeBSD on a FreeBSD system, not in a chroot on Linux. I expected this would 
work, learned that it doesn't and on the way also learned that one can build 
NetBSD on Linux or probably anything ;-)

So in a while, I expect to have set up 
https://reproducible.debian.net/freebsd/ as well as 
https://reproducible.debian.net/netbsd/ - but no promises (yet), but these are 
my plans ;-)

And to reply to some of you...

On Donnerstag, 7. Mai 2015, Michael Fuckner wrote:
  I'm one of the people involved in
  https://wiki.debian.org/ReproducibleBuilds and have set up
  https://reproducible.debian.net which continously tests all packages in
  the Debian archive for build reproducibility (so far on amd64 only).
 what is this good for? Testing the Compiler, track changes or check
 hardware (errors on memory or disk)

Reproducible builds enable anyone to reproduce bit by bit identical binary 
packages from a given source, so that anyone can verify that a given binary 
derived from the source it was said to be derived.  - right now you have to 
*believe* someone that the binary really comes from said source. And you need 
to *believe* the system building it wasn't compromised...

This is explained in more detail in our wiki or in the talks given, which are 
linked in the wiki as well.

On Freitag, 8. Mai 2015, Julian Elischer wrote:
 also: By FreeBSD do you mean the kernel? or the whole system?
 Unlike Linux, FreeBSD includes most of what the Linux world would
 consider to be the domain of the base distro..  e.g. cat, ls, cc, etc.

I mean the whole system (what you get when you run make world) as well as 
the ports.

https://wiki.freebsd.org/ReproducibleBuilds claims there are 3 known issues 
(for make world AIUI) for HEAD, I would like to build twice and verify 
myself.

https://wiki.freebsd.org/PortsReproducibleBuilds says Of the 23599 packages 
which were built in both runs, 15164 have the same checksum when using the 
previously mentioned patch, giving 64.25% reproducible packages. - I'm also 
curious to re-confirm this - and set up a test bed, which can be triggered 
regularily and easily. Our jenkins set up allows this and I'm interested to do 
this.

(And I wouldn't be surprised nor disappointed if it took me til August or 
September until I actually get around to tests the ports. The base system I 
definitly want to have results on in July.)
 
 There may also be a better mailing list for this...

which?

On Montag, 11. Mai 2015, Ed Maste wrote:
 A lot of this depends on the motivation for pursuing reproducible
 FreeBSD builds. If it's to help FreeBSD overall with reproducible
 builds, then using the FreeBSD build infrastructure on a FreeBSD
 kernel (e.g., a FreeBSD jail on Debian kFreeBSD) is an important part
 of the story. If it's specifically for reproducible kernel builds for
 kFreeBSD then the FreeBSD build infrastructure isn't relevant.

My interest is to help FreeBSD with reproducible builds as I want to see 
reproducible builds become the norm in the free software world and as I 
believe FreeBSD is an important part of this world. And also because I'm 
curious. :)

As such, I'll set up a FreeBSD host on jenkins.debian.net (in that virtual 
datacenter providing that host), running FreeBSD kernel and userland - to test 
FreeBSD on Debian ressources :-) Because we care and we can.

Debian's kfreebsd-amd64 to me here is just another Debian architecture 
(sorry Steven!), which will (hopefully) benefit from the Debian reproducible 
builds like all the other Debian architectures. 

(And I wrote hopefully because kfreebsd-amd64 was a bit special for jessie 
and hopefully will be a proper architecture for stretch, the release coming in 
two years.)

I'll come back once these FreeBSD tests are set up.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds