Re: Raising the severity of reproduciblity issues to "important"

[Holger Levsen]

On Fri, Sep 01, 2017 at 06:34:38PM +0100, Ian Campbell wrote:
> On Fri, 2017-09-01 at 12:43 +0200, Helmut Grohne wrote:
> > Whatever point you were trying to make around NEW, your argument is not
> > very convincing. I think Holger is right here: Where the package is
> > built should not matter. Presence of .buildinfo and reproducibility
> > does.
> 
> Appollogies if this is covering well worn ground but does this mean we
> therefore need to check that everything referenced in .buildinfo was
> present in the archive at some point as a step during accepting a
> package (new or not new) into the archive?

well, yes, but we should check this today too, just for proper builds, 
independently of reproducible builds.

> Where "was present in the archive at some point" is a proxy for "is
> present on snapshots.d.o". If that can also be checked directly that
> might be cool, although it might be considered a bit rude to a
> maintainer to reject a package for what was a snapshot.do.o issue.

well, yes, the check should be done using a local (copy of the) snapshot.d.o
database. It's also rude to upload a package built with packages not in the
archive, and we should prevent that.

> https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles suggests that
> the build environment contains the versions of packages but not their
> hashes -- so there is a possibility that a developer might be building
> with a non-canonical version of the package. Perhaps they installed a
> local dev version of the build-dep, perhaps because they maintain both
> and we doing a quasi-simultaneous upload. That's perhaps not indicative
> of best practice, but mistakes do happen.
 
yup, and we should try to catch as many automatically as we can.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Ian Campbell]

On Fri, 2017-09-01 at 12:43 +0200, Helmut Grohne wrote:
> Whatever point you were trying to make around NEW, your argument is not
> very convincing. I think Holger is right here: Where the package is
> built should not matter. Presence of .buildinfo and reproducibility
> does.

Appollogies if this is covering well worn ground but does this mean we
therefore need to check that everything referenced in .buildinfo was
present in the archive at some point as a step during accepting a
package (new or not new) into the archive?

Where "was present in the archive at some point" is a proxy for "is
present on snapshots.d.o". If that can also be checked directly that
might be cool, although it might be considered a bit rude to a
maintainer to reject a package for what was a snapshot.do.o issue.

https://wiki.debian.org/ReproducibleBuilds/BuildinfoFiles suggests that
the build environment contains the versions of packages but not their
hashes -- so there is a possibility that a developer might be building
with a non-canonical version of the package. Perhaps they installed a
local dev version of the build-dep, perhaps because they maintain both
and we doing a quasi-simultaneous upload. That's perhaps not indicative
of best practice, but mistakes do happen.

Ian.

Re: Raising the severity of reproduciblity issues to "important"

[Helmut Grohne]

On Fri, Sep 01, 2017 at 11:07:17AM +0100, Simon McVittie wrote:
> The problem with maintainer-built binaries around NEW is that if they
> wait in the NEW queue for (let's say) 1 month, then by the time they
> reach the archive, they were built with a 1 month old toolchain and
> build-dependencies, not an up-to-date toolchain and dependencies.
> Reproducible builds don't help with this, because a package can
> typically only be reproducible when holding the toolchain and
> dependencies constant.

I fail to see the problem here. Of course the maintainer-built binaries
should be accompanied with the relevant .buildinfo file telling what
versions of dependencies were used - just like any buildd-built package.

Packages can sit in unstable for months - even years - without being
updated and thus their binaries do use years old toolchain and
build-dependencies. What you describe does happen to buildd-built
packages. The key to reproducibility here is using the .buildinfo and
replicating the installation used for the original build (using
snapshot.d.o).

Whatever point you were trying to make around NEW, your argument is not
very convincing. I think Holger is right here: Where the package is
built should not matter. Presence of .buildinfo and reproducibility
does.

Regarding Guillem's point: I don't think disallowing binary uploads is
going to be a problem to bootstrapping. Ubuntu has been doing this for
years. They have a small set of people who can (carefully) inject binary
packages and that mechanism is sufficient. Restricting binary uploads to
a small subgroup of Debian Developers does make sense to me from a
bootstrap pov, because uploading binaries could be a rare thing to do.

We should be less shy in copying the good stuff from Ubuntu. :)

Helmut

Re: normal bugs (Re: Raising the severity of reproduciblity issues to "important")

[Adrian Bunk]

On Fri, Sep 01, 2017 at 09:43:54AM +, Holger Levsen wrote:
> On Fri, Sep 01, 2017 at 09:34:53AM +0300, Adrian Bunk wrote:
> > On Sun, Aug 23, 2015 at 12:48:50PM +0200, Chris Lamb wrote:
> > >...
> > > However, based on an informal survey at DebConf (and to reflect the
> > > feeling towards software reproducibility in the free software community
> > > in general) unless there are strong objections I intend to raise the
> > > severity of these wishlist issues to "important" once the toolchain
> > > changes to dpkg and debhelper land in sid.
> > I would strongly object to raising the severity to "important" for 
>  
> sigh. you are replying to a mail from 2015, were you aware of that?
>...

No, I have too many emails in my debian-devel folder
and did not have enough black tea in the morning.

Sincere apologies and please ignore my emails, it is my fault that
I failed to read the year of the emails I replied to and responded
to an obsolete proposal.

Sorry
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Raising the severity of reproduciblity issues to "important"

[Simon McVittie]

On Fri, 01 Sep 2017 at 09:40:25 +, Holger Levsen wrote:
> On Fri, Sep 01, 2017 at 09:26:44AM +0300, Adrian Bunk wrote:
> > AFAIK the only place where we currently still need binary packages that 
> > have been built on a maintainer machine is for [...]
>  
> the fun part is that once a package builds bit by bit identically, it doesnt
> matter anymore where it's been built…! :-)

The problem with maintainer-built binaries around NEW is that if they
wait in the NEW queue for (let's say) 1 month, then by the time they
reach the archive, they were built with a 1 month old toolchain and
build-dependencies, not an up-to-date toolchain and dependencies.
Reproducible builds don't help with this, because a package can
typically only be reproducible when holding the toolchain and
dependencies constant.

S

normal bugs (Re: Raising the severity of reproduciblity issues to "important")

[Holger Levsen]

On Fri, Sep 01, 2017 at 09:34:53AM +0300, Adrian Bunk wrote:
> On Sun, Aug 23, 2015 at 12:48:50PM +0200, Chris Lamb wrote:
> >...
> > However, based on an informal survey at DebConf (and to reflect the
> > feeling towards software reproducibility in the free software community
> > in general) unless there are strong objections I intend to raise the
> > severity of these wishlist issues to "important" once the toolchain
> > changes to dpkg and debhelper land in sid.
> I would strongly object to raising the severity to "important" for 
 
sigh. you are replying to a mail from 2015, were you aware of that?

Last thing Chris (and we all) said (at+during DebConf, so just 2 weeks ago),
is that we plan to treat such bugs as severity "normal" bugs.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Holger Levsen]

On Fri, Sep 01, 2017 at 09:26:44AM +0300, Adrian Bunk wrote:
> AFAIK the only place where we currently still need binary packages that 
> have been built on a maintainer machine is for [...]
 
the fun part is that once a package builds bit by bit identically, it doesnt
matter anymore where it's been built…! :-)


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Guillem Jover]

Hi!

On Fri, 2017-09-01 at 09:26:44 +0300, Adrian Bunk wrote:
> AFAIK the only place where we currently still need binary packages that 
> have been built on a maintainer machine is for NEW, and after someone
> has implemented a solution for that there is no blocker left for 
> allowing only source-only uploads from maintainers.

Bootstrapping (either for new ports, or for build-dep cycles) is still
also a blocker. Although that is already being worked on, spearheaded
by Helmut Grohne as part of the great rebootstrap effort [R].

Thanks,
Guillem

[R] 

Re: Raising the severity of reproduciblity issues to "important"

[Adrian Bunk]

On Sun, Aug 23, 2015 at 12:48:50PM +0200, Chris Lamb wrote:
>...
> However, based on an informal survey at DebConf (and to reflect the
> feeling towards software reproducibility in the free software community
> in general) unless there are strong objections I intend to raise the
> severity of these wishlist issues to "important" once the toolchain
> changes to dpkg and debhelper land in sid.

Which definition of "reproducibility" are you using for the purpose of 
this proposal?

As of today, the reproducibility definition in policy and what is being 
tested in the r-b infrastructure are quite different.

I would strongly object to raising the severity to "important" for 
packages where it is not proven that they are not reproducible according 
to current [1] policy.

> Regards,

cu
Adrian

[1] current at the time of the severity raise

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Raising the severity of reproduciblity issues to "important"

[Adrian Bunk]

On Mon, Aug 24, 2015 at 11:41:21PM +0200, Vincent Bernat wrote:
>  ❦ 24 août 2015 22:30 +0100, Colin Tuckley  :
> 
> >> We have pushed other archive-wide goals that were not shared by
> >> all upstreams. For example, we have enabled hardening build flags
> >> on almost all packages and for packages that don't obey to the
> >> appropriate flags, bugs with severity "important" were filed.
> >> That's not that different of a reproducible build.
> >
> > Sorry, but it's a *completely* different situation. The hardening
> > initiative made applications more secure and tamper resistant. The r-b
> > changes do nothing useful post-build.
> 
> Letting people independently check that the builds are not tampered is
> also a security application of reproducible builds. This is notably
> important for the binary packages that have been built on a maintainer
> machine instead of a builder.

The latter point is moot - if we still allow binary packages that have 
been built on a maintainer machine [1] into the archive by the time
everything installed on your computer will be reproducible, this would
be a huge fail itself.

AFAIK the only place where we currently still need binary packages that 
have been built on a maintainer machine is for NEW, and after someone
has implemented a solution for that there is no blocker left for 
allowing only source-only uploads from maintainers.

cu
Adrian

[1] these also have other frequent issues,
most notably unclean built environments

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Raising the severity of reproduciblity issues to "important"

[Wouter Verhelst]

On Mon, Aug 24, 2015 at 10:30:45PM +0100, Colin Tuckley wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> On 24/08/15 22:02, Vincent Bernat wrote:
> 
> > We have pushed other archive-wide goals that were not shared by
> > all upstreams. For example, we have enabled hardening build flags
> > on almost all packages and for packages that don't obey to the
> > appropriate flags, bugs with severity "important" were filed.
> > That's not that different of a reproducible build.
> 
> Sorry, but it's a *completely* different situation. The hardening
> initiative made applications more secure and tamper resistant. The r-b
> changes do nothing useful post-build.

Sorry, but this is not correct. You may not think it important, but that
doesn't mean it is useless post-build. The ability to independently
verify that the built binary did indeed come from a given source is a
*huge* benefit.

-- 
It is easy to love a country that is famous for chocolate and beer

  -- Barack Obama, speaking in Brussels, Belgium, 2014-03-26

Re: Raising the severity of reproduciblity issues to "important"

[Steve Langasek]

On Mon, Aug 24, 2015 at 10:25:01PM +0200, Niels Thykier wrote:
> > It is really so much difficult to make this in stages?
> > For example:

> > Stage 1. Make it a policy *recommendation*, with normal severity.
> > Stage 2. Make it a policy "should", with important severity.
> > Stage 3. Make it a release goal, policy "must", with serious severity.

> It would be great if Policy was updated so frequently that was up to
> speed with current initiatives in Debian.  However, historically, it
> often lacks behind on many key parts.  Including but not limited to
> Multi-Arch (released with Wheezy), which is currently still not
> documented in policy (nor in git for the next upload).

Multiarch filesystem paths are included in Policy; if they hadn't been, the
use of multiarch would have been forbidden because of the FHS.

Description of the multiarch control fields is not in Policy, because this
was a lower priority; there were other places maintainers could find
documentation, and it wasn't necessary to document this in Policy right away
for maintainers to know what was required for interoperation, and nobody was
looking to make multiarch a Policy requirement.  So no one has done the work
yet to craft suitable language to describe these fields in Policy - though
at DebConf it was clear that people would like to see this happen now.

None of that justifies a claim that Policy is slow to update.  If someone
wants to create a new requirement for packages in the archive, Policy is the
right way to accomplish this; and if someone has drafted suitable policy
language I see no reason that Policy would be slow to incorporate it.


> Also, contrary to popular belief:

>   Policy does *not* decide if something is an RC bug or not.

> That is currently delegated to the Release Team[1], which means that

>   https://release.debian.org/stretch/rc_policy.txt

> is the canonical policy for what is release critical and what is not.

It would be bad form for the Release Team to add something to this list that
was not already documented in Policy.  This document has its origins as a
list of *which* Policy requirements the Release Team will enforce as
release-critical.  Letting that list get out of sync with Policy would be
pretty bad for Debian.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Chris Lamb]

> Quoting Holger: "This is a lie" (pointing to a graph that was being
> shown on the screen). The current figures we are handling right now
> refer to a modified build environment (i.e. sid + the special
> sources.list line from alioth).

I do not intend to change anything until these changes have landed in
sid.

> Also, we should better categorize the reasons why packages do not
> build reproducibly.

We currently have 158 distinct categories of issues, each with a brief
description of the issue and their current status such as a link to a
BTS or upstream patch, or simply initial ideas on how to start fixing
them. Is there anything else that we could be doing that you think would
help here?


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Re: Raising the severity of reproduciblity issues to "important"

[Santiago Vila]

On Mon, Aug 24, 2015 at 10:25:01PM +0200, Niels Thykier wrote:
> In your opinion, how much of the archive should be fixed before one can
> start bumping the severity?

I don't know, but I think we should have better statistics before
deciding about that.

Quoting Holger: "This is a lie" (pointing to a graph that was being
shown on the screen). The current figures we are handling right now
refer to a modified build environment (i.e. sid + the special
sources.list line from alioth).

Also, we should better categorize the reasons why packages do not
build reproducibly.

For example: I am asked in gettext that xgettext honors the
SOURCE_DATE_EPOCH environment variable (see ##792687).

This could be a good idea by itself, but so far the packages where I
see that this change really helps are packages providing a .pot file
in the source that also regenerate it in the build process as well.

This is probably a bug even if we don't take R-B in account: Packages
should ideally not modify their own source in the build process.
However, if I apply the suggested patch, those bugs would be hidden
and we would never know about them.


So, let's see how things evolve before playing with severities.

Re: Raising the severity of reproduciblity issues to "important"

[Christoph Biedl]

Santiago Vila wrote...

> Making a great percentage of packages in the archive to be "suddenly"
> buggy is unacceptable.

Nobody would consider making failing r12y "serious" at the current
state where 13 to 17 percent of the packages fail, depending on how
you read the numbers.

> We all want Debian to build reproducibly, but goals are achieved by
> submitting bugs, changing packages and making uploads, not by rising
> severities.

Yes, and this work is currently being done. Or rather, has been done
to a suprisingly huge extent. A *lot* of patches have been prepared
and submitted to BTS, just waiting for the maintainers to pick them
up.

The question is, how many packages do need attention beyond that, i.e.
fail for reasons that still need investigation? At the moment the
number is somewhere between 200 and (wild guessing) 1000. If it's less
than 50 in a year, this seems realistic, why not finish the job?
Setting these to "important" then seems acceptable. Making this a
release goal still could be left for stretch+1.

Christoph

Re: Raising the severity of reproduciblity issues to "important"

[Vincent Bernat]

 ❦ 24 août 2015 22:30 +0100, Colin Tuckley  :

>> We have pushed other archive-wide goals that were not shared by
>> all upstreams. For example, we have enabled hardening build flags
>> on almost all packages and for packages that don't obey to the
>> appropriate flags, bugs with severity "important" were filed.
>> That's not that different of a reproducible build.
>
> Sorry, but it's a *completely* different situation. The hardening
> initiative made applications more secure and tamper resistant. The r-b
> changes do nothing useful post-build.

Letting people independently check that the builds are not tampered is
also a security application of reproducible builds. This is notably
important for the binary packages that have been built on a maintainer
machine instead of a builder.
-- 
Write and test a big program in small pieces.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: Raising the severity of reproduciblity issues to "important"

[Colin Tuckley]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 24/08/15 22:02, Vincent Bernat wrote:

> We have pushed other archive-wide goals that were not shared by
> all upstreams. For example, we have enabled hardening build flags
> on almost all packages and for packages that don't obey to the
> appropriate flags, bugs with severity "important" were filed.
> That's not that different of a reproducible build.

Sorry, but it's a *completely* different situation. The hardening
initiative made applications more secure and tamper resistant. The r-b
changes do nothing useful post-build.

Colin

- -- 
Colin Tuckley  |  +44(0)1223 830814  |  PGP/GnuPG Key Id
Debian Developer   |  +44(0)7799 143369  | 0x38C9D903

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJV240FAAoJEPoMQQc4ydkDHmsP/R5evll9FAHE6luNzgU7dzuS
mF0/J4jdqi9TLBZ3IbrLFEBdOM2ojgfRnW+QMBMc5CSorbidSSsHcXg50/1rdImQ
X5/EVBa33YafGDEWg26HZnB1I0NAsLbiNVtafDs2KFrSrC0DomsVyveOr82HhoUD
BRzYE5eiOpovITKgkIcNLoa20COYiuW3BQvFK5rTHFmAw3jL70GdTd3j/1qlXg5p
dVDUo8vtcJeh0ReQg3j4oVEquCINEayktl//Gxh826urB5Xha+UbRzlucsq1XiUd
T1gZFDfvBisSTQtfixUeweQFGBnp6lG7Igy289GMQ+2GVY3q+vhQIP+Gbd5yLMFk
c2JFNuy6iGIb53qZYOF/BaOSHTiSD7Sfwy1f4CgqXTQeHl+gjG+yDQTSHP8sCaf5
iaz3wwxB0ERaQpwQ6FM7tINfdmEIocxcukcNw62hhvlpszerF1Td+V1wY3Eh9DGh
wHKETWBHZZFGlZVKMorSO0hjskfWiBc95/o65KLeuFeiNDFV0HSf0prvSW3Frk6p
dvIFPmSX/qYZeOqK/rjTURNBaVyB9dysKJ2BsoCq6yPJ3sOxGWtJjyp9kPPWZrJV
Eq97SxHvI6uv/Le86FZCygb8Fbm6yqdcNVuCI7lxJki8zpfwccQtg4IifwJHpwCK
80Tbq5VtQ8kcodoJxTeE
=XGSt
-END PGP SIGNATURE-

Re: Raising the severity of reproduciblity issues to "important"

[Russ Allbery]

Niels Thykier  writes:
> On 2015-08-24 21:24, Santiago Vila wrote:

>> We all want Debian to build reproducibly, but goals are achieved by
>> submitting bugs, changing packages and making uploads, not by rising
>> severities.

> I agree in general that people should make an effort to improve the
> situation before trying to solve this by simply "inflating" the
> severity.  However, in this case, I feel that the reproducibility team
> has done quite an effort to reach this point already.

> In your opinion, how much of the archive should be fixed before one can
> start bumping the severity?

>  * Personally, I considered the >80% fixed is quite convincing, but I
>would like to hear you take on this.

Yeah, Niels speaks for me here too.  That was exactly what I was thinking.

Putting aside technical issues for a moment, I would also encourage people
to think about the social aspects of this.  Reproducible builds is one of,
if not *the*, most successful large-scale, distribution-wide change that
I've seen anyone try to do in Debian for quite a while.  It came out of
DebConf, it's been very well-handled, the people involved have done all of
the things we ask people to do when making global changes but that people
rarely are willing to invest the time and energy to do, and throughout
this process the reproducible build team have been an absolute model of
how to make a change happen in Debian.

I feel like this is something that should be rewarded.  Personally, I've
done so by making reproducible build issues a much higher personal
priority for myself than I would otherwise, since this is exactly the sort
of project that I want to collaborate with.  The social approach taken by
the reproducible build team has been so good that it's gotten me to care
about something that I originally didn't care that much about.

Now, all of that doesn't mean that it has to be a project-wide goal, but I
think it's worth considering how we, as a community, want to support a
part of our community that cares greatly about something.  This is
something Debian has generally been fairly good at (with occasional
problems and occasional hard choices when there are competing priorities
that different people have been enthusiastic about).  This one seems to
not have as many priority conflicts as many, and people enthusiastically
working on clearing every technical bar.  For me, this means a *lot*.

-- 
Russ Allbery (r...@debian.org)   

Re: Raising the severity of reproduciblity issues to "important"

[Colin Tuckley]

On 24/08/15 21:42, Niels Thykier wrote:

> Are you aware that 37 out of 40 of your packages can currently be build
> reproducible in unstable using the patched toolchain (e.g. dpkg and
> debhelper).  This (I presume) is without you having done anything to
> make them explicitly reproducible.

Actually, I've incorporated patches in a few of them that were attached
to R-B bugs, some of them were a good idea and some of them I didn't
really agree with but they were not harmful.

> Reproducibility is not mutually exclusive with following upstream.  Many
> developers have forwarded patches upstream - which I hope you will
> consider doing, so our upstreams will benefit from these improvements as
> well.

I've forwarded some upstream, the ones that I considered to be fixes for
real bugs.

Colin

-- 
Colin Tuckley  |  +44(0)1223 830814  |  PGP/GnuPG Key Id
Debian Developer   |  +44(0)7799 143369  | 0x38C9D903

Re: Raising the severity of reproduciblity issues to "important"

[Vincent Bernat]

 ❦ 24 août 2015 21:12 +0100, Colin Tuckley  :

>> Well, I object strongly.
>
> Same here, in my view reproducibility is a 'nice to have' it should
> *never* be forced on a package.
>
> We are in the business of packaging upstream software for
> distribution. We should not make arbitrary changes to upstream
> software, such as changing the way a date is added to a man page, just
> to make the build reproducible.

We have pushed other archive-wide goals that were not shared by all
upstreams. For example, we have enabled hardening build flags on almost
all packages and for packages that don't obey to the appropriate flags,
bugs with severity "important" were filed. That's not that different of a
reproducible build.
-- 
Make sure all variables are initialised before use.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: Raising the severity of reproduciblity issues to "important"

[Niels Thykier]

Hi,

On 2015-08-24 22:06, Matthias Klose wrote:
> [...] So what about identifying categories which should be fixed in any
> case, and maybe which should have special rules for accelerated NMUs and such?

Personally, I find that proposal quite interesting.

> Categories would include:
> 
>  - running dh-autoreconf during the build to accommodate new archs.
> 
>  - respecting dpkg-buildflags for current "security" settings, and
>obsoleting the old manually coded -O0/-O2 for debug builds.
> 

Both of these seems very reasonable to me.

In fact, I could be convinced to help provide patches to maintainers
that would be interested in switching to the dh-sequencer at the same time.

>  - respecting DEB_BUILD_OPTIONS=parallel=. While this doesn't
>improve package quality, it will help to better use buildd resources.
>almost every architecture now uses multi cores. It will help Debian
>to faster process binNMUs / transitions at least for release archs.
> 

Indeed - I have been pondering on making the next compat level in
debhelper default to --parallel.

> [...]
> 
> So what about generalizing these categories into something which doesn't need
> asking again for every single package to be applied?
> 
> Matthias
> 

Conceptually, I am positive about the idea.  The problem will probably
be what "categories" we can agree to.  On the other hand, this can
trivially become an Achilles heel in this proposal - though I sincerely
hope I am wrong here.

Thanks,
~Niels






signature.asc
Description: OpenPGP digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Niels Thykier]

Hi,


On 2015-08-24 22:12, Colin Tuckley wrote:
> [...]
> Same here, in my view reproducibility is a 'nice to have' it should
> *never* be forced on a package.
> 
> We are in the business of packaging upstream software for
> distribution. We should not make arbitrary changes to upstream
> software, such as changing the way a date is added to a man page, just
> to make the build reproducible.
> 

Are you aware that 37 out of 40 of your packages can currently be build
reproducible in unstable using the patched toolchain (e.g. dpkg and
debhelper).  This (I presume) is without you having done anything to
make them explicitly reproducible.

>> [...]
>> We all want Debian to build reproducibly
> 
> Do we?

I think it was a figure of speech.

The extended version: A substantial part of the developer body seems to
be positive about it.  It seems that external parties are also quite
interested in this effort. Core Infrastructure Initiative donated
200,000 US dollars to support the reproducibility effort.

> Personally I'd rather stay true to the upstream.
> 

Reproducibility is not mutually exclusive with following upstream.  Many
developers have forwarded patches upstream - which I hope you will
consider doing, so our upstreams will benefit from these improvements as
well.


Thanks,
~Niels




signature.asc
Description: OpenPGP digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Niels Thykier]

On 2015-08-24 21:24, Santiago Vila wrote:
> [...]
> 

Hi Santiago,

> Making a great percentage of packages in the archive to be "suddenly"
> buggy is unacceptable.
> 

I can see where you are coming from.  I have to admit that I am
personally not too concerned with the severity change.  Given it is not
release critical, it is not going to throw packages out of testing or
anything like that.  But as Enrico taught us: YKINMKBYKIOK :)

> We all want Debian to build reproducibly, but goals are achieved by
> submitting bugs, changing packages and making uploads, not by rising
> severities.
> 

I agree in general that people should make an effort to improve the
situation before trying to solve this by simply "inflating" the
severity.  However, in this case, I feel that the reproducibility team
has done quite an effort to reach this point already.

In your opinion, how much of the archive should be fixed before one can
start bumping the severity?

 * Personally, I considered the >80% fixed is quite convincing, but I
   would like to hear you take on this.

> It is really so much difficult to make this in stages?
> For example:
> 
> Stage 1. Make it a policy *recommendation*, with normal severity.
> Stage 2. Make it a policy "should", with important severity.
> Stage 3. Make it a release goal, policy "must", with serious severity.
> 

It would be great if Policy was updated so frequently that was up to
speed with current initiatives in Debian.  However, historically, it
often lacks behind on many key parts.  Including but not limited to
Multi-Arch (released with Wheezy), which is currently still not
documented in policy (nor in git for the next upload).

Thanks,
~Niels

-- 

Also, contrary to popular belief:

  Policy does *not* decide if something is an RC bug or not.

That is currently delegated to the Release Team[1], which means that

  https://release.debian.org/stretch/rc_policy.txt

is the canonical policy for what is release critical and what is not.

[1] https://lists.debian.org/debian-devel-announce/2013/12/msg7.html

"""
 * Release Team members define the content of the suites listed above,
   that is:

   + They define the packages that are part of those suites. Generally,
 that is achieved:
 - by deciding which issues are release-critical (RC) -- making the
   affected packages not suitable for stable releases -- usually by
   setting the corresponding bug's severity to serious, grave or
   critical;
"""






signature.asc
Description: OpenPGP digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Colin Tuckley]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 24/08/15 20:24, Santiago Vila wrote:

> Well, I object strongly.

Same here, in my view reproducibility is a 'nice to have' it should
*never* be forced on a package.

We are in the business of packaging upstream software for
distribution. We should not make arbitrary changes to upstream
software, such as changing the way a date is added to a man page, just
to make the build reproducible.

> Making a great percentage of packages in the archive to be
> "suddenly" buggy is unacceptable.

+1

> We all want Debian to build reproducibly

Do we? Personally I'd rather stay true to the upstream.

If the reproducible builds advocates want to insist on something then
it should be a way of specifying an 'override' similar to that used
for Lintian errors and warnings.

Colin

- -- 
Colin Tuckley  |  +44(0)1223 830814  |  PGP/GnuPG Key Id
Debian Developer   |  +44(0)7799 143369  | 0x38C9D903

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJV23rDAAoJEPoMQQc4ydkDuN8QAKxAqoVSyW91571ZHJ5cPl5D
9m+aA6emwEHLpeVxUPxEoD7+M8uPuRTX8lUVzNUWiPN8nmcnhwJAam/8OZSLU5Dm
zMent+1V3y0Cc8+W+JVh2FUIE1gxtuRxZc4Oe0EoP8rsFmfSJwhnrSX4kbuKGZlT
MeAJNT0vP9apv3JPVJBUtC4xn0nI8yXwnYukvnhJKj0tcPhZGBNqMDc1SxL+rJOh
hBMI49E/DS8F3rTw25KPmuK3eT3XyPtqbmay7EFnNWbB1uKTYHXfbVIKvfkSn+Q/
s6l5CMVbYQDY0H3CJlLb+TSi3DITnNcpG1TIgpAlMHPhnoBYXOxOW2xKkH+uyCBB
y2qVKXa9y0feIXIhD9t8rcQjqqip3U5b4VetwzdFg5Jumh4/xjCrf3gu14N5cSYd
Is+iWM64wKp2FiBqV0KGpLt+ZrV5l5t3EUIinvChU7jW9OcmPnnum1sxLhJGtcMj
iPXXXrMfViwElla9My6hwRJgacRn783amgdvg++EpqJW1b/V60J9h8fxqA/y5Kbu
a1JjKJe4G+D73LbyWcdxyK9I67U1AGSu575/CI/E9S1YFty/6oajTWTXvOKaxkp8
RT8IbdQubAota1xwR75t5KEb4hv2Y+rKejyAnXxd5nrnlNxcqjuXfcCDaTa87XWI
jRGKisFKdFK7Th27Zn+u
=vXeV
-END PGP SIGNATURE-

Re: Raising the severity of reproduciblity issues to "important"

[Matthias Klose]

On 08/23/2015 12:48 PM, Chris Lamb wrote:
> Hi -devel,
> 
> The reproducible-builds team are currently contributing patches with
> "wishlist" severity.
> 
> This is because it is not currently possible to build reproducible
> packages within sid itself - we maintain a separate repository whilst
> our changes to the toolchain are pending review and consultation.
> 
> Filing these bugs with a higher severity -- which would require
> developers to use this repository to fully test any modifications --
> would be unacceptable.
> 
> However, based on an informal survey at DebConf (and to reflect the
> feeling towards software reproducibility in the free software community
> in general) unless there are strong objections I intend to raise the
> severity of these wishlist issues to "important" once the toolchain
> changes to dpkg and debhelper land in sid.

I don't like this kind of "ad hoc" decision.  Sure, there are reasons to raise
the severity for this particular category, however we have a lot of issues that
affect a lot of packages, and which are not fixed or don't get any review for a
long time.  So what about identifying categories which should be fixed in any
case, and maybe which should have special rules for accelerated NMUs and such?
Categories would include:

 - running dh-autoreconf during the build to accommodate new archs.

 - respecting dpkg-buildflags for current "security" settings, and
   obsoleting the old manually coded -O0/-O2 for debug builds.

 - respecting DEB_BUILD_OPTIONS=parallel=. While this doesn't
   improve package quality, it will help to better use buildd resources.
   almost every architecture now uses multi cores. It will help Debian
   to faster process binNMUs / transitions at least for release archs.

 - verbose build logs on the buildds. Doesn't add to quality immediately,
   but from my point of view helps a lot for analyzing build failures
   and other issues.

 - cross build-ability. surely this wouldn't be important, however
   would raise awareness for this requirement.

The good thing for the reproducibility issues is that you have an easy way to
find these.  It's maybe more difficult for the issues mentioned above, but maybe
not impossible.

So what about generalizing these categories into something which doesn't need
asking again for every single package to be applied?

Matthias

Re: Raising the severity of reproduciblity issues to "important"

[Santiago Vila]

On Sun, Aug 23, 2015 at 12:48:50PM +0200, Chris Lamb wrote:
> The reproducible-builds team are currently contributing patches with
> "wishlist" severity.
> 
> This is because it is not currently possible to build reproducible
> packages within sid itself - we maintain a separate repository whilst
> our changes to the toolchain are pending review and consultation.
> 
> Filing these bugs with a higher severity -- which would require
> developers to use this repository to fully test any modifications --
> would be unacceptable.
> 
> However, based on an informal survey at DebConf (and to reflect the
> feeling towards software reproducibility in the free software community
> in general) unless there are strong objections I intend to raise the
> severity of these wishlist issues to "important" once the toolchain
> changes to dpkg and debhelper land in sid.

Well, I object strongly.

Making a great percentage of packages in the archive to be "suddenly"
buggy is unacceptable.

We all want Debian to build reproducibly, but goals are achieved by
submitting bugs, changing packages and making uploads, not by rising
severities.

It is really so much difficult to make this in stages?
For example:

Stage 1. Make it a policy *recommendation*, with normal severity.
Stage 2. Make it a policy "should", with important severity.
Stage 3. Make it a release goal, policy "must", with serious severity.

Switching from wishlist to important in a single step does not make
sense to me.

Re: Raising the severity of reproduciblity issues to "important"

[Antonio Terceiro]

On Sun, Aug 23, 2015 at 12:48:50PM +0200, Chris Lamb wrote:
> Hi -devel,
> 
> The reproducible-builds team are currently contributing patches with
> "wishlist" severity.
> 
> This is because it is not currently possible to build reproducible
> packages within sid itself - we maintain a separate repository whilst
> our changes to the toolchain are pending review and consultation.
> 
> Filing these bugs with a higher severity -- which would require
> developers to use this repository to fully test any modifications --
> would be unacceptable.
> 
> However, based on an informal survey at DebConf (and to reflect the
> feeling towards software reproducibility in the free software community
> in general) unless there are strong objections I intend to raise the
> severity of these wishlist issues to "important" once the toolchain
> changes to dpkg and debhelper land in sid.

I think that reprocibility is an important issue, and marking
reprocibility bugs as important makes sense to me. That, please also
include a script to test builds for reprocibility in e.g.  devscripts
(see #786755) so that maintainers can actually test it themselves.

-- 
Antonio Terceiro 


signature.asc
Description: Digital signature

Re: Raising the severity of reproduciblity issues to "important"

[Olivier Berger]

Hi.

Chris Lamb  writes:

> Hi -devel,
>
> The reproducible-builds team are currently contributing patches with
> "wishlist" severity.
>
> This is because it is not currently possible to build reproducible
> packages within sid itself - we maintain a separate repository whilst
> our changes to the toolchain are pending review and consultation.
>
> Filing these bugs with a higher severity -- which would require
> developers to use this repository to fully test any modifications --
> would be unacceptable.
>
> However, based on an informal survey at DebConf (and to reflect the
> feeling towards software reproducibility in the free software community
> in general) unless there are strong objections I intend to raise the
> severity of these wishlist issues to "important" once the toolchain
> changes to dpkg and debhelper land in sid.
>

For the records, I guess the recording of the conference [0] about
reproducibility at Debconf 15 should be a must watch :

http://meetings-archive.debian.net/pub/debian-meetings/2015/debconf15/Stretching_out_for_trustworthy_reproducible_builds_creating_bit_by_bit_identical_binaries.webm

My 2 cents,

Best regards,

P.S.: many thanks for this important work !

[0] 
https://summit.debconf.org/debconf15/meeting/353/stretching-out-for-trustworthy-reproducible-builds-creating-bit-by-bit-identical-binaries/
-- 
Olivier BERGER 
http://www-public.telecom-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8
Ingenieur Recherche - Dept INF
Institut Mines-Telecom, Telecom SudParis, Evry (France)

Raising the severity of reproduciblity issues to "important"

[Chris Lamb]

Hi -devel,

The reproducible-builds team are currently contributing patches with
"wishlist" severity.

This is because it is not currently possible to build reproducible
packages within sid itself - we maintain a separate repository whilst
our changes to the toolchain are pending review and consultation.

Filing these bugs with a higher severity -- which would require
developers to use this repository to fully test any modifications --
would be unacceptable.

However, based on an informal survey at DebConf (and to reflect the
feeling towards software reproducibility in the free software community
in general) unless there are strong objections I intend to raise the
severity of these wishlist issues to "important" once the toolchain
changes to dpkg and debhelper land in sid.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-