Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
hi Guillem, On Fri, Nov 09, 2018 at 11:55:38AM +0100, Guillem Jover wrote: > Actually, I guess the other option that might be an option for stable is > to make dpkg-buildpackage generate the buildinfo file itself, and on > source-only uploads force the name to be _source.buildinfo regardless > of the options passed down to dpkg-genbuildinfo (even if the contents > will end up not matching the name). > > This would seem rather less intrusive, as that only changes the > behavior in a "corner-case" (even though documented and recommended > one), when using «dpkg-buildpackage --changes-option=-S». And while it > could be considered to produce confusing filenames, it sticks to the > current pattern. It would also fix the other bug where running > dpkg-genbuildinfo leaves debian/files around, even on source only > builds. > > So, I might go with that instead. any update on this? the security team people still have to workaround this manually regularily, eg today, and would really like to see this fixed... -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C "... the premise [is] that privacy is about hiding a wrong. It's not. Privacy is an inherent human right, and a requirement for maintaining the human condition with dignity and respect." (Bruce Schneier) signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi,
On Sat, Aug 17, 2019 at 10:37:43AM +0200, Salvatore Bonaccorso wrote:
> Hi Ansgar,
>
> On Wed, Jun 19, 2019 at 08:39:50AM +0200, Salvatore Bonaccorso wrote:
> > Hi Ansgar,
> >
> > On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote:
> > [...]
> > > > Sure, I understand that things works like that, I'm just showing a few
> > > > design points that could potentially be done differently.
> > >
> > > We could also just not accept .buildinfo uploads when they don't contain
> > > useful information about published binaries, that is for source-only
> > > uploads.
> > >
> > > Maybe I should reenable the check for this at least on security-master?
> > > It was rejecting uploads that are okay for unstable/experimental so I
> > > disabled it again the last time.
> >
> > Thank you I think that would be a good compromise. Source-only uploads
> > remain possible for security uploads, and ftp-masters and security
> > team members do not need to roundtrip reuploading binary builds
> > (download, rename, resign ... reupload) and instead uploads which
> > contain a buildinfo file rejected giving the uploader a explanation
> > why, and the possiblity to just reupload a "proper" source only one.
>
> We had a couple of occasions still where we needed to do fetch the
> builds, rename, resign and reupload the packages.
>
> Any chance to enable the above check again? That would solve most of
> or issues and people which did upload a source only upload but
> containing a ${arch}.buildinfo would get their uploads rejected.
We still have recurring this issue, where people continue to upload
_sources.changes (even more frequently now I would say, given the
workflow for unstable/testing) but which include a _amd64.buildinfo.
The last one was the (not yet released) unbound update.
So if we can have this check enabled again for the security archive
this would help us a lot not having to workaround those rejected
uploads :)
Thanks for all your work!
Regards,
Salvatore
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi, On Fri, Mar 02, 2018 at 01:25:51AM +0100, Guillem Jover wrote: > On Thu, 2018-03-01 at 15:22:30 +, Holger Levsen wrote: > > On Wed, Jan 24, 2018 at 04:05:39PM +0100, Salvatore Bonaccorso wrote: > > > Any news regarding this proposal from Ansgar? We were biten now > > > several times already by this (e.g. php update, curl via > > > security.d.o). > > > > Guilem, what's your stance on this bug? > > I think it should be fixed. I've just not come up with something that > would seem a generic/clean way to do that. :( > > > We (reproducible builds) really dont want "our" tools (=.buildinfo files) > > to cause grief to other teams in Debian, and especially not on a regular > > basis... so as a first step to fix this, I'd like to collect opinions > > how to best fix this issue here. > > The problem got introduced when I proposed changing the filename > format, and dropping the annoying timestamp. I also though there was > talk at some point initially about DAK renaming those files to cope > with possible multiple uploads if the conflicting names? > > Renaming the arch buildinfo to _source.buildinfo seems wrong, and even > then I'm not sure how to cleanly transfer that knowledge from > dpkg-buildpackage to dpkg-genbuildinfo. > > I guess, the ideal solution would be to qualify the buildinfo file > with the builder user and hostname, because that in a way denotes the > build environment. But that seems like too much leakage. As in: > > pkgfoo_1.0.0-1_mips64el_username@hostname.buildinfo > > Perhaps just using the maintainer email address might be enough though, > the one from the -m option or from the changelog, which AFAIR buildds > do set? But this seems like it can produce quite ugly filenames: > > > pkgfoo_1.0.0-1_mips64el_buildd_mipsel-mipsel-sil...@buildd.debian.org.buildinfo > > not to mention that both of these "break" the conventional pattern, which > is still used by things like the debian/files parser and injector. I submitted a merge request to sbuild to add a suffix like this to the buildinfo and changes files: https://salsa.debian.org/debian/sbuild/merge_requests/6 This would work around this issue, without needing any changes in maintainer workflows. The suffix could just be 'buildd', no need to have the buildd name in there. The filename just needs to be different from the filename uploaded by the maintainer. Obviously it could contain the buildd name if people think that would be useful. This change in sbuild only needs to happen on the buildd system. No change is needed in the build chroots. So it should work for stretch and newer without the need for changes to dpkg (or any other package) in those releases (jessie doesn't have buildinfo files). The patch also adds the suffix to the .changes file, to avoid a similar issue there. This happens when the maintainer manually removes binaries from a changes file to do a source-only upload. It would also potentially happen if dak supported throw-away binaries. Cheers, Ivo ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi Ansgar,
On Wed, Jun 19, 2019 at 08:39:50AM +0200, Salvatore Bonaccorso wrote:
> Hi Ansgar,
>
> On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote:
> [...]
> > > Sure, I understand that things works like that, I'm just showing a few
> > > design points that could potentially be done differently.
> >
> > We could also just not accept .buildinfo uploads when they don't contain
> > useful information about published binaries, that is for source-only
> > uploads.
> >
> > Maybe I should reenable the check for this at least on security-master?
> > It was rejecting uploads that are okay for unstable/experimental so I
> > disabled it again the last time.
>
> Thank you I think that would be a good compromise. Source-only uploads
> remain possible for security uploads, and ftp-masters and security
> team members do not need to roundtrip reuploading binary builds
> (download, rename, resign ... reupload) and instead uploads which
> contain a buildinfo file rejected giving the uploader a explanation
> why, and the possiblity to just reupload a "proper" source only one.
We had a couple of occasions still where we needed to do fetch the
builds, rename, resign and reupload the packages.
Any chance to enable the above check again? That would solve most of
or issues and people which did upload a source only upload but
containing a ${arch}.buildinfo would get their uploads rejected.
Regards,
Salvatore
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Mon, 08 Jul 2019 16:39:30 -0700 Vagrant Cascadian wrote: > On 2019-07-09, Francesco Poli wrote: [...] > > Maybe it's a naive question: what's the use of including a .buildinfo > > file into a source-only upload? Is it superfluous? > > It's an attestation from the uploader that they built the package, what > the hashes of their build are, and the build environment used to produce > those binary packages. If there are differences between the buildd build > of the package and the uploader's build of the package, it may be > possible to compare the results. [...] > > This is certainly desireable from a reproducible builds perspective. I see: this makes perfect sense, sure. Thanks a lot for the clear explanation! [...] > > Would it be better, if dpkg-genchanges completely refrained from > > including .buildinfo files into source-only .changes files? > > Or calling the .buildinfo by a different name, e.g. source.buildinfo or > source-arch.buildinfo, etc. Maybe an alternative strategy could work as follows: • dpkg-genchanges should continue to do exactly what it currently does • the autobuilders should rename their own .buildinfo files on the fly, to something like package_version_arch-buildd.buildinfo This way, we could avoid name clashes, without the need to fix the tools used by all the people who upload packages... Could this strategy be the way to go? Does it have any important drawback? -- http://www.inventati.org/frx/ There's not a second to spare! To the laboratory! . Francesco Poli . GnuPG key fpr == CA01 1147 9CD2 EFDF FB82 3925 3E1C 27E1 1F69 BFFE pgpUtoXh2S3Zy.pgp Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On 2019-07-09, Francesco Poli wrote:
> On Wed, 19 Jun 2019 08:39:50 +0200 Salvatore Bonaccorso wrote:
>
> [...]
>> On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote:
> [...]
>> > We could also just not accept .buildinfo uploads when they don't contain
>> > useful information about published binaries, that is for source-only
>> > uploads.
>> >
>> > Maybe I should reenable the check for this at least on security-master?
>> > It was rejecting uploads that are okay for unstable/experimental so I
>> > disabled it again the last time.
>>
>> Thank you I think that would be a good compromise. Source-only uploads
>> remain possible for security uploads,
> [...]
>> and instead uploads which
>> contain a buildinfo file rejected giving the uploader a explanation
>> why, and the possiblity to just reupload a "proper" source only one.
> [...]
>
>
> Hello all,
> sorry for adding noise to this long discussion.
>
> Maybe it's a naive question: what's the use of including a .buildinfo
> file into a source-only upload? Is it superfluous?
It's an attestation from the uploader that they built the package, what
the hashes of their build are, and the build environment used to produce
those binary packages. If there are differences between the buildd build
of the package and the uploader's build of the package, it may be
possible to compare the results.
I usually use sbuild's --source-only-changes feature to produce a
.changes that also includes the .buildinfo that produced my local debs
without uploading the actual .deb files to the archive.
This is certainly desireable from a reproducible builds perspective.
> If it's indeed superfluous and it also causes issues on some queues,
> maybe source-only .changes files should _never_ include .buildinfo
> files...
.buildinfo files without any hashes of .deb files do seem less useful,
unless you consider the building of the .dsc as a build process
itself.
> But dpkg-genchanges shipped in dpkg-dev/1.19.7 seems to
> include .buildinfo files into every .changes file (even for source-only
> uploads):
>
> [snip from /usr/bin/dpkg-genchanges]
> 317 if (defined $file->{package} && $file->{package_type} eq 'buildinfo')
> {
> 318 # We always distribute the .buildinfo file.
> 319 $checksums->add_from_file("$uploadfilesdir/$f", key => $f);
> 320 next;
> 321 }
> 322
> 323 # If this is a source-only upload, ignore any other artifacts.
> 324 next if build_has_none(BUILD_BINARY);
>
> This puzzles me.
>
> Would it be better, if dpkg-genchanges completely refrained from
> including .buildinfo files into source-only .changes files?
Or calling the .buildinfo by a different name, e.g. source.buildinfo or
source-arch.buildinfo, etc.
> At the same time, dak could reject any source-only upload which also
> include a .buildinfo file.
I hope this doesn't end up being the solution.
Though a workable solution would be really welcome at this point...
live well,
vagrant
signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Wed, 19 Jun 2019 08:39:50 +0200 Salvatore Bonaccorso wrote:
[...]
> On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote:
[...]
> > We could also just not accept .buildinfo uploads when they don't contain
> > useful information about published binaries, that is for source-only
> > uploads.
> >
> > Maybe I should reenable the check for this at least on security-master?
> > It was rejecting uploads that are okay for unstable/experimental so I
> > disabled it again the last time.
>
> Thank you I think that would be a good compromise. Source-only uploads
> remain possible for security uploads,
[...]
> and instead uploads which
> contain a buildinfo file rejected giving the uploader a explanation
> why, and the possiblity to just reupload a "proper" source only one.
[...]
Hello all,
sorry for adding noise to this long discussion.
Maybe it's a naive question: what's the use of including a .buildinfo
file into a source-only upload? Is it superfluous?
If it's indeed superfluous and it also causes issues on some queues,
maybe source-only .changes files should _never_ include .buildinfo
files...
But dpkg-genchanges shipped in dpkg-dev/1.19.7 seems to
include .buildinfo files into every .changes file (even for source-only
uploads):
[snip from /usr/bin/dpkg-genchanges]
317 if (defined $file->{package} && $file->{package_type} eq 'buildinfo') {
318 # We always distribute the .buildinfo file.
319 $checksums->add_from_file("$uploadfilesdir/$f", key => $f);
320 next;
321 }
322
323 # If this is a source-only upload, ignore any other artifacts.
324 next if build_has_none(BUILD_BINARY);
This puzzles me.
Would it be better, if dpkg-genchanges completely refrained from
including .buildinfo files into source-only .changes files?
At the same time, dak could reject any source-only upload which also
include a .buildinfo file.
Perhaps I am completely off-track.
Please someone clarify!
--
http://www.inventati.org/frx/
There's not a second to spare! To the laboratory!
. Francesco Poli .
GnuPG key fpr == CA01 1147 9CD2 EFDF FB82 3925 3E1C 27E1 1F69 BFFE
pgpqLg6m_Cf1g.pgp
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi Ansgar, On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote: [...] > > Sure, I understand that things works like that, I'm just showing a few > > design points that could potentially be done differently. > > We could also just not accept .buildinfo uploads when they don't contain > useful information about published binaries, that is for source-only > uploads. > > Maybe I should reenable the check for this at least on security-master? > It was rejecting uploads that are okay for unstable/experimental so I > disabled it again the last time. Thank you I think that would be a good compromise. Source-only uploads remain possible for security uploads, and ftp-masters and security team members do not need to roundtrip reuploading binary builds (download, rename, resign ... reupload) and instead uploads which contain a buildinfo file rejected giving the uploader a explanation why, and the possiblity to just reupload a "proper" source only one. Regards, Salvatore ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Tue, 2019-06-18 at 21:20 +0200, Mattia Rizzolo wrote: > That would indeed be a fine workaround for me, and reduce the load the > security team is experience, since it's the team which is the most > affect by this. > (Incidentally, it also is the same way launchpad works, there you can't > upload a .buildinfo for an arch that you aren't uploading, and humans > can't upload binaries, so you can't upload .buildinfo for binaries at > all). But some tools (at least pbuilder) generate such (meaningful, since there was a binary build) files. It's already been said earlier but if the uploads are considered invalid it should be consistent accross the archive (not just on security-master) and tools should not generate them. Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAl0J1xkACgkQ3rYcyPpX RFsxowgA7bIn+1RfeJ6J7xv7Gxjh+WE5xZGbhOv+sDWgVwkJDiPAiW4tIMKU6qrw 17ghR+m7jo1PNZqr+boDZ851UVQOD5ii4SsyWBbesbMLPn2hNaBZN93El3pe4ni0 EgIeePe2d6wez+zZjiubdKEAZMuf7ezq3+9EuXuQDjSKmWV6PSu90i5/ncl6AByW /3SWQmt4sgUlr6HoR60B586d3eVVg82Hd/0GQBPinkgyp57G+R4z7HpRTPrYFmM3 QRIkcBhBcvG4FI7AdV/b1ki0iXPvwXrucOTxzBKWoehqFwA3kvJUZf+vBi9I93VW THIx8duOD0M7VoLNS6ohJHWZ8MyWrg== =xZJB -END PGP SIGNATURE- ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Tue, Jun 18, 2019 at 09:03:23PM +0200, Ansgar Burchardt wrote: > > Also here, it feels to me that once something is accepted into a policy > > queue, dak should already consider it something controlled by itself, > > store checksums in the database and be done, not keep the .changes > > around as a "source of truth" for additional processing, imho. > > Throwing away .changes doesn't help with .buildinfo name conflicts. It helps in the sense that once the .changes is thrown away, dak would be free to rename the .buildinfo however it likes. > We could also just not accept .buildinfo uploads when they don't contain > useful information about published binaries, that is for source-only > uploads. > > Maybe I should reenable the check for this at least on security-master? > It was rejecting uploads that are okay for unstable/experimental so I > disabled it again the last time. That would indeed be a fine workaround for me, and reduce the load the security team is experience, since it's the team which is the most affect by this. (Incidentally, it also is the same way launchpad works, there you can't upload a .buildinfo for an arch that you aren't uploading, and humans can't upload binaries, so you can't upload .buildinfo for binaries at all). -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Tue, Jun 18, 2019 at 06:29:12PM +0200, Ansgar Burchardt wrote: > The .buildinfo files are referred to in the .changes files; renaming > them would require updating the .changes file. The .changes files are > used to upload the security updates to ftp-master. With .changes being ephemeral, it feels to me that using them to cross the archive is not really a good solution, and whatever is used to copy packages from one archive to another (is it dak itself?) should instead re-create the upload and re-sign it. Also because that way it would be perfectly able to "upload" all of the sources+binaries from sec-master to ftp-master in a single go, which can't be bad. > ftp-master also has the same problem when uploads end up in policy > queues (the renaming to .buildinfo.N is only done when dak is "done" > with the file and will never touch it again). Also here, it feels to me that once something is accepted into a policy queue, dak should already consider it something controlled by itself, store checksums in the database and be done, not keep the .changes around as a "source of truth" for additional processing, imho. Sure, I understand that things works like that, I'm just showing a few design points that could potentially be done differently. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Mon, 2019-06-17 at 13:12 -0700, Vagrant Cascadian wrote: > > This behaviour is really causing issues for the security-archive so in > > one way or the other there needs to be a solution. Regularly we need > > to fetch the buildd changes and build binary packages, resign them and > > reupload them due to this bug. > > What's unclear to me is why the workaround in DAK for the main archive, > which adds .buildinfo.N for duplicate .buildinfo filenames, can't be or > hasn't been applied for the security archive. Is there something > fundamentally different with the security archive? The .buildinfo files are referred to in the .changes files; renaming them would require updating the .changes file. The .changes files are used to upload the security updates to ftp-master. ftp-master also has the same problem when uploads end up in policy queues (the renaming to .buildinfo.N is only done when dak is "done" with the file and will never touch it again). As most uploads go to unstable and experimental, one mostly doesn't notice the issue. Ansgar ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On 2019-06-17, Salvatore Bonaccorso wrote: > On Sun, Jun 16, 2019 at 01:49:24PM -0400, Daniel Kahn Gillmor wrote: >> On Sun 2019-06-16 15:50:55 +0200, Ivo De Decker wrote: >> > As "--changes-option=-S" creates an upload that is broken from the point of >> > view of the archive, it might make sense not to recommend (or even allow) >> > this >> > for now. Just building with "-S" instead should create a buildinfo file >> > with >> > _source, which won't trigger this issue. >> >> For the rest of the regular archive, --changes-option=-S is definitely >> *not* broken. I use that regularly, and strongly prefer it. It allows >> me to document what i managed to build, while still ensuring that the >> distributed binaries are created by debian's buildd network, and not my >> own machinery. >> >> I would be pretty sad if --changes-option=-S was explicitly deprecated >> in any part of the debian archive. > > This behaviour is really causing issues for the security-archive so in > one way or the other there needs to be a solution. Regularly we need > to fetch the buildd changes and build binary packages, resign them and > reupload them due to this bug. What's unclear to me is why the workaround in DAK for the main archive, which adds .buildinfo.N for duplicate .buildinfo filenames, can't be or hasn't been applied for the security archive. Is there something fundamentally different with the security archive? It seems quite late in the freeze cycle to get this fixed in dpkg even for buster, so it seems worth considering fixing in the archive, unless I'm missing something? > Prefered for us would defintively to find a solution though which does > not mean the need to disable source only uploads for security-master, > that IMHO would be a read drawback. > > That said, sorry it looks I'm repeating myself, but I wanted to > express again that this causes real issues for the work on releasing > security-updates via the security archive. Really hard to see that this has dragged on for almost two years now without resolution! live well, vagrant signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi Daniel, On Sun, Jun 16, 2019 at 01:49:24PM -0400, Daniel Kahn Gillmor wrote: > On Sun 2019-06-16 15:50:55 +0200, Ivo De Decker wrote: > > As "--changes-option=-S" creates an upload that is broken from the point of > > view of the archive, it might make sense not to recommend (or even allow) > > this > > for now. Just building with "-S" instead should create a buildinfo file with > > _source, which won't trigger this issue. > > For the rest of the regular archive, --changes-option=-S is definitely > *not* broken. I use that regularly, and strongly prefer it. It allows > me to document what i managed to build, while still ensuring that the > distributed binaries are created by debian's buildd network, and not my > own machinery. > > I would be pretty sad if --changes-option=-S was explicitly deprecated > in any part of the debian archive. This behaviour is really causing issues for the security-archive so in one way or the other there needs to be a solution. Regularly we need to fetch the buildd changes and build binary packages, resign them and reupload them due to this bug. Prefered for us would defintively to find a solution though which does not mean the need to disable source only uploads for security-master, that IMHO would be a read drawback. That said, sorry it looks I'm repeating myself, but I wanted to express again that this causes real issues for the work on releasing security-updates via the security archive. Regards, Salvatore ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Sun, 2019-06-16 at 15:50 +0200, Ivo De Decker wrote: > > We regularly get biten by this issue when contributors to security > > uploads, most recently with the bind9 upload but as well others. > > Is it clear in what cases this issue happens? Guillem mentioned > "dpkg-buildpackage --changes-option=-S" in https://bugs.debian.org/869184#75 > Are there any other use cases that trigger it? > > As "--changes-option=-S" creates an upload that is broken from the point of > view of the archive, it might make sense not to recommend (or even allow) this > for now. Just building with "-S" instead should create a buildinfo file with > _source, which won't trigger this issue. I'm my case I'm just using pbuilder with SOURCE_ONLY_CHANGES=yes, so it builds a complete (arch:all + arch:any) package, then generate a .changes for source- only upload (but there's the amd64.buildinfo included). Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAl0GWt8ACgkQ3rYcyPpX RFtQ8ggAnZSlfTWlEx4sLTfq59wI7ooeFwqRn84tuF5o9wGkmu6TiqvL5iKiw2l5 4j8x/bmwvTw4VE4QW7tMCnZ3VEfiZA4NXBkFVoCvwhiDFKWhzZL058yfoRqMMIhf O/GAnZ0FIjwn3s0k5K6TEFPHNA9CdIhHWtLBnzVfwIZt3QeuVYE0CTE9ZehTrGZe ygr2mlyNjO+izUwqlacwyDV7vH6p0789I6ulYKn/2AfxRxf/S7C+GmKbIrXy8e+9 xTWDcuudK9BmZU2WjtzY43ER7gyzFx8AHv89AXmWZ9jcaBiFcbd0K7pKnAPsY/+x +FwaxjrWWoIquJezCQ0RGmtPdxpE3Q== =L3gm -END PGP SIGNATURE- ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi, On Thu, May 09, 2019 at 07:24:56PM +0200, Salvatore Bonaccorso wrote: > > On Sun, Nov 11, 2018 at 08:38:36AM +0100, Salvatore Bonaccorso wrote: > > > On Fri, Nov 09, 2018 at 11:48:27AM +0100, Guillem Jover wrote: > > > > On Thu, 2018-11-08 at 20:28:57 +, Holger Levsen wrote: > > > > > On Thu, Nov 08, 2018 at 09:24:01PM +0100, Salvatore Bonaccorso wrote: > > > > > in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869184#40 Guillem > > > > > wrote: > > > > > > > > > >Perhaps the simplest and more correct might be to name it using > > > > >something like source+amd64 as the arch name, which seems like a > > > > >dubious arch, but at least is accurate and might be trivial to > > > > >implement, taking care of not ending up with such fake arch in > > > > >unexpected places… > > > > > > > > > > and I cannot find anything wrong with this simple solution and have > > > > > already asked Guillem in August to implement this ;) > > > > > > > > So, as I mentioned at the time this would require modifing the internal > > > > filtering of the debian/files entries to cover this case in several of > > > > the tools. It also modifies the documented filename pattern in > > > > deb-buildinfo(5). This is all solvable and I should probably just do it. > > > > But this breaks previous public filename "interfaces", seems rather > > > > intrusive, and entirely inappropriate for a stable update, which means > > > > it would not fix your immediate problems anyway, only starting with > > > > Buster. :/ > > > Although this would not help us for stretch-security uploads, such a > > > move starting from buster would be very appreciated! Guillem, back in November Salvatore said they would appreciate this "source+amd64 as the arch name" solution for this bug (see above), while now (because nothing happened I believe) he suggests disabling source all uploads for security builds, which IMO would be a *very* bad and sad outcome, as I believe source only security uploads are even more desired than regular source only uploads... > We regularly get biten by this issue when contributors to security > uploads, most recently with the bind9 upload but as well others. > > Would it be possible to at least workaround this on dak's side? > Disabling source-only uploads completely would seem to be a step back > on that regards. > > Though if that's the only way out of having to regularly fetch the > rejected builds, do manual renamings and resigning and reuploading of > files, then we should then just disable source-only uploads for the > security suites again. > > So I think we pretty much would prefer to be able to continue so. > > Just to be clear, thanks a lot for your work, this is not meant as > critique, just hilighting that we have recurring issues due to this > bug when people perform uploads for security. sigh, understandable... -- tschau, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi
The following question goes maybe specifically to Ansgar, from
dak/ftp-master perspective:
On Sun, Nov 11, 2018 at 08:38:36AM +0100, Salvatore Bonaccorso wrote:
> Hi Guillem!
>
> On Fri, Nov 09, 2018 at 11:48:27AM +0100, Guillem Jover wrote:
> > Hi!
> >
> > On Thu, 2018-11-08 at 20:28:57 +, Holger Levsen wrote:
> > > On Thu, Nov 08, 2018 at 09:24:01PM +0100, Salvatore Bonaccorso wrote:
> > > > We were again biten by this issue for some security-updates (most
> > > > recent one nginx). Do any involved parties know, was there any
> > > > progress in adressing this problem?
> >
> > Sorry, had your earlier mail from July marked for reply, but it seems
> > I missed that. :/
> >
> > > in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869184#40 Guillem
> > > wrote:
> > >
> > >Perhaps the simplest and more correct might be to name it using
> > >something like source+amd64 as the arch name, which seems like a
> > >dubious arch, but at least is accurate and might be trivial to
> > >implement, taking care of not ending up with such fake arch in
> > >unexpected places…
> > >
> > > and I cannot find anything wrong with this simple solution and have
> > > already asked Guillem in August to implement this ;)
> >
> > So, as I mentioned at the time this would require modifing the internal
> > filtering of the debian/files entries to cover this case in several of
> > the tools. It also modifies the documented filename pattern in
> > deb-buildinfo(5). This is all solvable and I should probably just do it.
> > But this breaks previous public filename "interfaces", seems rather
> > intrusive, and entirely inappropriate for a stable update, which means
> > it would not fix your immediate problems anyway, only starting with
> > Buster. :/
>
> Although this would not help us for stretch-security uploads, such a
> move starting from buster would be very appreciated!
>
> Thanks a lot for your time investing in it, very much appreicated.
We regularly get issue still with that given one easily forgets about
the issue (the last occurence whas the php7.0 upload for
stretch-security as of yesterday). I wonder thus: would it be easily
workaroundable on ftp-master/dak's side to perform some additional
checks in that direction and reject such uploads wich would contain a
_${arch}.buildinfo file?
Any help in this regard would be very welcome from security team's
perspective, as we -- although an easy step -- need to fetch rejected
packages, rename files, resign and upload.
Sorry for always returning with this issue to both you and dpkg
maintainers.
Regards,
Salvatore
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
On Fri, 2018-11-09 at 11:55:38 +0100, Guillem Jover wrote: > Actually, I guess the other option that might be an option for stable is > to make dpkg-buildpackage generate the buildinfo file itself, and on > source-only uploads force the name to be _source.buildinfo regardless > of the options passed down to dpkg-genbuildinfo (even if the contents > will end up not matching the name). > > This would seem rather less intrusive, as that only changes the > behavior in a "corner-case" (even though documented and recommended > one), when using «dpkg-buildpackage --changes-option=-S». And while it > could be considered to produce confusing filenames, it sticks to the > current pattern. It would also fix the other bug where running > dpkg-genbuildinfo leaves debian/files around, even on source only > builds. Err, ugh, disregard all the above. This would still generate the _.buildinfo pattern, because we are under a full build, and the only option passed is for .changes, an option which we do not parse from dpkg-buildpackage anyway. If the .changes file was generated by dpkg-genchanges itself then it could end up generating _source.changes, but that would not help here at all anyway… Thanks, Guillem ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi! On Fri, 2018-11-09 at 11:48:27 +0100, Guillem Jover wrote: > On Thu, 2018-11-08 at 20:28:57 +, Holger Levsen wrote: > > On Thu, Nov 08, 2018 at 09:24:01PM +0100, Salvatore Bonaccorso wrote: > > > We were again biten by this issue for some security-updates (most > > > recent one nginx). Do any involved parties know, was there any > > > progress in adressing this problem? > > Sorry, had your earlier mail from July marked for reply, but it seems > I missed that. :/ > > > in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869184#40 Guillem > > wrote: > > > >Perhaps the simplest and more correct might be to name it using > >something like source+amd64 as the arch name, which seems like a > >dubious arch, but at least is accurate and might be trivial to > >implement, taking care of not ending up with such fake arch in > >unexpected places… > > > > and I cannot find anything wrong with this simple solution and have > > already asked Guillem in August to implement this ;) > > So, as I mentioned at the time this would require modifing the internal > filtering of the debian/files entries to cover this case in several of > the tools. It also modifies the documented filename pattern in > deb-buildinfo(5). This is all solvable and I should probably just do it. > But this breaks previous public filename "interfaces", seems rather > intrusive, and entirely inappropriate for a stable update, which means > it would not fix your immediate problems anyway, only starting with > Buster. :/ Actually, I guess the other option that might be an option for stable is to make dpkg-buildpackage generate the buildinfo file itself, and on source-only uploads force the name to be _source.buildinfo regardless of the options passed down to dpkg-genbuildinfo (even if the contents will end up not matching the name). This would seem rather less intrusive, as that only changes the behavior in a "corner-case" (even though documented and recommended one), when using «dpkg-buildpackage --changes-option=-S». And while it could be considered to produce confusing filenames, it sticks to the current pattern. It would also fix the other bug where running dpkg-genbuildinfo leaves debian/files around, even on source only builds. So, I might go with that instead. Thanks, Guillem ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi! On Thu, 2018-11-08 at 20:28:57 +, Holger Levsen wrote: > On Thu, Nov 08, 2018 at 09:24:01PM +0100, Salvatore Bonaccorso wrote: > > We were again biten by this issue for some security-updates (most > > recent one nginx). Do any involved parties know, was there any > > progress in adressing this problem? Sorry, had your earlier mail from July marked for reply, but it seems I missed that. :/ > in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869184#40 Guillem > wrote: > >Perhaps the simplest and more correct might be to name it using >something like source+amd64 as the arch name, which seems like a >dubious arch, but at least is accurate and might be trivial to >implement, taking care of not ending up with such fake arch in >unexpected places… > > and I cannot find anything wrong with this simple solution and have > already asked Guillem in August to implement this ;) So, as I mentioned at the time this would require modifing the internal filtering of the debian/files entries to cover this case in several of the tools. It also modifies the documented filename pattern in deb-buildinfo(5). This is all solvable and I should probably just do it. But this breaks previous public filename "interfaces", seems rather intrusive, and entirely inappropriate for a stable update, which means it would not fix your immediate problems anyway, only starting with Buster. :/ Thanks, Guillem ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#869184: dpkg: source uploads including _amd64.buildinfo cause problems
Hi We were again biten by this issue for some security-updates (most recent one nginx). Do any involved parties know, was there any progress in adressing this problem? Sorry I know, probably patches and ideas welcome, but I cannot contribute here, take my question please just from my "users" point of view. Regards, Salvatore ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
