when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Hi, On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: > this is just a pledge to you all fellow debian developers to update your > build environment before you build a package. I want all binary packages to be rebuild on *.debian.org hosts. Everything else is just an ugly workaround. amen, Holger P.S.: and reproducible builds after that, then... signature.asc Description: This is a digitally signed message part.
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
No kidding! How many uploaded binaries might include malware? A lack of binary determinism in the build process basically ensures that it isn't feasible to discover an answer to this question. :( All the best, Jacob On 2/13/14, Holger Levsen wrote: > Hi, > > On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: >> this is just a pledge to you all fellow debian developers to update your >> build environment before you build a package. > > I want all binary packages to be rebuild on *.debian.org hosts. Everything > else is just an ugly workaround. > > > amen, > Holger > > P.S.: and reproducible builds after that, then... > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cafggdf1igi23fahwhsswhsace1xsvviwnob+i5n+skyh4bq...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 06:36:15PM +, Jacob Appelbaum wrote: > No kidding! > > How many uploaded binaries might include malware? > > A lack of binary determinism in the build process basically ensures > that it isn't feasible to discover an answer to this question. :( > > All the best, > Jacob I'm sure the https://wiki.debian.org/ReproducibleBuilds team would love some help! Cheers, Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
* Jacob Appelbaum , 2014-02-13, 18:36: How many uploaded binaries might include malware? *shrug* It's not like it's difficult to hide malicious code in source packages. How many configure scripts that we never rebuild from source contains trojans? -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140213184653.ga5...@jwilk.net
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 13 February 2014 16:13, Holger Levsen wrote: > Hi, > > On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: >> this is just a pledge to you all fellow debian developers to update your >> build environment before you build a package. > > I want all binary packages to be rebuild on *.debian.org hosts. Everything > else is just an ugly workaround. > All that's needed, I guess, is for someone to write a patch to dak / wanna-build ... and schedule _all.deb builds on amd64 ? Or if arch-restricted package, on one of the arches it will build on? -- Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhluizs-yu8-nnojv7z7fyyxsdmrl-6ff2zgeskqv8jc4...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 07:46:53PM +0100, Jakub Wilk wrote: > *shrug* It's not like it's difficult to hide malicious code in > source packages. > > How many configure scripts that we never rebuild from source > contains trojans? Just like my favourite Russ quote: Basically, people got tired of portability problems in building shared libraries so they hid them all inside a multi-thousand line shell script where no one can ever find them because everyone who tries goes blind. -- Russ Allbery -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140213211244.ga31...@riva.ucam.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 09:10:15PM +, Dimitri John Ledkov wrote: > On 13 February 2014 16:13, Holger Levsen wrote: > > Hi, > > > > On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: > >> this is just a pledge to you all fellow debian developers to update your > >> build environment before you build a package. > > > > I want all binary packages to be rebuild on *.debian.org hosts. Everything > > else is just an ugly workaround. > > > > All that's needed, I guess, is for someone to write a patch to dak / > wanna-build ... and schedule _all.deb builds on amd64 ? > Or if arch-restricted package, on one of the arches it will build on? The LP folks wanted to sync on this. I also wrote this for Debile, I called it arch affinity (I think LP uses the same name). We should pick a name and use it for all 3. I also think it should be a static arch (rather then any arch that can build it), so that things are a bit easier to duplicate and debug. Might as well. > -- > Regards, > > Dimitri. Cheers, Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Hi, On Donnerstag, 13. Februar 2014, Dimitri John Ledkov wrote: > All that's needed, I guess, is for someone to write a patch to dak / > wanna-build ... and schedule _all.deb builds on amd64 ? > Or if arch-restricted package, on one of the arches it will build on? nope, it's worse than you think: the arch specific package built on the developers machine (in a "random"^wnon predicatable environment) will not be rebuild, there are also no build logs available. See https://buildd.debian.org/status/package.php?p=html2text - you can only hope that I've build it in a clean environment and there aint a logfile for the amd64 build of that arch:any package. cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 4:13 PM, Paul Tagliamonte wrote: > On Thu, Feb 13, 2014 at 09:10:15PM +, Dimitri John Ledkov wrote: > > On 13 February 2014 16:13, Holger Levsen wrote: > > > Hi, > > > > > > On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: > > >> this is just a pledge to you all fellow debian developers to update > your > > >> build environment before you build a package. > > > > > > I want all binary packages to be rebuild on *.debian.org hosts. > Everything > > > else is just an ugly workaround. > > > > > > > All that's needed, I guess, is for someone to write a patch to dak / > > wanna-build ... and schedule _all.deb builds on amd64 ? > > Or if arch-restricted package, on one of the arches it will build on? > > The LP folks wanted to sync on this. I also wrote this for Debile, I > called it arch affinity (I think LP uses the same name). We should pick > a name and use it for all 3. > (to be clear, "a name" in this case is a name for the field in d/control, sorry, damn you anaphora) > > I also think it should be a static arch (rather then any arch that > can build it), so that things are a bit easier to duplicate and debug. > Might as well. > > > -- > > Regards, > > > > Dimitri. > > Cheers, > Paul > > -- > .''`. Paul Tagliamonte | Proud Debian Developer > : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 > `. `'` http://people.debian.org/~paultag > `- http://people.debian.org/~paultag/conduct-statement.txt > -- :wq
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 14/02/14 08:13, Paul Tagliamonte wrote:
> On Thu, Feb 13, 2014 at 09:10:15PM +, Dimitri John Ledkov wrote:
>> On 13 February 2014 16:13, Holger Levsen wrote:
>>> Hi,
>>>
>>> On Donnerstag, 13. Februar 2014, Ondřej Surý wrote:
this is just a pledge to you all fellow debian developers to update your
build environment before you build a package.
>>>
>>> I want all binary packages to be rebuild on *.debian.org hosts. Everything
>>> else is just an ugly workaround.
>>>
>>
>> All that's needed, I guess, is for someone to write a patch to dak /
>> wanna-build ... and schedule _all.deb builds on amd64 ?
>> Or if arch-restricted package, on one of the arches it will build on?
>
> The LP folks wanted to sync on this. I also wrote this for Debile, I
> called it arch affinity (I think LP uses the same name). We should pick
> a name and use it for all 3.
>
> I also think it should be a static arch (rather then any arch that
> can build it), so that things are a bit easier to duplicate and debug.
> Might as well.
Right, "arch affinity" is the term we use for the desirable feature of
being able to override the architecture that by default builds
architecture-independent packages ("nominated arch-indep" in LP
parlance). We have https://bugs.launchpad.net/launchpad/+bug/217427
about supporting that. It's simple to fix, but has mostly been blocked
on needing to decide on a field name with other interested parties. It
sounds like we should really just work something out together -- this
has been going on long enough :)
(There are still some cases for which the semantics of the Architecture
field are unclear, even with arch affinity.
https://bugs.launchpad.net/launchpad/+bug/1063188 is one such example:
hhsuite is "Architecture: amd64 all", our nominated arch-indep is i386,
so the arch-indep bits don't get built. Presumably once we have an arch
affinity field we can assume that its absence means we can just pick one
of the buildable archs and do indep there, but wanna-build, LP and other
implementations should probably decide on the same behaviour here. Other
than that it all seems like mostly a naming thing.)
William.
signature.asc
Description: OpenPGP digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 2/13/14, Jakub Wilk wrote: > * Jacob Appelbaum , 2014-02-13, 18:36: >>How many uploaded binaries might include malware? > > *shrug* It's not like it's difficult to hide malicious code in source > packages. > It is much harder for you to hide source code changes as a third party than binary changes. Many projects have code review processes with people who read each commit and reason about the contents of each code change. Very few public projects have a similar process for reverse engineering binaries or understanding the output of a compiler for releases. Those few projects generally aim for deterministic or reproducible builds - this still has the problem of missing a solid understanding of the output, by the way. > How many configure scripts that we never rebuild from source contains > trojans? There are several serious problems involved in this topic. One expects that we at least have processes for finding malicious configure scripts, as we find buffer overflows in C programs or as we find portability problems in assembler code - no such (Debian) process currently exists for inspecting binaries that are uploaded by developers. Perhaps I'm mistaken? Perhaps there is a project for actively inspecting and reverse engineering binaries that are uploaded by developers? A key problem here is not the badly behaving developer - it is the developer that is compromised and unwittingly becomes party to harming others. Minimizing this exposure is an important goal and worth consideration. All the best, Jacob -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAFggDF1XXuGoK_DR2LpGTDQX9NNPtou8F2JOJY=qj7ob7dk...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
> "Colin" == Colin Watson writes: Colin> On Thu, Feb 13, 2014 at 07:46:53PM +0100, Jakub Wilk wrote: >> *shrug* It's not like it's difficult to hide malicious code in >> source packages. >> >> How many configure scripts that we never rebuild from source >> contains trojans? Colin> Just like my favourite Russ quote: Colin> Basically, people got tired of portability problems in Colin> building shared libraries so they hid them all inside a Colin> multi-thousand line shell script where no one can ever find Colin> them because everyone who tries goes blind. -- Russ Allbery I assure you, that even if you get past the being blind bit, it's still impossible to figure out what's going on. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/01442d2fdd79-9235a03c-59d2-426a-9e7f-767912027c43-000...@email.amazonses.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 10:17:46PM +0100, Holger Levsen wrote: > See https://buildd.debian.org/status/package.php?p=html2text - you can only > hope that I've build it in a clean environment and there aint a logfile for > the amd64 build of that arch:any package. I'm told there's at least some magic address you can mail the logs to, but I never remember what it is. (It's all a workaround anyway.) -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140213235201.ga2...@riva.ucam.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 13 February 2014 21:17, Holger Levsen wrote: > Hi, > > On Donnerstag, 13. Februar 2014, Dimitri John Ledkov wrote: >> All that's needed, I guess, is for someone to write a patch to dak / >> wanna-build ... and schedule _all.deb builds on amd64 ? >> Or if arch-restricted package, on one of the arches it will build on? > > nope, it's worse than you think: the arch specific package built on the > developers machine (in a "random"^wnon predicatable environment) will not be > rebuild, there are also no build logs available. > > See https://buildd.debian.org/status/package.php?p=html2text - you can only > hope that I've build it in a clean environment and there aint a logfile for > the amd64 build of that arch:any package. > My understanding is that arch:any binary debs will be discouraged from uploading, and hence not a problem. (And/or eventually rejected, or binary uploads must go through binary upload queue and/or some such.) This got me thinking, given the problem of "where should arch:all build happen", would it be easier to enable source-only uploads for src+arch:any [!arch:all] packages? (similar how currently mixed src+any+all, can be uploaded as src+all) That would solve most of the uploads, given how boring arch:all binary packages are. -- Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhlujfoewma_pqvh6adzbr8i8e1cf7fkysjueeta8r6a+...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 14 February 2014 05:46, Jakub Wilk wrote: > How many uploaded binaries might include malware? >> > > *shrug* It's not like it's difficult to hide malicious code in source > packages. > After the damage is done, probably easier to find the malware that did it if you can rely on the source code being an accurate representation of what was running (not that this would be any easy task). -- Brian May
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
All rants aside, I believe there's a fairly wide agreement that we should throw away binaries from builds. I seem to recall ftp-master sending out mail to debian-devel-announce describing the steps along that process a while ago. I think it's fine to ask where that project is, and to volunteer resources to help. However, I'd hope we're all agreed that we need to get there. --Sam -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/01442dcab45a-f088c559-58fa-4337-adef-4a28964e06c5-000...@email.amazonses.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Heya Sam, On 2/14/14, Sam Hartman wrote: > All rants aside, I believe there's a fairly wide agreement that we > should throw away binaries from builds. I'd encourage something slightly different and then I'd expand on it a bit. I think it would be useful to have an historical archive of each binary, as uploaded by a developer or produced by a build server. In the event that a binary is imperfect, storing a copy of each binary may be useful for later analysis. This could help us spot bugs for a compiler specific problem (such as optimizing something out of the binary that was otherwise in the source) or when the binary is simply malicious (such as a backdoor inserted by a compiler or build process). I'd say that we might consider *using* that uploaded binary as an assertion by a given developer. In general it is the expected correct output binary and it is signed by that developer. As we move into the deterministic/reproducible world, the archive will serve an interesting purpose. Each developer would upload their respective binary and signature per architecture for a given source package. Their output may be represented as GnuPG signatures over hashes of source code packages and binary packages. In theory, we could have an assertion from another developer with another upload and so on. Even though we should have matching hashes, in some cases, we may not learn that we have mismatching hashes for hours or days. Thus keeping a copy of all the built binary parts is useful for later analysis. This will allow us to verify that the sources correspond to a specific binary output, we could also ensure that very important packages require a threshold of signatures before the binaries are consider properly built. Those that upload a binary should not be able to delete the uploaded binary - thus when a build server or developer builds a *different* version, we'll be able to inspect *all files* for differences. So for each developer uploading, we'd keep a copy. We could discard identical binaries and ensure that a copy is only kept in the long run when there are no strange events. If a developer is compromised, we're able to detect it and ensure that the developer's system cannot be used to erase evidence of such a compromise. If the build system is compromised, it will be useful to download the binary from the archive and to inspect it. Similarly the build system may not erase items from the archive. This could add to the overall security of the process and to begin to give us some semblance of a review process for binaries served to users. Until something like that happens, I think we should probably not *ship* the developer built binaries to users. We will however ship some binary parts to the user and hopefully those will be built by the build servers, as well as archived somewhere for all time, in case we're already compromised. ( Append only data store ideas like Chisel may have a non-evil use, hooray! ) > > I seem to recall ftp-master sending out mail to debian-devel-announce > describing the steps along that process a while ago. > > I think it's fine to ask where that project is, and to volunteer > resources to help. > However, I'd hope we're all agreed that we need to get there. > I've been reading the deterministic build page and I think it could use some further discussions. All the best, Jacob -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAFggDF0kOQsNbdQ-5-LSuHr0z0zTnOWDnSXV+H5qA1khMKF=w...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Fri, Feb 14, 2014 at 04:44:21AM +, Jacob Appelbaum wrote: > Heya Sam, > > On 2/14/14, Sam Hartman wrote: > > All rants aside, I believe there's a fairly wide agreement that we > > should throw away binaries from builds. > > I'd encourage something slightly different and then I'd expand on it a bit. > > I think it would be useful to have an historical archive of each > binary, as uploaded by a developer or produced by a build server. Perhaps you're interested in helping with http://snapshot.debian.org/ then? :) (I've got tons of these links!) Cheers, Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 2/14/14, Paul Tagliamonte wrote: > On Fri, Feb 14, 2014 at 04:44:21AM +, Jacob Appelbaum wrote: >> Heya Sam, >> >> On 2/14/14, Sam Hartman wrote: >> > All rants aside, I believe there's a fairly wide agreement that we >> > should throw away binaries from builds. >> >> I'd encourage something slightly different and then I'd expand on it a >> bit. >> >> I think it would be useful to have an historical archive of each >> binary, as uploaded by a developer or produced by a build server. > > Perhaps you're interested in helping with http://snapshot.debian.org/ > then? :) > I'm glad to see that the really hard part of the process is already done! Amazing! > (I've got tons of these links!) > I'm enjoying reading them all. Thank you! :) All the best, Jacob -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAFggDF18ETcHJh4HJBw5=bjiqzvm0woy4pm4y6lpxcp21qk...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 11:52:01PM +, Colin Watson wrote: > On Thu, Feb 13, 2014 at 10:17:46PM +0100, Holger Levsen wrote: > > See https://buildd.debian.org/status/package.php?p=html2text - you can only > > hope that I've build it in a clean environment and there aint a logfile for > > the amd64 build of that arch:any package. > > I'm told there's at least some magic address you can mail the logs to, > but I never remember what it is. (It's all a workaround anyway.) l...@buildd.debian.org The mail's subject has to be in the format that buildd uses, though: Subject: Log for successful build of package_version on amd64 (dist=sid) e.g., Log for successful build of nbd_1:3.7-1_amd64 (dist=sid) would work. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140214130140.gc7...@grep.be
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
* Wouter Verhelst , 2014-02-14, 14:01: I'm told there's at least some magic address you can mail the logs to, but I never remember what it is. (It's all a workaround anyway.) l...@buildd.debian.org The mail's subject has to be in the format that buildd uses, though: Subject: Log for successful build of package_version on amd64 (dist=sid) e.g., Log for successful build of nbd_1:3.7-1_amd64 (dist=sid) would work. Are buildd people happy with humans sending their logs this way? If yes, it would be nice if somebody wrote a log-submitting script to be included in devscripts. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140214131038.ga20...@jwilk.net
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Fri, Feb 14, 2014 at 02:10:38PM +0100, Jakub Wilk wrote: > * Wouter Verhelst , 2014-02-14, 14:01: > >>I'm told there's at least some magic address you can mail the > >>logs to, but I never remember what it is. (It's all a > >>workaround anyway.) > > > >l...@buildd.debian.org > > > >The mail's subject has to be in the format that buildd uses, though: > > > >Subject: Log for successful build of package_version on amd64 (dist=sid) > > > >e.g., > > > >Log for successful build of nbd_1:3.7-1_amd64 (dist=sid) > > > >would work. > > Are buildd people happy with humans sending their logs this way? Well, I am, but it's probably not my call. > If yes, it would be nice if somebody wrote a log-submitting script > to be included in devscripts. Good plan. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140214153925.gf16...@grep.be
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Fri, Feb 14, 2014 at 04:39:25PM +0100, Wouter Verhelst wrote: > > Are buildd people happy with humans sending their logs this way? > > Well, I am, but it's probably not my call. Which keyring does it use to validate? Can DMs send logs? Does it validate at all, or can some script kiddies use it as a pastebin service? :) Cheers, Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Fri, Feb 14, 2014 at 10:42:16AM -0500, Paul Tagliamonte wrote: > Which keyring does it use to validate? Can DMs send logs? Does it > validate at all, or can some script kiddies use it as a pastebin > service? :) The logs aren't signed, so it only validates the Subject line. This has been un-abused for many years, though perhaps that will change shortly. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140214160421.ga16...@scru.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
previously on this list Brian May contributed: > After the damage is done, probably easier to find the malware that did it Assuming the damage is visible? > All rants aside, I believe there's a fairly wide agreement that we > should throw away binaries from builds. >> I'd encourage something slightly different and then I'd expand on it a >> bit. Sounds like a plan but perhaps if they are not already? these uploads should be enabled within their own apt sources line. Whilst malicious code can be hidden in source and accompanying packages such as via sidechannel attacks, any additional threats should be avoided or users enabled to avoid them where possible. -- ___ 'Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface' (Doug McIlroy) In Other Words - Don't design like polkit or systemd ___ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/783597.44818...@smtp131.mail.ir2.yahoo.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 2014-02-14 16:42, Paul Tagliamonte wrote: On Fri, Feb 14, 2014 at 04:39:25PM +0100, Wouter Verhelst wrote: > Are buildd people happy with humans sending their logs this way? Well, I am, but it's probably not my call. Which keyring does it use to validate? Can DMs send logs? Does it validate at all, or can some script kiddies use it as a pastebin service? :) That's why I was careful to publish the address nowhere. We do some checks, but we still need to be able to receive mail from non-DSA'ed buildds. Otherwise I would have blocked mail reception from the internets. (We do receive some few spam mails. Now there will be plenty. Thanks.) Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/38ac36ebdcf91d9faeed2039ab369...@hub.kern.lc
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Sat, 15 Feb 2014, Philipp Kern wrote: > That's why I was careful to publish the address nowhere. We do some Unfortunately, that cat is out of the bag, now. Whether it will get spammed or attacked, I don't know. However, it is not like we ever could trust the logs anyway for any security purposes, as they are not signed by the same key that signed the respective changes file for the upload. Currently, they're useful for debug purposes, and that's it (which is fine). -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140215143421.gb5...@khazad-dum.debian.net
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
* Philipp Kern , 2014-02-15, 15:19: Are buildd people happy with humans sending their logs this way? Well, I am, but it's probably not my call. Which keyring does it use to validate? Can DMs send logs? Does it validate at all, or can some script kiddies use it as a pastebin service? :) That's why I was careful to publish the address nowhere. It's published here: http://www.debian.org/devel/buildd/wanna-build-states archive.org tells me it's been there since at least 2004. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140215161151.ga3...@jwilk.net
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 2014-02-15 15:34, Henrique de Moraes Holschuh wrote: On Sat, 15 Feb 2014, Philipp Kern wrote: That's why I was careful to publish the address nowhere. We do some Unfortunately, that cat is out of the bag, now. Whether it will get spammed or attacked, I don't know. However, it is not like we ever could trust the logs anyway for any security purposes, as they are not signed by the same key that signed the respective changes file for the upload. Currently, they're useful for debug purposes, and that's it (which is fine). Yeah, they could be, if sbuild is extended to sign the logs. That said, we do trust our mail-in. We can trust the received headers to some degree (for timestamps and where the mail came from), but we cannot rule out on-disk tampering. Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/43b8290b3d8f1f2f37b01636e9337...@hub.kern.lc
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
I'd like to chime in on this whole build thing. I've been trying to get pbuilder working for a few days now, on a package from backports. It should be a simple task, but I need a minor modification in the form of an extra repository for dependencies. It's been incredibly difficult to get the package built. The process needs some refinement and some minor enhancements at least. I'll be documenting my process soon enough for others to be able to have a reasonable account of the process as it stands now. On Feb 15, 2014 8:15 AM, "Philipp Kern" wrote: > On 2014-02-15 15:34, Henrique de Moraes Holschuh wrote: > >> On Sat, 15 Feb 2014, Philipp Kern wrote: >> >>> That's why I was careful to publish the address nowhere. We do some >>> >> >> Unfortunately, that cat is out of the bag, now. Whether it will get >> spammed >> or attacked, I don't know. >> >> However, it is not like we ever could trust the logs anyway for any >> security >> purposes, as they are not signed by the same key that signed the >> respective >> changes file for the upload. Currently, they're useful for debug >> purposes, >> and that's it (which is fine). >> > > Yeah, they could be, if sbuild is extended to sign the logs. That said, we > do trust our mail-in. We can trust the received headers to some degree (for > timestamps and where the mail came from), but we cannot rule out on-disk > tampering. > > Kind regards > Philipp Kern > > > -- > To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact > listmas...@lists.debian.org > Archive: http://lists.debian.org/43b8290b3d8f1f2f37b01636e93370 > a...@hub.kern.lc > >
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 13/02/14 22:10, Dimitri John Ledkov wrote: > On 13 February 2014 16:13, Holger Levsen wrote: >> > Hi, >> > >> > On Donnerstag, 13. Februar 2014, Ondřej Surý wrote: >>> >> this is just a pledge to you all fellow debian developers to update your >>> >> build environment before you build a package. >> > >> > I want all binary packages to be rebuild on *.debian.org hosts. Everything >> > else is just an ugly workaround. >> > > All that's needed, I guess, is for someone to write a patch to dak / > wanna-build ... and schedule _all.deb builds on amd64 ? > Or if arch-restricted package, on one of the arches it will build on? I guess this could be an interesting project to do as part of the GSoC, if someone is willing to mentor it. https://lists.debian.org/debian-devel-announce/2014/02/msg1.html signature.asc Description: OpenPGP digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 13, 2014 at 10:17:46PM +0100, Holger Levsen wrote: > nope, it's worse than you think: the arch specific package built on the > developers machine (in a "random"^wnon predicatable environment) will not be > rebuild, there are also no build logs available. > > See https://buildd.debian.org/status/package.php?p=html2text - you can only > hope that I've build it in a clean environment and there aint a logfile for > the amd64 build of that arch:any package. If your source package also builds an architecture independent package (not the case for html2text), you can do an architecture independent build locally. ftp happily takes your upload without architecture specific packages and builds the missing binaries including all logs (except for the architecture independent ones). Hope this helps Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140216090634.ga17...@alf.mars
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Sam Hartman debian.org> writes: [ autotools ] > I assure you, that even if you get past the being blind bit, it's still > impossible to figure out what's going on. And even then, even when you did the unbelievable and, say, ported libtool to MirBSD and Interix (consuming a whole bottle of wine in the process), you *still* can’t tell tails from heads afterwards, often wonder why t f you did something, and have a blackout the morning after it… bye, //mira“BTDT”bilos -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140226t140012...@post.gmane.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Paul Tagliamonte debian.org> writes: > > On Fri, Feb 14, 2014 at 04:39:25PM +0100, Wouter Verhelst wrote: > > > Are buildd people happy with humans sending their logs this way? > > > > Well, I am, but it's probably not my call. > > Which keyring does it use to validate? Can DMs send logs? Does it > validate at all, or can some script kiddies use it as a pastebin > service? :) The latter… I had asked for being allowed to upload my cowbuilder logs to the debian-ports equivalent. For years, repeatedly. I never got a positive answer on it, even after eventually having figured out the address and the format to use from reading source code. I also assumed all of those details were deliberately not published. I assume that the main archive would be at least as strict, if not stricter, than debian-ports. bye, //mirabilos ObCaptcha: “null” (looks like GMane only has a dozen or so words) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140226t140710-...@post.gmane.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Carlos Alberto Lopez Perez igalia.com> writes: > On 13/02/14 22:10, Dimitri John Ledkov wrote: > > All that's needed, I guess, is for someone to write a patch to dak / > > wanna-build ... and schedule _all.deb builds on amd64 ? > > Or if arch-restricted package, on one of the arches it will build on? > > I guess this could be an interesting project to do as part of the GSoC, > if someone is willing to mentor it. First, we need new syntax to specify the architectures an arch:all package may be built on. (There may be cases where this cannot be deducted from the other binary packages it builds – if any. Heck, there may even be cases where a source package has multiple arch:all packages that need to be built on *different* architectures.) Then we need to have this syntax supported by dpkg in stable, AIUI… bye, //mirabilos ObCaptcha: nature -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140226t141033-...@post.gmane.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Wed, Feb 26, 2014 at 01:11:55PM +, Thorsten Glaser wrote: > First, we need new syntax to specify the architectures an arch:all > package may be built on. (There may be cases where this cannot be > deducted from the other binary packages it builds – if any. Heck, > there may even be cases where a source package has multiple arch:all > packages that need to be built on *different* architectures.) > > Then we need to have this syntax supported by dpkg in stable, AIUI… Yo, tg, I was going to send a mail about this yesterday. I've decided I'm going to start a quest to support this. I settled on Build-Indep-Architecture myself. Anyway, dpkg doesn't need to be involved besides not choking on the d/control line. We can just change wanna-build, debile and launchpad to pass --arch-all to sbuild[1], which sbuild totes already supports. The launchpad team seems willing, and i've not talked with wanna-build, but I'm sure they're up for it. BTW; the syntax would define a single arch; you know, in the spirit of reproducability. Someone willing to take this work up from me would clear my plate up to do other thing, but it's something I want to see :) Cheers, Paul -- .''`. Paul Tagliamonte | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
* Paul Tagliamonte , 2014-02-26, 08:39: First, we need new syntax to specify the architectures an arch:all package may be built on. (There may be cases where this cannot be deducted from the other binary packages it builds – if any. Heck, there may even be cases where a source package has multiple arch:all packages that need to be built on *different* architectures.) “multiple arch:all packages that need to be built on different architectures” is not something that dpkg-buildpackage supports currently anyway. Let's not over-engineer… Then we need to have this syntax supported by dpkg in stable, AIUI… Yo, tg, I was going to send a mail about this yesterday. I've decided I'm going to start a quest to support this. I settled on Build-Indep-Architecture myself. FWIW, as an alternative for the new field, we could use Packages-arch-specific. BTW; the syntax would define a single arch; you know, in the spirit of reproducability. I have mixed feeling about this. On one hand, most[0] of arch:all packages can be built on more than one architecture, so “single arch” sounds like an artificial limitation. On the other hand, we very much don't want the same arch:all package to be built by multiple buildds… [0] Likely s/Most/All/ if you take into account hypothetical architectures that nobody has boostrapped (yet?!), e.g. musl-linux-m68k. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140226145537.ga7...@jwilk.net
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Wed, Feb 26, 2014 at 03:55:37PM +0100, Jakub Wilk wrote:
> >BTW; the syntax would define a single arch; you know, in the
> >spirit of reproducability.
>
> I have mixed feeling about this. On one hand, most[0] of arch:all
> packages can be built on more than one architecture, so “single
> arch” sounds like an artificial limitation.
You're not wrong; I'd just really hate for this to lead to a situation
where the arch:all build causes a FTBFS on ${EXOTIC_ARCH}
(HURD/kFreeBSD/MIPS) because it's assumed it works fine (I could see
something like endianess breaking things) and have it transitively break
the package (or worse yet; render data that isn't good to run)
I'd tend to lean to being explicit about where it should build (rather
then say "any" and call it a day), and have it explicitly reproducable
rather than some toss-up.
If dpkg-buildpckage doesn't care (and I don't see any reason why it
would), I'm sure it can just ignore this field and leave it as advice to
the build software.
Yeah, not great, I know :\
> On the other hand, we
> very much don't want the same arch:all package to be built by
> multiple buildds…
>
>
> [0] Likely s/Most/All/ if you take into account hypothetical
> architectures that nobody has boostrapped (yet?!), e.g.
> musl-linux-m68k.
Point taken.
Cheers,
Paul
--
.''`. Paul Tagliamonte | Proud Debian Developer
: :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87
`. `'` http://people.debian.org/~paultag
`- http://people.debian.org/~paultag/conduct-statement.txt
signature.asc
Description: Digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
Hi, Le 26/02/2014 15:55, Jakub Wilk a écrit : > * Paul Tagliamonte , 2014-02-26, 08:39: >>> First, we need new syntax to specify the architectures an arch:all >>> package may be built on. (There may be cases where this cannot be >>> deducted from the other binary packages it builds – if any. Heck, >>> there may even be cases where a source package has multiple arch:all >>> packages that need to be built on *different* architectures.) > > “multiple arch:all packages that need to be built on different > architectures” is not something that dpkg-buildpackage supports > currently anyway. Let's not over-engineer… I assume there will be full-archive rebuilds attempted before this is deployed in production anyway, I guess this should catch the few weird cases, if any... The only semi-legit case I can think of (for several arch-indep packages which need to be built on distinct arches from the same source) is for a source package which is intended to retrieve information on a each arch at build time and distribute it to all architectures in binary packages. Why exactly you would want to do that is not clear to me, perhaps to retrieve and distribute this sort of information: https://wiki.debian.org/ArchitectureSpecificsMemo In all other cases (I can think of), that means that the package is somehow buggy on all architectures. Kind regards, Thibaut. signature.asc Description: OpenPGP digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
[Paul Tagliamonte] > I was going to send a mail about this yesterday. I've decided > I'm going to start a quest to support this. I settled on > Build-Indep-Architecture myself. Sorry for the bikeshedding, but don't we already have ways to express exactly what we mean? Build-Depends-Indep: some-pkg-only-avail-on-i386-and-amd64 ...that being I guess the common case where you need to build arch:all on a specific architecture: because of availability of build tools. And if there are any cases even more exotic (you need to restrict the arch but _not_ because of build-dep availability): Build-Conflicts-Indep: build-essential [!i386] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140226151520.gb4...@p12n.org
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On 27/02/14 00:39, Paul Tagliamonte wrote: > On Wed, Feb 26, 2014 at 01:11:55PM +, Thorsten Glaser wrote: >> First, we need new syntax to specify the architectures an arch:all >> package may be built on. (There may be cases where this cannot be >> deducted from the other binary packages it builds – if any. Heck, >> there may even be cases where a source package has multiple arch:all >> packages that need to be built on *different* architectures.) >> >> Then we need to have this syntax supported by dpkg in stable, AIUI… > > Yo, tg, > > I was going to send a mail about this yesterday. I've decided I'm going > to start a quest to support this. I settled on Build-Indep-Architecture > myself. > > Anyway, dpkg doesn't need to be involved besides not choking on the > d/control line. We can just change wanna-build, debile and launchpad to > pass --arch-all to sbuild[1], which sbuild totes already supports. > > The launchpad team seems willing, and i've not talked with wanna-build, > but I'm sure they're up for it. Launchpad already passes -A to sbuild on i386, and that's all worked fine for the last 8 years. We just need to override the i386 default when this field is present. > BTW; the syntax would define a single arch; you know, in the spirit of > reproducability. I'm not sure this going to be sufficient long-term. A prioritised list sounds better to me; consider the proliferation of x86, ARM, MIPS and PowerPC architectures, where it's likely that any member of the family can build the platform firmware, and rebuilds or derivatives may support just a subset of the processor's ABIs. I'd probably define "Build-Indep-Architecture: armhf armel" to mean "build with -A on armhf if you have it, otherwise armel, otherwise nowhere". But maybe it would be better for "otherwise nowhere" to be "otherwise anywhere"? On the other hand, it's relatively easy to expand from a single arch to an ordered list later without breaking compatibility. And given the number of packages involved it's probably not completely terrible to change entirely if it doesn't work out. > Someone willing to take this work up from me would clear my plate up to > do other thing, but it's something I want to see :) signature.asc Description: OpenPGP digital signature
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
On Thu, Feb 27, 2014 at 7:51 AM, William Grant wrote: > I'd probably define "Build-Indep-Architecture: armhf armel" to mean > "build with -A on armhf if you have it, otherwise armel, otherwise > nowhere". But maybe it would be better for "otherwise nowhere" to be > "otherwise anywhere"? You could use this for "otherwise anywhere": Build-Architecture: armhf armel any Or this for "otherwise any Linux arch": Build-Architecture: armhf armel linux-any -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caktje6hi+fux30vgnxzporqki78am3jsoedgxgwu9jbap54...@mail.gmail.com
Re: when will we finally throw away binary uploads (Re: Please upgrade your build environment when you are affected by transition
* Peter Samuelson , 2014-02-26, 09:15: And if there are any cases even more exotic (you need to restrict the arch but _not_ because of build-dep availability): Build-Conflicts-Indep: build-essential [!i386] To be pedantically correct, one should conflict with a build-essential package (such as "dpkg-dev"), not with THE "build-essential" package. dpkg-buildpackage currently requires the "build-essential" package to be installed, but that's an implementation detail that shouldn't be relied upon. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140227122342.ga1...@jwilk.net
