Re: On doing 3000 no-source-change source-only uploads in January 2021
On 2021-01-04 at 04:27, Adrian Bunk wrote: > On Thu, Dec 31, 2020 at 02:16:23PM +0100, Bernd Zeimetz wrote: > >> Although the high number of packages makes me wonder, if at least a >> quick MIA check of the maintainers is warranted, or - if those packages >> are needed in bullseye at all. > > Maintainership status is a very poor indicator whether users might need > a package. > > Some obscure stuff is well maintained, like m68k and Hurd being among > our architectures with the most active maintainers. > > It is very hard and high effort for people who are not already active in > Debian development to get a change into Debian or take responsibility > for a package. Debian is not a welcoming place for new contributors. > > A normal user won't even notice that an important package is missing > before bullseye is stable. As a demonstrating example of this last point: I track testing, rather than stable, and subscribe to debian-devel, and therefore am not even a "normal" user in this sense. A package I use regularly (moosic) was removed from testing back in November of 2019, and from unstable back in April of 2020. I didn't notice anything until October or November of 2020 (when the Python 3 transition tried to remove the package from my computer), and didn't realize that what I had noticed meant the package had been removed from the archive - even testing, much less unstable - until December of 2020. I've adopted it upstream (it had been abandoned by its original author back in 2011) and fixed the issues which had led to its removal, and its Debian maintainer - who hadn't noticed the removal from unstable either, until looking at it again after I provided a new upstream version - has agreed to handle the packaging again, but it's not at all clear whether it'll be ready and make it through NEW again ahead of the release freeze. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: On doing 3000 no-source-change source-only uploads in January 2021
On Thu, Dec 31, 2020 at 02:16:23PM +0100, Bernd Zeimetz wrote: >... > Although the high number of packages makes me wonder, if at least a > quick MIA check of the maintainers is warranted, or - if those packages > are needed in bullseye at all. >... Maintainership status is a very poor indicator whether users might need a package. Some obscure stuff is well maintained, like m68k and Hurd being among our architectures with the most active maintainers. It is very hard and high effort for people who are not already active in Debian development to get a change into Debian or take responsibility for a package. Debian is not a welcoming place for new contributors. A normal user won't even notice that an important package is missing before bullseye is stable. > Bernd cu Adrian
Re: On doing 3000 no-source-change source-only uploads in January 2021
On Thu, Dec 31, 2020 at 08:14:20PM +0100, Eduard Bloch wrote: > You might want to update your database and rerun that analysis. At least > one of the packages was updated following the current rules a few days > ago. > https://tracker.debian.org/pkg/mail-expire well, the list is correct from the POV I care about: packages in *testing* without .buildinfo files - you've thankfully just uploaded mail-expire to unstable, so soon the fix will land in testing :) that said, Ivo just gave me an updated filter ignoring packages which have a different version in unstable, so from now on I will use that and thus I'll be happily ignoring packages which have been fixed in unstable recently. Today I've went through the first 114 packages from the list of 3051 packages posted here yesterday and I only had to upload 107 packages, because 7 packages (of those 114) have been uploaded very recently. Yay & thank you for that! -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C ⠈⠳⣄ signature.asc Description: PGP signature
Re: On doing 3000 no-source-change source-only uploads in January 2021
Hi Holger, On 31/12/2020 13:50, Holger Levsen wrote: > hi, > > as described in Message-ID: <20201231124509.gb3...@layer-acht.org> > or > http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ > I plan to do 3000 NMUs soon. > > Attached is the list of affected packages, maintainers and uploaders. Please > check > if you are on it and if so, please help me by uploading these packages before > me. Thanks for working on this! I added an item to pkg-perl Team "Low Hanging Fruit" monthly meeting [1] agenda [2], which will happen on Jan 21th. I don't think I can commit to do some of them before that date, or guarantee all will be done after the meeting, but I'll try my best to participate in this effort. Cheers, -- nodens [1] https://wiki.debian.org/Teams/DebianPerlGroup/LHF [2] https://wiki.debian.org/Teams/DebianPerlGroup/LHF/Agenda
Re: On doing 3000 no-source-change source-only uploads in January 2021
Hello Holger, I will do my part myself ASAP Kind regards Mechtilde Am 31.12.20 um 14:21 schrieb Holger Levsen: > Hi Bernd, > > On Thu, Dec 31, 2020 at 02:16:23PM +0100, Bernd Zeimetz wrote: >>> as described in Message-ID: <20201231124509.gb3...@layer-acht.org> >>> or >>> http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ >>> I plan to do 3000 NMUs soon. >> Thanks for your work! > > thanks! > >> Although the high number of packages makes me wonder, if at least a >> quick MIA check of the maintainers is warranted, or - if those packages >> are needed in bullseye at all. >> Of course, there are some packages which are working just fine with the >> same source for years, but at least some debhelper/compat or policy >> check and upload at some point makes sense. >> >> I think Debian should sometimes be better and faster in removing >> unmaintained stuff. > > while I agree in principle I have to say that in practice it's not doable > for me, uploading 3000 packages doing just the basic checks that I do is a > major task already, if I add any other QA measure which requieres human > consideration (like removal) it becomes a much much bigger task, deluting > my original goal of fixing the issue of missing .buildinfo files. Even > just running lintian-brush would at least double the amount, though I believe > the impact would be even higher. > > that said, I filed a bug to get src:embedian-archive-keyring removed from > sid and then bullseye. That was one striking example I found going through > those first 570 packages... > > -- Mechtilde Stehmann ## Apache OpenOffice ## Freie Office Suite für Linux, MacOSX, Windows und OS/2 ## Debian Developer ## PGP encryption welcome ## F0E3 7F3D C87A 4998 2899 39E7 F287 7BBA 141A AD7F OpenPGP_signature Description: OpenPGP digital signature
Re: On doing 3000 no-source-change source-only uploads in January 2021
Hallo, * Holger Levsen [Thu, Dec 31 2020, 12:50:30PM]: > as described in Message-ID: <20201231124509.gb3...@layer-acht.org> > or > http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ > I plan to do 3000 NMUs soon. > > Attached is the list of affected packages, maintainers and uploaders. Please > check > if you are on it and if so, please help me by uploading these packages before > me. You might want to update your database and rerun that analysis. At least one of the packages was updated following the current rules a few days ago. https://tracker.debian.org/pkg/mail-expire Apart from that, thanks for tackling this. Best regards, enjoy NYE and have a good start into 2021! Eduard.
Re: On doing 3000 no-source-change source-only uploads in January 2021
Hi Bernd, On Thu, Dec 31, 2020 at 02:16:23PM +0100, Bernd Zeimetz wrote: > > as described in Message-ID: <20201231124509.gb3...@layer-acht.org> > > or > > http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ > > I plan to do 3000 NMUs soon. > Thanks for your work! thanks! > Although the high number of packages makes me wonder, if at least a > quick MIA check of the maintainers is warranted, or - if those packages > are needed in bullseye at all. > Of course, there are some packages which are working just fine with the > same source for years, but at least some debhelper/compat or policy > check and upload at some point makes sense. > > I think Debian should sometimes be better and faster in removing > unmaintained stuff. while I agree in principle I have to say that in practice it's not doable for me, uploading 3000 packages doing just the basic checks that I do is a major task already, if I add any other QA measure which requieres human consideration (like removal) it becomes a much much bigger task, deluting my original goal of fixing the issue of missing .buildinfo files. Even just running lintian-brush would at least double the amount, though I believe the impact would be even higher. that said, I filed a bug to get src:embedian-archive-keyring removed from sid and then bullseye. That was one striking example I found going through those first 570 packages... -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C ⠈⠳⣄ There are no jobs on a dead planet. (Also many other things but people mostly seem to care about jobs.) signature.asc Description: PGP signature
Re: On doing 3000 no-source-change source-only uploads in January 2021
Hi Holger, > > as described in Message-ID: <20201231124509.gb3...@layer-acht.org> > or > http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ > I plan to do 3000 NMUs soon. Thanks for your work! Although the high number of packages makes me wonder, if at least a quick MIA check of the maintainers is warranted, or - if those packages are needed in bullseye at all. Of course, there are some packages which are working just fine with the same source for years, but at least some debhelper/compat or policy check and upload at some point makes sense. I think Debian should sometimes be better and faster in removing unmaintained stuff. Bernd -- Bernd ZeimetzDebian GNU/Linux Developer http://bzed.dehttp://www.debian.org GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
Re: On doing 3000 no-source-change source-only uploads in January 2021
Le jeudi, 31 décembre 2020, 13.50:30 h CET Holger Levsen a écrit : > hi, > > as described in Message-ID: <20201231124509.gb3...@layer-acht.org> > or > http://layer-acht.org/thinking/blog/20201231-no-source-change-source-upload > s/ I plan to do 3000 NMUs soon. Fantastic job, thanks a lot for doing it! -- OdyX signature.asc Description: This is a digitally signed message part.
Re: On doing 540 no-source-change source-only uploads in two weeks
Holger Levsen, le jeu. 31 déc. 2020 12:45:09 +, a ecrit: > I'll post the list of packages (sorted by ddlist) to debian-devel@lists.d.o > shortly and will then amend this blog post to link to that mail. I was about to ask for such a list :D I'll gladly upload my long-no-upload packages, probably they actually have changes in the git tree that deserve uploading anyway. Thanks for your care! Samuel
On doing 3000 no-source-change source-only uploads in January 2021
hi, as described in Message-ID: <20201231124509.gb3...@layer-acht.org> or http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ I plan to do 3000 NMUs soon. Attached is the list of affected packages, maintainers and uploaders. Please check if you are on it and if so, please help me by uploading these packages before me. -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C ⠈⠳⣄ That morning, the young barista woman told me that a customer came in with a mask, but not wearing it. When she asked the customer to put on her mask please, the woman said: "Why? There's no-one in here." ddlist.gz Description: application/gzip signature.asc Description: PGP signature
On doing 540 no-source-change source-only uploads in two weeks
hi, the following was first published on http://layer-acht.org/thinking/blog/20201231-no-source-change-source-uploads/ and is repeated here for the benefit of the audience not reading Planet Debian, who else might not understand the referred (and soon to be posted) follow-up mail. # On doing 540 no-source-change source-only uploads in two weeks So I've been doing 540 no-source-change source-only uploads in the last two weeks and am planning to do 3000 more in January 2021. We'll see how that goes ;) Let me explain what I have been doing and why. So, https://lists.debian.org/debian-devel-announce/2019/07/msg2.html;>starting with the Bullseye release cycle the Release Team changed policy: only packages which were build on buildds are allowed to migrate to testing. Which is pretty nice for https://reproducible-builds.org/;>reproducible builds as this also ensures that a https://manpages.debian.org/testing/dpkg-dev/deb-buildinfo.5.en.html;>.buildinfo file is available for anyone wanting to reproduce the binaries of that package. However, there are many binary (and source) packages in Debian which were uploaded before 2016 (which is when .buildinfo files were introduced) or were uploaded with binaries until that change in release policy July 2019. Then Ivo De Decker scheduled binNMUs for all the affected packages but due to the way binNMUs work, he couldn't do anything about arch:all packages as they currently cannot be rebuilt with binNMUs. Ivo and myself discussed what could be done about the remaining packages and (besides long complicated changes to Debian's workflows) the only thing deemed possible was doing many many source uploads with just a new changelog entry: * Non maintainer upload by the Reproducible Builds team. * No source change upload to rebuild on buildd with .buildinfo files. These packages are all inherently buggy, because Debian policy mandates that packages should be reproducible and without .buildinfo files one cannot reproducibly rebuild packages. So instead of filing many many bugs we've decided to just fix these bugs by doing a no-source-change source uploads. One nice aspect of these uploads is that there's no follow-up work imposed on the maintainer: whether they keep that changelog entry or whether they discard it, it does not matter. So Ivo had developed an SQL query which showed 570 packages needing an update roughly two weeks ago, on December 18 and so I started slowly. This is the amount of NMUs I did in the last days: for i in $(seq 18 30) ; do echo -n "Dec $i: " ; ls -lart1 done/*upload|grep -c "Dec $i" ; done Dec 18: 12 Dec 19: 0 Dec 20: 3 Dec 21: 13 Dec 22: 13 Dec 23: 16 Dec 24: 4 Dec 25: 28 Dec 26: 0 Dec 27: 38 Dec 28: 198 Dec 29: 206 Dec 30: 9 About ten packages had FTBFS bugs preventing an upload and seven packages were uploaded by the maintainer before me. I've seen two cases of sudden maintainer uploads after 8 and 10 years of no activity! So what did I do for each upload? * pre upload work: * test build with pbuilder in sid * check PTS for last upload date (and having arch:all binaries) and open RC bugs * modify d/changelog * check debdiff between two sources (just the changelog entry...!) * upload * (some times filing bugs or modifying bug meta data etc) * post upload: * check PTS for testing migration, so for this I've still got >500 browser tabs open and will keep them open until the packages migrates Much to my surprise I didn't get much feedback, there were like 6 people on the #debian-reproducible channel cheering and one on #debian-qa, though that person is a Release Team member so that was kind of important cheering. And I've seen some maintainer uploads to packages which haven't seen uploads since some years. And really nice: no-one complained so far. I hope this will stay this way with the plan to do 3000 more uploads of this kind: Those 570 packages were only https://udd.debian.org/cgi-bin/key_packages.yaml.cgi;>key packages but there are 3000 more source packages which have a binary in bullseye for which no .buildinfo file exists. So I plan to upload them all in January 2021 and you can help me doing so, by uploading your packages before me - and maybe fixing some other bugs in the process! I'll post the list of packages (sorted by ddlist) to debian-devel@lists.d.o shortly and will then amend this blog post to link to that mail. Many thanks to Ivo and the whole Release Team for their support of Reproducible Builds and generally speaking for the many many enhancements to the release process we've seen over the years. Debian in 2021 will rock'n'roll more than ever! So thank you all, once again, for making Debian what it is and what it will be! -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C ⠈⠳⣄ Dance like no on
Re: Re: source-only uploads
Andrey Rahmatullin writes ("Re: source-only uploads"):
> On Fri, Sep 01, 2017 at 12:47:41PM +0200, Emmanuel Bourg wrote:
> > Just yesterday I completely broke a key package used to build
> > many Java packages, and I couldn't even rebuild it to fix the issue.
>
> Why? Does it B-D on itself?
And, if it does, can it not be built using stretch ?
Ian.
I don't know about Java but I had an issue with freepascal not so long ago
(back when Jessie was stable and stretch was testing).
A change in glibc broke freepascal on powerpc stretch/sid to the point it
wouldn't install. Freepascal needs itself to build. Sids freepascal would not
build in jessie due to using newer debhelper features.
To fix this I had to take sid's freepascal, apply the upstream patch for the
glibc issue, hack it up so it would build in a jessie environment, build it in
a jessie environment on the porterbox, install the binaries from that build
into a sid environmentin qemu (because self-built packages can't be installed
on porterboxes).
This kind of stuff does happen and we need to be able to deal with it.
Having said that I believe the default should be to throw away maintainer-built
binaries, they should only be accepted if the developer explicitly asks for it.
Re: source-only uploads
On Fri, Sep 01, 2017 at 07:18:58PM +0100, Ian Jackson wrote:
> Andrey Rahmatullin writes ("Re: source-only uploads"):
> > On Fri, Sep 01, 2017 at 12:47:41PM +0200, Emmanuel Bourg wrote:
> > > Just yesterday I completely broke a key package used to build
> > > many Java packages, and I couldn't even rebuild it to fix the issue.
> >
> > Why? Does it B-D on itself?
>
> And, if it does, can it not be built using stretch ?
How will that help?
--
WBR, wRAR
signature.asc
Description: PGP signature
Re: source-only uploads
Andrey Rahmatullin writes ("Re: source-only uploads"):
> On Fri, Sep 01, 2017 at 12:47:41PM +0200, Emmanuel Bourg wrote:
> > Just yesterday I completely broke a key package used to build
> > many Java packages, and I couldn't even rebuild it to fix the issue.
>
> Why? Does it B-D on itself?
And, if it does, can it not be built using stretch ?
Ian.
Re: source-only uploads
On Fri, Sep 01, 2017 at 12:47:41PM +0200, Emmanuel Bourg wrote: > Just yesterday I completely broke a key package used to build > many Java packages, and I couldn't even rebuild it to fix the issue. Why? Does it B-D on itself? -- WBR, wRAR signature.asc Description: PGP signature
Re: source-only uploads
> and after someone > has implemented a solution for that there is no blocker left for > allowing only source-only uploads from maintainers. I'm all for source-only uploads and I adopted them recently, but I hope this restriction won't happen, or at least not without a derogation mechanism. Just yesterday I completely broke a key package used to build many Java packages, and I couldn't even rebuild it to fix the issue. I had to install a missing package locally and rebuild the binary on my machine. It would have been very difficult to fix that without binary uploads. Emmanuel Bourg
Re: First steps towards source-only uploads
On Wed, Aug 13, 2014, at 14:02, Guillem Jover wrote: Hi! On Mon, 2014-08-04 at 02:35:46 -0400, Joey Hess wrote: #756975 dpkg-dev: dpkg-genchanges option to only include arch:all debs This is now available in dpkg 1.17.11, and as mentioned on the bug report, you can use it in at least these ways: # Source and arch-indep only build, will fail if the package does # not produce any arch-indep binary, in which case you'd use -S. $ dpkg-buildpackage -g or # Full build, but filter the generated .changes file to only inlcude # source and possibly arch-indep binaries, will not fail if the # latter are missing. $ dpkg-buildpackage --changes-option=-g Any success of passing this to git-pbuilder? I had to modify pdebuild to not pass $DEBBUILDOPTS to dpkg-buildpackage invocation and only pass those options to the build inside of pbuilder. Ondrej -- Ondřej Surý ond...@sury.org Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS server -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1409041723.583972.156795125.3f34c...@webmail.messagingengine.com
Re: First steps towards source-only uploads
On Fri, 2014-08-15 at 11:27:00 +0200, Johannes Schauer wrote: Quoting Guillem Jover (2014-08-13 13:48:11) On Tue, 2014-08-12 at 13:27:38 +0200, Ansgar Burchardt wrote: There are also other problems that need to be eventually addressed: as far as I know there are some source packages producing arch:all binaries that cannot be built on all architectures. A Build-Architecture-Indep field was proposed to indicate where it should be built in this case[1], but for now I think we can require that maintainers have to upload the arch:all packages in this case. I think all proposed field names in that thread are rather confusing. In Debian packaging lingo build means several things, it can mean at least the build machine (!= host machine), or it can mean the act of building. In the case of Build-Depends style fields, it's referring to the act of building, but Architecture is related to the host system/compiler, so mixing the different meanings would be messy, think for example about cross-compiling. But is the question not *on* which architecture arch:all packages are built, and would the term build in Build-Architecture-Indep thus not be correct in both of its meanings (as the architecture I'm building *on* as well as the act of building)? Exactly, that was precisely my point, thus why this would be very confusing. (I guess I didn't make myself clear enough :) Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140823010451.gb12...@gaara.hadrons.org
Re: First steps towards source-only uploads
Hello, 2014-08-15 16:04 GMT+02:00 Ondřej Surý ond...@sury.org: I have encountered a situation where the FTBFS bug was caused by segfault in other package. This has forced me to split opendnssec-doc to arch:all package (which was good thing anyway), so there are cases where you want to build the arch:all on most stable and fastest arch. Could we just pick amd64 and be done? :) We currently got powerpc or s390x machines producing faster builds than amd64, or would you instead base your arch selection on archive build coverage which amd64 is probably closer to 100% than any other arch. If amd64 was to be picked, what would happen to packages producing arch:all packages that do not build on such architecture? Regards, -- Héctor Orón -.. . -... .. .- -. -.. . ...- . .-.. --- .--. . .-. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caodfweeabq7nuxldcjd6ce9vaw9jwmzhkkjcnbsch92ugm1...@mail.gmail.com
Re: First steps towards source-only uploads
On Tue, 19 Aug 2014, Hector Oron wrote: If amd64 was to be picked, what would happen to packages producing arch:all packages that do not build on such architecture? They would get a FTBFS bug and they would get fixed. Or, if it's not a bug, the package would be uploaded by the maintainer himself (until we have the required support to document in a field the architecture that should build the package). Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140819141244.gb13...@x230-buxy.home.ouaza.com
Re: First steps towards source-only uploads
On Tue, 19 Aug 2014, Hector Oron wrote: 2014-08-15 16:04 GMT+02:00 Ondřej Surý ond...@sury.org: I have encountered a situation where the FTBFS bug was caused by segfault in other package. This has forced me to split opendnssec-doc to arch:all package (which was good thing anyway), so there are cases where you want to build the arch:all on most stable and fastest arch. Could we just pick amd64 and be done? :) We currently got powerpc or s390x machines producing faster builds than amd64, or would you instead base your arch selection on archive build coverage which amd64 is probably closer to 100% than any other arch. The *use* coverage is the major reason, IMO. AMD64 is the arch most likely to have the best quality assurance, because it is directly used by the vast majority of the packagers. s390X is not generally available and we cannot own or replace them easily, so using it would be a bad idea IMHO: we need to keep our independence. AFAIK, very powerful AMD64 boxes are easier and cheaper to buy/replace than very powerful powerpc ones. Especially outside the USA and EU. If amd64 was to be picked, what would happen to packages producing arch:all packages that do not build on such architecture? It is a FTBFS error that must be fixed. Nothing new there. I do think we should ignore FTBFS bugs for arch:all on arches where the build failure happens due to insufficient resources (memory, storage, address space, etc). So, if the FTBFS bug for the arch:all package happens in some other arch than AMD64 *and* it is not due to resource restrictions on that arch, it is a FTBFS bug that has to be fixed just the same. This does not discriminate against other arches. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140819191532.ga19...@khazad-dum.debian.net
Re: First steps towards source-only uploads
Hi, Quoting Guillem Jover (2014-08-13 13:48:11) On Tue, 2014-08-12 at 13:27:38 +0200, Ansgar Burchardt wrote: There are also other problems that need to be eventually addressed: as far as I know there are some source packages producing arch:all binaries that cannot be built on all architectures. A Build-Architecture-Indep field was proposed to indicate where it should be built in this case[1], but for now I think we can require that maintainers have to upload the arch:all packages in this case. I think all proposed field names in that thread are rather confusing. In Debian packaging lingo build means several things, it can mean at least the build machine (!= host machine), or it can mean the act of building. In the case of Build-Depends style fields, it's referring to the act of building, but Architecture is related to the host system/compiler, so mixing the different meanings would be messy, think for example about cross-compiling. But is the question not *on* which architecture arch:all packages are built, and would the term build in Build-Architecture-Indep thus not be correct in both of its meanings (as the architecture I'm building *on* as well as the act of building)? cheers, josch -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140815092700.14069.93664@hoothoot
Re: First steps towards source-only uploads
Hello, 2014-08-13 22:59 GMT+02:00 Philipp Kern pk...@debian.org: On 2014-08-13 14:34, Raphael Hertzog wrote: On Wed, 13 Aug 2014, Colin Watson wrote: I don't think there's a good reason to build them separately, and some good reasons not to (for example, it would waste a good deal of buildd time for a number of packages without very hygienic separation of their build rules). My suggestion would be to just build them along with the architecture-dependent binaries for some nominated architecture, which could be package-specific or not depending on what you all have time for, and be done with it. In kali, that's exactly what I have been doing. Any package that builds an arch: all is sent to an amd64 buildd and sbuild on the amd64 buildd gets called with -A. It doesn't matter whether it builds only the arch: all or another package too. We need to convey if the arch:all is actually needed, though, otherwise dak will reject the package. Or could we simply build it always and have it discarded if the maintainer's copy had been accepted? Even building arch:all packages in one architecture might solve the issue, I do not like that approach, as it holds other arches from building until that primary arch has built arch:all packages. It also leads to not fully buildable packages for the rest of architectures, i.e. #690260 (openjdk-7: build empty doc packages on arm). Building arch:all on every architecture can also be seen as a waste of build time but effective. Another approach would be to find out if arch:all packages are available for build, if not, then build them along package in whichever architecture, otherwise skip them. That looks the most fair approach to me, but it can possibly lead to interesting races, depending on implementation. Regards, -- Héctor Orón -.. . -... .. .- -. -.. . ...- . .-.. --- .--. . .-. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caodfwegv40bnpuj_u5i-4nzvr5ua4wdo5wy+8xj+7wqs1lu...@mail.gmail.com
Re: First steps towards source-only uploads
On Fri, 15 Aug 2014, Hector Oron wrote: Even building arch:all packages in one architecture might solve the issue, I do not like that approach, as it holds other arches from building until that primary arch has built arch:all packages. I understand the concern at the philosophical level but on a practical level, using the fastest architecture we have is actually the way to ensure that all arches get their arch all packages in the fastest possible way. It also leads to not fully buildable packages for the rest of architectures, i.e. #690260 (openjdk-7: build empty doc packages on arm). Those are bugs that can be found as part of QA activities, we don't have to complicate our build infrastructure just to ensure that this specific class of bugs gets found during build. On the contrary, this is a sure way to create unexpected problems (by letting bad arch all packages enter the archive) that maintainers won't be able to anticipate as they do test-build their packages on i386/amd64 mainly. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140815130004.gc17...@x230-buxy.home.ouaza.com
Re: First steps towards source-only uploads
On Fri, 15 Aug 2014, Raphael Hertzog wrote: On Fri, 15 Aug 2014, Hector Oron wrote: Even building arch:all packages in one architecture might solve the issue, I do not like that approach, as it holds other arches from building until that primary arch has built arch:all packages. I understand the concern at the philosophical level but on a practical level, using the fastest architecture we have is actually the way to ensure that all arches get their arch all packages in the fastest possible way. And any package that cannot build arch:all on a released arch for which it produces binary packages potentially has a FTBFS bug, anyway, which can be reported by any interested parties. Exceptions would be arches that are too resource-constrained to build it in the first place. IMHO, it is NOT the job of the autobuilders to track down and find every FTBFS bug in Debian. They cannot be expected to waste resources tracking down FTBFS bugs that have no effect on the current operations of the autobuilder network. It also leads to not fully buildable packages for the rest of architectures, i.e. #690260 (openjdk-7: build empty doc packages on arm). Those are bugs that can be found as part of QA activities, we don't have to complicate our build infrastructure just to ensure that this specific class of bugs gets found during build. Agreed. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140815133712.gc26...@khazad-dum.debian.net
Re: First steps towards source-only uploads
On Fri, Aug 15, 2014, at 15:37, Henrique de Moraes Holschuh wrote: And any package that cannot build arch:all on a released arch for which it produces binary packages potentially has a FTBFS bug, anyway, which can be reported by any interested parties. Exceptions would be arches that are too resource-constrained to build it in the first place. I have encountered a situation where the FTBFS bug was caused by segfault in other package. This has forced me to split opendnssec-doc to arch:all package (which was good thing anyway), so there are cases where you want to build the arch:all on most stable and fastest arch. Could we just pick amd64 and be done? :) Cheers, -- Ondřej Surý ond...@sury.org Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS server -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1408111452.213427.153089893.5c8a7...@webmail.messagingengine.com
Re: First steps towards source-only uploads
On Fri, 15 Aug 2014, Ondřej Surý wrote: On Fri, Aug 15, 2014, at 15:37, Henrique de Moraes Holschuh wrote: And any package that cannot build arch:all on a released arch for which it produces binary packages potentially has a FTBFS bug, anyway, which can be reported by any interested parties. Exceptions would be arches that are too resource-constrained to build it in the first place. I have encountered a situation where the FTBFS bug was caused by segfault in other package. This has forced me to split opendnssec-doc to arch:all package (which was good thing anyway), so there are cases where you want to build the arch:all on most stable and fastest arch. Could we just pick amd64 and be done? :) Yes, please :-) -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140815140823.ga1...@khazad-dum.debian.net
Re: First steps towards source-only uploads
Hi! [ I had a note to reply to the previous thread, but never got to it. ] On Tue, 2014-08-12 at 13:27:38 +0200, Ansgar Burchardt wrote: On 08/12/2014 12:33, Hector Oron wrote: 2014-08-01 9:37 GMT+02:00 Ansgar Burchardt ans...@debian.org: We will allow not including arch:all packages in uploads once we have sorted out how to get them built. Has it been already discussed? If so, where? There are also other problems that need to be eventually addressed: as far as I know there are some source packages producing arch:all binaries that cannot be built on all architectures. A Build-Architecture-Indep field was proposed to indicate where it should be built in this case[1], but for now I think we can require that maintainers have to upload the arch:all packages in this case. I think all proposed field names in that thread are rather confusing. In Debian packaging lingo build means several things, it can mean at least the build machine (!= host machine), or it can mean the act of building. In the case of Build-Depends style fields, it's referring to the act of building, but Architecture is related to the host system/compiler, so mixing the different meanings would be messy, think for example about cross-compiling. [1] https://lists.debian.org/debian-devel/2014/02/msg01149.html Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140813114811.ga13...@gaara.hadrons.org
Re: First steps towards source-only uploads
Hi! On Mon, 2014-08-04 at 02:35:46 -0400, Joey Hess wrote: #756975 dpkg-dev: dpkg-genchanges option to only include arch:all debs This is now available in dpkg 1.17.11, and as mentioned on the bug report, you can use it in at least these ways: # Source and arch-indep only build, will fail if the package does # not produce any arch-indep binary, in which case you'd use -S. $ dpkg-buildpackage -g or # Full build, but filter the generated .changes file to only inlcude # source and possibly arch-indep binaries, will not fail if the # latter are missing. $ dpkg-buildpackage --changes-option=-g The advantage of the second is that the package is fully built so that the maintainer can test that it builds and can install and test the resulting packages. Unfortunately as arch-specific packages are filtered out from the .changes file, lintian will not be able to find them. So you migth want to do a normal build followed by one with either -S or -g. Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140813120220.gb13...@gaara.hadrons.org
Re: First steps towards source-only uploads
Hi, On Tue, 12 Aug 2014, Ansgar Burchardt wrote: First, w-b has to recognize that arch:all packages need to be built. And they need to be scheduled on a buildd which builds the arch:all packages (and only the arch:all packages?). For the latter I assume sbuild would need an option to build only the arch:all packages using dpkg-buildpackage -A. It would also be interesting to test which packages FTBFS in this case. FWIW, sbuild supports -A (--arch-all) and this will result in calling dpkg-buildpackage -b (instead of -B). On Wed, 13 Aug 2014, Colin Watson wrote: I don't think there's a good reason to build them separately, and some good reasons not to (for example, it would waste a good deal of buildd time for a number of packages without very hygienic separation of their build rules). My suggestion would be to just build them along with the architecture-dependent binaries for some nominated architecture, which could be package-specific or not depending on what you all have time for, and be done with it. +1 In kali, that's exactly what I have been doing. Any package that builds an arch: all is sent to an amd64 buildd and sbuild on the amd64 buildd gets called with -A. It doesn't matter whether it builds only the arch: all or another package too. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140813123433.gb6...@x230-buxy.home.ouaza.com
Re: First steps towards source-only uploads
Guillem Jover wrote: # Full build, but filter the generated .changes file to only inlcude # source and possibly arch-indep binaries, will not fail if the # latter are missing. $ dpkg-buildpackage --changes-option=-g The advantage of the second is that the package is fully built so that the maintainer can test that it builds and can install and test the resulting packages. Unfortunately as arch-specific packages are filtered out from the .changes file, lintian will not be able to find them. So you migth want to do a normal build followed by one with either -S or -g. I tend to use debuild from devscripts, primarily because it automatically runs lintian. If dpkg-buildpackage gained native support for invoking lintian, it could do so on the unfiltered changes file, and then emit the filtered changes file. Would you consider including support for that in dpkg-buildpackage? - Josh Triplett -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140813192409.GA3056@jtriplet-mobl1
Re: First steps towards source-only uploads
On Wed, 2014-08-13 at 12:24:49 -0700, Josh Triplett wrote: Guillem Jover wrote: # Full build, but filter the generated .changes file to only inlcude # source and possibly arch-indep binaries, will not fail if the # latter are missing. $ dpkg-buildpackage --changes-option=-g The advantage of the second is that the package is fully built so that the maintainer can test that it builds and can install and test the resulting packages. Unfortunately as arch-specific packages are filtered out from the .changes file, lintian will not be able to find them. So you migth want to do a normal build followed by one with either -S or -g. I tend to use debuild from devscripts, primarily because it automatically runs lintian. If dpkg-buildpackage gained native support for invoking lintian, it could do so on the unfiltered changes file, and then emit the filtered changes file. Would you consider including support for that in dpkg-buildpackage? dpkg-buildpackage does support running a checker like lintian since 1.17.6 (see --check-command and DEB_CHECK_COMMAND in the man page). Doing what you request would require running dpkg-genchanges twice, but only sometimes, I'm not sure I like how that would smell as an interface, but I'll ponder about it (maybe just adding an option to request the filtering, so that the build type option is passed to dpkg-genchanges for the final .changes file, but otherwise dpkg-buildpackage performs a normal full build, hmmm). What comes to mind though, although slightly cumbersome, is that right now there's also this other option, which should cover all requirements (except being succint :): $ export DEB_CHECK_COMMAND=lintian $ dpkg-buildpackage -us -uc $ dpkg-genchanges -g ../pkgname_version_src+all.changes $ debsign ../pkgname_version_src+all.changes (dpkg-genchanges really needs a -O[filename] option, which I'll be implementing soon, and signing should eventually be disabled by default anyway as discussed some time ago here.) Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140813201603.ga11...@gaara.hadrons.org
Re: First steps towards source-only uploads
On 2014-08-13 14:34, Raphael Hertzog wrote: On Wed, 13 Aug 2014, Colin Watson wrote: I don't think there's a good reason to build them separately, and some good reasons not to (for example, it would waste a good deal of buildd time for a number of packages without very hygienic separation of their build rules). My suggestion would be to just build them along with the architecture-dependent binaries for some nominated architecture, which could be package-specific or not depending on what you all have time for, and be done with it. +1 In kali, that's exactly what I have been doing. Any package that builds an arch: all is sent to an amd64 buildd and sbuild on the amd64 buildd gets called with -A. It doesn't matter whether it builds only the arch: all or another package too. We need to convey if the arch:all is actually needed, though, otherwise dak will reject the package. Or could we simply build it always and have it discarded if the maintainer's copy had been accepted? Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/99bea068419b09ad368191a1cd596...@hub.kern.lc
Re: First steps towards source-only uploads
Hello, 2014-08-01 9:37 GMT+02:00 Ansgar Burchardt ans...@debian.org: We will allow not including arch:all packages in uploads once we have sorted out how to get them built. Has it been already discussed? If so, where? Regards, -- Héctor Orón -.. . -... .. .- -. -.. . ...- . .-.. --- .--. . .-. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAODfWeFagKRU8wE-N=wSFqaqBo4Ysg1Qpd9z3o+0=2w51sn...@mail.gmail.com
Re: First steps towards source-only uploads
Hi, On 08/12/2014 12:33, Hector Oron wrote: 2014-08-01 9:37 GMT+02:00 Ansgar Burchardt ans...@debian.org: We will allow not including arch:all packages in uploads once we have sorted out how to get them built. Has it been already discussed? If so, where? Not the part to actually implement it. But from my memories people in the buildd team had no principal objections, it's just that someone needs to spend time to implement everything needed for this. I asked in #-buildd recently what needs to be done to start building arch:all packages in the buildd network: First, w-b has to recognize that arch:all packages need to be built. And they need to be scheduled on a buildd which builds the arch:all packages (and only the arch:all packages?). For the latter I assume sbuild would need an option to build only the arch:all packages using dpkg-buildpackage -A. It would also be interesting to test which packages FTBFS in this case. There are also other problems that need to be eventually addressed: as far as I know there are some source packages producing arch:all binaries that cannot be built on all architectures. A Build-Architecture-Indep field was proposed to indicate where it should be built in this case[1], but for now I think we can require that maintainers have to upload the arch:all packages in this case. Ansgar [1] https://lists.debian.org/debian-devel/2014/02/msg01149.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53e9fa2a.5090...@debian.org
Re: First steps towards source-only uploads
On Tue, Aug 12, 2014 at 01:27:38PM +0200, Ansgar Burchardt wrote: First, w-b has to recognize that arch:all packages need to be built. And they need to be scheduled on a buildd which builds the arch:all packages (and only the arch:all packages?). For the latter I assume sbuild would need an option to build only the arch:all packages using dpkg-buildpackage -A. It would also be interesting to test which packages FTBFS in this case. I don't think there's a good reason to build them separately, and some good reasons not to (for example, it would waste a good deal of buildd time for a number of packages without very hygienic separation of their build rules). My suggestion would be to just build them along with the architecture-dependent binaries for some nominated architecture, which could be package-specific or not depending on what you all have time for, and be done with it. -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140812232335.ga14...@riva.ucam.org
Re: Bug#756835: First steps towards source-only uploads
Hi, Quoting Matthias Urlichs (2014-08-07 07:54:26) Also, build profiles are not explained anywhere in Policy (unless that's been added after 3.9.5), so how would I discover which values are allowed / make sense? right. For the purpose of documenting the Package-List its usage for build profiles can just be omitted until build profiles are documented. Somehow I thought that a bug to document the restriction list syntax and build profiles had long been filled but apparently that wasn't the case so now there is #757760. cheers, josch -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140811075507.14069.52073@hoothoot
Re: First steps towards source-only uploads
Hi, Quoting Charles Plessy (2014-08-06 07:41:40) what do you think about the patch I sent to the Policy, for describing the syntax of the current optional fields of the Packages-List field ? Do you think that modifications are needed ? Would you second it ? https://bugs.debian.org/756835 Have a nice day, I had worded it a bit differently: diff --git a/policy.sgml b/policy.sgml index fbbe4a3..840951c 100644 --- a/policy.sgml +++ b/policy.sgml @@ -3824,10 +3824,17 @@ Checksums-Sha256: the source package, considering every architecture. The first line of the field value is empty. Each one of the next lines describes one binary package, by listing its name, type, section and priority - separated by spaces. Fifth and subsequent space-separated items - may be present and parsers must allow them. See the - qref id=f-Package-TypePackage-Type/qref field for a list of - package types. + separated by spaces. + See the qref id=f-Package-TypePackage-Type/qref field for a + list of package types. + Fifth and subsequent space-separated items are key/value pairs of + the form ttkey=value1,value2/tt. The currently used keys are + the ttarch/tt key which lists the architectures a binary + package builds for and the ttprofile/tt key which lists the + build profiles a binary package builds in. + footnoteThe key/value syntax for all fields after the fourth was + introduced in prgndpkg/prgn version tt1.17.7/tt in + emJessie/em./footnote /p /sect1 cheers, josch -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140806060316.19236.55079@hoothoot
Re: First steps towards source-only uploads
Ansgar Burchardt ans...@debian.org (2014-08-01): as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. We will allow not including arch:all packages in uploads once we have sorted out how to get them built. Source-only uploads to NEW might also come in the future, but dak needs a bit more work to be able to handle them. This is absolutely awesome. Many many thanks. Mraw, KiBi. signature.asc Description: Digital signature
Re: First steps towards source-only uploads
hi can you please take this adress off the list ? many thanks! LG Von meinem iPod gesendet Am 01.08.2014 um 09:37 schrieb Ansgar Burchardt ans...@debian.org: Hi, as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. We will allow not including arch:all packages in uploads once we have sorted out how to get them built. Source-only uploads to NEW might also come in the future, but dak needs a bit more work to be able to handle them. Ansgar -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/896d20a3-161c-4fd6-8c4e-5a6dad258...@gmx.ch
Re: First steps towards source-only uploads
Hi, Quoting Ansgar Burchardt (2014-08-01 12:25:05) I want to understand purpose and syntax of this new field. The field was originally discussed in https://bugs.debian.org/619131 to allow easier processing of packages that build arch-specific binaries such as src:linux[1]. The architecture information was only (re)introduced later, as far as I know to help with bootstrapping, but it turns out that I also want this for source-only uploads to catch uploads introducing NEW binary packages. The field started out with the binary package name, type, section and priority. For bootstrapping it is necessary to know for which architectures and build profiles a binary package builds without having access to the unpacked source package but just from looking at the Packages/Sources files. Thus this information was added as optional fields. Every field that comes after the first four will be of the form key=value. The arch information will always be printed and the profile field only for a build with a build profile. Guillem explains the new field here: https://lists.debian.org/20140421140417.ga1...@gaara.hadrons.org cheers, josch -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140806051637.19236.26348@hoothoot
Re: First steps towards source-only uploads
Control: retitle -1 Extension of the syntax of the Packages-List field. Le Wed, Aug 06, 2014 at 07:16:37AM +0200, Johannes Schauer a écrit : The field started out with the binary package name, type, section and priority. For bootstrapping it is necessary to know for which architectures and build profiles a binary package builds without having access to the unpacked source package but just from looking at the Packages/Sources files. Thus this information was added as optional fields. Every field that comes after the first four will be of the form key=value. The arch information will always be printed and the profile field only for a build with a build profile. Guillem explains the new field here: Hi Johannes, Ansgar, Guillem, and everybody, what do you think about the patch I sent to the Policy, for describing the syntax of the current optional fields of the Packages-List field ? Do you think that modifications are needed ? Would you second it ? https://bugs.debian.org/756835 Have a nice day, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140806054139.ga13...@falafel.plessy.net
Re: First steps towards source-only uploads
Filed a few bug reports on this: #756975 dpkg-dev: dpkg-genchanges option to only include arch:all debs #756978 dgit: .tar-less push The possibility of the second one is .. pretty amazing! -- see shy jo signature.asc Description: Digital signature
Re: First steps towards source-only uploads
On 08/01/2014 06:15 PM, Holger Levsen wrote: Hi Ansgar, On Freitag, 1. August 2014, Ansgar Burchardt wrote: as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: [...] whoooh! Yay! Thanks a lot to those who made this real! This is *great* news! cheers, Holger Likewise! Thanks for working on this. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53ddf708.6070...@debian.org
Re: First steps towards source-only uploads
Hi, Kurt Roeckx k...@roeckx.be writes: Please also make sure you rename the changes files to not conflict with the .changes files the buildd is going to use. As of today, that's no longer required. :) Ansgar -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87zjfl46kd@deep-thought.43-1.org
Re: First steps towards source-only uploads
Ansgar Burchardt wrote: * Architecture-independent (arch:all) packages must be included in uploads. That can be read 2 different ways.. I hope it means: If you have an arch:all, you have to upload it, but if there is none, you can upload with no .debs. Is that correct? -- see shy jo, still on dialup, so could probably find excuses to add arch:all to most of his packages if he needed to.. signature.asc Description: Digital signature
Re: First steps towards source-only uploads
On Sun, Aug 3, 2014 at 8:13 PM, Joey Hess jo...@debian.org wrote: Ansgar Burchardt wrote: * Architecture-independent (arch:all) packages must be included in uploads. That can be read 2 different ways.. I hope it means: If you have an arch:all, you have to upload it, but if there is none, you can upload with no .debs. Is that correct? Yes, that's correct. You can upload just the source package itself if it doesn't build any arch-indep packages. Regards, Vincent -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caczd_tcxlluqd2p7dgamboh38wqsl4okmpt9f15j7rxgveh...@mail.gmail.com
Bug#756835: First steps towards source-only uploads
Package: debian-policy Severity: wishlist Version: 3.9.5.0 Le Fri, Aug 01, 2014 at 12:25:05PM +0200, Ansgar Burchardt a écrit : On 08/01/2014 12:17, martin f krafft wrote: also sprach Paul Wise p...@debian.org [2014-08-01 11:33 +0200]: * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. Can we read up more on this somewhere? It is the default if you are using dpkg-dev from jessie and you don't need to do anything other than generating your .dsc with dpkg-source as per usual. I want to understand purpose and syntax of this new field. The field was originally discussed in https://bugs.debian.org/619131 to allow easier processing of packages that build arch-specific binaries such as src:linux[1]. The architecture information was only (re)introduced later, as far as I know to help with bootstrapping, but it turns out that I also want this for source-only uploads to catch uploads introducing NEW binary packages. Hi Ansgar, Martin, and Policy Editors, here is a proposed update for the description of the Package-List field in the Policy's section 5.6.27. (patch attached). For the busy people, here is the current content of §5.6.27–8. - 5.6.27 Package-List Multiline field listing all the packages that can be built from the source package, considering every architecture. The first line of the field value is empty. Each one of the next lines describes one binary package, by listing its name, type, section and priority separated by spaces. Fifth and subsequent space-separated items may be present and parsers must allow them. See the Package-Type field for a list of package types. 5.6.28 Package-Type Simple field containing a word indicating the type of package: deb for binary packages and udeb for micro binary packages. Other types not defined here may be indicated. In source package control files, the Package-Type field should be omitted instead of giving it a value of deb, as this value is assumed for paragraphs lacking this field. - Everybody's suggestions are welcome ! Have a nice week-end, and big thank you to Ansgar and the other FTP Masters for the source-only uploads ! -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan From b1aa1bf72723e4594557f9898da97afe23f0c900 Mon Sep 17 00:00:00 2001 From: Charles Plessy ple...@debian.org Date: Sat, 2 Aug 2014 19:30:57 +0900 Subject: [PATCH] =?UTF-8?q?Document=20the=20=E2=80=98arch=E2=80=99=20and?= =?UTF-8?q?=20=E2=80=98profile=E2=80=99=20options=20in=20the=20Package-Lis?= =?UTF-8?q?t=20field.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- policy.sgml | 8 1 file changed, 8 insertions(+) diff --git a/policy.sgml b/policy.sgml index ae9d173..33c4f1a 100644 --- a/policy.sgml +++ b/policy.sgml @@ -3826,6 +3826,14 @@ Checksums-Sha256: qref id=f-Package-TypePackage-Type/qref field for a list of package types. /p + p + The optional fields currently in use add informations related to +architectures build profiles, with a syntax such as +ttname=value1,value2/tt and names ttarch/tt and +ttprofile/ttfootnote + They were introduced in prgndpkg/prgn version + tt1.17.7/tt in emJessie/em/footnote. + /p /sect1 sect1 id=f-Package-Type -- 2.0.1
Re: First steps towards source-only uploads
01.08.2014 11:37, Ansgar Burchardt wrote: Hi, as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. Is there an easy way to produce such uploads? Thanks! /mjt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53db47b6.7010...@msgid.tls.msk.ru
Re: First steps towards source-only uploads
On Fri, 01 Aug 2014, Michael Tokarev wrote: 01.08.2014 11:37, Ansgar Burchardt wrote: as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. Is there an easy way to produce such uploads? If your source builds arch-all packages, this has actually worked for a long time already. The changestool utility from the reprepro package can be used to manipulate a .changes file as needed. For instance, the way my tor-package build machinery works, I end up with a source-only .changes file (created by dpkg-genchanges -S) and then add the .all package using changestool adddeb. Cheers, -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140801080951.gl30...@anguilla.noreply.org
Re: First steps towards source-only uploads
On Fri, Aug 1, 2014, at 09:54, Michael Tokarev wrote: 01.08.2014 11:37, Ansgar Burchardt wrote: Hi, as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. Is there an easy way to produce such uploads? Build binary packages as usual, but sed-out _(amd64|i386).deb from resulting .changes before signing it. Ondrej -- Ondřej Surý ond...@sury.org Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS server -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1406880972.2158370.147993346.05a1a...@webmail.messagingengine.com
Re: First steps towards source-only uploads
also sprach Ansgar Burchardt ans...@debian.org [2014-08-01 09:37 +0200]: as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: Wow. This is great news! Thank you so much for your perseverance. * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. Can we read up more on this somewhere? -- .''`. martin f. krafft madduck@d.o @martinkrafft : :' : proud Debian developer `. `'` http://people.debian.org/~madduck `- Debian - when you have better things to do than fixing systems without a god, life is only a matter of opinion. -- douglas adams digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: First steps towards source-only uploads
On Fri, Aug 1, 2014 at 4:38 PM, martin f krafft wrote: also sprach Ansgar Burchardt ans...@debian.org [2014-08-01 09:37 +0200]: * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. Can we read up more on this somewhere? It is the default if you are using dpkg-dev from jessie and you don't need to do anything other than generating your .dsc with dpkg-source as per usual. -- bye, pabs https://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAKTje6Eijt3Eft8ZJv6=wa3gckioqpvfnasqsan9eobmfmn...@mail.gmail.com
Re: First steps towards source-only uploads
also sprach Paul Wise p...@debian.org [2014-08-01 11:33 +0200]: * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. Can we read up more on this somewhere? It is the default if you are using dpkg-dev from jessie and you don't need to do anything other than generating your .dsc with dpkg-source as per usual. I want to understand purpose and syntax of this new field. -- .''`. martin f. krafft madduck@d.o @martinkrafft : :' : proud Debian developer `. `'` http://people.debian.org/~madduck `- Debian - when you have better things to do than fixing systems quidquid latine dictum sit, altum viditur. digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
Re: First steps towards source-only uploads
Hi Ansgar, On Freitag, 1. August 2014, Ansgar Burchardt wrote: as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: [...] whoooh! Yay! Thanks a lot to those who made this real! This is *great* news! cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: First steps towards source-only uploads
On 08/01/2014 12:17, martin f krafft wrote: also sprach Paul Wise p...@debian.org [2014-08-01 11:33 +0200]: * The source package includes a Package-List field that also has an arch=* column. dpkg (= 1.17.7) will include this. Can we read up more on this somewhere? It is the default if you are using dpkg-dev from jessie and you don't need to do anything other than generating your .dsc with dpkg-source as per usual. I want to understand purpose and syntax of this new field. The field was originally discussed in https://bugs.debian.org/619131 to allow easier processing of packages that build arch-specific binaries such as src:linux[1]. The architecture information was only (re)introduced later, as far as I know to help with bootstrapping, but it turns out that I also want this for source-only uploads to catch uploads introducing NEW binary packages. Ansgar [1] This is currently broken in dak, but teaching process-new to handle source-only uploads will also address this. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53db6b01.70...@debian.org
Re: First steps towards source-only uploads
Thanks a lot; this is great news! Richard Sent by mobile; excuse my brevity.
Re: First steps towards source-only uploads
On Fri, Aug 01, 2014 at 10:16:12AM +0200, Ondrej Surý wrote: On Fri, Aug 1, 2014, at 09:54, Michael Tokarev wrote: 01.08.2014 11:37, Ansgar Burchardt wrote: Hi, as a first step towards source-only uploads, the archive will now accept source-only uploads provided the following conditions are met: * The source package is not NEW and does not build NEW binaries. * Architecture-independent (arch:all) packages must be included in uploads. Is there an easy way to produce such uploads? Build binary packages as usual, but sed-out _(amd64|i386).deb from resulting .changes before signing it. Please also make sure you rename the changes files to not conflict with the .changes files the buildd is going to use. Kurt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140801205857.ga9...@roeckx.be
Re: First steps towards source-only uploads
* Kurt Roeckx k...@roeckx.be, 2014-08-01, 22:58: Build binary packages as usual, but sed-out _(amd64|i386).deb from resulting .changes before signing it. Please also make sure you rename the changes files to not conflict with the .changes files the buildd is going to use. Can we fix dak not to care about names of *.changes? These names are not cryptographically signed, so they shouldn't be trusted anyway. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140801215149.ga8...@jwilk.net
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Sun, Nov 25, 2012 at 01:32:16PM -0500, Barry Warsaw wrote: On Nov 23, 2012, at 03:06 PM, YunQiang Su wrote: you always need to build for one arch and test, then why not upload it? I think there are a lot of good reasons to do source-only uploads, even when you should be building locally for testing purposes. * Reproducibility - buildds provide a more controlled environment than developers' machines. [snip] * Testability - Is there any guarantee that a package's tests have been run during the local build process? [snip] These both would be provided by throwing away the built component and rebuilding in a closed environment, which is (I believe) the current thinking of the best way forward. Neil -- signature.asc Description: Digital signature
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Nov 23, 2012, at 03:06 PM, YunQiang Su wrote: you always need to build for one arch and test, then why not upload it? I think there are a lot of good reasons to do source-only uploads, even when you should be building locally for testing purposes. * Reproducibility - buildds provide a more controlled environment than developers' machines. This means that it's less likely that some local environmental factor creeps into your binary packages, or is silently relied upon to produce a successful build. * Testability - Is there any guarantee that a package's tests have been run during the local build process? I think it's a good thing to enable more packages tests (e.g. through dh_auto_tests or DEP 8 tests), so ensuring that DEP 8 tests for example are always run before the package is published is, IMO a good thing. Cheers, -Barry signature.asc Description: PGP signature
Re: release goal for jessie! (Re: Source-only uploads
]] Gunnar Wolf Didier Raboud dijo [Wed, Nov 21, 2012 at 09:21:19PM +0100]: Actually, I like that way to put it as it leaves us with multiple ways forward: * accept source-only; * drop uploaded binaries; I would join this camp as well. Without the working knowledge of being a DSA or buildd-admin, I cannot assure how much would this increase our workload, but it would probably just mean rebuilding for the most popular architectures (that is, AMD64 or i386), hardware for which is readily available and should pose no additional effort to get. And it would mean IMO a good leap forward in ensuring buildability — Even more with arch:all I doubt it would make any change in the workload for us in the DSA. I assume it will lead to a slight increase in workload for the buildd maintainers. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d2z3fc3t@xoog.err.no
Re: release goal for jessie! (Source-only uploads
Thomas Goirand zigo at debian.org writes: Though I'm in the favor of dropping binaries rather than source-only, This could even help the cases of packages that need itself to be built. When a packager does a source+binary upload of foo (= 1.2-1), it would be built in a clean, minimal chroot (yes, I’m adding the minimal here, with several side-stabs…) whose sources.list would contain unstable, and experimental for packages targetting that, as right now, plus ANOTHER, NEW repository, which is created by buildd “on the fly”, which contains the binaries that the packager uploaded, but is marked to never be used automatically just like experimental, so the binaries by the packager *CAN* be used to build the binaries that will make it into the archive, but *only* if the packager explicitly states it, here by doing a B-D on foo (= 1.2-1), or even foo (= 1.2) would work in this case. That one-package APT repo and its content would then be dropped afterwards, never to be seen again, no matter whether the build succeeded or not. bye, //mirabilos -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20121125t005535-...@post.gmane.org
Re: release goal for jessie! (Source-only uploads
On 25/11/12 00:00, Thorsten Glaser wrote: Thomas Goirandzigoat debian.org writes: Though I'm in the favor of dropping binaries rather than source-only, This could even help the cases of packages that need itself to be built. When a packager does a source+binary upload of foo (= 1.2-1), it would be built in a clean, minimal chroot (yes, I’m adding the minimal here, with several side-stabs…) whose sources.list would contain unstable, and experimental for packages targetting that, as right now, plus ANOTHER, NEW repository, which is created by buildd “on the fly”, which contains the binaries that the packager uploaded, but is marked to never be used automatically just like experimental, so the binaries by the packager *CAN* be used to build the binaries that will make it into the archive, but *only* if the packager explicitly states it, here by doing a B-D on foo (= 1.2-1), or even foo (= 1.2) would work in this case. That one-package APT repo and its content would then be dropped afterwards, never to be seen again, no matter whether the build succeeded or not. bye, //mirabilos If you need binaries from a package in its build process then you use the ones in the build tree in preference to any that might be installed. It's just a shame that automake can't use LD_LIBRARY_PATH to prefer shared libraries from the build tree in the same way. Regards, Philip -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/50b1bb8c.30...@philipashmore.com
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
Didier Raboud dijo [Wed, Nov 21, 2012 at 09:21:19PM +0100]: I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? Both. I'd argue that it's a bug in both. BTW, can we have this as a release goal for jessie, that all packages have been build on Debian buildd infrastructure? ;-) Actually, I like that way to put it as it leaves us with multiple ways forward: * accept source-only; * drop uploaded binaries; I would join this camp as well. Without the working knowledge of being a DSA or buildd-admin, I cannot assure how much would this increase our workload, but it would probably just mean rebuilding for the most popular architectures (that is, AMD64 or i386), hardware for which is readily available and should pose no additional effort to get. And it would mean IMO a good leap forward in ensuring buildability — Even more with arch:all * (optionally: ) diff built and uploaded binaries, blame; This can be a bit more tricky. Of course, diffing the .build fails would not work, to begin with, because of the pathnames. But even diffing the shipped files — two shipped files are not guaranteed to be bit-by-bit identical, even if compiled in the same hardware. What is yet unclear is if we want to build all (as in arch:any+all) or all (as in arch:any) packages on buildds. Rebuilding arch:all packages is quite important IMO. I would probably add a rebuild when entering testing where this to be a perfect world, to ensure continued buildability. But I know it's probably too much to ask... And it would still be incomplete (as rebuilding anything that build-depends on this package could still be added — And down this path we can find madness...) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121123163002.ga6...@gwolf.org
Re: release goal for jessie! (Re: Source-only uploads
On 11/24/2012 12:30 AM, Gunnar Wolf wrote: I would join this camp as well. Without the working knowledge of being a DSA or buildd-admin, I cannot assure how much would this increase our workload, but it would probably just mean rebuilding for the most popular architectures (that is, AMD64 or i386), hardware for which is readily available and should pose no additional effort to get. And it would mean IMO a good leap forward in ensuring buildability — Even more with arch:all +1 to all of the above Though I'm in the favor of dropping binaries rather than source-only, so that later it would be possible to do some sanity checks with the buildd and DD version of the binary packages (it doesn't have to be a bit-by-bit compare, but I'm sure we can find some interesting tests to do, like checking if control files are identical, etc...). Just a 2 cents idea, since I wont be implementing it, Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/50afd5a3.6090...@debian.org
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Wed, Nov 21, 2012 at 09:21:19PM +0100, Didier Raboud wrote: What is yet unclear is if we want to build all (as in arch:any+all) or all (as in arch:any) packages on buildds. Are there any reasons to not built arch:all on buildds aside from technical problems? -- WBR, wRAR signature.asc Description: Digital signature
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
you always need to build for one arch and test, then why not upload it? On Thu, Nov 22, 2012 at 4:33 PM, Andrey Rahmatullin w...@wrar.name wrote: On Wed, Nov 21, 2012 at 09:21:19PM +0100, Didier Raboud wrote: What is yet unclear is if we want to build all (as in arch:any+all) or all (as in arch:any) packages on buildds. Are there any reasons to not built arch:all on buildds aside from technical problems? -- WBR, wRAR -- YunQiang Su
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Fri, Nov 23, 2012 at 03:06:22PM +0800, YunQiang Su wrote: you always need to build for one arch and test, then why not upload it? How is that related to my question? Also, please don't top-post and dont send me copies. On Thu, Nov 22, 2012 at 4:33 PM, Andrey Rahmatullin w...@wrar.name wrote: On Wed, Nov 21, 2012 at 09:21:19PM +0100, Didier Raboud wrote: What is yet unclear is if we want to build all (as in arch:any+all) or all (as in arch:any) packages on buildds. Are there any reasons to not built arch:all on buildds aside from technical problems? -- WBR, wRAR signature.asc Description: Digital signature
release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
Hi, On Dienstag, 20. November 2012, Henrique de Moraes Holschuh wrote: I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? Both. I'd argue that it's a bug in both. BTW, can we have this as a release goal for jessie, that all packages have been build on Debian buildd infrastructure? ;-) cheers, Holger -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201211212059.03249.hol...@layer-acht.org
Re: release goal for jessie! (Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
Le mercredi, 21 novembre 2012 20.59:02, Holger Levsen a écrit : Hi, On Dienstag, 20. November 2012, Henrique de Moraes Holschuh wrote: I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? Both. I'd argue that it's a bug in both. BTW, can we have this as a release goal for jessie, that all packages have been build on Debian buildd infrastructure? ;-) Actually, I like that way to put it as it leaves us with multiple ways forward: * accept source-only; * drop uploaded binaries; * (optionally: ) diff built and uploaded binaries, blame; What is yet unclear is if we want to build all (as in arch:any+all) or all (as in arch:any) packages on buildds. OdyX -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201211212121.19776.o...@debian.org
Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On 20 November 2012 12:23, Henrique de Moraes Holschuh h...@debian.org wrote: On Tue, 20 Nov 2012, Dmitrijs Ledkovs wrote: I am sorry, if I was not clear. I am aware of the last iteration, but I am not enquiring about the default policy within debian as to how we should upload by default. I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? Both. Where is this policy documented? I have checked Debian Policy and ftp-masters mini-site faq/rejects sections. Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhlujq19s9rvymg8_o6eireyh6fvhlhmzh6u5j-bv8_nc...@mail.gmail.com
Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Tue, Nov 20, 2012 at 11:10:37AM +, Dmitrijs Ledkovs wrote: Source-only uploads are not allowed. Why not? May I request a binNMU for the architecture (amd64) I upload? I currently do not have facilities to build the package in question with the host running Debian's kernel. For the package in question it is important to build in the same environment as the rest of the packages in the distribution. The sole purpose of the package is to gather as much detail about the environment as possible, which has been proven to be highly valuable for security analysis and fixing highly strict test-suites. The last iteration of this was started at http://lists.debian.org/debian-devel/2012/10/msg00296.html -- WBR, wRAR signature.asc Description: Digital signature
Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On 20 November 2012 11:14, Andrey Rahmatullin w...@wrar.name wrote: On Tue, Nov 20, 2012 at 11:10:37AM +, Dmitrijs Ledkovs wrote: Source-only uploads are not allowed. Why not? May I request a binNMU for the architecture (amd64) I upload? I currently do not have facilities to build the package in question with the host running Debian's kernel. For the package in question it is important to build in the same environment as the rest of the packages in the distribution. The sole purpose of the package is to gather as much detail about the environment as possible, which has been proven to be highly valuable for security analysis and fixing highly strict test-suites. The last iteration of this was started at http://lists.debian.org/debian-devel/2012/10/msg00296.html I am sorry, if I was not clear. I am aware of the last iteration, but I am not enquiring about the default policy within debian as to how we should upload by default. I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? If it's a policy enforcement, I am ok with it. Otherwise, I'd would like to see dak accept those. I have a vague recollection of a UDD presentations which did list count of DDs doing source-only uploads. Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhlujhme6yf+xsjorlkj_xtf62s6xehew8f+zehojpktk...@mail.gmail.com
Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Tue, 20 Nov 2012, Dmitrijs Ledkovs wrote: I am sorry, if I was not clear. I am aware of the last iteration, but I am not enquiring about the default policy within debian as to how we should upload by default. I am asking why, when I had a reason to do so, was not able to do a source-only upload. Is this a feature of dak, or a policy enforcement? Both. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121120122309.gb22...@khazad-dum.debian.net
Re: Source-only uploads (was: procenv_0.9-1_source.changes REJECTED)
On Tue, Nov 20, 2012 at 12:08:13PM +, Dmitrijs Ledkovs wrote: If it's a policy enforcement, I am ok with it. Otherwise, I'd would like to see dak accept those. I have a vague recollection of a UDD presentations which did list count of DDs doing source-only uploads. source+all uploads probably? Also, I'm subscribed. -- WBR, wRAR signature.asc Description: Digital signature
Re: throw away debs and source only uploads
Hi, Chipping in late... sorry for that. 2011/6/7 Don Armstrong d...@debian.org: On Tue, 07 Jun 2011, Andreas Barth wrote: Non-buildd binaries should still be allowed, but they should be followed immediately by a binNMU. [Are there any cases where we wouldn't want to rebuild the package after it was bootstrapped?] There are cases where porters need to upload, because of funny issues. Or hand-builds within sane buildd chroots where the buildd software refuses. Or similar. (I think I did less than 10 such uploads recently.) Ok. Am I correct that these odd cases are bugs which should be fixed? If so, it seems reasonable to queue a binNMU, and then file bugs appropriately if it failed. FWIW, there are corner cases, besides cyclic build dependencies were a bootstrap is needed. Some emulators might need some ROM code compiled on one architecture to be run on another architecture. I have also attended some requests where people was wanting to access porter boxes to hand-build packages, IIRC, mlton compiler was one of such cases. Having said that I believe our build daemon infrastructure is quite good, but still there are several corner cases where manual uploads are needed and there is nothing bad about it, even policy is not against that. If ever consider source-only uploads, please make an exception to the rule to allow corner cases like the ones discussed, some to be followed by binNMU, some others not to be followed by binNMU. Best regards, -- Héctor Orón -.. . -... .. .- -. -.. . ...- . .-.. --- .--. . .-. free spam -- Would you like to make a donation for Debian Conference? ** http://debconf11.debconf.org/payments.xhtml ** /free spam -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTikafaw-Br0B4M=_n+t7vevjtrk...@mail.gmail.com
Re: throw away debs and source only uploads
* Don Armstrong (d...@debian.org) [110607 04:25]: On Mon, 06 Jun 2011, Philipp Kern wrote: I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Non-buildd binaries should still be allowed, but they should be followed immediately by a binNMU. [Are there any cases where we wouldn't want to rebuild the package after it was bootstrapped?] There are cases where porters need to upload, because of funny issues. Or hand-builds within sane buildd chroots where the buildd software refuses. Or similar. (I think I did less than 10 such uploads recently.) Andi -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110607074538.gu15...@mails.so.argh.org
Re: throw away debs and source only uploads
On Tue, 07 Jun 2011, Andreas Barth wrote: * Don Armstrong (d...@debian.org) [110607 04:25]: On Mon, 06 Jun 2011, Philipp Kern wrote: I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Non-buildd binaries should still be allowed, but they should be followed immediately by a binNMU. [Are there any cases where we wouldn't want to rebuild the package after it was bootstrapped?] There are cases where porters need to upload, because of funny issues. Or hand-builds within sane buildd chroots where the buildd software refuses. Or similar. (I think I did less than 10 such uploads recently.) Ok. Am I correct that these odd cases are bugs which should be fixed? If so, it seems reasonable to queue a binNMU, and then file bugs appropriately if it failed. Don Armstrong -- [T]he question of whether Machines Can Think, [...] is about as relevant as the question of whether Submarines Can Swim. -- Edsger W. Dijkstra The threats to computing science http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110607155947.gv4...@rzlab.ucr.edu
Re: throw away debs and source only uploads
* Don Armstrong (d...@debian.org) [110607 18:11]: On Tue, 07 Jun 2011, Andreas Barth wrote: * Don Armstrong (d...@debian.org) [110607 04:25]: On Mon, 06 Jun 2011, Philipp Kern wrote: I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Non-buildd binaries should still be allowed, but they should be followed immediately by a binNMU. [Are there any cases where we wouldn't want to rebuild the package after it was bootstrapped?] There are cases where porters need to upload, because of funny issues. Or hand-builds within sane buildd chroots where the buildd software refuses. Or similar. (I think I did less than 10 such uploads recently.) Ok. Am I correct that these odd cases are bugs which should be fixed? Usually yes. If so, it seems reasonable to queue a binNMU, and then file bugs appropriately if it failed. It's reasonable to queue a binNMU, but it's not to assume that it's successful. Or that there might be transient issues, e.g. a hand-build to just complete the transition to testing, and the next source version is uploaded directly after the transition and built normally. Or issues, where we don't need to wait for the binNMU to fail, but just directly file the bug - of course I'm happy to fail the build by hand in such cases. As said, all that are exceptions to the rule. Andi -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110607165245.gh2...@mails.so.argh.org
Re: throw away debs and source only uploads
Hi! On Sun, 2011-04-17 at 11:20:07 +0200, Stefano Zacchiroli wrote: On Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli wrote: The main decision which needs to be made is whether, as a project, we want source only uploads or to throw away DD built non-all debs. There's not entire agreement amongst the ftpmasters about this (I err on the side of the source-only uploads to avoid making people waste time and bandwidth but that's not the only opinion). snip Once a decision is made, implementation is easy, but I'd like some form of consensus before we go down that route. Further round of update on this one, after some more discussion on list and a brief chat of mine with Mark. - There seems to be consensus to go ahead with throw-away debs; they require a bit of work though so either be patient or, better, volunteer with FTP masters to help out with the implementation of the remaining bits. I think this was mentioned in some previous incarnation of this discussion, but throwing away debs unconditionally, or at least w/o having a way to specify they must not be thrown away is going to be an issue when bootstrapping packages. Those cases where we have a cyclic build dependency chain, which is not uncommon: 1) Some compilers written in the same language they target. 2) Build tools: pkg-config Build-Depends on libglib2.0-dev, libglib2.0-dev Build-Depends on pkg-config. pkg-config used to bundle an ancient glib 1.x to be able to automatically bootstrap but that was removed with some recent upstream release. 3) IDL generators: mig Build-Depends on gnumach-dev, gnumach Build-Depends on mig. 4) ... Having to request ftp-masters each time one of these is first bootstrapped anew on an architecture is going to be annoying, both for the porter/packager and for the ftp-masters. regards, guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110606081601.gc16...@gaara.hadrons.org
Re: throw away debs and source only uploads
On 2011-06-06, Guillem Jover guil...@debian.org wrote: - There seems to be consensus to go ahead with throw-away debs; they require a bit of work though so either be patient or, better, volunteer with FTP masters to help out with the implementation of the remaining bits. I think this was mentioned in some previous incarnation of this discussion, but throwing away debs unconditionally, or at least w/o having a way to specify they must not be thrown away is going to be an issue when bootstrapping packages. Those cases where we have a cyclic build dependency chain, which is not uncommon: You should still be able to upload binaries. So two uploads, one with the source and one with the binaries only should still let them be accepted. I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrniup5u4.l9s.tr...@kelgar.0x539.de
Re: throw away debs and source only uploads
On Mon, 2011-06-06 at 09:03:00 +, Philipp Kern wrote: On 2011-06-06, Guillem Jover guil...@debian.org wrote: I think this was mentioned in some previous incarnation of this discussion, but throwing away debs unconditionally, or at least w/o having a way to specify they must not be thrown away is going to be an issue when bootstrapping packages. Those cases where we have a cyclic build dependency chain, which is not uncommon: You should still be able to upload binaries. So two uploads, one with the source and one with the binaries only should still let them be accepted. Ah right, and while still slightly cumbersome, it would at least not be as annoying as needing to prod people around, etc. I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Yes, please. thanks, guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110606162543.ga32...@gaara.hadrons.org
Re: throw away debs and source only uploads
On 06/06/2011 10:16 AM, Guillem Jover wrote: Hi! On Sun, 2011-04-17 at 11:20:07 +0200, Stefano Zacchiroli wrote: On Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli wrote: The main decision which needs to be made is whether, as a project, we want source only uploads or to throw away DD built non-all debs. There's not entire agreement amongst the ftpmasters about this (I err on the side of the source-only uploads to avoid making people waste time and bandwidth but that's not the only opinion). snip Once a decision is made, implementation is easy, but I'd like some form of consensus before we go down that route. Further round of update on this one, after some more discussion on list and a brief chat of mine with Mark. - There seems to be consensus to go ahead with throw-away debs; they require a bit of work though so either be patient or, better, volunteer with FTP masters to help out with the implementation of the remaining bits. I think this was mentioned in some previous incarnation of this discussion, but throwing away debs unconditionally, or at least w/o having a way to specify they must not be thrown away is going to be an issue when bootstrapping packages. Those cases where we have a cyclic build dependency chain, which is not uncommon: 1) Some compilers written in the same language they target. 2) Build tools: pkg-config Build-Depends on libglib2.0-dev, libglib2.0-dev Build-Depends on pkg-config. pkg-config used to bundle an ancient glib 1.x to be able to automatically bootstrap but that was removed with some recent upstream release. 3) IDL generators: mig Build-Depends on gnumach-dev, gnumach Build-Depends on mig. 4) ... Having to request ftp-masters each time one of these is first bootstrapped anew on an architecture is going to be annoying, both for the porter/packager and for the ftp-masters. Are you saying they cannot be bootstrapped with older versions (which are already in the archive)??! Cheers Luk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ded107b.3050...@debian.org
Re: throw away debs and source only uploads
On Mon, 2011-06-06 at 19:38:03 +0200, Luk Claes wrote: Are you saying they cannot be bootstrapped with older versions (which are already in the archive)??! By definition if they need to be manually bootstrapped it's because their build dependencies are not available. The usual cases for that are a set of NEW packages, existing packages for new architectures, or for existing architectures where those packages have never built before. Take mig on kfreebsd-i386 as an example. To build it we'd first need to unpack gnumach, manually run “debian/rules build/config.status” and “make -C build install-data” to just install the headers. Then unpack mig, remove gnumach-dev from the Build-Depends, build and install the new mig.deb. Now we can build a clean gnumach, and install the resulting gnumach-dev. And finally just to make sure, we rebuild a clean mig (and possibly a cleaner gnumach with the clean mig). We've just bootstrapped mig ang gnumach on kfreebsd-i386. regards, guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110607015208.ga13...@gaara.hadrons.org
Re: throw away debs and source only uploads
On Mon, 06 Jun 2011, Philipp Kern wrote: I.e. I think we should still allow non-buildd binaries, e.g. for those cases you mentioned. Non-buildd binaries should still be allowed, but they should be followed immediately by a binNMU. [Are there any cases where we wouldn't want to rebuild the package after it was bootstrapped?] Don Armstrong -- Identical parts aren't. -- Beach's Law http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110607022454.gu4...@rzlab.ucr.edu
Re: throw away debs and source only uploads
On Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli wrote:
The main decision which needs to be made is whether, as a project, we
want source only uploads or to throw away DD built non-all debs.
There's not entire agreement amongst the ftpmasters about this (I err
on the side of the source-only uploads to avoid making people waste
time and bandwidth but that's not the only opinion).
snip
Once a decision is made, implementation is easy, but I'd like some
form of consensus before we go down that route.
Further round of update on this one, after some more discussion on list
and a brief chat of mine with Mark.
- There seems to be consensus to go ahead with throw-away debs; they
require a bit of work though so either be patient or, better,
volunteer with FTP masters to help out with the implementation of the
remaining bits.
- There seems to be consensus also on going ahead with source-only
uploads, as long as they are not the default. The override method to
enable them is still undecided though. A possibility is to have an
explicit extra field in .changes file to state yes, I'm sure I want
to do a source-only upload; another is a simple override at dput
upload time. FTP master will discuss the various possibilities and let
us now.
Either way, we will need to document in devref that the recommended
way is to do binary-ful uploads and how to override if needed. (A bug
report is in order, but it's pointless to submit it before we have
further technical details.)
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Quando anche i santi ti voltano le spalle, | . |. I've fans everywhere
ti resta John Fante -- V. Capossela ...| ..: |.. -- C. Adams
signature.asc
Description: Digital signature
Re: throw away debs and source only uploads
On Thu, Apr 07, 2011 at 04:55:12PM +0200, Stefano Zacchiroli wrote: - going ahead with throw away debs seems to be largely uncontroversial; can we haz zem please? :-) Will that throw away Arch: all packages as well? If there are no technical issues/implementation missing with this (somebody mentioned that), I would suggest doing that as well, for consistency (having all build logs on buildd.d.o, e.g.), and because we've seen breakage due to that from time to time. Michael -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110417103406.ga31...@nighthawk.chemicalconnection.dyndns.org
Re: throw away debs and source only uploads
Hi, Am Sonntag, den 17.04.2011, 11:20 +0200 schrieb Stefano Zacchiroli: - There seems to be consensus to go ahead with throw-away debs; they require a bit of work though so either be patient or, better, volunteer with FTP masters to help out with the implementation of the remaining bits. will this infrastructure then also be able to do archAllBinNMUs, for cases when problems in arch:all packages can be fixed by simply rebuilding without source changes? That would be nice to have. Although it would come almost for free with source-only-uploads, as then one can mechanically fetch the latest source, add a changelog entry and upload the otherwise unmodified source. But still, something even more automatic is cleaner and less error prone. In any case, I am looking forward to these changes, and kudos in advance to anyone implementing them. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: throw away debs and source only uploads
[ Bcc:-ing ftpmasters ]
Time to wrap up the current state of this discussion, at least as far as
I see it.
- going ahead with throw away debs seems to be largely uncontroversial;
can we haz zem please? :-)
- there seems to be no substantial objections either on the fact the
source only upload would be fine, as long as they are not the default
but can be enabled upon request
What's still largely undecided is how uploaders would require a source
only upload. Several presented use cases --- both here on list and in
private replies to me --- show that my proposal, as well as all possible
source-based solutions, are suboptimal. In other words, people seem to
really need per-upload, or even per-batch upload, white listing.
FWIW, I wouldn't like much the idea of introducing yet another list of
people which are allowed to do source only uploads as that would be a
potential bottleneck which seems unwarranted here (at least if you buy
my argument that for what concerns the risk of non-buildable-uploads,
the defaults matter more than strict controls).
How about just relying on dpkg-buildpackage -S, maybe coupling it with
the need of using dput -f (extending a bit the current semantics of
-f) which would refuse to upload a source only binary by default?
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Quando anche i santi ti voltano le spalle, | . |. I've fans everywhere
ti resta John Fante -- V. Capossela ...| ..: |.. -- C. Adams
signature.asc
Description: Digital signature
Re: throw away debs and source only uploads
Lars Wirzenius writes (Re: throw away debs and source only uploads): Most uploads are done using dput or dupload. We could add code to them to verify that there's binaries corresponding to the source that is about to be built. We could have the archive scripts insist that the .debs have to exist in the .changes file, but not to check that the files are actually uploaded or to accept empty files. If we want to check the manifest we could require that the .changes file list manifests (output of dpkg-deb --contents, or something) for any missing .debs. Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/19860.31129.223881.836...@chiark.greenend.org.uk
Re: throw away debs and source only uploads
Hi, Am Mittwoch, den 30.03.2011, 16:18 +0200 schrieb Stefano Zacchiroli: The main use case I've seen mentioned on list to favor source only uploads over throw away debs is that of low bandwidth or bandwidth limits. Most likely, that use case applies to very few people and the vast majority of uploads could happen without suffering of those problems. I have another use case: Uploading packages with little (typos, changes in debian/control) or no (aka rebuild triggering) changes, but applying such a change to 200 packages – something that comes up with the Haskell packaging team often. Often I am confident that my change is correct even without manual testing. Nevertheless I have to fire up a pbuilder, build several dozen packages in the right order (something that wanna-build is much better in figuring out), some of which build fast and some take quite some time. A lot of work goes into useless manual labor here despite the fact that we (almost) have the infrastructure to automate that. I, for one, welcome our new alien ant^W^W source only uploads. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
throw away debs and source only uploads
On Mon, Mar 28, 2011 at 03:37:05PM +0100, Mark Hymers wrote: Ok, the situation here is the following: Thanks a lot for taking the time of clarifying. The main decision which needs to be made is whether, as a project, we want source only uploads or to throw away DD built non-all debs. There's not entire agreement amongst the ftpmasters about this (I err on the side of the source-only uploads to avoid making people waste time and bandwidth but that's not the only opinion). snip Once a decision is made, implementation is easy, but I'd like some form of consensus before we go down that route. So, let me see if I can help out in shaping the discussion around this. First of all my gut feeling is the same as Don's [1]. I've the impression that throw away debs is a rather uncontroversial step to take and that we could take independently from source only uploads. [1] http://lists.debian.org/debian-devel/2011/03/msg01063.html Clearly that would not solve some of the issues that source only uploads would solve, most notably the wasted bandwidth issue. At the same time: 1) it won't be worse than the status quo in that respect either, and 2) would be better than the status quo in terms of package uniformity wrt their build environments. I saw only one potential drawback in going forward with throw away debs, namely the risk of doing some job to implement them and them throw it away the day, potentially near, we further switch to source only uploads. My interpretation of your mail is that this risk is pretty low, as the involved work is low as well. (Yes, I've also seen the various interesting proposals of comparing metadata of uploaded packages against those of rebuilt packages and I understand that implementing those would require significantly more work. But none of those proposals seem to be *requirements* to go ahead.) If my gut feeling will be shown to be wrong on the consensus around throw away debs (with evidence coming as angry replies to this mail), I propose to have a poll on throw away debs. Bottom line #1: I propose to go ahead and deploy throw away debs, as soon as technically feasible. One objection to source-only is the perception that people will throw untested things at the buildds and see what sticks. That strikes me as a social problem, but as a project we probably want to think how to handle it. Regarding source only upload, well, it's tricky. There is the usual tension about the principle desire of trusting every DD to do the right thing and the reality-check observation that enabling people to upload only sources *will* mean that some people will upload packages without having even built them once; let's call this latter problem developer sloppiness. (Shameless [academic] plug: developer sloppiness is fairly common all over computer science. That's part of the reason why we have developed over time programming languages which are more and more strict in *forcing* developers to do the right™ thing, at least by default.) The main use case I've seen mentioned on list to favor source only uploads over throw away debs is that of low bandwidth or bandwidth limits. Most likely, that use case applies to very few people and the vast majority of uploads could happen without suffering of those problems. To address developer sloppiness, sane defaults could be enough. If this is the case, a solution that might work is a scheme in which source only uploads are disallowed by default, but at the same time can be enabled if the uploader really needs to. If the needed explicit step to have a source only upload gets through is costly enough, sloppy developers won't implement it (after all they are sloppy, aren't they?) and we should be able to avoid a good deal of gratuitous additional FTBFS. An implementation which might cut the deal can rely on lintian. We might for instance have a lintian error which is triggered by a changes file missing .deb files and have such an error be valid ground for automatic rejection by dak. A maintainer who really needs to do a source only upload for that package will simply have to add the proper override. Bonus features: maintainers which rely too much on source only uploads (assuming that can be defined...) can be easily spotted as the overrides are in the archive. Drawback of this solution: overrides will be per package and will have the tendency of stick around packages; that should not be a big deal either, assuming the current defaults and recommended practices of dpkg-buildpackage friend will still be about building binary packages by default. The above is just an idea, little more than a brain-dump, for finding a compromise among the real needs of people with bandwidth problem and the social issues revolving around developer sloppiness. Once more: it's material for discussion which IMHO should not be in the way of the implementation of the throw away debs scheme. Cheers. -- Stefano Zacchiroli -o- PhD in Computer Science
Re: throw away debs and source only uploads
On Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli wrote:
[...]
The above is just an idea, little more than a brain-dump, for
finding a compromise among the real needs of people with bandwidth
problem and the social issues revolving around developer
sloppiness.
[...]
I expect part of the concern on both sides simply resolves around
conservation, doing the right thing as a project and not wasting
Internet/computing resources (I know bandwidth is much more of a
commodity these days, but lots of people are still instilled with
the careful principles of yesteryear).
One possible compromise which I think was already mentioned in one
of the earlier discussions (but now I can't find a reference) was to
initially attempt builds of source-only uploads on one arch and
delay building on the rest until it was proven not to FTBFS. This
strikes a balance between wasting buildd resources and not assuming
devs are irresponsibly sloppy (disclaimer: this was not my
impression of course, and IANADD anyway). It probably increases
latency for package entry into the archive, but any uploader
concerned about that can simply do a normal upload including their
locally-built binary packages to vouch for buildablity of the
source.
--
{ IRL(Jeremy_Stanley); WWW(http://fungi.yuggoth.org/); PGP(43495829);
WHOIS(STANL3-ARIN); SMTP(fu...@yuggoth.org); FINGER(fu...@yuggoth.org);
MUD(kin...@katarsis.mudpy.org:6669); IRC(fu...@irc.yuggoth.org#ccl);
ICQ(114362511); YAHOO(crawlingchaoslabs); AIM(dreadazathoth); }
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110330155231.go1...@yuggoth.org
Re: throw away debs and source only uploads
On Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli wrote: [...] Regarding source only upload, well, it's tricky. There is the usual tension about the principle desire of trusting every DD to do the right thing and the reality-check observation that enabling people to upload only sources *will* mean that some people will upload packages without having even built them once; let's call this latter problem developer sloppiness. (Shameless [academic] plug: developer sloppiness is fairly common all over computer science. That's part of the reason why we have developed over time programming languages which are more and more strict in *forcing* developers to do the right™ thing, at least by default.) [...] Someone (I forget who) previously suggested that a source-only changes file should have to be accompanied by a build log. This would need a bit of infrastructure to file the build log away. Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110330163345.gu2...@decadent.org.uk
Re: throw away debs and source only uploads
On ke, 2011-03-30 at 17:33 +0100, Ben Hutchings wrote: Someone (I forget who) previously suggested that a source-only changes file should have to be accompanied by a build log. This would need a bit of infrastructure to file the build log away. Most uploads are done using dput or dupload. We could add code to them to verify that there's binaries corresponding to the source that is about to be built. -- Blog/wiki/website hosting with ikiwiki (free for free software): http://www.branchable.com/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1301503405.11500.13.ca...@havelock.liw.fi
Re: throw away debs and source only uploads
I definitely agree we want to throw away developer-built debs (arch all arch any) in almost all situations. I don't think I would want the lintian solution for source-only uploads, I would prefer something on a per-upload basis that requires an explicit human decision per-upload rather than something that can be scripted away. It is also important to have an audit trail for this. Maybe a mail bot on ftp-master that a developer needs to mail before the upload will be accepted. The overrides could be published on the ftp-master website and replies to them be CCed to -devel or another list. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/AANLkTinjBCS0eR+oiTXVbiTG_fFt2RAWb+07QP=t=g...@mail.gmail.com
Re: throw away debs and source only uploads
Paul Wise p...@debian.org writes: I definitely agree we want to throw away developer-built debs (arch all arch any) in almost all situations. I don't think I would want the lintian solution for source-only uploads, I would prefer something on a per-upload basis that requires an explicit human decision per-upload rather than something that can be scripted away. It is also important to have an audit trail for this. We could add another queue similar to the DELAYED queues, maybe. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/877hbgkxnc@windlord.stanford.edu
Re: throw away debs and source only uploads
Le Wed, Mar 30, 2011 at 04:18:34PM +0200, Stefano Zacchiroli a écrit : Regarding source only upload, well, it's tricky. There is the usual tension about the principle desire of trusting every DD to do the right thing and the reality-check observation that enabling people to upload only sources *will* mean that some people will upload packages without having even built them once Hi Stefano, I think that one important factor is how often such errors will happen. We can imagine all sorts of scenarios and devise counter-mesures against them, but are they all worth the effort, and worth the damage as well ? Because it is always frustrating to read top-down comments about simple Debian developers to be sloppy and untrustable. Let's not make it a common assumption. In what I have seen in the packaging teams that I follow, often when a package fails to build on all architectures, it is because the developer has not tested them in a minimal build environment. Making sure that binary packages have been built together with the source package is not solving that problem. On the binary side as well, what the maintainer can do to test the packages by hand is also limited, not to mention that testing more than one architecture is time consuming (so I usually never do). Build-time regression tests and facilities to let users run the same tests on the binary packages they downloaded (DEP8 ?) will altogether do much more for the quality of Debian than using the presence of binary packages accompaniying the source upload as an evidence of significant qualitative difference compared to a source-only upload. Asking developers to publish their build logs is far more interesting, and in the short term, does not require any infrastructure change. Currently, I store my build logs in the git repositories where my source packages are managed, and in the case of subversion packages, I send the logs on the maintainer mailing list. If the uploaded package turns out to be problematic, we have a starting point for troubleshooting. And when developers repeatedly commit the same mistake, refuse to realise the damage they do, and persist in ignoring the solution to the problems they cause, let'se withdraw the trust we gave them. But does that happen that often ? My experience is that people learn and improve their skills, not the reverse. In that sense, occasional failures to build, while not a goal by themselves, are an inevitable byproduct of increasing our workforce. Automated reporting of build failures can circumvent the nuisance to the package maintainers themselves. Cheers, -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110331001718.ga...@merveille.plessy.net
