Providing documentation for developers (in the website) related to the buildd network
Hi everyone, After trying to build my own autobuilder machine at home [1] I've found that there's not that much documentation on how the buildd network works, and it's quite spread out (in different locations, including mailing lists and people.debian.org directories) I'm considering uploading the attached content to the webwml sources (i.e. the Debian website) at w.d.o/debian/buildd/. This information tries to complement the information available elsewhere (i.e. buildd.debian.org, the Developer's Reference) and aggregate some of the information produced by developers (people.debian.org stuff). That's why some of you are in the mail so you can review and approve the reuse of your content. I would appreciate some comments on the content attached (html pages provided for convenience, local hyperlinks will not work due to how the website builds for content negotiation). If no one objects I will upload this to the website shortly. The only reason I'm doing this is because I think there's not enough developer-oriented information on how buildd works (both the software and the network) and I would rather have this in a single place than spread out in multiple locations. Also, it would be nice to write some additional information on both where to obtain the software (buildd was previously available at db.debian.org, but it seems to not be there any longer) and how to setup an autobuilder system from scratch. That information could reside here or, maybe, in the Developer's Reference. Thanks for your help Javier [1] To try and recompile all the Debian packages both to: - recompile it with the Gcc+SSP patch and see if we can integrate whatever Adamantix has done back into Debian through a new 'architecture' - automatically audit the source code of built packages (source code audit of unbuilt packages breaks in some cases because of extensive usage of dpatch) buildd.tgz Description: GNU Unix tar archive signature.asc Description: Digital signature
Re: Providing documentation for developers (in the website) related to the buildd network
On Mon, Nov 08, 2004 at 10:59:19AM +0100, Javier Fernández-Sanguino Peña wrote: Hi everyone, After trying to build my own autobuilder machine at home [1] I've found that there's not that much documentation on how the buildd network works, and it's quite spread out (in different locations, including mailing lists and people.debian.org directories) Indeed. I'm considering uploading the attached content to the webwml sources (i.e. the Debian website) at w.d.o/debian/buildd/. As most of this content is developer-oriented, shouldn't this be /devel/buildd/ ? This information tries to complement the information available elsewhere (i.e. buildd.debian.org, the Developer's Reference) and aggregate some of the information produced by developers (people.debian.org stuff). That's why some of you are in the mail so you can review and approve the reuse of your content. I would appreciate some comments on the content attached (html pages provided for convenience, local hyperlinks will not work due to how the website builds for content negotiation). If no one objects I will upload this to the website shortly. I have no problem with my content being reused for www.debian.org; however, in that case I would appreciate it if I could maintain it there (i.e., if someone could give me CVS access). However, one little correction: --- index.wml.orig 2004-11-08 11:46:02.670732286 +0100 +++ index.wml 2004-11-08 11:47:55.032348605 +0100 @@ -57,7 +57,8 @@ for all architectures (in state emNeeds-Build/em). Build machines will query the build database for packages in this state, and will routinely take packages from that list. The list is -is prioritized by urgency and number of packages build-depending on a package. +is prioritized by previous compilation state, priority, section, and finally +package name. PIf the build succeeds in all architectures, the maintainer will not need to do anything. All those binary packages will be uploaded to the The idea that this list would be prioritized by urgency is a myth which was sent into the world by my people.d.o page; however, it is incorrect. The only reason I'm doing this is because I think there's not enough developer-oriented information on how buildd works (both the software and the network) and I would rather have this in a single place than spread out in multiple locations. I couldn't agree more. In fact, I wanted to do this a while ago, but never got around to actually do it. Thanks for taking this up. Also, it would be nice to write some additional information on both where to obtain the software (buildd was previously available at db.debian.org, but it seems to not be there any longer) No, there simply is no source package, and it is therefore not referenced in the Sources file available on db.debian.org. However, if you check the version number of the package in the Packages file, and check where the Sources file has its base URL, you can download the source tarball. I agree that it would be better if the source would be readily downloadable, but that is currently not the case. -- EARTH smog | bricks AIR -- mud -- FIRE soda water | tequila WATER -- with thanks to fortune
Re: Providing documentation for developers (in the website) related to the buildd network
On Mon, Nov 08, 2004 at 11:54:15AM +0100, Wouter Verhelst wrote: I'm considering uploading the attached content to the webwml sources (i.e. the Debian website) at w.d.o/debian/buildd/. As most of this content is developer-oriented, shouldn't this be /devel/buildd/ ? A. Correct, it's devel/buildd (and that's where I have it in my local sources, the previous reference was a typo) I have no problem with my content being reused for www.debian.org; however, in that case I would appreciate it if I could maintain it there (i.e., if someone could give me CVS access). I cannot provide that access, but I'm sure the www maintainers would gladly give it to you once this has been setup. However, one little correction: (.. patch ..) Fixed, thanks. However, if you check the version number of the package in the Packages file, and check where the Sources file has its base URL, you can download the source tarball. I can't find the package in the sid Packages file, is it 'buildd'? A previous mail from you [1] points to http://db.debian.org/debian-admin/ but that location is not available any more (I believe you are already aware of that) I agree that it would be better if the source would be readily downloadable, but that is currently not the case. Yes, most unfortunate. I've been able to find a buildd package, however, at http://people.debian.org/~rleigh/buildd/ I wonder if this one is recommended/used? Regards Javier [1] http://lists.debian.org/debian-devel/2004/06/msg007006.html signature.asc Description: Digital signature
Re: Providing documentation for developers (in the website) related to the buildd network
On Mon, Nov 08, 2004 at 01:47:29PM +0100, Javier Fernández-Sanguino Peña wrote: On Mon, Nov 08, 2004 at 11:54:15AM +0100, Wouter Verhelst wrote: I'm considering uploading the attached content to the webwml sources (i.e. the Debian website) at w.d.o/debian/buildd/. As most of this content is developer-oriented, shouldn't this be /devel/buildd/ ? A. Correct, it's devel/buildd (and that's where I have it in my local sources, the previous reference was a typo) Okay :) I have no problem with my content being reused for www.debian.org; however, in that case I would appreciate it if I could maintain it there (i.e., if someone could give me CVS access). I cannot provide that access, but I'm sure the www maintainers would gladly give it to you once this has been setup. Whom do I ask? [...] However, if you check the version number of the package in the Packages file, and check where the Sources file has its base URL, you can download the source tarball. I can't find the package in the sid Packages file, is it 'buildd'? In the db.debian.org packages file, not the sid one. wget http://db.debian.org/debian-admin/Packages look for 'buildd' in Packages, and see what the version number is. construct a file name 'wanna-build_' plus the version number of buildd as found in Packages. wget http://db.debian.org/debian-admin/wanna-build_$version.tgz A previous mail from you [1] points to http://db.debian.org/debian-admin/ but that location is not available any more (I believe you are already aware of that) I agree that it would be better if the source would be readily downloadable, but that is currently not the case. Yes, most unfortunate. I've been able to find a buildd package, however, at http://people.debian.org/~rleigh/buildd/ I wonder if this one is recommended/used? Neither am I. [1] http://lists.debian.org/debian-devel/2004/06/msg007006.html That link renders a 404. -- EARTH smog | bricks AIR -- mud -- FIRE soda water | tequila WATER -- with thanks to fortune
Re: Providing documentation for developers (in the website) related to the buildd network
Can you please NOT use dyndns for my email? On Mon, Nov 08, 2004 at 01:47:29PM +0100, Javier Fernández-Sanguino Peña wrote: Yes, most unfortunate. I've been able to find a buildd package, however, at http://people.debian.org/~rleigh/buildd/ [EMAIL PROTECTED]:~wajig policy buildd buildd: Installed: 2004.09.06 Candidate: 2004.09.06 Version Table: *** 2004.09.06 0 500 http://db.debian.org debian-admin/ Packages 100 /var/lib/dpkg/status The question is, is that public information? I guess it is: http://lists.debian.org/debian-devel/2004/08/msg00389.html Christian
Re: Providing documentation for developers (in the website) related to the buildd network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Wouter Verhelst wrote: | On Mon, Nov 08, 2004 at 01:47:29PM +0100, Javier Fernández-Sanguino Peña wrote: | |On Mon, Nov 08, 2004 at 11:54:15AM +0100, Wouter Verhelst wrote: [...] |However, if you check the version number of the package in the Packages |file, and check where the Sources file has its base URL, you can |download the source tarball. | |I can't find the package in the sid Packages file, is it 'buildd'? | | | In the db.debian.org packages file, not the sid one. | | wget http://db.debian.org/debian-admin/Packages | | look for 'buildd' in Packages, and see what the version number is. | construct a file name 'wanna-build_' plus the version number of buildd | as found in Packages. | | wget http://db.debian.org/debian-admin/wanna-build_$version.tgz | | |A previous mail from you [1] points to http://db.debian.org/debian-admin/ |but that location is not available any more (I believe you are already |aware of that) | | |I agree that it would be better if the source would be readily |downloadable, but that is currently not the case. | |Yes, most unfortunate. I've been able to find a buildd package, however, at |http://people.debian.org/~rleigh/buildd/ | |I wonder if this one is recommended/used? | | | Neither am I. | | |[1] http://lists.debian.org/debian-devel/2004/06/msg007006.html | | | That link renders a 404. I think he meant [2] (there was one 0 too many) [2] http://lists.debian.org/debian-devel/2004/06/msg00706.html Cheers Luk -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBj3Wm5UTeB5t8Mo0RAloUAJsGg4OipbPfcpm+8k2plcBXhvkB5gCgpekC 7nRZvpjkujBiOXqPDLhrZfs= =Cwaq -END PGP SIGNATURE-
Re: Providing documentation for developers (in the website) related to the buildd network
On Mon, Nov 08, 2004 at 02:03:48PM +0100, Wouter Verhelst wrote:
I cannot provide that access, but I'm sure the www maintainers would gladly
give it to you once this has been setup.
Whom do I ask?
w.d.o/devel/website/using-cvs points you to
http://people.debian.org/~joey/misc/webwml.html
(I wonder why that is not on the website instead of in p.d.o). Basicly mail
[EMAIL PROTECTED] (Matt Krai and Josip Rodin) and Cc: [EMAIL PROTECTED]
However, if you check the version number of the package in the Packages
file, and check where the Sources file has its base URL, you can
download the source tarball.
I can't find the package in the sid Packages file, is it 'buildd'?
In the db.debian.org packages file, not the sid one.
wget http://db.debian.org/debian-admin/Packages
Ok.
look for 'buildd' in Packages, and see what the version number is.
construct a file name 'wanna-build_' plus the version number of buildd
as found in Packages.
wget http://db.debian.org/debian-admin/wanna-build_$version.tgz
Doesn't work (wget)=:
--16:03:35-- http://db.debian.org/debian-admin/wanna-build_2004.09.06.deb
(...)
16:09:29 ERROR 404: Not Found.
Actually it's
http://db.debian.org/debian-admin/wanna-build_2004.09.06.tar.gz
However the changelog says:
wanna-build (2004.09.06) unstable; urgency=high
* Initial non-release. [*DO NOT USE THESE DEBS*]
-- Ryan Murray [EMAIL PROTECTED] Mon, 6 Sep 2004 03:15:41 -0700
:-?
The sources are _really_ different from those found in the m68k CVS
(from diffstat: 47 files changed, 764 insertions(+), 3505 deletions(-))
Yes, most unfortunate. I've been able to find a buildd package, however, at
http://people.debian.org/~rleigh/buildd/
I wonder if this one is recommended/used?
Neither am I.
They are somewhat different (configuration files are located in different
places) and Roger Leigh's version does provide manual pages for all of the
buildd commands (wanna-build, wanna-build-mail, wanna-build-statistics,
wanna-build-update...). Also, the version at db.debian.org provides some
functionality that is missing in Roger's (like a 'clean-db' call to
wanna-build). However, I'm not sure if the security fix that was recently
introduced was included in those packages... It looks like it was not, is
this it:
- if (!/^(\S+)\s*(\(\s*(|=)\s*(\S+)\s*\))?\s*$/) {
+ if (!/^(\S+)\s*(\(\s*([=]+)\s*(\S+)\s*\))?\s*$/) {
return 0 if $verify;
- warn( parse_deplist: bad dependency $_\n );
+ logger( parse_deplist: bad dependency $_\n );
next;
}
These forks are really messy :-(
Regards.
Javier
Re: Providing documentation for developers (in the website) related to the buildd network
However, I'm not sure if the security fix that was recently introduced was included in those packages... It looks like it was not, is this it: () Scratch that, after reviewing wanna-build-apt's NEW file (dated 2004-09-05) I see that it is probably still vulnerable since buildd-mail in Roger's buildd packages has a CVS id dated 2003/01/09. Regards Javier signature.asc Description: Digital signature
