On Thu, Jan 17, 2013 at 09:34:16AM +0100, Johannes Schauer wrote: > Hi, > > On Thu, Jan 17, 2013 at 08:33:57AM +0100, Wouter Verhelst wrote: > > If wanna-build is updated to support these two fields, then I imagine > > it can run the bootstrapping dependency algorithm. While you wouldn't > > want to upload a package to the debian.org archive when the > > architecture is as yet incomplete, the same isn't true for the > > debian-ports.org archive. > > > > It would require some patches for wanna-build to understand that package > > "foo" would need to be rebuilt once a particular profile is fully built > > and available, but I don't think that's impossible. > > I'm not sure if wanna-build is the right tool to do this
Why not? It already needs to do build-dependency tracking, marking packages as "can't be built yet because build-depends aren't there yet" all the time. That's exactly the sort of thing you need to be doing, no? Of couse, it's entirely possible that I'm missing something obvious here -- you're the one who spent half a year figuring out what needs to be done, not me ;-) > but we also did not yet implement anything which would require a > specific tool for the build process. > > The output of the current version of the algorithm produces is an ASCII, > comma separated list of source packages. All packages which can be built > in parallel are given on the same line. Source packages that need to be > built with reduced build dependencies are annotated as such. The > rebuilding of profile built packages is part of this output and > triggered after all source packages of a strongly connected component > finished building. I see. > After all profile built packages successfully rebuilt > fully, *all* source packages of the strongly connected component are > rebuilt. This is to make sure that every source package was built once > with all build dependencies available. If you have enough information on what a package needs to build properly, you wouldn't need to do this. But I agree that's a pretty big "if". > This list gave me so far an idea of the build order and a way to check > whether it made sense. By coming up with a more machine readable version > of this output, a tool can be triggered/controlled which then gives the > actual build jobs and does things like you mentioned above. Right. > > However, to do that, there's one thing I'm missing in your mail: there > > will be cases where packages, when built in a particular profile do > > not support some functionality. That is, the package is available and > > does most of what the full package does, but because some > > build-dependency is missing, it doesn't support some feature or other > > -- for example, let's say that a document translation package (which > > we'll call "foo") which supports many input formats does not support > > XML as an input format when built in the stage1 profile. The output of > > its stage1 build would not change the number of binary packages built > > with the source package, it would just build a binary package with > > reduced functionality. > > > > Now it might be that a package build-depends on our package foo because > > it needs to translate documents in that XML format into something else, > > With your proposal, there's no way for the build-depending package to > > declare that it needs a version of foo that was built at a richer > > profile than stage1. > > > > Is this something you've considered? > > Glad you bring this up because we indeed forgot to mention this in our > initial email. Yes, what you mention can indeed happen and this is also > why extensive rebuilding is part of the build order: to make sure that > all features are always compiled into the binaries. > > We have thought about this issue but did not manage to come up with a > global solution. A solution would require a way to depend on "features" > of a package and therefor create an overcomplicated dependency syntax > which would not justify its purpose. I'm not sure I agree with that. Your build profiles are essentially creating additional "variants" of a particular binary package. I imagine we should be able to come up with a syntax for the build-depends header which would allow one to declare you depend on a particular variant of a binary package. Let's say something like the following (extending the syntax as suggested by Ian Jackson earlier): Build-Depends: foo (profile:stage1 >= 1.3) [profile:stage1], foo (>= 1.3) [!profile:stage1] Essentially, with this syntax I'm saying that to build a the package which contains this build-dependency header, under normal circumstances you'd need the package "foo" at version 1.3 or above, as built with no profiles enabled; but if you enable the "stage1" profile for this package, you can use the "stage1" version of the "foo" package instead. I don't think this is too confusing or complicated, and I think it makes sense to add the profile to the version information in this manner; a () qualifier for a package could then be interpreted as a statement of *what* a package needs to comply with (version or profile of the build-dependency), whereas a [] qualifier would state *when* we need a particular build-dependency (architecture or profile of our *current* build). > There are two things that a developer could do to avoid such FTBFS > errors during the bootstrap process when he modifies a source package > with reduced "stage1" build dependencies: > > - only remove build dependencies which do not modify the binary > packages which are produced (instead, let some binary packages not be > produced, like foo-doc, foo-java or foo-gtk) > > - look at the reverse dependencies in the generated dependency graph to > make sure that features which are changed in built binary packages > are not depended upon (naturally, those reverse dependencies will > change over time...) > > Both solutions are very fragile. This is also why the developed > algorithms try to build as few source packages as possible with reduced > build dependencies. That makes sense, I suppose. > Do we have some experience of how common such errors are during a manual > bootstrap? Not me, I'm afraid. -- Copyshops should do vouchers. So that next time some bureaucracy requires you to mail a form in triplicate, you can mail it just once, add a voucher, and save on postage. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130117205132.gg14...@grep.be