The way bitbake works today, it can't build *anything* unless it's parsed *everything*, because anything could have PROVIDES / RPROVIDES what you're wanting to build. If you want to shift things to a server, which I agree would be nice, you'd have to have a daemon running there to make available the full dependency information.
On Mon, Jul 26, 2010 at 1:59 PM, C Michael Sundius <[email protected]>wrote: > every time I see a conversation on this list that says something to the > effect of "we have xxx number of recipes that do this" or "if we make this > sort of change we'll have to change yyy number of recipes" I cringe. I ran: > > [msund...@msundius-lnx1 recipes]$ find . -type f | grep "\.bb$" | wc > 8581 8581 276766 > [msund...@msundius-lnx1 recipes]$ ls -1 | wc > 2023 2023 18851 > > 8500 recipes just doesn't seem like a scalable number of recipes to manage > w/ our development community. Its problematic for the following reasons: > > 1) making a change that affects compatibility is difficult if not > impossible > 2) most people working w/ embedded distros want only a small percentage of > this beast > 3) building involves bitbake parsing all of these recipes before going to > do > the tree of what actually needs to be built. > there are probably more complaints from the masses > > It seems to me that we need to decouple in a formal way the recipes from OE > (classes and config files) and from each other. > I am proposing that we do the following: > - that we define exactly what it means to conform to a particular version > of > the OE Recipe API (or whatever you call it). > - that we "fetch" recipes from well known repositories/mirrors (or locally) > in the same way we fetch tarballs. > - we parse recipes as we build the dependency tree. > > That way we don't have to look through the entire list 8500 recipes > (parsing > them all) before we build our tiny little embedded application that > controls > a refrigerator's ice making machine or whatever you make. > > For us, we simply don't use most of all of the recipes and I have found it > hard to manage the disparity of the entire official tree vs the recipes > that > we use. I have simply created my own (sub)tree that is a subset to the > greater tree. > > I also like the idea that we would be able to append our own configuration > data patches etc for selected standard recipes (that could be "fetched" > from > somewhere) so that we would not really have to merge all of those specifics > of our own system into the repository of recipes used by the whole world > (i.e. noone cares about the busybox config file we use or a patch to the > kernel to fix a dumb thing our HW engineers did). somthing akin to the > .bbappend discussion as of late > > I realize that this is off topic here. I also am not qualified to propose > how to implement such a solution. I'm just wondering if that might have > already occurred to any of us that there's a fundamental issue that > detracts > us from creating a truly independent build framework. I'm also not trying > to > be a complainer.. I've been thinking about this for a long time and I'd > like > to help... I'm just not that good w/ python :] > > Back to the discussion at hand, If we have a (versioned) compatibility > standard for recipes then we could simply bump the number and exclude old > recipes until their maintainer or someone who needed it, got around to > upgrading the recipe. > > Mike > > On Sat, Jul 24, 2010 at 2:57 AM, Frans Meulenbroeks < > [email protected]> wrote: > > > Dear all, > > > > The topic of legacy staging has been on the table for probably 8 months > or > > so. > > Still we have a lot of recipes that use legacy staging. > > This email tries to stir up the discussion on how to get rid of these. > > > > Most of the major recipes seem to be converted. > > Koen reported 53 recipes with legacy staging after building > > angstrom-gnome-image and openjdk. [1] > > This seems an indication that lots of common and actively used recipes > are > > converted. > > However there are still 1100+ recipes and about 150 .inc files that have > a > > do_stage rule [2] > > > > This indicates that quite some work is still needed. > > Let's have a look at the options. What I see as possibilities are: > > > > A: accept that we will have lots of recipes around that use legacy > staging > > B: update and test all recipes (at least up to the level that it is > > verified > > that the files stage properly after removing the legacy staging > > C: with a sed script or so remove all do_stage functions and hope the > best > > for it. > > D: remove the recipes that still use legacy staging as apparently no-one > is > > enough interested in them to update them. > > > > Let us now look at the pro's and con's of these possiblities: > > > > From the various calls to fix this that I have seen on the mailing list A > > is > > not really too desired. > > > > B would be nice, but is a hell lot of work. With 1100 recipes, 150 inc > > files > > and 5 minutes per recipe, this takes about 100 hours. > > Even with one minute per recipe it would be about 20 hours. > > Given that lots of the recipes for which this applies seem to be rarely > > used > > or are older versions of recipes that are not used any more, this seems > > somewhat a waste of time. > > Unless someone stands up as volunteer or someone develops an automated > > solution, I feel this is not going to happen. > > (and no: I feel no desire at all to spend hours and hours of my spare > time > > to convert recipes most of which I am very unlikely to use). > > > > C is a quick hack without warranty that the recipe is not broken. > > I've no idea how you feel about this, but in my opinion I'd rather have a > > legacy staging recipe which works than a non-legacy staging one which > might > > or might not be broken. > > > > That leaves option D. Of course removing all recipes that still use > legacy > > staging is not desired, as that would also mean e.g. removal of the 53 > > recipes identified by Koen. [1] > > However, the idea has some merits. Lots of the recipes with legacy > staging > > seem to be old recipes. See e.g. the alsa-lib example [3]. By removing > > these > > at least the time and work needed for B would be less. > > > > Now how to proceed? > > Well that is the reason for this email. > > I would like to hear your opinions on this, so feel free to voice them. > > If there is consensus we can start deploying things. If not we might ask > > the > > TSC for some guidance. > > > > To start off the discussion let me give you my personal view. > > > > I would be in favour to remove all recipes that use legacy staging and > that > > do not fit into one of the following categories: > > - it is the most recent version of the package that is build by the > recipe > > - it is not the most recent version but all more recent versions have > > DEFAULT_PREFERENCE = "-1" > > - it is pinned by a distro > > - it is a toolchain recipe (gcc, binutils, automake, autoconf and > probably > > glibc) > > - it is a kernel or u-boot recipe > > The rationale behind this is that it removes a lot of recipes (and hence > a > > lot of work converting). > > Note that the recipes are not gone. They will remain in the stable 2009 > > branch and they can always be retrieved from git. > > So should someone for whatever reason need a recipe he/she can recover > it, > > fix it and put it back. > > > > After that we can make an inventory of the work remaining. > > If there are relatively few recipes remaining it will become a lot > simpler > > to find volunteers to clean up those. > > If there are many e.g. because an orphaned distro or machine pins lots of > > legacy recipes) we might consider a different scenario. > > > > This is my personal view, but ofc I would like to have a discussion on > this > > and hear other opinions so preferably we can come to a consensus. > > The only request I have is that if you advocate a certain solution that > you > > are willing to participate in realising that solution. > > E.g. it is easy to say that B is the desired scenario and that others > > should > > implement this. > > > > Best regards, Frans. > > > > PS: if the consensus is to start off removing the legacy recipes as I > > proposed above, I am more than willing to participate in that. > > and if someone has a good idea on how to automate identification of > > qualifying recipes (especially weeding out from the list, the ones we > still > > want to retain), I'd love to learn about that too. > > > > [1] > > > > > http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-July/021870.html > > [2] > > > > > http://lists.linuxtogo.org/pipermail/openembedded-devel/2010-July/021901.html > > [3] > > > > > http://www.mail-archive.com/[email protected]/msg08150.html > > _______________________________________________ > > Openembedded-devel mailing list > > [email protected] > > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > > > _______________________________________________ > Openembedded-devel mailing list > [email protected] > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics _______________________________________________ Openembedded-devel mailing list [email protected] http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
