Re: [gentoo-dev] Re: GLEP 55 Version 2
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2009.06.07 10:34, Ulrich Mueller wrote: > > On Sun, 07 Jun 2009, Steven J Long wrote: > > > I'd just like to know what the implications would be for users if > we > > kept the .ebuild extension, and a new PMS were rolled out stating > > that the mangler were allowed to find the EAPI without sourcing > (and > > giving the restrictions) once portage 2.2 was stable, or the > ability > > to handle this backported to 2.1.6, and issued in a release? > > Even if we do only a one-time change of the file extension, can we > ever get rid of the old extension? Or are we then stuck with two > extensions in the tree until the end of time? > > Let's assume for the moment that we change from ".ebuild" to ".eb". > Then we obviously cannot change all ebuilds in the tree to ".eb", > otherwise old Portage versions would see an empty tree and there > would > be no upgrade path. > > Or am I missing something? > > Ulrich First, lets be clear that the upgrade path related problems are driven by the EAPI change, not the .ebuild to .eb change. That serves only to allow the new EAPI to be introduced immedately and hide the change from older package managers The upgrade path issue remains even if we do the usual Gentoo introduce new features to the package manager then wait a year or so before they are deployed. Without the .ebuild to .eb change. late upgraders will get the error messages in Appendix A of the GLEP when these features are relesed. To keep an upgrade path, package managers and their dependencies need to use EAPI 0, 1 or 2. (EAPI 3 is not yet out) How far back should the upgrade path extend? As you suggest, even with .ebuild to .eb change.its not a clean break with the past but would happen over time, with ebuilds for new package versions being migrated to the new format. For example we would have foo-2.1-r1.ebuild foo-3.0.eb portage-2.3.ebuild portage-3.0.eb Portage-2.3 will understand the later EAPI but will itself, use only EAPI, 0, 1 or 2. With the .ebuild to .eb change. users who do not upgrade portage will not see newer versions of foo. Without it, they will get the error messages in Appendix A of the GLEP. This will have the side effect of driving the adoption of the newer package managers. It is not suffcient to have portage-2.3.ebuild as understanding later EAPI files. All of its dependencies need to keep to EAPI 0, 1 or 2. These must be the last packages to migrate to later EAPIs. Thats just portage. The same reasoning applies to any other package manager and there are at least three. This begs the question of how friendly do we want to be to derivitive distros that use our tree but their own package manager? Upgrade path issues need to be addressed in the GLEP. I will add something like the above in the next version. - -- Regards, Roy Bamford (NeddySeagoon) a member of gentoo-ops forum-mods treecleaners trustees -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.11 (GNU/Linux) iEYEARECAAYFAkoryrUACgkQTE4/y7nJvashtQCgkO+feHG2BBGLOTObrwq72iOx nI4AoNZ67Mhq4yEKYTzfBOPmu98Py9Mc =GB62 -END PGP SIGNATURE-
Re: [gentoo-dev] Re: GLEP 55 Version 2
Patrick Lauer wrote: And if you really absolutely have to do that you can change the sync location on every disruptive change, but (imo) that should be avoided. If mirroring and other practical concerns weren't an issue what you're essentially describing is just moving to a CVS/git/etc repository and using such a tool to do all the syncs (ie not using rsync at all). Then all you need is an update page that has a list of commands like: emerge --sync --date "2008-01-01" emerge -u world emerge --sync --date "2008-08-10" Follow this xorg update doc: emerge --sync --date "2034-04-02" emerge -u portage so that you have support for the finally-approved glep55 And so on... :) That actually gives you a best-of-both-worlds option: continue to use rsync for normal use to ease distribution, but have a cvs tree that anybody can read from which could be used in upgrade guides for out-of-date users.
[gentoo-dev] Re: GLEP 55 Version 2
Richard Freeman posted 4a2baaa9.4030...@gentoo.org, excerpted below, on Sun, 07 Jun 2009 07:55:21 -0400: > As far as an upgrade path goes - we could provide a one-time tarball > that will update portage (and its essential dependencies) to a version > that can get users out of this bind. If a user has a system THAT old > then they might just want to extract a stage1 tarball (manually - > without overwriting /etc without care) and go from there. We've done the tarball thing a couple times before, with portage I think, with amd64/gcc for certain, as it was needed to get out of some sort of multilib and profile based bind IIRC, and with the in-tree profiles (from pre-cascade profiles) at least once too, IIRC. > I'm not sure that gentoo generally supports graceful upgrades from very > ancient systems to modern ones without keeping up to date. Other > distros can do it since they do ~annual releases and users could just > apply those sequentially. For portage we don't keep around all the > files needed to do a sequential upgrade like this - if a user were to > try to upgrade to a 3-year-old version of some package most likely it > wouldn't be mirrored and upstream might not have it either. AFAIK from what I've read here over the years, Gentoo tries to keep smooth in-tree upgrades to a year out. Beyond that, we don't usually deliberately break it without some warning and a tarball or similar upgrade path for another six months to a year, but it's by no means guaranteed it'll be a smooth upgrade after a year even if we aren't deliberately breaking it. Generally, beyond a year, it's recommended that one uses the stage tarball to get something at least operationally modern, and goes from there. Simply put, Gentoo's NOT in practice a distribution for the folks who like to lollygag around for years between updates. Tho we do try to support it up to a year out and to provide at least some form of likely non-routine upgrade option beyond that, it definitely works best and with the least trouble for those updating every month or at least once a quarter, with things getting progressively more difficult and troublesome the further out beyond that you go, simply because of lack of testing if nothing else. > We obviously need to give some thought to not breaking old versions of > portage, but given that portage will be only one of many problems if a > user doesn't do an emerge -u world for 5 years I'm not sure we need a > bulletproof solution... I just realized that I'm right about at my Gentoo 5-year anniversary, with an original installation of 2004.1. (I tried 2004.0 but it didn't work for some reason I never did figure out, but perhaps related to the then new NPTL, which I was trying to enable.) I can't /imagine/ first installing it then, and coming back to it now, expecting anything but a full reinstall from stage tarball (assuming as suppose I would be if I had been that out of it, that was still even /using/ stage tarballs as it was then). Imagine people wondering what happened to xfree86, among other things. I mean, talk about a time- traveler getting confused by the future! -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: GLEP 55 Version 2
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ulrich Mueller wrote: >> On Sun, 07 Jun 2009, Steven J Long wrote: > >> I'd just like to know what the implications would be for users if we >> kept the .ebuild extension, and a new PMS were rolled out stating >> that the mangler were allowed to find the EAPI without sourcing (and >> giving the restrictions) once portage 2.2 was stable, or the ability >> to handle this backported to 2.1.6, and issued in a release? > > Even if we do only a one-time change of the file extension, can we > ever get rid of the old extension? unfortunately, no. > Or are we then stuck with two > extensions in the tree until the end of time? > Let's assume for the moment that we change from ".ebuild" to ".eb". better put this new ebuild type in a new tree; such a massive change to the tree its not recommended. > Then we obviously cannot change all ebuilds in the tree to ".eb", > otherwise old Portage versions would see an empty tree and there would > be no upgrade path. leaving actual ".ebuild"s as they are now (good healthy state :)) and making new development of ".eb" ebuilds happen in a new tree (I said new tree, but it could be any way that hides those new ebuild to OLD package managers) would help. only newer versions of package managers are required to support this, that is they will look for .eb (in new or current tree, not sure what's best) and then for .ebuilds, and ideally this should be transparent to old package managers and allow an upgrade path. - -- mescali...@g.o -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkortVIACgkQV/B5axfzrPsTiACeJCJb3F8Up/+CjHIwC3Slhn/6 yZgAoLcJgNn2d3W/JeZPkK85arUPW9vV =fR4T -END PGP SIGNATURE-
Re: [gentoo-dev] Re: GLEP 55 Version 2
On Sunday 07 June 2009 11:34:12 Ulrich Mueller wrote: > > On Sun, 07 Jun 2009, Steven J Long wrote: > > > > I'd just like to know what the implications would be for users if we > > kept the .ebuild extension, and a new PMS were rolled out stating > > that the mangler were allowed to find the EAPI without sourcing (and > > giving the restrictions) once portage 2.2 was stable, or the ability > > to handle this backported to 2.1.6, and issued in a release? > > Even if we do only a one-time change of the file extension, can we > ever get rid of the old extension? Or are we then stuck with two > extensions in the tree until the end of time? > > Let's assume for the moment that we change from ".ebuild" to ".eb". > Then we obviously cannot change all ebuilds in the tree to ".eb", > otherwise old Portage versions would see an empty tree and there would > be no upgrade path. > > Or am I missing something? I come to the same conclusion. Which means that the easiest way to get things migrated will be a one-time change of the _sync_ location so that users have a chance to get to an upgrade-ready state on their system, then change sync location, then upgrade to the current state. In which case we actually do not have to change the file name anymore ... Of course things aren't always that easy, but if you add a "generation file" somewhere in the rsync checkout it is easy to see if your current rsync location is "old and deprecated" or "new and improved". And if you really absolutely have to do that you can change the sync location on every disruptive change, but (imo) that should be avoided. Less disruptive changes is better :) hth, Patrick
Re: [gentoo-dev] Re: GLEP 55 Version 2
Ulrich Mueller wrote: Let's assume for the moment that we change from ".ebuild" to ".eb". Then we obviously cannot change all ebuilds in the tree to ".eb", otherwise old Portage versions would see an empty tree and there would be no upgrade path. Or am I missing something? That is a good point. Although things would probably break more gracefully than having old portage versions attempting to parse new ebuilds (maybe, maybe not). That actually makes me wonder - if we didn't change the extension at all could we somehow poison the portage tree so that old versions of portage would abort before doing anything dumb with a meaningful error message? As far as an upgrade path goes - we could provide a one-time tarball that will update portage (and its essential dependencies) to a version that can get users out of this bind. If a user has a system THAT old then they might just want to extract a stage1 tarball (manually - without overwriting /etc without care) and go from there. I'm not sure that gentoo generally supports graceful upgrades from very ancient systems to modern ones without keeping up to date. Other distros can do it since they do ~annual releases and users could just apply those sequentially. For portage we don't keep around all the files needed to do a sequential upgrade like this - if a user were to try to upgrade to a 3-year-old version of some package most likely it wouldn't be mirrored and upstream might not have it either. We obviously need to give some thought to not breaking old versions of portage, but given that portage will be only one of many problems if a user doesn't do an emerge -u world for 5 years I'm not sure we need a bulletproof solution...
[gentoo-dev] Re: GLEP 55 Version 2
> On Sun, 07 Jun 2009, Steven J Long wrote: > I'd just like to know what the implications would be for users if we > kept the .ebuild extension, and a new PMS were rolled out stating > that the mangler were allowed to find the EAPI without sourcing (and > giving the restrictions) once portage 2.2 was stable, or the ability > to handle this backported to 2.1.6, and issued in a release? Even if we do only a one-time change of the file extension, can we ever get rid of the old extension? Or are we then stuck with two extensions in the tree until the end of time? Let's assume for the moment that we change from ".ebuild" to ".eb". Then we obviously cannot change all ebuilds in the tree to ".eb", otherwise old Portage versions would see an empty tree and there would be no upgrade path. Or am I missing something? Ulrich
[gentoo-dev] Re: GLEP 55 Version 2
Roy Bamford wrote: > I've spent some time reading all of this years emails on GLEP55 and > added a few lines to version 1.5 which is the last offical version. > Thanks for all the hard work. My apologies for my mistaken comment at the end of the last Council meeting. Clearly the mangler /does/ need to know the EAPI without sourcing for future extensibility. I'd just like to know what the implications would be for users if we kept the .ebuild extension, and a new PMS were rolled out stating that the mangler were allowed to find the EAPI without sourcing (and giving the restrictions) once portage 2.2 was stable, or the ability to handle this backported to 2.1.6, and issued in a release? Would it be unacceptable to have a clear upgrade path for any users who didn't actually update portage? (Perhaps a news item so they'd be notified as and when they ran emerge.) It strikes me that we went through a similar situation with bash-3.17 I think it was, and that once we're past this, there'll never be any need to worry in the future. So, given that it's taken so long and so much discussion, why not just do this last big bump, and not worry about about using another extension at all? After all, the issue would only arise once Council approved an EAPI requiring one of the incompatible changes, and 3 has only recently come out. Also, you asked for proponents of either side to provide statistics as to the time difference between the various options. It's rather hard for me to patch paludis, nor do I have the inclination. I am not being facetious, simply pointing out that the comparison has to be made within a single app. Comparing an approach implemented in portage vs one in paludis is comparing apples and oranges. Also, Patrick brought up cache improvements (like not having a single cache file per ebuild, but rather per package. This could have the EAPI and versions first, followed by metadata per version.) I feel we should bear in mind that there are other areas where we can improve things, many of which we have not even thought of, or at least discussed. With respect to time gains in interactivity, much has been made of the speed difference between portage and paludis. I have yet to see any comparative numbers between pkgcore and paludis in this regard. (If we are going to compare manglers and not algorithms.) I think we are agreed now that an EAPI which can be extracted before source does fulfil all the criteria (as others have said, this is actually what the GLEP is about; ascertaining the EAPI without needing to source.) If not, could someone please explain why not. Regards, Steve. -- #friendly-coders -- We're friendly but we're not /that/ friendly ;-)
[gentoo-dev] Re: GLEP 55: another approach: display pretty messages with old PMs
Michael Haubenwallner posted 1243610264.27150.293.ca...@sapc154.salomon.at, excerpted below, on Fri, 29 May 2009 17:17:44 +0200: > Wouldn't it be possible to avoid both the extension change and another > extended wait period for new incompatible(*) EAPIs, when we do this > early and silent exit hack for unsupported ebuilds with old PMs that > still do full interpretation for EAPI detection? Possibly. I forgot about the context (the inherit eapi.eclass hack) when I wrote the previous (until /after/ I posted, naturally, probably when you noticed that 4.ebuild thing too, of course =:^). But the possibility had been proposed before and I don't remember what came of it, nor have I been following close enough to know if there's another caveat somewhere that shoots down the eapi.eclass hack, or not. I'm sure someone else will supply the reason it didn't go anywhere. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] Re: GLEP 55: another approach: display pretty messages with old PMs
On Fri, 2009-05-29 at 10:42 +, Duncan wrote: > Michael Haubenwallner posted on Fri, 29 > May 2009 10:14:46 +0200: > > > Ohw, the latter would be necessary here, or '4.ebuild' would not be > > found. > > s/4.ebuild/4.eclass/ I assume. Indeed. > Except... since an ebuild must presently be sourced to (properly) > determine EAPI, it still doesn't work for changes that break sourcing or > other pre-EAPI processing (like parsing PN and PVR out of the filename), > so the changes allowed with a simple EAPI change are still rather limited. As long as the *whole* ebuild content (including the filename) needs to be successfully interpreted just for EAPI detection, we will have to change the extension or wait the extended period for each incompatible EAPI change. So this full interpretation for EAPI detection doesn't feel like a good way to go at all. > That's why the change to GLEP55 or alternative, whether in-filename or > in-file-itself, will again require either an extended wait after > introduction (the old way) or at minimum, a one-time change to extension > such that old PM versions don't even see the currently EAPI incompatible > changes. Wouldn't it be possible to avoid both the extension change and another extended wait period for new incompatible(*) EAPIs, when we do this early and silent exit hack for unsupported ebuilds with old PMs that still do full interpretation for EAPI detection? And after another extended wait period(**), we can start dropping the silent exit hack, so we finally have switched to EAPI detection without full interpretation, but still have the .ebuild extension. (*) The incompatibility of EAPIs must not begin (meaning the bytewise ebuild content) before the end of both the ebuild's EAPI value definition and the silent exit hack. But this IMO is an acceptable compromise. (**) After this wait period, the incompatibility of EAPIs can start after the end of the ebuild's EAPI value definition. Thanks! /haubi/ -- Michael Haubenwallner Gentoo on a different level
[gentoo-dev] Re: GLEP 55: another approach: display pretty messages with old PMs
Michael Haubenwallner posted 1243584886.27150.33.ca...@sapc154.salomon.at, excerpted below, on Fri, 29 May 2009 10:14:46 +0200: > Ohw, the latter would be necessary here, or '4.ebuild' would not be > found. s/4.ebuild/4.eclass/ I assume. > Btw.: What do non-EAPI-aware PMs do with ebuilds using EAPI 1 and 2 - > how become they masked _now_? What did I miss here? They used the old "wait an extended period (loosely speaking, a year) after initial stabilization of a new feature before use" method. EAPI was supposed to do away with that, since once EAPI aware PMs could be assumed (after the year or whatever waiting period), any EAPI a PM didn't understand was supposed to be rejected. Except... since an ebuild must presently be sourced to (properly) determine EAPI, it still doesn't work for changes that break sourcing or other pre-EAPI processing (like parsing PN and PVR out of the filename), so the changes allowed with a simple EAPI change are still rather limited. That's why the change to GLEP55 or alternative, whether in-filename or in-file-itself, will again require either an extended wait after introduction (the old way) or at minimum, a one-time change to extension such that old PM versions don't even see the currently EAPI incompatible changes. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: GLEP 55: another approach: display pretty messages with old PMs
On Thu, 2009-05-28 at 19:17 +0200, Michael Haubenwallner wrote: > inherit eapi 4 > > Now when the PM is capable of pre-source EAPI detection, it will set > EAPI before sourcing, eapi.eclass can see EAPI already being set and not > do the 'exit' in global scope. Or even the PM's inherit-implementation > expects to be first called with arguments "eapi 4", and not reading the > eapi.eclass at all, so the 'eapi.eclass' does not need to check for > anything, just needs to 'exit' when inherited. Ohw, the latter would be necessary here, or '4.ebuild' would not be found. eapi.eclass could also be renamed to sth. like eapisupport.eclass or eapivalidation.eclass, to write this way: EAPI="4" inherit eapisupport But then the eclass has to detect which EAPI's are supported by the running PM to 'exit' upon an unsupported one. Btw.: What do non-EAPI-aware PMs do with ebuilds using EAPI 1 and 2 - how become they masked _now_? What did I miss here? Thanks! /haubi/ -- Michael Haubenwallner Gentoo on a different level
Re: [gentoo-dev] Re: GLEP 55 updated
2009/5/18 Steven J Long : > David Leverton wrote: > >> 2009/5/17 Ben de Groot : >>> I think the way eapi-2 was introduced into the tree wasn't particularly >>> problematic. >> >> I think there might be a misunderstanding here. Ciaran means functions >> provided by the package manager that ebuilds can call during metadata >> generation, for example built-in versionator-like functionality, not >> new phase functions like src_prepare and src_configure. > > Ugh. Firstly versionator is a piece of bloated crap that should have died a > long time ago; all it does is stop Gentoo newbs learning the basics[1] of > their implementation language, which leaves them open to nonsensical > arguments about printf -v (glad you finally learnt that one, btw.) Yes, it should have died a long time ago, that's why we're talking about adding equivalent built-in functionality. Please try to keep up. (And it's always amusing to see your bizarre delusions about what I do and don't know.) > Secondly, please explain fully and clearly, but concisely, why we can't > simply state that EAPI=NN allows the ebuild to use the EAPI functions in > global scope. Because by the time the package manager knows the EAPI and is in a position to provide the appropriate functions, the ebuild will have already tried to use them. > Bear in mind that you have to deal with the case of the mangler which can > get EAPI from an ebuild without sourcing, as portage currently can, I > believe. Interesting > Relaxing that restriction strikes me as much saner than relaxing all the > other restrictions which make interoperability easier. > > (Frankly, I'm amazed at having to state that to people trusted to write a > specification. Follow up to -project on this point please, as it's about > process, not technique.) You're amazed that people trusted to write a specification don't already know what "strikes you as much saner"? Believe it or not, the world does not revolve around you and your opinions.
Re: versionator.eclass terminator, was [gentoo-dev] Re: GLEP 55 updated
On Mon, 18 May 2009 17:42:19 +0200 Robert Buchholz wrote: > I'm not following. Why should it be discouraged? > I was happy with it until now. Versionator is a lot better than what people were doing before I wrote it. It's just nowhere near as good as what a package manager provided solution combined with a reduced need for MY_PV hacks would give. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: versionator.eclass terminator, was [gentoo-dev] Re: GLEP 55 updated
On Monday 18 May 2009, Jeroen Roovers wrote: > On Mon, 18 May 2009 16:16:46 +0100 > > Ciaran McCreesh wrote: > > Why do you think I wrote the awful hack that is versionator? > > Why don't you explain why, historically, you put that in the tree? It > would help us now if you were to simply record your mistakes for > everybody else to easily avoid. It's still being used in the tree and > should be discouraged. I'm not following. Why should it be discouraged? I was happy with it until now. Robert signature.asc Description: This is a digitally signed message part.
Re: versionator.eclass terminator, was [gentoo-dev] Re: GLEP 55 updated
On Mon, 18 May 2009 17:28:00 +0200 Jeroen Roovers wrote: > On Mon, 18 May 2009 16:16:46 +0100 > Ciaran McCreesh wrote: > > Why do you think I wrote the awful hack that is versionator? > > Why don't you explain why, historically, you put that in the tree? It > would help us now if you were to simply record your mistakes for > everybody else to easily avoid. It's still being used in the tree and > should be discouraged. Versionator was created because the alternative was worse: developers were hard-coding versions in ebuilds, writing dodgy bash substitutions that wouldn't reliably convert between versions and using things like sed and tr in global scope. The problem is, versionator was written before the current version rules. It doesn't handle some things that are now legal, and it still uses the old meanings for numeric components. The correct solution is two-fold: * Replace versionator with package-manager internal functions that use the package manager's rules for version parsing. * Reduce the need to use MY_PV by extending the version rules. Both of these are things that can't be done with current EAPI rules. > > Anything that finally lets us kill that off has to be good... > > Loosening VERSION requirements won't fix the problem. This will: > > 1) Discourage its use by putting a QA ewarn in the eclass. > 2) Have all ebuilds converted either through QA bugs or a nice > Saturday afternoon coding spree. > 3) Announce its removal. > 4) Remove. You can't discourage versionator until the replacement's in place. Going back to the old way of doing things is a loot worse than using versionator. So no, the way to fix it is: 1) Change the EAPI rules to allow global scope and version suffix changes. 2) Bring in a versionator replacement done as internals in a new EAPI. 3) Extend the version format rules in a new EAPI. 4) Disallow versionator use in the first EAPI that has 2) and 3). -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55 updated
Ciaran McCreesh wrote: > On Mon, 18 May 2009 16:07:20 +0100 > Steven J Long wrote: >> I missed the clamour of developers complaining about this >> oh-so-burdensome restriction that they've been dealing with for at >> least 5 years. > > Why do you think I wrote the awful hack that is versionator? > Anything that finally lets us kill that off has to be good... I have to disagree. As Steve said, the fact that Gentoo has a strict way to specify versions brings clarity and uniformity to our tree, regardless of the myriad upstream conventions. I do not think that allowing all of those upstream conventions in our filenames is a benefit. It is actually quite ugly and would make the tree harder to comprehend. Someone looking at various packages in our tree would need to learn each specific upstream format in order to make sense of the filename content. The current consistency in versions in the tree is a great feature, IMHO. Using versionator and $MY_PV is, as I see it, a translation method. It gives us the best of both worlds: the ability to deal with upstream versions and a consistent Gentoo version format. These mechanisms could certainly be improved upon, of course, and handling some cases is currently difficult, as is handing the case in which upstream's tarball file does not contain the version, but these are fixable issues. -Joe
[gentoo-dev] Re: GLEP 55 updated
David Leverton wrote: > 2009/5/17 Ben de Groot : >> I think the way eapi-2 was introduced into the tree wasn't particularly >> problematic. > > I think there might be a misunderstanding here. Ciaran means functions > provided by the package manager that ebuilds can call during metadata > generation, for example built-in versionator-like functionality, not > new phase functions like src_prepare and src_configure. Ugh. Firstly versionator is a piece of bloated crap that should have died a long time ago; all it does is stop Gentoo newbs learning the basics[1] of their implementation language, which leaves them open to nonsensical arguments about printf -v (glad you finally learnt that one, btw.) Secondly, please explain fully and clearly, but concisely, why we can't simply state that EAPI=NN allows the ebuild to use the EAPI functions in global scope. Bear in mind that you have to deal with the case of the mangler which can get EAPI from an ebuild without sourcing, as portage currently can, I believe. Relaxing that restriction strikes me as much saner than relaxing all the other restrictions which make interoperability easier. (Frankly, I'm amazed at having to state that to people trusted to write a specification. Follow up to -project on this point please, as it's about process, not technique.) [1] (watch wrap) http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_02 http://wooledge.org/mywiki/BashFAQ/073 man bash /Parameter Expansion -- #friendly-coders -- We're friendly but we're not /that/ friendly ;-)
Re: versionator.eclass terminator, was [gentoo-dev] Re: GLEP 55 updated
On Mon, 18 May 2009 16:16:46 +0100 Ciaran McCreesh wrote: > Why do you think I wrote the awful hack that is versionator? Why don't you explain why, historically, you put that in the tree? It would help us now if you were to simply record your mistakes for everybody else to easily avoid. It's still being used in the tree and should be discouraged. > Anything that finally lets us kill that off has to be good... Loosening VERSION requirements won't fix the problem. This will: 1) Discourage its use by putting a QA ewarn in the eclass. 2) Have all ebuilds converted either through QA bugs or a nice Saturday afternoon coding spree. 3) Announce its removal. 4) Remove.
Re: [gentoo-dev] Re: GLEP 55 updated
On Mon, 18 May 2009 16:07:20 +0100 Steven J Long wrote: > I missed the clamour of developers complaining about this > oh-so-burdensome restriction that they've been dealing with for at > least 5 years. Why do you think I wrote the awful hack that is versionator? Anything that finally lets us kill that off has to be good... -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] Re: GLEP 55 updated
Joe Peterson wrote: > Ciaran McCreesh wrote: >>> 3. "Extend versioning rules in an EAPI - for example, addition of the >>> scm suffix - GLEP54 [1] or allowing more sensible version formats like >>> 1-rc1, 1-alpha etc. to match upstream more closely." >>> Apart from GLEP54, I believe our versioning scheme works reasonably >>> well. I don't see any need to match upstream more closely. I'd rather >>> like to keep the more uniform way of handling suffixes like rc and >>> alpha, that we have now. >> >> Please explain why 1.2_rc3 is legal but 1.2-rc3 is not. > > I actually like the current format in that it does *not* allow "-" in > the version. For example, pkg-2.3.1_rc5 makes it clear that the string > from "2" to "rc5" is the version. If were were to allow pkg-2.3.1-rc5, > this could get visually confusing (looks a bit like pkg-2.3.1-r5). In > this case, *less* flexibility and more strict rules serve a good > purpose, I think. > Agreed; the purpose of an internal format specification is not to allow NN variants on a theme all over the place. It should nail things down to ONE variant which everybody can collaborate around. I missed the clamour of developers complaining about this oh-so-burdensome restriction that they've been dealing with for at least 5 years. Until I see that happening independently of this current hooha, I'm going to consider this 'reason' to be yaf "straw man". -- #friendly-coders -- We're friendly but we're not /that/ friendly ;-)
[gentoo-dev] Re: GLEP 55 updated
Just a heads up that I wrote a more detailed description of the peformance hit that EAPI in the ebuild introduces. Might come up with some numbers later too. [1] - http://dev.gentoo.org/~peper/glep-0055.html#easily-fetchable-eapi-inside-the-ebuild -- Best Regards, Piotr Jaroszyński
[gentoo-dev] Re: GLEP 55 updated
On Sun, 17 May 2009 19:18:14 +0100 Ciaran McCreesh wrote: > On Sun, 17 May 2009 12:15:24 -0600 > Ryan Hill wrote: > > I'd like 2 if we could have multiple same-versioned ebuilds of > > different EAPI. 3 is good enough for me. > > We couldn't. Allowing multiple equal but different ebuilds gets highly > crazy -- EAPIs aren't orderable, so it's not obvious which one the > package manager should select. Ah, right. I knew there was a reason. -- gcc-porting, by design, by neglect treecleaner, for a fact or just for effect wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662 signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55 updated
On Sun, 17 May 2009 12:15:24 -0600 Ryan Hill wrote: > I'd like 2 if we could have multiple same-versioned ebuilds of > different EAPI. 3 is good enough for me. We couldn't. Allowing multiple equal but different ebuilds gets highly crazy -- EAPIs aren't orderable, so it's not obvious which one the package manager should select. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55 updated
2009/5/17 Ryan Hill : > On Sun, 17 May 2009 17:56:06 +0200 > Piotr Jaroszyński wrote: > >> Hello, >> >> I have just updated GLEP 55 [1], hopefully making it a bit clearer. >> >> Just FYI, my order of preference of solutions is: >> >> 1. EAPI-suffixed ebuilds (obviously) >> 2. EAPI in the filename with one-time extension change >> 3. Easily fetchable EAPI inside the ebuild and one-time extension change >> >> I can live with any of these if that's what it takes to move forward. > > I'd like 2 if we could have multiple same-versioned ebuilds of different > EAPI. 3 is good enough for me. That's covered in the GLEP: "Note that it is still not permitted to have more than one ebuild with equal category, package name, and version. Although it would have the advantage of allowing authors to provide backwards compatible ebuilds, it would introduce problems too. The first is the requirement to have strict EAPI ordering, the second is ensuring that all the ebuilds for a single category/package-version are equivalent, i.e. installing any of them has exactly the same effect on a given system." I don't see a way to overcome these problems in a sensible way. -- Best Regards, Piotr Jaroszyński
[gentoo-dev] Re: GLEP 55 updated
On Sun, 17 May 2009 17:56:06 +0200 Piotr Jaroszyński wrote: > Hello, > > I have just updated GLEP 55 [1], hopefully making it a bit clearer. > > Just FYI, my order of preference of solutions is: > > 1. EAPI-suffixed ebuilds (obviously) > 2. EAPI in the filename with one-time extension change > 3. Easily fetchable EAPI inside the ebuild and one-time extension change > > I can live with any of these if that's what it takes to move forward. I'd like 2 if we could have multiple same-versioned ebuilds of different EAPI. 3 is good enough for me. -- gcc-porting, by design, by neglect treecleaner, for a fact or just for effect wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662 signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
Fernando J. Pereda wrote: On 10 Jun 2008, at 15:33, Joe Peterson wrote: Luca Barbato wrote: Check if exists a line EAPI=*$, if does and the rest of the string matches an understood eapi, go on sourcing, otherwise ignore/mask it... And placing it out-of-band (like "# EAPI=...") avoids any sourcing errors, makes parsing faster, etc. No, it doesn't make parsing faster. Had you bothered to profile any package manager you'd know that. How much is parsing speed relevant when users in majority of cases already have the metadata cache from the rsync servers? For devs (or users) hacking on an ebuild they have to source it anyway so the addition pre-parsing is negligible... VB -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
"Santiago M. Mola" <[EMAIL PROTECTED]> wrote: > On Thu, Jun 12, 2008 at 4:07 AM, Jim Ramsay <[EMAIL PROTECTED]> wrote: > > Why not just bump the filename suffix when it is required to > > support a new EAPI that breaks the sourcing rules of previous EAPIs? > > > > Or will backwards-incompatible changes be happening so frequently > > that the package suffix will have to change for every EAPI bump > > anyway, which would make this proposal equivalent to GLEP55? > > That works. Although, we'd have to keep track of two parameters when > setting our EAPI. One being the EAPI itself and the suffix needed for > that EAPI. So this doesn't seem to make the problem simpler. I agree that it doesn't make things simpler. Though it doesn't make things that much more complex. That said, I'm becoming more convinced that a lot of the changes that really need to be made will indeed break EAPI sourcing rules, so maybe the latter part of my original quote above will indeed be the case - This would be equivalent to GLEP55. I should add that I am not personally opposed to having the EAPI in the filename, but this may find slightly more acceptance from the folks who think that solution is incorrect. -- Jim Ramsay Gentoo Developer (rox/fluxbox/gkrellm) signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On 2008-06-10 15:50, Fernando J. Pereda uttered these thoughts: > > On 10 Jun 2008, at 15:46, Joe Peterson wrote: >> Also, I'm not sure reading XML is a problem at all - python has good >> libs for this already. > > Reading XML files is easy, but it makes certain codepaths much much slower. > Not a good 'feature'. ... AND there's no requirement for the PM to be written in python or any other language currently in use. -- () The ASCII Ribbon Campaign - against HTML Email /\ and proprietary formats. pgpuqnCKU4eqn.pgp Description: PGP signature
[gentoo-dev] Re: GLEP 55
Duncan <[EMAIL PROTECTED]> posted [EMAIL PROTECTED], excerpted below, on Wed, 11 Jun 2008 12:52:24 +: > Ciaran McCreesh <[EMAIL PROTECTED]> posted > [EMAIL PROTECTED], excerpted below, on Tue, 10 Jun > 2008 15:00:18 +0100: > >> On Tue, 10 Jun 2008 09:49:04 -0400 >> Richard Freeman <[EMAIL PROTECTED]> wrote: >>> 4) Putting EAPI inside the ebuild, but in a manner that does not >>> require sourcing using bash (ie comment at top of file). > >> - it doubles the number of file reads necessary during resolution. > > No comment, except that it should be cached after the first one, so the > second read should be a memory read. Yeah, replying to myself... Having read the metadata-file/ebuild discussion, I see that despite the current sourcing "requirement", it's not being done in practice for dependency building during which the EAPI is a necessary data point. Only the metadata is being read, thus speeding up the process where seeks between metadata and the ebuild would otherwise be needed. While a single additional seek per ebuild would suffice (it'd be in cache after that), that could still add up to a LOT of extra seeks (with the resultant time and cache usage increase), especially on an --emptytree world or similar. Still, it's essentially required by the current spec even if the now- current EAPIs manage to avoid it due to similarity of the current EAPIs, so it'd be no new imposition. The question then becomes, if everything else needed is stored in the pre- generated metadata cache, why isn't this? Shouldn't it be there along with the rest of the (meta)data used for building the dependency tree, thus avoiding the extra seek? Why not put it in the ebuild as is the dependency data, then along with said dependency data, copy it to metadata cache when it's generated? As for backward compatibility on the metadata, append or separate (single per repository) file. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Thu, Jun 12, 2008 at 4:07 AM, Jim Ramsay <[EMAIL PROTECTED]> wrote: > Why not just bump the filename suffix when it is required to support a > new EAPI that breaks the sourcing rules of previous EAPIs? > > Or will backwards-incompatible changes be happening so frequently that > the package suffix will have to change for every EAPI bump anyway, > which would make this proposal equivalent to GLEP55? That works. Although, we'd have to keep track of two parameters when setting our EAPI. One being the EAPI itself and the suffix needed for that EAPI. So this doesn't seem to make the problem simpler. Regards, -- Santiago M. Mola Jabber ID: [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Wed, Jun 11, 2008 at 12:05 PM, Olivier Galibert <[EMAIL PROTECTED]> wrote: > On Tue, Jun 10, 2008 at 03:14:47PM +0100, Ciaran McCreesh wrote: >> > > *Then* would be the time to change the extension. As long as the > ebuild is bash-parseable with an appropriate environment, it doesn't > make sense to change the extension because a env-variable set or a > comment are more natural methods. > > If/when the format changes to something not parseable by bash, then it > will be time to change the extension. And then how to mark > (sub-)version will depend on the chosen new format, in case of xml > that would be the dtd information. > > I suspect the rejection of the extension change will be there as long > as the fundamental format (bash script) doesn't change. > If the extension was based on the fact that ebuilds are bash scripts, they'd have .bash extension. The .ebuild extension means a specific kind of bash script and it doesn't seem wrong to change the extension if that "specific kind" changes, even if bash is still the interpreter. Even if we switched to sh or zsh I doubt we'd use the .sh or .zsh extension. Regards, -- Santiago M. Mola Jabber ID: [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Olivier Galibert <[EMAIL PROTECTED]> wrote: > On Tue, Jun 10, 2008 at 03:14:47PM +0100, Ciaran McCreesh wrote: > > > > *Then* would be the time to change the extension. As long as the > ebuild is bash-parseable with an appropriate environment, it doesn't > make sense to change the extension because a env-variable set or a > comment are more natural methods. > > If/when the format changes to something not parseable by bash, then it > will be time to change the extension. And then how to mark > (sub-)version will depend on the chosen new format, in case of xml > that would be the dtd information. > > I suspect the rejection of the extension change will be there as long > as the fundamental format (bash script) doesn't change. Well said. This is something that I've heard bandied about on IRC now and then, and sounds to me (notably *not* a package manager developer) like a fairly reasonable compromise. To the proponents of GLEP55: Is there some reason that GLEP55 is preferable to this? Are there reasons why a particular filename extension could not apply to a range of EAPIs? Why not just bump the filename suffix when it is required to support a new EAPI that breaks the sourcing rules of previous EAPIs? Or will backwards-incompatible changes be happening so frequently that the package suffix will have to change for every EAPI bump anyway, which would make this proposal equivalent to GLEP55? -- Jim Ramsay Gentoo/Linux Developer (rox,gkrellm) signature.asc Description: PGP signature
[gentoo-dev] Re: GLEP 55
Ciaran McCreesh <[EMAIL PROTECTED]> posted [EMAIL PROTECTED], excerpted below, on Tue, 10 Jun 2008 15:00:18 +0100: > On Tue, 10 Jun 2008 09:49:04 -0400 > Richard Freeman <[EMAIL PROTECTED]> wrote: >> 4) Putting EAPI inside the ebuild, but in a manner that does not >> require sourcing using bash (ie comment at top of file). > - it doubles the number of file reads necessary during resolution. No comment, except that it should be cached after the first one, so the second read should be a memory read. > - it heavily restricts future syntax and meaning of EAPIs I don't see this. Putting it at a defined place in the ebuild and parsing it pre-source nails down the problem such that if a future format change is further incompatible, a primary EAPI can be defined that defines a further extension, a second line to look at in the ebuild, some external file, the filename, whatever, for the additional sub-eapi or whatever detail, much as extensions to various Internet RFCs have done when they've wanted to extend beyond what would fit in the then-current definitions. It does little to restrict the ultimate combined primary/secondary EAPI, where the primary simply defines where and how to look for the secondary. > - it makes comments have meaning Only if we choose the comment format, and then no more than shebang and similar solutions do. In the latter case it's an already recognized *ix solution. In the former, it's entirely possible to use a backward compatible EAPI= simple assignment that can be pre-parsed, and use the sub-eapi (minor part, in terms discussed elsewhere) if necessary to further define it using functions or the like. That said, I don't see the big deal to putting it in the file extension, when we're already breaking traditional content-defines-type rules by decreeing .ebuild extensions in the first place. I'd personally like to keep it a one-time fixed extension change, if only to force some discipline on the proliferation by forcing each new change to be reauthorized, meaning it's /really/ needed or it's simply not worth the trouble, but really, the precedent was already set when we accepted metadata in filename with the .ebuild thing in the first place, so there's little reason to fight it now, unless the proposal also eliminates that, and backward compatibility with it. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Wed, Jun 11, 2008 at 12:06 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > On Wed, 11 Jun 2008 08:31:45 +0200 > Thomas de Grenier de Latour <[EMAIL PROTECTED]> wrote: >> On 2008/06/11, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: >> > You're missing the cases where the cache isn't usable. >> >> I was not talking about generating cache entries, and neither were >> you. I've replied to you because you were suggesting that the "EAPI in >> ebuilds contents" solution had extra cost when _using_ valid cache >> entries (need to extract the EAPI from the ebuild before reading this >> cache entry), which i think can be easily avoided. > > And it does, since EAPI has to be known to use the cache contents. The > way it's handled currently is a hack that doesn't work with future EAPIs > defining new metadata. Fix that, then. And I understand that the code is already there in both portage and pkgcore to store the cache as key-value pairs rather than one-slot-per-key, and would be relatively trivial to add to paludis. Regards, -- Arun Raghavan (http://nemesis.accosted.net) v2sw5Chw4+5ln4pr6$OFck2ma4+9u8w3+1!m?l7+9GSCKi056 e6+9i4b8/9HTAen4+5g4/8APa2Xs8r1/2p5-8 hackerkey.com -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 03:14:47PM +0100, Ciaran McCreesh wrote: > *Then* would be the time to change the extension. As long as the ebuild is bash-parseable with an appropriate environment, it doesn't make sense to change the extension because a env-variable set or a comment are more natural methods. If/when the format changes to something not parseable by bash, then it will be time to change the extension. And then how to mark (sub-)version will depend on the chosen new format, in case of xml that would be the dtd information. I suspect the rejection of the extension change will be there as long as the fundamental format (bash script) doesn't change. OG. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Wed, Jun 11, 2008 at 04:14:58AM +0100, Ciaran McCreesh wrote: > On Tue, 10 Jun 2008 19:14:11 +0200 > Olivier Galibert <[EMAIL PROTECTED]> wrote: > > On Tue, Jun 10, 2008 at 03:02:28PM +0100, Ciaran McCreesh wrote: > > > Except that currently, the ebuild file isn't opened for read. So > > > it's not in memory at all. > > > > Why would you need the EAPI before the time when you actually want to > > interpret the contents? > > You need the EAPI before you use the metadata. But you don't need the > ebuild to get the metadata in the common case. The metadata should carry its version indicator too. OG. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Kelly wrote: | Wrong. Thanks for that. I'm gonna assume you meant "I think you're wrong". | Sure, because of how the algorithm works, people could potentially do | both, but the GLEP makes it pretty clear that they shouldn't. It doesn't just say they shouldn't, it *says what happens if they do*: pkg-4.ebuild-2, EAPI="1" ~pre-source EAPI is 2, post-source EAPI is 1. EAPI 1 is used So to live up to the GLEP specification, package managers must source the ebuild to determine the correct EAPI. | Basically, you don't set the post-source EAPI. It is only supposed to | be used when the pre-source EAPI cannot be determined. If that's what was meant, the GLEP needs changing to say that. Something like: pkg-4.ebuild-2, EAPI="X" ~Pre-source EAPI is 2, post-source EAPI is X. In this instance the post-source EAPI is ignored, since a pre-source EAPI is set, so EAPI 2 is used. It's not a big deal, but it will need a change to the GLEP in its current form, if that's what's meant. Mike 5:) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkhPdRUACgkQu7rWomwgFXq1twCgqnFzBqQI8vl63faZ1cq27MF7 7ekAn0aA73nic1v/ovwAUuHsaL44MrTE =StC2 -END PGP SIGNATURE- -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Wed, 11 Jun 2008 08:31:45 +0200 Thomas de Grenier de Latour <[EMAIL PROTECTED]> wrote: > On 2008/06/11, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > > You're missing the cases where the cache isn't usable. > > I was not talking about generating cache entries, and neither were > you. I've replied to you because you were suggesting that the "EAPI in > ebuilds contents" solution had extra cost when _using_ valid cache > entries (need to extract the EAPI from the ebuild before reading this > cache entry), which i think can be easily avoided. And it does, since EAPI has to be known to use the cache contents. The way it's handled currently is a hack that doesn't work with future EAPIs defining new metadata. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On 2008/06/11, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > You're missing the cases where the cache isn't usable. > I was not talking about generating cache entries, and neither were you. I've replied to you because you were suggesting that the "EAPI in ebuilds contents" solution had extra cost when _using_ valid cache entries (need to extract the EAPI from the ebuild before reading this cache entry), which i think can be easily avoided. On 2008/06/10, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > Currently we don't touch the ebuild's content *at all* for metadata > operations, except where there's no or stale metadata cache (which is > rare). We can get away with this currently because 0 and 1 have > identical cache layouts and PMS has some (necessary) weasel wording; > future EAPIs likely won't, so we're back to the chicken / egg problem. > > So... We either have the EAPI from the extension (which we already > have, since we have to read the dir to know what versions are > available), in which case we know how to read the metadata cache file. > Or we have to open up a file that would otherwise not be opened, just > to parse one line so we know how to read the cache file. -- TGL. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Wed, 11 Jun 2008 09:52:17 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: > On Wed, Jun 11, 2008 at 8:44 AM, Ciaran McCreesh > <[EMAIL PROTECTED]> wrote: > [...] > >> Why would you need the EAPI before the time when you actually want > >> to interpret the contents? > > > > You need the EAPI before you use the metadata. But you don't need > > the ebuild to get the metadata in the common case. > > Does the cache format _really_ need to be extensible the extent that > we're jumping hoops to support arbitrary cache formats? The cache format needs to be able to have keys added and removed from it. But the cache format is largely irrelevant here. There aren't any non-trivial EAPI related problems introduced by cache that don't already exist without cache. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Wed, Jun 11, 2008 at 8:44 AM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] >> Why would you need the EAPI before the time when you actually want to >> interpret the contents? > > You need the EAPI before you use the metadata. But you don't need the > ebuild to get the metadata in the common case. Does the cache format _really_ need to be extensible the extent that we're jumping hoops to support arbitrary cache formats? -- Arun Raghavan (http://nemesis.accosted.net) v2sw5Chw4+5ln4pr6$OFck2ma4+9u8w3+1!m?l7+9GSCKi056 e6+9i4b8/9HTAen4+5g4/8APa2Xs8r1/2p5-8 hackerkey.com -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 19:14:11 +0200 Olivier Galibert <[EMAIL PROTECTED]> wrote: > On Tue, Jun 10, 2008 at 03:02:28PM +0100, Ciaran McCreesh wrote: > > Except that currently, the ebuild file isn't opened for read. So > > it's not in memory at all. > > Why would you need the EAPI before the time when you actually want to > interpret the contents? You need the EAPI before you use the metadata. But you don't need the ebuild to get the metadata in the common case. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Wed, 11 Jun 2008 01:36:01 +0200 Thomas de Grenier de Latour <[EMAIL PROTECTED]> wrote: > Or you apply to future EAPI's cache formats one of the solutions that > have been proposed for the ebuild side of the very same "chicken / egg > problem": for instance, you could use $EAPI as cache filename > extension. > > And that's it, you can get EAPI from the cache, hence no more extra > reading of ebuild files, and no more perfs issues. > > It sounds so simple, am I missing something obvious? You're missing the cases where the cache isn't usable. -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] Re: GLEP 55
Bernd Steinhauser <[EMAIL PROTECTED]> posted [EMAIL PROTECTED], excerpted below, on Tue, 10 Jun 2008 05:46:47 +0200: > No, not really. If you have .txt, .txt-2, .text or .footext in a dir, > you would still realize, that those should be text files. The first three, yes, by long tradition, footext, not necessarily, as it's too ambiguous. Just like the penisland dot com domain, which is read as pen-island and sells pens, BTW (at least last time I looked, after it came up as an example in a discussion similar to this), is that foo-text or foot-ext (the way I would read it by default) or foote-XT or...? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 2008/06/10, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > Currently we don't touch the ebuild's content *at all* for metadata > operations, except where there's no or stale metadata cache (which is > rare). We can get away with this currently because 0 and 1 have > identical cache layouts and PMS has some (necessary) weasel wording; > future EAPIs likely won't, so we're back to the chicken / egg problem. > > So... We either have the EAPI from the extension (which we already > have, since we have to read the dir to know what versions are > available), in which case we know how to read the metadata cache file. > Or we have to open up a file that would otherwise not be opened, just > to parse one line so we know how to read the cache file. Or you apply to future EAPI's cache formats one of the solutions that have been proposed for the ebuild side of the very same "chicken / egg problem": for instance, you could use $EAPI as cache filename extension. And that's it, you can get EAPI from the cache, hence no more extra reading of ebuild files, and no more perfs issues. It sounds so simple, am I missing something obvious? -- TGL. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Richard Freeman wrote:
> On the other hand, this is a big change from the present, and I'm not
> convinced that it will actually be a big improvement over some of the
> other EAPI ideas being floated around. However, it is a
> potentially-neat idea...
Rich, interesting thoughts! But yeah, I agree that for now that is a lot.
My idea for the "#!" thing was much smaller-scale and is a way to add simple
syntax to allow declaration of the EAPI in the file with no sourcing and no
filename extension mangling (plus, no pre-source/post-source issues):
Just have one "shebang-like" line in the header before any real bash code
("#!EAPI=3", "#EAPI=3", or whatever is agreed-upon). This is out-of-band meta
info, so it's OK that it's in a "comment". It's ignored by bash, but it can be
read trivially without sourcing. To accelerate things for the tree (I've seen
others mention this idea too), store the EAPI in the portage cache when it is
generated.
-Joe
--
gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Santiago M. Mola wrote: On Tue, Jun 10, 2008 at 10:20 PM, Federico Ferri <[EMAIL PROTECTED]> wrote: The so-called shebang; very good in my opinion! Works very well for true shell scripts. why it can't work for ebuilds? This option was already discussed when GLEP 55 was proposed, and in my opinion it's totally discarded. The concept of shebang doesn't apply here at all. Plus /usr/bin/ebuild is a binary provided by portage which has nothing to do with the process of sourcing ebuilds nor even with the internals of portage (not to speak of other package managers). I had thought of this option as well - I agree that it has a lot of issues. I agree that ebuild wouldn't be the right tool, but in the right framework I could see how something like this might work. Here is how a portage tree might be handled by a future package manager that uses shebangs to implement any number of EAPIs that have almost no restrictions on file contents other than the shebang: When new "ebuilds" are synced into the repository, they are executed. If the interpreter doesn't exist, then it is an unsupported EAPI and the failure is silently ignored (or logged or whatever). If the interpreter does exist it will interpret the file properly - perhaps using a EAPI command-line option from the shebang line or whatever is appropriate for that EAPI which the interpreter obviously understands. If the interpreter determines that the file uses an EAPI it doesn't know, it aborts execution and skips the package. That makes the scheme always backwards compatible (well, at least until the switch to this approach - current package managers wouldn't be compatible). When the new ebuild is executed, it will call appropriate functions to register itself into the local repository. That registration might include callbacks of some kind or whatever - the whole way the ebuild works could vary by EAPI - since the interpreter used by the EAPI could change. The only restriction for compatibility is a standard shebang line. Each ebuild would only be executed upon a change when it is synced, so the overhead isn't huge. At that point all the data is stored in a local cache and the ebuilds themselves don't get touched until they are installed. Installation will work in an EAPI-dependent way - perhaps by executing the ebuild with a particular parameter or environment setting or whatever. The package manager will know since it has already been established that this is a supported EAPI by the fact that the ebuild got processed when it was synced in. This kind of system also makes it easy to support some scenarios that are currently hard: 1. Download a random ebuild off the internet and want to add it to your repository? Rather than having to put it in a local portage tree you could just execute it from anywhere and it would register itself in its present location - and act like any other package in the tree. 2. No need for utilities like ebuild to manipulate the install process - instead just execute the ebuild with the appropriate parameters or whatever to get it to do the install process. On the other hand, this is a big change from the present, and I'm not convinced that it will actually be a big improvement over some of the other EAPI ideas being floated around. However, it is a potentially-neat idea... -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Mike Auty wrote:
The speed issues aren't really a concern, since the GLEP suggests that
the ebuild must be sourced anyway. The GLEP allows for a .ebuild-2 file
to contain EAPI="1".
Wrong. From the GLEP: "note that one should *never* explicitly set both
EAPIs to different values."
Basically, you don't set the post-source EAPI. It is only supposed to be
used when the pre-source EAPI cannot be determined. This is entirely for
backwards compatability with EAPI={0,1}. Once people start using
suffixed EAPIs, EAPI should not (probably "must not") be set in the
ebuild. Sure, because of how the algorithm works, people could
potentially do both, but the GLEP makes it pretty clear that they shouldn't.
--
Mike Kelly
--
gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 10:20 PM, Federico Ferri <[EMAIL PROTECTED]> wrote: > > The so-called shebang; very good in my opinion! > > Works very well for true shell scripts. why it can't work for ebuilds? This option was already discussed when GLEP 55 was proposed, and in my opinion it's totally discarded. The concept of shebang doesn't apply here at all. Plus /usr/bin/ebuild is a binary provided by portage which has nothing to do with the process of sourcing ebuilds nor even with the internals of portage (not to speak of other package managers). Regards, -- Santiago M. Mola Jabber ID: [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Joe Peterson ha scritto:
It was mentioned that "comments are to be ignored", but you point out a
perfect and very fundamental example of where this is not true:
#!/usr/bin/env bash
Putting another line close to this one with:
#EAPI=42
or
#!EAPI=42
if you like (conforms more to the shell script specifier), is not too
muchh of a stretch.
The so-called shebang; very good in my opinion!
Works very well for true shell scripts. why it can't work for ebuilds?
First, I think of two alternatives:
1) the interpreter is the EAPI (with /usr/bin/ebuild being EAPI=0)
this will be the first line of a EAPI=0 ebuild:
#!/usr/bin/ebuild
this will be a EAPI=paludis ebuild:
#!/usr/bin/ebuild-paludis
apparently it's the same mechanism the shell uses to execute a shell
script, except for the fact that:
* ebuild is not an executable file
* an external program is used to determine the EAPI (i.e.: extract the
shebang)
2) fixed interpreter name, use arguments for switching EAPIs
this is another option, wich carries itself a GREAT power of leaving an
open door when in the future will happen something similar to what we
have now, where switching the EAPI (or replace EAPI with something else)
is critical and breaks compatibility.
example - EAPI=0:
#!/usr/bin/ebuild
...
another eapi:
#!/usr/bin/ebuild -eapi paludis
This allows virtually unlimited cases similar to this; for example when
switching to , another switch will
be used:
#!/usr/bin/ebuild -eapi foo -fluffy bar
Advantages of both solutions:
* easy to parse
* doesn't break compatibility
What it does not address:
* doesn't break compatibility
In fact, it should break compatibility.
again I can think of two solutions:
1) change the ebuild extension to a different value, once and
(hopefully) forever, e.g. from .ebuild to .xbuild, and leave the minimal
set of .ebuild-s for the upgrade path
2) use two repositories: the legacy one, needed for the upgrade path,
and the shyny-new-repository with the new ebuilds format
best regards,
Federico Ferri
--
#include
main(){printf("%x",4275974592);}
-BEGIN GEEK CODE BLOCK-
Version: 3.12
GCS d-(--) a- C++$ UL+++$ L+++$ E--- W++@ w--@
PE PGP+ R- tv-- DI++>+++ D++ h+
--END GEEK CODE BLOCK--
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] Re: GLEP 55
Olivier Galibert a écrit : On Tue, Jun 10, 2008 at 12:01:00PM +0200, Rémi Cardona wrote: Ciaran McCreesh a écrit : Kills the upgrade path completely. No good. Lemme sum this up in layman's terms : 1) EAPI _has_ to be known before sourcing an ebuild. There's no way to avoid that for various reasons, all 100% valid. "sourcing" != "reading the first line/n first lines/first block and grepping". IO-wise, it's the same, as most ebuilds fit inside a kernel memory page (ie, 4KB on most arches). So with a cold cache, the time required to actually run bash to source the ebuild won't matter much. But we're getting lost on this thread, let's let it cool off for a little while :) Cheers, Rémi -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 03:02:28PM +0100, Ciaran McCreesh wrote: > Except that currently, the ebuild file isn't opened for read. So it's > not in memory at all. Why would you need the EAPI before the time when you actually want to interpret the contents? OG. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 12:01:00PM +0200, Rémi Cardona wrote: > Ciaran McCreesh a écrit : > >Kills the upgrade path completely. No good. > > Lemme sum this up in layman's terms : > > 1) EAPI _has_ to be known before sourcing an ebuild. There's no way to > avoid that for various reasons, all 100% valid. "sourcing" != "reading the first line/n first lines/first block and grepping". OG. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ciaran McCreesh wrote: | No, it results in a new open() on a file that's elsewhere on disk, which | results in two new seeks. You get about fifty seeks per second. The speed issues aren't really a concern, since the GLEP suggests that the ebuild must be sourced anyway. The GLEP allows for a .ebuild-2 file to contain EAPI="1". This requires the package manager to source the ebuild to find out exactly what EAPI should be being used. The GLEP doesn't strictly outline what happens to a .ebuild-1 containing EAPI="2" for a PM that supports EAPI="2" (which needs to be addressed before the GLEP gets put into action). It does say a developer should not specify both a pre- and post- source EAPI, but the GLEP says that if both exist the post-source one is used. ~ That means all package managers would have to source the ebuilds. Personally, as a developer, I don't want to be messing around with yet more numbers in the ebuild filenames, and I think it looks ugly. Not great arguments, granted. It also feels like a bad idea to separate information required to process the contents of the file from the contents of the file itself. P/PN/PV variables are used in clearly defined locations of the file, and the script could run under a different name and version (as proved by every version bump around). The suggestion seems to be that an ebuild could not run under a different EAPI. In that case, the EAPI version should be embedded in the contents of the file. As people have pointed out though, there's no need to rush this decision. We're simply not going to achieve backwards compatibility forever (we haven't in the past), so why not choose a time period to allow for upgrades (6 months, a year?) and implement a new EAPI definition mechanism that's present in the file and offers a slightly better compromise between the various parties than the current suggestion? It will require a little more patience, but it offers time for a thought out and well designed choice. What's the rush? Mike 5:) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkhOpDEACgkQu7rWomwgFXoFvgCeMxSfRiQLEZr3QyT+Bx4b5aNe /9EAnicrcCQTXxsliAeM4mdisgjO7abg =tGD8 -END PGP SIGNATURE- -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 11:08:21 -0400 Richard Freeman <[EMAIL PROTECTED]> wrote: > I'm curious as to what operation in particular we're looking at. > Let's say I type in "paludis --sync": paludis --sync doesn't use metadata. > Next, suppose I type in "paludis -pi world": If it's straight after a --sync, then yes, some things are pre-loaded by rsync. Similarly, if it's straight after other operations, then yes, some things are pre-loaded. But we don't plan for "just the use cases that make our life easy". > Why wouldn't everything you need not already be in the cache? And if > something wasn't, then why is reading in the EAPI any slower than > reading in (R)DEPEND/KEYWORDS/IUSE/etc? Currently we don't touch the ebuild's content *at all* for metadata operations, except where there's no or stale metadata cache (which is rare). We can get away with this currently because 0 and 1 have identical cache layouts and PMS has some (necessary) weasel wording; future EAPIs likely won't, so we're back to the chicken / egg problem. So... We either have the EAPI from the extension (which we already have, since we have to read the dir to know what versions are available), in which case we know how to read the metadata cache file. Or we have to open up a file that would otherwise not be opened, just to parse one line so we know how to read the cache file. > What specific operation (from an end-user perspective) is improved by > putting EAPI in the filename? I'm not interested in theoretical > operations like "given a portage tree, find the EAPI of every file" - > users don't do those operations routinely in isolation, but as part > of larger operations where doing an open and a getline now just > speeds up the open and getline that would be executed 500 lines later > anyway. paludis -pi anything on a cold cache. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh wrote: On Tue, 10 Jun 2008 19:40:22 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: On Tue, Jun 10, 2008 at 7:32 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] I don't think that filename-vs-first-line is going to make a big difference in practical performance. It's about a factor of five difference in cold-cache resolution performance for Paludis. Could you please share more details on the experiment that showed this kind of performance degradation and the numbers, if possible? Shove an open and a getline in when doing what would otherwise be a successful cache read. It adds a couple of thousand new open()s on files that would otherwise be left alone, and changes the nice linear "slurp up things in this directory in order until we don't need anything else" pattern into "bounce backwards and forwards between lots of files in two different directories". On a cold cache, which is the most common use case, this is very very nasty. I'm curious as to what operation in particular we're looking at. Let's say I type in "paludis --sync": Obviously the first step is to rsync the portage tree. Then we find all modified files (already output by rsync) and update the caches. We already need to fully parse every file to create a dependency tree, so why is it slow to cache the EAPI for each file while we're at it? Next, suppose I type in "paludis -pi world": Why wouldn't everything you need not already be in the cache? And if something wasn't, then why is reading in the EAPI any slower than reading in (R)DEPEND/KEYWORDS/IUSE/etc? What specific operation (from an end-user perspective) is improved by putting EAPI in the filename? I'm not interested in theoretical operations like "given a portage tree, find the EAPI of every file" - users don't do those operations routinely in isolation, but as part of larger operations where doing an open and a getline now just speeds up the open and getline that would be executed 500 lines later anyway. Again, I'm not completely opposed to the idea of putting EAPI in the filename, but I'm not convinced that the arguments in favor of it are as clear-cut as some are suggesting. If everybody was open about the pros/cons of each solution and not suggesting that one solution is near-perfect and the others are total non-starters then this whole debate would probably be a whole lot less contentious. When people perceive that somebody isn't honest about the shortcomings of their own proposal then they're less likely to trust them - it is just a human thing... -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 10 Jun 2008, at 16:14, Ciaran McCreesh wrote: On Tue, 10 Jun 2008 19:38:52 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: On Tue, Jun 10, 2008 at 7:30 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] - it doubles the number of file reads necessary during resolution. The first read will cause the file to be cached for subsequent reads anyway, so the performance hit boils down to an additional read() call (which will probably be buffered by your file I/O library anyway, so it's unlikely to even result in a context switch). And even without, it is well worth the lack of fugliness in the ebuild name. No, it results in a new open() on a file that's elsewhere on disk, which results in two new seeks. You get about fifty seeks per second. Plus path resolution, which isn't exactly free - ferdy -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh a écrit : No no. Doing the seek to open a file in a different directory and then seeking back to your original directory over and over when otherwise you'd be doing nice linear opens on adjacent inodes in a single directory is where the performance hit is. Paludis is pretty much seek bound in a lot of cases. Ok It's moving the relevant information further and further away from where it's supposed to be. It doubles the number of files a developer has to check in order to do simple ebuild maintenance. I said one file per package, not per ebuild. It only doubles the number of files if there's only one ebuild in a given package :) -- Rémi Cardona LRI, INRIA [EMAIL PROTECTED] [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 19:54:33 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: > Well, most file systems have a local structure for this data (=> block > group), so it's not going to be a seek that's very far. Secondly, how > many ebuilds do you need to read directly to get this data in a > typical case? Isn't this what the metadata cache is for? It's a nice local structure when you're just using a) a nice, linear, readahead-friendly-ish slurp of the ebuild directory and b) a nice, linear slurp of files in the metadata cache, which is how things are now. When you move that to having to make alternating use of the metadata cache and the ebuild files, you're suddenly seeking backwards and forwards between two places over and over. > >> > - it heavily restricts future syntax and meaning of EAPIs > >> > >> Not by much. It's just a header. > > > > > > Do we want to keep the spec so wide open that we support any format > under the Sun that we fancy? Seems like overgeneralizing to me. We want to keep it open enough that we're not going to have to go through yet another backwards compatibility breaking change. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 16:11:49 +0200 Rémi Cardona <[EMAIL PROTECTED]> wrote: > Ciaran McCreesh a écrit : > > The package manager does not currently source the whole thing with > > bash to get the EAPI, nor does it open the ebuild file at all for > > metadata. You're talking doubling the number of file operations > > here, and going from extremely good filesystem locality (which > > means very few seeks) to jumping backwards and forwards all over > > the place. > > Yeah, grepping through the ebuild is just bound to get a massive > performance hit, I don't have numbers but that just looks obvious. No no. Doing the seek to open a file in a different directory and then seeking back to your original directory over and over when otherwise you'd be doing nice linear opens on adjacent inodes in a single directory is where the performance hit is. Paludis is pretty much seek bound in a lot of cases. > Ciaran, what other pitfalls do you see for using one EAPI file per > package (except for the broken compatibility, I know it's a big one) ? It's moving the relevant information further and further away from where it's supposed to be. It doubles the number of files a developer has to check in order to do simple ebuild maintenance. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 19:40:22 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: > On Tue, Jun 10, 2008 at 7:32 PM, Ciaran McCreesh > <[EMAIL PROTECTED]> wrote: > [...] > >> I don't think that filename-vs-first-line is going to make a big > >> difference in practical performance. > > > > It's about a factor of five difference in cold-cache resolution > > performance for Paludis. > > Could you please share more details on the experiment that showed this > kind of performance degradation and the numbers, if possible? Shove an open and a getline in when doing what would otherwise be a successful cache read. It adds a couple of thousand new open()s on files that would otherwise be left alone, and changes the nice linear "slurp up things in this directory in order until we don't need anything else" pattern into "bounce backwards and forwards between lots of files in two different directories". On a cold cache, which is the most common use case, this is very very nasty. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 7:44 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] >> The first read will cause the file to be cached for subsequent reads >> anyway, so the performance hit boils down to an additional read() call >> (which will probably be buffered by your file I/O library anyway, so >> it's unlikely to even result in a context switch). And even without, >> it is well worth the lack of fugliness in the ebuild name. > > No, it results in a new open() on a file that's elsewhere on disk, which > results in two new seeks. You get about fifty seeks per second. Well, most file systems have a local structure for this data (=> block group), so it's not going to be a seek that's very far. Secondly, how many ebuilds do you need to read directly to get this data in a typical case? Isn't this what the metadata cache is for? >> > - it heavily restricts future syntax and meaning of EAPIs >> >> Not by much. It's just a header. > > Do we want to keep the spec so wide open that we support any format under the Sun that we fancy? Seems like overgeneralizing to me. Regards, -- Arun Raghavan (http://nemesis.accosted.net) v2sw5Chw4+5ln4pr6$OFck2ma4+9u8w3+1!m?l7+9GSCKi056 e6+9i4b8/9HTAen4+5g4/8APa2Xs8r1/2p5-8 hackerkey.com -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Richard Freeman wrote: > Some object to parsing out the EAPI without sourcing the ebuild (only > bash can source bash). I disagree with this argument - every time you > run a shell script it is sourced by something other than bash - the > kernel has to figure out what script interpreter to use by parsing the > first line. There is no reason we can't use a magic number in the same > way with the EAPI. That isn't reason enough on its own to put the EAPI > in the filename, but it is a start. +1 It was mentioned that "comments are to be ignored", but you point out a perfect and very fundamental example of where this is not true: #!/usr/bin/env bash Putting another line close to this one with: #EAPI=42 or #!EAPI=42 if you like (conforms more to the shell script specifier), is not too muchh of a stretch. > Most software packages store version information internal to a file > format. I'm actually not aware of many that put it in the filename. Only a few, mainly Windows, I believe. Like .WSn (as pointed out on the Filename_extension wikipedia page). But oddballs like this suggest to me that a hack had to be done because the version could not be gleaned in a more subtle way from the file itself (e.g. MS Word does this transparently - all are ".doc"). -Joe -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 19:38:52 +0530 "Arun Raghavan" <[EMAIL PROTECTED]> wrote: > On Tue, Jun 10, 2008 at 7:30 PM, Ciaran McCreesh > <[EMAIL PROTECTED]> wrote: > [...] > > - it doubles the number of file reads necessary during resolution. > > The first read will cause the file to be cached for subsequent reads > anyway, so the performance hit boils down to an additional read() call > (which will probably be buffered by your file I/O library anyway, so > it's unlikely to even result in a context switch). And even without, > it is well worth the lack of fugliness in the ebuild name. No, it results in a new open() on a file that's elsewhere on disk, which results in two new seeks. You get about fifty seeks per second. > > - it heavily restricts future syntax and meaning of EAPIs > > Not by much. It's just a header. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh a écrit : The package manager does not currently source the whole thing with bash to get the EAPI, nor does it open the ebuild file at all for metadata. You're talking doubling the number of file operations here, and going from extremely good filesystem locality (which means very few seeks) to jumping backwards and forwards all over the place. Yeah, grepping through the ebuild is just bound to get a massive performance hit, I don't have numbers but that just looks obvious. Ciaran, what other pitfalls do you see for using one EAPI file per package (except for the broken compatibility, I know it's a big one) ? This is basic stuff you really need to know before you can comment sensibly on a discussion about package metadata. Then, thanks for explaining those things :) We are learning stuff as we go. -- Rémi Cardona LRI, INRIA [EMAIL PROTECTED] [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Fernando J. Pereda wrote: On 10 Jun 2008, at 15:48, Luca Barbato wrote: Fernando J. Pereda wrote: No, it doesn't make parsing faster. *Had you bothered to profile any package manager you'd know that.* Do you have any number to share? What number are you interested in? Profiling numbers... lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 7:32 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] >> I don't think that filename-vs-first-line is going to make a big >> difference in practical performance. > > It's about a factor of five difference in cold-cache resolution > performance for Paludis. Could you please share more details on the experiment that showed this kind of performance degradation and the numbers, if possible? Thanks, -- Arun Raghavan (http://nemesis.accosted.net) v2sw5Chw4+5ln4pr6$OFck2ma4+9u8w3+1!m?l7+9GSCKi056 e6+9i4b8/9HTAen4+5g4/8APa2Xs8r1/2p5-8 hackerkey.com -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, Jun 10, 2008 at 7:30 PM, Ciaran McCreesh <[EMAIL PROTECTED]> wrote: [...] > - it doubles the number of file reads necessary during resolution. The first read will cause the file to be cached for subsequent reads anyway, so the performance hit boils down to an additional read() call (which will probably be buffered by your file I/O library anyway, so it's unlikely to even result in a context switch). And even without, it is well worth the lack of fugliness in the ebuild name. > - it heavily restricts future syntax and meaning of EAPIs Not by much. It's just a header. > - it makes comments have meaning Just as much as #!/bin/bash and # vim: ... do Regards, -- Arun Raghavan (http://nemesis.accosted.net) v2sw5Chw4+5ln4pr6$OFck2ma4+9u8w3+1!m?l7+9GSCKi056 e6+9i4b8/9HTAen4+5g4/8APa2Xs8r1/2p5-8 hackerkey.com -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 09:56:18 -0400 Richard Freeman <[EMAIL PROTECTED]> wrote: > Joe Peterson wrote: > > No, I have not profiled PMs to try this, but you are saying that > > reading the first few lines of a file is not faster than sourcing > > the whole thing with bash? Remember that it could abort the minute > > it sees a non '#' or blank line, which would be after the first few. > > > > Actually, if you specified that the EAPI goes on the first line, then > you'd only need to read the first line. > > And to the extent that the rest got read into memory anyway by the > kernel, well then it is already in the cache when whatever mechanism > is invoked to parse the rest of the file. Except that currently, the ebuild file isn't opened for read. So it's not in memory at all. > I don't think that filename-vs-first-line is going to make a big > difference in practical performance. It's about a factor of five difference in cold-cache resolution performance for Paludis. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 09:49:04 -0400 Richard Freeman <[EMAIL PROTECTED]> wrote: > 4) Putting EAPI inside the ebuild, but in a manner that does not > require sourcing using bash (ie comment at top of file). > > + it solves 1) > + it keeps pretty file names > + syntax/implementation is trivial > - it breaks backwards compatibility (eventually - hacks might delay > this) > - it does force future ebuilds to have the EAPI line in it - it doubles the number of file reads necessary during resolution. - it heavily restricts future syntax and meaning of EAPIs - it makes comments have meaning > Most software packages store version information internal to a file > format. I'm actually not aware of many that put it in the filename. Most software doesn't have to care about backwards / forwards compatibility. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 07:49:31 -0600 Joe Peterson <[EMAIL PROTECTED]> wrote: > > No, it doesn't make parsing faster. Had you bothered to profile > > any package manager you'd know that. > > No, I have not profiled PMs to try this, but you are saying that > reading the first few lines of a file is not faster than sourcing the > whole thing with bash? Remember that it could abort the minute it > sees a non '#' or blank line, which would be after the first few. The package manager does not currently source the whole thing with bash to get the EAPI, nor does it open the ebuild file at all for metadata. You're talking doubling the number of file operations here, and going from extremely good filesystem locality (which means very few seeks) to jumping backwards and forwards all over the place. This is basic stuff you really need to know before you can comment sensibly on a discussion about package metadata. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
Joe Peterson wrote: No, I have not profiled PMs to try this, but you are saying that reading the first few lines of a file is not faster than sourcing the whole thing with bash? Remember that it could abort the minute it sees a non '#' or blank line, which would be after the first few. Actually, if you specified that the EAPI goes on the first line, then you'd only need to read the first line. And to the extent that the rest got read into memory anyway by the kernel, well then it is already in the cache when whatever mechanism is invoked to parse the rest of the file. I don't think that filename-vs-first-line is going to make a big difference in practical performance. Any time lost in determining EAPI would be time saved in parsing the file. And even though you could ignore unknown EAPIs, I think that in a typical case users would be using an up-to-date package manager that would just end up parsing everything all the time anyway. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 10 Jun 2008, at 15:48, Luca Barbato wrote: Fernando J. Pereda wrote: No, it doesn't make parsing faster. Had you bothered to profile any package manager you'd know that. Do you have any number to share? What number are you interested in? - ferdy -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Rémi Cardona wrote: Ciaran McCreesh a écrit : Kills the upgrade path completely. No good. Lemme sum this up in layman's terms : 1) EAPI _has_ to be known before sourcing an ebuild. There's no way to avoid that for various reasons, all 100% valid. 2) Putting the EAPI in the filename : + it solves 1) + it keeps backward compatibility because old PM won't recognize the filenames - it's not very "pretty" 3) Putting the EAPI in metadata.xml or in another external file + it solves 1) + it keeps pretty file names - it breaks backwards compatibility - (specific to metadata.xml) PM will have to learn to read XML (yuck) That's about it, right? You missed some options - I'll try to be honest about their pros/cons: 4) Putting EAPI inside the ebuild, but in a manner that does not require sourcing using bash (ie comment at top of file). + it solves 1) + it keeps pretty file names + syntax/implementation is trivial - it breaks backwards compatibility (eventually - hacks might delay this) - it does force future ebuilds to have the EAPI line in it 5) Putting EAPI inside the ebuild as in #4, but do a one-time-only change of the filenames. + it solves 1) + it sort-of-keeps pretty file names (two pretty extensions instead of one) + syntax/implementation is trivial + it preserves backwards compatibility - it does force future ebuilds to have the EAPI line in it 6) Putting EAPI inside the ebuild as in #4, and do a one-time rsync location change + it solves 1) + the filenames don't change at all + syntax is trivial, future ebuilds are trivial + it preserves backwards compatibility - it does force future ebuilds to have the EAPI line in it - it seems like quite a hack - and you end up with a dummy portage tree sitting out there for some period of time Personally, I tend to lean towards either just putting the EAPI in the filename, or putting it in the first line and breaking backwards compatibility. The whole reason we're in this mess is the ebuild file format does not include a version field or the equivalent of extensible headers used in other sorts of files (tar, etc). We've certainly broken backwards-compatibility before with other changes within the distro. All we need to do is make it easy to update to a package manager that fixes the break and post instructions prominently, and make sure that package managers support the new process for a few months before taking advantage of it. Some object to parsing out the EAPI without sourcing the ebuild (only bash can source bash). I disagree with this argument - every time you run a shell script it is sourced by something other than bash - the kernel has to figure out what script interpreter to use by parsing the first line. There is no reason we can't use a magic number in the same way with the EAPI. That isn't reason enough on its own to put the EAPI in the filename, but it is a start. Most software packages store version information internal to a file format. I'm actually not aware of many that put it in the filename. -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 10 Jun 2008, at 15:46, Joe Peterson wrote: Also, I'm not sure reading XML is a problem at all - python has good libs for this already. Reading XML files is easy, but it makes certain codepaths much much slower. Not a good 'feature'. - ferdy -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Fernando J. Pereda wrote: > On 10 Jun 2008, at 15:33, Joe Peterson wrote: > >> Luca Barbato wrote: >>> Check if exists a line EAPI=*$, if does and the rest of the string >>> matches an understood eapi, go on sourcing, otherwise ignore/mask >>> it... >> And placing it out-of-band (like "# EAPI=...") avoids any sourcing >> errors, makes parsing faster, etc. > > No, it doesn't make parsing faster. Had you bothered to profile any > package manager you'd know that. No, I have not profiled PMs to try this, but you are saying that reading the first few lines of a file is not faster than sourcing the whole thing with bash? Remember that it could abort the minute it sees a non '#' or blank line, which would be after the first few. -Joe -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Fernando J. Pereda wrote: No, it doesn't make parsing faster. Had you bothered to profile any package manager you'd know that. Do you have any number to share? lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Rémi Cardona wrote: > Ciaran McCreesh a écrit : >> Kills the upgrade path completely. No good. > > Lemme sum this up in layman's terms : > > 1) EAPI _has_ to be known before sourcing an ebuild. There's no way to > avoid that for various reasons, all 100% valid. > > 2) Putting the EAPI in the filename : > > + it solves 1) > + it keeps backward compatibility because old PM won't recognize the > filenames > - it's not very "pretty" I'd say the problems go way beyond just being not pretty. That longish email I wrote yesterday has a bunch of reasons I don't like it. And "pretty" makes the issue sound unimportant or superficial. > 3) Putting the EAPI in metadata.xml or in another external file > > + it solves 1) > + it keeps pretty file names > - it breaks backwards compatibility > - (specific to metadata.xml) PM will have to learn to read XML (yuck) > > That's about it, right? Good summary, except I think we can find ways to deal with compatibility (several ideas have been put forth already: giving time for PM to be upgraded, or a one-time tree or extension bump - and I'm sure there are even better ones yet to be discussed). I do not believe that the filename mangling solution is the "one and only way" as some people are insisting. Also, I'm not sure reading XML is a problem at all - python has good libs for this already. -Joe -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 10 Jun 2008, at 15:33, Joe Peterson wrote: Luca Barbato wrote: Check if exists a line EAPI=*$, if does and the rest of the string matches an understood eapi, go on sourcing, otherwise ignore/mask it... And placing it out-of-band (like "# EAPI=...") avoids any sourcing errors, makes parsing faster, etc. No, it doesn't make parsing faster. Had you bothered to profile any package manager you'd know that. - ferdy -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Luca Barbato wrote: > Check if exists a line EAPI=*$, if does and the rest of the string > matches an understood eapi, go on sourcing, otherwise ignore/mask it... And placing it out-of-band (like "# EAPI=...") avoids any sourcing errors, makes parsing faster, etc. -Joe -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Rémi Cardona wrote: > Ciaran McCreesh a écrit : >> picard_facepalm.jpg > > I don't think any of us are completely thrilled by either proposals, but > the EAPI-in-a-separate-file does have the potential for more > flexibility, ie package-wide EAPI. > > And it does keep filenames simple enough. +1 -Joe -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Tiziano Müller wrote:
>>> And then how do we deal with EAPI 3, where the syntax changes again?
>> Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
>> from there. If you change the way you declare EAPI each time, yeah,
>> that's a problem, but I'm not sure why that would ne necessary.
> No, that is not the problem.
>
> Example:
> In EAPI 42 we define that the package manager must provide a global function
> extract_depend_from_setup_py() such that it is callable at a global level
> in an ebuild like this
>
> *snip start*
>
> # Copyright 1999-2007 Gentoo Foundation
> # Distributed under the terms of the GNU General Public License v2
> # $Header: $
>
> EAPI=42
>
> DESCRIPTION="A library aiming to support agile and test-driven python
> development on various levels."
> SRC_URI="http://codespeak.net/download/${PN}/${P}.tar.gz";
> HOMEPAGE="http://codespeak.net/py/";
> KEYWORDS="~amd64 ~x86"
> SLOT="0"
> LICENSE="MIT"
> IUSE=""
>
> extract_depend_from_setup_py
>
> *snip end*
>
> Now, a package manager (or a tool) not knowing EAPI 42 will fail when it
> tries to source the above ebuild to determine the EAPI version (as it is
> being currently done as far as I understood it) because
> extract_depend_from_setup_py is undefined.
> So it won't even be able to read out the EAPI.
>
> With the EAPI in the filename tools now knowing EAPI-42 will either ignore
> the above foo-1.0.ebuild-42 or mask it because they may identify the
> EAPI-version without sourcing the ebuild.
>
> And: No, just sourcing the ebuild and simply masking it because we can't
> source it is a no-go (since you're abusing error-handling as a case
> switch).
Agreed, and that's why I like the out-of-band solution better (i.e.
header line with "# EAPI=42". No need for mangled filenames, and no
nead to actually source.
-Joe
--
gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On 10 Jun 2008, at 12:30, Rémi Cardona wrote: Ciaran McCreesh a écrit : picard_facepalm.jpg I don't think any of us are completely thrilled by either proposals, but the EAPI-in-a-separate-file does have the potential for more flexibility, ie package-wide EAPI. And it does keep filenames simple enough. And it is not backwards compatible. - ferdy-- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh a écrit : picard_facepalm.jpg I don't think any of us are completely thrilled by either proposals, but the EAPI-in-a-separate-file does have the potential for more flexibility, ie package-wide EAPI. And it does keep filenames simple enough. -- Rémi Cardona LRI, INRIA [EMAIL PROTECTED] [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh a écrit : Kills the upgrade path completely. No good. Lemme sum this up in layman's terms : 1) EAPI _has_ to be known before sourcing an ebuild. There's no way to avoid that for various reasons, all 100% valid. 2) Putting the EAPI in the filename : + it solves 1) + it keeps backward compatibility because old PM won't recognize the filenames - it's not very "pretty" 3) Putting the EAPI in metadata.xml or in another external file + it solves 1) + it keeps pretty file names - it breaks backwards compatibility - (specific to metadata.xml) PM will have to learn to read XML (yuck) That's about it, right? -- Rémi Cardona LRI, INRIA [EMAIL PROTECTED] [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 11:40:35 +0200 Luca Barbato <[EMAIL PROTECTED]> wrote: > Ciaran McCreesh wrote: > > Kills the upgrade path completely. No good. > > Not really, you change the rsync paths and older portage will pick a > repo that just has the necessary to upgrade to the next portage. > > This kind of things would work better using an scm supporting > branches and tags a bit better. picard_facepalm.jpg -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP 55
Tiziano Müller wrote:
Joe Peterson wrote:
Ciaran McCreesh wrote:
And a file extension is far less obscurely complex than enforcing
arbitrary syntax restrictions upon ebuilds.
I disagree. One is exposed to devs only as ebuild syntax; the other is
exposed in an inappropriate location to everyone looking at the portage
tree.
No it can't. EAPI has to be known before the source can start. Bash
doesn't provide traps for executing code upon changed variables.
Doing it out-of-band solve this.
No, it's only needed once per non-trivial change. So we might as well
just change it for every EAPI.
Huh? If the "new" portage knows how to determine the EAPI definitively
(and that would be defined), it can deal with the differences.
And then how do we deal with EAPI 3, where the syntax changes again?
Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
from there. If you change the way you declare EAPI each time, yeah,
that's a problem, but I'm not sure why that would ne necessary.
No, that is not the problem.
Example:
In EAPI 42 we define that the package manager must provide a global function
extract_depend_from_setup_py() such that it is callable at a global level
in an ebuild like this
*snip start*
# Copyright 1999-2007 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
EAPI=42
DESCRIPTION="A library aiming to support agile and test-driven python
development on various levels."
SRC_URI="http://codespeak.net/download/${PN}/${P}.tar.gz";
HOMEPAGE="http://codespeak.net/py/";
KEYWORDS="~amd64 ~x86"
SLOT="0"
LICENSE="MIT"
IUSE=""
extract_depend_from_setup_py
*snip end*
Now, a package manager (or a tool) not knowing EAPI 42 will fail when it
tries to source the above ebuild to determine the EAPI version (as it is
being currently done as far as I understood it) because
extract_depend_from_setup_py is undefined.
So it won't even be able to read out the EAPI.
With the EAPI in the filename tools now knowing EAPI-42 will either ignore
the above foo-1.0.ebuild-42 or mask it because they may identify the
EAPI-version without sourcing the ebuild.
Check if exists a line EAPI=*$, if does and the rest of the string
matches an understood eapi, go on sourcing, otherwise ignore/mask it...
lu
--
Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero
--
gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Ciaran McCreesh wrote: Kills the upgrade path completely. No good. Not really, you change the rsync paths and older portage will pick a repo that just has the necessary to upgrade to the next portage. This kind of things would work better using an scm supporting branches and tags a bit better. lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
Tiziano Müller wrote: Another ugly solution: Having the EAPI on a per-package (like $portagedir/cat/package-1) or per-tree basis ($portagedir/profiles/eapi) I like the per tree basis and I already asked about that (since makes things clearer and older portage can be tricked by rsync. lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: GLEP 55
On Tue, 10 Jun 2008 10:33:34 +0200 Tiziano Müller <[EMAIL PROTECTED]> wrote: > Another ugly solution: Having the EAPI on a per-package (like > $portagedir/cat/package-1) or per-tree basis > ($portagedir/profiles/eapi) and start providing our tree as overlays > of more than one tree (will end up in a mess of dependencies, but it > would still be nice to specify the EAPI for a complete overlay > instead of having the name all the ebuilds like .eapi-X :-). > In addition: it wouldn't be possible to identify the EAPI of an > ebuild by just looking at it... Kills the upgrade path completely. No good. -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] Re: GLEP 55
Ciaran McCreesh wrote: > On Mon, 9 Jun 2008 22:35:25 -0700 > Donnie Berkholz <[EMAIL PROTECTED]> wrote: >> Did anyone already propose specifying this in metadata.xml? > > Yup. That's a no-go, since metadata.xml is quite rightly treated as > being "not suitable for anything the package manager really needs". > > It also moves the EAPI definition even further away from the ebuild, > which makes it even harder to work with. > > And, of course, it's not backwards compatible, so it'd still need a > file extension change. > Another ugly solution: Having the EAPI on a per-package (like $portagedir/cat/package-1) or per-tree basis ($portagedir/profiles/eapi) and start providing our tree as overlays of more than one tree (will end up in a mess of dependencies, but it would still be nice to specify the EAPI for a complete overlay instead of having the name all the ebuilds like .eapi-X :-). In addition: it wouldn't be possible to identify the EAPI of an ebuild by just looking at it... -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: GLEP 55
Joe Peterson wrote:
> Ciaran McCreesh wrote:
>> And a file extension is far less obscurely complex than enforcing
>> arbitrary syntax restrictions upon ebuilds.
>
> I disagree. One is exposed to devs only as ebuild syntax; the other is
> exposed in an inappropriate location to everyone looking at the portage
> tree.
>
>> No it can't. EAPI has to be known before the source can start. Bash
>> doesn't provide traps for executing code upon changed variables.
>
> Doing it out-of-band solve this.
>
>> No, it's only needed once per non-trivial change. So we might as well
>> just change it for every EAPI.
>
> Huh? If the "new" portage knows how to determine the EAPI definitively
> (and that would be defined), it can deal with the differences.
>
>> And then how do we deal with EAPI 3, where the syntax changes again?
>
> Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
> from there. If you change the way you declare EAPI each time, yeah,
> that's a problem, but I'm not sure why that would ne necessary.
No, that is not the problem.
Example:
In EAPI 42 we define that the package manager must provide a global function
extract_depend_from_setup_py() such that it is callable at a global level
in an ebuild like this
*snip start*
# Copyright 1999-2007 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
EAPI=42
DESCRIPTION="A library aiming to support agile and test-driven python
development on various levels."
SRC_URI="http://codespeak.net/download/${PN}/${P}.tar.gz";
HOMEPAGE="http://codespeak.net/py/";
KEYWORDS="~amd64 ~x86"
SLOT="0"
LICENSE="MIT"
IUSE=""
extract_depend_from_setup_py
*snip end*
Now, a package manager (or a tool) not knowing EAPI 42 will fail when it
tries to source the above ebuild to determine the EAPI version (as it is
being currently done as far as I understood it) because
extract_depend_from_setup_py is undefined.
So it won't even be able to read out the EAPI.
With the EAPI in the filename tools now knowing EAPI-42 will either ignore
the above foo-1.0.ebuild-42 or mask it because they may identify the
EAPI-version without sourcing the ebuild.
And: No, just sourcing the ebuild and simply masking it because we can't
source it is a no-go (since you're abusing error-handling as a case
switch).
--
gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: [GLEP 55] EAPI subdirectories instead of file name suffixes
Piotr Jaroszy?ski wrote: > On Saturday 22 of December 2007 02:41:02 Petteri Räty wrote: >> Piotr Jaroszy?ski kirjoitti: >> > This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds >> > (for example, foo-1.2.3.ebuild-1). >> >> It seems many people don't like the idea of having it in the filename > > Seems you are counting the posts, not the people. > >> but how about having subdirectories for different eapis. This should >> even be faster for the package manager as it can just ignore the >> directories it can't understand instead of having to parse the file >> names. > > It was already proposed, but didn't seem to get much support. That would be counting posts, not people. It just hasn't been discussed. > It is > equivalent to using the suffixes, but I see it rather as perfomarnce hit, > not improvement. The package manger would have to look for ebuilds in the > main dir and all the subdirs in case it doesn't have/can't use the cache. > Yeah but this isn't about performance (or so I heard. I took that to mean it was about ebuilds which can't be sourced, but you might want to check exactly what McCreesh meant, since I am apparently incapable of understanding his missives.) Given that, the subdirectories would do the job fine, and end-users don't have to worry about building the cache anyway. -- [EMAIL PROTECTED] mailing list
