Re: [gentoo-portage-dev] Transports
On Tue, May 31, 2005 at 02:38:58AM -0400, Alec Warner wrote: > I was looking at Transport code last night and I noticed it only > supported HTTP/HTTPS/FTP, which I thought was kind of limited. Thoughts > on merging the Sync code in with Transports, having the transport lib > covering all...well..file transport code within portage? The Rsync code > is strikingly similar, and I was thinking of adding scp as well so > people have a lot of options. > > Thoughts, objections...donuts? I like cheese. ? scp doesn't support resume, so it differs from existing transports if added. For merging of sync and transports, transports is specifically single file network io requests, sync can mangle multiple files. Dunno, possible. Honestly sync and transports could use a mild set of touch ups, although not sure about collapsing/combining the sync/transports bit. Reasons for it, aside from having a few more protocols able to be handled? Hadn't thought about the possibility of supporting cvs for SRC_URI- that would be nifty, although would need a way to specify a required atom for protocols (if this cpv has cvs://blar in it, it requires dev-util/cvs, or preferably a virtual should dev-util/cvs ever move... ~brian pgpsdjAB32JZ1.pgp Description: PGP signature
Re: [gentoo-portage-dev] Transports
On Tue, May 31, 2005 at 11:20:56AM -0400, Alec Warner wrote: > Brian Harring wrote: > Mostly was looking at a unified transport lib, with connection > abstractions, which is basically what we have now, but lacks SCP/RSYNC > support, and I think people would want that functionality, Remote repos > with rsync for proto fex. That way the remote repo code only depends on > Transports and all the proto mess is abstracted away. Maybe just keep > em seperate and copy the rsync code? I kind of like the single vs multi > approach now that I think about it more :P Unifying the actual 'transports' crap into transports would make sense, although I'd still posit a wrapper layer over whatever transport layer in use would be required; iow, transports/rsync, but sync/rsync still that imports portage.transports.rsync .. ~harring pgpLhjBI8dwe3.pgp Description: PGP signature
Re: [gentoo-portage-dev] dispatch-conf
On Thu, Jun 02, 2005 at 08:29:24AM +0900, Jason Stubbs wrote: > It's not fixed as far as I know and I've been the one fixing bugs with > dispatch-conf and I just came across that bug the other day. To confirm we're > talking about the same thing: > > myfile (regular file) > ._cfg_myfile (symlink to otherfile) > > After running dispatch-conf and choosing to "use new", myfile will be a > regular file holding the same contents as otherfile. > > As for splitting it out, I'd say either both or neither. As it stands now, > etc-update has more bugs (mostly usability) filed against it than > dispatch-conf does. It'd be nice to have dispatch-conf become the > recommended. This could still happen if they were to be split out though. > We'd just need a virtual/cfg-updater or something that they both provide and > just make portage depend on it. Hmm. I was just about to argue that we need at least one bundled with portage, but the thing is, we don't really. virtual seems like slight overkill, but is the proper route... ~brian pgpPOyd8Bo0Oa.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: properly handle metadata transfer on first sync of an empty tree (#96410)
On Fri, Aug 05, 2005 at 08:20:52PM +0900, Jason Stubbs wrote: > On Friday 05 August 2005 12:31, Brian D. Harring wrote: > > Hola all, patch (incvs now) to fix up a traceback on first sync with > > an empty tree; bug #96410 > > http://bugs.gentoo.org/show_bug.cgi?id=96410 > > > > The fix isn't exactly what I'd call pretty (creating an intermediate > > portdbapi and config instance to do the updates), but it's a corner > > case; config's categories is a bit of a hack, and tearing it out in > > stable is more work then worth... so this. > > > > Either way, it's attached, poke at it kindly :) > > You know the caching fairly well, so I'll ask rather than check. ;) > > How does this affect the new version of portage check that follows? Will > that use the old cache, find that it's out of date and then regen it? The ugly inlining of code in portage.py makes this play nice actually, due to the forced reload of portage prior to checking for an updated portage being available (specifically, the config instance is recreated). ~harring pgpKJ3AFvs2Av.pgp Description: PGP signature
[gentoo-portage-dev] PATCH: if doing a piped call in unpack, check pipestatus
Hola.
Bug 101035
http://bugs.gentoo.org/show_bug.cgi?id=101035
InCVS now also...
~harring
Index: ebuild.sh
===
RCS file: /var/cvsroot/gentoo-src/portage/bin/ebuild.sh,v
retrieving revision 1.201.2.39
diff -u -r1.201.2.39 ebuild.sh
--- ebuild.sh 7 Aug 2005 04:33:45 - 1.201.2.39
+++ ebuild.sh 9 Aug 2005 11:21:01 -
@@ -363,7 +363,8 @@
tar xzf "${DISTDIR}/${x}" ${tarvars} || die
"$myfail"
;;
tbz2)
- bzip2 -dc "${DISTDIR}/${x}" | tar xf -
${tarvars} || die "$myfail"
+ bzip2 -dc "${DISTDIR}/${x}" | tar xf -
${tarvars}
+ assert "$myfail"
;;
ZIP|zip)
unzip -qo "${DISTDIR}/${x}" || die "$myfail"
@@ -377,7 +378,8 @@
;;
bz2)
if [ "${y}" == "tar" ]; then
- bzip2 -dc "${DISTDIR}/${x}" | tar xf -
${tarvars} || die "$myfail"
+ bzip2 -dc "${DISTDIR}/${x}" | tar xf -
${tarvars}
+ assert "$myfail"
else
bzip2 -dc "${DISTDIR}/${x}" > ${x%.*}
|| die "$myfail"
fi
pgpM7Sq5P2KHX.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Next major version
On Fri, Aug 12, 2005 at 12:04:34PM -0400, Kristian Benoit wrote: > I remember, when I started using Gentoo, reading that portage is a stand > alone tool, it is not bind into Gentoo in anyway, someone could use it > on redhat, debian, lfs... Nice intention, but impossible with stable/alpha code- the abstractions are missing. No config abstractions, but more importantly no format abstractions; no true package object. > Back then I was using lfs so I thought portage could be the way to go on > lfs, but I realized that Gentoo fit my needs and I did'nt have to > compile everything by hand anymore and still have everything compiled by > my machines :) OH JOY !!! Heh, came via the route I did... > But 5 years or so later, the only official place to get portage releases > is still in the gentoo mirrors. There is no RSS feed or anything like > that. I still believe that portage has the potential to be so powerful > that redhat, debian, ... could be building their packages using portage, > managing their own tree, having night build. > > The problem is see, is that the initial portage vision (or perhaps my > initial vision, a vision I still have) has not been carried along with > it's developpement. The vision got blocked by the implementation. Try busting all of the globals out of portage, then abstracting all ebuild specific actions (doebuild) behind package apis, so that different formats can be swapped on the fly. Hell, binding dbapi and *tree classes together into one, and having them properly inherited from a base is required, rather then lots of duplicated code. From there, how do you represent the *depends of a package, so that the resolver can be reused across different configurations of package format (this box being rpm, that being ebuild fex); need to break it down into restrictions, handing the actual depends matching off to repositories, with the resolver shifting sets of returned packages/restrictions around to build up a graph. Either way, look at gentoo-src/portage/portage and gentoo-src/portage/rewrite-misc Work is underway, help is needed, jump in and start digging :) The design *should* allow for lots of crazy crap, although anyone who sees a flaw please speak up now :) > Having an official web site, doc, ... will help getting visibility and > effort from the rest of the world thus we'll have better tools and > eventually extend portage beyond Gentoo. API for tools, a *sane* api moreso :) ~harring pgptbc6oHdG5T.pgp Description: PGP signature
Re: [gentoo-portage-dev] os.path.normpath patch ( bug 90444 )
On Tue, Aug 16, 2005 at 02:39:36PM -0400, Alec Joseph Warner wrote:
> I kind of see Brian's point in being defensive in filenames though
> (os.path.normpath("///var/db/pkg/fex") being invalid in a CONTENTS
> file but still usable by most(?) calls in python, which will just figure
> it all out anyway.
If we aren't defensive on it, and it causes something to implode, it's
a runtime bt...
Re: equery, equery is a special case walking the entire contents...
> If portage errored on invalid paths in CONTENTS file, I would see
> leaving it there, but if it doesn't help anything ( because it doesn't
> hurt anything either, right? :) ) then why do the work?
What about finding a faster approach then normpath?
Take a look at normpath, it splits it up into strings, and rejoins;
that's a bit of a hit right there- potential point for optimization if
I keep screaming "the data returned may not be right, albeit usable"
:)
~harring
pgpYl7htd0bwL.pgp
Description: PGP signature
[gentoo-portage-dev] [PATCH] sane USE_EXPAND + IUSE check
Hola-
basically, use_expand'd vars need to be exempted from IUSE checks, as
long as the USE_EXPAND var is in IUSE.
This does that.
~harring
Index: ebuild.sh
===
RCS file: /var/cvsroot/gentoo-src/portage/bin/ebuild.sh,v
retrieving revision 1.201.2.40
diff -u -r1.201.2.40 ebuild.sh
--- ebuild.sh 9 Aug 2005 11:25:44 - 1.201.2.40
+++ ebuild.sh 17 Aug 2005 00:50:27 -
@@ -15,6 +15,7 @@
if [ -f "${T}/environment" ]; then
source "${T}/environment" &>/dev/null
fi
+ USE_EXPAND=$(echo ${USE_EXPAND} | tr A-Z a-z)
fi
if [ -n "$#" ]; then
@@ -130,7 +131,19 @@
# Make sure we have this USE flag in IUSE
if ! hasq "${u}" ${IUSE} ${E_IUSE} && ! hasq "${u}" ${PORTAGE_ARCHLIST}
selinux; then
- echo "QA Notice: USE Flag '${u}' not in IUSE for
${CATEGORY}/${PF}" >&2
+ local x
+ local invalid=1
+ for x in ${USE_EXPAND}; do
+ if [ "${u:0:${#x}}" == "${x}" ]; then
+ if hasq "${x}" ${IUSE} ${E_IUSE}; then
+ unset invalid
+ fi
+ break
+ fi
+ done
+ if [ -n "${invalid}" ]; then
+ echo "QA Notice: USE Flag '${u}' not in IUSE for
${CATEGORY}/${PF}" >&2
+ fi
fi
for x in ${USE}; do
pgpVlG8eJOxdX.pgp
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] sane USE_EXPAND + IUSE check
On Wed, Aug 17, 2005 at 10:41:12AM +0900, Jason Stubbs wrote: > You hijacked a thread again... Bleh, headers (yes I've been lazy and just hitting reply on previous messages to lift the to :P) > On Wednesday 17 August 2005 09:52, Brian Harring wrote: > > basically, use_expand'd vars need to be exempted from IUSE checks, as > > long as the USE_EXPAND var is in IUSE. > > I don't really like the idea of this without a companion patch that provides > some way to get a list of relevant env vars and their possible settings to > the user without having to look at the ebuild. The list of possiblities > could live in use.desc or a similar file, but at minimum the list of vars > needs to be provided. Fix in the meantime since people are poking about it; a proper lock down of IUSE makes sense, exempting A) addition of Yet Another File To Profiles B) if you do it within portdir (as per the norm), it effectively changes IUSE for all ebuilds viewed, meaning QA complaints in binpkgs and overlay ebuilds (where it may be valid in that context). > Another patch is also required that will allow disabling of the QA check on > certain vars, preferably defined in a file in the tree similar to > info_vars. This would be used for vars such as USERLAND which are profile > defined. Why whitelist some vars, but not others? If you're locking the USE_EXPAND'd vars down for full IUSE check, any profile defined var should be limited in the same way, imo. A var not used in a profile is effectively dead ebuild code, even if the profile has disappeared; that said, still suffers the same issues as B above. > In other words, don't kill the QA check without addressing the issue that > the QA check is warning about. ;) Issues above kind of strike me as it being more trouble then worth- personally, I'd rather see IUSE for that ebuild have the known USE_EXPAND'd var variations it uses listed, with some form of filtering to keep people from attempting dumb things like userland_BSD when they're on userland_GNU (hypothetical example). ~harring pgpcPbhCq6bc6.pgp Description: PGP signature
Re: [gentoo-portage-dev] Re: Environment Whitelisting
On Mon, Aug 22, 2005 at 12:24:13PM -0700, Zac Medico wrote: > warnera6 wrote: > >>>My preference would go 4, 3, 2 then 1. While Makefiles and configure > >>>scripts may be "broken" upstream, how long is it before the breakage > >>>goes unnoticed? More importantly, what's the chances of a dev finding > >>>the breakage before users? Cleansing the environment to me is akin to > >>>using sandbox. It offers protection against misbehaving packages... > >>> > >> > >>Good point. How about if we add environment sandboxing support (in > >>addition to filesystem sandboxing) to sandbox. With an environment > >>sandbox, we could detect specifically which variables a build is > >>fragile with regard to. The sandbox would have both filesystem access > >>and environment access violation summaries. > > > >"environmental sandbox" being similar to sandbox, or the cleansing of > >the environment? The latter is easy, the former...I am not sure how you > > begin to detect variable use in bash :/ > > > > AFAIK we can intercept getenv() calls the same way that we intercept > filesystem calls. IMO the white/black/override lists would best be > implemented at this level. Don't think this is the appropriate method, imo- remember sandbox doesn't exist on bsd, so the solution wouldn't be across the board (resulting in ebuild devs inventing their own that is when required). Better approach is abusing the env-filtering capabilities written into 2.1 already- it wouldn't require much to slip it into ebuild_processor. ~harring pgpATunzqlr3M.pgp Description: PGP signature
Re: [gentoo-portage-dev] Environment Whitelisting
On Mon, Aug 22, 2005 at 11:33:23PM +0200, Marius Mauch wrote: > Theoretical discussions about this are pointless IMO without > numbers/facts to back things up. I'd posit theroetical discussions about this are pointless without getting ebuild dev's to give a yay/nay on whether they want it or not; not much for trying to force it down their throats if they don't want it (more work, essentially). ~harring pgpGgF8NBM8lW.pgp Description: PGP signature
Re: [gentoo-portage-dev] Environment Whitelisting
On Mon, Aug 22, 2005 at 11:59:54PM +0200, Marius Mauch wrote: > On 08/22/05 Brian Harring wrote: > > > On Mon, Aug 22, 2005 at 11:33:23PM +0200, Marius Mauch wrote: > > > Theoretical discussions about this are pointless IMO without > > > numbers/facts to back things up. > > I'd posit theroetical discussions about this are pointless without > > getting ebuild dev's to give a yay/nay on whether they want it or not; > > > > not much for trying to force it down their throats if they don't want > > it (more work, essentially). > > That too. But providing them with some numbers will certainly have an > effect on their decision (especially if it shows that it doesn't really > affect them ;) Rather hard to back it up though, without specialized knowledge in (effectively) the whole tree- either we do it, or we ask nicely those who are supposed to have such knowledge :) I can rattle off a couple of env vars that screw things up, but how many of us are aware that an exported ARCH screws with kernel builds fex? I'd punt it to them, and find out what they think (tiz the route I took when I brought this up last). Explicit whitelisting is great for getting closer to deterministic builds, but it's a helluva overhead on a side note. ~harring pgpIkTIe3KmQe.pgp Description: PGP signature
[gentoo-portage-dev] stripping implementation in portage
Hola all. Short version, the nostrip feature is a bit funky as an option. What I'm after is effectively building all packages *with* debugging information as default, and leaving it up to the repository you're merging the package to, to decide on stripping or not. IOW, if you prefer stripped binaries on your livefs, the stripping occurs while merging to the livefs- this leaves you the option of having binpkgs that *do* carry non-stripped binaries/libs. Situation can be reversed also, for the embedded crowd. Downside, for people who flat out want stripping across the board, it's a bit more flipping it on, although that's addressed via inherit support within the underlying config (just take my word on that one :) Also involves a bit more logic, but that's just implementation voodoo. So... thoughts? I'd be particularly curious about any package where this wouldn't be viable. Aside from that, cc'ing both lists, would prefer the discussion on dev since the implementation can go either way; preference of if that flexibility is desired or not is a user thing, so we discuss it in their ml. ~harring pgpAX6p2No9XM.pgp Description: PGP signature
Re: [gentoo-portage-dev] Environment Whitelisting
On Mon, Aug 22, 2005 at 06:36:03PM -0400, Alec Warner wrote: > I'm kinda with genone on implementing both ( since they are similar ). > If it's decided that blacklisting is easier to maintain, I can always > make up my own whitelist for pkg-foo and apply it and if it works submit > it as a bug ( or even some other whitelist database? ) and thus can > gaurentee that my package was built 'correctly'(TM). I think this would > be important in fex, an enterprise distribution type deal where the > build env is important to some. Put the whitelists in the tree and have > them --excluded by default, so only the users that use them have to > downlaod them. Sticking Yet Another File In Profiles (hence forth known as yafip since it comes up a lot :) is a quick fix, but forces anyone trying to do remote repo's to add hacks to expose that information; effectively requiring two querying modes for the repo, which doesn't seem clean. If it goes anyways, profile would get my vote- base profile exists for spreading settings like this throughout all profiles also; profile and repo are seperated entities, as such the repo implementation doesn't get muddied up, just requires extra keys pulled for profile implementation, which is what it's designed for anyways. ~harring pgpnBbxksQR1o.pgp Description: PGP signature
Re: [gentoo-portage-dev] Environment Whitelisting
On Tue, Aug 23, 2005 at 08:28:08AM +0900, Jason Stubbs wrote: > On Tuesday 23 August 2005 06:40, Brian Harring wrote: > > On Mon, Aug 22, 2005 at 11:33:23PM +0200, Marius Mauch wrote: > > > Theoretical discussions about this are pointless IMO without > > > numbers/facts to back things up. > > > > I'd posit theroetical discussions about this are pointless without > > getting ebuild dev's to give a yay/nay on whether they want it or not; > > not much for trying to force it down their throats if they don't want > > it (more work, essentially). > > I don't really see what it has to do with ebuild devs... We're talking about > the user's environment leaking into the portage build environment, no? > Environment vars used by ebuilds can/should be set by users in a portage > configuration file rather than being added to the environment. The only > issue i see here is user customizations - fex, a hypothetical colorgcc that > gets its config info from the env. Ixnaying user env leaking in will lead to bugs where ebuilds *allow* for that, along with pissed off ebuild devs if they've not been made aware of it oncoming. Hell, they may not even agree on the deterministic bit re: env; this would explicitly block developers from fooling with autotool vars (WANT_AUTOMAKE fex) for testing without mangling the ebuild. So yeah, I'm trying to ensure we're not screamed at for deploying what (imo) is a good change, but may piss people off if it's not stated up front (akin to glep 33). ~harring pgpFEMfSbI52o.pgp Description: PGP signature
Re: [gentoo-portage-dev] Re: Environment Whitelisting
On Mon, Aug 22, 2005 at 09:57:46PM -0400, Kristian Benoit wrote: > On Mon, 2005-08-22 at 12:24 -0700, Zac Medico wrote: > > AFAIK we can intercept getenv() calls the same way that we intercept > > filesystem calls. IMO the white/black/override lists would best be > > implemented at this level. > > getenv is'nt a system call. Nor is glibc's open call, although it hands off to a syscall . LD_PRELOAD just inserts (essentially) it's functions into the symbol resolution order; your symbols get used rather then those from the lib. It's possible, I just don't like it. :) ~harring pgpj4b3eP79gD.pgp Description: PGP signature
[gentoo-portage-dev] removal of vars from ebuild env
Fair warning, To anyone relying on the vars BUILD_PREFIX, BUILDDIR to be available in the ebuild env, they're going to be yanked down the line; right now, going by scans nobody relies on them- so... please keep it that way. Thanks, ~harring pgpF0Md5boscN.pgp Description: PGP signature
[gentoo-portage-dev] future restrictions to DISTDIR access from the ebuild env
Hola all. robbat2 made the suggestion, and after a bit of playing I think it's best- in short, to support multiple DISTDIR's, we need either intelligent querying of portage from bash side as to a files true location, or a directory full of symlinks pointing at the ebuild's stated files. Latter is easiest imo, and a bit cleaner for a dev to be able to look at it and see why things are breaking; upshot, adding this indirection allows portage (with sane fetcher(s)) to do N DISTDIR, and it also causes any ebuild that has unstated SRC_URI deps to bail. The forced bail pretty much makes it so that an ebuild dev can't easily screw up, they hit the unpack failure rather then a commit and a user hitting it. If you've got issues with it, give a yell; just added this in the rewrite. ~harring pgpfx797qrV4w.pgp Description: PGP signature
[gentoo-portage-dev] portage rewrite snapshot (was RFC - Gentoo on the Lab)
On Tue, Aug 23, 2005 at 06:58:53PM -0400, Kristian Benoit wrote: > Yeah, I'd really like having a snapshot, even if I'd prefer having > cvs/svn access. You can send me it by mail or make it available > somewhere. Pardon the delay, wanted to iron out building code before pushing the snapshot up- it's available at http://dev.gentoo.org/~ferringb/new-08-25-05.tar.bz2 anonsvn is in the air, but in the interim I'll be doing snapshot's as stuff comes in. /me notes again, that it is not usable yet. You can build packages with it, but the classes for merge operations aren't in place, meaning no merge. Bugs probably exist up the ying yang also; if you want to hack on it, then by all means go for it, otherwise kindly wait (general disclaimer that one). ~harring pgp5L3MlPzDDR.pgp Description: PGP signature
Re: [gentoo-portage-dev] Additional information to output: emerge -pv
On Thu, Aug 25, 2005 at 10:15:26AM -0400, Sandy McArthur wrote: > How about making it so you need two -v switchs (eg: -vv) to get the > long listing? I use `emerge world -puv` all the time and don 't want > to have to scroll through so much stuff. Conflicts with -vv usage in >=2.1 which pulls from metadata, dumping maintainer/herd. Or just extends it... /me shrugs, eenie meenie on that one ~harring pgpykhKmX7bdV.pgp Description: PGP signature
Re: [gentoo-portage-dev] Additional information to output: emerge -pv
On Thu, Aug 25, 2005 at 11:14:50AM -0500, Brian Harring wrote: > On Thu, Aug 25, 2005 at 10:15:26AM -0400, Sandy McArthur wrote: > > How about making it so you need two -v switchs (eg: -vv) to get the > > long listing? I use `emerge world -puv` all the time and don 't want > > to have to scroll through so much stuff. > Conflicts with -vv usage in >=2.1 which pulls from metadata, dumping > maintainer/herd. > > Or just extends it... > /me shrugs, eenie meenie on that one Correction, I'm wrong- that's emerge -sv I'm thinking about. ~harring pgp4jyhNM5wzm.pgp Description: PGP signature
Re: Re: [gentoo-portage-dev] Additional information to output: emerge -pv
On Thu, Aug 25, 2005 at 05:44:47PM +, Dustin Spicuzza wrote: > My guess on the lack of feekback on the bug is that a feature like this > wouldn't be as useful to someone who isn't new to gentoo, and in fact > (as mentioned above), could actually be quite annoying. But for someone > who is still new to gentoo.. it would be useful until they get a grasp > on what USE flags do what. Actually, I'd find it useful personally. People add *weird* use flags sometimes. Lack of feedback is partially the fact I'd rather see it as a class based approach; can tag it into the rewrite whenever UI work gets in full gear, but adding new features into stable that will have to be reimplemented in the rewrite doesn't seem wise (hence the feature freeze declaration). ~harring pgpvyucVTTfGg.pgp Description: PGP signature
Re: [gentoo-portage-dev] Additional information to output: emerge -pv
On Thu, Aug 25, 2005 at 06:15:36PM +, Dustin Spicuzza wrote:
>
> >
> >Actually, I'd find it useful personally.
> >People add *weird* use flags sometimes.
> >
> >
> LoL... yes, thats true.
>
> A suggestion possibly to keep in mind is what if all functionality
> related to USE flags were somehow implemented in a seperate class/set of
> functions... or maybe that's already being done. But I mean, the job of
> determining which flags apply to what package and managing all that and
> how it works... I noticed that (at least in stable) a lot of it is in
> portage.config, but maybe it could be a seperate class that
> portage.config uses.. and other things (like deps, etc). Then stuff like
> outputting USE flags, or adding additional ways to determine which USE
> flags to actually use for each package, would all be in one location,
> lending itself to code reuse...
>
> Of course, this is just from my looking at portage stable for a week or
> so... maybe its already being done. :)
It's being done :)
Essentially, you have this
config
domain
|--repo(s)
| |--cache(s)
| |--sync
|--vdb(s)
|--profile(s)
Domain is generated from configuration, as are all of the other
objects; the config class just holds config data, and instantiates
objects, shuffling them around dependant on an external file that
states the rules for the config (in the process, stating how to
instantiate the objs).
Config build data is domain data; your normal ebuild repository is an
'unconfigured' repository, iow, you can toggle stuff on it to change
the metadata of packages it returns.
The domain during instantiation notes that it has an unconfigured repo
in it's list, inspects the repo instance for what is needed to
configure it (and the func to call), calls the func with the
configurables supplied, and uses the returned repo instead of the
unconfigured repo.
Via this, the config data (USE, build, etc) is pushed down into the
repo. The configured repo wraps the unconfigured repo's returned
packages with it's own wrapper that binds the config data into it, and
hands that back.
Probably not a great description, but the short version is it works
pretty well, and is pretty well encapsulated.
What I'd *like* to see is the use.{local.,}desc information bound into
the repository, and have it supplied from the repository object; doing
so keeps that data in the repo backend, not screwing up any
abstractions that are carefully defined to allow for remote
replacements.
Haven't proposed it to -dev yet, but I'd like to see package.* and
use.* moved out of profiles/ , and into it's own base directory in the
ebuild repository.
Reasoning is that profile objs are seperate from repo objs; the fs
layout should reflect that to some degree. It's cleaner anyways, imo.
:)
~harring
pgpqBz2pOhTE0.pgp
Description: PGP signature
[gentoo-portage-dev] Re: [gentoo-dev] EBUILD_FORMAT support
Pardon the delay, been putting this one off since it's going to be a fun one to address, and will be a bit long :) On Thu, Aug 25, 2005 at 12:34:00PM +0200, Paul de Vrieze wrote: > What I mean is compatibility with current portage versions. Current > versions do not understand EAPI. There would be a good chance that they > could choke on packages with all kinds of new features, even in the sync > phase. A different extension would ensure that those portage versions > would still work (crippled) on a new tree. Of course such an extension > change should only be done once. Once the API versions are available this > is not an issue. General portage stance towards EAPI is unset EAPI == 0 (current stable ebuild format); if EAPI > then portage internal EAPI, unable to merge, which should be able to be detected during buildplan. Current portage doesn't know about EAPI; boned in that respect I'll admit, but it's the case for all new features rolled out- three options for dealing with this 1) Usual method, deploy support, N months later use support. 2) tweak stable so it's aware and can complain. Still requires people to upgrade, just makes it so that they're not forced into upgrading to 3.x; this is mainly a benefit for those who may don't care to try the first few releases of 3.x when it hits (akin to people dodging the first release or two of a gcc release). Worth noting that one rather visibile aspect of EAPI=1 is that (assuming the council votes on it, and yay's it) glep33 *will* result in current eclasses being effectively abandoned w/in the N months after an EAPI capable portage is released. Sound kind of bad, but people will have to upgrade for the capabilities. If EAPI was pegged into portage/ebuilds already it wouldn't be an issue, issues could be detected prior. Unfortunately it's not, and introduction of it (and use of it) is going to involve a few road bumps. Plus side, once it's in, portage *will* know if the ebuild is incompatible with the pythonic/bash ebuild code, and portage/the UI can act accordingly. Meanwhile, the changes that are being pushed into EAPI are addition of configure phase (broken out from compile), elib addition, and eclass2 support (same beast, different rules due to env save/restoration). The potential for horkage on sync'ing isn't there due to the fact that's purely python side; ebuild*sh doesn't play into it. Re: regen, issue isn't really there either; if you try and merge an eapi=0 on a non eapi aware portage, it works, same as it did before. If you try to merge an eapi=1 ebuild you hit either an issue with inherit, or a bail immediately in src_compile, due to the fact eapi=1 ebuilds will seperate configure out from compile (eapi=0 portage won't know to call it; no configure == failed compile). That said, there also quite likely is a change coming down the pipe to the tree's cache; the change will shift the rsync'd metadata cache over to a key/val based cache. Why oh why, yet another cache change? Simple. The change moves away from list based format to key:value pairs; in short it's a change that once made, means keys can be added to the cache from that point on without causing cache complaints on sync'ing. Last cache breakage, I swear :P EAPI addition being the next key tagged in; stable (not surprising) needs to be released with a version capable of reading both old and new format; once that's done, time for the usual "yo, upgrade people, something's coming down the line". Same version that supports old and new cache format can also include rudimentary eapi awareness. At least that's what I'm thinking. It's roughly inline with the previous forced cache breakages, just in this case slipping in some extra support in the process. Notices obviously would go out prior to moving on this also, along with a good chunk of waiting. > > > ps. I would also suggest requiring that EAPI can be retrieved by a > > > simple line by line parsing without using bash. (This allows for > > > changing the parsing system) > > > > No, that yanks EAPI setting away from eclasses. > > If the eclasses follow similar rules that would be easilly parseable. > (taking inherit ...) into account is easy as long as the inherit line is > on one line of it's own. (unconditionally) These rules that would > allready be followed out of style reasons would make various kinds of > parsers able to parse them. while it's insane, people *can* use indirection (eg inherit $var) for inherit's as long as it's deterministic, always the same inherit call for that ebuild's data. Don't see a good reason to ixnay that, which means we'd have to parse the whole enchilada, eclasses and the ebuild. Effectively, raiding a single var out wouldn't fly; eclasses could override an ebuild's eapi setting for example, just like any other metadata key (imo). A *true* format change, moving away from bash for example or moving to an executing design o
[gentoo-portage-dev] proposed shift of files in the tree of non profiles files into seperate dir
Hola all. Straight to the point, I'm proposing that the following files- arch.list categories use.desc use.local.desc package.mask updates be moved out of the profiles directory in the tree, and into the existing metadata directory personally, due to the fact that the files above are essentially repository metadata. Why move them *now* when they've been around forever? Those files should never have been placed there. They're repository specific data (global data for the repo), and do not belong jammed into profiles which is a seperate entity. Further, while no one has yet proposed anything concrete, people have been after revamping the profile implementation. Quite likely if/when this occurs, it's going to require a seperate directory to avoid any issues with older portage installations accessing it. Shifting the files now while changes are being made addresses that concern, and makes things a bit more logical. Not surprising, few issues (and ways to deal with it). For backwards compatability with portage tools, symlinks are needed to prevent older portage versions and tools from suddenly being broke; just do a relative link to the new location, no complaints from portage. CVS does not play nice with symlinks however- fortunately rsync sucks a bit less, so the symlinks would have to be added by the rsync regen script (minor to do). Two scenarios for how this will result in visible issues for people- 1) CVS users, aka, devs. Devs *should* be running latest portage, which would know about the shift. If they're running an older portage version and aren't willing to upgrade, they tag the symlinks themselves. It's a minor annoyance frankly; assuming they read -dev (like they're suppossed to :P ), they'll know in advance it's coming. 2) users storing their tree on an fs that lacks symlink support. They ought to be a miniscule minority, but any issues they hit would be addressed by upgrading portage and any portage tools they use that is reliant on those files. This is the worst case user scenario, but again, it should be effectively non-existant. Finally, lang.desc I can't find any reference to in portage tools- zhen commited it almost 3 years back, and it hasn't been touched since. Unless somebody knows what the heck that file is for, it's a good candidate for removal. I realize this may seem like a minor/stupid change, but it's a matter of cleanup getting things a bit more consistant for when portage is capable of dealing with more then one $PORTDIR. ~harring pgpfB6TLsedFv.pgp Description: PGP signature
[gentoo-portage-dev] PATCH: initial EAPI awareness
Hola.
Attached is a patch that
A) adds EAPI awareness to portage; mainly, if >0, complain and be
unwilling to merge the package
B) tweaks to portage_db_flat, addition of portage_db_metadata, and
portage_db_flat_hash
Flat_hash is the replacement cache format for metadata/cache; assuming
nobody does anything stupid with EAPI, it will allow us to deploy new
metadata in the tree without forcing a breakage every time we do so.
portage_db_metadata.py is a compability implementation; basically
detects if it's flat_list (portage_db_flat), or flat_hash and calls
accordingly.
Aside from that, tried to keep changes as minimal as possible. The
patch isn't complete due to the fact we need to add an EAPI cache
compatibility check for --metadata transfer, but can't do that till I
get some feedback from -dev ml regarding a bit of tree cleanup that
will go along with it.
Please test this out; if you want to test the EAPI checking, tag
EAPI=1 into an ebuild, and try making emerge bail.
If you're less adventurous, please test the compatibility cache
testing;
in /etc/portage/modules
portdbapi.metadbmodule=portage_db_metadata.database
Will enable the autodetection. Additionally,
portdbapi.auxdbmodule=portage_db_flat_hash.database
being flipped on would be appreciated, although you will have to wipe
your cache and run emerge --metadata; with the settings above, you're
testing the auto-detect functionality, and the new format.
Thanks,
~harring
diff -urN portage-old/bin/emerge portage/bin/emerge
--- portage-old/bin/emerge 2005-08-27 05:28:09.0 -0500
+++ portage/bin/emerge 2005-08-27 05:27:17.0 -0500
@@ -612,7 +612,7 @@
help()
sys.exit(0)
elif portage.secpass!=2:
- if myaction in ["search", "info", "regen"]:
+ if myaction in ["search", "info", "regen", "metadata"]:
pass
elif (not myaction) and (not myfiles):
pass
@@ -1497,7 +1497,7 @@
myfetchlist=[]
for x in mylist:
fetch=" "
-
+ stupid_if_logic_hack = True
if x[0]=="blocks":
addl=""+red("B")+" "+fetch+" "
resolved=portage.db[x[1]]["vartree"].resolve_key(x[2])
@@ -1512,7 +1512,22 @@
print red("(is blocking
"+x[3]+")")
else:
print
+ stupid_if_logic_hack = False
+
else:
+ if x[0] == "ebuild":source = "porttree"
+ elif x[0] == "binary": source = "bintree"
+ elif x[0] == ("nomerge", "blocker"):source
= None
+ else:
+ print "EAPI check, unknown source
"+str(x[0])+" for "+x[2]+", this shouldn't occur",mylist
+ sys.exit(1)
+ if source:
+ needed =
portage.db[x[1]][source].dbapi.aux_get(x[2], ["EAPI"])[0]
+ if needed > portage.EAPI:
+ print '[%s] %s %s, EAPI:%i
installed portage EAPI:%i' % (x[0].ljust(13), x[2], red("UNMERGABLE"),needed,
portage.EAPI)
+ stupid_if_logic_hack = True
+
+ if stupid_if_logic_hack:
if (x[0]!="binary") and ("fetch" in
string.split(portage.portdb.aux_get(x[2],["RESTRICT"])[0])):
fetch = red("F")
if portage.portdb.fetch_check(x[2],
portage.settings):
@@ -1830,6 +1845,34 @@
self.pkgsettings["FEATURES"]=string.join(myfeat)
mergecount=0
+
+ unmergable = []
+ for source, root, cpv, action in mymergelist:
+ if source == "ebuild": source = "porttree"
+ elif source == "binary": source = "bintree"
+ elif source in ("nomerge", "blocks"):
+ continue
+ else:
+ print "EAPI check, unknown source
"+str(source)+" for "+cpv+", this shouldn't occur",mymergelist
+ sys.exit(1)
+ needed = portage.db[root][source].dbapi.aux_get(cpv,
["EAPI"])[0]
+ if portage.EAPI < needed:
+ unmergable.append((needed, cpv))
+
+ if len(unmergable):
+ print
+ print red("!!!")
+ print red("!!!")+" %s" % red("Ebuild API
incompatibility")
+ print red("!!!")+" installed po
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Sun, Aug 28, 2005 at 02:46:28PM +0900, Jason Stubbs wrote: > On Saturday 27 August 2005 19:53, Brian Harring wrote: > > Please test this out; if you want to test the EAPI checking, tag > > EAPI=1 into an ebuild, and try making emerge bail. > > + print "EAPI check, unknown source "+str(x[0])+" for "+x[2]+", this > shouldn't occur",mylist > + sys.exit(1) > > These two are in the patch. Can that be made an exception please? That way > --debug could still be useful if the bug hits. That's debugging; not knowing (due to design flaws) all potentials for x[0], stuck that in debugging for anyone running the patch; people hitting it reporting back (hopefully), for debugging Final version won't include it. ~harring pgpWLv5Adox8k.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Sun, Aug 28, 2005 at 02:31:24AM -0700, Zac Medico wrote:
> Brian Harring wrote:
> >Please test this out; if you want to test the EAPI checking, tag
> >EAPI=1 into an ebuild, and try making emerge bail.
>
> I needed to patch ebuild.sh so that EAPI would be parsed. It bails out
> properly now.
Crud, didn't include that file.
Should be echo `echo "${EAPI:-0}"` actually
~harring
pgpCH7bAJN24M.pgp
Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Fri, Sep 02, 2005 at 08:31:26AM +0200, Marius Mauch wrote: > On 08/27/05 Brian Harring wrote: > > > Hola. > > > > Attached is a patch that > > A) adds EAPI awareness to portage; mainly, if >0, complain and be > >unwilling to merge the package > > Actually I just wrote also a patch for it (for 2.1), however instead of > complaining I just masked them (without unmask ability), just add a > check to gvisible() and getmaskingstatus() (actually just calling > dbapi.eapicheck()). That way it should be more transparent to users IMO. > Won't stop people from using ebuild(1) though. Masking makes a bit more sense, 'cept that information needs to be dumped out when resolution cannot occur. Further, it's a mild gotcha for users who wonder wth portage is masking a node. Better approach imo though, and would like to see that implemented rather then what I did with the stable patch. Incremental'ing EAPI during inherit is a no go; EAPI1 and EAPI0 are incompatible for execution, due to the src_compile configure/make breakup. Somebody care to split a masking patch for stable rather then the emerge modifications I did btw? I'm poking at ensuring an eapi=0 portage's generated eapi=1 cache entries are not used by an eapi=1 portage without a forced regeneration atm. That and the fact the 2.1 state should be decided, if we're going to have (effectively) two branches of development going at once, vs developmental line and maintenance branch. ~harring pgprNaO25CXkf.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Mon, Aug 29, 2005 at 01:52:50PM -0700, Zac Medico wrote:
> Brian Harring wrote:
> >
> >Please test this out; if you want to test the EAPI checking, tag
> >EAPI=1 into an ebuild, and try making emerge bail.
> >
>
> Well, it bails too often. :)
Mainly cause I generated something of a crap patch leaving a bit of
stuff out from the looks of it :)
> It seems that an explicit integer conversion is needed for > and <
> comparisons with mixed types
> (2.0.51-eapi-awareness-emerge-int-conversion.patch).
> >If you're less adventurous, please test the compatibility cache
> >testing;
> >in /etc/portage/modules
> >portdbapi.metadbmodule=portage_db_metadata.database
> >
>
> We need to make sure EAPI defaults to 0 when using old metadata. It seems
> like db_template.__getitem__() is a nice central place to do that
> (2.0.51-eapi-awareness-db_template.patch). I made another patch for
> bindbapi.aux_get() which does the same for old metadata from binpkgs
> (2.0.51-eapi-awareness-bindbapi.aux_get.patch).
The default'ing should occur outside of the cache layer imo, due to
the fact it has to occur in every dbapi.
Attached is a patch doing so, using gvisible to mask ebuild's EAPI >
portage's EAPI, and a check in doebuild to block people from trying to
do things with an ebuild that has an EAPI greater then portage's.
The doebuild mod is strictly for non depend phases; the data from a
depend's phase gets nuked and EAPI set to -EAPI if it's a later
version that what portage knows about.
Discussed on irc a bit, but repeating the reasons for those on the ml
who've missed the logic; effectively, an EAPI=0 portage generating an
EAPI=1 cache entry cannot be guranteed to generate it correctly,
contain all needed metadata.
So you flag it specially (negated EAPI), and don't store anything.
Add a few checks in so when an EAPI=1 portage finds the EAPI=1 cache
entry, which has been negated to mean effectively "someone who is
EAPI=1, please regenerate me", it gets regenerated, and the entry can
be trusted to hold all required EAPI1 metadata.
Attached is a patch culminating the feedback, with a few tweaks.
Should work a helluva lot better this time around :)
~harring
diff -purN portage-old/bin/ebuild.sh portage/bin/ebuild.sh
--- portage-old/bin/ebuild.sh 2005-08-29 02:16:41.0 -0500
+++ portage/bin/ebuild.sh 2005-08-29 17:02:00.0 -0500
@@ -1831,6 +1831,7 @@ for myarg in $*; do
echo `echo "$CDEPEND"` >> $dbkey
echo `echo "$PDEPEND"` >> $dbkey
echo `echo "$PROVIDE"` >> $dbkey
+ echo `echo "${EAPI:-0}"` >> $dbkey
echo `echo "$UNUSED_01"` >> $dbkey
echo `echo "$UNUSED_02"` >> $dbkey
echo `echo "$UNUSED_03"` >> $dbkey
@@ -1838,7 +1839,6 @@ for myarg in $*; do
echo `echo "$UNUSED_05"` >> $dbkey
echo `echo "$UNUSED_06"` >> $dbkey
echo `echo "$UNUSED_07"` >> $dbkey
- echo `echo "$UNUSED_08"` >> $dbkey
set +f
#make sure it is writable by our group:
exit 0
diff -purN portage-old/pym/portage.py portage/pym/portage.py
--- portage-old/pym/portage.py 2005-08-27 05:28:02.0 -0500
+++ portage/pym/portage.py 2005-08-29 17:11:30.0 -0500
@@ -81,7 +81,7 @@ try:
MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE,
MAKE_DEFAULTS_FILE, \
DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE,
CONFIG_MEMORY_FILE,\
- INCREMENTALS, STICKIES
+ INCREMENTALS, STICKIES, EAPI
from portage_data import ostype, lchown, userland, secpass, uid,
wheelgid, \
portage_uid, portage_gid
@@ -2402,6 +2396,15 @@ def doebuild(myebuild,mydo,myroot,mysett
raise
except:
pass
+ try:
+ eapi = abs(db[root][tree].dbapi.aux_get(mycpv,
["EAPI"])[0])
+ if portage_const.EAPI != eapi:
+ # can't do anything with this.
+ raise Exception("Unable to do any operations on
'%s', due to the fact it's EAPI is higher then this portage versions. Please
upgrade to a portage version that supports EAPI %i" % (mycpv, eapi))
+ except SystemExit, e:
+ raise
+ except Exception, e:
+ raise Exception("Unabl
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
Round 3, fixed all uglyness.
You *will* see uglyness for the changeover from flat_list to flat_hash
if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one
time hit, and is the reason we blow away the cache on portage
upgrades.
Either way, full patch, just correction of a few instances where it
user visible warnings were popping up, but not required.
~harring
diff -purN portage-old/bin/ebuild.sh portage/bin/ebuild.sh
--- portage-old/bin/ebuild.sh 2005-08-29 02:16:41.0 -0500
+++ portage/bin/ebuild.sh 2005-08-29 17:02:00.0 -0500
@@ -1831,6 +1831,7 @@ for myarg in $*; do
echo `echo "$CDEPEND"` >> $dbkey
echo `echo "$PDEPEND"` >> $dbkey
echo `echo "$PROVIDE"` >> $dbkey
+ echo `echo "${EAPI:-0}"` >> $dbkey
echo `echo "$UNUSED_01"` >> $dbkey
echo `echo "$UNUSED_02"` >> $dbkey
echo `echo "$UNUSED_03"` >> $dbkey
@@ -1838,7 +1839,6 @@ for myarg in $*; do
echo `echo "$UNUSED_05"` >> $dbkey
echo `echo "$UNUSED_06"` >> $dbkey
echo `echo "$UNUSED_07"` >> $dbkey
- echo `echo "$UNUSED_08"` >> $dbkey
set +f
#make sure it is writable by our group:
exit 0
diff -purN portage-old/pym/portage.py portage/pym/portage.py
--- portage-old/pym/portage.py 2005-08-27 05:28:02.0 -0500
+++ portage/pym/portage.py 2005-08-29 19:28:34.0 -0500
@@ -81,7 +81,7 @@ try:
MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE,
MAKE_DEFAULTS_FILE, \
DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE,
CONFIG_MEMORY_FILE,\
- INCREMENTALS, STICKIES
+ INCREMENTALS, STICKIES, EAPI
from portage_data import ostype, lchown, userland, secpass, uid,
wheelgid, \
portage_uid, portage_gid
@@ -2402,6 +2396,15 @@ def doebuild(myebuild,mydo,myroot,mysett
raise
except:
pass
+ try:
+ eapi = abs(db[root][tree].dbapi.aux_get(mycpv,
["EAPI"])[0])
+ if portage_const.EAPI != eapi:
+ # can't do anything with this.
+ raise Exception("Unable to do any operations on
'%s', due to the fact it's EAPI is higher then this portage versions. Please
upgrade to a portage version that supports EAPI %i" % (mycpv, eapi))
+ except SystemExit, e:
+ raise
+ except Exception, e:
+ raise Exception("Unable to pull EAPI from cpv %s, tree
%s; can't confirm that it's supported by this portage, thus unable to merge it:
Exception was '%s'" % (mycpv, tree, e))
if mysplit[2] == "r0":
mysettings["PVR"]=mysplit[1]
@@ -3911,7 +3914,10 @@ def getmaskingstatus(mycpv):
rValue.append("package.mask")
# keywords checking
- mygroups = portdb.aux_get(mycpv, ["KEYWORDS"])[0].split()
+ mygroups, eapi = portdb.aux_get(mycpv, ["KEYWORDS", "EAPI"])
+ if abs(eapi) > portage_const.EAPI:
+ return ["required EAPI %i, supported EAPI %i" % (eapi,
portage_const.EAPI)]
+ mygroups = mygroups.split()
pgroups=groups[:]
myarch = settings["ARCH"]
pkgdict = settings.pkeywordsdict
@@ -4531,7 +4537,12 @@ class bindbapi(fakedbapi):
else:
myval = string.join(myval.split(),' ')
mylist.append(myval)
-
+ if "EAPI" in wants:
+ idx = wants.index("EAPI")
+ try:
+ mylist[idx] = abs(int(mylist[idx]))
+ except ValueError:
+ mylist[idx] = 0
return mylist
@@ -4811,6 +4822,12 @@ class vardbapi(dbapi):
else:
myd = ""
results.append(myd)
+ if "EAPI" in wants:
+ idx = wants.index("EAPI")
+ try:
+ results[idx] = abs(int(wants[idx]))
+ except ValueError:
+ results[idx] = 0
return results
@@ -5077,9 +5094,9 @@ auxdbkeys=[
'DEPEND','RDEPEND', 'SLOT', 'SRC_URI',
'RESTRICT', 'HOMEPAGE', 'LICENSE', 'DESCRIPTION',
'KEYWORDS', 'INHERITED', 'IUSE', 'CDEPEND',
- 'PDEPEND', 'PROVIDE',
+ 'PDEPEND', 'PROVIDE', 'EAPI',
'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
- 'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 'UNUSED_08',
+ 'UNU
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Tue, Aug 30, 2005 at 12:38:16PM +0200, Marius Mauch wrote: > On 08/30/05 Paul de Vrieze wrote: > > > On Monday 29 August 2005 22:52, Zac Medico wrote: > > > Brian Harring wrote: > > > > Please test this out; if you want to test the EAPI checking, tag > > > > EAPI=1 into an ebuild, and try making emerge bail. > > > > > > Well, it bails too often. :) > > > > > > It seems that an explicit integer conversion is needed for > and < > > > comparisons with mixed types > > > (2.0.51-eapi-awareness-emerge-int-conversion.patch). > > > > I don't think it is a wise path to interpret EAPI's as integers at > > all. There should not be guarantees of forward or backward > > compatibility between versions. Interpreting them as integers seems > > to imply that. Basically I think that EAPI could be anything wanted, > > similar to SLOTS. Of course using digits is easy for humans. > > I've to agree with Paul here, portage should hold a list of all EAPI > values it understands, not a maximum value. Stable knows of one version- so containment check versus > is moot; negated values are disallowed for eapi anyways. Note that the rewrite will do containment tests instead, but rewrite will also support eapi1 and eapi0 rather then just one. ~harring pgpQ9RYM8c6Tk.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Tue, Aug 30, 2005 at 03:15:15PM +0200, Marius Mauch wrote: > On 08/30/05 Brian Harring wrote: > Problem is that you then rely on python always evaluating "somestring" > > 0 as True which I don't think is a good idea (it holds true even for "0" > > 0), if you treat 0 as a string you get problems (as some strings are > > "smaller" than 0) and you can't convert all strings to ints. In the initial proposal of EAPI, it was intended as version ebuild format #; think format specifications. Float is probably better, but strings being slipped in I don't see any good reason to allow, nor has it been requrested (it was specifically ixnayed when the idea was hashed out actually). Aside from that, again, stable is capable of a single eapi version; if it's a string, the code catches the value error and knocks it down to eapi0 due to the reasons described above. Further reason why string is a no go indicated in the code; if eapi0 portage tries regening an eapi1 cache entry, it stores negated eapi version with no other metadata. Allowing strings nukes that approach, unless you disallow - as the first character (which would be demonstration of strings not being incredibly well suited for eapi settings imo). > What's the point of using > anyway? Simplicity in the code right now, since stable will *never* support anything but eapi0. It's an easy check. ~harring pgpqwMWMA9j4f.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Tue, Aug 30, 2005 at 04:47:51PM +0200, Paul de Vrieze wrote: > On Tuesday 30 August 2005 15:28, Brian Harring wrote: > > On Tue, Aug 30, 2005 at 03:15:15PM +0200, Marius Mauch wrote: > > > On 08/30/05 Brian Harring wrote: > > > Problem is that you then rely on python always evaluating > > > "somestring" > 0 as True which I don't think is a good idea (it holds > > > true even for "0" > > > > > > > 0), if you treat 0 as a string you get problems (as some strings > > > > are "smaller" than 0) and you can't convert all strings to ints. > > > > In the initial proposal of EAPI, it was intended as version ebuild > > format #; think format specifications. > > > > Float is probably better, but strings being slipped in I don't see any > > good reason to allow, nor has it been requrested (it was > > specifically ixnayed when the idea was hashed out actually). > > What the format of the values of EAPI is is irrelevant. The matter is that > there is no way to know that version 5 is in any way compatible with > version 4, which in turn is also incompatible with version 3. Seeing > versions as numbers would imply an ordering in the versions. Such an > ordering would seem to imply compatibility or something the like, while > versions could differ significantly. There actually is an ordering to the versions though- they're going to map out to later portage releases (eapi3 release's portage version >= eapi4 release's portage version >= eapi5 releases' portage version). Why am I being so damned stubborn about numbers for this? Cause I don't want users having to dink around with knowing that eapi="really cool version" somehow maps out to 3.1. Further, eapi version *likely* will wind up as the slotting for any split out portage-ebuild package, and slots traditionally have always been ints. > About the ordering, what if there was some new format that would enable to > very easilly write packages for perl modules but sucked at anything else? > This would be a different format alltogether, not a different version. > > If EAPI's were more than just strings, at some point it might even be > possible to associate packages with EAPI's so that portage would > automatically signal the EAPI was unsupported, find the relevant package > and merge the plugin. Restart itself and merge the ebuild that needed the > new EAPI. The plugin ebuild could ensure that the portage version is > compatible. The slotting comment above is in regard to it. People will hit annoyances along the way, but slot==EAPI for portage-ebuild ought to allow a sane resolver to dig it's way back. Won't work for stable, due to the fact stable's resolver isn't smart enough, but rewrite's should support the "lets hop from portage version to portage version in upgrading" bit. Aside from that, EAPI is specific to *ebuilds*. New format is a seperate beast that shouldn't be shoe horned into ebuilds imo. > > Aside from that, again, stable is capable of a single eapi version; if > > it's a string, the code catches the value error and knocks it down to > > eapi0 due to the reasons described above. > > Then the code is broken. It should check whether the value is "0" or the > empty string "" or unset. If it is the latter it should change it to "0", > if it is anything else it should just treat it as an unsupported format > and ignore that particular ebuild. (and try to use other versions of the > package that are still in the old format). Yielding on this, since I agree on it. Will update the next round of patches so it's stricter. > > Further reason why string is a no go indicated in the code; if eapi0 > > portage tries regening an eapi1 cache entry, it stores negated eapi > > version with no other metadata. Allowing strings nukes that approach, > > unless you disallow - as the first character (which would be > > demonstration of strings not being incredibly well suited for eapi > > settings imo). > > As long as it does not generate "-0" this broken behaviour should not > break anything as long as portage 2.1 does a forced regen or cache > evaluation on installation (when detecting a 2.0 version present) or > first run. Wouldn't call it broken behaviour; the reasoning behind doing this is that people, regardless of any warning we give them, *will* have caches that are used by multiple portage versions. Bad setups will results in older portage installations updating a central cache; the negating allows for older portage installations and newer to play nice in the cache, n
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Tue, Aug 30, 2005 at 07:46:24PM +0200, Marius Mauch wrote: > On 08/29/05 Brian Harring wrote: > > That and the fact the 2.1 state should be decided, if we're going to > > have (effectively) two branches of development going at once, vs > > developmental line and maintenance branch. > > Well, basically I wanted to deploy elog and Jason mentioned some other > changes as well, and while talking about it we couldn't find much in > head that we didn't want out. Unfortunately you weren't around at that > time. Also I think we could use 2.1 to add all the hacks we need for > transitioning (like the EAPI and Manifest stuff). I'd rather tag the hacks into stable release, as what the EAPI patch is intended to do. Reasoning is pretty straightforward; I trust stable code to hold less user visible bugs now, then what 2.1 would hold- stable has been shoved through the ringer; 2.1 hasn't been shoved through a comparable ringer. Further, if we're tagging compatibility hacks for 2.0.51 -> 3.0, the thing that matters is the compatibility additions, not extra (potentially buggy) features. Don't get me wrong- I'm still watching 2.1 bugs, but mainly for correction of stuff w/in rewrite. 2.1 *could* be made into a full release line, I just am convinced the time to do so has come and gone already. Rewrite isn't complete, but the base of it is saner then 2.x's, and people (beyond me) are actively working on it. Further, people are sniffing around re: capabilities the rewrite has natively, N portdir's for example for the -osx crew. My 2 cents, at least. ~harring pgpskUMjjL4Mq.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
Hokay, this is pretty much my response to the string vs int stuff,
slammed into one email.
First clarifying the assumption I'm after a max test for eapi testing;
not the case- stated so already in this thread.
Restating it again, for stable, a check of if it's > 0 (with int
conversion failures being knocked up to 1 fex) is effectively the same
thing as if str(portage_const.EAPI) in ("", "0"). Stated also that
for rewrite, this approach won't work, and you're stuck specifying
either a range of supported EAPI (max *and* min), or specifying every
single individual eapi identifier, which I'll clarify before is going
to burn any maintainer of such code.
On Wed, Aug 31, 2005 at 12:52:53PM +0200, Marius Mauch wrote:
> On 08/30/05 Brian Harring wrote:
>
> > Why am I being so damned stubborn about numbers for this? Cause I
> > don't want users having to dink around with knowing that eapi="really
> > cool version" somehow maps out to 3.1. Further, eapi version *likely*
> > will wind up as the slotting for any split out portage-ebuild package,
> > and slots traditionally have always been ints.
>
> Thanks for that last comment, it actually prooves our point ;)
> Yes, SLOTs are *generally* numeric strings, as they are generally tied
> to major versions. But as soon as more than the major part of the
> version is important ints don't work anymore (and never ever mention
> float again, version numbers aren't floats), and in some cases you find
> non-version data in SLOT (like $CTARGET).
Summation of string based failings.
First off, a modifier of an EAPI, say EAPI=3-strict isn't valid; why?
strict is most likely a restriction, a specific negation of some
aspect of eapi3.
Restrict is the place for it, or a similar metadata key. Yes this
doesn't hold across all cases (someone could've just decided 3-strict
was a cool name fex), but throwing it out there to point out that eapi
is ebuild templates, not restrictions.
Further, any deviation where loss of backwards compatibility would
occur (limiting or extending bash syntax) is implicitly another
format, thus beyond eapi's keen. EAPI is ebuild (the format) spec/api
*only*, see comments below re: format support for further explanation.
Re: tagging EAPI at the top of a file, infra would probably shoot me
for doing such- till a live, fully compatible and *roughly* equivalent
parser is available, portage would have to do a bit of grepping,
jacking up the regen times.
One thing I'm going to clarify also, is that the rewrite *does not*
make it hard to tag new formats in. There's no reason to jam
everything into ebuilds, especially considering we are bound by
existing definitions, and a body of 20,000 live ebuilds users/devs
would get rather pissed off about if we forced a change across it.
If it's not an adjustment, a tweak, an extension/subtraction of ebuild
functionality, eapi isn't applicable imo, and a seperate format is the
route to go (one that could easily live alongside in the tree).
EAPI is just for ebuild format. alt formats (whether sh based or
otherwise) would be wise to version their version also.
Either way, getting to the point of why strings suck badly in the
point where it matters- as format version identifiers for the code
that does the actual work.
Example code for numeric, say we have eapi 0-5 at this point.
def configure(self):
if self.pkg.eapi > 0:
...call configure phase...
return
example code for strings
def configure(self):
if self.pkg.eapi in ("0", "1", "2", "3", "4", "5"):
...call configure phase...
return
This sucks.
the response is "well, stick it into a list somewhere". Can do, but
you have to mangle each list everytime you add a new eapi. That's a
good way to lead to dumb ass bugs when tinkering with a new eapi
level.
or... capabilities, as suggested.
capabilities = {"0":["configure"],
"1":["configure"],
"2":["configure"],
"3":["configure"],
"4":["configure"],
"5":["configure"],}
def has_capability(eapi, capability):
try:return capability in capabilities[eapi]:
except KeyError:return False
def configure(self):
if has_capability(self.pkg.eapi, "configure"):
...call configure phase...
return
or (slightly nastier example)
# eapi5 being eapi4, just with bash side changes. implicit is that
# their is bash side changes between each eapi, just stating it
# explicitly to a
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Wed, Aug 31, 2005 at 11:55:40AM +0200, Paul de Vrieze wrote: > pps. For being able to state that one eclass/ebuild is compatible with two > (overlapping) formats I also argue to make EAPI a space separated list of > supported API's. As an example for why this would make sense take for > example the depend.apache eclass. This eclass is compatible with both the > proposed EAPI1 as the current EAPI0 as it does not use/provide > src_compile. Writing separate eclasses with just different EAPI versions > is rather pointless, so allowing a list of supported ones makes sense. I > do realise that this makes things a bit harder as first the EAPI of an > ebuild(including it's eclasses) must be decided, and only then might > proper parsing be possible to happen. Preferred solution (imo) for such a case is eapi defined prior to the inherit iff the eclass is capable of multiple eapi levels. It's effectively the same thing, but allows for the ebuild/eclass authors to have total control over it, and doesn't require a second duff eclass. Regarding having the eclass specify compatible eapi versions, and portage somehow tracking ebuild eapi and comparing the two (and bailing if the containment test fails), not much for. Reasoning is thus- new eapi3 is available, ebuild dev bumps (fex) apache so that it uses eapi3; if the eclass limits to <3, the ebuild *cannot* use the eclass. Need a duplicate eclass, or the ebuild gets the eclasses code jammed into it, or (I really don't like this case) eclass gets eapi3 tagged in, despite the fact the eclass may not be eclass compatible without the ebuild doing a little dance (for eapi0/1, defining it's own src_configure and src_compile fex). Or the ebuild is kept at eapi2, till eapi3 is available in the eclass, slightly saner approach, but also effectively adoption of new eapis due to the fact that you cannot localize the build up of eapi3 compatibility crap in the ebuild, then port it into the eclass (then marking the eclass as explicitly eapi3 compatible). Eclass templates I prefer to keep as flexible as possible; granted people can shoot themselves in the foot, but that's always the case (add a safety to the gun, they'll find the safety and maybe gutshot themselves instead while playing with the safety :). Point I'm getting at is that artificial limiters people find a way around, with the workaround being usually a helluva worse then what the limitation is trying to block. Imo, at least. Either way, it's something that once applied, cannot be reversed; would rather leave it loose, then tighten it down the line if people seem chronically apt to screw it up. ~harring pgpd6EqmkkEli.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Wed, Aug 31, 2005 at 08:10:35AM -0700, Zac Medico wrote: > Brian Harring wrote: > >Round 3, fixed all uglyness. > >You *will* see uglyness for the changeover from flat_list to flat_hash > >if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one > >time hit, and is the reason we blow away the cache on portage > >upgrades. > > > >Either way, full patch, just correction of a few instances where it > >user visible warnings were popping up, but not required. > >~harring > > > > This patch worked well for me with your suggested settings in > /etc/portage/modules: > > portdbapi.metadbmodule=portage_db_metadata.database > portdbapi.auxdbmodule=portage_db_flat_hash.database > > With "emerge metadata" I was able to convert the existing metadata from the > rsync mirrors into flat_hash format. There were no problems emerging > ebuilds and I was also able to emerge binpkgs with old metadata. cool, thank you for testing that. ~harring pgpCCI70eDTdz.pgp Description: PGP signature
Re: [gentoo-portage-dev] Extended package.mask
On Wed, Aug 31, 2005 at 06:44:33PM +0200, Fabian Zeindl wrote: > Hi > > I'm using gentoo and the best package manager of the world for over one year > now, and I'm quite satisfied :-). > One idea I had: I regularly stumble across ebuilds which are masked in > package.mask because they break certain other packages or something like that. > It would be genial if portage wouldn't just say > "this ebuild is masked cause it breaks XYZ", but detect whether I have XYZ > installed and warns me if so resp. emerges normally otherwise. If I install > package XYZ some time later portage must of course detect this and say "you > have package ABC installed, which makes problem with the package you wan't to > install now, what do you want to do?". > > Is this an insane idea? It would make portage even better if it could resolve > situations/environments/configurations where emerging/unmerging something will > cause problems. You forgot the inverse of it, you've merge the package that's loosely masked as you're proposing, portage now has to tell block you from merging XYZ (due to the package breaking xyz). Offhand... I'm not much for this. The scenarios where it occurs, and a package *could* released are a bit rare imo; basically, gain vs cost. > PS: The second question I want to ask: Why are some packages so long marked ~ > unstable when their upstream is stable. Is there a policy how long to mark new > ebuilds or something like that? Upstream stability does not match actual stability; we stable dependant on our experiences, bugs, and user feedback with a particular version. ~harring pgpuRHwSvseDq.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: if doing a piped call in unpack, check pipestatus
On Thu, Sep 01, 2005 at 11:12:02PM +0900, Jason Stubbs wrote: > On Tuesday 09 August 2005 20:28, Brian Harring wrote: > > Hola. > > Bug 101035 > > http://bugs.gentoo.org/show_bug.cgi?id=101035 > > > > InCVS now also... > > I can't find this code in stable at all. Is this something else? Twas in stable, then somebody removed unpack from stable's ebuild.sh; I'll tag it back in. ~harring pgpfq69NlJnC0.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Thu, Sep 01, 2005 at 10:27:44PM -0700, Zac Medico wrote: > Paul de Vrieze wrote: > >On Wednesday 31 August 2005 14:57, Brian Harring wrote: > > > >>Re: tagging EAPI at the top of a file, infra would probably shoot me for > >>doing such- till a live, fully compatible and *roughly* equivalent parser > >>is available, portage would have to do a bit of grepping, jacking up the > >>regen times. > > > > > >If in cache EAPI can be gotten from the cache. If not, I don't think it > >matters where in the file EAPI occurs from the standpoint of getting it's > >value. The only thing would be that in the future a fast EAPI parser could > >be made that would just look at EAPI and get its version. I could easilly > >write you such a parser. > > It is impossible write a parser for an unconstrained and unknown format > that may exist in the future. If we put a constraint on the format, in > order to parse the EAPI, then we contradict our original goal (to > unconstrain the format). > > A better approach IMO would be to store the EAPI in a separate file such as > metadata.xml. This would allow *absolute* flexibility in the "ebuild" > format. Portage would be able to select an appropriate parser with no need > to examine the "ebuild" itself. Disallows eclasses from ever setting eapi. Like I've said, EAPI is ebuild specific. Ebuild is a format; eapi defines revisions of it, in my mind a minor revision of the ebuild 1 format. Any form of loss of backwards compatability *should* be a different format, .ebuild2 for all I care. Trying to use EAPI to allow for N different formats into one format is wrong from where I sit; you would need a container format for it, which ebuild wasn't designed for (nor is it easily extensible to be made so I posit). EAPI's original specification was for handling addition of new funcs, different hooks in the ebuild; I prefer it remain as this. The core rewrite is format agnostic, if a new format is defined (whether a massively managled version of ebuild or flat out new), it's a seperate format and should be handled via the core, not via ebuild specific package handling. There's no reason a repository can't hold multiple formats internally; the capability is there, use that rather then trying to jam too much into EAPI, imo at least. ~harring pgpG3l5aSg1zX.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: initial EAPI awareness
On Fri, Sep 02, 2005 at 10:53:05AM +0200, Paul de Vrieze wrote: > On Friday 02 September 2005 08:04, Brian Harring wrote: > > > > Like I've said, EAPI is ebuild specific. Ebuild is a format; eapi > > defines revisions of it, in my mind a minor revision of the ebuild 1 > > format. Any form of loss of backwards compatability *should* be a > > different format, .ebuild2 for all I care. > > The new proposed format loses backwards compatibility. If there is > backwards compatibility no new format or api version is needed. EAPI > should work on the python level, not only on the ebuild.sh level. The two are utterly intertwined. Doesn't matter if portage thinks it's eapi 1 or 2, what matters is if the ebuild*sh env *creates* an eapi 1 or 2 env for execution. Python side just behaves a bit differently, ebuild*sh is where all of the actual execution occurs (and varies dependant on eapi). > > Trying to use EAPI to allow for N different formats into one format is > > wrong from where I sit; you would need a container format for it, > > which ebuild wasn't designed for (nor is it easily extensible to be > > made so I posit). > > No it would state that the eclass is 100% compatible with both by the > formats overlapping and the ebuild not threading outside the overlapped > area. And how is this going to work when it's more then just a split of a func? I'm not saying it can't be done, I'm saying it *shouldn't* be done. It's begging for screwups; relying on portage to pick and choose between eapi levels for an ebuild's execution env dependant on an eclass *requires* the ebuild to be fully compatible to those eapis, same for the eclass author. That's not so easy to do, and will lead to a buttload of case tests of EAPI, which isn't going to improve the code (imo). People aren't going to do it, is my opinion. It's unnecessarily complex with minimal gain; the gain being decreased due to the inherent complexity of A) pulling this off in portage B) comprehending *exactly* what portage will do in each case C) the ebuild/eclass author not only managing to get usage of *one* eapi correct, getting potentially N eapi's correct. D) eclasses that inherit eclasses requiring the same eapi compatability checks E) within two bodies of code, devs keeping seperated in their mind the differences between each eapi, and *never* blurring them. Note I'm not pulling the "everyone else is ignorant" crap that popped up on -dev ml; I'm stating that writing to an api is manual work, and people will screw up on it, just the same as in writing the implementation of that api, *we* will screw up on it. Bugs per line, it's unavoidable. Not advocating that we dumb everything down, advocating we don't add something that (assuming we get it right in implementation) is one hell of a pandora's box from the standpoint of complex ebuilds combined with complex eclasses. That's also not even getting into the fact that what you're proposing, effectively greping EAPI from the file is A) helluva lot slower on regen B) trying to turn ebuilds into a container format. They're not a container format, nor should they be (kitchen sink shouldn't be included). C) core's more then capable of supporting seperated formats; the 'container' should be the repository, not jammed within the package data. D) It's a helluva lot easier, and less chance for screwup to just define a new format, use a new extension, and have the repository implementation use a different pkg format for that file. E) New, non eapi=1 style change with a different extension wouldn't even be *seen* by the noncompliant repositories. Helluva lot simpler. Not advocating we define a new format every day; I'm advocating that we define a format (v1), we do tweaks to it, and define a new *seperate* format when we need stuff that is beyond the keen of the existing format. Shifting away from declarative syntax, moving away from bash syntax, etc. If it isn't akin to our existing definition of an ebuild, it's not an ebuild and should *not* be jammed into the ebuild format. It's a new beast, and should be seperated. That's also ignoring the additional interaction of elibs thrown into the mix. If what you're suggesting is implemented, eclasses *and* ebuilds would be litered with if [ "$EAPI" == 2 ]; then eapi2 specific stuff elif [ "$EAPI" == 1]; then eapi1 specific stuff else general, all eapi stuff fi Note that the code above also has a nice hidden bug in it that's not obvious till you think about it; there is no way to predict at the time of the ebuilds writing that EAPI3 will work for the els
Re: [gentoo-portage-dev] PATCH: ebuild unpack sources ${T}/environment during setup phase (bug 85803)
On Wed, Sep 07, 2005 at 07:20:44PM -0700, Zac Medico wrote: > http://bugs.gentoo.org/show_bug.cgi?id=85803 > > This trivial patch seems to work for me but I'm not sure if it will cause > regressions or not. Feedback is appreciated. Questionable in relation to binpkgs; binpkg phase execution is rather haphazard/broken anyways, but... question is if the build env (unpack -> install phases) needs to bleed through to the > install phases; if so, for binpkgs this change will break that. 'protecting' the ebuild env state when setup is executed would address this bug; basically, filtering what's saved/reloaded for the env dumps. Dunno. On the fence on this one- I went the route of just locking down chunks of the supplied ebuild*sh env, such that they were bound to the portage installation, not the ebuild's generated env; think that's the best long term solution, but it's not something that can be deployed in stable yet. ~harring pgp7Qvmq6jRuY.pgp Description: PGP signature
[gentoo-portage-dev] PATCH emerge --metadata and typo fix
Hola...
basically... got ticked off and knocked the --metadata code down to
absolute explicit settings, resetting category, dumping
PORTDIR_OVERLAY, and ignoring failures on cache cleansing.
The important thing is transfering the metadata; cache cleansing is
good (very good imo), but it's not worthy of a full tb from where I
sit, since it's a greater annoyance to users to lack the pregenerated
metadata, vs having extra files sitting around.
Either way... this should eliminate pretty much any remaining horkages
in it.
That and a typo in the "you're trying to unmerge a pkg that is in
system set" is fixed.
Patch has been tested by a couple of people, looking for a few more
testers; should be pretty much golden, but would like people to check
this over so that any remaining --metadata horkages disappear.
~harring
Index: emerge
===
--- emerge (revision 1992)
+++ emerge (working copy)
@@ -2205,7 +2205,7 @@
#avoid cluttering the preview printout with stuff that
isn't getting unmerged
continue
if not (pkgmap[x]["protected"] or pkgmap[x]["omitted"]) and (x
in syslist):
- print red("\a\n\n!!! '%s' is part of your system
profile. '%s'" % (mykey))
+ print red("\a\n\n!!! '%s' is part of your system
profile." % mykey)
print yellow("\a!!! Unmerging it may be damaging to
your system.\n")
if "--pretend" not in myopts and "--ask" not in myopts:
global EMERGE_WARNING_DELAY
@@ -2685,8 +2685,19 @@
pass
# we don't make overlay trees cache here, plus we don't trust
portage.settings.categories
porttree_root = portage.portdb.porttree_root
- pdb = portage.portdbapi(porttree_root,
portage.config(config_profile_path=portage.settings.profile_path[:], \
- config_incrementals=portage.settings.incrementals[:]))
+ conf =
portage.config(config_profile_path=portage.settings.profile_path[:], \
+ config_incrementals=portage.settings.incrementals[:])
+
+ conf["PORTDIR_OVERLAY"] = ''
+ conf.categories = portage.grabfile(os.path.join(porttree_root,
"profiles", "categories"))
+ try:
+ i = conf.categories.index("virtual")
+ if i != -1:
+ conf.categories.remove(i)
+ except (ValueError, IndexError):
+ pass
+
+ pdb = portage.portdbapi(porttree_root, conf)
cp_list = pdb.cp_all()
if len(cp_list) == 0:
print "no metadata to transfer, exiting"
@@ -2699,19 +2710,23 @@
current=1
def cleanse_cache(pdb, cat, saves, porttree_root=porttree_root):
- if len(saves):
- d={}
- for v in saves:
- d[portage.catsplit(v)[1]] = True
- for pv in pdb.auxdb[porttree_root][cat].keys():
- if pv not in d:
-
pdb.auxdb[porttree_root][cat].del_key(pv)
- else:
- try:
- pdb.auxdb[porttree_root][cat].clear()
- del pdb.auxdb[porttree_root][cat]
- except KeyError:
- pass
+ try:
+ if len(saves):
+ d={}
+ for v in saves:
+ d[portage.catsplit(v)[1]] = True
+ for pv in
pdb.auxdb[porttree_root][cat].keys():
+ if pv not in d:
+
pdb.auxdb[porttree_root][cat].del_key(pv)
+ else:
+ try:
+
pdb.auxdb[porttree_root][cat].clear()
+ del
pdb.auxdb[porttree_root][cat]
+ except KeyError:
+ pass
+ except KeyError:
+ # stop breaking things, cleansing is minor.
+ pass
savelist = []
catlist = []
pgp9kdcOVA6pO.pgp
Description: PGP signature
[gentoo-portage-dev] PATCH glep31 checking
Hola.
http://glep.gentoo.org/glep-0031.html<-- the details
http://bugs.gentoo.org/106544<-- the bug
http://bugs.gentoo.org/attachment.cgi?=68828 <-- the patch
Attached the patch also; one additional tweak is that file.size is now
a fatal check, since the tree seem's to finally be clean.
~harring
Index: repoman
===
--- repoman (revision 1992)
+++ repoman (working copy)
@@ -13,6 +13,13 @@
sys.path = ["/usr/lib/portage/pym"]+sys.path
version="1.2"
+allowed_filename_chars="a-zA-Z0-9._-+:"
+allowed_filename_chars_set = {}
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('a'),
ord('z')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('A'),
ord('Z')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('0'),
ord('9')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, map(ord, [".", "-", "_",
"+", ":"])))
+
import string,signal,re,pickle,tempfile
import portage
@@ -21,6 +28,8 @@
import portage_dep
import cvstree
import time
+import codecs
+
from output import *
#bold, darkgreen, darkred, green, red, turquoise, yellow
@@ -85,6 +94,8 @@
"filedir.missing":"Package lacks a files directory",
"file.executable":"Ebuilds, digests, metadata.xml, Manifest, and
ChangeLog do note need the executable bit",
"file.size":"Files in the files directory must be under 20k",
+ "file.name":"File/dir name must be composed of only the following
chars: %s " % allowed_filename_chars,
+ "file.UTF8":"File is not UTF8 compliant",
"KEYWORDS.missing":"Ebuilds that have a missing KEYWORDS variable",
"LICENSE.missing":"Ebuilds that have a missing LICENSE variable",
"DESCRIPTION.missing":"Ebuilds that have a missing DESCRIPTION
variable",
@@ -146,7 +157,6 @@
"IUSE.invalid",
"ebuild.minorsyn",
"ebuild.badheader",
-"file.size",
"metadata.missing",
"metadata.bad",
"virtual.versioned"
@@ -663,6 +673,29 @@
stats["file.executable"] += 1
fails["file.executable"].append(checkdir+"/"+y)
digestlist=[]
+
+ for y in checkdirlist:
+ for c in y.strip(os.path.sep):
+ if c not in allowed_filename_chars_set:
+ stats["file.name"] += 1
+ fails["file.name"].append("%s/%s: char '%s'" %
(checkdir, y, c))
+ break
+
+ if not (y in ("ChangeLog", "metadata.xml") or
y.endswith(".ebuild")):
+ continue
+ try:
+ line = 1
+ for l in codecs.open(y, "r", "utf8"):
+ line +=1
+ except UnicodeDecodeError, ue:
+ stats["file.UTF8"] += 1
+ s = ue.object[:ue.start]
+ l2 = s.count("\n")
+ line += l2
+ if l2 != 0:
+ s = s[s.rfind("\n") + 1:]
+ fails["file.UTF8"].append("%s/%s: line %i, just after:
'%s'" % (checkdir, y, line, s))
+
if isCvs:
try:
mystat=os.stat(checkdir+"/files")[0]
@@ -799,6 +832,13 @@
stats["file.size"] += 1
fails["file.size"].append("("+
str(mystat.st_size/1024) + "K) "+x+"/files/"+y)
+ for c in y.strip(os.path.sep):
+ if c not in allowed_filename_chars_set:
+ stats["file.name"] += 1
+ fails["file.name"].append("%s/%s: char
'%s'" % (checkdir, y, c))
+ break
+
+
if "ChangeLog" not in checkdirlist:
stats["changelog.missing"]+=1
fails["changelog.missing"].append(x+"/ChangeLog")
pgpoxv44vqNjL.pgp
Description: PGP signature
Re: [gentoo-portage-dev] PATCH glep31 checking
On Mon, Sep 19, 2005 at 04:12:08PM -0500, Brian Harring wrote: > Attached the patch also; one additional tweak is that file.size is now > a fatal check, since the tree seem's to finally be clean. Dropped the file.size becoming fatal change on the bug, and intend to for the final version. Either tweak the patch yourself, or gank it from the bug... it's a one line reversion. :) Pls test, kthx. ~harring pgpSn9miVa1jT.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: gentoolkit: Make portage.config object a global object
On Tue, Sep 20, 2005 at 12:37:27PM -0500, Paul Varner wrote: > On Fri, 2005-09-16 at 11:59 -0500, Paul Varner wrote: > > http://bugs.gentoo.org/show_bug.cgi?id=90680 > > > > Author: Paul Varner > > > > The current implementation of gentoolkit creates a portage.config object > > for every package object that it creates. While this is the correct > > thing to do from an object-oriented programming point of view, this > > implementation consumes an excessive amount of memory and CPU. The > > proposed patch changes the portage.config object for each package object > > to point to a single global object. > > > > Updated patch to add a semaphore to control access to the global > portage.config object. Unless anyone sees any other issues with this > patch, I will be placing it into gentoolkit. Reason for a semaphore over threading.Lock ? ~harring pgpmp2V7TRpXm.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: gentoolkit: Make portage.config object a global object
On Tue, Sep 20, 2005 at 06:55:44PM -0500, Paul Varner wrote: > On Tue, 2005-09-20 at 18:34 -0500, Brian Harring wrote: > > > Updated patch to add a semaphore to control access to the global > > > portage.config object. Unless anyone sees any other issues with this > > > patch, I will be placing it into gentoolkit. > > Reason for a semaphore over threading.Lock ? > > No reason other than I'm used to thinking of them as semaphores instead > of locks, so semaphore is what I searched for in the Python docs. Need to make some chunks of the rewrite thread safe, so I poked a bit; python -m timeit -n 1 -r 3 -s 'import threading;s=threading.Lock()' 's.acquire();s.release()' around 1us python -m timeit -n 1 -r 3 -s 'import threading;s=threading.Semaphore()' 's.acquire();s.release()' 20us Granted... It's being anal. Just surprised me, and figured I'd poke to find out if you had a specific reason for using sem over the simpler (pretty much straight cpython) Lock :) ~harring pgpIo2YQGJJ18.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: ebuild unpack sources ${T}/environment during setup phase (bug 85803)
On Wed, Sep 07, 2005 at 07:20:44PM -0700, Zac Medico wrote:
> http://bugs.gentoo.org/show_bug.cgi?id=85803
>
> This trivial patch seems to work for me but I'm not sure if it will cause
> regressions or not. Feedback is appreciated.
>
> Zac
> Index: portage-2.0.51.22/bin/ebuild.sh
> ===
> --- portage-2.0.51.22.orig/bin/ebuild.sh
> +++ portage-2.0.51.22/bin/ebuild.sh
> @@ -11,7 +11,7 @@ if [ ! -z "${PORTAGE_GPG_DIR}" ]; then
> SANDBOX_PREDICT="${SANDBOX_PREDICT}:${PORTAGE_GPG_DIR}"
> fi
>
> -if [ "$*" != "depend" ] && [ "$*" != "clean" ] && [ "$*" != "nofetch" ]; then
> +if [ "$*" != "depend" ] && [ "$*" != "clean" ] && [ "$*" != "nofetch" ] && [
> "$*" != "setup" ]; then
> if [ -f "${T}/environment" ]; then
> source "${T}/environment" &>/dev/null
> fi
Any further thoughts on this?
I'd like to see something done about the bug, but the binpkg setup
issue stands still.
~harring
pgpV8YuBlut9g.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Improved user information and communication
On Sat, Oct 01, 2005 at 11:57:01PM +0200, Daniel Stiefelmaier wrote: > > >>man emerge provides information on possible options, why should there > >>not be a way to get information on an ebuilds option??? > > > > > >because emerge is the tool, not the object. You wouldn't expect the > >openoffice documentation to cover examples for different kinds of > >letters, would you? > > err.. i think you got me wrong... i do not expect emerge to have a > built-in list of use flags. > The description should be in the .ebuilds or metadata.xml > But i hope you do agree, that eix or emerge are the appropriate tools to > dig such information. Nope, I don't agree. > (maybe eix more than emerge) > Just like "emerge -vv" will print information about the ebuild > maintainers in future release, if i got that right. Lifted from metadata.xml . Jamming a 'built-in list of use flags' into ebuilds/metadata.xml first of all is vague, second of all is going to jack up the tree's size, third of all will lead to a buttload of redundant information across the tree. So... nail it down, instead of the vague "eix/emerge should do this". If you're suggesting it read use.* info from profiles/use.*, state so. > >>The useflag "xprint" sounds like printing support, but doesn't tell > >>if you need it if you use cups or the kde-printing system or... > >>whatever. > > > > > >~ $ grep xprint /usr/portage/profiles/use.desc > >xprint - Support for xprint, http://www.mozilla.org/projects/xprint/ > > > >what do you need more? > > - ease of use > - elegance Eye of the beholder, not an objective point > - not need to know about every portage file (especially if new to gentoo) > - time efficiency (for dozens of flags) for x in flag1 flag2 flag3; do grep /usr/portage/profiles/use.*; done > - non-global flags See above. > eix also provides only information that you could grep in a more or less > elegant way. > or using google... Portage does you mean, since eix is just a tool that read directly from portage underlying files (and is going to get boned by portage changes to the underlying cache at some point). Use ufed, is my opinion on this, or write a tool/extension of existing tools. > >>Why do you think just because YOU don't need it, noone will? > > > > > >This is not a personal debate. The most important reason I see against > >this idea is that portage is a package manager, not a documentation > >center. What you're not groking here is that people work on A) what they find interesting B) what they think *needs* to be done. Via that ordering, the subjective bit you're throwing out is nulled; use flag display is minor compared to fixing portage's screwups from where I sit. Additionally turning the question, Why do you think just because YOU think it's needed, others will? > most programs, even those for gurus, print information about their > option or AT LEAST how to GET information. still, these programs are > "not a documentation center". This makes no sense. > >Why should the ebuild contain links to documentation? To be honest, > >not even the HOMEPAGE info is needed, it's just for the user's > >convenience. > > even emerge is "just for the user's convenience" > even distributions are "just for the user's convenience", who needs them? > i never heard someone argueing that a feature is needless because it is > convenient. > features are there to be convenient. Stretching the example to the extreme doesn't prove your point (don't do it if you want people to actually respond). > >I tend to refer to the UNIX principle: The right tool for the right > >job. For your problem, google (or any other search engine of course) > >is the right tool. > > what should i say? don't you have bookmarks of good sites? do you always > google for them? > of course you will get what you want in many cases but not always. > > another unix principle is that everybody can do everything the way he > likes. in this case, i prefer to have a "readers choice" instead of > googling and digging the perls. You've got the unix principle slightly wrong there- implicit is that if no alternative exists, the person persuing the alternative *implements* it themselves rather then riding others to do it. > also, i agree that emerge may not be the right tool for that. may be > "eix" or a new one. > what this is about, is including additional information (only one link > that will not hurt you) in the ebuilds or metadata.xml Guessing you're not aware of the cache, nor the dtd for metadata.xml Slapping stuff into the cache requires portage modifications, both for the package and for the rsync cache generation. It can be done, but the question before doing it is exactly what this is going to yield, is it really the right route, and what is going to be broken by this (since tools *do* read directly from cache entries even if it's daft to do so). > >Do you think we're all sadists? > > No, but hard to believe that you are n
Re: [gentoo-portage-dev] [PATCH] EAPI cleanups and fixes
On Tue, Oct 04, 2005 at 01:06:35AM +0900, Jason Stubbs wrote: > Don't like the size of this patch, but it's quite repetitive so... Wouldn't worry on the repetitive, it's repetitive due to the fact the *dbapi classes don't (ab|)use inheritance... > * Make all aux_get() functions return a list of strings again Why is this a good thing for EAPI? > * Move the EAPI validity check into a separate function > * Raise a specific UnsupportedAPIException instead of plain Exception > * Mark metadata of unsupported EAPIs as "INVALID" rather than -1 This doesn't really fly imo. You mark it as invalid, and _no_ portage version (regardless of it's ability to handle that EAPI) will _ever_ regenerate that entry. An old portage version updating the cache would make certain nodes never usable. The -1 is wrong, should be -EAPI. This however is getting back into the "eapi should be freeform, not just ints", which I thought I clarified why it should be ints (or people shut up instead of listening to me argue it). :) ~harring pgpF7iBOUVmQH.pgp Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] EAPI cleanups and fixes
On Tue, Oct 04, 2005 at 08:27:09AM +0900, Jason Stubbs wrote: > On Tuesday 04 October 2005 03:30, Brian Harring wrote: > > On Tue, Oct 04, 2005 at 01:06:35AM +0900, Jason Stubbs wrote: > > > Don't like the size of this patch, but it's quite repetitive so... > > > > Wouldn't worry on the repetitive, it's repetitive due to the fact the > > *dbapi classes don't (ab|)use inheritance... > > > > > * Make all aux_get() functions return a list of strings again > > > > Why is this a good thing for EAPI? > > Consistency. There is no mapping of names to types so any tool that uses > aux_get to enumerate values and assumes that they are strings (as they have > always been and still are for every other key) would break. What tools are querying EAPI via aux_get already? I'm not much for making it a string purely because everything else is strings. > > > * Move the EAPI validity check into a separate function > > > * Raise a specific UnsupportedAPIException instead of plain Exception > No problem with these two? Cleaner, so nope, no complaints. > > > * Mark metadata of unsupported EAPIs as "INVALID" rather than -1 > > > > This doesn't really fly imo. You mark it as invalid, and _no_ portage > > version (regardless of it's ability to handle that EAPI) will _ever_ > > regenerate that entry. > > > > An old portage version updating the cache would make certain nodes > > never usable. > > > > The -1 is wrong, should be -EAPI. > > So negative numbers signals invalid cache entries in the scheme? sort of. Indication of what EAPI capable portage is needed that is capable of regenerating that entry correctly. This gets back to why the original patch was doing >=0 tests. > > This however is getting back into > > the "eapi should be freeform, not just ints", which I thought I > > clarified why it should be ints (or people shut up instead of > > listening to me argue it). :) > > The only reasoning that I recall without checking is that it wouldn't > complicate certain code paths. That's not a valid reason, in my opinion. > > Can leave that debate alone though. s/INVALID/\-1/ in the patch works for me. -EAPI... -1 implies 3.x would be capable of regenerating it (may not be the case). It was partially code simplicity, but moreso it's sanity from where I sit. Refinements of specs, packages, projects, whatever, is numerical and increasing to indicate later revisions. EAPI="the horny toad" ain't going to happen (nor should it), imo. ~harring pgpMgLIyZgHds.pgp Description: PGP signature
[gentoo-portage-dev] [PATCH] bug 107770 ebuild screwing up A when executing phase by phase
Quicky description of the bug is that A was being defined to '' in the
ebuild env; due to the fact ebuild.sh automatically stomps the current
passed in env with the previous env (it's bad, we know it already :),
this resulted in A getting auto set to a bad value, and the value
lingering.
Attached is a patch that fixes this particular issue. Old bug from
the looks of it also, since that declare's been there for quite some
time.
Already in svn also, posting for those interested, and those affected.
~harring
Index: bin/ebuild.sh
===
--- bin/ebuild.sh (revision 2082)
+++ bin/ebuild.sh (working copy)
@@ -1733,8 +1733,10 @@
unset E_IUSE E_DEPEND E_RDEPEND E_CDEPEND E_PDEPEND
-declare -r T P PN PV PVR PR A D EBUILD EMERGE_FROM O PPID FILESDIR
-declare -r PORTAGE_TMPDIR
+for x in T P PN PV PVR PR A D EBUILD EMERGE_FROM O PPID FILESDIR
PORTAGE_TMPDIR; do
+ [[ ${!x-UNSET_VAR} != UNSET_VAR ]] && declare -r ${!x}
+done
+unset x
# Turn of extended glob matching so that g++ doesn't get incorrectly matched.
shopt -u extglob
pgpLkmFz6MzPi.pgp
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] bug 107770 ebuild screwing up A when executing phase by phase
On Tue, Oct 04, 2005 at 09:17:22AM -0500, Brian Harring wrote:
Responding to myself, because I'm an idiot, attached is the correct
patch.
~harring
Index: bin/ebuild.sh
===
--- bin/ebuild.sh (revision 2082)
+++ bin/ebuild.sh (working copy)
@@ -1733,8 +1733,10 @@
unset E_IUSE E_DEPEND E_RDEPEND E_CDEPEND E_PDEPEND
-declare -r T P PN PV PVR PR A D EBUILD EMERGE_FROM O PPID FILESDIR
-declare -r PORTAGE_TMPDIR
+for x in T P PN PV PVR PR A D EBUILD EMERGE_FROM O PPID FILESDIR
PORTAGE_TMPDIR; do
+ [[ ${!x-UNSET_VAR} != UNSET_VAR ]] && declare -r ${x}
+done
+unset x
# Turn of extended glob matching so that g++ doesn't get incorrectly matched.
shopt -u extglob
pgpRqUu2IkbBe.pgp
Description: PGP signature
Re: [gentoo-portage-dev] allow extra info to be echod on die
On Wed, Oct 05, 2005 at 04:33:17PM +, Thomas Matthijs wrote:
> Hi,
>
> I would really like a way to echo more information when a die happens so
> the users can paste it in their bug report
>
> Only needs a very simple change to ebuild.sh:
>
EBUILD_DEATH_HOOKS=''
diefunc() {
local funcname="$1" lineno="$2" exitcode="$3"
shift 3
@@ -289,6 +291,7 @@
echo "!!! ${*:-(no error message)}" >&2
echo "!!! If you need support, post the topmost build error, NOT this
status message." >&2
echo >&2
+ for x in $EBUILD_DEATH_HOOKS; do
+ ${x} "$1" "$2" "$3"
+ done
exit 1
}
^^^ something like that I think is a bit better; passing the diefunc
args to the hooks also.
~harring
pgpiX7sVt5WS1.pgp
Description: PGP signature
Re: [gentoo-portage-dev] allow extra info to be echod on die
Just an addendum to this... it went in, will be available in
2.0.51_rc4 and up.
If you want all registered death funcs to kick in,
EBUILD_DEATH_HOOKS="$EBUILD_DEATH_HOOKS the_name_of_your_func"
or if you want just your func to run
EBUILD_DEATH_HOOKS="the_name_of_your_func"
CC'ing gentoo-dev, since I'd expect some devs would be interested in
it.
~harring
On Wed, Oct 05, 2005 at 11:48:01AM -0500, Brian Harring wrote:
> On Wed, Oct 05, 2005 at 04:33:17PM +, Thomas Matthijs wrote:
> > Hi,
> >
> > I would really like a way to echo more information when a die happens so
> > the users can paste it in their bug report
> >
> > Only needs a very simple change to ebuild.sh:
> >
> EBUILD_DEATH_HOOKS=''
>
> diefunc() {
> local funcname="$1" lineno="$2" exitcode="$3"
> shift 3
> @@ -289,6 +291,7 @@
> echo "!!! ${*:-(no error message)}" >&2
> echo "!!! If you need support, post the topmost build error, NOT this
> status message." >&2
> echo >&2
> + for x in $EBUILD_DEATH_HOOKS; do
> + ${x} "$1" "$2" "$3"
> + done
> exit 1
> }
> ^^^ something like that I think is a bit better; passing the diefunc
> args to the hooks also.
> ~harring
pgpe8Zpe3I0pn.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
Yay, time for another flame war (just what I'd love to spend my time on). On Wed, Oct 05, 2005 at 04:06:47PM -0400, Alec Joseph Warner wrote: > >Hey Folks- > > > >I'm working on trying to get prefixed installs working. (As such, I'm > >using some code kindly modified by Michael Haubenwallner. ) I'm now > >in python code (portage) and would like to compare what I have with > >gentoo proper. > >Is this the location of the latest up to date portage code (in CVS, I > >realized devs might have more "up to date" code on their boxes): > >http://www.gentoo.org/cgi-bin/viewcvs.cgi/portage/pym/?root=gentoo-src > > >I looked through the dev documentation but couldn't find anywhere > >where it stated the actual location of the code in CVS. Any pointers > >would be great. We've moved to svn, which will be available via viewcvs shortly. Anonymous svn is available if you poke your head into irc.freenode.net #gentoo-portage, and pull it from the /topic. > >The issue I found is with pym/cache/fs_template.py. If I'm running as > >root (GID = 0) then this fails: > > > >def __init__(self, label, auxdbkeys, basepath=None, gid=-1, > >perms=0664, **config): > > """throws InitializationError if needs args aren't > > specified""" > > if not gid: > > raise > >cache_errors.InitializationError(self.__class__, "must specify gid!") Judging by location, that's 2.1. The extra opts are directly changable via configuration under the rewrite's code, so setting gid isn't hard. > >Shouldn't the logic be "if gid != -1"? I don't have access to a > >gentoo proper box right now... Yah. That said that code's been corrected under the rewrite. > I thought that part of brian's domain stuff in Savior was to cover this. > In either case no one should be writing any real code at this point > since no one has agreed on any sane way to pull this off. There needs > to be plenty of healthy discussion the pro's and con's of how things > should be done in regards to *-prefix. There was plenty of discussion about it last time around. It got sunk due to the fact that ciaran was a bit hell bent on having HOME integrated into it. All or none, was effectively the issue. The current line of thought on it is global prefix going in as an addition to an EAPI release, _strictly_ global prefix. The home crap (interdomain deps, querying information/location of package X) being a later release. Why? Doing prefix is hard. Doing prefix + home is harder. A global offset for PREFIX is going to require a fair amount of work. Tacking home into it (something that a global offset does _not_ require) is sadling a requested feature with a lesser feature. Do 'em seperate. Those who want interdomain, they can do the work. Those who want global offset, they can do that chunk. To head off the "it's not going to work for vim-*", yah, you'll be boned and have to install duplicate vim-* into the global prefix. Bluntly, either you dive in and start wading through the problems (fixing them as you go), or you sit back listening to how it's never going to work (thus accomplishing nothing). ~harring pgpNB0H1f7KlS.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 01:48:03PM -0700, m h wrote: > On 10/5/05, Brian Harring <[EMAIL PROTECTED]> wrote: > > Yay, time for another flame war (just what I'd love to spend my time > > on). > > Sorry, I'm really not trying to kindle any flames here. Heh, you're not, I'm just mildly pissy due to recurrent flamewars :) > So, on the topic of rewrite. Does there happen to be any testcases > for portage? Unittests, etc? I'd be nice to verify that rewrite > behaves properly (well, actually I want testcases for selfish reasons, > so I don't break code if I change anything) Niadda. Would love it if someone stepped up on that, since I don't particularly have the time right now :) > > The current line of thought on it is global prefix going in as an addition > > to an EAPI release, _strictly_ global prefix. The home crap (interdomain > > deps, querying information/location of package X) being a later release. > > > > Sorry, I don't know what EAPI means Ebuild API. Stable portage's ebuild env, funcs available, etc, is EAPI=0. Think of it as specifying the lib you want; the ebuild.sh specific changes that will be in the rewrite (elib, src_configure phase addition, deprived setup phase) would be EAPI=1 Basically, it gives us a way to introduce changes for how ebuilds are called/executed, what they expect from the env, without breaking things. It also gives us a way to break compatibility between EAPI versions; as long as the metadata exported is correct for the ebuild's EAPI and we're still doing a declarative design for calling into the ebuild, we can basically do whatever we want. > > Do 'em seperate. Those who want interdomain, they can do the work. > > Those who want global offset, they can do that chunk. > > I understand the interdomain stuff to be that prefixed packages can > depend on packages outside of their prefix? If so, I don't want this > "feature". I want an isolated sandbox. (Again, I realize others have > different needs) Pretty much. Best description is dependencies between root's. Global prefix (for osx) would either A) have a vdb for that prefix that represented the package.provided nodes B) have a domain for root=/, and do interdomain. A is likely route due to it being a helluva lot simpler; B is better/cleaner (imo), but it requires more work. Baby steps. :) > > To head off the "it's not going to work for vim-*", yah, you'll be > > boned and have to install duplicate vim-* into the global prefix. > > Bluntly, either you dive in and start wading through the problems > > (fixing them as you go), or you sit back listening to how it's never > > going to work (thus accomplishing nothing). > > ~harring > > > > So, I figure I'm sortof diving in with Haubi's code (against the > advice of those wanted a complete spec) since I think my needs seem to > be the most minimum subset of what others want in this feature. I > think it's a good way to help me understand the innards of portage > (though the code is pretty spaghetti right now). I presume you think > I should start with "rewrite" as a base? What is the current status > of rewrite? Rewrite's code is a heck of a lot cleaner; oop based for starters :) There is some nastyness, but it's encapsulated, and pretty much required. Current state of it is that I'm atm stuck on plugin code, and a slight change to the config handling code. Building/fetching are done, full immutable ebuild tree and vdb are done, immutable binpkg repository is done sans a package class. The mutable thing is basically querying the db; for vdb and binpkg, they need to be modifiable, able to add a package to the repository (merging). I'm working on that atm. Jason's doing resolver work, state of that I can't comment on (that's his thing). ebuild*sh side of it's already done- if you were looking to test out prefix building (experiment) I'd probably start there. > Sorry for all the questions, again I'm not looking to start a flame > war, I'm looking for some features and like what gentoo has done and > what can be done with it. I'm not trying to work against or outside > what the mainline developers are doing Wouldn't worry about it, my initial response was phrased strongly due to the fact whenever implementing global prefix is brought up the screams against it start prior to even being able to iron out, or discuss it. ~harring pgp8q9X1JECTK.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 09:57:03PM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 15:24:29 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | To head off the "it's not going to work for vim-*", yah, you'll be > | boned and have to install duplicate vim-* into the global prefix. > | Bluntly, either you dive in and start wading through the problems > | (fixing them as you go), or you sit back listening to how it's never > | going to work (thus accomplishing nothing). > > It can be made to work, so long as you don't a) jump in without proper > planning, b) assume that you'll not have to modify ebuilds and c) > demand that as soon as it's available, it works for all ebuilds. I'd point out that the last time prefix was brought up, all of those points were addressed. A) would like to hear what you think is required planning wise compared to the previous haubi prototype patch. B) Modifying the ebuilds is required, although modifying the underlying base defaults does cover a good swath of ebuilds C) See previous comments from last round of this stating "those who want it, have to do the work". ~harring pgpjCoEJ44DaB.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 04:16:06PM -0500, Kito wrote:
> My first question would be how to identify ebuilds that respect $
> {prefix}?
>
> A separate profile/keyword seems wrong.
Agreed.
>
> ICANINSTALLTO was the best idea presented, but that implies it would
> be a list of known working prefixes, which seems unrealistic.
DOMAIN is a saner name imo. Aside from that, listing every potential
target (/usr/bin, $HOME/bin fex) is whacky and getting into the HOME
stuff that comes after.
I would suggest root, and offset as the initial targets.
root would be implicit if DOMAIN isn't defined, and references / as
the prefix. offset is just as it sounds, a root offset, everything
from that tree installed at an offset.
> Maybe
> it would be better to have portage error check that globally at the
> load_config stage against a list of known stupid prefixes,
> stupidprefixes=["/usr","/","/bin"] etc. etc.
Bit beyond the ken of this discussion, but yah, agreed.
~harring
pgpon9LigF1m7.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 02:52:59PM -0700, m h wrote: > Hmmm, I'm not clear yet on the value of interdomain, but I'm sure > someone will enlighten me along the way... interdomain would be how ciaran's HOME request would be pulled off; user's 'domain', configuration settings + prefix offset would try to satisfy deps from the global domain '/'. When it can't, would resort to installing to the user's domain, assuming the ebuild is capable of it. > OK, so since I have a deadline (end of next week) I'd like to have a > simple prototype working. IE me running apache on FC4 with portage. > If I can do that then it's very likely that I'll be able to devote a > bit of time towards working on this (where this could be rewrite) (bug > fixes, polishing it up, unittests, etc). On that note, I need to know > what the timeframe for rewrite is. If it's to a state where I can > work with it then maybe I should try and start from there. But I'm > assumming it will be easier/quicker to get haubi's stuff working. Haubi's is what I'd recommend. Rewrite isn't anywhere near that sort of timeline. I'd also note realistically pulling it off within 10 days might be tricky, due to the fact it requires ebuild modifications plus dusting off and updating haubi's patch, along with buttload of testing. Or I'm just slow at implementing stuff. :) ~harring pgp7j0hS9Ee6H.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 11:29:56PM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 16:16:06 -0500 Kito <[EMAIL PROTECTED]> wrote: > | > b) assume that you'll not have to modify ebuilds > | > | I don't think anyone(devs) has made this naive assumption have they? > > pvdabeel has for pathspec. > > | > and c) > | > demand that as soon as it's available, it works for all ebuilds. > | > | I don't think anyone(devs) has made this naive demand have they? > > pvdabeel has for pathspec. You're not talking to pvdabeel right now, so kindly be quiet about him. Further, don't argue the dev point, no one cares- we're discussing prefix, not getting pulled into tangents of pathspec nor slams at other devs. > | ICANINSTALLTO was the best idea presented, but that implies it would > | be a list of known working prefixes, which seems unrealistic. Maybe > | it would be better to have portage error check that globally at the > | load_config stage against a list of known stupid prefixes, > | stupidprefixes=["/usr","/","/bin"] etc. etc. > > The plan was to have ICANINSTALLTO="root home prefixed" or somesuch. > The name isn't particularly clear on that... See email naming it as domain. Home being a later target; not sure about prefixed, but it's as good of a name as any other (arbitrary, in other words). ~harring pgpdiKyxShUiG.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 11:31:32PM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 16:13:06 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | A) would like to hear what you think is required planning wise > | compared to the previous haubi prototype patch. > > There has been no serious discussion on how *ebuilds* will use the > prefix system. Hacking econf and expecting PREFIX to be sufficient is > naive from a tree-perspective. econf isn't the only change required; the point is that whatever is decided, would have to be added to econf thus covering a good chunk of ebuilds in the tree that don't require fancy voodoo. The basic proposal of haubi's glep (ignoring the portage innard modifications) came down to addition of a prefix var, that would be required slipped in for any fs installation paths (--prefix=$PREFX fex). Beyond that, there is the shebang issue which can be addresses via a combination of automated scans/fixes, and fixing bugs as it's hit. Hardcoded vars in scripts for the path to a binary are an issue also, although again, scans can be done to at least check for it. Leaves mangling the build process so that the build framework of the package uses the prefix offset files, rather then / . For c/c++ source, usual trick from fink afaik involves a mangling of cflags with -I tacked in. Kinda ugly, although I'd expect there is a better route. Packages that pull include/compile settings/args from a utility (thinking python configuration tools, and pkgconfig) shouldn't be too horrid to change, since it's a matter of modifying it in one place (theoretically :). ~harring pgpcAHreE2uGq.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Wed, Oct 05, 2005 at 04:01:54PM -0700, m h wrote: > On 10/5/05, Brian Harring <[EMAIL PROTECTED]> wrote: > > On Wed, Oct 05, 2005 at 02:52:59PM -0700, m h wrote: > > > Hmmm, I'm not clear yet on the value of interdomain, but I'm sure > > > someone will enlighten me along the way... > > interdomain would be how ciaran's HOME request would be pulled off; > > user's 'domain', configuration settings + prefix offset would try to > > satisfy deps from the global domain '/'. When it can't, would resort > > to installing to the user's domain, assuming the ebuild is capable of > > it. > > > > > OK, so since I have a deadline (end of next week) I'd like to have a > > > simple prototype working. IE me running apache on FC4 with portage. > > > If I can do that then it's very likely that I'll be able to devote a > > > bit of time towards working on this (where this could be rewrite) (bug > > > fixes, polishing it up, unittests, etc). On that note, I need to know > > > what the timeframe for rewrite is. If it's to a state where I can > > > work with it then maybe I should try and start from there. But I'm > > > assumming it will be easier/quicker to get haubi's stuff working. > > Haubi's is what I'd recommend. > > Rewrite isn't anywhere near that sort of timeline. > > > > Great, so on that note, since Haubi's work is based on portage 2.1, > can anyone point me to the cvs repository? (Or are they all moved to > svn?) svn co svn://twobit.net/portage/main/trunk # <-- 2.1 branch ~harring pgppJS0M3o4YM.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 12:03:18AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 17:53:36 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | On Wed, Oct 05, 2005 at 11:29:56PM +0100, Ciaran McCreesh wrote: > | > On Wed, 5 Oct 2005 16:16:06 -0500 Kito <[EMAIL PROTECTED]> wrote: > | > | > b) assume that you'll not have to modify ebuilds > | > | > | > | I don't think anyone(devs) has made this naive assumption have > | > | they? > | > > | > pvdabeel has for pathspec. > | > > | > | > and c) > | > | > demand that as soon as it's available, it works for all ebuilds. > | > | > | > | I don't think anyone(devs) has made this naive demand have they? > | > > | > pvdabeel has for pathspec. > | > | You're not talking to pvdabeel right now, so kindly be quiet about > | him. > > Oh come off it. You claim that no dev has made this naive demand when > at least one quite clearly has on several occasions. All three of my > points are entirely relevant to the discussion, since every other time > this has come up at least one said mistakes has been made. Again, do you see anyone here claiming all are simple? It's the same points made last time around, those who want it do the work, it's going to be bumpy and will require work, but those who want it are the ones who have to do it. Bluntly, stick to the discussion of prefix, or go elsewhere. Bash pvdabeel if you like but sure as hell not on this list since no one cares, further since it's not relevant to the _current_ discussion, where people are actually attempting to pull this off. If as you posit, it's not been discussed/planned properly, I suggest you stick to contributing to the discussion and planning of it. The alternative is that people get fed up with the the ancillary bullshit, and they implement it themselves (which I'm sure you would dislike). ~harring pgpAkSIYj84Zj.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 12:14:30AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 18:00:12 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | Beyond that, there is the shebang issue which can be addresses via a > | combination of automated scans/fixes, and fixing bugs as it's hit. > | Hardcoded vars in scripts for the path to a binary are an issue also, > | although again, scans can be done to at least check for it. > > This one's a far bigger issue than might be initially obvious. It would > involve rewriting a whole load of autotools innards... Clarify. My knowledge of autotool innards is that it relies on $PATH for lookup of the tools it uses. > | Leaves mangling the build process so that the build framework of the > | package uses the prefix offset files, rather then / . For c/c++ > | source, usual trick from fink afaik involves a mangling of cflags > | with -I tacked in. Kinda ugly, although I'd expect there is a better > | route. > > Again, autoconf tinkering. Statement above is from digging through generated configure script. ~harring pgp3yrxiLkcfh.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 12:38:35AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 18:22:37 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | On Thu, Oct 06, 2005 at 12:14:30AM +0100, Ciaran McCreesh wrote: > | > On Wed, 5 Oct 2005 18:00:12 -0500 Brian Harring > | > <[EMAIL PROTECTED]> > | > wrote: > | > | Beyond that, there is the shebang issue which can be addresses > | > | via a combination of automated scans/fixes, and fixing bugs as > | > | it's hit. Hardcoded vars in scripts for the path to a binary are > | > | an issue also, although again, scans can be done to at least > | > | check for it. > | > > | > This one's a far bigger issue than might be initially obvious. It > | > would involve rewriting a whole load of autotools innards... > | > | Clarify. My knowledge of autotool innards is that it relies on $PATH > | for lookup of the tools it uses. > > It does in some places, it doesn't in others. It especially doesn't for > things that aren't normally found via PATH. It's a hell of a mess. Examples? ~harring pgpqc9GgtRZ5U.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 01:13:53AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 18:40:46 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | > It does in some places, it doesn't in others. It especially doesn't > | > for things that aren't normally found via PATH. It's a hell of a > | > mess. > | > | Examples? > > Of stuff in PATH? /bin/sh is assumed throughout to be a Bourne > compatible shell (and SHELL and CONFIG_SHELL aren't universally > honoured). uname, hostname and sed are called with hard paths (with > various fallbacks) in several early on stages. Of stuff not in path? > There's no standard and widely used way of digging up where libexec > tools are. What's being raised here is issues with making ebuilds handle prefix _perfectly_. Where are the portage issues, so that people can actually jump in and start testing out actual solutions, rather then conjecturing about what may or may not break? ~harring pgpvDKijE0RXh.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 02:07:32AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 20:01:34 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | What's being raised here is issues with making ebuilds handle prefix > | _perfectly_. Where are the portage issues, so that people can > | actually jump in and start testing out actual solutions, rather then > | conjecturing about what may or may not break? > > The issue is that you need to fix autoconf before you can claim that > any non-trivial test case works correctly. And how are you going to verify autoconf works perfectly without testing it? The point I'm making is that the only thing required of *portage*, is the prefix var being used internally, and handed down to the ebuilds. Ironing out the ebuild cruft is left to those who want it. Again, where is the hold up for *portage*? You're not offering any good reasons why people can't jump in and experiment with it despite the vaunted "plan it out first". Gurantee attempting prefix offset, things will *always* be missed for handling all issues if you're just planning it. Basically, I'm not seeing what point you're making beyond the fact there is work that's going to have to be done from the ebuild side of things (work you won't be able to predict I'd posit). Last time around, prefix was ixnayed due to the fact there was work. People want it, and are willing to do the legwork. What's the problem? Why the 101 holes before they even can attempt it? If you're after shooting the idea down (as I suspect), state so rather then death by a thousand cuts. Saves us time, really. Hell, haubi's patch already lays the ground work for testing it. I'm not seeing why you're being negative about people even working on it. ~harring pgpW7dLHJukQB.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 02:23:47AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 20:17:40 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | > The issue is that you need to fix autoconf before you can claim that > | > any non-trivial test case works correctly. > | > | And how are you going to verify autoconf works perfectly without > | testing it? > > Can't. Dead easy to verify that it will break without testing it, > though. Just look at the source. Break is a bit heavy, considering autoconf's usage of /bin/sh is pretty limited in terms of syntax. Looking at the source, it'll revert to / for certain bins. A forced autoreconf with a patched autoconf/autotools is required, but again, doesn't do jack with out testing. > | The point I'm making is that the only thing required of *portage*, is > | the prefix var being used internally, and handed down to the ebuilds. > | > | Ironing out the ebuild cruft is left to those who want it. Again, > | where is the hold up for *portage*? > > That's rather short-sighted... Portage is irrelevant without the > ebuilds. And ebuilds are irrelevant without portage. Point? My point experimentation can start for addressing the issues you keep pointing at still stands. > > | What's the problem? Why the 101 holes before they even can attempt > | it? If you're after shooting the idea down (as I suspect), state so > | rather then death by a thousand cuts. Saves us time, really. > | > | Hell, haubi's patch already lays the ground work for testing it. I'm > | not seeing why you're being negative about people even working on it. > > Because a botched solution is worse than no solution at all. You've > seen the mess we end up with when people start working with a > half-arsed initial setup. Well plan the sucker out then, as you advocated, or sit back and let them jump in and start working it out. Perhaps they'll decide it's completely unworkable (I doubt it, considering the fact fink's crappy handling of building has made it this far), or perhaps they'll get it working and you won't have as many holes to point at. Regardless, it's their time to spend working on it. Either chip in, or offer _constructive_ advice, or plain step back and let them try to get it working. Note the constructive. Stating it won't work and pulling a new reason why isn't constructive, pointing out each issue as you see it so they can address it (if it's an issue) is a bit more constructive. ~harring pgpLpvBGoQzGq.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 02:40:58AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 20:32:20 -0500 Brian Harring <[EMAIL PROTECTED]> > > Portage is considerably less work than the tree. Saving as much effort > as possible from an ebuild perspective should be a major consideration, > even if it makes the portage side more complicated. Think of how all > the ebuild-related problems are going to be solved first. Don't leave > it as an afterthought. Round and round we go. The ebuild related problems aren't going to be solved in portage till someone has a general solution that can be pushed into portage/ebuild.sh base template. That's something that requires people diving in and screwing with it. > | My point experimentation can start for addressing the issues you keep > | pointing at still stands. > > The sensible place to start experimenting is by adapting existing > ebuilds and tinkering with ebuild.sh, not by adding something which may > or may not end up being relevant to portage proper. Bluntly, what the hell do you think we're talking about here? In case you haven't caught on, there *are* portage modifications that have to go with it, meaning more then ebuild.sh. Regardless, I'll backport haubi's patch to stable if anyone is after screwing with it, unless michael's has a version that applies cleanly to .53_rc4. Enough dancing, would rather hand it off to those who are interested, and see what they come up with rather then fencing via email (and accomplishing nothing). Michael, got anything I can mold to .5*, or just backport the 2.1 mod? ~harring pgpmLvkkFehel.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 03:01:12AM +0100, Ciaran McCreesh wrote: > On Wed, 5 Oct 2005 20:48:26 -0500 Brian Harring <[EMAIL PROTECTED]> > wrote: > | > The sensible place to start experimenting is by adapting existing > | > ebuilds and tinkering with ebuild.sh, not by adding something which > | > may or may not end up being relevant to portage proper. > | > | Bluntly, what the hell do you think we're talking about here? In > | case you haven't caught on, there *are* portage modifications that > | have to go with it, meaning more then ebuild.sh. > Clarification of two things. First- this is external, including the patch. So comparing it to attempts that were done in a live tree is a bit of intentional bullshit/rhetoric. The entire intention of this is to work it out, *outside* of the tree, something those involved know. You seem to be out of the loop, not surprising considering your attitude towards this whole attempt. Second- Not having a clue about what the full set of modifications are going to be until you solve the ebuild side of it is *exactly* why people have to jump in and actually test the damn thing. You can solve as much of it up front as you know will be an issue, testing will reveal the additional issues. Under your suggested route, nothing is accomplished (potentially the reason you're suggesting all issues up front be addressed regardless of whether or not if they'll actually _be_ issues). What you're offering as a proposed/sane route is a route that produces nothing due to the fact you think everything must be solved up front, regardless if it turns out to be an issue or not (let alone identifying everything that may or may not be an issue) . Get the basic portage support up, they iron out the base mods initially needed, and jump in and identify the bugs that crop up further. Essentially, lets see how well this actually works out, rather then listening to you run your mouth about how it's a bad idea whenever it comes up, and all of these things will be issues (/bin/sh usage isn't an issue for the initial test target of osx). Either way, they're doing the work, you aren't, and you really don't have *any* say over their efforts until they finalize a solution and bring it to *devs* (not just you) for merging into mainline. So bluntly, shut up and let those who you think are being retarded, be retarded. Discussions on this list regarding those attempts shouldn't be heckled unless you're contributing to those efforts (and I truly mean *contributing*, not trying to punch holes in embryonic efforts that are trying to get off the ground addressing the major issues up front). Regardless, your points (repeatedly restated in the varying forms) have been noted, and those who are interested have no reason to not move forward with ironing out an implementation, and testing it. I suggest you sit quietly and let them do their work, rather then riding their asses. You might be surprised at what they come up with. If/When they push for inclusion, the merits of their efforts their solution (and outstanding issues) will be evaluated then. Back off and let them do their work, it's not affecting you in anyway, so again, you really don't have any say in it till they push for mainline. So... yeah. I'll post the prefix support backport patch once it's done, few days I'd expect since there is a fair amount of autotooling to deal with also. Those interested, I suggest you chip in, whether to spite ciaran (good or bad reasons, doesn't matter, result does >:), or to get this feature you want realized. ~harring pgpadtl900v4S.pgp Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 02:14:32PM +1000, Finn Thain wrote:
>
>
> On Wed, 5 Oct 2005, Kito wrote:
>
> [snip]
> >
> > My first question would be how to identify ebuilds that respect ${prefix}?
> >
> > A separate profile/keyword seems wrong.
> >
> > ICANINSTALLTO was the best idea presented, but that implies it would be
> > a list of known working prefixes, which seems unrealistic.
>
> What problem was ICANINSTALLTO intended to solve? IIRC, it was discussed
> on -dev in the context of vim plug-ins. Apart from vim plugins, has anyone
> found other problem packages?
>
> I'm wondering, would a constraint to the effect that "certain deps of pkg
> foo must be on the same prefix as foo" suffice for the vim plugin case?
>
> Or maybe that would work better if expressed, "pkg blah can not satisfy a
> dep from any pkg on a different prefix". Such constraints would be
> possible to implement with a new file in the profile (say, package.local).
That gets into more of ciaran's HOME installation target feature.
Current form for global prefix offset is
DOMAIN="root offset" , with offset == prefix offset.
If an ebuild doesn't have DOMAIN="offset", and you're doing a prefix
offset, it's unusable. No question of "can I use a dep from another
prefix", with prefix offset you're doing the deps full in an offset.
As stated earlier in this mess of a thread, HOME crap is being left to
those who want it, it'll be implemented by those who want it after
global prefix is ironed out (if it is accepted also).
~harring
pgpEHAG0zrtZw.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Questions about CVS locations and GID...
On Thu, Oct 06, 2005 at 07:29:12PM +0100, Ciaran McCreesh wrote: > On Thu, 06 Oct 2005 09:07:12 -0400 Alec Warner <[EMAIL PROTECTED]> > wrote: > | Then I suggest you file this e-mail away and when the time comes that > | the code is to be merged and it sucks ( and by sucks I mean more than > | "it doesn't meet Ciaran's software engineering standards" ) you can > | waive the mail in everyone's face. No project, specs or not, is ever > | written right the first time. At the very least someone can take his > | code and improve on it in the future. > > Yeah, right, because saying "I told you so" after the shit hits the fan > is ever so helpful. Right. What's stated above, and what this thread has come down to is that you're putting a lot of reqs and pointing at whatever roadblock you can for anyone who attempts it. Kudos, although bluntly, talk is cheap. You're not chipping in any way, you're fitting the exact definition of drobbin's "freak" http://www.gentoo.org/doc/en/articles/making-the-distro-p1.xml in how you're interacting with those attempting to pull this off (and I'm not even remotely commenting on your flaming of those merely asking questions). Produce a patch and maybe people will listen to you, otherwise those of us who are tired of listening to others state how it should be done (but do nothing but talk), are going to jump in and do it, addressing the issues thus pointed out. In other words, their really is no point in debating people attempting it. They're going too, and as I made pretty bluntly clear, the only say you've got on this is as a member of the dev community when/if they push for it to go mainline. Kind of hard to block it when nothing has been produced after all. Regardless, this list doesn't have flamewars, and I'd thank all participants to kindly back off (and yes, I'm being a damn hypocrite in stating it). If people are up for the continuing pissing match, take it private. ~harring pgphwQR49pGmo.pgp Description: PGP signature
[gentoo-portage-dev] ECONF_EXTRA handling: bug 38618
Anyone got a good reason bug 38618 shouldn't go in?
Essentially it's
./configure ...
-${EXTRA_ECONF} "$@"
+"$@" ${EXTRA_ECONF}
It allows for users to override ebuild defined configure options,
potentially shooting themselves in the foot, but in the same token
they can already shoot themselves in the foot via EXTRA_ECONF...
~harring
pgpcybpBOH5Nq.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Custom eclass question
On Sun, Oct 09, 2005 at 06:52:24PM -0500, Mikey wrote: > http://codeserver.wherever.net/pman/package_ids.php?action=package&id=10105 > [snip bits about wget screwing up] Others have already clarified that's it python side rather then bash so eclass is no go, but out of curiousity any got a good reason we can't modify FETCHCOMMAND to include FILE rather then just DISTDIR? With wget, it's just -O. Curl, piping probably (although haven't looked at their opts)... ~harring pgpQJZHqJeJYn.pgp Description: PGP signature
Re: [gentoo-portage-dev] Changelogs
On Mon, Oct 10, 2005 at 08:35:03PM +0200, Paul de Vrieze wrote: > > Hi guys, > > Is there any particular reason that the changelog was dropped from recent > portage versions. I'd like to have it to see what changed. Especially with > release candidates that sometimes develop issues. Intending on dumping the actual svn log messages, and mangling it into ChangeLog. Been a bit hit or miss afaik for rc3, but that's the intention. Easier for us; no duplication, the actual log message is sitting right in the vcs rather then as a change to a file in the vcs. So... yah, it's not being killed off, just a change of how it's done. ~harring pgpfNOMxY8Q1y.pgp Description: PGP signature
[gentoo-portage-dev] Cache rewrite backport
Hola. So... short version, for those who weren't aware, cache subsystem got gutted and rewrote in >=2.1 base code. Couple of reasons why the old design needed to be pitched 1) Lots of forced stat calls for no good reason. 2) designed around per category, which doesn't fly when you're doing a remote cache- pooling being *required* for rdbms for a single repository cache sucks. 3) Two seperate caches per repository effectively; eclass_cache (which is global), and repository cache. So... those are the main 3. Cache rewrite that's in 2.1, and 3.x (latter being saner since it's actively worked on) I've backported to stable, and attached to this email. It's... kind of large, but there's a couple of good reasons for it. The original cache cleansing code is included, throwing out emerge's reimplementation of it (this one is a bit more extensible)- beyond that, eclass_cache got ripped out of portage.py and into it's own module. Portage.py modifications are pretty minimal also. I'd be *very* interested in timing runs for with, vs without- the cache rewrite lacks a lastX cache that was used in stable's cache to cover up performance issues. Probably will need to throw in a wrapper to stable re-adding it, due to the fact stable's calling patterns to the cache are bad (instead of making a single request for all keys needed, makes N calls resulting in N io actions rather then 1), but I'd like to get stats backing that up before adding a bandaid back in. Either way, test, feeding stats back, etc, would be appreciated ;) Thanks, ~harring 3.0-cache-backport.patch.bz2 Description: Binary data pgpSXbMdeHtPM.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Tue, Oct 11, 2005 at 02:10:46AM -0500, Brian Harring wrote: > Either way, test, feeding stats back, etc, would be appreciated ;) Just don't report emerge --metadata --quiet throwing a traceback to me. I already know about that one. :) Why doesn't it surprise me no bugs throw N runs, then one turns up <5m after I send... meh. ~harring pgp3RL4iR3zXQ.pgp Description: PGP signature
[gentoo-portage-dev] [PATCH] pre/post phase hooks for users
Since axxo is being a slacker (:-P) and hasn't posted this, did a
quicky implentation for stable ebuild.sh of pre/post phase hooks.
The intention of these hooks are for users to define funcs in their
/etc/portage/bashrc; the phase to be hooked, say pkg_setup , is hooked
via
echo $'
pre_pkg_setup() { echo "I am called prior to pkg_setup"; }
post_pkg_setup() { echo "I am called after pkg_setup"; }
' >> /etc/portage/bashrc
for example.
setup, unpack, compile, test, install, preinst, postinst, prerm,
postrm, and config via this patch have pre/post hooks.
This hooking approach is preferred to just sticking stuff in bashrc
and having it detect the phase- reasoning is that the bashrc will be
sourced once, and only once when ebd is default.
So... it's in users interests to convert over to this setup sooner
rather then later.
Note also, that these hooks aren't phase additions- ebuilds/eclasses
are not to touch them (this will be enforced under ebd also).
That said, their will be an exemption for java ebuilds due to the fact
that they're blocked by ebuild.sh env handling- they need ebd for
things to work properly, and in the meantime this gives them a method
to have things work properly. Downside is that the pre/post hooks are
not available for users for java ebuilds.
Either way, play with it, feedback would be appreciated.
~harring
Index: ebuild.sh
===
--- ebuild.sh (revision 2122)
+++ ebuild.sh (working copy)
@@ -631,11 +631,14 @@
dyn_setup()
{
+ [ "$(type -t pre_pkg_setup)" != "file" ] && pre_pkg_setup
pkg_setup
+ [ "$(type -t post_pkg_setup)" != "file" ] && post_pkg_setup
}
dyn_unpack() {
trap "abort_unpack" SIGINT SIGQUIT
+ [ "$(type -t pre_src_unpack)" != "file" ] && pre_src_unpack
local newstuff="no"
if [ -e "${WORKDIR}" ]; then
local x
@@ -662,6 +665,7 @@
if [ -e "${WORKDIR}" ]; then
if [ "$newstuff" == "no" ]; then
echo ">>> WORKDIR is up-to-date, keeping..."
+ [ "$(type -t post_src_unpack)" != "file" ] &&
post_src_unpack
return 0
fi
fi
@@ -673,6 +677,9 @@
touch "${BUILDDIR}/.unpacked" || die "IO Failure -- Failed 'touch
.unpacked' in BUILDIR"
echo ">>> Source unpacked."
cd "$BUILDDIR"
+
+ [ "$(type -t post_src_unpack)" != "file" ] && post_src_unpack
+
trap SIGINT SIGQUIT
}
@@ -854,6 +861,9 @@
dyn_compile() {
trap "abort_compile" SIGINT SIGQUIT
+
+ [ "$(type -t pre_src_compile)" != "file" ] && pre_src_compile
+
[ "${CFLAGS-unset}" != "unset" ] && export CFLAGS
[ "${CXXFLAGS-unset}"!= "unset" ] && export CXXFLAGS
[ "${LIBCFLAGS-unset}" != "unset" ] && export LIBCFLAGS
@@ -892,6 +902,7 @@
echo ">>> It appears that ${PN} is already compiled; skipping."
echo ">>> (clean to force compilation)"
trap SIGINT SIGQUIT
+ [ "$(type -t post_src_compile)" != "file" ] && post_src_compile
return
fi
if [ -d "${S}" ]; then
@@ -948,6 +959,9 @@
if hasq nostrip $FEATURES $RESTRICT; then
touch DEBUGBUILD
fi
+
+ [ "$(type -t post_src_compile)" != "file" ] && post_src_compile
+
trap SIGINT SIGQUIT
}
@@ -972,8 +986,10 @@
dyn_test() {
+ [ "$(type -t pre_src_test)" != "file" ] && pre_src_test
if [ ${BUILDDIR}/.tested -nt "${WORKDIR}" ]; then
echo ">>> It appears that ${PN} has already been tested;
skipping."
+ [ "$(type -t post_src_test)" != "file" ] && post_src_test
return
fi
trap "abort_test" SIGINT SIGQUIT
@@ -992,6 +1008,7 @@
cd "${BUILDDIR}"
touch .tested || die "Failed to 'touch .tested' in ${BUILDDIR}"
+ [ "$(type -t post_src_test)" != "file" ] && post_src_test
trap SIGINT SIGQUIT
}
@@ -1001,6 +1018,7 @@
dyn_install() {
trap "abort_install" SIGINT SIGQUIT
+ [ "$(type -t pre_src_install)" != "file" ] && pre_src_install
rm -rf "${BUILDDIR}/image"
mkdir "${BUILDDIR}/image"
if [ -d "${S}" ]; then
@@ -1204,6 +1222,7 @@
echo ">>> Completed installing ${PF} into ${D}"
echo
cd ${BUILDDIR}
+ [ "$(type -t post_src_install)" != "file" ] && post_src_install
trap SIGINT SIGQUIT
}
@@ -1211,6 +1230,8 @@
# set IMAGE depending if this is a binary or compile merge
[ "${EMERGE_FROM}" == "binary" ] && IMAGE=${PKG_TMPDIR}/${PF}/bin \
|| IMAGE=${D}
+
+ [ "$(type -t pre_pkg_preinst)" != "file" ] && pre_pkg_preinst
pkg_preinst
@@ -1319,6 +1340,9 @@
echo "!!! Unable to set SELinux security labels"
fi
fi
+
+
Re: [gentoo-portage-dev] Cache rewrite backport
On Tue, Oct 11, 2005 at 02:10:46AM -0500, Brian Harring wrote: > Either way, test, feeding stats back, etc, would be appreciated ;) Lets see how many times I can respond to myself... :) Joys of having 4 different branches of portage (2.0, 2.0-prefix, 2.1, and 3.x) all in varying use locally let a rewrite specific import slip through in cache/fs_template.py Additionally, missed inclusion of mappings.py in the patch; thanks to zmedico for pointing out issues. Corrected it, posting an updated patch. ~harring 3.0-cache-backport.patch.bz2 Description: Binary data pgpWEfutSpAne.pgp Description: PGP signature
Re: [gentoo-portage-dev] Custom eclass question
On Tue, Oct 11, 2005 at 10:37:43AM +0300, Marius Mauch wrote: > Brian Harring wrote: > >On Sun, Oct 09, 2005 at 06:52:24PM -0500, Mikey wrote: > > > >>http://codeserver.wherever.net/pman/package_ids.php?action=package&id=10105 > >> > > > >[snip bits about wget screwing up] > > > >Others have already clarified that's it python side rather then bash > >so eclass is no go, but out of curiousity any got a good reason we > >can't modify FETCHCOMMAND to include FILE rather then just DISTDIR? > > > >With wget, it's just -O. Curl, piping probably (although haven't > >looked at their opts)... > > Where do you get the filename from if not SRC_URI? Additional metadata tagged in, or mangling of the syntax in some way. Kind of a moot question if we can't extend FETCHCOMMAND/RESUMECOMMAND for it though, hence the question about if any fetchers will seriously choke on this requirement. ~harring pgphItwkMl5RN.pgp Description: PGP signature
Re: [gentoo-portage-dev] Custom eclass question
On Tue, Oct 11, 2005 at 09:13:01PM +0200, Marius Mauch wrote:
> On Tue, 11 Oct 2005 13:59:42 +0200 (CEST)
> "Matthias Waechter" <[EMAIL PROTECTED]> wrote:
>
> > > Brian Harring wrote:
> > >>>Where do you get the filename from if not SRC_URI?
> > >>
> > >> Additional metadata tagged in, or mangling of the syntax in some
> > >> way.
> > >
> > > I don't want to imagine how ugly that will get, syntax mangling will
> > > probably break things and additional metadata needs a way to
> > > properly map entries.
> >
> > SRC_URI="rename:${PN}-${PV}.tar.bz2:http://and-so-on/";
> >
> > or add some slashes after rename to fit whatever slashy syntax.
>
> Breaks anything that assumes that SRC_URI is actually a (list of) URI.
> fetch() might not be so hard to adjust, but there are many more tools
> involved. Not a good idea to change semantics of established
> datastructures (especially if it's only a workaround for broken
> upstream packages).
EAPI change...
/me goes back to what he was doing and leaves people to come up with a
sane proposal
~harring
pgpj4gIVaCm8v.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Wed, Oct 12, 2005 at 12:01:12AM +0200, Bastian Balthazar Bux wrote: > Sorry, but here the results are not those expected: .51.22 vs .53_rc5... try with a vanilla .53_rc5 please > time emerge --metadata; 1st run; 2.0.51.22-r3 > real2m24.419s > user0m12.329s > sys 0m3.644s > > time emerge --metadata; 2nd run; 2.0.51.22-r3 > real1m17.700s > user0m12.257s > sys 0m2.976s > > time emerge --metadata; 1st run; 2.0.53_rc5 patched > real3m14.073s > user0m12.917s > sys 0m9.433s > > time emerge --metadata; 2nd run; 2.0.53_rc5 patched > real3m42.874s > user0m12.869s > sys 0m9.333s Wasn't expecting a massive improvement, although wasn't sure as hell wasn't expecting a 3x increase in sys. :) Should've seen a large tweak for the first .53_rc5 run also, since it (essentially) would be a forced rewrite of the cache due to INHERITED vs _eclasses_ key changes (moving eclass_cache into the backend). Not running anything additional via /etc/portage/modules I'd bet, but asking to verify also... Meanwhile, thanks for testing; contrary to other results, but _any_ regression I'm after. ~harring pgp0XVgP7qMOk.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Wed, Oct 12, 2005 at 03:49:44AM +0200, Bastian Balthazar Bux wrote: > Brian Harring ha scritto: > > On Wed, Oct 12, 2005 at 12:01:12AM +0200, Bastian Balthazar Bux wrote: > > > >>Sorry, but here the results are not those expected: > > > > .51.22 vs .53_rc5... try with a vanilla .53_rc5 please > > here they are, also added a test with a dirty trick to precharge the > portage dir and see what happen. Look like there is a small improvement. > Now it's late. > > time emerge --metadata; 1st run; 2.0.53_rc5 vanilla > real9m44.449s > user4m51.034s > sys 0m24.754s > > time emerge --metadata; 2nd run; 2.0.53_rc5 vanilla > real2m50.932s > user0m12.597s > sys 0m3.836s > > time emerge --metadata; 3rd run; 2.0.53_rc5 vanilla > real1m55.445s > user0m12.501s > sys 0m3.416s > > tar -c /usr/portage/* >/dev/null & time emerge --metadata > ; 4th run; 2.0.53_rc5 vanilla > real 1m10.275s > user 0m13.377s > sys 0m4.740s I'd wonder how this ^^^ compares to round 5 of the backport patch- (attached). It's a bit further then what's in 3x, although assuming it behaves well I'll merge that into the rewrite also. > tar -c /usr/portage/* >/dev/null & time emerge --metadata > ; 4th run; 2.0.53_rc5 patched > real 4m12.131s > user 0m13.661s > sys 0m10.329s Difference is due to the fact the rewrite's metadata->local cache is cloning code (think cp) rather then updating... Forgot about that, wrote that code a while back you see :) Made some somewhat tricky changes to the code; stable's best case is 2 stats, 2 reads, 2 opens, and 2 closes per ebuild. Worst case is a write thrown in, plus utime. Best case with this patch is 2 stats, worst is still the same but occurs only when eclasses change. Alec and zach tested it fair bit, sounds of it it's performing fairly well. Would be curious about stats from those running truly crap hardware, 20m metadata transfer for example. Forewarning, if you're switching between the patch and .53_rc3, you'll get some screwwy stats on the first run since it's a cache invalidation, eg wipe the cache via an rm -rf. :) ~harring 3.0-cache-backport-experimental-4.patch.bz2 Description: Binary data pgp0X7QGqEUYn.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Wed, Oct 12, 2005 at 03:49:44AM +0200, Bastian Balthazar Bux wrote:
> Brian Harring ha scritto:
> > On Wed, Oct 12, 2005 at 12:01:12AM +0200, Bastian Balthazar Bux wrote:
> >
> >>Sorry, but here the results are not those expected:
> >
> > .51.22 vs .53_rc5... try with a vanilla .53_rc5 please
>
> here they are, also added a test with a dirty trick to precharge the
> portage dir and see what happen. Look like there is a small improvement.
> Now it's late.
>
> time emerge --metadata; 1st run; 2.0.53_rc5 vanilla
> real9m44.449s
> user4m51.034s
> sys 0m24.754s
>
> time emerge --metadata; 2nd run; 2.0.53_rc5 vanilla
> real2m50.932s
> user0m12.597s
> sys 0m3.836s
>
> time emerge --metadata; 3rd run; 2.0.53_rc5 vanilla
> real1m55.445s
> user0m12.501s
> sys 0m3.416s
>
> tar -c /usr/portage/* >/dev/null & time emerge --metadata
> ; 4th run; 2.0.53_rc5 vanilla
> real 1m10.275s
> user 0m13.377s
> sys 0m4.740s
>
>
> time emerge --metadata; 1st run; 2.0.53_rc5 patched
> real 4m30.186s
> user 0m12.757s
> sys 0m9.921s
>
> time emerge --metadata; 2nd run; 2.0.53_rc5 patched
> real 4m41.021s
> user 0m12.597s
> sys 0m9.297s
>
> time emerge --metadata; 3rd run; 2.0.53_rc5 patched
> real 4m44.544s
> user 0m12.521s
> sys 0m9.457s
>
> tar -c /usr/portage/* >/dev/null & time emerge --metadata
> ; 4th run; 2.0.53_rc5 patched
> real 4m12.131s
> user 0m13.661s
> sys 0m10.329s
>
> >
> >
> >
> >> time emerge --metadata; 1st run; 2.0.51.22-r3
> >>real2m24.419s
> >>user0m12.329s
> >>sys 0m3.644s
> >>
> >> time emerge --metadata; 2nd run; 2.0.51.22-r3
> >>real1m17.700s
> >>user0m12.257s
> >>sys 0m2.976s
> >>
>
> [snip]
> the 2.0.51.22-r3 ones are still much faster on "real", please shade a
> light into my ignorance
Cache had to have been mostly full already; note the 4m51 for .53_rc5;
.22-r3 would display the same if the cache was invalid (going from
cache rewrite patch to .53_rc5 vanilla invalidates the local cache).
So... pretty much I'm ignoring the first .53_rc5 run, 2nd/3rd match up
somewhat with .53.22-r3; main difference that comes to mind is that
.53_rc5's --metadata code had a collection of extra checks/steps
thrown in to protect against a lot of annoying tracebacks that were
rearing there heads, and EAPI was added which would result in
rewriting the cache entry on the fly.
Don't think it's the case though due to no matching user increase;
difference in sys pretty much points at some extra IO occuring
somewhere.
> > Meanwhile, thanks for testing; contrary to other results, but _any_
> > regression I'm after.
> > ~harring
>
> No, thank you to work on this, every time I've tryed to dive in portage
> I needed some day of hospital.
I'd suggest hitting the jim bean personally. Replace the pounding
headache with something you at least control... ;)
> look also at this additional try:
>
> cp -a cache/* /dev/shm/
> mount -obind /dev/shm /usr/portage/metadata/cache/
> tar -c /usr/portage/* >/dev/null & time emerge --metadata
> ; Nth run; 2.0.53_rc5 patched
>
> real 3m43.653s
> user 0m12.937s
> sys 0m9.817s
The copy instead of update accounts for that, which shouldn't occur
with the experimental-4 patch in the other email.
> IMHO the "real" time of an emerge --metadata could be improved acting in
> two ways:
> 1) preload as much as possible data (stats included) from disk before to
> parse it
> 2) separating disk read from disk writes (i.e many disk read followed
> by many disk writes followed ...)
I actually tried threading the bugger a while back... the improvement
wasn't quite what I was hoping for, partially due to hitting issues
with the global interpretter lock in python.
That said, could attempt it again, code for it is mostly
straightforward
> Fetching virtual information from a cache save thousand of disk reads
> # find /var/db/pkg/ -name PROVIDE \
> \( -exec echo -n {}\: \; -and -exec cat {} \; \) \
> | egrep -v "PROVIDE:$"
a central cache of providers for the vdb would certainly make
python -c'import portage' a helluva lot faster I'd expect.
Any takers to prototype it?
~harring
pgp5fltO0xhx4.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
Rather then keep posting large patches here, just going to post them to dev.gentoo.org/~ferringb/portage/2.0 ... for example, http://dev.gentoo.org/~ferringb/portage/2.0/3.0-cache-backport-experimental-4.patch should be pulled rather then previous patch. Helluva lot easier for me since I can correct dumb ass mistakes like an import pdb for testing sliding into a patch :) Think that's probably a sign it's time for some sleep... ~harring pgpGR5wwPYAWo.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Wed, Oct 12, 2005 at 02:56:38PM +0200, Bastian Balthazar Bux wrote: > Brian Harring ha scritto: > > Rather then keep posting large patches here, just going to post them > > to dev.gentoo.org/~ferringb/portage/2.0 ... for example, > > http://dev.gentoo.org/~ferringb/portage/2.0/3.0-cache-backport-experimental-4.patch > > > > > > should be pulled rather then previous patch. > > > > Helluva lot easier for me since I can correct dumb ass mistakes like > > an import pdb for testing sliding into a patch :) > > > > Think that's probably a sign it's time for some sleep... > > ~harring > > premessa: > emerge metadata does not recreate the cache after a > "rm -rf $PORTDIR/metadata/cache/*" . > The first thing that came in mind is that I'm missing something obvious. Can't trigger that myself... so would be curious. > Today working on a different box, slower cpu, faster disk (ratio 2:1) > > sys-apps/portage-2.0.53_rc5 patched version > time emerge --metadata three times, first time with error trace > real3m43.162s 0m14.986s 0m14.124s > user0m13.580s 0m10.790s 0m11.150s > sys 0m7.990s 0m3.280s 0m2.960s > > sys-apps/portage-2.0.53_rc5 vanilla > time emerge --metadata three times > real0m47.913s 0m52.677s 0m37.448s > user0m32.900s 0m31.720s 0m31.870s > sys 0m7.020s 0m6.440s 0m5.530s > > sys-apps/portage-2.0.53_rc5 patched version again > time emerge --metadata three times, first time with error trace > real0m17.386s 0m14.029s 0m13.873s > user0m12.110s 0m10.980s 0m10.820s > sys 0m3.950s 0m3.020s 0m3.040s > > === cache file example > cat $PORTDIR/metadata/cache/sys-apps/portage-2.0.53_rc5 > >=dev-lang/python-2.2.1 > !build? ( >=sys-apps/sed-4.0.5 dev-python/python-fchksum > >=dev-lang/python-2.2.1 userland_GNU? ( sys-apps/debianutils ) > >=app-shells/bash-2.05a ) !userland_Darwin? ( sys-apps/sandbox ) > selinux? ( >=dev-python/python-selinux-2.15 ) > 0 > mirror://gentoo/portage-2.0.53_rc5.tar.bz2 > http://dev.gentoo.org/~jstubbs/releases/portage-2.0.53_rc5.tar.bz2 > > http://www.gentoo.org/ > GPL-2 > The Portage Package Management System. The primary package management > and distribution system for Gentoo. > ~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc-macos ~ppc64 ~s390 > ~sh ~sparc ~x86 > multilib toolchain-funcs > build selinux ^^^ that's a flat_list entry, if/when this faster code is deployed a wipe of the cache probably would occur since flat_list and flat_hash use the same location (need to change that someday imo). Explains the traceback that followed. :) _Still_ need to modify the damn thing to cover another angle for PORTDIR being wholesale moved, but posted an experimental-6 that will be a bit slower, but is a bit more correct about handling the cases above, and dealing with eclass staleness checks. ~harring pgpRJbE9OpRWt.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Wed, Oct 12, 2005 at 05:15:15PM -0500, Paul Varner wrote: > On Tue, 2005-10-11 at 02:10 -0500, Brian Harring wrote: > > Either way, test, feeding stats back, etc, would be appreciated ;) > > Thanks, Hokay... so experimental-6 seems to be at least mostly stable now, no tracebacks, no --metadata regressions... Curious about feedback from general usage, emerge -s, emerge -Dup world, etc. timing runs would be appreciated ~harring pgpB5Miu4uD5v.pgp Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Thu, Oct 13, 2005 at 10:17:26AM +0200, Nagatoro wrote:
> Brian Harring wrote:
> >Curious about feedback from general usage, emerge -s, emerge -Dup
> >world, etc. timing runs would be appreciated
> portage 2.0.53_rc5 - 3.0-cache-backport-experimental-7
>
> emerge -s portage:
>
> real0m10.855s
> user0m2.610s
> sys 0m0.537s
>
> emerge -Dup:
>
> real0m40.443s
> user0m18.300s
> sys 0m3.465s
Don't spose you could attempt the tests above with the cache patches,
plus the patch I've attached? It adds lastX caching back in, should
reduce sys a _bit_, although personally I'm not expecting a helluva
lot out of it.
Part of the reason is that by it's nature, it only improves sequential
calls for the same keys. emerge does it on occasion, but I'm not
betting their would be a helluva lot of gain. repoman may benefit,
since it's code is horrid though.
Either way... locally, I'm getting pretty much no gain detectable in
the usual testing noise, so curious if anyone else can see anything
out of it.
If it has a usable affect, it's going to show on systems with piss
poor IO, and minimal kernel buffering.
~harring
diff -urN portage-base/pym/cache/mappings.py portage/pym/cache/mappings.py
--- portage-base/pym/cache/mappings.py 2005-10-13 03:54:01.0 -0500
+++ portage/pym/cache/mappings.py 2005-10-13 04:11:57.0 -0500
@@ -102,3 +102,63 @@
self.pull = None
return key in self.d
+
+class CachingDict(UserDict.DictMixin):
+ """
+ given an initial dict, this wraps that dict storing changes in a
secondary dict, protecting
+ the underlying dict from changes
+ """
+ __slots__=("_cache", "_d")
+
+ def __init__(self, d):
+ self._d = d
+ self._cache_keys = []
+ self._cache_vals = []
+
+ def __getitem__(self, key):
+ if key in self._cache_keys:
+ return self._cache_vals[self._cache_keys.index(key)]
+ val = self._d[key]
+ if len(self._cache_keys) == 3:
+ del self._cache_keys[2]
+ del self._cache_vals[2]
+ self._cache_keys.insert(0, key)
+ self._cache_vals.insert(0, val)
+ return val
+
+ def __delitem__(self, key):
+ del self._d[key]
+ if key in self._cache_keys:
+ i = self._cache_keys.index(key)
+ del self._cache_keys[i]
+ del self._cache_vals[i]
+
+ def __setitem__(self, key, val):
+ self._d[key] = val
+ i = -1
+ if key in self._cache_keys:
+ i = self._cache_keys.index(key)
+ elif len(self._cache_keys) == 3:
+ i = 2
+ if i != -1:
+ del self._cache_keys[i]
+ del self._cache_vals[i]
+ self._cache_keys.insert(0, key)
+ self._cache_vals.insert(1, val)
+
+ def __iter__(self):
+ return iter(self.keys())
+
+ def keys(self):
+ return self._d.keys()
+
+ def has_key(self, key):
+ return key in self
+
+ def __contains__(self, key):
+ if key in self._cache_keys:
+ return True
+ return key in self._d
+
+ def sync(self, *a, **kw):
+ return self._d.sync(*a, **kw)
diff -urN portage-base/pym/portage.py portage/pym/portage.py
--- portage-base/pym/portage.py 2005-10-13 03:53:52.0 -0500
+++ portage/pym/portage.py 2005-10-13 04:03:24.0 -0500
@@ -5087,9 +5087,10 @@
# XXX: REMOVE THIS ONCE UNUSED_0 IS YANKED FROM auxdbkeys
# ~harring
filtered_auxdbkeys = filter(lambda x: not
x.startswith("UNUSED_0"), auxdbkeys)
+ from cache.mappings import CachingDict
for x in self.porttrees:
# location, label, auxdbkeys
- self.auxdb[x] =
self.auxdbmodule(portage_const.DEPCACHE_PATH, x, filtered_auxdbkeys,
gid=portage_gid)
+ self.auxdb[x] =
CachingDict(self.auxdbmodule(portage_const.DEPCACHE_PATH, x,
filtered_auxdbkeys, gid=portage_gid))
def close_caches(self):
for x in self.auxdb.keys():
pgpsfFhrhd7LO.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Cache rewrite backport
On Thu, Oct 13, 2005 at 01:05:40PM +0200, Nagatoro wrote: > Brian Harring wrote: > Hmm, when trying to update the cache (emerge --metadata) I get this: > --- > >>> Updating Portage cache: > Traceback (most recent call last): > File "/usr/bin/emerge", line 2734, in ? > cache.util.mirror_cache(source, cm, pdb.auxdb[porttree_root], > eclass_cache=ec, verbose_instance=noise_maker) > File "/usr/lib/portage/pym/cache/util.py", line 22, in mirror_cache > if not trg_cache.autocommits: > AttributeError: CachingDict instance has no attribute 'autocommits' > --- > but if I remove the lastX patch it workes. *cough*, that's cause lastX is a dirty hack externally, rather then implemented w/in the cache backend for testing. That and I forgot to define that method. :) Basically it wraps the cache repo obj- if it proved to make a decent difference, would go through and integrate it better. > portage 2.0.53_rc5 - 3.0-cache-backport-experimental-7 + lastx > emerge -s portage: > > real0m12.891s > user0m3.030s > sys 0m0.576s > > After running this with and without the lastx patch I seem to get > slighly longer times (real and user) with it (~0.5s real and user) and > slightly shorter sys times (~0.2s). > > emerge -Dup world: > > real0m38.113s > user0m18.186s > sys 0m2.916s > > And here I seem to get slightly shorter real and sys times but > slightly longer user times (~0.5s real and user, ~0.2s sys). > > But the times varies way to much to "prove" this. Pretty much is exactly what I've seen Haven't gotten any conclussive stats out of it. fex, testing via- cd /usr/portage ; for x in 1 2; do time repoman scan &> /dev/null ; done ; cd /usr/lib/portage/pym/; patch -p2 -i ~/lastx.patch ; cd /usr/portage ; for x in 1 2; do time repoman scan &> /dev/null ; done ; cd /usr/lib/portage/pym/; patch -p2 -i ~/lastx.patch -R real53m30.754s user34m25.549s sys 6m11.663s real49m16.141s user32m48.835s sys 5m52.882s patching file cache/mappings.py patching file portage.py real51m4.228s user34m24.005s sys 6m4.447s real51m16.202s user34m21.253s sys 6m1.315s patching file cache/mappings.py patching file portage.py About the only thing I've seen with the patch is less variation in the results, although never hitting the best runs unpatched does. Either way, thanks for testing it :) ~harring pgp4vHN8GQe9n.pgp Description: PGP signature
Re: [gentoo-portage-dev] exclude cache from sync, was:Cache rewrite backport
On Thu, Oct 13, 2005 at 07:12:42PM +0200, Francesco R. wrote: > Elaborating on the previous subject an idea come in mind, > avoid the $PORTDIR/metadata/cache sync, It's 80 MB, 2 files > > So I've tryed the first python patch of my life, obviously it's also the > first portage one (subliminal message, take the result with care and > check it yourself) > > how? the patch add the option "--withregen", the to > exclude /metadata/cache from the sync and to issue a regen of the cache > before to update the metadata. > calling "emerge --withregen --sync" do the work. > > The following test has run with a dual opteron as rsync server and a > Athlon XP 2500+ with an old and slow disk, cabled with gigabit > ethernet. The rsync is forced removing the timestamp. > > The portage tree has been synced _before_ starting the bench, so the > overhead of rsync is really reduced at the bare minimum, check the > differences between the files on server and client. > > The surprising result is that recreating the cache is faster than rsync > it. (real time) Nuke the cache and try it ;) Being slightly sarcastic there, fastest I've managed to get a full regen down to was around 22 minutes for ebd, with stable being around 34m http://dev.gentoo.org/~ferringb/ebuild-daemon/stats/paired-stats I'd expect the gap has narrowed a bit since those timing runs, although it should still be sizable. Either way, the longer timing run (stable ebuild.sh) is the one to note :) So... that --metadata run has an extra stat call per check best case, but the cost of getting a cache entry is pretty heavily different. For metadata, just is a file read/write, for regen, exec(bash -c ebuild.sh) which, via quicky commandline test (that sets a floor for it), time bash /usr/lib/portage/bin/ebuild.sh , you're looking at around .1s per call. Pretty heavy difference on updates, eg, worst case- something a user hits on first sync, or nuking of the tree :) > ===|=== > ===|=== > 1st try, portage [2.0.53_rc5] + cac|1st try, portage [2.0.53_rc5] + cac > ===|=== > sync with cache and _no_ regen |sync without cache and regen after >|Regenerating cache entries... >|done regen! >| > real6m23.727s |real4m14.837s > user0m12.373s |user0m18.681s > sys 0m13.849s |sys 0m7.744s > ===|=== > ===|=== > 2nd try, portage [2.0.53_rc5] + cac|2nd try, portage [2.0.53_rc5] + cac > ===|=== > sync with cache and _no_ regen |sync without cache and regen after >|Regenerating cache entries... >|done regen! >| > real6m53.649s |real2m40.794s > user0m12.361s |user0m18.361s > sys 0m14.117s |sys 0m6.800s > ===|=== > ===|=== > 3rd try, portage [2.0.53_rc5] + cac|3rd try, portage [2.0.53_rc5] + cac > ===|=== > sync with cache and _no_ regen |sync without cache and regen after >|Regenerating cache entries... >|done regen! >| > real6m46.973s |real4m19.261s > user0m12.605s |user0m18.593s > sys 0m13.733s |sys 0m7.648s >| That said, I'm curious wth is triggering the 2x sys activity, which probably is to blame for the ~90s difference Anyone game for profiling a --metadata run, vs --regen run via profile.run? ~harring pgpEITgRx5D51.pgp Description: PGP signature
Re: [gentoo-portage-dev] Status of SVN trunk?
On Thu, Oct 13, 2005 at 08:08:43PM +0200, Thomas de Grenier de Latour wrote: > Hi, > > I've been using an old portage CVS snapshot for a few months, and > today i've finaly decided to sync it with SVN trunk. Ouch... that > was not such a great idea... First, some *.sh files were not > sourcing because of some typos. Then, once fixed, i've seen > portage starting to play with my root filesystem (missing $IMAGE in > dyn_preinst is not good...). Finally, looking at the SVN > changelog, i see that it doesn't get most of the fixes/improvements > that goes in 2.0. > So, i'm wondering: should i properly report this issues, or is > trunk completly given up? And the corollary question: if i was > about to update a few old small patches i have sitting on b.g.o, > what would be the right branch to work on? Is 2.0 opened to new > features again? Any updates to 2.1, I'd be curious about since it's bash core is the basis of 3.x Regarding features, that's jasons thing; the intention was to nail 2.0 down, and move forward (last time this was discussed). ~harring pgpprlfLVj5C9.pgp Description: PGP signature
Re: [gentoo-portage-dev] Status of SVN trunk?
First... stating thank you. This is an ongoing pain in the ass, trying to track what everyone has stuck into stable and ensure it's in ebd codebase is mildly maddening on a given day. On Fri, Oct 14, 2005 at 03:01:30PM +0200, Thomas de Grenier de Latour wrote: > On Thu, 13 Oct 2005 13:21:21 -0500 > Brian Harring <[EMAIL PROTECTED]> wrote: > > > Any updates to 2.1, I'd be curious about since it's bash core is > > the basis of 3.x > > The savior's bash code seems sane compared to the trunk's one. Trunk being a frozen snapshot of the codebase... kept working on it. :) > Now, compared to 2.0 (and just reading changelog), i see: > - some patches missing in both branches: some are new stuffs > i'm sure you're already aware of (EBUILD_DEATH_HOOK, EAPI support, > etc.), other are fixes (lchown/lchgrp for instance, but i also see > something about .tar unpacking in changelog which seems to apply, > etc.). Just added EBUILD_DEATH_HOOK and EAPI assert fixes for unpack are in also (.tar bit) the lchown/lchgrp changes I'm not pushing forward. That code I'm going to yank out of the bash side, and shove it into python, since contents scans will be needed, and should be more efficient. > I'm okay to try to make some more detailed report with patches, but > i would first like to know whether trunk is going anywhere. I mean, > it's obvious that atm noone uses it since it's not even > syntacticaly correct. That, plus some bugs closed as dups of #102126 > without beeing fixed, makes me wonder... Oh, and also this recent > email from Marius on gentoo-dev@: > > > On Tue, 11 Oct 2005 17:00:56 + > > Alec Joseph Warner <[EMAIL PROTECTED]> wrote: > > > > > FYI elog is implemented in CVS ( 2.1 ). When it will be > > > released is anyone's guess. > > > > 2.1? probably never, but elog will almost certainly be backported > > to the 2.0 branch. > > > > Marius > > Anyway, don't take me wrong, i'm not trying to flame anyone about > how portage development is done, or whatever. If the new route is > to backport some of the long awaited features to 2.0 until 3.x is > ready, it's fine for me. Surprising thing, people are using it- that said, I'm not (nor have I been for a few months). Was using it over stable due to it having stuff I wanted, but due to reasons listed below, reverted to a hacked up stable... Meanwhile, spose I'll be the one to do the deed since nobody else is speaking up, but yah, 2.1 is dead as a release line. Pretty much was the case from it's initial release, since by that time I'd already moved on past it (meaning the ebd related bugs in 2.1 I was no longer fixing in 2.1). Reasons come down to several- A) lot of underlying structural changes, lot bash side, and shims python side to play with bash side B) said shims (namely the continued existance of of doebuild akin code) means we're still going to get idiotic phase bugs like we do with stable. Env handling bash side is bordering on total control, but what goes into the env python side is still a god awful mess. C) backporting chunks of code (cache fex) to 2.0.x line (non-ebd line, basically) isn't too hard, and worth the effort. D) 2.1's base is approaching a year old. Time for stabling for that beast came and went long ago from where I'm sitting, 'specially considering the issues in A/B require python side modifications (core rewrite), which is what 3.0 is. We could slide by with the bastardly python side ebuild shim, but personally I don't think it's a productive route to follow. So... onto the bugs. > It's really just that i have a bit of > free time this days, so i was thinking of updating a few stuffs i > would like to see included someday (bugs #85786, #83613, #90444, > #84884, #37498 and #34964 comes to mind), and for that i need to > know what would be the prefered target. #85786 (faster config.pusedict) Stable target from where I'm sitting, since it's a fairly simple modification from the standpoint of how much code it touches Needs testers before it would be considered (get cracking peeps). Not so much a feature as a more efficient version of existing, so... no major complaints personally. #83613 (abspath profile/parent support) Hesitant on this one; target would have to be stable since the changes won't really fit into 3.x (profile being a seperate config obj, can go nuts with it). Either way, hesitant since A) changes to profile need to play nice in a backwards compatible way, we already have enough issues with .50 profile screwups still. :) B) if limited to /etc/portage/profile (something of the sort), less of an issue. C) interaction with any code that resets POR
Re: [gentoo-portage-dev] [PATCH] pre/post phase hooks for users
On Fri, Oct 14, 2005 at 06:31:45PM +0200, Thomas Matthijs wrote:
> > Index: ebuild.sh
> > ===
> > --- ebuild.sh (revision 2122)
> > +++ ebuild.sh (working copy)
> > @@ -631,11 +631,14 @@
> >
> > dyn_setup()
> > {
> > + [ "$(type -t pre_pkg_setup)" != "file" ] && pre_pkg_setup
> > pkg_setup
> > + [ "$(type -t post_pkg_setup)" != "file" ] && post_pkg_setup
> > }
>
> On my system
> type -t non-existand-function returns ""
> and when it does it gives me "function"
>
> s/!= "file"/== "function"/g ??
Yah, good catch.
Jason, your thoughts on this 53 wise?
~harring
pgpEydoVOagXI.pgp
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] pre/post phase hooks for users
On Fri, Oct 14, 2005 at 05:02:02PM -0500, Brian Harring wrote:
> On Fri, Oct 14, 2005 at 06:31:45PM +0200, Thomas Matthijs wrote:
> > > Index: ebuild.sh
> > > ===
> > > --- ebuild.sh (revision 2122)
> > > +++ ebuild.sh (working copy)
> > > @@ -631,11 +631,14 @@
> > >
> > > dyn_setup()
> > > {
> > > + [ "$(type -t pre_pkg_setup)" != "file" ] && pre_pkg_setup
> > > pkg_setup
> > > + [ "$(type -t post_pkg_setup)" != "file" ] && post_pkg_setup
> > > }
> >
> > On my system
> > type -t non-existand-function returns ""
> > and when it does it gives me "function"
> >
> > s/!= "file"/== "function"/g ??
> Yah, good catch.
>
> Jason, your thoughts on this 53 wise?
Bleh, pardon, meant .54 for inclussion
> ~harring
pgprEdEH3PlJI.pgp
Description: PGP signature
Re: [gentoo-portage-dev] Suggestion: Backup option in make.conf
On Sat, Oct 15, 2005 at 06:28:53AM +0200, Rafael Fern?ndez L?pez wrote: >Hi, >My suggestion for portage is: when you do an "emerge -vuD world", you >have to work at the end of the emerging with the /etc config files and >see the differences to know if you want to overwrite file by file or >not. >Well, what I want is to have a backup of all them (one backup, and >only one). For that goal, it can be used a new option in make.conf >like "backup_etc_files=yes" if you want to turn this on. >By turning this on, an "emerge -vuD world" can be done in one shot, I >mean, you can do an emerge -vuD world, and all the files could be >replaced (overwrited) by the new ones (etc-update), but you'll be able >to access to your backups if anything is wrong, so you don't have to >go file by file looking at the differences between them. >I suggest to name them "whatever.conf" (our original file) ==> >"whatever.conf.backup" (our backuped file) This breaks down pretty quickly... a peacemeal pam upgrade, for example can result in you locked out of your system. Baselayout updates to /etc/passwd can accomplish the same thing, and not accounting for /etc/fstab resulting in an unbootable system... ~harring pgpzkREvMSNua.pgp Description: PGP signature
Re: [gentoo-portage-dev] The road ahead...
On Sat, Oct 15, 2005 at 01:45:42PM +0900, Jason Stubbs wrote: > Since I've fallen into the terrible pit of "trying to make everybody happy" > and since IRC sucks for making decisions due to lack of continuity, let's > battle it out here. ;) /me refills the napalm, and gets ready... > Where we're at: > > Which leaves us with 2.0 and a set of refactorings and features. I think it's > pretty much decided that these will be backported to 2.0. The only question > at this stage is when. The only complicating factor here is the current 225 > open bug reports. That and what to call 2.0+refactorings+features. > > So, there's pretty much three ways we can go: > > 1) Backport refactorings+features and release. > 2) Fix more bugs, backport refactorings+features and release. > 3) Fix more bugs, release, backport refactorings+features and release. > > There's still a lot of bugs that can be fixed without too much work, so I'd > like to go with 2) or 3). I was thinking to go with 3) with the backported > stuff being named 2.1.0, which is how we arrived at this thread. Aside from the 2.1 name being already slightly abused, prefer option 4, bug/release work, integrating chunks in as they're ready and releasing when things are stable. Basically... when the chunks are ready to be integrated, they've been tested (ala cache patch + some more time), yank the suckers in, and continue with stabilising towards a release. On the subject of versions, hich ever version the chunks get included under, they're going to need integration testing, so versioning isn't as much an issue to me as time. The delta between 2.1 and 2.0, last time I generated it was half a meg; pulling chunks from 2.1 into 2.0 requires rewriting a chunk of glue, so minimizing the time/delta is something of a concern- eg, doing 2.0.* for a while, then a 2.1 I'm not totally much for. My two cents, at least. ~harring pgppLGjuUd5pB.pgp Description: PGP signature
Re: [gentoo-portage-dev] The road ahead...
On Sat, Oct 15, 2005 at 01:45:42PM +0900, Jason Stubbs wrote: > Since I've fallen into the terrible pit of "trying to make everybody happy" > and since IRC sucks for making decisions due to lack of continuity, let's > battle it out here. ;) Something I forgot to tag into the other email... Personally, I can live with which ever direction things go. What I'm mainly after is that whatever route/plan choosen, is stuck to. Really not much for 2.1 (existing ebd based) being brought up as a release line N months down the line, since 2 branches of development is a pita enough, let alone 3 with fairly radical differences between each branch. Not saying the path can't be tweaked as we're proceeding, but would *really* like to know that as a group/community, this is what we're going towards rather then in N different directions. ~harring pgpsWQn7FaoQe.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: emerge sync with cvs:// behaves oddly with initial checkout (bug 100478)
On Tue, Oct 18, 2005 at 01:30:56AM -0700, Zac Medico wrote: > As brought to my attention by Brian, the problem with my previously posted > patch is that the "emerge sync" cvs checkout will be owned by root and cvs > doesn't allow commits by root. Apparently we need to devise a scheme to > set the privileges appropriately for a non-root user and/or group. I'd suggest taking a look at the sync refactoring in 2.1; the SYNC syntax will need to be extended for this, since you'll need to specify both remote and local user. Pulling from sync/cvs #new format. #cvs://[CVS_RSH binary:[EMAIL PROTECTED]:cvs_root:module #example #cvs://ssh:[EMAIL PROTECTED]:/var/cvsroot:gentoo-x86 #old format #cvs://[EMAIL PROTECTED]:cvsroot #implicit gentoo-x86 module, and ext w/ ssh. The new format doesn't cover that case, although it's not too hard to mangle it in; say (and I realize this is counter to normal uri specification) #cvs://[CVS_RSH binary:][local [EMAIL PROTECTED] user][:remote [EMAIL PROTECTED]:cvs_root:module example... cvs://ssh:[EMAIL PROTECTED]@cvs.gentoo.org:/var/cvsroot:gentoo-x86 say sf.net cvs://[EMAIL PROTECTED]:[EMAIL PROTECTED]:whatever-it-is:diffball Either way, the old SYNC CVS uri handling doesn't really cut it and needs overhauling; the CVS_RSH trickery should be left in also, since not everyone is going to have ssh remotely. ~harring pgpvJhOWPVkuc.pgp Description: PGP signature
