[gentoo-portage-dev] confcache integration
Yo.
Looking to integrate confcache support into trunk some time in the
near future- had users testing it for about 2 months (give or take),
so far it's behaved pretty decently. A few packages eat themselves
when ran with --cache (bad autotooling), hence the addition of
restrict="confcache" being added to the patch.
Either way, patch is attached, looking for any complaints regarding it
(or integrating confcache)- will be sending an email off to gentoo-dev
in the next few days to get feedback from the general dev populace,,
but sending this email to get feedback on the implementation in the
meantime.
~harring
Index: pym/portage.py
===
--- pym/portage.py (revision 2490)
+++ pym/portage.py (working copy)
@@ -2660,7 +2660,30 @@
print "!!! Perhaps: rm -Rf",mysettings["BUILD_PREFIX"]
print "!!!",str(e)
return 1
-
+ try:
+ if "confcache" in features:
+ if not mysettings.has_key("CONFCACHE_DIR"):
+ mysettings["CONFCACHE_DIR"] =
os.path.join(mysettings["PORTAGE_TMPDIR"], "confcache")
+ if not
os.path.exists(mysettings["CONFCACHE_DIR"]):
+
os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775)
+ os.chown(mysettings["CONFCACHE_DIR"],
portage_uid, -1)
+ else:
+ st =
os.stat(mysettings["CONFCACHE_DIR"])
+ if not (st.st_mode & 0) == 0775:
+
os.chmod(mysettings["CONFCACHE_DIR"], 0775)
+ if not st.st_uid == portage_uid:
+
os.chown(mysettings["CONFCACHE_DIR"], portage_uid, -1)
+ for x in listdir(mysettings["CONFCACHE_DIR"]):
+ p =
os.path.join(mysettings["CONFCACHE_DIR"], x)
+ st = os.stat(p)
+ if not (st.st_mode & 0) & 07600 ==
0600:
+ os.chmod(p, (st.st_mode & 0777)
| 0600)
+ if not st.st_uid == portage_uid:
+ os.chown(p, portage_uid, -1)
+
+ except OSError, e:
+ print "!!! Failed resetting perms on confcachedir %s" %
mysettings["CONFCACHE_DIR"]
+ return 1
#try:
# mystat=os.stat(mysettings["CCACHE_DIR"])
# if (mystat[stat.ST_GID]!=portage_gid) or
((mystat[stat.ST_MODE]&02070)!=02070):
Index: bin/ebuild.sh
===
--- bin/ebuild.sh (revision 2490)
+++ bin/ebuild.sh (working copy)
@@ -459,7 +459,22 @@
LOCAL_EXTRA_ECONF="--libdir=${CONF_LIBDIR_RESULT}
${LOCAL_EXTRA_ECONF}"
fi
- echo "${ECONF_SOURCE}/configure" \
+ local TMP_CONFCACHE_DIR CONFCACHE_ARG
+ if hasq confcache $FEATURES && ! hasq confcache $RESTRICT; then
+ CONFCACHE="$(type -p confcache)"
+ if [ -z "${CONFCACHE}" ]; then
+ ewarn "disabling confcache, binary cannot be
found"
+ else
+ CONFCACHE="${CONFCACHE/ /\ }"
+
TMP_CONFCACHE_DIR="${CONFCACHE:+${CONFCACHE_DIR:-${PORTAGE_TMPDIR}/confcache}}"
+ TMP_CONFCACHE_DIR="${TMP_CONFCACHE_DIR/ /\ }"
+ CONFCACHE_ARG="--confcache-dir"
+ fi
+ else
+ CONFCACHE=
+ fi
+
+ echo ${CONFCACHE} ${CONFCACHE_ARG} ${TMP_CONFCACHE_DIR}
"${ECONF_SOURCE}/configure" \
--prefix=/usr \
--host=${CHOST} \
--mandir=/usr/share/man \
@@ -470,7 +485,7 @@
"$@" \
${LOCAL_EXTRA_ECONF}
- if ! "${ECONF_SOURCE}/configure" \
+ if ! ${CONFCACHE} ${CONFCACHE_ARG} ${TMP_CONFCACHE_DIR}
"${ECONF_SOURCE}/configure" \
--prefix=/usr \
--host=${CHOST} \
--mandir=/usr/share/man \
pgpu5EBoUtgFS.pgp
Description: PGP signature
Re: [gentoo-portage-dev] confcache integration
On Tue, 24 Jan 2006 04:50:31 -0800 Brian Harring <[EMAIL PROTECTED]> wrote: > Yo. > > Looking to integrate confcache support into trunk some time in the > near future- had users testing it for about 2 months (give or take), > so far it's behaved pretty decently. A few packages eat themselves > when ran with --cache (bad autotooling), hence the addition of > restrict="confcache" being added to the patch. > > Either way, patch is attached, looking for any complaints regarding > it (or integrating confcache)- will be sending an email off to > gentoo-dev in the next few days to get feedback from the general dev > populace,, but sending this email to get feedback on the > implementation in the meantime. That directory stuff really needs a wrapper function, it's duplicated way too often. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] confcache integration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian Harring wrote: > Yo. > > Looking to integrate confcache support into trunk some time in the > near future- had users testing it for about 2 months (give or take), > so far it's behaved pretty decently. A few packages eat themselves > when ran with --cache (bad autotooling), hence the addition of > restrict="confcache" being added to the patch. > > Either way, patch is attached, looking for any complaints regarding it > (or integrating confcache)- will be sending an email off to gentoo-dev > in the next few days to get feedback from the general dev populace,, > but sending this email to get feedback on the implementation in the > meantime. > > ~harring > > I don't see a manpage/make.conf patch in there :) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQ9YzA2zglR5RwbyYAQKCTg/+NtqHZSIrApdkm6+TULFE2W2CIHQD9Vd8 VdLyKaDXOigsK460iZehY1pnxZRA8qes9Ed4H51kWL5HMCO/9FHyUZq2SAYHwQ/M moGMML+XyPanUc9Kvy2JREc/onzCbvPdYXkcJRNuE2XFhrfnKTXWin8JJBqyP7C/ 9S/NItRH+lAE6/1Az/6iJcSto0WscN/W18shwitCUB0igVUYcaBal4W3oJ6SvHEB NtX/xjL6zHTlS1iCsawVXAzgwcvFBNOSWilFobN6Qj2V1cn9a74Pio4axl33pEzK 6Mz4csL/UCZb/RisABOU03i3tTBEnehwgvKaD1BxHVfTUNwr8f9Oc1GdLJoZKlp8 H+4mZ7K82bhJ8CB0io9Zz5FZFUrp2cU2mQCh53vlVHZ2UlH+rAXaCLp2Lm0Fi1yQ 5cYyp0tH4eGNrYBrykZZ/gImaGYhIJ2RR8AETROltzRLk9bXq5dMgvWF2+38DdBI RqJKMYMmHSYJ/Ccp0kLs4G6xhOw5VXpRIdPMOFPOh76szdJX801s8S0ZHWAPcc7q /lkfnc9JwKcLSQIGth0IllEfhe3/OtaZoi+SDV0hIuOwJ0AgesUzNnUIa41pAfW5 npGTImc8kXYSp1JoDyflpUgsY6Cv2w48K96QUTWpj5oOPtu4l2t0wsof0boekFjP BR12OBv/wB4= =cCKG -END PGP SIGNATURE- -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] confcache integration
On Tuesday 24 January 2006 21:50, Brian Harring wrote: +os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775) +os.chown(mysettings["CONFCACHE_DIR"], portage_uid, -1) This will die when running as non-root, no? "ebuild" is now oft used by devs running as normal users which will be broken by this. No clues on the bash stuff; it seems there's an external confcache binary but I can't tell much beyond that. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] confcache integration
On Wed, 2006-01-25 at 00:30 +0900, Jason Stubbs wrote: > On Tuesday 24 January 2006 21:50, Brian Harring wrote: > > +os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775) > +os.chown(mysettings["CONFCACHE_DIR"], portage_uid, -1) > > This will die when running as non-root, no? "ebuild" is now oft used by devs > running as normal users which will be broken by this. Your right. I've hit that bug myself in the past switching between root merges and being a real life dev and running ebuild foo.build clean unpack compile; -- solar <[EMAIL PROTECTED]> Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] confcache integration
On Tue, Jan 24, 2006 at 03:37:07PM -0500, solar wrote:
> On Wed, 2006-01-25 at 00:30 +0900, Jason Stubbs wrote:
> > On Tuesday 24 January 2006 21:50, Brian Harring wrote:
> >
> > +os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775)
> > +os.chown(mysettings["CONFCACHE_DIR"], portage_uid, -1)
> >
> > This will die when running as non-root, no? "ebuild" is now oft used by devs
> > running as normal users which will be broken by this.
>
> Your right. I've hit that bug myself in the past switching
> between root merges and being a real life dev and running ebuild
> foo.build clean unpack compile;
Meh. should've told me :P
Attached is round 2, including man/make.conf mods (ignore the fact
it's a single make.conf, if accepted the alt make.conf's will be
updated with the same text).
~harring
Index: pym/portage.py
===
--- pym/portage.py (revision 2583)
+++ pym/portage.py (working copy)
@@ -2669,7 +2669,38 @@
print "!!! Perhaps: rm -Rf",mysettings["BUILD_PREFIX"]
print "!!!",str(e)
return 1
+ try:
+ if "confcache" in features:
+ if not mysettings.has_key("CONFCACHE_DIR"):
+ mysettings["CONFCACHE_DIR"] =
os.path.join(mysettings["PORTAGE_TMPDIR"], "confcache")
+ if not
os.path.exists(mysettings["CONFCACHE_DIR"]):
+ if not os.getuid() == 0:
+ # we're boned.
+ features.remove("confcache")
+ mysettings["FEATURES"] = "
".join(features)
+ else:
+
os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775)
+
os.chown(mysettings["CONFCACHE_DIR"], -1, portage_gid)
+ else:
+ st =
os.stat(mysettings["CONFCACHE_DIR"])
+ if not (st.st_mode & 0) == 0775:
+
os.chmod(mysettings["CONFCACHE_DIR"], 0775)
+ if not st.st_gid == portage_gid:
+
os.chown(mysettings["CONFCACHE_DIR"], -1, portage_gid)
+ # check again, since it may have been disabled.
+ if "confcache" in features:
+ for x in listdir(mysettings["CONFCACHE_DIR"]):
+ p =
os.path.join(mysettings["CONFCACHE_DIR"], x)
+ st = os.stat(p)
+ if not (st.st_mode & 0) & 07600 ==
0600:
+ os.chmod(p, (st.st_mode & 0777)
| 0600)
+ if not st.st_gid == portage_gid:
+ os.chown(p, -1, portage_gid)
+
+ except OSError, e:
+ print "!!! Failed resetting perms on confcachedir %s" %
mysettings["CONFCACHE_DIR"]
+ return 1
#try:
# mystat=os.stat(mysettings["CCACHE_DIR"])
# if (mystat[stat.ST_GID]!=portage_gid) or
((mystat[stat.ST_MODE]&02070)!=02070):
Index: cnf/make.conf
===
--- cnf/make.conf (revision 2583)
+++ cnf/make.conf (working copy)
@@ -259,6 +259,8 @@
# 'buildpkg'causes binary packages to be created of all packages that
#are being merged.
# 'ccache' enables ccache support via CC.
+# 'confcache' enable confcache support; speeds up autotool based configure
+#calls
# 'collision-protect'
#prevents packages from overwriting files that are owned by
#another package or by no package at all.
Index: bin/ebuild.sh
===
--- bin/ebuild.sh (revision 2583)
+++ bin/ebuild.sh (working copy)
@@ -493,7 +493,22 @@
LOCAL_EXTRA_ECONF="--libdir=${CONF_LIBDIR_RESULT}
${LOCAL_EXTRA_ECONF}"
fi
- echo "${ECONF_SOURCE}/configure" \
+ local TMP_CONFCACHE_DIR CONFCACHE_ARG
+ if hasq confcache $FEATURES && ! hasq confcache $RESTRICT; then
+ CONFCACHE="$(type -p confcache)"
+ if [ -z "${CONFCACHE}" ]; then
+ ewarn "disabling confcache, binary cannot be
found"
+
Re: [gentoo-portage-dev] confcache integration
On Wed, Jan 25, 2006 at 12:30:22AM +0900, Jason Stubbs wrote: > it seems there's an external confcache binary but I can't tell much > beyond that. Yes, it's external (standalone) now- dev-util/confcache. ~harring pgp4K5t0iJcYM.pgp Description: PGP signature
Re: [gentoo-portage-dev] confcache integration
On Wed, 2006-01-25 at 05:18 -0800, Brian Harring wrote: > On Tue, Jan 24, 2006 at 03:37:07PM -0500, solar wrote: > > On Wed, 2006-01-25 at 00:30 +0900, Jason Stubbs wrote: > > > On Tuesday 24 January 2006 21:50, Brian Harring wrote: > > > > > > +os.makedirs(mysettings["CONFCACHE_DIR"], mode=0775) > > > +os.chown(mysettings["CONFCACHE_DIR"], portage_uid, -1) > > > > > > This will die when running as non-root, no? "ebuild" is now oft used by > > > devs > > > running as normal users which will be broken by this. > > > > Your right. I've hit that bug myself in the past switching > > between root merges and being a real life dev and running ebuild > > foo.build clean unpack compile; > Meh. should've told me :P I know I should have. Sorry time has been rather limited as of recent. > Attached is round 2, including man/make.conf mods (ignore the fact > it's a single make.conf, if accepted the alt make.conf's will be > updated with the same text). Putting this into local testing. -- solar <[EMAIL PROTECTED]> Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
