[gentoo-dev] [RFC] EAPI

2005-08-26 Thread Kristian Benoit 
On the EAPI subject Brian just brought back, I had this idea that we
could use the same approch XML took with HTML.

The ebuild could define which EAPI to use, but instead beiing a version,
the EAPI would be an ebuild API definition. The equivalent to the XML's
dtd. The ebuild could point to a directory named
$PORTDIR/eapi// which would contain a python script named
.py. If not already loaded, that plugable eapi would be
loaded before processing the ebuild.

That way, there is no outdated ebuild format. There is just a default
format which is the actual format.

It could also be an XML defining the ebuild's build sequence and other
particularities a group of ebuild could have.

Kristian

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC - Gentoo on the Lab

2005-08-23 Thread Kristian Benoit 
On Tue, 2005-08-23 at 12:27 -0500, Brian Harring wrote:
> On Tue, Aug 23, 2005 at 12:25:03PM -0400, Kristian Benoit wrote:
> > On Mon, 2005-08-22 at 15:39 -0500, Brian Harring wrote:
> > > That and help would always be welcome :P
> > 
> > Then where do I find the code (I'm an official dev yet, so I only have
> > access to what's in the mirrors and the patchs on mailing lists)
> If you're after a snapshot, give a yell; right now still waiting for 
> svn crap to straighten out (mainly for the rewrite to be moved fully 
> to svn), but once done I can point you at wherever I dump it in my 
> devspace till anonsvn is up.

Yeah, I'd really like having a snapshot, even if I'd prefer having
cvs/svn access. You can send me it by mail or make it available
somewhere.

> No clue on eta of anonsvn, that's infra's thing unfortunately.
> ~harring

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC - Gentoo on the Lab

2005-08-23 Thread Kristian Benoit 
On Mon, 2005-08-22 at 21:41 +0100, Stephen Bennett wrote:
> On Mon, 22 Aug 2005 13:49:14 -0400
> Kristian Benoit <[EMAIL PROTECTED]> wrote:
> 
> > I do agree with that, portage probably need a rewrite/better
> > modularization anyway. There is/was a project called portage-ng () you
> > might want to have a look at. I did a little in that direction
> > recently, and it seems that there is not too many people working on
> > it since drobbins left, but you can contact Pieter
> > ([EMAIL PROTECTED]). I might get on that too at some point in the
> > future too.
> 
> portage-ng is dead. There is a rewrite going on, but it'll take a while
> to get anywhere near usable.

Here is my recent communication with Pieter:

On Sat, 2005-08-13 at 21:59 +0200, Pieter Van den Abeele wrote: 
> On 13 Aug 2005, at 19:16, Kristian Benoit wrote:
> 
> > I've heard that you might be the last to know something about
> > portage-ng. What's the actual status,
> 
> Here's what I've done so far:
> 
> I've build upon Andreas' Zeller idea of using Smolka's feature logic  
> for software configuration management. Zeller's approach was ok for  
> determining consistency/inconsistency of software configurations. In  
> his phd thesis he described the algorithm involved and discussed time  
> complexity (which goes to NP if you allow for quantification). My  
> impression after implementing his idea was that for automated  
> construction there were a few things that were lacking, and some  
> things were incorrect. I revisited Zellers feature logic, and ended  
> up with a clean implementation of a declarative language which has  
> some very desirable properties for automated software construction.  
> I'd say my approach is closer to the spirit of Smolka's feature logic  
> than Zellers approach. My non-destructive feature unification has a  
> worst case time complexity of O(n x m) where n is the length of the  
> feature term describing your system, and m is the length of the  
> feature term describing the component to be added. In practice  
> performance is very good. An emerge --vp --emptytree kde with the  
> regular portage takes about 55 seconds on my Open Desktop Workstation  
> and produces a list of 200 packages. A similar action using my  
> implementation:
> 
> =
> Total time: 14.55 seconds
> =
> Predicate   Box Entries =Calls+Redos Time
> =
> vunify/4341,900 =  341,900+072.6%
> $garbage_collect/1  326 =  326+013.6%
> lists:append/3  684,320 =  684,320+0 4.0%
> genterm/2   520 =  520+0 3.9%
> hunify/4520 =  520+0 3.3%
> is/2342,420 =  342,420+0 1.8%
>  >/2 342,160 =  342,160+0 0.8%
> buildsystem/2 1 =1+0 0.0%
> val/3 5,200 =5,200+0 0.0%
> unify/3 260 =  260+0 0.0%
> 
> % 9,531,861 inferences, 13.96 CPU in 14.55 seconds (96% CPU, 682798  
> Lips)
> 
> The search space is kept as small as possible because I've introduced  
> lazy feature evaluation and both multi valued and open features.  
> Those concepts are missing in Zellers approach. Comparing current  
> portage and my implementation performance-wise is difficult. In  
> general mine is faster, but current portage doesn't use sql either.  
> What is for sure is that mine allows you to express various  
> constraints. You can prevent a dependency from being built with a  
> specific use flag. My implementation automatically solves 'blockers'.  
> Circular dependencies are also solved correctly. For instance, if you  
> want to install unixodbc with the qt use flag enabled and you want to  
> install qt wit the unixodbc use flag enabled, my portage knows that  
> it needs to:
> 
> emerge unixodbc without qt
> emerge qt with unixodbc
> remerge unixodbc with qt support
> 
> So it makes revdep-rebuild unnecessary basically.
> 
> 
> Once I was done implementing this new declarative language in which  
> for instance ebuild concepts can be easily expressed (but also rpm,  
> deb, fink, darwinports). I implemented a knowledge base in which  
> those feature terms can be stored/looked up efficiently. I used an  
&

Re: [gentoo-dev] RFC - Gentoo on the Lab

2005-08-23 Thread Kristian Benoit 
On Mon, 2005-08-22 at 15:39 -0500, Brian Harring wrote:
> That and help would always be welcome :P

Then where do I find the code (I'm an official dev yet, so I only have
access to what's in the mirrors and the patchs on mailing lists)

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC - Gentoo on the Lab

2005-08-22 Thread Kristian Benoit 
On Mon, 2005-08-22 at 16:38 +0200, Marius Mauch wrote:

> Anyway, I hope you realize that your project doesn't only involve
> hacking on portage, but rewriting almost all of it for the client part.
> Actually I'd rather suggest you start from scratch

I do agree with that, portage probably need a rewrite/better
modularization anyway. There is/was a project called portage-ng () you
might want to have a look at. I did a little in that direction recently,
and it seems that there is not too many people working on it since
drobbins left, but you can contact Pieter ([EMAIL PROTECTED]). I might
get on that too at some point in the future too.

There is also Patrick Mclean ([EMAIL PROTECTED]) that has a nice
lab setup using Gentoo at McGill. It's a diskless system so might not be
to much what you are looking for, but as you probably want to dig before
getting to work, you should really see his setup. He did not released
anything yet, but we talked about starting a wiki for gentoo labs
sometime and have his setup released there.

Kristian

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [patch] preconfig to allow user configuration before compilation

2005-07-15 Thread Kristian Benoit 
On Fri, 2005-07-15 at 20:13 +0900, Jason Stubbs wrote:
> By the way, you seemed to attach a unified diff between your modified
> portage and an empty directory. Please don't do the same for busybox.
> Sending a 1.4MB patch to a public mailing list is not very nice -
> especially not in one go.

I'm sincerilly truly sorry, I'll try to triple check in the futur.
I just can't believe what I did.
I ran the same command and here's the output ???

> Patches belong on bugs.gentoo.org anyway.

Sorry, I'm getting use to the LKML.
As I sent a bad patch, I'll send the correction here, but I'll send
patch in bugs.gentoo.org in the futur.

Thanks

Kristian
diff -uNrp portage-2.0.51.19/bin/ebuild.sh portage-2.0.51.19-preconfig/bin/ebuild.sh
--- portage-2.0.51.19/bin/ebuild.sh	2005-02-26 06:22:37.0 -0500
+++ portage-2.0.51.19-preconfig/bin/ebuild.sh	2005-07-14 20:21:38.0 -0400
@@ -548,6 +548,11 @@ src_unpack() {
 	fi	
 }
 
+src_preconfig() {
+	echo "default preconfig"
+	return `false`
+}
+
 src_compile() {
 	if [ -x ./configure ]; then
 		econf
@@ -685,6 +690,20 @@ dyn_unpack() {
 	trap SIGINT SIGQUIT
 }
 
+dyn_preconfig() {
+	echo ">>> Preconfiguring source..."
+	cd ${S}
+	local preconfigured = 1
+	echo "some more stuff"
+	src_preconfig && preconfigured=0 || preconfigured=1
+
+	if [ preconfigured ]; then
+		echo ">>> Source preconfigured."
+	else
+		echo ">>> Could not preconfigure the source."
+	fi
+}
+
 dyn_clean() {
 	if [ "$USERLAND" == "BSD" ] && type -p chflags &>/dev/null; then
 		chflags -R noschg,nouchg,nosappnd,nouappnd,nosunlnk,nouunlnk \
@@ -1778,7 +1797,7 @@ for myarg in $*; do
 			set +x
 		fi
 		;;
-	unpack|compile|test|clean|install)
+	unpack|preconfig|compile|test|clean|install)
 		if [ "${SANDBOX_DISABLED="0"}" == "0" ]; then
 			export SANDBOX_ON="1"
 		else
diff -uNrp portage-2.0.51.19/pym/portage.py portage-2.0.51.19-preconfig/pym/portage.py
--- portage-2.0.51.19/pym/portage.py	2005-03-03 11:45:01.0 -0500
+++ portage-2.0.51.19-preconfig/pym/portage.py	2005-07-14 20:53:15.0 -0400
@@ -2253,9 +2253,14 @@ def spawnebuild(mydo,actionmap,mysetting
 			retval=spawnebuild(actionmap[mydo]["dep"],actionmap,mysettings,debug,alwaysdep=alwaysdep,logfile=logfile)
 			if retval:
 return retval
+
+	# Do not log for preconfig as it break interactivity.
+	if mydo=="preconfig":
+		logfile=None
+
 	# spawn ebuild.sh
 	mycommand = EBUILD_SH_BINARY + " "
-	if selinux_enabled and ("sesandbox" in features) and (mydo in ["unpack","compile","test","install"]):
+	if selinux_enabled and ("sesandbox" in features) and (mydo in ["unpack","preconfig","compile","test","install"]):
 		con=selinux.getcontext()
 		con=string.replace(con,mysettings["PORTAGE_T"],mysettings["PORTAGE_SANDBOX_T"])
 		selinux.setexec(con)
@@ -2295,7 +2300,7 @@ def doebuild(myebuild,mydo,myroot,mysett
 
 	validcommands = ["help","clean","prerm","postrm","preinst","postinst",
 	"config","setup","depend","fetch","digest",
-	"unpack","compile","test","install","rpm","qmerge","merge",
+	"unpack","preconfig","compile","test","install","rpm","qmerge","merge",
 	"package","unmerge", "manifest"]
 
 	if mydo not in validcommands:
@@ -2533,7 +2538,7 @@ def doebuild(myebuild,mydo,myroot,mysett
 			return unmerge(mysettings["CATEGORY"],mysettings["PF"],myroot,mysettings)
 
 	# if any of these are being called, handle them -- running them out of the sandbox -- and stop now.
-	if mydo=="clean":
+	if mydo in ["clean","config"]:
 		logfile=None
 	if mydo in ["help","clean","setup"]:
 		return spawn(EBUILD_SH_BINARY+" "+mydo,mysettings,debug=debug,free=1,logfile=logfile)
@@ -2623,13 +2628,18 @@ def doebuild(myebuild,mydo,myroot,mysett
 			  "depend": { "args":(0,1)}, # sandbox  / portage
 			  "setup":  { "args":(1,0)}, # without  / root
 			 "unpack":  {"dep":"setup",   "args":(0,1)}, # sandbox  / portage
-			"compile":  {"dep":"unpack",  "args":(nosandbox,1)}, # optional / portage
+		  "preconfig":  {"dep":"unpack",  "args":(nosandbox,1)}, # optional / portage
 			   "test":  {"dep":"compile", "args":(nosandbox,1)}, # optional / portage
 			"install":  {"dep":"test","args":(0,0)}, # sandbox  / root
 			"rpm":  {"dep":"install", "args":(0,0)}, # sandbox  / root
 	"package":  {"dep":"install", "args":(0,0)}, # sandbox  / root
 	}
-	
+
+	if "preconfig" in mysettings["FEATURES"].split():
+		actionmap["compile"] =  {"dep":"preconfig",  "args":(nosandbox,1)} # optional / portage
+	else:
+		actionmap["compile"] =  {"dep":"unpack",  "args":(nosandbox,1)} # optional / portage
+
 	if mydo in actionmap.keys():	
 		if mydo=="package":
 			for x in ["","/"+mysettings["CATEGORY"],"/All"]:
--- busybox-1.00-r4.ebuild	2005-07-14 20:39:47.0 -0400
+++ busybox-1.00-r6.ebuild	2005-07-14 20:38:51.0 -0400
@@ -91,6 +91,7 @@ src_unpack() {
 		done
 		if [[ -r ${S}/.config ]] ; then
 			einfo "

[gentoo-dev] [patch] config interactiveness fix.

2005-07-14 Thread Kristian Benoit 
Actually, when executing pkg_config with "ebuild some/ebuild.ebuild
config", the stdin/stdout are broken cause we are not writing to a
terminal.

I fixed it with this patch:

diff -uNpr portage-2.0.51.19/pym/portage.py
portage-2.0.51.19-config/pym/portage.py
--- portage-2.0.51.19/pym/portage.py2005-03-03 11:45:01.0
-0500
+++ portage-2.0.51.19-config/pym/portage.py 2005-07-14
20:48:36.0 -0400
@@ -2533,7 +2533,7 @@ def doebuild(myebuild,mydo,myroot,mysett
return
unmerge(mysettings["CATEGORY"],mysettings["PF"],myroot,mysettings)
 
# if any of these are being called, handle them -- running them
out of the sandbox -- and stop now.
-   if mydo=="clean":
+   if mydo in ["clean","config"]:
logfile=None
if mydo in ["help","clean","setup"]:
return spawn(EBUILD_SH_BINARY+"
"+mydo,mysettings,debug=debug,free=1,logfile=logfile)


The only bad side is that we loose the log for that part. But it might
not be loggable anyway. As Jason Stubbs said:

"Actually, I promote interactive code in pkg_config(). There's no
standard as to what it will do, ..."

Kristian


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-embedded] Interactive command

2005-07-14 Thread Kristian Benoit 
On Thu, 2005-07-14 at 12:01 -0400, Michael Cummings wrote:
> Not sure of the mechanics on how
> this is done for the portage emerge process, but even stopping the
> emerge process for those packages that *must* have a preconfig done
> would help (are there any though? I can think of one maybe, but not
> really anything to demand new functionality in portage yet).

This problem is mostly present in the embedded world. As the packages
have to be fine tuned (as a kernel is) at compiled time for the system
on which it is beiing build. For example, uClibc and busybox uses a
kernel like config.

As described on busybox's home page:

"BusyBox combines tiny versions of many common UNIX utilities into a
single small executable. It provides replacements for most of the
utilities you usually find in GNU fileutils, shellutils, etc. The
utilities in BusyBox generally have fewer options than their
full-featured GNU cousins; however, the options that are included
provide the expected functionality and behave very much like their GNU
counterparts. BusyBox provides a fairly complete environment for any
small or embedded system."

With so many applications/options provided by a single package, that aim
to go in small memory/storage system, one must understand that users
must fine tune it.

Kristian

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-embedded] Interactive command

2005-07-14 Thread Kristian Benoit 
On Thu, 2005-07-14 at 21:37 +0900, Jason Stubbs wrote:
> On Thursday 14 July 2005 20:58, Ned Ludd wrote:
> >   echo "being that no portage dev in his/her right mind would ever"
> >   echo "allow interactive code in an ebuild we use bashrc tricks"
> 
> Actually, I promote interactive code in pkg_config(). There's no standard as 
> to what it will do, so there's no real solution other than telling the user 
> and then waiting for confirmation.
> 
> As for the other phases, they should of course be 100% non-interactive. 
> However, a pkg_presetup() or some such couldn't go astray - as long as it was 
> purely optional. It would be much better to wait until portage supports 
> arbitrary per-package env for it to be of any real use though.

I just checked and pkg_config does not handle make menuconfig correctly
either :( Probably a bug.

One of my option was to unpack, config, merge (it would do the same, but
you would have to know you must config after unpack)... I'm goiing to
check how to fix that. :)

Kristian

> Regards,
> Jason Stubbs

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Interactive command

2005-07-14 Thread Kristian Benoit 
On Wed, 2005-07-13 at 16:49 -0700, Donnie Berkholz wrote:
> BTW, it's not in good form to post to multiple lists where only
> subscribers can respond.

I appologise.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-embedded] Interactive command

2005-07-14 Thread Kristian Benoit 
On Thu, 2005-07-14 at 08:43 +0200, Patrick Lauer wrote:
> On Thu, 2005-07-14 at 01:13 -0400, Kristian Benoit wrote:
> > What I want is "emerge busybox uclibc vanilla-sources nano". Should
> > unpack only the 3 first packages, show me busybox menuconfig, uclibc
> > menuconfig and vanilla-sources menuconfig and only then perform the rest
> > of the installation as usual (without re-unpacking the 3 packages).
> I really really hope that you want this to be a totally optional feature
> that is never ever enabled by default.

Of course it has to be totally optional and preconfig should be called
before any packages compilation. So that you get every preconfig
interactive prompt before everything else is done.

> When I run emerge -uD world on a server I don't accept any ebuild
> waiting for interactive input. For stagebuilding and other activities
> this behaviour is also unacceptable. 

I wont accept that either :).

> >  And
> > save the configs for futur use. So if my configs does not fit my need,
> > re-emerging busybox with USE=savedconfig should get me the busybox
> > menuconfig back using the previously configured ".config".
> Patch the ebuilds with your .config

Nope, it would just use the actual solution, use the one
in /etc/busybox/busybox.config

> > But anyway, the question is not whether the idea is good or not, but how
> > to get a menuconfig that works within ebuild.sh (called from spawn in
> > portage_exec ... called from the user with ebuild or emerge)?
> *shudder* I hope this stays limited to your overlay.
>
> wkr,
> Patrick

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: [gentoo-embedded] Interactive command

2005-07-14 Thread Kristian Benoit 
On Thu, 2005-07-14 at 07:58 -0400, Ned Ludd wrote:
> On Thu, 2005-07-14 at 01:13 -0400, Kristian Benoit wrote:
> > On Wed, 2005-07-13 at 20:02 -0400, Ned Ludd wrote:
> > > On Wed, 2005-07-13 at 18:44 -0400, Kristian Benoit wrote:
> > > > I'm trying to add a new command to ebuild (preconfig) for packages like
> 
> 
> > But anyway, the question is not whether the idea is good or not, but how
> > to get a menuconfig that works within ebuild.sh (called from spawn in
> > portage_exec ... called from the user with ebuild or emerge)?
> 
> Well portage is a non-interactive tool and it's staying that way.
> For what you want you will need to edit the busybox/-sources/uClibc 
> ebuild and manually add in a call to make menuconfig. 
> Or add something like this untested example to your 
> local /etc/portage/bashrc

I'm not trying to make it more interactive than it actually is.

stated from:
http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=6#doc_chap3

"Performing Configuration Actions

Some applications include instructions that configure the package
further on your system. These instructions can be interactive and are
therefore not automatically executed. To run these configuration steps,
which are enlisted in the ebuild's (optional) config() function, use
ebuild's config functionality:


Code Listing 11: Configuring a package


# ebuild path/to/ebuild config"

I'm trying to do the same as config does, but in as a preconfig that
would only be also called on user request.

My goal is actually mostly learning portage internal so it might never
do it's way into production and it might be fine like that.

Thanks for the bashrc tip, I did'nt know that.

Kristian

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: [gentoo-embedded] Interactive command

2005-07-13 Thread Kristian Benoit 
On Wed, 2005-07-13 at 20:02 -0400, Ned Ludd wrote:
> On Wed, 2005-07-13 at 18:44 -0400, Kristian Benoit wrote:
> > I'm trying to add a new command to ebuild (preconfig) for packages like
> > busybox that has to be configured prior to compile and that would be a
> > cannot really use USE as it probably have undreads of USE that does not
> > yet exist.
> > 
> > That new command would be called between src_unpack and src_compile and
> > only if FEATURES contain preconfig.
> 
> What you want already exists. Enable USE=savedconfig
> Then add an /etc/busybox/busybox.config
> Then merge busybox.

I know that this option exist, but I must either download busybox by
myself, make menuconfig by myself and copy it to the right location by
myself or ebuild unpack and do the rest by myself. It look pretty much
as a patch to me, cause portage cant handle packages that has to be user
configured before compile time.

What I want is "emerge busybox uclibc vanilla-sources nano". Should
unpack only the 3 first packages, show me busybox menuconfig, uclibc
menuconfig and vanilla-sources menuconfig and only then perform the rest
of the installation as usual (without re-unpacking the 3 packages). And
save the configs for futur use. So if my configs does not fit my need,
re-emerging busybox with USE=savedconfig should get me the busybox
menuconfig back using the previously configured ".config".

But anyway, the question is not whether the idea is good or not, but how
to get a menuconfig that works within ebuild.sh (called from spawn in
portage_exec ... called from the user with ebuild or emerge)?

Kristian

> Have fun.
> 
> 
> -- 
> Ned Ludd <[EMAIL PROTECTED]>
> 

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Interactive command

2005-07-13 Thread Kristian Benoit 
I'm trying to add a new command to ebuild (preconfig) for packages like
busybox that has to be configured prior to compile and that would be a
cannot really use USE as it probably have undreads of USE that does not
yet exist.

That new command would be called between src_unpack and src_compile and
only if FEATURES contain preconfig.

Anyway, the problem is when I run "make menuconfig" from src_preconfig,
I get a dumb terminal (I cannot send keys to the menuconfig, but it
works for vim):

pressing down prints "^[OB"
pressing up prints "^[OA"
the COLUMNS/LINES are unset...

So I wonder if there is a way to make interactive command work from an
ebuild. I know the config command should allow interactiveness.

Kristian 

-- 
gentoo-dev@gentoo.org mailing list