Re: MacPorts AutoBuild

2008-08-02 Thread Rainer Müller
Jordan K. Hubbard wrote:
 
 On Jun 20, 2008, at 5:06 PM, Bryan Blackburn wrote:
 
 Some basic stuff at http://trac.macports.org/wiki/MPAB, the readme  
 in the tarball has more info currently than that wiki page.
 
 Is there a place for ongoing development of this?  I've already got some 
 Leopard-specific changes which will result in a lot more of the 
 xcodeproject ports building, but I'm not really clear on how or where to 
 submit them.  Thanks.

I imported it to contrib/mpab today. I also added the Subversion url to 
the wiki page at http://trac.macports.org/wiki/MPAB.

Everybody with commit access can contribute to it.

Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-08-02 Thread Rainer Müller
Rainer Müller wrote:
 I imported it to contrib/mpab today. I also added the Subversion url to 
 the wiki page at http://trac.macports.org/wiki/MPAB.

Can somebody with the appropriate permissions please remove the tarball 
from http://trac.macports.org/wiki/MPAB to avoid confusion?

 Everybody with commit access can contribute to it.

Reading my own sentence again, this is not correct. Of course anybody 
can contribute, you just need to send a patch! ;-)

Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-08-02 Thread William Siegrist

On Aug 2, 2008, at 8:51 PM, Rainer Müller wrote:


Rainer Müller wrote:
I imported it to contrib/mpab today. I also added the Subversion  
url to

the wiki page at http://trac.macports.org/wiki/MPAB.


Can somebody with the appropriate permissions please remove the  
tarball

from http://trac.macports.org/wiki/MPAB to avoid confusion?




Done.

-Bill






smime.p7s
Description: S/MIME cryptographic signature
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-07-05 Thread Bryan Blackburn
On Jul 3, 2008, at 10:45 PM, Jordan K. Hubbard wrote:

 On Jun 20, 2008, at 5:06 PM, Bryan Blackburn wrote:

 Some basic stuff at http://trac.macports.org/wiki/MPAB, the readme
 in the tarball has more info currently than that wiki page.

 Is there a place for ongoing development of this?  I've already got  
 some Leopard-specific changes which will result in a lot more of the  
 xcodeproject ports building, but I'm not really clear on how or  
 where to submit them.  Thanks.


At the moment, no, just the tarball on the wiki.

Bryan


 - Jordan


___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-07-03 Thread Jordan K. Hubbard


On Jun 20, 2008, at 5:06 PM, Bryan Blackburn wrote:


Some basic stuff at http://trac.macports.org/wiki/MPAB, the readme
in the tarball has more info currently than that wiki page.


Is there a place for ongoing development of this?  I've already got  
some Leopard-specific changes which will result in a lot more of the  
xcodeproject ports building, but I'm not really clear on how or where  
to submit them.  Thanks.


- Jordan

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Contrib area (was: Re: MacPorts AutoBuild)

2008-07-02 Thread Rainer Müller
Ryan Schmidt wrote:
 The page says MPAB is to be downloaded from the attachment on that  
 wiki page. Why not put it in the repository somewhere, like in /users/ 
 blb?

Maybe we should create a 'contrib area' in our repository? That would be 
another directory like /trunk/contrib or /trunk/tools for stuff like this.

I consider /users to be the private room of developers and I don't want 
to tread on someones toes by committing there (of course it's a path 
like any other, but still there is some psychological barrier...).

We currently have some stuff lying around which is not necessarily bound 
to only one user and could therefore be on a more prominent place. Also, 
this might attract more developers to work on it.

I suggest to move the following things from other places
into a 'contrib area':

   * MPWA (from /users/jberry/mpwa/)
   * MPAB (from the wiki)
   * select (used for {gcc,python}_select, from /users/mww/select)
   * MacPorts Framework [1]
   * Pallet (from /users/rhwood/Pallet)

[1] I assume that the plans are to put the framework into base
 once it is finished. George is currently working in
 his own branch on it, so it would be better to delay this.

Of course I understand if someone wants total control over his project 
(and rather wants to see patches than commits from others), this could 
remain in the /users space.

Comments?

Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-24 Thread Ryan Schmidt

On May 31, 2008, at 16:24, Bryan Blackburn wrote:

 I just finished uploading what I've been working on for the last few
 weeks off and on, my redo of my old DarwinPorts AutoBuild scripts.   
 See

 http://trac.macports.org/wiki/MPAB

The page says MPAB is to be downloaded from the attachment on that  
wiki page. Why not put it in the repository somewhere, like in /users/ 
blb?

 Right now it builds ports fine in the chroot (though now with 10.5.3 I
 need to rebuild a newer version of my chroot), but does have a few
 limitations.  One is that I've currently only tested it with 10.5 and
 Xcode 3.0.  See the Todo section in the ReadMe.txt file that comes
 with the archive.

 I've had it successfully build about 145 ports so far, with several
 failing for various reasons.  It starts to slow down when you are
 dealing when building a port with many dependencies as it will
 uninstall all ports between each build attempt for better cleanroom
 building.  My MBP's HD gets a bit slow when frequently extracting then
 removing thousands of files

 Give it a try if you'd like and reply here or update the wiki page.

Does MPAB just try to build the port with its default variants, or  
does it try all permutations of variants the port will accept? I  
imagine it's not uncommon for maintainers to test a new version of a  
port with the default variants but forget (or not have the time) to  
test all variants, so variants might break over time. Patches only  
added in variants might need to be updated. Some variants might  
conflict with one another but the maintainer may not have marked  
that. Etc.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-24 Thread Jordan K. Hubbard

On Jun 23, 2008, at 10:51 PM, Bryan Blackburn wrote:

 So far it's working fine, it's tried to build about 170 ports (23
 failed, some because of dependencies failing).  Watching things like
 the chroot/opt/local/[lib|bin] show things coming and going as they
 should, which should allow ports with undeclared dependencies to fail
 when not finding what they want.

Is this work on progress checked in on a branch in macports anywhere,  
or...?

- Jordan

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-23 Thread Bryan Blackburn
On Jun 22, 2008, at 8:46 AM, Daniel J. Luke wrote:

 On Jun 20, 2008, at 8:00 PM, Bryan Blackburn wrote:

 Hmm, that's an interesting thought; since activate only creates hard
 links it'll definitely be faster than extracting the archive followed
 by creating those hard links.  While inactive, ports shouldn't be
 looking in /opt/local/var/macports/software for stuff, so should be
 safe.  Thanks for the idea.

 If it doesn't work for some reason (like #7361 or some other issue)  
 using direct mode instead of image mode would be an easy minor  
 performance improvement (saving the time used to make the  
 hardlinks). Probably not as much of a win as being able to use  
 activate/deactivate, though.


So far it's working fine, it's tried to build about 170 ports (23  
failed, some because of dependencies failing).  Watching things like  
the chroot/opt/local/[lib|bin] show things coming and going as they  
should, which should allow ports with undeclared dependencies to fail  
when not finding what they want.

I didn't do any stopwatch runs before/after so I can't say if things  
are any better, but since I've usually been sitting at my machine when  
much of the build is going, it definitely feels like it's getting  
through more ports.

Bryan


 --
 Daniel J. Luke
 ++
 | * [EMAIL PROTECTED] * |
 | *-- http://www.geeklair.net -* |
 ++
 |   Opinions expressed are mine and do not necessarily   |
 |  reflect the opinions of my employer.  |
 ++

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-22 Thread Daniel J. Luke

On Jun 20, 2008, at 8:00 PM, Bryan Blackburn wrote:

Have you tried using activate/deactivate instead of archives/
uninstall? I don't know if it would be significantly faster or not,
but it would probably be worth testing. I might eventually have time
to investigate this (but probably not for the next week or so).


Hmm, that's an interesting thought; since activate only creates hard
links it'll definitely be faster than extracting the archive followed
by creating those hard links.  While inactive, ports shouldn't be
looking in /opt/local/var/macports/software for stuff, so should be
safe.  Thanks for the idea.


If it doesn't work for some reason (like #7361 or some other issue)  
using direct mode instead of image mode would be an easy minor  
performance improvement (saving the time used to make the hardlinks).  
Probably not as much of a win as being able to use activate/ 
deactivate, though.


--
Daniel J. Luke
++
| * [EMAIL PROTECTED] * |
| *-- http://www.geeklair.net -* |
++
|   Opinions expressed are mine and do not necessarily   |
|  reflect the opinions of my employer.  |
++





PGP.sig
Description: This is a digitally signed message part
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Daniel J. Luke

On May 31, 2008, at 5:24 PM, Bryan Blackburn wrote:

I've had it successfully build about 145 ports so far, with several
failing for various reasons.  It starts to slow down when you are
dealing when building a port with many dependencies as it will
uninstall all ports between each build attempt for better cleanroom
building.  My MBP's HD gets a bit slow when frequently extracting then
removing thousands of files

Give it a try if you'd like and reply here or update the wiki page.



I'm running it now, and it seems pretty cool.

I don't know if you have plans for future development, but I was  
thinking the following would be good:


- Add a way to add more macports.conf settings (enabling parallel  
builds perhaps)
- Use archive mode or activate/inactivate to speed things up for ports  
with lots of dependencies (unless I'm missing something, I think that  
ports end up getting rebuilt multiple times as dependencies since  
everything gets uninstalled between port builds)


We could also do a 'distributed' build test by having the build app  
ask a macports server for a port (or list of ports) it should build  
test and then it could send back the success/failure + log which could  
be stored. We could then send out nag emails (or open tickets)  
automatically for ports that don't build, which would be nice.


Alternatively, we could set up a dedicated build-test system somewhere  
that could test ports as they are changed (perhaps a post-commit hook  
could add to a list of ports to test), but I don't think that  
macosforge has resources for that (yet?)


Thoughts?
--
Daniel J. Luke
++
| * [EMAIL PROTECTED] * |
| *-- http://www.geeklair.net -* |
++
|   Opinions expressed are mine and do not necessarily   |
|  reflect the opinions of my employer.  |
++





PGP.sig
Description: This is a digitally signed message part
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Daniel J. Luke

On Jun 20, 2008, at 1:10 PM, Daniel J. Luke wrote:
- Add a way to add more macports.conf settings (enabling parallel  
builds perhaps)
- Use archive mode or activate/inactivate to speed things up for  
ports with lots of dependencies (unless I'm missing something, I  
think that ports end up getting rebuilt multiple times as  
dependencies since everything gets uninstalled between port builds)


nevermind, I just noticed that you already have it set up to do this :)

--
Daniel J. Luke
++
| * [EMAIL PROTECTED] * |
| *-- http://www.geeklair.net -* |
++
|   Opinions expressed are mine and do not necessarily   |
|  reflect the opinions of my employer.  |
++





PGP.sig
Description: This is a digitally signed message part
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread William Siegrist


On Jun 20, 2008, at 10:10 AM, Daniel J. Luke wrote:


On May 31, 2008, at 5:24 PM, Bryan Blackburn wrote:

I've had it successfully build about 145 ports so far, with several
failing for various reasons.  It starts to slow down when you are
dealing when building a port with many dependencies as it will
uninstall all ports between each build attempt for better cleanroom
building.  My MBP's HD gets a bit slow when frequently extracting  
then

removing thousands of files

Give it a try if you'd like and reply here or update the wiki page.



Alternatively, we could set up a dedicated build-test system  
somewhere that could test ports as they are changed (perhaps a post- 
commit hook could add to a list of ports to test), but I don't think  
that macosforge has resources for that (yet?)





Once MacPorts has the software ready for a build system, I can start  
requesting hardware to support it. I have not looked at MPAB yet to  
see how close it is to an automatic build system. I also remember  
James saying MPWA would be used as the front-end for such a system,  
and I dont think MPWA is ready either.


So, if someone wants to lead the initiative to assemble the parts and  
explain how I can deploy it on hardware, I'd be happy to ask for more  
servers here.  If no one wants to take charge of this, I will  
eventually work on it, but dont know when I'll have time.



-Bill


---
William Siegrist
Mac OS Forge
http://macosforge.org/









smime.p7s
Description: S/MIME cryptographic signature
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Jordan K. Hubbard

On Jun 20, 2008, at 11:11 AM, William Siegrist wrote:

 Once MacPorts has the software ready for a build system, I can start  
 requesting hardware to support it. I have not looked at MPAB yet to  
 see how close it is to an automatic build system. I also remember  
 James saying MPWA would be used as the front-end for such a system,  
 and I dont think MPWA is ready either.

 So, if someone wants to lead the initiative to assemble the parts  
 and explain how I can deploy it on hardware, I'd be happy to ask for  
 more servers here.  If no one wants to take charge of this, I will  
 eventually work on it, but dont know when I'll have time.

We have a considerable collection of hardware lying around which could  
be repurposed to such a role rather easily.   There's a machine you  
use every day for backups, for example, which was, in fact, originally  
conceived and allocated to this exact purpose (you must have wondered  
at that attached RAID).  Of course that simply never happened, at  
least not beyond my own primitive attempts, all of which yielded such  
terrible results [50% failure rates] that I came to have dark  
suspicions about my methodology for creating the build chroot (as well  
as dark suspicions about how many of our ports actually build at any  
given time) and went on to other things.

Anyway, if MPAB/MPWA are starting to generate good results on whatever  
development hardware is being used, and by good results I mean all  
of the below:

o   Building, or at least iterating through, all the ports in the system  
and generating helpful status information for each (even if it's  
falls over immediately, that's good to know)

o   Taking at least some pains to isolate the build products from the  
builder host machine, both in files read and files written.

o   Has had all reasonable precautions taken after doing a reasonably  
pragmatic analysis of the security implications of executing all that  
open-ended Tcl code and how a rogue port might attack the builder,  
either deliberately or through carelessness.

Then I'd say it's time for us to start thinking seriously about  
putting this into early production.   This is the same checklist the  
project is going to have to go down before anyone will be willing to  
even BETA this on more private hardware anyway, so I don't think  
it's an unreasonable one.

- Jordan



___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Bryan Blackburn
On Jun 20, 2008, at 11:10 AM, Daniel J. Luke wrote:

...

 I don't know if you have plans for future development, but I was  
 thinking the following would be good:

 - Add a way to add more macports.conf settings (enabling parallel  
 builds perhaps)

Currently, I simply punt on this issue, since it'd be tough to figure  
out what settings for various macports.conf values would be best for a  
given use.  You can use './mpab mount' to mount the chroot and 'vi  
mpchroot/opt/local/etc/macports/macports.conf' to change whatever you  
need.


 - Use archive mode or activate/inactivate to speed things up for  
 ports with lots of dependencies (unless I'm missing something, I  
 think that ports end up getting rebuilt multiple times as  
 dependencies since everything gets uninstalled between port builds)

As you noticed, it does use archive mode, but since it uninstalls all  
installed ports after each port-build attempt, you'll see it reinstall  
popular dependencies (eg, pkgconfig, zlib, etc) quite a bit.  This way  
it should catch ports that have missed necessary deps.  Fortunately  
with archive mode it only has to reinstall the files, but if you're on  
a slower disk, some ports with 20+ deps can still take a while to do  
these; especially larger ports with lots of files, like ncursesw with  
over 3000 files alone, seriously IO bound.

Bryan




...
 --
 Daniel J. Luke
 ++
 | * [EMAIL PROTECTED] * |
 | *-- http://www.geeklair.net -* |
 ++
 |   Opinions expressed are mine and do not necessarily   |
 |  reflect the opinions of my employer.  |
 ++

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Bryan Blackburn
On Jun 20, 2008, at 12:11 PM, William Siegrist wrote:

...


 Once MacPorts has the software ready for a build system, I can start  
 requesting hardware to support it. I have not looked at MPAB yet to  
 see how close it is to an automatic build system. I also remember  
 James saying MPWA would be used as the front-end for such a system,  
 and I dont think MPWA is ready either.


MPAB works now, though I've only built maybe 5% of MacPorts's ports on  
my MBP, and even that has generated several trac tickets (and a few I  
think I forgot to file as well).

 So, if someone wants to lead the initiative to assemble the parts  
 and explain how I can deploy it on hardware, I'd be happy to ask for  
 more servers here.  If no one wants to take charge of this, I will  
 eventually work on it, but dont know when I'll have time.


If you look at the readme in the tarball, I currently claim the  
necessary parts are 10.5.[23], Xcode 3.0, and Apple's X11 (including  
X11SDK from Xcode of course); then a tarball of MacPorts itself.   
After that, 'sudo ./mpab' should start.  It builds in a chroot, so the  
first run will take some time as it builds the chroot (4.5G or so).

Bryan



 -Bill


 ---
 William Siegrist
 Mac OS Forge
 http://macosforge.org/


___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Bryan Blackburn
On Jun 20, 2008, at 1:28 PM, Jordan K. Hubbard wrote:

...
 Of course that simply never happened, at
 least not beyond my own primitive attempts, all of which yielded such
 terrible results [50% failure rates] that I came to have dark
 suspicions about my methodology for creating the build chroot (as well
 as dark suspicions about how many of our ports actually build at any
 given time) and went on to other things.


While what I've written does have some of it early lineage from your  
buildall.sh script, it's definitely changed a bit.  The good news is  
that of the 150+ or so ports I had MPAB try to build here, a vast  
majority of them built successfully.  The failures were all  
explainable at the MacPorts level, not MPAB (eg missing distfiles, a  
perl module needing 'port -f install', and so on).

 Anyway, if MPAB/MPWA are starting to generate good results on whatever
 development hardware is being used, and by good results I mean all
 of the below:

 o Building, or at least iterating through, all the ports in the system
 and generating helpful status information for each (even if it's
 falls over immediately, that's good to know)


MPAB currently builds the list of ports to attempt by sorting them by  
dependency: the early ones it builds are dependencies of later ones.   
The reason for this is that if Y depends on X and X failed, it'll  
simply skip Y.  Also, of course, if X did build successfully, all  
other ports depending on it will simply cause X do install from the  
portarchive instead of rebuilding again.  If you extract the MPAB  
tarball, the chroot-scripts/genportlist.tcl is the script which builds  
this initial list.

 o Taking at least some pains to isolate the build products from the
 builder host machine, both in files read and files written.


It's a chroot, so other than the initial building of the chroot, the  
host machine's files should be left alone.  Of course, the bad thing  
is chroot needs root to run...

 o Has had all reasonable precautions taken after doing a reasonably
 pragmatic analysis of the security implications of executing all that
 open-ended Tcl code and how a rogue port might attack the builder,
 either deliberately or through carelessness.


The biggest issue here is breaking out of the chroot, otherwise if  
something malicious happens then all future build attempts would most  
likely fail quite spectacularly.

Bryan


 Then I'd say it's time for us to start thinking seriously about
 putting this into early production.   This is the same checklist the
 project is going to have to go down before anyone will be willing to
 even BETA this on more private hardware anyway, so I don't think
 it's an unreasonable one.

 - Jordan


___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Daniel J. Luke

On Jun 20, 2008, at 4:47 PM, Bryan Blackburn wrote:

Currently, I simply punt on this issue, since it'd be tough to figure
out what settings for various macports.conf values would be best for a
given use.  You can use './mpab mount' to mount the chroot and 'vi
mpchroot/opt/local/etc/macports/macports.conf' to change whatever you
need.


Yeah, I modified where you turned on archivemode to also tweak the  
conf to do parallel build with some of the cores on my workstation.



- Use archive mode or activate/inactivate to speed things up for
ports with lots of dependencies (unless I'm missing something, I
think that ports end up getting rebuilt multiple times as
dependencies since everything gets uninstalled between port builds)


As you noticed, it does use archive mode, but since it uninstalls all
installed ports after each port-build attempt, you'll see it reinstall
popular dependencies (eg, pkgconfig, zlib, etc) quite a bit.  This way
it should catch ports that have missed necessary deps.  Fortunately
with archive mode it only has to reinstall the files, but if you're on
a slower disk, some ports with 20+ deps can still take a while to do
these; especially larger ports with lots of files, like ncursesw with
over 3000 files alone, seriously IO bound.



Have you tried using activate/deactivate instead of archives/ 
uninstall? I don't know if it would be significantly faster or not,  
but it would probably be worth testing. I might eventually have time  
to investigate this (but probably not for the next week or so).


--
Daniel J. Luke
++
| * [EMAIL PROTECTED] * |
| *-- http://www.geeklair.net -* |
++
|   Opinions expressed are mine and do not necessarily   |
|  reflect the opinions of my employer.  |
++





PGP.sig
Description: This is a digitally signed message part
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Jordan K. Hubbard

On Jun 20, 2008, at 2:07 PM, Bryan Blackburn wrote:

 While what I've written does have some of it early lineage from your
 buildall.sh script, it's definitely changed a bit.  The good news is
 that of the 150+ or so ports I had MPAB try to build here, a vast
 majority of them built successfully.  The failures were all
 explainable at the MacPorts level, not MPAB (eg missing distfiles, a
 perl module needing 'port -f install', and so on).

That's excellent - it sounds like you've definitely made much more  
progress!

 MPAB currently builds the list of ports to attempt by sorting them by
 dependency: the early ones it builds are dependencies of later ones.
 The reason for this is that if Y depends on X and X failed, it'll
 simply skip Y.  Also, of course, if X did build successfully, all
 other ports depending on it will simply cause X do install from the
 portarchive instead of rebuilding again.  If you extract the MPAB
 tarball, the chroot-scripts/genportlist.tcl is the script which builds
 this initial list.

Does it also support picking up where it left off, or is it an  
assumption that once you've generated this initial list of ports,  
you're committed to trying to build the whole thing from beginning to  
end?   The latter scenario is fine, I'm just wondering if there's  
currently any bookkeeping associated with which ports on that list  
have been tried and which still need to be built.  If there were, it  
would obviously make it easier for volunteer builders to start and  
stop batch builds in order to take advantage of slack time on  
borrowed hardware at ${theirInstitution} and
we could also discuss the notion of jobbing out builds to remote  
builders as the next logical enhancement.

I know, I know, I want everything. :-)

 It's a chroot, so other than the initial building of the chroot, the
 host machine's files should be left alone.  Of course, the bad thing
 is chroot needs root to run...

Well, until we figure out some way of simulating chroot with some  
future trace mode on steroids I don't see any easy way around this,  
so I'm just happy you're at least using a chroot! :)

 The biggest issue here is breaking out of the chroot, otherwise if
 something malicious happens then all future build attempts would most
 likely fail quite spectacularly.

It's hard enough to break out of a chroot that I'm pretty comfortable  
with the level of security this represents.  It's the trace-mode-on- 
steroids approach that's a bit more difficult to get right, and I  
wasn't sure if you had gone that route or not.

Where does one check out the current state of MPAB goodness so far  
again?  I must have missed that somewhere in this discussion chain.   
I've got some unused hardware I wouldn't mind throwing at testing this  
for awhile...

Thanks!

- Jordan

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Bryan Blackburn
On Jun 20, 2008, at 3:13 PM, Daniel J. Luke wrote:

...


 Have you tried using activate/deactivate instead of archives/ 
 uninstall? I don't know if it would be significantly faster or not,  
 but it would probably be worth testing. I might eventually have time  
 to investigate this (but probably not for the next week or so).


Hmm, that's an interesting thought; since activate only creates hard  
links it'll definitely be faster than extracting the archive followed  
by creating those hard links.  While inactive, ports shouldn't be  
looking in /opt/local/var/macports/software for stuff, so should be  
safe.  Thanks for the idea.

Bryan


 --
 Daniel J. Luke
 ++
 | * [EMAIL PROTECTED] * |
 | *-- http://www.geeklair.net -* |
 ++
 |   Opinions expressed are mine and do not necessarily   |
 |  reflect the opinions of my employer.  |
 ++


___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Bryan Blackburn
On Jun 20, 2008, at 4:50 PM, Jordan K. Hubbard wrote:

...

 Does it also support picking up where it left off, or is it an
 assumption that once you've generated this initial list of ports,
 you're committed to trying to build the whole thing from beginning to
 end?   The latter scenario is fine, I'm just wondering if there's
 currently any bookkeeping associated with which ports on that list
 have been tried and which still need to be built.  If there were, it
 would obviously make it easier for volunteer builders to start and
 stop batch builds in order to take advantage of slack time on
 borrowed hardware at ${theirInstitution} and
 we could also discuss the notion of jobbing out builds to remote
 builders as the next logical enhancement.

 I know, I know, I want everything. :-)


Yeah, don't we all?  What it does is check to see if there's already a  
built package (from using archive mode) for the version/revision it's  
about to build; if there is, it doesn't bother trying to build again.   
If not (eg, it failed before, or never tried before) then it obviously  
tries to build.  And it does use trap to allow Ctrl-C, moving the  
pertinent logs out of the chroot and doing cleanup before stopping.   
That's how I was able to build all those ports initially on a MBP  
while still using said machine.

...

 Where does one check out the current state of MPAB goodness so far
 again?  I must have missed that somewhere in this discussion chain.
 I've got some unused hardware I wouldn't mind throwing at testing this
 for awhile...


Some basic stuff at http://trac.macports.org/wiki/MPAB, the readme  
in the tarball has more info currently than that wiki page.

Bryan


 Thanks!

 - Jordan

___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev


Re: MacPorts AutoBuild

2008-06-20 Thread Joshua Root
Bryan Blackburn wrote:
 On Jun 20, 2008, at 3:13 PM, Daniel J. Luke wrote:
 
 ...

 Have you tried using activate/deactivate instead of archives/ 
 uninstall? I don't know if it would be significantly faster or not,  
 but it would probably be worth testing. I might eventually have time  
 to investigate this (but probably not for the next week or so).

 
 Hmm, that's an interesting thought; since activate only creates hard  
 links it'll definitely be faster than extracting the archive followed  
 by creating those hard links.  While inactive, ports shouldn't be  
 looking in /opt/local/var/macports/software for stuff, so should be  
 safe.  Thanks for the idea.

I assume you're using trunk, so it should be fine, but I just thought 
I'd mention that it won't work with 1.6.0 because of #7361: 
http://trac.macports.org/ticket/7361.

- Josh
___
macports-dev mailing list
macports-dev@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev