experimenting with build layout

2013-09-07 Thread Marc Espie
got some new diskspace, so I decided to experiment with new layouts for
"full" builds, src + x + ports.

The idea being I'm a bit fed up with wiping the machine and rebuilding
packages from scratch, so I'm going to build some stuff in a chroot...

Sharing this, because I think that would be interesting...


/build for chrooting for build
 obj tree rooted at /build/obj
/build/xobj
/build/pobj
/build/source (4G, default layout /usr/src + /usr/obj) for holding all 
3 source trees:
 /build/source/src
 /build/source/xenocara
 /build/source/ports

/build/data for more permanent junk, such as distfiles, built packages, logs.

can NFS export /build/source and /build/data to other machines set up in a
similar way.

Note that /build itself is async, noatime (I don't care at all if I lose
it), /build/source and /build/data will probably be softdep,noatime

I will get a tmpfs /build/tmp  whenever we manage to get that working :(

and that makes 3 partitions for everything, kind of keep things segregated
away.

Build plan looks like:
- wipe build
- normal build for src
- normal build for X
- reinstall src and X with  DESTDIR=/build
(can even bake release and do things that way)

- do a chroot to /build. Run dpb in there.

update packages from new stuff built after it's finished building.

of course, kernel, src, X builds will disrupt normal functioning a bit,
but we're talking an hour or two vs days for full bulks.

Just food for thought. Maybe stuff to improve some more.



Re: experimenting with build layout

2013-09-07 Thread Marc Espie
On Sat, Sep 07, 2013 at 10:45:35AM +0200, Marc Espie wrote:
> got some new diskspace, so I decided to experiment with new layouts for
> "full" builds, src + x + ports.
> 
> The idea being I'm a bit fed up with wiping the machine and rebuilding
> packages from scratch, so I'm going to build some stuff in a chroot...
> 
> Sharing this, because I think that would be interesting...
> 
> 
> /build for chrooting for build
>  obj tree rooted at /build/obj
>   /build/xobj
> /build/pobj
> /build/source (4G, default layout /usr/src + /usr/obj) for holding all 
> 3 source trees:
>  /build/source/src
>  /build/source/xenocara
>  /build/source/ports
> 
> /build/data for more permanent junk, such as distfiles, built packages, logs.
> 
> can NFS export /build/source and /build/data to other machines set up in a
> similar way.
> 
> Note that /build itself is async, noatime (I don't care at all if I lose
> it), /build/source and /build/data will probably be softdep,noatime
> 
> I will get a tmpfs /build/tmp  whenever we manage to get that working :(
> 
> and that makes 3 partitions for everything, kind of keep things segregated
> away.
> 
> Build plan looks like:
> - wipe build
> - normal build for src
> - normal build for X
> - reinstall src and X with  DESTDIR=/build
> (can even bake release and do things that way)
> - do a chroot to /build. Run dpb in there.
> 
> update packages from new stuff built after it's finished building.
> 
> of course, kernel, src, X builds will disrupt normal functioning a bit,
> but we're talking an hour or two vs days for full bulks.

For the chroot: don't forget to allow devs and suid in the partition

initial src install with cd src/etc && make distribution
don't forget to create user, edit sudo. Setup localtime. run ldconfig.
run MAKEDEV.
likewise for X (release-clean + release-install) does it.

Don't forget dpb reads config on the first machine, AND expects everything
to be set up in a similar way on the other side. e.g., you want to
have your chroot to appear in the correct location on a distant machine.
(haven't delved into setting up proper chroot on the distant machine
and teaching dpb that just yet).



Re: experimenting with build layout

2013-09-10 Thread Marc Espie
On Sat, Sep 07, 2013 at 04:25:53PM +0200, Marc Espie wrote:
> On Sat, Sep 07, 2013 at 10:45:35AM +0200, Marc Espie wrote:
> > got some new diskspace, so I decided to experiment with new layouts for
> > "full" builds, src + x + ports.
> > 
> > The idea being I'm a bit fed up with wiping the machine and rebuilding
> > packages from scratch, so I'm going to build some stuff in a chroot...
> > 
> > Sharing this, because I think that would be interesting...
> > 
> > 
> > /build for chrooting for build
> >  obj tree rooted at /build/obj
> > /build/xobj
> > /build/pobj
> > /build/source (4G, default layout /usr/src + /usr/obj) for holding all 
> > 3 source trees:
> >  /build/source/src
> >  /build/source/xenocara
> >  /build/source/ports
> > 
> > /build/data for more permanent junk, such as distfiles, built packages, 
> > logs.
> > 
> > can NFS export /build/source and /build/data to other machines set up in a
> > similar way.
> > 
> > Note that /build itself is async, noatime (I don't care at all if I lose
> > it), /build/source and /build/data will probably be softdep,noatime
> > 
> > I will get a tmpfs /build/tmp  whenever we manage to get that working :(
> > 
> > and that makes 3 partitions for everything, kind of keep things segregated
> > away.
> > 
> > Build plan looks like:
> > - wipe build
> > - normal build for src
> > - normal build for X
> > - reinstall src and X with  DESTDIR=/build
> > (can even bake release and do things that way)
> > - do a chroot to /build. Run dpb in there.
> > 
> > update packages from new stuff built after it's finished building.
> > 
> > of course, kernel, src, X builds will disrupt normal functioning a bit,
> > but we're talking an hour or two vs days for full bulks.
> 
> For the chroot: don't forget to allow devs and suid in the partition
> 
> initial src install with cd src/etc && make distribution
> don't forget to create user, edit sudo. Setup localtime. run ldconfig.
> run MAKEDEV.
> likewise for X (release-clean + release-install) does it.
> 
> Don't forget dpb reads config on the first machine, AND expects everything
> to be set up in a similar way on the other side. e.g., you want to
> have your chroot to appear in the correct location on a distant machine.
> (haven't delved into setting up proper chroot on the distant machine
> and teaching dpb that just yet).

Okay, dpb knows how to do chroot on the distant machine.

So my setup is fairly consistant.
Local partition /build (currently async, WITH dev and suid)
NFS mount for /build/source
NFS mount for /build/xenocara
NFS mount for /build/ports

building source without chroot: first kernel build on one box,
then install on both boxes, then reboot

objdir as /build/obj
after make build, initial chroot population by:
- cd /base/source/etc && sudo make distribution DESTDIR=/build
- cd /build/dev && sudo ./MAKEDEV all
- chroot /build
from within: 
adduser that will do the build, 
set up sshkey for that user
edit sudoers,
create proper localtime link
create pkg.conf and mk.conf with correct setup
copy resolv.conf over !
ldconfig /usr/local/lib /usr/X11R6/lib

mk.conf:
SUDO=sudo -E
BUILD=
BSDSRCDIR=${BUILD}/source/src
XSRCDIR=${BUILD}/source/xenocara
PORTSDIR=${BUILD}/source/ports
BSDOBJDIR=${BUILD}/obj
XOBJDIR=${BUILD}/xobj
WRKOBJDIR=${BUILD}/pobj

pkg.conf:
nochecksum  = yes

(done on both boxes, make sure user matches)

Further builds can be normal builds, followed by make install DESTDIR=/build

make sure dpb user has sufficient limits on both boxes (outside the chroot).


for xenocara:
normal build
then
make release-instal DESTDIR=/build
(done on both boxes)


dpb builds: create a hosts file under the chroot with
distant chroot=/build
localhost

go chroot:
chroot -u joe /build
exec ssh-agent sh
ssh-add
dpb -h hosts



this is still half-manual, there are probably a few steps to be automated.
Note the src/xenocara is a normal "build from source", there's just an
extra step to populate the chroot (it could be wiped each build actually).
A similar setup can be done with snapshot installs...



Re: experimenting with build layout

2013-09-21 Thread Marc Espie
Things work just fine.
Got an improvement though...
On Tue, Sep 10, 2013 at 07:16:22PM +0200, Marc Espie wrote:
> > > - normal build for X
> > > - reinstall src and X with  DESTDIR=/build

normal build for X on *one* host.

Use rsync to synch the distant hosts, and the chroots.
(just don't forget to ldconfig -R appropriately).

*much* faster, impressively so in fact.

this avoids some nasty half-rebuild issues that take time when you install
X twice... :)