experimenting with build layout
got some new diskspace, so I decided to experiment with new layouts for "full" builds, src + x + ports. The idea being I'm a bit fed up with wiping the machine and rebuilding packages from scratch, so I'm going to build some stuff in a chroot... Sharing this, because I think that would be interesting... /build for chrooting for build obj tree rooted at /build/obj /build/xobj /build/pobj /build/source (4G, default layout /usr/src + /usr/obj) for holding all 3 source trees: /build/source/src /build/source/xenocara /build/source/ports /build/data for more permanent junk, such as distfiles, built packages, logs. can NFS export /build/source and /build/data to other machines set up in a similar way. Note that /build itself is async, noatime (I don't care at all if I lose it), /build/source and /build/data will probably be softdep,noatime I will get a tmpfs /build/tmp whenever we manage to get that working :( and that makes 3 partitions for everything, kind of keep things segregated away. Build plan looks like: - wipe build - normal build for src - normal build for X - reinstall src and X with DESTDIR=/build (can even bake release and do things that way) - do a chroot to /build. Run dpb in there. update packages from new stuff built after it's finished building. of course, kernel, src, X builds will disrupt normal functioning a bit, but we're talking an hour or two vs days for full bulks. Just food for thought. Maybe stuff to improve some more.
Re: experimenting with build layout
On Sat, Sep 07, 2013 at 10:45:35AM +0200, Marc Espie wrote: > got some new diskspace, so I decided to experiment with new layouts for > "full" builds, src + x + ports. > > The idea being I'm a bit fed up with wiping the machine and rebuilding > packages from scratch, so I'm going to build some stuff in a chroot... > > Sharing this, because I think that would be interesting... > > > /build for chrooting for build > obj tree rooted at /build/obj > /build/xobj > /build/pobj > /build/source (4G, default layout /usr/src + /usr/obj) for holding all > 3 source trees: > /build/source/src > /build/source/xenocara > /build/source/ports > > /build/data for more permanent junk, such as distfiles, built packages, logs. > > can NFS export /build/source and /build/data to other machines set up in a > similar way. > > Note that /build itself is async, noatime (I don't care at all if I lose > it), /build/source and /build/data will probably be softdep,noatime > > I will get a tmpfs /build/tmp whenever we manage to get that working :( > > and that makes 3 partitions for everything, kind of keep things segregated > away. > > Build plan looks like: > - wipe build > - normal build for src > - normal build for X > - reinstall src and X with DESTDIR=/build > (can even bake release and do things that way) > - do a chroot to /build. Run dpb in there. > > update packages from new stuff built after it's finished building. > > of course, kernel, src, X builds will disrupt normal functioning a bit, > but we're talking an hour or two vs days for full bulks. For the chroot: don't forget to allow devs and suid in the partition initial src install with cd src/etc && make distribution don't forget to create user, edit sudo. Setup localtime. run ldconfig. run MAKEDEV. likewise for X (release-clean + release-install) does it. Don't forget dpb reads config on the first machine, AND expects everything to be set up in a similar way on the other side. e.g., you want to have your chroot to appear in the correct location on a distant machine. (haven't delved into setting up proper chroot on the distant machine and teaching dpb that just yet).
Re: experimenting with build layout
On Sat, Sep 07, 2013 at 04:25:53PM +0200, Marc Espie wrote: > On Sat, Sep 07, 2013 at 10:45:35AM +0200, Marc Espie wrote: > > got some new diskspace, so I decided to experiment with new layouts for > > "full" builds, src + x + ports. > > > > The idea being I'm a bit fed up with wiping the machine and rebuilding > > packages from scratch, so I'm going to build some stuff in a chroot... > > > > Sharing this, because I think that would be interesting... > > > > > > /build for chrooting for build > > obj tree rooted at /build/obj > > /build/xobj > > /build/pobj > > /build/source (4G, default layout /usr/src + /usr/obj) for holding all > > 3 source trees: > > /build/source/src > > /build/source/xenocara > > /build/source/ports > > > > /build/data for more permanent junk, such as distfiles, built packages, > > logs. > > > > can NFS export /build/source and /build/data to other machines set up in a > > similar way. > > > > Note that /build itself is async, noatime (I don't care at all if I lose > > it), /build/source and /build/data will probably be softdep,noatime > > > > I will get a tmpfs /build/tmp whenever we manage to get that working :( > > > > and that makes 3 partitions for everything, kind of keep things segregated > > away. > > > > Build plan looks like: > > - wipe build > > - normal build for src > > - normal build for X > > - reinstall src and X with DESTDIR=/build > > (can even bake release and do things that way) > > - do a chroot to /build. Run dpb in there. > > > > update packages from new stuff built after it's finished building. > > > > of course, kernel, src, X builds will disrupt normal functioning a bit, > > but we're talking an hour or two vs days for full bulks. > > For the chroot: don't forget to allow devs and suid in the partition > > initial src install with cd src/etc && make distribution > don't forget to create user, edit sudo. Setup localtime. run ldconfig. > run MAKEDEV. > likewise for X (release-clean + release-install) does it. > > Don't forget dpb reads config on the first machine, AND expects everything > to be set up in a similar way on the other side. e.g., you want to > have your chroot to appear in the correct location on a distant machine. > (haven't delved into setting up proper chroot on the distant machine > and teaching dpb that just yet). Okay, dpb knows how to do chroot on the distant machine. So my setup is fairly consistant. Local partition /build (currently async, WITH dev and suid) NFS mount for /build/source NFS mount for /build/xenocara NFS mount for /build/ports building source without chroot: first kernel build on one box, then install on both boxes, then reboot objdir as /build/obj after make build, initial chroot population by: - cd /base/source/etc && sudo make distribution DESTDIR=/build - cd /build/dev && sudo ./MAKEDEV all - chroot /build from within: adduser that will do the build, set up sshkey for that user edit sudoers, create proper localtime link create pkg.conf and mk.conf with correct setup copy resolv.conf over ! ldconfig /usr/local/lib /usr/X11R6/lib mk.conf: SUDO=sudo -E BUILD= BSDSRCDIR=${BUILD}/source/src XSRCDIR=${BUILD}/source/xenocara PORTSDIR=${BUILD}/source/ports BSDOBJDIR=${BUILD}/obj XOBJDIR=${BUILD}/xobj WRKOBJDIR=${BUILD}/pobj pkg.conf: nochecksum = yes (done on both boxes, make sure user matches) Further builds can be normal builds, followed by make install DESTDIR=/build make sure dpb user has sufficient limits on both boxes (outside the chroot). for xenocara: normal build then make release-instal DESTDIR=/build (done on both boxes) dpb builds: create a hosts file under the chroot with distant chroot=/build localhost go chroot: chroot -u joe /build exec ssh-agent sh ssh-add dpb -h hosts this is still half-manual, there are probably a few steps to be automated. Note the src/xenocara is a normal "build from source", there's just an extra step to populate the chroot (it could be wiped each build actually). A similar setup can be done with snapshot installs...
Re: experimenting with build layout
Things work just fine. Got an improvement though... On Tue, Sep 10, 2013 at 07:16:22PM +0200, Marc Espie wrote: > > > - normal build for X > > > - reinstall src and X with DESTDIR=/build normal build for X on *one* host. Use rsync to synch the distant hosts, and the chroots. (just don't forget to ldconfig -R appropriately). *much* faster, impressively so in fact. this avoids some nasty half-rebuild issues that take time when you install X twice... :)