This entire exercise has suggested that we're doing something very inefficient with the way we build these VMs.
The original goal of efs_virtual_machine was to automate the entire setup of these machines, assuming that the starting point is a VM "template" that is a purely vanilla installation of the distribution. The first thing the script does is install a bunch of packages, of course. I'm pretty sure none of you have rebuilt things anywhere close to the number of times I have. What I've found is that the slowest part of the process is the software installation. Once that's done, all the OS configuration is pretty quick. Now that I've added OpenAFS to the mix, it's a LOT more complicated, because OpenAFS packages are not part of the default network of "yum" servers for CentOS, and you need to install packages that are specific to the kernel patch level of the local machine. This is really tedious, and really slow. I think it makes sense to factor out the package installation, and do that ONCE, per VM template. If we want to upgrade the OpenAFS kernel version, then we can upgrade the templates first. So... I'm going to rebuild my VM templates today, and take notes on HOW I did it (we've always left that as an exercise for the developer, and it's not trivial). This will deal with another nit that's been annoying me. When you try to tun efs_virtual_machine on a couple of the platforms, you have to first install a couple of packages anyway (perl on Debian, IIRC, and a few things on FreeBSD). I think it makes sense to create VM templates with minimally installed software, but my point is that the way we do it now is *too* minimal. My notes are going to be specific to how I do this in VMWare Fusion on MacOS X, and while the OS install part will be generic, the VMware part won't. I'll leave it up to others to patch those notes with instructions specific to the other VMware products. The big win here will be that we can deal with the OpenAFS stuff ONCE, and then forget about it, until we need to upgrade. Then, we'll be upgrading the templates, once per platform. On Fri, Oct 8, 2010 at 12:39 PM, Phillip Moore <[email protected]>wrote: > After I got the test.efs setup done with 1.4.12, I immediately wanted to > rebuild it all with 1.5.77, since there are some key features in the 1.5.* > code that EFS is going to leverage. In particular, I plan to require the > newer "fs mkmount" syntax that let's me avoid creating temporary volume > mounts (man, do I wish I had that back when I wrote AFS/VMS...) > > This has been a LOT of work, and not everything's automated yet, but here's > what I've accomplished. In Jerry's boot.efs environment, I built > openafs/core/1.5.77-build001, and managed to get a complete build of the > user-mode binaries, but I built with --disable-kernel-module for that > release. The goal is to use openafs/core/* releases for the servers I'll be > building, but that comes later (I have a really sick idea for leveraging > /efs/boot, but as usual, I digress...) > > In order to build the test.efs setup, I needed a set of 1.5.77 rpms. > Building the rpms is a little tricky. First, you have to build the srpm, > and then you use that srpm to build the binary rpms. This is partially > automated in openafs/rpms/1.5.77-build001, but I quickly learned that you > have to build a kmod-openafs-* binary rpm for EVERY SINGLE kernel patch > level. Simon Wilkerson wrote a script that mass-produces these, but I > haven't figure out how it works yet. The stuff in the install tree in > openafs/rpms is what was built automatically, but it only includes the > kmod-openafs rpm for the kernel version that is used on the compile servers. > > As soon as I tried to use that kernel rpm on MY CentOS 5.4 machines, I > discovered that Jerry's setup runs: > > 2.6.18-164.el5 > > but mine are running: > > 2.6.18-164.15.1.el5 > > This required building another kmod-openafs rpm for that specific kernel. > What that means is that the simple approach I took for openafs/rpms will not > work. This is why Simon wrote the script he did, and I will need to figure > out how to setup the build environment, which mostly involves collecting all > the kernel-devel-* rpms needed to populate /usr/src/kernels with the right > directories. I'll know more when I dig into it. > > In order to make the test.efs environment reproducible, which is my > immediate goal, I've uploaded the rpms I'm working with to: > > ftp-osl.osuosl.org:~openefs/data/packages/openafs/1.5.77/$platform > > The $platform values are x86-32.rhel.5 and x86-64.rhel.5 of course. This > will be an entirely different effort for Debian/Ubuntu (although that will > probably be much easier, since Russ Allbery owns that code, and he does a > fantastic job of managing OpenAFS for at least Debian), and the other OSes, > as well of course. > > The setup process for getting test.efs configured is significantly more > complicated now. For one thing, in order to NOT require everyone else to > setup AFS, I've decoupled the setup into a separate script. For another > thing, you have to run it twice, and it costs you another pair of reboots, > too. > > There's lots of todo items for test.efs still, and I'm nailing these one by > one. I do NOT make any PAM changes at all, so authentication is still > manual, but since the test suite automated this, it's just the human users > who have to deal with it for now. > > Next, I'm going to start extending efs-core and laying the framework for > the OpenAFS support. One nice thing is that I will be able to use the same > test.efs setup to test AFS and non-AFS, so I will be able to make sure that > basic NFS support doesn't break. > > I said I'd have the test.efs done by the end of this week -- mission > accomplished. > > By the end of NEXT week, I'm hoping to know a lot more about how efs-core > has to change in order to support both. >
_______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
