With the latest commit to efs-core, I've closed this gap. I'm running the complete test suite using an /efs/test tree I exported using efs...@madefsd01:~/tmp/export_efs_test. Basically post-processes the pure export to reduce the size of the final tarball, which is still 55MB.
This means we can start building infrastructure we want to test with in boot.efs first, and then export it to the efs-core test environment. When the tests pass, we'll be deploying with the exact same dependencies. The key here is that everything works from the same efsdeploy.conf file, with the same specific set of dependencies. More to follow.... On Mon, Oct 11, 2010 at 11:31 AM, Phillip Moore <[email protected]>wrote: > We have a huge gap between how we test EFS, and how we deploy it, and I've > been trying to figure out how we close it. Here's the problem.... > > When we deploy EFS, we build it into a real EFS release in efs/core, using > efsdeploy. This is done by creating a real distribution out of the source > tree, and then installing it pretty much like any other CPAN module. This > works very well, and we are already running efsd out of /efs/dist in the > boot.efs environment. Getting to the point where that was automated was a > lot of work. The various dependencies are satisfied by the > EFS::Perl::Depends mechanism, and the metadata in the lib/efs/metadata.conf > file. > > When we test EFS, however, we obviously don't have any of the EFS runtime > environment, so we have to satisfy all of the necessary dependencies by some > other means. We currently do this by installing local rpms for the various > C library dependencies, and then building perl and all of the CPAN modules > we need into /usr/efs. This works fairly well, but these two environments > are NEVER in sync. > > We can use cpan2efs to build and control our CPAN dependencies in the > runtime/deployment environment, but when we build the test environment, we > can either do so from the "head" of CPAN (whatever's current when you build > things), or from a fixed minicpan snapshot. The result is that these two > environments play leapfrog and are never the same. Given out sensitivity > to changes in out CPAN dependencies, this is a very real problem. We test > with one set of dependencies, and then deploy with an entire different set, > which greatly reduces our confidence level that the deployment will in fact > work. > > I started working on some tools to bridge this gap. First, I wrote > efsdeploy_export_tree, which let's me create an exported /efs/test tree > which is the subset of /efs/dist that represents the efs/core dependencies, > but converted to work out of /efs/test. The goal is to be able to create a > test environment with IDENTICAL dependencies as the runtime environment you > would then use to deploy after a successful test run. > > I can create the /efs/test tree fine. The problem is using it in the test > suite. Normally, we have a single perl installation with the all the > dependencies bundled, so perl just finds everything. Now, we have to figure > out how to set EFS_PERL_DEPENDS_RUNTIME so that the test suite can locate > everything, and this is where the toolkit is incomplete. This comes back > to how we are managing (or not) the efsdeploy.conf metadata. Those files > are domain specific, since they list the specific MPRs that are in the local > domain, but the general rules they contain are specific to the product not > the domain. > > I have another tool, efsdeploy_depends that really just extracts the > dependencies from efsdeploy.conf, and you can use it this way: > > EFS_PERL_DEPENDS_RUNTIME=$( efsdeploy_depends --depends > perl_runtime,perl_test,perl_build ) > > I might be able to export efsdeploy into the /efs/test tree, and then use > this script from there, but the efsdeploy tools have been designed to work > with real EFS releases, and they don't work outside of /efs/dev/.../src. > IOW, as implemented, they don't work well in an arbitrary source tree. > After all, efsdeploy is designed to automate integration of pre-packaged > software products, and it was not designed to be a development tool. That > may have to change, but it won't be easy. > > Testing EFS is a little special, of course. Almost everything else can be > tested in the real EFS environment, but obviously not EFS itself. This may > call for some very special tools, since the problems EFS faces may simply > not be generic enough to justify making them general purpose. > > This is the kind of complex problem I could really use some help on, and > this is why I need the rest of the team to catch up and learn to use the > various tools I've written, because until you understand the building > blocks, you'll never get a handle on the bigger picture issues we face. > > > >
_______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
