Brooks Davis skrev:
On Wed, Jan 30, 2008 at 07:20:23PM +0100, Erik Cederstrand wrote:
I'd like a situation where I can very quickly set up a slave with a
specific version of FreeBSD to run additional tests or provide shell access
to a developer. This currently involves adding an entry to a queue,
rebooting and waiting 2 minutes. Quick and easy, but the archiving strategy
is obviously very inefficient.
I'm thinking of a couple of options:
1. Having one full install per month and archiving the rest as diffs
against that by recursively bsdiff'ing every file in the tree (I
could bsdiff a whole tarball, but bsdiff is very memory-intensive).
Quick test: 25 mins.
2. Make a hash of all files and only store the binaries where the hash
is different from the monthly tarball. Faster than 1., but less
effective. Quick test: 5 mins.
3. Use some kind of VCS. My experience with Subversion and binary files
is that it's very slow.
4. Throw hardware at the problem.
I'd say it should not take more than 10 mins to recreate an archived
version. Any thoughts?
It seems like you should be able to combine 1 and 2 with checksums to
decide if you need to run diffs. I'd think that would be quite fast.
I finally got around to testing this, and with a combination of mtree
comparing md5 hashes, bsdiff compacting changed files and hardlinking
unchanged files I get a reduction in size from 256MB to 10MB. Pretty
good, and the whole operation only takes a few minutes.
I have one peculiarity, though. I install python2.5 into the directory
containing the build, and even though the python version has not
changed, I still get mismatching md5 sums on every .pyo and .pyc file.
Any thoughts on this?
Erik
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
