On Sat, May 26, 2007 at 01:59:38PM -0700, Petcher, Daniel wrote: > > I *REALLY* need to buy and mount more disk space, but I don't have the > budget or time for that project this quarter.
That is indeed what you need to do. Plan on purchasing 3x space times what you are backing up. Any less and you are filling the backup disks too often. > Surely I can't be the first person to have wanted this functionality. Does > anyone else out there have a suggestion to simplify my life? Management of what goes INTO the backups is key. This involves both changing how the client machines are configured, what gets backed up, and how dirvish is set up. It is a little late now, but when you set up the dirvish archives, you can use branches to merge similar machines into sorta-kinda the same vaults using branches. For example, my wife and I both have T30 laptops with RHEL5 . Those, and a couple of other RHEL5 machines, are branches off the same basic "RHEL5boot" and "RHEL5root" vaults. This saves some space for all the common stuff. Also a little late, but build your backup drives with small blocks (2K?) and lots of inodes. You are storing a huge number of small directory files every day, typically a kbyte or so, and those can chew up 16K block ext3 partitions quickly. Avoid making big files that have small daily changes - Do NOT use dirvish for giant files that change often, but only a little. Mail on my systems is stored in "maildir" format, rather than "mbox" format. Now the addition of a single mail does not make a big new file in the backups. Another space saver is to use "datext" style logrotate ( originally in SuSE, recently in Red Hat). This makes log files use date extensions instead of rotating through .1, .2, etc. Change logs to daily instead of weekly on big logs. Now you are not storing a new version of a big log file every day. Yet another space saver is to exclude big files like database files from the dirvish backups, and use some other method. Database files usually have some kind of dedicated backup technique anyway. If you run something like VMWare, with big virtual disks, those are another classic "little changes often" file. If you can, arrange your VMWare virtual disks to be as small as possible, and use internal samba shares (stored as linux files) for everything possible. The virtual disks will still change, but slowly - exclude them from the filesystem vault and store them some other way. Are you storing a whole lot of downloaded ISO files to make CDs and DVDs from? There is little point in letting dirvish and rsync manage these, if you are short of disk space. Just make some extra CDs and DVDs. When you have made other arrangements for 10 or so big files, and stop backing up other big files that are stored elsewhere, you will find the remaining moderate sized files store very nicely. You can point to big files from a client directory with soft links, and that directory can be the target of a special big-file vault. Use aggressive expires. This may be a good application of something like rdiffbackup. This is a PITA for restores, but it is only a few files, which if necessary you can move and check by hand. The bottom line is that you need enough storage to do this, and rsync, the core of dirvish, doesn't do everything well. For the vast bulk of your files, it is magic. For giant files, all it can do is make new daily copies of them, chewing up disk space rapidly. I hope that helps ... Keith -- Keith Lofstrom [EMAIL PROTECTED] Voice (503)-520-1993 KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon" Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs _______________________________________________ Dirvish mailing list [email protected] http://www.dirvish.org/mailman/listinfo/dirvish
