On Mon, 2008-08-11 at 02:21 -0400, Rick Thomas wrote: > On Aug 10, 2008, at 2:12 AM, Brian Wells wrote: > > > Hi. I'm using svn-fast-backup (in the subversion-tools package) to > > make > > rsync backups of my subversion repository. The place I'm backing > > up to > > is on an ext3-formatted usb flash drive, and I'm wondering if this > > is a > > wise thing to do. > > > > I know that flash drive blocks wear out after a certain number of > > writes, and rsync uses hard links to avoid duplicate backup files. My > > question is, does this mean that the inodes' reference counts will be > > changed frequently enough that it might wear out that block of the > > flash > > drive any time soon? (I currently backup about 900 files one or two > > times a day.) If so, is ext3 capable of using another block for the > > inode, or will I lose the ability to read/write some files completely? > > > > Thanks, > > Brian > > > Here are some basic facts about USB flash drives: > > > The underlying NAND flash RAM is guaranteed for 100K writes (older > technology) or 1M (more recent). After that it's retention time > starts to decrease, sometimes to as little as a few weeks, and with > continued use eventually to complete failure. The retention time of > a new, fully-functional NAND flash RAM is hundreds of years. > > USB flash drives have controller chips in the drive that, among other > things, talk USB protocol to the host. One of the other things they > do is try to distribute the writes over the available blocks, > extending the lifetime of the device beyond the raw lifetime of the > underlying NAND flash. But the firmware in the controller usually > assumes you are using a FAT file-system. If, for example, you use > EXT3 rather than FAT, you will probably confuse it. I don't know > what the effect will be on device life, but I'm guessing it will > probably be something like a return to the behavior of the raw NAND > flash. > > Write blocks are large (4-8K or more). If you update one inode in a > block, you have to read the entire block, update the inode and write > the entire block back again. Each write to an inode causes an > effective write to all the inodes in the same block. So there's > roughly a 64x multiplier (8Kbytes/block divided by 128bytes/inode = > 64 inodes/block) in the number of writes. Thus any given single > inode is good for roughly 16K updates before it wears out. > > Let's suppose your rsync backup causes a small number of updates for > each inode it touches. Call it 10, conservatively (it very well > could be as low as 1 or 2 -- I don't know enough of the details of > how rsync works at the filesystem level). That gives you about 1600 > rsync runs before you start to wear out the inodes. At twice a day, > you can safely use your USB flash drive for 800 days, or between 2 > and 3 years. > > So here's what I'd do. Use your USB flash drive for a year, then > retire it to the long-term archives and replace it with a new one. > Let that be one of those things you do on or near your birthday, like > schedule a checkup with your Doctor. Also: dismount the flash drive, > force an fsck, and remount it, every time you do a backup. This will > read and sanity-check every inode on the drive, without doing any > writes. If the fsck shows random errors, replace the drive because > it's probably starting to wear out. > > Sound reasonable? > > > Rick > >
It does sound reasonable, with one sticking point: a lot of space in the flash drive will never be used. I should have mentioned that; despite having 900 files, the repository is currently only about 5 MB, and the drive itself is 4 GB. I'm looking at an alternative utility, svn-backup-dumps, in the same package. I could do ~800 full backups, or many more incremental backups with a full backup here and there, before running out of space. That would only write most blocks once or twice, and once it filled up I could erase old backups with only one write and reuse it, right? (If I do this, should I reformat to FAT so the controllers won't be confused?) Thanks, Brian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]