On Mon, Oct 13, 2003 at 02:30:32PM -0700, Ben Escoto wrote: > On Mon, 13 Oct 2003 15:43:35 -0400 > "Jason M. Felice" <[EMAIL PROTECTED]> wrote: > > dirvish looks interesting. One of the requirements that I now realize I > > didn't write into the proposal was the ability to store only a single > > copy of duplicate files... duplicate as determined by file contents, not > > naming or inode or anything. (Rationale: will likely need to back up > > large numbers of clients with lots of files in common.)
A re-read of this paragraph highlighted the multiple-client issue which dirvish does address rather well through branches. Mail me off-list if you need specifics of how to set that up. The more general problem of coalescing duplicate files is a real performance pig. To do that what must be done is to hash(checksum) every single file, store the files based on that hash and then use a database to associate the paths with the files with a separate handling of hardlinks. Then while doing all of that you somehow want to be able to use the rsync algorithm between versions of the same file. Doing that is far outside the scope of rsync. > > Check out Craig Barrett's BackupPC at > > http://backuppc.sourceforge.net/ > > It has this feature and I have heard good things about it. And since > I'm posting this I might as well mention my project at > http://rdiff-backup.stanford.edu. It does versions and uses the rsync > algorithm, but doesn't notice duplicate files. I generally don't wish to appear to be pitching my own project (dirvish) which is why i don't often provide a link (google does that for me) and generally mention it in the context of other tools. But the other tools list is starting to get a bit long as there are so many permutations of the wheel. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html