Thanks
On Tue, May 25, 2010 at 1:26 PM, Benjamin Watkins <ben-l...@constant-technologies.com> wrote: > On 5/25/2010 6:41 AM, Mag Gam wrote: >> >> I know rsync can do many things but I was wondering if anyone is using >> it for data deduplication on a large filesystem. I have a filesystem >> which is about 2TB and I want to make sure I don't have the same data >> in a different place of a filesystem. Is there an algorithm for that? >> > > While rsync is not an appropriate tool for this, I have successfully used > dupseek in the past. > > http://freshmeat.net/projects/dupseek/ > > It is a perl script, so I expect you should be able to use it on any > platform you need. It show support for POSIX/Linux, but I expect it can run > under Windows as well if you are comfortable with Cygwin. > > I'm sure there are many more tools like this. I used this one because it > was optimized for large files. > > -Ben > > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
