On 05.12.2013 22:46, Chuck Munro wrote: >> On 04.12.2013 14:05, John Doe wrote: >>>> From: Lists<li...@benjamindsmith.com> >>>> >>>>>> Our next big test is to try out ZFS filesystem send/receive in >>>>>> lieu >>>>>> of >>>>>> our current backup processes based on rsync. Rsync is a fabulous >>>>>> tool, >>>>>> but is beginning to show performance/scalability issues dealing >>>>>> with >>>>>> the >>>>>> many millions of files being backed up, and we're hoping that ZFS >>>>>> filesystem replication solves this. >>>> >>>> Not sure if I already mentioned it but maybe have a look at: >>>> ?http://code.google.com/p/lsyncd/ >> I'm not so sure inotify works well with millions of files, not to >> mention it uses rsync. :D >> >> -- Sent from the Delta quadrant using Borg technology! Nux! > > I can attest to the usefulness of 'lsyncd' for large numbers of files > (our file server has almost 2 million in active use, with a second > backup server that's lsync'd to the first. > > Things to note: > - Yes, lsyncd does use rsync, but it issues an 'exclude *' followed by > the list of only the file(s) that need updating at that moment. > > - The inotify service can be jacked waaaay up (three kernel > parameters) > to handle millions of files if you wish. Just make sure you have lots > of RAM.
Be careful with it. Sadly I found out that inotify would consistently fail on InnoDB files (ibd); I had to use stupid while loops and check mtimes to perform some stuff that inotify-cron would've done much more elegantly ... -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos