On Tue, Dec 15, 2009 at 06:27:41AM -0600, Carl Wilhelm Soderstrom wrote: > On 12/14 04:25 , Robin Lee Powell wrote: > > Not with large trees it isn't. I have 3.5 million files, and > > more than 300GiB of data, in one file system. The last > > incremental took *twenty one hours*. I have another backup > > that's 4.5 million files, also more than 300 GiB of data, also > > in one file system. The full took 20 hours; it hasn't succeeded > > at an incremental yet. That's over full 100BaseT, if not better > > (I'm not the networking person). > > Pardon me if this has been mentioned already and I missed it, but > I presume you've already gone over both ends of the connection and > checked for possible optimizations?
Just to give a sense of scale here: # date ; find /pictures -xdev -type f -printf "%h\n" >/tmp/dirs ; date Tue Dec 15 12:50:57 PST 2009 Tue Dec 15 17:26:44 PST 2009 (something I ran to try to figure out how to partition the tree) The other one isn't even close to finishing, as far as I can tell. In the face of it taking nigh-on 5 hours just to *walk the tree*, from the local host, I haven't been focusing on little things like ssh encryption choices too much. :) As a general comment to everybody: I *am* reading what you're saying, I do appreciate the help, and I'm going to try out some of the things you suggested. Thank you. -Robin -- They say: "The first AIs will be built by the military as weapons." And I'm thinking: "Does it even occur to you to try for something other than the default outcome?" See http://shrunklink.com/cdiz http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/ ------------------------------------------------------------------------------ This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/