Thanks Chris. I will give it a shot and see if I can make it behave in any
way... was hoping for a bit of a magic bullet, I suppose :)
On Tue, Nov 24, 2009 at 10:41 PM, Chris Bennett <ch...@ceegeebee.com> wrote:
> Hi,
>
> > Thanks for the reply. The data is, in fact, "all time" in the sense that
> it
> > goes back years, but it's sorted by filename, rather than date; it's
> > essentially equivalent to how BackupPC stores data in cpool/, i.e. the
> first
> > 3 characters of the filename will generate 3 levels of subdirectories.
> The
> > best I was able to do, to date, was to make 10 shares, 1-9, and back up
> 10
> > separate backup trees. But that was before, when I had about 100k
> files... I
> > tried this recently, and seem to have made it go under. So I guess I'd
> need
> > to make TWO levels of shares, so 1/0-1/9, 2/0-2/9, etc. Then, maybe, once
> I
> > go through the full loop, it'll be easier to perform future incrementals
> > since the delta will be small.
>
> Yeah, I've been able to archive large pools of files that have aged,
> so that backuppc doesn't have to consider such a large filelist. I'm
> not too sure on the mechanics of backuppc and overhead - e.g. what
> amount of work does backuppc perform to perform a full and
> incremental.. how much memory is consumed per considered file. I
> expect someone else can more succintly answer these kind of questions
> to help you build a more scalable configuration.
>
> > My BackupPC box doesn't swap too much, it doesn't behave like it's under
> > massive load at all; but then again, I think my IO subsystem (Dell Perc6
> +
> > 4x WD Greens in RAID5) hopefully outperforms the speed of the link+any
> > overhead :) I haven't tried stracing rsync on the remote server. Any
> > suggestions on how to use it? I've never tried it before.
>
> Get the pid of your rsync process on the source of data.
>
> Then perform something like
> # -s3000 specified 3000 characters printed per system call
> strace -p <pid> -s3000
>
> This will give you insight into the open/stat/read/close cycle that
> rsync will be doing when copying data. I would expect it to be
> cycling faster than you can read, although in the case where I've seen
> high swap activity, you'll see batches of the cycle followed by
> pauses.
>
> Similarly, running:
> vmstat 1
>
> in another console and looking at the bi/bo columsn that represent
> blocks in/out helps you to know whether swap is being heavily used.
>
> Good luck and let me know if you find a good solution to your problem.
>
> Regards,
>
> Chris Bennett
> cgb
>
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
