> [...]The box is running redhat 9 with 2.4.20 kernel and ext3 filesystem. ... sendsize[27747]: time 0.156: spawning /usr/lib/amanda/runtar in pipeline
sendsize[27747]: time 11114.835: .....
sendsize[27747]: estimate time for /home level 0: 11114.679
sendsize[27747]: estimate size for /home level 0: 419847640 KB
sendsize[27747]: time 11114.835: waiting for /bin/tar "/home" child
sendsize[27747]: time 11114.835: after /bin/tar "/home" wait
sendsize[27747]: time 11114.882: getting size via gnutar for /home level 6
sendsize[27747]: time 11115.510: spawning /usr/lib/amanda/runtar in pipeline
[...]
sendsize[27747]: time 18815.342: Total bytes written: 69237104640 (64GB, 8.6MB/s)
sendsize[27747]: time 18815.372: .....
sendsize[27747]: estimate time for /home level 6: 7699.861
sendsize[27747]: estimate size for /home level 6: 67614360 KB
sendsize[27747]: time 18815.372: waiting for /bin/tar "/home" child
sendsize[27747]: time 18815.372: after /bin/tar "/home" wait
sendsize[27747]: time 18815.409: done with amname '/home', dirname '/home', spindle -1
sendsize[27717]: time 18815.493: child 27747 terminated normally
sendsize: time 18815.503: pid 27717 finish time Tue Jul 20 06:13:36 2004
That was 11000 seconds for a level 0 estimate plus 7700 seconds for a level 6. It could have been worse, when amanda also wanted a level 7
estimate to choose from.
A long time ago, I had also an "long estimates" problem, where a UFS filesystem on Solaris 2.6 . It was a filesystem with many small files. GNutar really takes a long time on that. Some larger filesystems on the same host estimated much quicker, but those had much less files (but larger ones).
I had the opportunity to migrate the troublesome filesystem to another host. I just tried ext3 on Linux, which was faster (maybe because the host also was faster), but not very much. Then I tried Reiserfs. The time went down incredibly: from over 2 hours on the old machine to less than 15 minutes on the new machine with Reiserfs.
Just a datapoint...
If you install dump for ext2, then you should also try out that one. Dump takes only a few seconds or minutes compared to gnutar for such filesystems.
Splitting the filesystem in smaller DLE's will not gain very much speed as almost all those little files still need to be estimated. This means it needs a stat() syscall on each file or directory to determine if the file is changed, and that's what makes gnutar slow. The stat-call on a reiserfs is really really very fast. (I never tested with JFS or XFS.)
A complete different approach is to base the estimates on some intelligent statistics from the previous runs. Jean-Louis is experimenting with that idea (or implementing already?).
-- Paul Bijnens, Xplanation Tel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM Fax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *********************************************************************** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***********************************************************************
