* John R. Jackson <[EMAIL PROTECTED]> (Sun, Jan 28, 2001 at 08:20:30PM -0500)
>>I have always wondered .. why does amanda pipe ufsdump output to ufsrestore
>>before sending it to the tape device?
> It's collecting the index data.
> The dump (or tar) output pipeline is rather complicated. The image data
> goes back to sendbackup who in turn tee's it to the restore program
> to gather the index information (if indexing is enabled) as well as
> sending the raw data (possibly through a compression program) back on
> the network to a dumper process on the server side. The restore program
> also feeds its results back through sendbackup to be sent to the dumper
> on a different socket (as I recall). So sendbackup is multiplexing five
> data streams:
>
> * reading the dump image coming in from the backup program
>
> * writing the image out to the index (restore) process
>
> * writing the image out the socket connected to dumper on the server
> or to a compression program
>
> * reading the output of the index process
>
> * writing the index data to another socket back to dumper
Allright,
that explains a lot.
Thanks very much.
>>If I ufsdump direct to tape, eg.
>>
>>ufsdump 0f /dev/rmt/0n /
>>
>>I consistently achieve 3mb/second (Exabyte mammoth).
>>
>>If amanda is dumping direct to tape (file systems that are bigger than the
>>holding disk), I'm lucky if i get 1mb/second.
>>
>>If it's going from the holding disk to tape, I get 3mb/second, as expected.
> But you're comparing apples and oranges. As you've noted, going from
> disk to tape on the same machine gets 3 MBytes/s whether you are using
> ufsdump or Amanda is using taper to copy a holding disk image.
> But that's not what happens when Amanda is dumping a client direct to
> tape. The data has to go across the network (even if it's all on the
> local machine it still goes through the kernel network stack). And,
> probably even more important, Amanda does compression when dumping,
> not when writing to tape.
Even with compression disabled, amanda is much slower.
Dumper gets between .5 and 3kbps (disk and tape all on server ).
Taper gets 7-8Kbps
[see below for more detailed report]
> So a dump to holding disk would be "slow" but the corresponding holding
> disk to tape operation would be "fast". But a direct to tape backup
> would pay the penalty and show the speed loss due to compression even
> though the tape I/O portion is going as fast as it is given data.
It's not the compression (or at leat not only the compression) that gives
the penalty, but more likely the 5 way split ..
As long as Im doing incrementals it isn't too bad, since those dump to
disk, and from disk to tape is fast.
When doing a 0dump of a 100+G tar directory it becomes painfull ;)
> You didn't mention what kind of dump rates Amanda reports. Those should
> more or less match your direct to tape numbers for large enough images
> to get a good sample and with similar clients.
> Note that I'm not saying something isn't wrong in Amanda. Just that we
> need to narrow down the list of culprits.
Using the atatched script, I got the following out of my latest amdump file
(all disks are local to the amanda server
all disks are wide scsi (80Mbps ) or better (LVD) )
(usage,
getbps.pl amdumpfile
)
DUMPER: 17.7
TAPER: 55.6
DUMPER: 8.7
TAPER: 47.7
DUMPER: 565.9
TAPER: 2886.3
DUMPER: 1431.6
TAPER: 9620.7
DUMPER: 294.6
TAPER: 6414.0
DUMPER: 3061.4
DUMPER: 2249.5
TAPER: 9787.8
TAPER: 2884.3
DUMPER: 1027.1
TAPER: 7059.3
DUMPER: 2077.8
TAPER: 6033.7
DUMPER: 1869.2
TAPER: 5516.5
DUMPER: 2913.1
TAPER: 7524.7
DUMPER: 1671.4
TAPER: 8072.4
DUMPER: 1423.8
TAPER: 6464.1
DUMPER: 1216.9
DUMPER: 2204.8
TAPER: 7347.8
TAPER: 6818.4
DUMPER: 1796.7
TAPER: 6793.8
DUMPER: 1584.8
TAPER: 7249.1
DUMPER: 2673.5
TAPER: 11384.8
DUMPER: 4331.9
TAPER: 4330.9
DUMPER: 2742.2
TAPER: 2742.1
Averages
Dumper 1758.13
Taper 5951.7
The same script run over the last 10orso amdump files I had still on disk
(output stripped to only show the averages) gives
Averages
Dumper 1124.84210526316
Taper 4952.77368421053
Averages
Dumper 1198.96
Taper 4038.22
Averages
Dumper 1762.46666666667
Taper 7299.05
Averages
Dumper 1758.13
Taper 5951.7
Averages
Dumper 1523.97777777778
Taper 5017.5
Averages
Dumper 899.25
Taper 4525.7625
Averages
Dumper 1771.47
Taper 5564.87
Averages
Dumper 1658.56
Taper 6897.71
Averages
Dumper 82.1
Taper 3085.6
Averages
Dumper 95.81
Taper 3129.7
Kind regards,
--
Gerhard den Hollander Phone +31-10.280.1515
Technical Support Jason Geosystems BV Fax +31-10.280.1511
(When calling please note: we are in GMT+1)
[EMAIL PROTECTED] POBox 1573
visit us at http://www.jasongeo.com 3000 BN Rotterdam
JASON.......#1 in Reservoir Characterization The Netherlands
This e-mail and any attachment is/are intended solely for the named
addressee(s) and may contain information that is confidential and privileged.
If you are not the intended recipient, we request that you do not
disseminate, forward, distribute or copy this e-mail message.
If you have received this e-mail message in error, please notify us
immediately by telephone and destroy the original message.
getbps.pl
