Slow creating lots of files...
Doing some further research into my NetBackup slow restore performance...I have noticed that if I create a lot of small filesit can take about 10 times as long as creating one big file of the same size. With a sample set of 500MB... Creating 500MB with of text files (about 30,000 of them) takes about 10 minutes Creating 1 500 MB file takes about 1 minute. Restoring 1 500MB file takes about 1 minute or so...a little more to allow the robot to mount the tape/position the tape etc. Restoring the 30,000 files with NetBackup takes about 20 minutes Restoring the 30,000 files to an alternate location takes about 40 minutes Now interms of backing this all up...well 2.5GB of it takes about 8 minutes. My hardware platform is a Compaq DL360 with dual PIII/933MHz CPUs (only one configured with the kernel to date). Two 36GB (10K RPM) drives configured with Hardware RAID1 1 GB RAM. (The OS is configured with a 2 GB swap space.) Here is the FreeBSD Slice... # /dev/idad0s1c: type: ESDI disk: idad0s1 label: flags: bytes/sector: 512 sectors/track: 32 tracks/cylinder: 255 sectors/cylinder: 8160 cylinders: 8320 sectors/unit: 67891200 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: #size offsetfstype [fsize bsize bps/cpg] a: 209715204.2BSD 2048 1638489 # (Cyl.0 - 257*) b: 4194304 2097152 swap# (Cyl. 257*- 771*) c: 678912000unused0 0 # (Cyl.0 - 8319) e: 8388608 62914564.2BSD 2048 1638489 # (Cyl. 771*- 1799*) f: 49152000 146800644.2BSD 2048 1638489 # (Cyl. 1799*- 7822*) g: 4059136 638320644.2BSD 2048 1638490 # (Cyl. 7822*- 8319*) mislog01 # df -h Filesystem Size Used Avail Capacity Mounted on /dev/idad0s1a 1008M50M 878M 5%/ /dev/idad0s1g 1.9G 4.0K 1.8G 0%/home /dev/idad0s1e 3.9G 976M 2.7G26%/usr /dev/idad0s1f23G 2.5G19G12%/var procfs 4.0K 4.0K 0B 100%/proc Have any of you seen this before? Is there some tuning that I can do? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Slow creating lots of files...
On Fri, Nov 21, 2003 at 02:57:41PM -0500, Marco Greene (Home) wrote: Doing some further research into my NetBackup slow restore performance...I have noticed that if I create a lot of small filesit can take about 10 times as long as creating one big file of the same size. Well, that doesn't sound too abnormal to me..creating lots of small files is a lot more work for the drive. You forgot to mention details of your FreeBSD version and configuration (such as FS mount options, kernel config, full hardware details, etc). Obviously, these can make all the difference to analyzing your problem. Kris pgp0.pgp Description: PGP signature
RE: Slow creating lots of files...
On Fri, Nov 21, 2003 at 02:57:41PM -0500, Marco Greene (Home) wrote: Doing some further research into my NetBackup slow restore performance...I have noticed that if I create a lot of small filesit can take about 10 times as long as creating one big file of the same size. Ever install Rainbow 6? That install could take almost an hour from 1 cd. If they had zip'ed the files transferred and unzipped, it would have been much faster. With lots of small files, you have seek, read, seek, read, etc. With one big file, seek, read, read, read... I am not an expert on this, but it doesn't sound very abnormal. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
