Re: LTO-3: optimizing blocksize
What drivers are you using for your drives? I found that with IBM Ultrium drives the IBM driver does far better than the stock Linux driver, particularly in error reporting and correction. I think it also sets the minimum block size to 256k. I get around 50-60MB/s writes (I think this is actually limited by disk bandwidth), and 80MB/s reads. -- -- Skylar Thompson ([EMAIL PROTECTED]) -- http://www.cs.earlham.edu/~skylar/ signature.asc Description: OpenPGP digital signature
Re: LTO-3: optimizing blocksize
* Jean-Francois Malouin [EMAIL PROTECTED] [20070801 14:05]: * Joshua Baker-LePain [EMAIL PROTECTED] [20070801 12:20]: On Wed, 1 Aug 2007 at 11:29am, Jean-Francois Malouin wrote Hi Joshua, Hardware: I've setup the default access device to the drives as non-compressing. The library is hooked through a LSI U320 PCI-X scsi card to a 4 DualCore2 Xeon with 8GB of RAM running Debian/Etch running a 64bit kernel 2.6.21.5-i686-64-smp. It should be beefy enough :) If you ever want to use both drives simultaneously, you'll need a dual channel SCSI card and each drive will need to be on its own channel. Trust me on this one -- I tried every trick I could think of with 2 drives on one channel (there should be plenty of bandwidth, right!?), but couldn't get decent speeds when using both drives. I tried running amtapetype simultaneously and on each tape on it's own and I didn't see any differences in the transfer rates but since they seem to throttle back to half-speed maybe I wasn't hitting their bottlenecks. My bad, replying to myself... Here's the outpout of 2 simultaneous running dd's on non-compressing tape devices daisy-chained to the same scsi hba (LSI U320 PCI-X): grumpy:~# time dd if=/dev/zero of=/dev/nst0 bs=32k dd: writing `/dev/nst0': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7003.39 seconds, 58.1 MB/s real116m43.419s user0m11.317s sys 3m7.260s grumpy:~# time dd if=/dev/zero of=/dev/nst1 bs=32k dd: writing `/dev/nst1': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7002.45 seconds, 58.1 MB/s real116m42.477s user0m11.677s sys 3m6.480s Looks good to me. Not sure why amtapetype tops at 40MBs. jf -- °
Re: LTO-3: optimizing blocksize
On Thu, Aug 02, 2007 at 12:03:35PM -0400, Jean-Francois Malouin wrote: My bad, replying to myself... Here's the outpout of 2 simultaneous running dd's on non-compressing tape devices daisy-chained to the same scsi hba (LSI U320 PCI-X): grumpy:~# time dd if=/dev/zero of=/dev/nst0 bs=32k dd: writing `/dev/nst0': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7003.39 seconds, 58.1 MB/s real116m43.419s user0m11.317s sys 3m7.260s grumpy:~# time dd if=/dev/zero of=/dev/nst1 bs=32k dd: writing `/dev/nst1': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7002.45 seconds, 58.1 MB/s real116m42.477s user0m11.677s sys 3m6.480s Looks good to me. Not sure why amtapetype tops at 40MBs. Size is less than amtapetype: Earlier you said amtapetype gave ~386GB. dd's decimal 407 gig is 379GB binary. As to speed, dd is only using the scsi controller for writing, does amtapetype use it to also read its data? jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: LTO-3: optimizing blocksize
* Jon LaBadie [EMAIL PROTECTED] [20070802 13:23]: On Thu, Aug 02, 2007 at 12:03:35PM -0400, Jean-Francois Malouin wrote: My bad, replying to myself... Here's the outpout of 2 simultaneous running dd's on non-compressing tape devices daisy-chained to the same scsi hba (LSI U320 PCI-X): grumpy:~# time dd if=/dev/zero of=/dev/nst0 bs=32k dd: writing `/dev/nst0': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7003.39 seconds, 58.1 MB/s real116m43.419s user0m11.317s sys 3m7.260s grumpy:~# time dd if=/dev/zero of=/dev/nst1 bs=32k dd: writing `/dev/nst1': No space left on device 12424636+0 records in 12424635+0 records out 407130439680 bytes (407 GB) copied, 7002.45 seconds, 58.1 MB/s real116m42.477s user0m11.677s sys 3m6.480s Looks good to me. Not sure why amtapetype tops at 40MBs. Size is less than amtapetype: Earlier you said amtapetype gave ~386GB. dd's decimal 407 gig is 379GB binary. Of course you're right! And I made the same mistake twice: actually the tapetype length was 386048 mbytes which is in fact 377GB binary *not* 386GB decimal as I posted on the first post of this thread. As to speed, dd is only using the scsi controller for writing, does amtapetype use it to also read its data? Not sure how it works. I guess I'll have to get dirty and look at the source... jf jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- °
Re: LTO-3: optimizing blocksize
On Wed, 1 Aug 2007 at 11:29am, Jean-Francois Malouin wrote Hardware: I've setup the default access device to the drives as non-compressing. The library is hooked through a LSI U320 PCI-X scsi card to a 4 DualCore2 Xeon with 8GB of RAM running Debian/Etch running a 64bit kernel 2.6.21.5-i686-64-smp. It should be beefy enough :) If you ever want to use both drives simultaneously, you'll need a dual channel SCSI card and each drive will need to be on its own channel. Trust me on this one -- I tried every trick I could think of with 2 drives on one channel (there should be plenty of bandwidth, right!?), but couldn't get decent speeds when using both drives. Running amtapetype with different blocksize gives me ~386MB for capacity (close enough to 400MB) but I never seem to get close to streaming: bs= speed= 32k 50482 kps 128k50531 kps 256k50508 kps 512k50521 kps 1024k 50512 kps 2048k 15780 kps 4096k 15875 kps Any hint on what I should try next? Interesting. What if you try with dd or tar rather than amtapetype? On my LTO3 drives testing with tar bs=32k yielded 41MiB/s while bs=2048k yielded 60MiB/s. Note also that LTO drives can throttle back to half of their native rate (80MB/s for LTO3) without shoe-shining. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: LTO-3: optimizing blocksize
* Joshua Baker-LePain [EMAIL PROTECTED] [20070801 12:20]: On Wed, 1 Aug 2007 at 11:29am, Jean-Francois Malouin wrote Hi Joshua, Hardware: I've setup the default access device to the drives as non-compressing. The library is hooked through a LSI U320 PCI-X scsi card to a 4 DualCore2 Xeon with 8GB of RAM running Debian/Etch running a 64bit kernel 2.6.21.5-i686-64-smp. It should be beefy enough :) If you ever want to use both drives simultaneously, you'll need a dual channel SCSI card and each drive will need to be on its own channel. Trust me on this one -- I tried every trick I could think of with 2 drives on one channel (there should be plenty of bandwidth, right!?), but couldn't get decent speeds when using both drives. I tried running amtapetype simultaneously and on each tape on it's own and I didn't see any differences in the transfer rates but since they seem to throttle back to half-speed maybe I wasn't hitting their bottlenecks. Running amtapetype with different blocksize gives me ~386MB for capacity (close enough to 400MB) but I never seem to get close to streaming: bs= speed= 32k 50482 kps 128k50531 kps 256k50508 kps 512k50521 kps 1024k 50512 kps 2048k 15780 kps 4096k 15875 kps Any hint on what I should try next? Interesting. What if you try with dd or tar rather than amtapetype? On my LTO3 drives testing with tar bs=32k yielded 41MiB/s while bs=2048k yielded 60MiB/s. That's what you were mentionning in a post last summer iirc. One test just completed: here's what I get on a non-compression tape device dd if=/dev/zero of=/dev/nst0 bs=32k dd: writing `/dev/nst0': No space left on device 12424637+0 records in 12424636+0 records out 407130472448 bytes (407 GB) copied, 7000.04 seconds, 58.2 MB/s Would you care posting the exact command that you used so that we can really compare the numbers? Some numbers I got from a local 1TB raidset (in a raid6 config) to the tape drive: tar -b 2048 -cf /dev/nst1 /export_raid04 (gives ~42MBs) tar -b 4096 -cf /dev/nst0 /export_raid03 (gives ~42MBs) so obviously the tapes went half-speed. I'm thinking of creating 2GB ramdisk, say, and filling it up with random garbage and shoving that directly in the drive to get numbers that bypasses the disks IO. I'll post them later. Note also that LTO drives can throttle back to half of their native rate (80MB/s for LTO3) without shoe-shining. I'm aware of that fact. One of the main reason why I purchased LTO3 in fact! thanks for the input. regards, jf -- Joshua Baker-LePain Department of Biomedical Engineering Duke University -- °
Re: LTO-3: optimizing blocksize
[LTO3 blocksize?] bs= speed= 32k 50482 kps 128k50531 kps 256k50508 kps 512k50521 kps 1024k 50512 kps 2048k 15780 kps 4096k 15875 kps I am in a similar situation with Quantum LTO-2 drives. I am using 32k blocksize, and my testing indicated that even at 512k the speed fell off from the max. I am seeing 23 MB/s write speed at the peak, and it's pretty consistent and the drive sounds good at that speed. I realize this is slower than the 40 MB/s that LTO-2 is supposed to do. This is with dd from /dev/zero, and an Adaptec 2940U2W with only the tape drive on the bus. I didn't run amtapetype - I just grabbed one. I've only written 50G max of 200G allowed, and usually less so I'm not worrying. On NetBSD and probably *BSD, 'systat vmstat' shows IO speed per device.
