Re: [Bacula-users] LTO3 tape capacity lower than expected
Wow. Since setting the Maximum Block Size to 262144, and the Maximum File Size to 4G, I am now getting 35MB/s which is up from 15-18MB/s. Now I am not spooling, these are copy jobs, so I don't know if this makes a difference. On 1/7/2010 4:46 PM, Jesper Krogh wrote: > 05-Jan 00:37 bacula-sd JobId 33734: Despooling elapsed time = 00:00:58, > Transfer rate = 172.4 M bytes/second > > On LTO3 tapes in the LTO4 drive it tops at around 120MB/s. > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Thomas Mueller wrote: >>> maybe this is the reason for the "extra mb/s". >> Modifying the Maximum Block Size to more than 262144 didn't change much >> here. But changing the File Size did. Much. > > I found a post from Kern saying that Quantum told him, that about 262144 > is the best blocksize - increasing it would increase error rate too. > >> Anyway, 40 MB/s seems a bit low, even with the defaults. Before tuning >> our setup I got ~75 MB/s. Are you spooling the data to disk or writing >> directly to tape? > > yes, i was surprised too that it is that slow. > > I'm spooling to disk first (2x 1TB disk as RAID0, dedicated to bacula for > spooling). i will also start a sequential read test to check if the disks > are the bottleneck. The slow job was the only one running. > > watching iotop i saw the "maximum file size" problem: it stops writing > after 1 GB (default file size) and writes to the DB and then continues > writing. so for a LTO-4 it stops nearly 800 times until the tape is full. I've also increased the Maximum File Size, I'm spooling of a ram-disk to LTO4 tapes seeing rates like this: 04-Jan 23:09 bacula-sd JobId 33739: Despooling elapsed time = 00:01:22, Transfer rate = 121.9 M bytes/second 04-Jan 23:15 bacula-sd JobId 33739: Despooling elapsed time = 00:01:24, Transfer rate = 119.0 M bytes/second 04-Jan 23:21 bacula-sd JobId 33739: Despooling elapsed time = 00:01:24, Transfer rate = 119.0 M bytes/second 04-Jan 23:26 bacula-sd JobId 33739: Despooling elapsed time = 00:01:21, Transfer rate = 123.4 M bytes/second 04-Jan 23:31 bacula-sd JobId 33739: Despooling elapsed time = 00:01:25, Transfer rate = 117.6 M bytes/second 04-Jan 23:40 bacula-sd JobId 33734: Despooling elapsed time = 00:01:06, Transfer rate = 151.5 M bytes/second 04-Jan 23:46 bacula-sd JobId 33734: Despooling elapsed time = 00:00:56, Transfer rate = 178.5 M bytes/second 04-Jan 23:53 bacula-sd JobId 33734: Despooling elapsed time = 00:01:00, Transfer rate = 166.6 M bytes/second 04-Jan 23:57 bacula-sd JobId 33734: Despooling elapsed time = 00:00:57, Transfer rate = 175.4 M bytes/second 05-Jan 00:02 bacula-sd JobId 33734: Despooling elapsed time = 00:01:02, Transfer rate = 161.2 M bytes/second 05-Jan 00:08 bacula-sd JobId 33734: Despooling elapsed time = 00:01:00, Transfer rate = 166.6 M bytes/second 05-Jan 00:12 bacula-sd JobId 33734: Despooling elapsed time = 00:00:52, Transfer rate = 192.3 M bytes/second 05-Jan 00:17 bacula-sd JobId 33734: Despooling elapsed time = 00:00:59, Transfer rate = 169.4 M bytes/second 05-Jan 00:22 bacula-sd JobId 33734: Despooling elapsed time = 00:00:58, Transfer rate = 172.4 M bytes/second 05-Jan 00:29 bacula-sd JobId 33734: Despooling elapsed time = 00:00:58, Transfer rate = 172.4 M bytes/second 05-Jan 00:37 bacula-sd JobId 33734: Despooling elapsed time = 00:00:58, Transfer rate = 172.4 M bytes/second On LTO3 tapes in the LTO4 drive it tops at around 120MB/s. -- Jesper -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Thomas Mueller schrieb: > > >> > With > >> > > >> > Maximum File Size = 5G > >> > Maximum Block Size = 262144 > >> > Maximum Network Buffer Size = 262144 > >> > > >> > I get up to 150M MB/s while despooling to LTO-4 drives. Maximum File > >> > Size gave me some extra MB/s, I think it's as important as the > >> > Maximum Block Size. > >> > > >> > > >> thanks for providing this hints. just searching why my lto-4 is writing > >> just at 40mb/s. will try them out! > >> > >> searching the "Maximum File Size" in the manual I found this: > >> > >> If you are configuring an LTO-3 or LTO-4 tape, you probably will want > >> to set the Maximum File Size to 2GB to avoid making the drive stop to > >> write an EOF mark. > >> > >> maybe this is the reason for the "extra mb/s". > > > > Modifying the Maximum Block Size to more than 262144 didn't change much > > here. But changing the File Size did. Much. > > I found a post from Kern saying that Quantum told him, that about 262144 > is the best blocksize - increasing it would increase error rate too. > > > > > Anyway, 40 MB/s seems a bit low, even with the defaults. Before tuning > > our setup I got ~75 MB/s. Are you spooling the data to disk or writing > > directly to tape? > > yes, i was surprised too that it is that slow. > > I'm spooling to disk first (2x 1TB disk as RAID0, dedicated to bacula for > spooling). i will also start a sequential read test to check if the disks > are the bottleneck. The slow job was the only one running. > > watching iotop i saw the "maximum file size" problem: it stops writing > after 1 GB (default file size) and writes to the DB and then continues > writing. so for a LTO-4 it stops nearly 800 times until the tape is full. That's strange. Bacula shouldn't stop writing, if the Max File Size is reached it writes a EOF marker and then continuous writing to the next file. Have you activated attribute spooling? Bacula should then only write to the DB at the end of a job. I've never used iotop but with dstat I see a more or less constant stream from the spool disks to the tape drive. If this is an HP drive, I would start with the HP LTT tool and a Media and Drive Assessment Test. There is also an performance test. This helped me several times to identify bad drives or tapes. http://h18000.www1.hp.com/products/storageworks/ltt/index.html There are only RPMs for linux, but with alien you can generate a deb package for debian if you need. I run the tests in CLF mode (command line). Ralf -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
>> > With >> > >> > Maximum File Size = 5G >> > Maximum Block Size = 262144 >> > Maximum Network Buffer Size = 262144 >> > >> > I get up to 150M MB/s while despooling to LTO-4 drives. Maximum File >> > Size gave me some extra MB/s, I think it's as important as the >> > Maximum Block Size. >> > >> > >> thanks for providing this hints. just searching why my lto-4 is writing >> just at 40mb/s. will try them out! >> >> searching the "Maximum File Size" in the manual I found this: >> >> If you are configuring an LTO-3 or LTO-4 tape, you probably will want >> to set the Maximum File Size to 2GB to avoid making the drive stop to >> write an EOF mark. >> >> maybe this is the reason for the "extra mb/s". > > Modifying the Maximum Block Size to more than 262144 didn't change much > here. But changing the File Size did. Much. I found a post from Kern saying that Quantum told him, that about 262144 is the best blocksize - increasing it would increase error rate too. > > Anyway, 40 MB/s seems a bit low, even with the defaults. Before tuning > our setup I got ~75 MB/s. Are you spooling the data to disk or writing > directly to tape? yes, i was surprised too that it is that slow. I'm spooling to disk first (2x 1TB disk as RAID0, dedicated to bacula for spooling). i will also start a sequential read test to check if the disks are the bottleneck. The slow job was the only one running. watching iotop i saw the "maximum file size" problem: it stops writing after 1 GB (default file size) and writes to the DB and then continues writing. so for a LTO-4 it stops nearly 800 times until the tape is full. - Thomas -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Thomas Mueller schrieb: > > >> > I tried to do that years ago but I believe this made all tapes that > >> > were already written to unreadable (and I now have 80) so I gave this > >> > up. With my 5+ year old dual processor Opteron 248 server I get > >> > 25MB/s to 45MB/s despools (which measures the actual tape rate) for > >> > my LTO2 drives. The reason for the wide range seems to be > >> > compression. > >> > >> Can anybody confirm or rebute this for 2.2.x? I'm currently fiddling > >> with Maximum Block Size and a shiny new tape. It looks like 1M is too > >> much for my tape drive, but 512K seems to work and it's making a huge > >> difference: btape fill reports > 60 MB/s right at the beginning, then > >> drops to abour 52 MB/s. > > > > With > > > > Maximum File Size = 5G > > Maximum Block Size = 262144 > > Maximum Network Buffer Size = 262144 > > > > I get up to 150M MB/s while despooling to LTO-4 drives. Maximum File > > Size gave me some extra MB/s, I think it's as important as the Maximum > > Block Size. > > > > thanks for providing this hints. just searching why my lto-4 is writing > just at 40mb/s. will try them out! > > searching the "Maximum File Size" in the manual I found this: > > If you are configuring an LTO-3 or LTO-4 tape, you probably will want to > set the Maximum File Size to 2GB to avoid making the drive stop to write > an EOF mark. > > maybe this is the reason for the "extra mb/s". Modifying the Maximum Block Size to more than 262144 didn't change much here. But changing the File Size did. Much. Anyway, 40 MB/s seems a bit low, even with the defaults. Before tuning our setup I got ~75 MB/s. Are you spooling the data to disk or writing directly to tape? Ralf -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
>> > I tried to do that years ago but I believe this made all tapes that >> > were already written to unreadable (and I now have 80) so I gave this >> > up. With my 5+ year old dual processor Opteron 248 server I get >> > 25MB/s to 45MB/s despools (which measures the actual tape rate) for >> > my LTO2 drives. The reason for the wide range seems to be >> > compression. >> >> Can anybody confirm or rebute this for 2.2.x? I'm currently fiddling >> with Maximum Block Size and a shiny new tape. It looks like 1M is too >> much for my tape drive, but 512K seems to work and it's making a huge >> difference: btape fill reports > 60 MB/s right at the beginning, then >> drops to abour 52 MB/s. > > With > > Maximum File Size = 5G > Maximum Block Size = 262144 > Maximum Network Buffer Size = 262144 > > I get up to 150M MB/s while despooling to LTO-4 drives. Maximum File > Size gave me some extra MB/s, I think it's as important as the Maximum > Block Size. > thanks for providing this hints. just searching why my lto-4 is writing just at 40mb/s. will try them out! searching the "Maximum File Size" in the manual I found this: If you are configuring an LTO-3 or LTO-4 tape, you probably will want to set the Maximum File Size to 2GB to avoid making the drive stop to write an EOF mark. maybe this is the reason for the "extra mb/s". - Thomas -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
> Is it possible that the low block size of 64k affects tape capacity? It > looks suspicious to me that all tapes "end" at about the same size... > I have never seen it go below native capacity. I usually get around 1.5 to 1 compression rate on my data with outliners being close to 1.1 to 1 and 5.5 to 1. However my tapes do not get reused that many times. Most of my 80 tapes are used for archive purposes (never get deleted) and the rest there are over 10 in rotation that lasts months before being recycled. John -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Put a new tape in, and run tapeinfo -f /dev/nst0 and it should report what block size range your drive can support. Alng with lots of other useful information. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
That should not matter. I have read somewhere that Kern said that very large blocks sizes can waste space, I don not know how this works tho. On 1/6/2010 8:20 AM, Tino Schwarze wrote: > > Is it possible that the low block size of 64k affects tape capacity? It > looks suspicious to me that all tapes "end" at about the same size... > > Thanks so far, > > Tino. > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
I believe they are rated for 250 complete passes. On 1/6/2010 9:05 AM, Tino Schwarze wrote: > All the same config, just an older tape. :-( I'm not sure, how often > it's been used because of our rather complicted multi-stage backup > scheme (daily/weekly/monthly/yearly pools). Is it possible that the > drive simply wastes capacity because of low throughput? > > What's your experience: How often may a tape be written to? > > Thanks, > > Tino. > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
On Wed, Jan 06, 2010 at 02:20:06PM +0100, Tino Schwarze wrote: > > > > It looks like btape is not happy. > > > > > > > > Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on > > > > device "Superloader-Drive" (/dev/nst0). > > > > > > > > Are your tapes old (still good)? Did you clean the drive? Latest > > > > Firmware? > > > > The drive requested cleaning just yesterday which I did. I'm not sure > > how old the tape in question really is. I used a dell update utility and > > it updated the drive to v2181 which seems to be the latest for a > > half-height SCSI tape drive. 30MB/s seems a bit low to me - it should be > > able to do 60MB/s per spec. Or is that just theory vs. real world? > > > > > I would add are you using LTO3 tapes or LTO2 tapes? > > > > They are labelled 400GB/800GB - all of the same kind, so they're > > definitely LTO3. The autoloader says: "Drive Idle Gen 3 Data" in it's web > > interface, so yes, I'm sure. > > > > I could check a very new tape tomorrow or try a shiny new one, just to > > be sure. > > Here are the results for a shiny new tape with block size 512k: > > > Wrote blk_block=75, dev_blk_num=651 VolBytes=391,110,459,392 > > rate=59638.7 KB/s > > 06-Jan 13:50 btape JobId 0: End of Volume "TestVolume1" at 397:1147 on > > device "Superloader-Drive" (/dev/nst0). Write of 524288 bytes got -1. > > 06-Jan 13:50 btape JobId 0: Re-read of last block succeeded. btape: > > btape.c:2345 Last block at: 397:1146 this_dev_block_num=1147 > > btape: btape.c:2379 End of tape 397:0. VolumeCapacity=391,370,506,240. > > Write rate = 59587.5 KB/s > > Done writing 0 records ... > > Wrote state file last_block_num1=1146 last_block_num2=0 > > > > > > 13:50:27 Done filling tape at 397:0. Now beginning re-read of tape ... > > 06-Jan 13:50 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > > command. > > 06-Jan 13:50 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > > Slot 1. > > 06-Jan 13:50 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > > command. > > 06-Jan 13:50 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > > Slot 1. > > 06-Jan 13:50 btape JobId 0: Ready to read from volume "TestVolume1" on > > device "Superloader-Drive" (/dev/nst0). > > Rewinding. > > Reading the first 1 records from 0:0. > > 1 records read now at 1:626 > > Reposition from 1:626 to 397:1146 > > Reading block 1146. > > > > The last block on the tape matches. Test succeeded. > > So this looks very promising: 390 GB and I'm also seeing almost 60MB/s. > I will repeat this test using one of the older tapes, then report back. > > Is it possible that the low block size of 64k affects tape capacity? It > looks suspicious to me that all tapes "end" at about the same size... It feels like the tapes really are worn out. I aborted the new btape test since the throughput was really low: 14:31:54 Begin writing Bacula records to tape ... Wrote blk_block=5000, dev_blk_num=1185 VolBytes=2,620,915,712 rate=24494.5 KB/s Wrote blk_block=1, dev_blk_num=464 VolBytes=5,242,355,712 rate=24845.3 KB/s Wrote blk_block=15000, dev_blk_num=1650 VolBytes=7,863,795,712 rate=26656.9 KB/s Wrote blk_block=2, dev_blk_num=929 VolBytes=10,485,235,712 rate=24613.2 KB/s Wrote blk_block=25000, dev_blk_num=208 VolBytes=13,106,675,712 rate=24590.4 KB/s Wrote blk_block=3, dev_blk_num=1394 VolBytes=15,728,115,712 rate=25532.7 KB/s 14:43:17 Flush block, write EOF All the same config, just an older tape. :-( I'm not sure, how often it's been used because of our rather complicted multi-stage backup scheme (daily/weekly/monthly/yearly pools). Is it possible that the drive simply wastes capacity because of low throughput? What's your experience: How often may a tape be written to? Thanks, Tino. -- "What we nourish flourishes." - "Was wir nähren erblüht." www.lichtkreis-chemnitz.de www.tisc.de -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Hi there, On Tue, Jan 05, 2010 at 07:30:44PM +0100, Tino Schwarze wrote: > > > It looks like btape is not happy. > > > > > > Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device > > > "Superloader-Drive" (/dev/nst0). > > > > > > Are your tapes old (still good)? Did you clean the drive? Latest Firmware? > > The drive requested cleaning just yesterday which I did. I'm not sure > how old the tape in question really is. I used a dell update utility and > it updated the drive to v2181 which seems to be the latest for a > half-height SCSI tape drive. 30MB/s seems a bit low to me - it should be > able to do 60MB/s per spec. Or is that just theory vs. real world? > > > I would add are you using LTO3 tapes or LTO2 tapes? > > They are labelled 400GB/800GB - all of the same kind, so they're > definitely LTO3. The autoloader says: "Drive Idle Gen 3 Data" in it's web > interface, so yes, I'm sure. > > I could check a very new tape tomorrow or try a shiny new one, just to > be sure. Here are the results for a shiny new tape with block size 512k: > Wrote blk_block=75, dev_blk_num=651 VolBytes=391,110,459,392 rate=59638.7 > KB/s > 06-Jan 13:50 btape JobId 0: End of Volume "TestVolume1" at 397:1147 on device > "Superloader-Drive" (/dev/nst0). Write of 524288 bytes got -1. > 06-Jan 13:50 btape JobId 0: Re-read of last block succeeded. btape: > btape.c:2345 Last block at: 397:1146 this_dev_block_num=1147 > btape: btape.c:2379 End of tape 397:0. VolumeCapacity=391,370,506,240. > Write rate = 59587.5 KB/s > Done writing 0 records ... > Wrote state file last_block_num1=1146 last_block_num2=0 > > > 13:50:27 Done filling tape at 397:0. Now beginning re-read of tape ... > 06-Jan 13:50 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > command. > 06-Jan 13:50 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > Slot 1. > 06-Jan 13:50 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > command. > 06-Jan 13:50 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > Slot 1. > 06-Jan 13:50 btape JobId 0: Ready to read from volume "TestVolume1" on device > "Superloader-Drive" (/dev/nst0). > Rewinding. > Reading the first 1 records from 0:0. > 1 records read now at 1:626 > Reposition from 1:626 to 397:1146 > Reading block 1146. > > The last block on the tape matches. Test succeeded. So this looks very promising: 390 GB and I'm also seeing almost 60MB/s. I will repeat this test using one of the older tapes, then report back. Is it possible that the low block size of 64k affects tape capacity? It looks suspicious to me that all tapes "end" at about the same size... Thanks so far, Tino. -- "What we nourish flourishes." - "Was wir nähren erblüht." www.lichtkreis-chemnitz.de www.tisc.de -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Tino Schwarze schrieb: > On Tue, Jan 05, 2010 at 04:55:51PM -0500, John Drescher wrote: > > > >> I'm not seeing anywhere close to 60M/s ( < 30 ). I think I just fixed > > >> that. I increased the block size to 1M, and that seemed to really > > >> increase the throughput, in the test I just did. I will see tomorrow, > > >> when it all runs. > > > > > > Yes, if you aren't already, whenever writing to tape you should almost > > > without exception (and certainly on any modern tape drive) be using the > > > largest block size that btape says your drive supports. > > > > I tried to do that years ago but I believe this made all tapes that > > were already written to unreadable (and I now have 80) so I gave this > > up. With my 5+ year old dual processor Opteron 248 server I get 25MB/s > > to 45MB/s despools (which measures the actual tape rate) for my LTO2 > > drives. The reason for the wide range seems to be compression. > > Can anybody confirm or rebute this for 2.2.x? I'm currently fiddling > with Maximum Block Size and a shiny new tape. It looks like 1M is too > much for my tape drive, but 512K seems to work and it's making a huge > difference: btape fill reports > 60 MB/s right at the beginning, then > drops to abour 52 MB/s. With Maximum File Size = 5G Maximum Block Size = 262144 Maximum Network Buffer Size = 262144 I get up to 150M MB/s while despooling to LTO-4 drives. Maximum File Size gave me some extra MB/s, I think it's as important as the Maximum Block Size. Ralf -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
On Tue, Jan 05, 2010 at 04:55:51PM -0500, John Drescher wrote: > >> I'm not seeing anywhere close to 60M/s ( < 30 ). I think I just fixed > >> that. I increased the block size to 1M, and that seemed to really > >> increase the throughput, in the test I just did. I will see tomorrow, > >> when it all runs. > > > > Yes, if you aren't already, whenever writing to tape you should almost > > without exception (and certainly on any modern tape drive) be using the > > largest block size that btape says your drive supports. > > I tried to do that years ago but I believe this made all tapes that > were already written to unreadable (and I now have 80) so I gave this > up. With my 5+ year old dual processor Opteron 248 server I get 25MB/s > to 45MB/s despools (which measures the actual tape rate) for my LTO2 > drives. The reason for the wide range seems to be compression. Can anybody confirm or rebute this for 2.2.x? I'm currently fiddling with Maximum Block Size and a shiny new tape. It looks like 1M is too much for my tape drive, but 512K seems to work and it's making a huge difference: btape fill reports > 60 MB/s right at the beginning, then drops to abour 52 MB/s. Thanks, Tino. -- "What we nourish flourishes." - "Was wir nähren erblüht." www.lichtkreis-chemnitz.de www.tisc.de -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Brian Debelius wrote: > That would be 16M. Isn't the SD hard limited to 1M? > "The maximun size-in-bytes possible is 2,000,000" From the SD Configuration Maximum block size directive. Regards, Richard -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
On Tue, Jan 5, 2010 at 3:00 PM, Phil Stracchino wrote: > Brian Debelius wrote: >> I'm not seeing anywhere close to 60M/s ( < 30 ). I think I just fixed >> that. I increased the block size to 1M, and that seemed to really >> increase the throughput, in the test I just did. I will see tomorrow, >> when it all runs. > > Yes, if you aren't already, whenever writing to tape you should almost > without exception (and certainly on any modern tape drive) be using the > largest block size that btape says your drive supports. > I tried to do that years ago but I believe this made all tapes that were already written to unreadable (and I now have 80) so I gave this up. With my 5+ year old dual processor Opteron 248 server I get 25MB/s to 45MB/s despools (which measures the actual tape rate) for my LTO2 drives. The reason for the wide range seems to be compression. John -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
That would be 16M. Isn't the SD hard limited to 1M? On 1/5/2010 3:00 PM, Phil Stracchino wrote: > Brian Debelius wrote: > >> I'm not seeing anywhere close to 60M/s (< 30 ). I think I just fixed >> that. I increased the block size to 1M, and that seemed to really >> increase the throughput, in the test I just did. I will see tomorrow, >> when it all runs. >> > Yes, if you aren't already, whenever writing to tape you should almost > without exception (and certainly on any modern tape drive) be using the > largest block size that btape says your drive supports. > > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
Brian Debelius wrote: > I'm not seeing anywhere close to 60M/s ( < 30 ). I think I just fixed > that. I increased the block size to 1M, and that seemed to really > increase the throughput, in the test I just did. I will see tomorrow, > when it all runs. Yes, if you aren't already, whenever writing to tape you should almost without exception (and certainly on any modern tape drive) be using the largest block size that btape says your drive supports. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
I'm not seeing anywhere close to 60M/s ( < 30 ). I think I just fixed that. I increased the block size to 1M, and that seemed to really increase the throughput, in the test I just did. I will see tomorrow, when it all runs. You should not be seeing any errors. On 1/5/2010 1:30 PM, Tino Schwarze wrote: > On Tue, Jan 05, 2010 at 12:48:53PM -0500, John Drescher wrote: > > >>> It looks like btape is not happy. >>> >>> Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device >>> "Superloader-Drive" (/dev/nst0). >>> >>> Are your tapes old (still good)? Did you clean the drive? Latest Firmware? >>> > > The drive requested cleaning just yesterday which I did. I'm not sure > how old the tape in question really is. I used a dell update utility and > it updated the drive to v2181 which seems to be the latest for a > half-height SCSI tape drive. 30MB/s seems a bit low to me - it should be > able to do 60MB/s per spec. Or is that just theory vs. real world? > > >> I would add are you using LTO3 tapes or LTO2 tapes? >> > They are labelled 400GB/800GB - all of the same kind, so they're > definitely LTO3. The autoloader says: "Drive Idle Gen 3 Data" in it's web > interface, so yes, I'm sure. > > I could check a very new tape tomorrow or try a shiny new one, just to > be sure. > > Thanks so far, > > Tino. > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
On Tue, Jan 05, 2010 at 12:48:53PM -0500, John Drescher wrote: > > It looks like btape is not happy. > > > > Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device > > "Superloader-Drive" (/dev/nst0). > > > > Are your tapes old (still good)? Did you clean the drive? Latest Firmware? The drive requested cleaning just yesterday which I did. I'm not sure how old the tape in question really is. I used a dell update utility and it updated the drive to v2181 which seems to be the latest for a half-height SCSI tape drive. 30MB/s seems a bit low to me - it should be able to do 60MB/s per spec. Or is that just theory vs. real world? > I would add are you using LTO3 tapes or LTO2 tapes? They are labelled 400GB/800GB - all of the same kind, so they're definitely LTO3. The autoloader says: "Drive Idle Gen 3 Data" in it's web interface, so yes, I'm sure. I could check a very new tape tomorrow or try a shiny new one, just to be sure. Thanks so far, Tino. -- "What we nourish flourishes." - "Was wir nähren erblüht." www.lichtkreis-chemnitz.de www.tisc.de -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
On Tue, Jan 5, 2010 at 12:37 PM, Brian Debelius wrote: > It looks like btape is not happy. > > Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device > "Superloader-Drive" (/dev/nst0). > > Are your tapes old (still good)? Did you clean the drive? Latest Firmware? > I would add are you using LTO3 tapes or LTO2 tapes? John -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity lower than expected
It looks like btape is not happy. Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device "Superloader-Drive" (/dev/nst0). Are your tapes old (still good)? Did you clean the drive? Latest Firmware? On 1/5/2010 9:06 AM, Tino Schwarze wrote: > Hi there, > > I'm struggling with my LTO3 autochanger (Quantum Superloader3). We're > using HP tapes of 400/800 GB capacity (uncompressed/compressed). > Everything has been running fine for about 3 years now (OS: OpenSuSE > 10.2, package bacula-postgresql-2.2.5-1), but we're starting to really > fill our tapes (we use a one tape per day strategy and they get recycled > before next use). > > I did a btape "test" and all worked fine. Then I did btape "fill" which > ended up like this: > > 4:06:26 Flush block, write EOF > Wrote blk_block=346, dev_blk_num=4000 VolBytes=223,211,455,488 > rate=30393.7 KB/s > Wrote blk_block=3465000, dev_blk_num=9000 VolBytes=223,534,015,488 > rate=30404.5 KB/s > Wrote blk_block=347, dev_blk_num=14000 VolBytes=223,856,575,488 > rate=30411.2 KB/s > Wrote blk_block=3475000, dev_blk_num=3500 VolBytes=224,179,135,488 > rate=30397.2 KB/s > Wrote blk_block=348, dev_blk_num=8500 VolBytes=224,501,695,488 > rate=30403.8 KB/s > wrote blk_block=3485000, dev_blk_num=13500 VolBytes=224,824,255,488 > rate=30414.5 KB/s > 05-Jan 14:07 btape JobId 0: End of Volume "TestVolume1" at 327:1 on device > "Superloader-Drive" (/dev/nst0). Write of 64512 bytes got -1. > 05-Jan 14:07 btape JobId 0: Re-read of last block succeeded. > btape: btape.c:2345 Last block at: 326:15500 this_dev_block_num=1 > btape: btape.c:2379 End of tape 327:0. VolumeCapacity=224,953,344,000. > Write rate = 30386.8 KB/s > Done writing 0 records ... > Wrote state file last_block_num1=15500 last_block_num2=0 > > > 14:07:31 Done filling tape at 327:0. Now beginning re-read of tape ... > 05-Jan 14:07 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > command. > 05-Jan 14:07 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > Slot 1. > 05-Jan 14:07 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > command. > 05-Jan 14:07 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > Slot 1. > 05-Jan 14:07 btape JobId 0: Ready to read from volume "TestVolume1" on > device "Superloader-Drive" (/dev/nst0). > Rewinding. > Reading the first 1 records from 0:0. > 1 records read now at 1:5084 > Reposition from 1:5084 to 326:15500 > Reading block 15500. > Error reading block: ERR=block.c:1008 Read zero bytes at 326:0 on device > "Superloader-Drive" (/dev/nst0). > > > There were no error messages in the system log nor dmesg. So it looks > like my 400 GB tape takes only about 224 GB of data? This number seems > to be different for different tapes, some take 230, some 240, some less. > The autoloader doesn't report any errors either. System load was > negligible. > > The device is attached to an > > Adaptec AIC79xx driver version: 3.0 > Adaptec 39320A Ultra320 SCSI adapter > aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 101-133Mhz, 512 SCBs > [...] > Target 4 Negotiation Settings > User: 320.000MB/s transfers (160.000MHz RDSTRM|DT|IU|RTI|QAS, > 16bit) > Goal: 160.000MB/s transfers (80.000MHz DT, 16bit) > Curr: 160.000MB/s transfers (80.000MHz DT, 16bit) > Channel A Target 4 Lun 0 Settings > Commands Queued 56783788 > Commands Active 0 > Command Openings 1 > Max Tagged Openings 0 > Device Queue Frozen Count 0 > Channel A Target 4 Lun 1 Settings > Commands Queued 1234 > Commands Active 0 > Command Openings 1 > Max Tagged Openings 0 > Device Queue Frozen Count 0 > > When I tried to write the original label back to the tape, I got: > > 05-Jan 14:57 btape: Fatal Error at dev.c:1651 because: > dev.c:1650 Attempt to WEOF on non-appendable Volume > 05-Jan 14:57 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" > command. > 05-Jan 14:57 btape JobId 0: 3302 Autochanger "loaded? drive 0", result is > Slot 1. > Wrote Volume label for volume "TestVolume1". > > Any hints what might be wrong here? Anything I could test? > > Thanks! > > Tino. > > -- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
