Re: Amtapetype wrongly reporting compression?

[Gene Heskett]

On Monday 15 January 2018 08:11:37 Jean-Louis Martineau wrote:

> Luc,
>
> You should keep the driver in variable block size:
> eg . /usr/bin/mt -f /dev/st0 setblk 0
>
> Jean-Louis
>
> On 12/01/18 03:20 PM, Luc Lalonde wrote:
> > Hello Gene,
> >
> > Would a variant like this:
> >
> > /usr/sbin/mtx load 1 0;
> > /usr/bin/mt -f /dev/st0 compression 0;
> > /usr/bin/mt -f /dev/st0 setblk 524288;
> > /usr/bin/dd if=/dev/zero of=/dev/st0 bs=512k count=1;
> > su amandabackup -c "/usr/sbin/amlabel -f Monthly-LTO7
> > 000321L7 slot 1";
> > /usr/sbin/mtx unload 1 0;
> >
> > work?
> >
> > During my tests with this, the hardware compression stays disabled
> > when I load a new tape.
> >
> > Thanks!
> >
> > On 2018-01-11 10:51 PM, Gene Heskett wrote:
> >> 1. rewind the tape.
> >> 1a. Do NOT remove tape from drive, or cause it to read the tape
> >> other than what I write here until after step 5.
> >> 2. read the label out to a 32k file.
> >> 3. rewind the tape.
> >> 4. Turn the compression off, probably with mt.
> >> 5. Immediately re-write that label while the tape is rewound, and
> >> the hidden tape id block in front of the label will get rewritten
> >> too, with that compression flag off.

The above quote of my previous message is Copyright 2018 by Maurice E. 
Heskett.

> This message is the property of CARBONITE, INC. and may contain
> confidential or privileged information. If this message has been
> delivered to you by mistake, then do not copy or deliver this message
> to anyone.  Instead, destroy it and notify me by reply e-mail

And I publically object to CARBINITE, INC's attempt to establish 
copyright claims on what I write by claiming its their property.  Yahoo 
used to do that too. Am I going to revert to a sig establishing my own 
copyright claims, something I haven't had to do in a decade or more?

I don't mind reading a "commercial" touting the product thats paying the 
net bills. I am a firm believer in the TANSTAAFL principle, but what I 
write is NOT CARBONITE INC.'s property.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: Amtapetype wrongly reporting compression?

[Jean-Louis Martineau]

amtapetype report the drive is in compressed mode because it can write 
compressible data a lot faster than uncompressible data.


amtapetpye write uncompressible data to determine the size, that's why 
you get the native tape size.


It is also strange that it report 'LEOM is not supported', i'm sure LEON 
is supported for theses drives.


Jean-Louis


On 13/01/18 04:48 PM, Luc Lalonde wrote:

Hello Jean-Louis,

I re-ran ‘amtapetype’ with a much larger blocksize (512k).  Here’s the 
result:


Checking for FSF_AFTER_FILEMARK requirement
Applying heuristic check for compression.
Wrote random (uncompressible) data at 139266114.064516 bytes/sec
Wrote fixed (compressible) data at 287816635.73 bytes/sec
Compression: enabled
Writing one file to fill the volume.
Wrote 6017046282240 bytes at 138187 kb/sec
Writing smaller files (60170436608 bytes) to determine filemark.
define tapetype unknown-tapetype {
comment "Created by amtapetype; compression enabled"
length 5876021760 kbytes
filemark 2839 kbytes
speed 138187 kps
blocksize 512 kbytes
}
# LEOM is not supported for this drive and kernel

After having unloaded and re-loaded this tape, I can confirm that the 
compression is disabled:


DataCompEnabled: no
DataCompCapable: yes
DataDeCompEnabled: yes

As I mentioned in my earlier mail, the compression test does not seem 
to be correctly reporting the compression status of the tape…  The 
‘length’ reported with ‘amtapetype’ is roughly the ‘native’ capacity 
of the LTO7 tapes.


Can you confirm this?

Best regards, Luc.

On Jan 11, 2018, at 3:09 PM, Jean-Louis Martineau 
> wrote:


Luc,

amtapetype use speed heuristic to detect if the drive is in compressed
mode, it might not be good for newer drives.

Why you didn't post the complete amtapetype output? I can't tell you why
the heuristic is bad without those numbers.

The default block size of 32k was good 15 years ago, but it is really
bad for new drives. You should really think about increasing it a lot.

Jean-Louis

On 11/01/18 02:56 PM, Luc Lalonde wrote:
> I sent the wrong output in the last email Here's the correct
> 'tapeinfo':
>
> Product Type: Tape Drive
> Vendor ID: 'IBM '
> Product ID: 'ULTRIUM-HH7 '
> Revision: 'G9Q1'
> Attached Changer API: No
> SerialNumber: '123456789A'
> MinBlock: 1
> MaxBlock: 8388608
> SCSI ID: 4
> SCSI LUN: 0
> Ready: yes
> BufferedMode: yes
> Medium Type: 0x78
> Density Code: 0x5c
> BlockSize: 32768
> DataCompEnabled: no
> DataCompCapable: yes
> DataDeCompEnabled: yes
> CompType: 0xff
> DeCompType: 0xff
> BOP: yes
> Block Position: 0
> Partition 0 Remaining Kbytes: -1
> Partition 0 Size in Kbytes: -1
> ActivePartition: 0
> EarlyWarningSize: 0
> NumPartitions: 0
> MaxPartitions: 3
>
> Further, here's my '/etc/stinit.def':
>
> manufacturer="IBM" model = "ULTRIUM-HH7" {
> scsi2logical=1
> can-bsr=1
> auto-lock=1
> two-fms=0
> drive-buffering=1
> buffer-writes
> read-ahead=1
> async-writes=1
> can-partitions=1
> fast-mteom=0
> sysv=1
> timeout=180
> long-timeout=14400
> mode1 blocksize=32768 compression=1
> mode2 blocksize=32768 compression=0
> mode3 disabled=1
> mode4 disabled=1
>
> Sorry, about that!
>
> On 2018-01-11 02:43 PM, Luc Lalonde wrote:
>> Hello Folks,
>>
>> I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).
>>
>> We're migrating from LTO5 to LTO7 and I'm getting strange results
>> when I run 'amtapetype'.
>>
>> Here's what I get after roughly 35 hours:
>>
>> define tapetype LTO7 {
>> comment "Created by amtapetype; compression enabled"
>> length 5874932832 kbytes
>> filemark 1813 kbytes
>> speed 118310 kps
>> blocksize 32 kbytes
>> }
>>
>> Why is it saying that it can write roughly 6Tb if it's supposedly
>> hardware compressed? LTO7 is supposed to give a compression ration
>> of 2.5 to 1.
>>
>> I've disabled compression... Here's the ouptut of 'tapeinfo:
>>
>> [root@ulysses ~]# tapeinfo -f /dev/sg12
>> Product Type: Tape Drive
>> Vendor ID: 'IBM '
>> Product ID: 'ULTRIUM-HH7 '
>> Revision: 'G9Q1'
>> Attached Changer API: No
>> SerialNumber: '123456789A'
>> MinBlock: 1
>> MaxBlock: 8388608
>> SCSI ID: 4
>> SCSI LUN: 0
>> Ready: yes
>> BufferedMode: yes
>> Medium Type: 0x78
>> Density Code: 0x5c
>> BlockSize: 32768
>> DataCompEnabled: yes
>> DataCompCapable: yes
>> DataDeCompEnabled: yes
>> CompType: 0xff
>> DeCompType: 0xff
>> Block Position: 2
>> Partition 0 Remaining Kbytes: -1
>> Partition 0 Size in Kbytes: -1
>> ActivePartition: 0
>> EarlyWarningSize: 0
>> NumPartitions: 0
>> MaxPartitions: 3
>>
>> Am I missing something?
>>
>> Thanks!|
>>
>


*Disclaimer*

This message is the property of*CARBONITE, INC.* 
and 
may contain confidential or privileged information.


If this message has been delivered to you by mistake, then do not 
copy or deliver this message to anyone. Instead, destroy it and 
notify me by reply e-mail.





This message is the property of CARBONITE, INC. and may contain 

Re: Amtapetype wrongly reporting compression?

[Jean-Louis Martineau]

Luc,

You should keep the driver in variable block size:
eg . /usr/bin/mt -f /dev/st0 setblk 0

Jean-Louis

On 12/01/18 03:20 PM, Luc Lalonde wrote:

Hello Gene,

Would a variant like this:

/usr/sbin/mtx load 1 0;
/usr/bin/mt -f /dev/st0 compression 0;
/usr/bin/mt -f /dev/st0 setblk 524288;
/usr/bin/dd if=/dev/zero of=/dev/st0 bs=512k count=1;
su amandabackup -c "/usr/sbin/amlabel -f Monthly-LTO7 000321L7 
slot 1";

/usr/sbin/mtx unload 1 0;

work?

During my tests with this, the hardware compression stays disabled 
when I load a new tape.


Thanks!


On 2018-01-11 10:51 PM, Gene Heskett wrote:

1. rewind the tape.
1a. Do NOT remove tape from drive, or cause it to read the tape other
than what I write here until after step 5.
2. read the label out to a 32k file.
3. rewind the tape.
4. Turn the compression off, probably with mt.
5. Immediately re-write that label while the tape is rewound, and the
hidden tape id block in front of the label will get rewritten too, with
that compression flag off.



This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: Amtapetype wrongly reporting compression?

[Gene Heskett]

On Friday 12 January 2018 15:20:51 Luc Lalonde wrote:

> Hello Gene,
>
> Would a variant like this:
>
>      /usr/sbin/mtx load 1 0;
>      /usr/bin/mt -f /dev/st0 compression 0;
>      /usr/bin/mt -f /dev/st0 setblk 524288;
>      /usr/bin/dd if=/dev/zero of=/dev/st0 bs=512k count=1;
>      su amandabackup -c "/usr/sbin/amlabel -f Monthly-LTO7
> 000321L7 slot 1";
>      /usr/sbin/mtx unload 1 0;
>
> work?
>
> During my tests with this, the hardware compression stays disabled
> when I load a new tape.
>
> Thanks!
>
Hey, if it works, its right. ;-)

> On 2018-01-11 10:51 PM, Gene Heskett wrote:
> > 1. rewind the tape.
> > 1a. Do NOT remove tape from drive, or cause it to read the tape
> > other than what I write here until after step 5.
> > 2. read the label out to a 32k file.
> > 3. rewind the tape.
> > 4. Turn the compression off, probably with mt.
> > 5. Immediately re-write that label while the tape is rewound, and
> > the hidden tape id block in front of the label will get rewritten
> > too, with that compression flag off.


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: Amtapetype wrongly reporting compression?

[Jon LaBadie]

On Fri, Jan 12, 2018 at 03:20:51PM -0500, Luc Lalonde wrote:
> Hello Gene,
> 
> Would a variant like this:
> 
>     /usr/sbin/mtx load 1 0;
>     /usr/bin/mt -f /dev/st0 compression 0;
>     /usr/bin/mt -f /dev/st0 setblk 524288;
>     /usr/bin/dd if=/dev/zero of=/dev/st0 bs=512k count=1;
>     su amandabackup -c "/usr/sbin/amlabel -f Monthly-LTO7 000321L7 slot
> 1";
>     /usr/sbin/mtx unload 1 0;
> 
> work?
> 
> During my tests with this, the hardware compression stays disabled when I
> load a new tape.
> 
The critical event is whether it remains disabled after the
amanda program opens the tape drive for reading (or R/W).
All amanda programs that deal with the tape drive do a read
of the tape header before doing anything else.

> Thanks!
> 
> 
> On 2018-01-11 10:51 PM, Gene Heskett wrote:
> > 1. rewind the tape.
> > 1a. Do NOT remove tape from drive, or cause it to read the tape other
> > than what I write here until after step 5.
> > 2. read the label out to a 32k file.
> > 3. rewind the tape.
> > 4. Turn the compression off, probably with mt.
> > 5. Immediately re-write that label while the tape is rewound, and the
> > hidden tape id block in front of the label will get rewritten too, with
> > that compression flag off.
> 
> -- 
> Luc Lalonde, analyste
> -
> Département de génie informatique:
> École polytechnique de MTL
> (514) 340-4711 x5049
> luc.lalo...@polymtl.ca
> -
>>> End of included message <<<

-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: Amtapetype wrongly reporting compression?

[Luc Lalonde]

Hello Gene,

Would a variant like this:

    /usr/sbin/mtx load 1 0;
    /usr/bin/mt -f /dev/st0 compression 0;
    /usr/bin/mt -f /dev/st0 setblk 524288;
    /usr/bin/dd if=/dev/zero of=/dev/st0 bs=512k count=1;
    su amandabackup -c "/usr/sbin/amlabel -f Monthly-LTO7 000321L7 
slot 1";

    /usr/sbin/mtx unload 1 0;

work?

During my tests with this, the hardware compression stays disabled when 
I load a new tape.


Thanks!


On 2018-01-11 10:51 PM, Gene Heskett wrote:

1. rewind the tape.
1a. Do NOT remove tape from drive, or cause it to read the tape other
than what I write here until after step 5.
2. read the label out to a 32k file.
3. rewind the tape.
4. Turn the compression off, probably with mt.
5. Immediately re-write that label while the tape is rewound, and the
hidden tape id block in front of the label will get rewritten too, with
that compression flag off.


--
Luc Lalonde, analyste
-
Département de génie informatique:
École polytechnique de MTL
(514) 340-4711 x5049
luc.lalo...@polymtl.ca
-

Re: Amtapetype wrongly reporting compression?

[Gene Heskett]

On Thursday 11 January 2018 14:43:49 Luc Lalonde wrote:

> Hello Folks,
>
> I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).
>
> We're migrating from LTO5 to LTO7 and I'm getting strange results when
> I run 'amtapetype'.
>
> Here's what I get after roughly 35 hours:
>
> define tapetype LTO7 {
>  comment "Created by amtapetype; compression enabled"
>  length 5874932832 kbytes
>  filemark 1813 kbytes
>  speed 118310 kps
>  blocksize 32 kbytes
> }
>
> Why is it saying that it can write roughly 6Tb if it's supposedly
> hardware compressed?  LTO7 is supposed to give a compression ration of
> 2.5 to 1.

That can vary widely, dependent on the data. If already compressed, don't 
waste your cpu's power trying to gain another 5%, 'tain't worth it.

>
> I've disabled compression... Here's the ouptut of 'tapeinfo:
>
> [root@ulysses ~]# tapeinfo -f /dev/sg12
> Product Type: Tape Drive
> Vendor ID: 'IBM '
> Product ID: 'ULTRIUM-HH7 '
> Revision: 'G9Q1'
> Attached Changer API: No
> SerialNumber: '123456789A'
> MinBlock: 1
> MaxBlock: 8388608
> SCSI ID: 4
> SCSI LUN: 0
> Ready: yes
> BufferedMode: yes
> Medium Type: 0x78
> Density Code: 0x5c
> BlockSize: 32768
> DataCompEnabled: yes
_
> DataCompCapable: yes
> DataDeCompEnabled: yes
> CompType: 0xff
> DeCompType: 0xff
> Block Position: 2
> Partition 0 Remaining Kbytes: -1
> Partition 0 Size in Kbytes: -1
> ActivePartition: 0
> EarlyWarningSize: 0
> NumPartitions: 0
> MaxPartitions: 3
>
> Am I missing something?
>
> Thanks!|

Yes, you wrote that tapes label with compression enabled, so despite your 
turning it off, when the drive scanned that tape to mount it, it found 
the compression was enabled in a hidden header only the drive can see, 
located in front of the label block, so it re-enabled it. BTDT, had fun, 
finally developed a method to turn it off:

1. rewind the tape.
1a. Do NOT remove tape from drive, or cause it to read the tape other 
than what I write here until after step 5. 
2. read the label out to a 32k file.
3. rewind the tape.
4. Turn the compression off, probably with mt.
5. Immediately re-write that label while the tape is rewound, and the 
hidden tape id block in front of the label will get rewritten too, with 
that compression flag off.

6. You must do this with every tape that was previously labeled with 
compression enabled else its a virus that will re-infect every tape 
after the one you missed.

I do not personally believe in using drive compression for the simple 
reason that it hides the true tape capacity from amanda. If you use 
local compression, gzip or ???, amanda counts bytes sent down the cable 
to the drive and this is already compressed data if your dumptype in use 
says to, and can then know pretty precisely how much more data this tape 
can hold. Otherwise its a SWAG*, and we all know just how far off that 
can be under the wrong circumstances.

*SWAG, thats a Scientific Wild Assed Guess.

In any event, I've likely started a flame war over where to do the 
compression, so I'll go get my nomex underwear out.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: Amtapetype wrongly reporting compression?

[Jean-Francois Malouin]

Hi,

I have been using a 2048k blocksize on LTO5 and LTO6 for a few years now
with good results. This is the tapetype that amtapetype generated with a
non-compression LTO6 tape device (without the part-* options, they are
of my own design) using this blocksize:

define tapetype lto6-tapetype {  
# LEOM is not supported for this drive and kernel
comment "Created by amtapetype; compression disabled"
length 2459914240 kbytes
filemark 3227 kbytes
speed 156842 kps
blocksize 2048 kbytes
part-size 200gb 
part-cache-max-size 200gb
part-cache-type disk
part-cache-dir "/holddisk"
}

Maybe it's overkill, 512k might give you the full perf of the drive.
Don't have a LTO7 to play with but I hope this helps!
jf

* Luc Lalonde  [20180111 15:44]:
> Hello Jean-Louis and Debra,
> 
> I'll re-run the 'amtapetype' with 512k blocksize and get back to
> soon (depending on how long it takes)
> 
> Thanks your your help, Luc.
> 
> 
> On 2018-01-11 03:31 PM, Debra S Baddorf wrote:
> >After research, I started using 512k blocks,  3 years ago,  for LTO5 tapes.
> >For LTO7  I might have to research even more, but certainly 512k or bigger.
> >
> >Debra Baddorf
> >Fermilab
> >
> >
> >
> >>On Jan 11, 2018, at 2:09 PM, Jean-Louis Martineau 
> >> wrote:
> >>
> >>Luc,
> >>
> >>amtapetype use speed heuristic to detect if the drive is in compressed
> >>mode, it might not be good for newer drives.
> >>
> >>Why you didn't post the complete amtapetype output? I can't tell you why
> >>the heuristic is bad without those numbers.
> >>
> >>The default block size of 32k was good 15 years ago, but it is really
> >>bad for new drives. You should really think about increasing it a lot.
> >>
> >>Jean-Louis
> >>
> >>On 11/01/18 02:56 PM, Luc Lalonde wrote:
> >>>I sent the wrong output in the last email Here's the correct
> >>>'tapeinfo':
> >>>
> >>>Product Type: Tape Drive
> >>>Vendor ID: 'IBM '
> >>>Product ID: 'ULTRIUM-HH7 '
> >>>Revision: 'G9Q1'
> >>>Attached Changer API: No
> >>>SerialNumber: '123456789A'
> >>>MinBlock: 1
> >>>MaxBlock: 8388608
> >>>SCSI ID: 4
> >>>SCSI LUN: 0
> >>>Ready: yes
> >>>BufferedMode: yes
> >>>Medium Type: 0x78
> >>>Density Code: 0x5c
> >>>BlockSize: 32768
> >>>DataCompEnabled: no
> >>>DataCompCapable: yes
> >>>DataDeCompEnabled: yes
> >>>CompType: 0xff
> >>>DeCompType: 0xff
> >>>BOP: yes
> >>>Block Position: 0
> >>>Partition 0 Remaining Kbytes: -1
> >>>Partition 0 Size in Kbytes: -1
> >>>ActivePartition: 0
> >>>EarlyWarningSize: 0
> >>>NumPartitions: 0
> >>>MaxPartitions: 3
> >>>
> >>>Further, here's my '/etc/stinit.def':
> >>>
> >>>manufacturer="IBM" model = "ULTRIUM-HH7" {
> >>>scsi2logical=1
> >>>can-bsr=1
> >>>auto-lock=1
> >>>two-fms=0
> >>>drive-buffering=1
> >>>buffer-writes
> >>>read-ahead=1
> >>>async-writes=1
> >>>can-partitions=1
> >>>fast-mteom=0
> >>>sysv=1
> >>>timeout=180
> >>>long-timeout=14400
> >>>mode1 blocksize=32768 compression=1
> >>>mode2 blocksize=32768 compression=0
> >>>mode3 disabled=1
> >>>mode4 disabled=1
> >>>
> >>>Sorry, about that!
> >>>
> >>>On 2018-01-11 02:43 PM, Luc Lalonde wrote:
> Hello Folks,
> 
> I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).
> 
> We're migrating from LTO5 to LTO7 and I'm getting strange results
> when I run 'amtapetype'.
> 
> Here's what I get after roughly 35 hours:
> 
> define tapetype LTO7 {
> comment "Created by amtapetype; compression enabled"
> length 5874932832 kbytes
> filemark 1813 kbytes
> speed 118310 kps
> blocksize 32 kbytes
> }
> 
> Why is it saying that it can write roughly 6Tb if it's supposedly
> hardware compressed? LTO7 is supposed to give a compression ration
> of 2.5 to 1.
> 
> I've disabled compression... Here's the ouptut of 'tapeinfo:
> 
> [root@ulysses ~]# tapeinfo -f /dev/sg12
> Product Type: Tape Drive
> Vendor ID: 'IBM '
> Product ID: 'ULTRIUM-HH7 '
> Revision: 'G9Q1'
> Attached Changer API: No
> SerialNumber: '123456789A'
> MinBlock: 1
> MaxBlock: 8388608
> SCSI ID: 4
> SCSI LUN: 0
> Ready: yes
> BufferedMode: yes
> Medium Type: 0x78
> Density Code: 0x5c
> BlockSize: 32768
> DataCompEnabled: yes
> DataCompCapable: yes
> DataDeCompEnabled: yes
> CompType: 0xff
> DeCompType: 0xff
> Block Position: 2
> Partition 0 Remaining Kbytes: -1
> Partition 0 Size in Kbytes: -1
> ActivePartition: 0
> EarlyWarningSize: 0
> NumPartitions: 0
> MaxPartitions: 3
> 
> Am I missing something?
> 
> Thanks!|
> 
> >>
> >>Disclaimer
> >>
> >>This message is the property of CARBONITE, INC. and may contain 
> >>confidential or privileged information.
> >>
> >>If this message has been delivered to you by mistake, then do not copy or 
> >>deliver this message to anyone. Instead, destroy it and notify me by reply 
> 

Re: Amtapetype wrongly reporting compression?

[Luc Lalonde]

Hello Jean-Louis and Debra,

I'll re-run the 'amtapetype' with 512k blocksize and get back to soon 
(depending on how long it takes)


Thanks your your help, Luc.


On 2018-01-11 03:31 PM, Debra S Baddorf wrote:

After research, I started using 512k blocks,  3 years ago,  for LTO5 tapes.
For LTO7  I might have to research even more, but certainly 512k or bigger.

Debra Baddorf
Fermilab




On Jan 11, 2018, at 2:09 PM, Jean-Louis Martineau  
wrote:

Luc,

amtapetype use speed heuristic to detect if the drive is in compressed
mode, it might not be good for newer drives.

Why you didn't post the complete amtapetype output? I can't tell you why
the heuristic is bad without those numbers.

The default block size of 32k was good 15 years ago, but it is really
bad for new drives. You should really think about increasing it a lot.

Jean-Louis

On 11/01/18 02:56 PM, Luc Lalonde wrote:

I sent the wrong output in the last email Here's the correct
'tapeinfo':

Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: no
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Further, here's my '/etc/stinit.def':

manufacturer="IBM" model = "ULTRIUM-HH7" {
scsi2logical=1
can-bsr=1
auto-lock=1
two-fms=0
drive-buffering=1
buffer-writes
read-ahead=1
async-writes=1
can-partitions=1
fast-mteom=0
sysv=1
timeout=180
long-timeout=14400
mode1 blocksize=32768 compression=1
mode2 blocksize=32768 compression=0
mode3 disabled=1
mode4 disabled=1

Sorry, about that!

On 2018-01-11 02:43 PM, Luc Lalonde wrote:

Hello Folks,

I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).

We're migrating from LTO5 to LTO7 and I'm getting strange results
when I run 'amtapetype'.

Here's what I get after roughly 35 hours:

define tapetype LTO7 {
comment "Created by amtapetype; compression enabled"
length 5874932832 kbytes
filemark 1813 kbytes
speed 118310 kps
blocksize 32 kbytes
}

Why is it saying that it can write roughly 6Tb if it's supposedly
hardware compressed? LTO7 is supposed to give a compression ration
of 2.5 to 1.

I've disabled compression... Here's the ouptut of 'tapeinfo:

[root@ulysses ~]# tapeinfo -f /dev/sg12
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
Block Position: 2
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Am I missing something?

Thanks!|



Disclaimer

This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.

If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone. Instead, destroy it and notify me by reply 
e-mail.





--
Luc Lalonde, analyste
-
Département de génie informatique:
École polytechnique de MTL
(514) 340-4711 x5049
luc.lalo...@polymtl.ca
-

Re: Amtapetype wrongly reporting compression?

[Debra S Baddorf]

After research, I started using 512k blocks,  3 years ago,  for LTO5 tapes.
For LTO7  I might have to research even more, but certainly 512k or bigger.

Debra Baddorf
Fermilab 



> On Jan 11, 2018, at 2:09 PM, Jean-Louis Martineau  
> wrote:
> 
> Luc,
> 
> amtapetype use speed heuristic to detect if the drive is in compressed 
> mode, it might not be good for newer drives.
> 
> Why you didn't post the complete amtapetype output? I can't tell you why 
> the heuristic is bad without those numbers.
> 
> The default block size of 32k was good 15 years ago, but it is really 
> bad for new drives. You should really think about increasing it a lot.
> 
> Jean-Louis
> 
> On 11/01/18 02:56 PM, Luc Lalonde wrote:
> > I sent the wrong output in the last email Here's the correct 
> > 'tapeinfo':
> >
> > Product Type: Tape Drive
> > Vendor ID: 'IBM '
> > Product ID: 'ULTRIUM-HH7 '
> > Revision: 'G9Q1'
> > Attached Changer API: No
> > SerialNumber: '123456789A'
> > MinBlock: 1
> > MaxBlock: 8388608
> > SCSI ID: 4
> > SCSI LUN: 0
> > Ready: yes
> > BufferedMode: yes
> > Medium Type: 0x78
> > Density Code: 0x5c
> > BlockSize: 32768
> > DataCompEnabled: no
> > DataCompCapable: yes
> > DataDeCompEnabled: yes
> > CompType: 0xff
> > DeCompType: 0xff
> > BOP: yes
> > Block Position: 0
> > Partition 0 Remaining Kbytes: -1
> > Partition 0 Size in Kbytes: -1
> > ActivePartition: 0
> > EarlyWarningSize: 0
> > NumPartitions: 0
> > MaxPartitions: 3
> >
> > Further, here's my '/etc/stinit.def':
> >
> > manufacturer="IBM" model = "ULTRIUM-HH7" {
> > scsi2logical=1
> > can-bsr=1
> > auto-lock=1
> > two-fms=0
> > drive-buffering=1
> > buffer-writes
> > read-ahead=1
> > async-writes=1
> > can-partitions=1
> > fast-mteom=0
> > sysv=1
> > timeout=180
> > long-timeout=14400
> > mode1 blocksize=32768 compression=1
> > mode2 blocksize=32768 compression=0
> > mode3 disabled=1
> > mode4 disabled=1
> >
> > Sorry, about that!
> >
> > On 2018-01-11 02:43 PM, Luc Lalonde wrote:
> >> Hello Folks,
> >>
> >> I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).
> >>
> >> We're migrating from LTO5 to LTO7 and I'm getting strange results 
> >> when I run 'amtapetype'.
> >>
> >> Here's what I get after roughly 35 hours:
> >>
> >> define tapetype LTO7 {
> >> comment "Created by amtapetype; compression enabled"
> >> length 5874932832 kbytes
> >> filemark 1813 kbytes
> >> speed 118310 kps
> >> blocksize 32 kbytes
> >> }
> >>
> >> Why is it saying that it can write roughly 6Tb if it's supposedly 
> >> hardware compressed? LTO7 is supposed to give a compression ration 
> >> of 2.5 to 1.
> >>
> >> I've disabled compression... Here's the ouptut of 'tapeinfo:
> >>
> >> [root@ulysses ~]# tapeinfo -f /dev/sg12
> >> Product Type: Tape Drive
> >> Vendor ID: 'IBM '
> >> Product ID: 'ULTRIUM-HH7 '
> >> Revision: 'G9Q1'
> >> Attached Changer API: No
> >> SerialNumber: '123456789A'
> >> MinBlock: 1
> >> MaxBlock: 8388608
> >> SCSI ID: 4
> >> SCSI LUN: 0
> >> Ready: yes
> >> BufferedMode: yes
> >> Medium Type: 0x78
> >> Density Code: 0x5c
> >> BlockSize: 32768
> >> DataCompEnabled: yes
> >> DataCompCapable: yes
> >> DataDeCompEnabled: yes
> >> CompType: 0xff
> >> DeCompType: 0xff
> >> Block Position: 2
> >> Partition 0 Remaining Kbytes: -1
> >> Partition 0 Size in Kbytes: -1
> >> ActivePartition: 0
> >> EarlyWarningSize: 0
> >> NumPartitions: 0
> >> MaxPartitions: 3
> >>
> >> Am I missing something?
> >>
> >> Thanks!|
> >>
> >
> 
> 
> Disclaimer
> 
> This message is the property of CARBONITE, INC. and may contain confidential 
> or privileged information.
> 
> If this message has been delivered to you by mistake, then do not copy or 
> deliver this message to anyone. Instead, destroy it and notify me by reply 
> e-mail.
> 

Re: Amtapetype wrongly reporting compression?

[Jean-Louis Martineau]

Luc,

amtapetype use speed heuristic to detect if the drive is in compressed 
mode, it might not be good for newer drives.


Why you didn't post the complete amtapetype output? I can't tell you why 
the heuristic is bad without those numbers.


The default block size of 32k was good 15 years ago, but it is really 
bad for new drives. You should really think about increasing it a lot.


Jean-Louis

On 11/01/18 02:56 PM, Luc Lalonde wrote:
I sent the wrong output in the last email Here's the correct 
'tapeinfo':


Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: no
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Further, here's my '/etc/stinit.def':

manufacturer="IBM" model = "ULTRIUM-HH7" {
scsi2logical=1
can-bsr=1
auto-lock=1
two-fms=0
drive-buffering=1
buffer-writes
read-ahead=1
async-writes=1
can-partitions=1
fast-mteom=0
sysv=1
timeout=180
long-timeout=14400
mode1 blocksize=32768 compression=1
mode2 blocksize=32768 compression=0
mode3 disabled=1
mode4 disabled=1

Sorry, about that!

On 2018-01-11 02:43 PM, Luc Lalonde wrote:

Hello Folks,

I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).

We're migrating from LTO5 to LTO7 and I'm getting strange results 
when I run 'amtapetype'.


Here's what I get after roughly 35 hours:

define tapetype LTO7 {
comment "Created by amtapetype; compression enabled"
length 5874932832 kbytes
filemark 1813 kbytes
speed 118310 kps
blocksize 32 kbytes
}

Why is it saying that it can write roughly 6Tb if it's supposedly 
hardware compressed?  LTO7 is supposed to give a compression ration 
of 2.5 to 1.


I've disabled compression... Here's the ouptut of 'tapeinfo:

[root@ulysses ~]# tapeinfo -f /dev/sg12
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
Block Position: 2
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Am I missing something?

Thanks!|




This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: Amtapetype wrongly reporting compression?

[Luc Lalonde]

I sent the wrong output in the last email Here's the correct 'tapeinfo':

Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: no
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Further, here's my '/etc/stinit.def':

manufacturer="IBM" model = "ULTRIUM-HH7" {
scsi2logical=1
can-bsr=1
auto-lock=1
two-fms=0
drive-buffering=1
buffer-writes
read-ahead=1
async-writes=1
can-partitions=1
fast-mteom=0
sysv=1
timeout=180
long-timeout=14400
mode1 blocksize=32768 compression=1
mode2 blocksize=32768 compression=0
mode3 disabled=1
mode4 disabled=1

Sorry, about that!

On 2018-01-11 02:43 PM, Luc Lalonde wrote:

Hello Folks,

I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).

We're migrating from LTO5 to LTO7 and I'm getting strange results when 
I run 'amtapetype'.


Here's what I get after roughly 35 hours:

define tapetype LTO7 {
    comment "Created by amtapetype; compression enabled"
    length 5874932832 kbytes
    filemark 1813 kbytes
    speed 118310 kps
    blocksize 32 kbytes
}

Why is it saying that it can write roughly 6Tb if it's supposedly 
hardware compressed?  LTO7 is supposed to give a compression ration of 
2.5 to 1.


I've disabled compression... Here's the ouptut of 'tapeinfo:

[root@ulysses ~]# tapeinfo -f /dev/sg12
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
Block Position: 2
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Am I missing something?

Thanks!|



--
Luc Lalonde, analyste
-
Département de génie informatique:
École polytechnique de MTL
(514) 340-4711 x5049
luc.lalo...@polymtl.ca
-

Amtapetype wrongly reporting compression?

[Luc Lalonde]

Hello Folks,

I'm wondering if there's a bug with 'Amtapetype' (version 3.5.1).

We're migrating from LTO5 to LTO7 and I'm getting strange results when I 
run 'amtapetype'.


Here's what I get after roughly 35 hours:

define tapetype LTO7 {
comment "Created by amtapetype; compression enabled"
length 5874932832 kbytes
filemark 1813 kbytes
speed 118310 kps
blocksize 32 kbytes
}

Why is it saying that it can write roughly 6Tb if it's supposedly 
hardware compressed?  LTO7 is supposed to give a compression ration of 
2.5 to 1.


I've disabled compression... Here's the ouptut of 'tapeinfo:

[root@ulysses ~]# tapeinfo -f /dev/sg12
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'G9Q1'
Attached Changer API: No
SerialNumber: '123456789A'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 4
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 32768
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
Block Position: 2
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Am I missing something?

Thanks!|

--
Luc Lalonde, analyste
-
Département de génie informatique:
École polytechnique de MTL
(514) 340-4711 x5049
luc.lalo...@polymtl.ca
-