Re: Backups to tape consistently under 60% tape capacity
Re-running amtapetype might be a very good idea. It might point to where the problem isn’t, at least! Do double check your cables. People have found problems in cables which look like reduced throughput. “Mpath” — I don’t know what it is, but could it have changed with your OS upgrade? Wouldn’t hurt to check that the tape driver setting haven’t changed with the OS work ….. but otherwise, it sounds good. Deb On Oct 21, 2014, at 7:43 PM, Tom Robinson tom.robin...@motec.com.au wrote: Hmm, I did check my tape driver settings. When I set the tape library up, I spent a long time getting the driver settings right (on OmniOS) and took copious notes on the settings. My queries reveal that I'm not using compression (which is what I wanted as the vtapes are already compressed). LTO5 native is 1.5T; compressed is 3T. All my tapes are 'newish' (about one year old). The tape unit is the same age. For months I was consistently getting over 110% (highest 117%), then capacity dropped once to 86% and then consistently to about 56% (lowest 42%). Is there a block size issue (2x56=112)? All the weekly dumps are local so network shouldn't be an issue. The tape unit is using redundant SAS connectors. Maybe it's a mpath thing? Should I re-run amtapetype to see what it thinks the 'new' tape capacity is after upgrading the OS? On 22/10/14 10:47, Debra S Baddorf wrote: Yeah, it sure does look like it ought to fit! Could the tapes be dirty and not holding as much any more??? I have no idea if that’s even possible. But it kinds seems like your tapes are saying “I don’t want that much data”.Compression issues? Your tapes were previously getting 117% capacity, and now are only doing 86%. Is that the general summary? Unless somebody else can suggest some network (cable to tape drive?) or system problems which might make the tapes appear smaller than before? Compression issues? Read the original message at the bottom of this email for the original problem complaint. Deb On Oct 21, 2014, at 6:20 PM, Tom Robinson tom.robin...@motec.com.au wrote: Hi Debra, A brilliant motivational speech. Thanks. Well worth the read. In homage, I strongly suggest anyone who hasn't read it to go and do that now. Here it is again for those whose mouse wheel is dysfunctional: http://www.appleseeds.org/Big-Rocks_Covey.htm I will try your suggestions but want to make clear that the virtual tapes you see are the product of a daily run which is disk only. The weekly run puts all those daily dumps onto tape which then leaves the building. So I have both virtual and real tapes. The issues I'm having are in the weekly run (the dump to real tape of a set of virtual tapes). The tapes can be viewed as a bunch of big/little rocks. The total amount of data, however they are stacked, should still fit on a single LTO5 tape (amtapetype told me: length 1483868160 kbytes): $ pwd /data/backup/amanda/vtapes/daily $ du -shc * 512 data 1.0Kinfo 119Gslot1 144Gslot2 115Gslot3 101Gslot4 80G slot5 157Gslot6 189Gslot7 117Gslot8 1019G total Plus: 4.2G/ 212M/export 212M/export/home 212M/export/home/tom So, it looks like I do still have some big rocks to put in first but on the surface of it, it looks like it should all fit in anyway (Did I sum that up wrongly? Looks less than my tape length.). Thanks, Tom (BTW, your last email is not so much a diatribe as a oratory or allocution). On 22/10/14 03:31, Debra S Baddorf wrote: Since nobody else is chiming in, I’ll have another go. I don’t think there IS a dry-run of the taping process, since so much depends on the timing of when a DLE is finished and ready to go to tape, and the physical fitting it onto tape (although, since you have a virtual tape, presumably that isn’t as subject to variation as a real tape might be). I wonder if your root (or boot or sys or whatever you call them) partitions are now just slightly bigger, after your operating system upgrade. That would affect the way things fit into the tape. One has to put the biggest things in first, then the next biggest that will still fit, etc to make the most of the tape size. (see http://www.appleseeds.org/Big-Rocks_Covey.htm for the life motivational analysis type speech that uses this principal too) Yet you, Tom, are telling amanda to finish all the small things first, and then put them onto tape as soon as they are done: dumporder “sssS” taperalgo first I have mine set to finish the big dumps first, so I can put them on the tape first dumporder “BTBTBTBTBT Then — I want amanda to wait until it has a whole tapeful before it starts writing — just so that all those “big pieces” are done and available to be chosen. flush-threshold-dumped100 And THEN —
Re: Backups to tape consistently under 60% tape capacity
Now I have to work out why my tape is reporting as smaller! amtapetype reports
my tape is only half
as big for the same block size...(was 1483868160 is now 743424512). :-/
Checking for FSF_AFTER_FILEMARK requirement
Applying heuristic check for compression.
Wrote random (uncompressible) data at 67415370.8307692 bytes/sec
Wrote fixed (compressible) data at 273874944 bytes/sec
Compression: enabled
Writing one file to fill the volume.
Wrote 761266700288 bytes at 57975 kb/sec
Got LEOM indication, so drive and kernel together support LEOM
Writing smaller files (7612661760 bytes) to determine filemark.
device-property FSF_AFTER_FILEMARK false
define tapetype ULT3580-TD5 {
comment Created by amtapetype; compression enabled
length 743424512 kbytes
filemark 987 kbytes
speed 57975 kps
blocksize 512 kbytes
}
# for this drive and kernel, LEOM is supported; add
# device-property LEOM TRUE
# for this device.
On 23/10/14 03:19, Debra S Baddorf wrote:
Re-running amtapetype might be a very good idea. It might point to where
the problem
isn’t, at least!
Do double check your cables. People have found problems in cables which
look like
reduced throughput. “Mpath” — I don’t know what it is, but could it have
changed
with your OS upgrade?
Wouldn’t hurt to check that the tape driver setting haven’t changed with
the OS work …..
but otherwise, it sounds good.
Deb
On Oct 21, 2014, at 7:43 PM, Tom Robinson tom.robin...@motec.com.au wrote:
Hmm, I did check my tape driver settings. When I set the tape library up, I
spent a long time
getting the driver settings right (on OmniOS) and took copious notes on the
settings. My queries
reveal that I'm not using compression (which is what I wanted as the vtapes
are already compressed).
LTO5 native is 1.5T; compressed is 3T.
All my tapes are 'newish' (about one year old). The tape unit is the same
age.
For months I was consistently getting over 110% (highest 117%), then
capacity dropped once to 86%
and then consistently to about 56% (lowest 42%). Is there a block size issue
(2x56=112)?
All the weekly dumps are local so network shouldn't be an issue. The tape
unit is using redundant
SAS connectors. Maybe it's a mpath thing?
Should I re-run amtapetype to see what it thinks the 'new' tape capacity is
after upgrading the OS?
On 22/10/14 10:47, Debra S Baddorf wrote:
Yeah, it sure does look like it ought to fit!
Could the tapes be dirty and not holding as much any more??? I have no
idea if that’s even possible.
But it kinds seems like your tapes are saying “I don’t want that much
data”.Compression issues?
Your tapes were previously getting 117% capacity, and now are only doing
86%. Is that the general summary?
Unless somebody else can suggest some network (cable to tape drive?) or
system problems which might make
the tapes appear smaller than before? Compression issues? Read the
original message
at the bottom of this email for the original problem complaint.
Deb
On Oct 21, 2014, at 6:20 PM, Tom Robinson tom.robin...@motec.com.au wrote:
Hi Debra,
A brilliant motivational speech. Thanks. Well worth the read. In homage, I
strongly suggest anyone
who hasn't read it to go and do that now. Here it is again for those whose
mouse wheel is
dysfunctional:
http://www.appleseeds.org/Big-Rocks_Covey.htm
I will try your suggestions but want to make clear that the virtual tapes
you see are the product of
a daily run which is disk only. The weekly run puts all those daily dumps
onto tape which then
leaves the building. So I have both virtual and real tapes. The issues I'm
having are in the weekly
run (the dump to real tape of a set of virtual tapes).
The tapes can be viewed as a bunch of big/little rocks. The total amount
of data, however they are
stacked, should still fit on a single LTO5 tape (amtapetype told me:
length 1483868160 kbytes):
$ pwd
/data/backup/amanda/vtapes/daily
$ du -shc *
512 data
1.0Kinfo
119Gslot1
144Gslot2
115Gslot3
101Gslot4
80G slot5
157Gslot6
189Gslot7
117Gslot8
1019G total
Plus:
4.2G/
212M/export
212M/export/home
212M/export/home/tom
So, it looks like I do still have some big rocks to put in first but on
the surface of it, it looks
like it should all fit in anyway (Did I sum that up wrongly? Looks less
than my tape length.).
Thanks,
Tom
(BTW, your last email is not so much a diatribe as a oratory or
allocution).
On 22/10/14 03:31, Debra S Baddorf wrote:
Since nobody else is chiming in, I’ll have another go.
I don’t think there IS a dry-run of the taping process, since so much
depends on the timing
of when a DLE is finished and ready to go to tape, and the physical
fitting it onto tape
(although, since you have a virtual tape, presumably that isn’t as
subject to
