virtual tape size
I am trying to replace a physical tape changer with a virtual tape changer on NFS mounted hard disk. I started with the virtual tape configuration on the amanda wiki at http://wiki.zmanda.com/index.php/How_To:Set_Up_Virtual_Tapes. The example tapetype definition on the wiki has a tape size of just 3Gb. But the real tapes I'm using have a capacity of 100Gb uncompressed and 500Gb compressed. That is such a huge disparity that I am uncertain as to how to proceed. I have a quota of 2Tb on the remote file system. My data is about 1.2Tb total. I want to keep about 2 weeks of incremental data and and I get about 20Gb of changes daily. So I think the 2Tb quota is about right but I am uncertain as to how to define the tapes. I googled for questions about amanda virtual tape size and I saw a message from someone who set it at over 100Gb. Someone else responded that it was kind of large but it should work. So I tried setting mine at 80. and configured 25 virtual tapes. That's 20Tb of space. But inspite of going 10x over my quota, I've run out of tapes. I'm guessing that 80Gb is too small. But it seems odd that the example on the wiki would define a tape size of 3Gb. 300Gb seems more realistic unless I am missing something. -- --- John G. Heim, 608-263-4189, jh...@math.wisc.edu
Re: virtual tape size
At Mon, 14 Oct 2013 10:20:16 -0500 John G. Heim jh...@math.wisc.edu wrote: I am trying to replace a physical tape changer with a virtual tape changer on NFS mounted hard disk. I started with the virtual tape configuration on the amanda wiki at http://wiki.zmanda.com/index.php/How_To:Set_Up_Virtual_Tapes. The example tapetype definition on the wiki has a tape size of just 3Gb. But the real tapes I'm using have a capacity of 100Gb uncompressed and 500Gb compressed. That is such a huge disparity that I am uncertain as to how to proceed. How 'full' are the tapes you have been using, on average? How many tapes has Amanda been using daily, on average? Peak? Just because your physical tapes have an arbitriarly large capacity (and the compressed capacity is very data dependent and is meaningless if Amanda is writing compressed dumps to the tape) does not mean that Amanda is actually using the whole tape (or even needs to). I have a quota of 2Tb on the remote file system. My data is about 1.2Tb total. I want to keep about 2 weeks of incremental data and and I get about 20Gb of changes daily. So I think the 2Tb quota is about right but I am uncertain as to how to define the tapes. If your *average* daily dump is 20Gb, then your tape size should be about 20Gb (or maybe a little larger), and you will use 1 'tape' a day (on average). When Amanda does a 'full', you will need more 'tapes' (eg your peak usage -- set numtapes to allow for that). I googled for questions about amanda virtual tape size and I saw a message from someone who set it at over 100Gb. Someone else responded that it was kind of large but it should work. So I tried setting mine at 80. and configured 25 virtual tapes. That's 20Tb of space. But inspite of going 10x over my quota, I've run out of tapes. You have to understand: Amanda will only put one day's worth of backups on a given tape. When amdump starts, it will *always* use fresh tapes. There is little point in making the tapes larger than the size of one day's backup. With your daily incrementals being about 20Gb, a tape size much larger than 20Gb is useless. You might be better off with *smaller* tapes and more of them. I'm guessing that 80Gb is too small. But it seems odd that the example on the wiki would define a tape size of 3Gb. 300Gb seems more realistic unless I am missing something. You are missing. 80Gb is probably too *large*. -- Robert Heller -- 978-544-6933 / hel...@deepsoft.com Deepwoods Software-- http://www.deepsoft.com/ () ascii ribbon campaign -- against html e-mail /\ www.asciiribbon.org -- against proprietary attachments
Re: aggressive bump
On Oct 14, 2013, at 5:39 PM, John G. Heim jh...@math.wisc.edu
wrote:
If I understand the concept of bumpting, then because I'm using virtual
tapes, I think I want total bumping to be done -- if there is such a thing.
I don't care at all how many tapes my backup is scattered over.
I inherited my amanda setup from my predecessor in this job and I have never
messed with the bump settings. But we are moving from physical tapes to
virtual tapes. Here are the current values:
bumpsize 500 mb # minimum savings (threshold) to bump level 1
- 2
bumpdays 1# minimum days at each level
bumpmult 4# threshold = bumpsize * bumpmult^(level-1)
I'm a little unclear on how backup levels and tape cycle interact. It would
be fine with me if there was always just one copy of each file on the virtual
tapes.
--
---
John G. Heim, 608-263-4189, jh...@math.wisc.edu
I think you are saying don't DON'T want bumping. Bumping is going down to a
level-2 backup from a level-1, so as to reduce the amount of data being saved.
I think you are saying you only want level 0 backups done. I do this for my
archival tapes, which should not contain level-1 backups and thus be reliant
on
some other tape. I use these settings:
dumpcycle 0# the number of days in the normal ARCHIVE cycle
tapecycle 9000 tapes # some infinite number; always fresh tape for archives
inside my dumptype's I include:
define dumptype dailyNormal {
BDnormal #read in the other file
strategy noinc #nothing but full backups; don't even calculate other
sizes
skip-incr yes
}
Am I hearing you correctly?
Deb Baddorf
Fermilab
Tape Compression: Is it on or off
amanda 3.3.3
OmniOS 151006
Hi,
This question has probably been answered elsewhere so maybe someone can
redirect me to the short
answer. I am still not sure if I'm using compression or not. My apologies in
advance for the long post.
I have an IBM-3580-LTO5 configured through the 'st' driver (and the 'sgen'
driver for the tape robot).
Simply, amtapetype reports that compression is on even though I'm using a
device node that doesn't
do compression:
$ amtapetype -f -t ULT3580-TD5 weekly /dev/rmt/0bn
Checking for FSF_AFTER_FILEMARK requirement
device-property FSF_AFTER_FILEMARK false
Applying heuristic check for compression.
Wrote random (uncompressible) data at 73561559.3650794 bytes/sec
Wrote fixed (compressible) data at 193099093.33 bytes/sec
Compression: enabled
Writing one file to fill the volume.
Wrote 1515988615168 bytes at 73464 kb/sec
Got LEOM indication, so drive and kernel together support LEOM
Writing smaller files (15159885824 bytes) to determine filemark.
define tapetype ULT3580-TD5 {
comment Created by amtapetype; compression enabled
length 1480457632 kbytes
filemark 0 kbytes
speed 73464 kps
blocksize 32 kbytes
}
# for this drive and kernel, LEOM is supported; add
# device-property LEOM TRUE
# for this device.
I had determined that device node /dev/rmt/0bn does NOT use compression but
still amtapetype reports
compression is being used.
Can shed some more light on this?
To determine which device node does/doesn't use compression I did the following:
Reading the man page I have determined that, for my configuration, unless I use
a device node that
is specifically for compression, then compress will not be used.
The 'st' driver uses a bit pattern for the tape device options. Mine is:
Property Value
options 0x1018619
ST_VARIABLE 0x0001
ST_BSF 0x0008
ST_BSR 0x0010
ST_KNOWS_EOD0x0200
ST_UNLOADABLE 0x0400
ST_NO_RECSIZE_LIMIT 0x8000
ST_MODE_SEL_COMP 0x1
ST_WORMABLE x100
AFAICT, compression is set on by ST_MODE_SEL_COMP. But you must understand the
following riddle to
really determine if compression will be used.
From 'man st':
ST_MODE_SEL_COMP
If the ST_MODE_SEL_COMP flag is set, the driver deter-
mines which of the two mode pages the device supports
for selecting or deselecting compression. It first tries
the Data Compression mode page (0x0F); if this fails, it
tries the Device Configuration mode page (0x10). Some
devices, however, may need a specific density code for
selecting or deselecting compression. Please refer to
the device specific SCSI manual. When the flag is set,
compression is enabled only if the c or u device is
used. Note that when the lower 2 densities of a drive
are identically configured and the upper 2 densities are
identically configured, but the lower and upper differ
from each other and ST_MODE_SEL_COMP is set, the m
node sets compression on for the lower density code (for
example, 0x42) and the c and u nodes set compression
on for the higher density (for example, 0x43). For any
other device densities, compression is disabled.
To make more sense on the above you need to know the status of the tape:
mt -f /dev/rmt/0cb config
IBM ULT3580-TD5, IBM ULT3580-TD5 , CFGIBMULT3580TD5;
CFGIBMULT3580TD5 =
2,0x3B,0,0x1018619,4,0x46,0x46,0x58,0x58,3,60,1500,600,16920,780,780,16380;
From the status ouput and reading more of the man page, I see that I have four
densities:
density 0 0x46
density 1 0x46
density 2 0x58
density 3 0x58
In other words, because the lower two compression are configured identically:
density 1 0x46
density 2 0x46
and the upper two compressions are configured identically:
density 3 0x58
density 4 0x58
and they differ from each other:
0x46 != 0x58
and ST_MODE_SEL_COMP is set:
ST_MODE_SEL_COMP = 0x1
then compression is set by using the correct device node.
Lower density compression:
/dev/rmt/0m
/dev/rmt/0mb
/dev/rmt/0mbn
/dev/rmt/0mn
Higher density compression:
/dev/rmt/0c
/dev/rmt/0cb
/dev/rmt/0cbn
/dev/rmt/0cn
or
/dev/rmt/0u
/dev/rmt/0ub
/dev/rmt/0ubn
/dev/rmt/0un
All other device densities have no compression:
/dev/rmt/0
/dev/rmt/0b
/dev/rmt/0bn
/dev/rmt/0h
/dev/rmt/0hb
/dev/rmt/0hbn
/dev/rmt/0hn
/dev/rmt/0l
/dev/rmt/0lb
/dev/rmt/0lbn
/dev/rmt/0ln
/dev/rmt/0n
Hopefully I have understood that.
Regards,
Tom
--
Tom Robinson
IT Manager/System Administrator
MoTeC Pty Ltd
121 Merrindale Drive
Croydon South
3136 Victoria
Australia
T: +61 3 9761 5050
F: +61 3 9761 5051
E: tom.robin...@motec.com.au
signature.asc
Description: OpenPGP digital signature
