Media rotation and backup scheduling
Hi All, We have data which is approximately 220GB in size. We have got an HP Ultrium 960 tape device for our backups with 10 numbers of media (400GB LTO3). I have a few queries about media rotation. I plan to use a dumpcycle of 5 days (5 working days of a week). Suppose I use a media set of 5 media for week1 and use the set of remaining 5 media for week2 and use the media sets alternately for alternate weeks. When I reuse the first media set for week3, will the earlier backup on the media set be overwritten or will the backup of week3 coexist with the earlier backup? The man page of amanda.conf says that The number of tapes in rotation must be larger than the number of tapes required for a complete dump cycle. What does this exactly mean? So in my case if my dumpcycle requires 5 media, do I need to use more than 5 tapes in my tape cycle and why? Kindly provide some suggestions and I have some more queries to ask. Thanks in advance. Yogesh Yahoo! Music Unlimited Access over 1 million songs. http://music.yahoo.com/unlimited
Re: Media rotation and backup scheduling
On Fri, 19 Jan 2007 at 3:05am, Yogesh Hasabnis wrote We have data which is approximately 220GB in size. We have got an HP Ultrium 960 tape device for our backups with 10 numbers of media (400GB LTO3). I have a few queries about media rotation. I plan to use a dumpcycle of 5 days (5 working days of a week). If you're only going to running 'amdump' on weekdays, use dumpcycle=1 week, not 5 days. Otherwise the weekend will confuse amanda. So, for your model: dumpcycle 1 week runspercycle 5 tapecycle 10 Suppose I use a media set of 5 media for week1 and use the set of remaining 5 media for week2 and use the media sets alternately for alternate weeks. When I reuse the first media set for week3, will the earlier backup on the media set be overwritten or will the backup of week3 coexist with the earlier backup? Amanda never appends to tapes -- that's a design decision. So when a tape gets reused, whatever is on it gets overwritten. The man page of amanda.conf says that The number of tapes in rotation must be larger than the number of tapes required for a complete dump cycle. What does this exactly mean? So in my case if my dumpcycle requires 5 media, do I need to use more than 5 tapes in my tape cycle and why? This all has to do with amanda not appending to tapes. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: include not included?
On 2007-01-18 20:22, Jean-Francois Malouin wrote:
Hi,
Got this client (Amanda-2.5.1p2 same as server) for which a few DLEs
with includes don't work, ie the include debug files are empty and the
DLE backups nothing. Turns out that some dirs leading to the DLEs are
700 to some user and so amanda can't stat below the
read-write-exec-user-only directory. First time I encounter this. I
though runtar being suid to root would not be affected by this...
DLE:
gertrude concussion2_jenkai_05
/export_raid02/data-concussion-concussion2/jenkai/McGill/analysis/fMRI/individuals
{
tar-bsdtcpauth-100
include ./hockey
}
In the diskpath .../McGill/... is 700 to some user:
# ls -ld /export_raid02/data-concussion-concussion2/jenkai/McGill
drwxr- 4 jenkai ptitolab 35 Jan 16 13:30
/export_raid02/data-concussion-concussion2/jenkai/McGill/
and the include is empty (after running amcheck):
ls -la
/tmp/amanda-conf9/selfcheck.concussion2_jenkai_05.20070118135050.include
-rw--- 1 amanda disk 0 Jan 18 13:50
/tmp/amanda-conf9/selfcheck.concussion2_jenkai_05.20070118135050.include
If I make that dir 755 (say) all is good and the include debug file
shows ./hockey as it should.
Is this the expected behaviour?
expected and documented :-)
From the amanda.conf man page:
include [ list|file ][[optional][ append ][ string ]+]
...
Note
For globbing to work at all, even the limited single
level, the top level directory of the DLE must be readable
by the Amanda user
The reason is that gnutar has no globbing on the --files-from=F option
(while the --exclude does use patterns). So Amanda builds the list
of files/dirs to include herself from the patterns in disklist.
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: include not included?
* Paul Bijnens [EMAIL PROTECTED] [20070119 10:03]: On 2007-01-18 20:22, Jean-Francois Malouin wrote: Hi, Got this client (Amanda-2.5.1p2 same as server) for which a few DLEs with includes don't work, ie the include debug files are empty and the DLE backups nothing. Turns out that some dirs leading to the DLEs are 700 to some user and so amanda can't stat below the read-write-exec-user-only directory. First time I encounter this. I though runtar being suid to root would not be affected by this... [...snip..] expected and documented :-) From the amanda.conf man page: That will teach me not to install the man pages from 2.5.x still have those from 2.4.5... Plain as the sun in the sky... regards jf include [ list|file ][[optional][ append ][ string ]+] ... Note For globbing to work at all, even the limited single level, the top level directory of the DLE must be readable by the Amanda user The reason is that gnutar has no globbing on the --files-from=F option (while the --exclude does use patterns). So Amanda builds the list of files/dirs to include herself from the patterns in disklist. -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... Are you sure? ... YES ... Phew ... I'm out * *** -- °
Sony AIT5 tapetype
I have a brand new Sony LIB162/A5BBLE -- tape changer with one AIT5 tape
drive. AIT5 just started shipping last fall, so I've had to do this
without any examples turning up in google. Therefore, I thought I would
toss it back to the list in case anyone else can use it.
I just finished using amtapetype to whack the drive overnight, and it
came up with the following results (I modified the comments and replaced
unknown-tapetype):
# /usr/local/sbin/amtapetype -f /dev/rmt/1n -e 400G
Writing 16384 Mbyte compresseable data: 682 sec
Writing 16384 Mbyte uncompresseable data: 682 sec
Estimated time to write 2 * 409600 Mbyte: 34100 sec = 9 h 28 min
wrote 12451840 32Kb blocks in 95 files in 16323 seconds (short write)
wrote 12451840 32Kb blocks in 190 files in 16335 seconds (short write)
define tapetype SONY-AIT5 {
comment SONY AIT5 8mm tape drive
# data provided by Chris Hoogendyk [EMAIL PROTECTED]
# produced by whacking it for 10 hrs or so with amtapetype
# on a Sun E250 with a Dual Ultra320 LVD SCSI PCI card
length 389120 mbytes
filemark 0 kbytes
speed 24401 kps
}
To get the tape drive working, I had to go to Sony tech support. They
hadn't released the unix configuration guide with the AIT5 updates yet.
But, they gave me the details in an email.
I'm working on a Sun E250 running Solaris 9.
I edited /kernel/drv/st.conf and added:
tape-config-list=
SONYSDX-1100, SONY AIT5 8mm,
SONY_AIT;
SONY_AIT = 2, 0x36,0, 0x19639, 4, 0x00, 0x00, 0x00, 0x00,
0, 60, 900, 900, 1500, 1200, 1200, 18000;
Of course, that requires a `touch /reconfigure` and `reboot`.
Then, I was able to do `mt -f /dev/rmt/1n status` and successfully run
ufsdump.
--
To get the changer working, I edited /kernel/drv/sgen.conf and added:
device-type-config-list=changer;
and uncommented the whole block of lines that had the form:
name=sgen class=scsi target=0 lun=0;
Of course, again, `touch /reconfigure` and `reboot`.
Then I downloaded and installed mtx (from sunfreeware as
mtx-1.2.17-sol9-sparc-local.gz). Checked the man pages, and was able to
do `/usr/local/sbin/mtx -f /dev/scsi/changer/c7t0d0 status`.
That worked and I could also move tapes around, load them, unload them,
etc. I could see bar codes as well.
I also have sonytape software that I downloaded from sony support.
Haven't used it yet. But it is supposed to handle things like firmware
upgrades. I used the web interface to change the password, because
scrolling through the entire ascii character set on the front panel was
just too much. Then I disconnected it. Command line from the server
should be totally adequate.
I haven't got amanda configured and running yet.
So I may still have some questions for the group.
But, I thought I would toss this information up on the list just in case
anyone else can make use of it.
---
Chris Hoogendyk
-
O__ Systems Administrator
c/ /'_ --- Biology Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~ - University of Massachusetts, Amherst
[EMAIL PROTECTED]
---
Erdös 4
Re: include not included?
On Fri, Jan 19, 2007 at 04:02:46PM +0100, Paul Bijnens wrote:
On 2007-01-18 20:22, Jean-Francois Malouin wrote:
Hi,
Got this client (Amanda-2.5.1p2 same as server) for which a few DLEs
with includes don't work, ie the include debug files are empty and the
DLE backups nothing. Turns out that some dirs leading to the DLEs are
700 to some user and so amanda can't stat below the
read-write-exec-user-only directory. First time I encounter this. I
though runtar being suid to root would not be affected by this...
DLE:
gertrude concussion2_jenkai_05
/export_raid02/data-concussion-concussion2/jenkai/McGill/analysis/fMRI/individuals
{
tar-bsdtcpauth-100
include ./hockey
}
In the diskpath .../McGill/... is 700 to some user:
# ls -ld /export_raid02/data-concussion-concussion2/jenkai/McGill
drwxr- 4 jenkai ptitolab 35 Jan 16 13:30
/export_raid02/data-concussion-concussion2/jenkai/McGill/
and the include is empty (after running amcheck):
ls -la
/tmp/amanda-conf9/selfcheck.concussion2_jenkai_05.20070118135050.include
-rw--- 1 amanda disk 0 Jan 18 13:50
/tmp/amanda-conf9/selfcheck.concussion2_jenkai_05.20070118135050.include
If I make that dir 755 (say) all is good and the include debug file
shows ./hockey as it should.
Is this the expected behaviour?
expected and documented :-)
From the amanda.conf man page:
include [ list|file ][[optional][ append ][ string ]+]
...
Note
For globbing to work at all, even the limited single
level, the top level directory of the DLE must be readable
by the Amanda user
Does the man page description need a bit of touch up?
Jean-Francois was not using globbing (no metachars) but simply ./hockey.
So it appears the entire include mechanism does not work unless the
directories are readable. Also, I suspect that execute permission is
needed as well.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Sony AIT5 tapetype
On Fri, 19 Jan 2007 at 11:01am, Chris Hoogendyk wrote
define tapetype SONY-AIT5 {
comment SONY AIT5 8mm tape drive
# data provided by Chris Hoogendyk [EMAIL PROTECTED]
# produced by whacking it for 10 hrs or so with amtapetype
# on a Sun E250 with a Dual Ultra320 LVD SCSI PCI card
length 389120 mbytes
filemark 0 kbytes
speed 24401 kps
}
I'm out of the AIT loop -- what's the rated speed? 24MB/s doesn't seem
all that fast (then again, I'm used to LTO3), and I wonder if you could
increase that with some tweaking.
--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University
Re: Sony AIT5 tapetype
Joshua Baker-LePain wrote:
On Fri, 19 Jan 2007 at 11:01am, Chris Hoogendyk wrote
define tapetype SONY-AIT5 {
comment SONY AIT5 8mm tape drive
# data provided by Chris Hoogendyk [EMAIL PROTECTED]
# produced by whacking it for 10 hrs or so with amtapetype
# on a Sun E250 with a Dual Ultra320 LVD SCSI PCI card
length 389120 mbytes
filemark 0 kbytes
speed 24401 kps
}
I'm out of the AIT loop -- what's the rated speed? 24MB/s doesn't
seem all that fast (then again, I'm used to LTO3), and I wonder if you
could increase that with some tweaking.
The rated speed is 24MB/s. I was pleased that my overall setup was tuned
to achieve that, and there were no bottlenecks or gotchas causing
reduced speed or stops starts. I've set it up with two 300G Seagate
disks for spooling, so I can back up all my servers at once and push
them to tape at maximum speed once they are done.
I expect my tweaking to come in the network area. I have a quad 100
network card in the backup server. We'll see how things work and
consider trunking and a backside network if necessary. I only have one
server that has been backed up over the network in the past (it didn't
have a dds/3 of its own), and I did it with ufsdump piped through ssh to dd.
Do you achieve 80MB/s on your LTO3?
---
Chris Hoogendyk
-
O__ Systems Administrator
c/ /'_ --- Biology Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~ - University of Massachusetts, Amherst
[EMAIL PROTECTED]
---
Erdös 4
Re: Sony AIT5 tapetype
On Fri, 19 Jan 2007 at 3:10pm, Chris Hoogendyk wrote Do you achieve 80MB/s on your LTO3? The numbers in the amdump reports range from 55-70MB/s. And that's running backups to both drives in my library. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
-auth vs 2.4.4p1 clients 2.5.1p2 server
What kind of configuration do I need, now that I've upgraded my server to 2.5.1p2? Most of the clients are 2.4.something and are no longer able to connect to do a recover, although dumps work fine. The server is itself a client. It can do a recover now, but if we change something I'll need to make this work too. Oh, and one new client is at 2.5.something. All are backing up fine, but what can I do about recover connecting? I got mixed versions of clients, I guess! Deb Baddorf
amidxtape req'd on clients?
My clients seem to fail at doing recovers unless I have the amidxtaped process installed (and in inetd.conf) on the client. Yet doing an install as client-only doesn't install this piece. I wind up doing server installs on most of my clients. What am I doing wrong? I gather that amidxtaped isn't supposed to be needed on the client, but I can't make do without it. Deb Baddorf
Re: -auth vs 2.4.4p1 clients 2.5.1p2 server
Since you are using the old amindexd/amidxtaped protocol, you must configure them in your server xinetd the same way you did for your amanda 2.4.5 server. Deb Baddorf wrote: What kind of configuration do I need, now that I've upgraded my server to 2.5.1p2? Most of the clients are 2.4.something and are no longer able to connect to do a recover, although dumps work fine. The server is itself a client. It can do a recover now, but if we change something I'll need to make this work too. Oh, and one new client is at 2.5.something. All are backing up fine, but what can I do about recover connecting? I got mixed versions of clients, I guess! Deb Baddorf
Re: amidxtape req'd on clients?
Tell amrecover to connect to your tape server with the -s and -t options. You don't need to install amidxtaped on the client Deb Baddorf wrote: My clients seem to fail at doing recovers unless I have the amidxtaped process installed (and in inetd.conf) on the client. Yet doing an install as client-only doesn't install this piece. I wind up doing server installs on most of my clients. What am I doing wrong? I gather that amidxtaped isn't supposed to be needed on the client, but I can't make do without it. Deb Baddorf
Re: Tape errors, hardware? (SOLVED)
Just for the archives, evidently the inability to write a filemark is a known failure mode on the AIT-3 drives (affecting about 1% of the drives, according to the library support folks). However, the really bizarre thing is that it can be fixed with a firmware update that can be installed via a utility downloadable from the Sony web site. Frank Frank Smith wrote: Just trying to verify that I'm having an actual hardware error. Backups on one tape server (that's been in use for years) failed with the following: taper: tape archive03 kb 0 fm 0 writing filemark: Input/output error taper: retrying q42:/d5/backups/oracle/Dmp.0 on new tape: [writing filemark: Input/output error] taper: tape archive04 kb 1796480 fm 1 writing file: Input/output error and everything remained on the holdingingdisk. From using dd I can see that Amanda is successfully updating the date in the header block. I can successfully run amlabel on the tapes, and I can use tar to write and read a tar file to/from the tape. However, 'mt eof' fails, and gives the same error in the system logs as when Amanda runs: kernel: st1: Error with sense data: 6st1: Current: sense key: Medium Error kernel: Additional sense: Write error kernel: Info fld=0x1 Is it definitely a drive failure when data can be written but not EOF marks, or is there something else I should check before replacing the drive? Thanks, Frank -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
