tuning questions
I just got Amanda 2.5.2p1 up and running and really like it. I have a couple
of questions regarding my configuration--just want to be sure I'm running
things optimally.
I defined a tapetype of HARDDISK with
define tapetype HARDDISK {
length 10 mbytes
}
Last night my backups used 74% of the tape space. What happens when the data I
want to backup exceeds the size I've defined? Should I increase the size in
the definition.
Also, I'm using the the default cycle parameters as follows
dumpcycle 4 weeks
runspercycle 20
tapecycle 25 tapes
with a crontab of
45 0 * * 2-6 /usr/sbin/amdump DailySet1
What if I want backups to occur Monday thru Saturday (actually, early Tuesday
thru early Sunday mornings)? Should I increase runspercycle and tapecycle to
24 and, say, 35? And change my crontab to
45 0 * * 2-7 /usr/sbin/amdump DailySet1
? Basically, I have new data being stored on Saturdays that I would like to
get backed up early Sunday morning.
Thanks,
Brandon
adding tapes to cycle
I am currently using the following default settings: dumpcycle 4 weeks runspercycle 20 tapecycle 25 tapes If wanted to, say, have an 8 week dumpcycle could I simply add and label 25 tapes and change the parameters above to dumpcycle 8 weeks runspercycle 40 tapecycle 50 tapes and expect Amanda to do the right thing? Thanks, Brandon
Re: adding tapes to cycle
- Chris Hoogendyk wrote: > > > Chris Hoogendyk wrote: > > > > > > Brandon Metcalf wrote: > >> I am currently using the following default settings: > >> > >> dumpcycle 4 weeks > >> runspercycle 20 > >> tapecycle 25 tapes > >> > >> If wanted to, say, have an 8 week dumpcycle could I simply add and > >> label 25 tapes and change the parameters above to > >> > >> dumpcycle 8 weeks > >> runspercycle 40 > >> tapecycle 50 tapes > >> > >> and expect Amanda to do the right thing? > > > > Short answer: Yes. Of course. > > > > A bit more: You can change your configuration and then feed it a few > > more tapes at a time until you reach your full number of tapes. As > > long as it has a new labeled tape each time it wants one, it will > > continue to work. When you run out of new labeled tapes, then there > > need to be enough tapes that there will be an old one that can be > > re-used. Typically, you have a few more tapes than specified in > > tapecycle. If runtapes is greater than 1, then you need more -- see: > > http://wiki.zmanda.com/index.php/Tapecycle. > > I want to amend my comments with one caveat. > > While Amanda will do "the right thing," to some extent it will be > hamstrung by your configuration. You will typically only have one full > backup at a time (within a dumpcycle). You will have the constant threat > of having a full for some DLE overwritten, and the incrementals that > relate to that full becoming meaningless, before you have another full. > In addition, a single tape failure can result in multiple DLE's not > being recoverable. > > If you changed your original scheme to > >dumpcycle 2 weeks >runspercycle 10 >tapecycle 25 tapes > > Then you would always have a full backup, and often two. > > If you add 25 tapes and make your new scheme > >dumpcycle 4 weeks >runspercycle 20 >tapecycle 50 tapes > > Then, again, you will always have a full backup. > > I don't know if your intention in stretching out the tapecycle is to > have more coverage, or whether you have too much data for your > network/server/tapes to handle in the shorter cycle. I would be > concerned about redundancy. If you aren't having trouble with too much > data, I would use a scheme like > >dumpcycle 1 week >runspercycle 5 >tapecycle 50 tapes > > or, perhaps, 2, 10 and 50. My motivation behind asking the question was to have more coverage. I certainly don't have a problem with too much data as I'm using virtual tapes on a large NAS RAID array. I was thinking that by increasing the number of tapes and the dumpcycle and runspercycle parameters, I could recover further back in time. It sounds like this may not be the case which is probably due to my lack of understand of exactly how Amanda uses these parameters in its algorithms. Assuming that I don't have a issue with too much data and I would like the following to happen * a full backup once per week * backups 6 days per week * the ability to go back a couple of months Would a config like the following work? dumpcycle 1 week runspercycle 6 tapecycle 75 And could I simply add all of the virtual tapes and once and change the above parameters and be OK? Thanks for the help. Brandon
amanda and smbclient
I realize this is more of a smbclient issue, but thought someone might have an answer as to how files residing on Windows shares can be backed up without ensuring they are not in use. The problem is sendbackup: info BACKUP=/usr/bin/smbclient sendbackup: info RECOVER_CMD=/bin/gzip -dc |/usr/bin/smbclient -xpGf - ... sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end | Domain=[MAI] OS=[Windows Server 2003 3790 Service Pack 1] Server=[Windows Server 2003 5.2] ? NT_STATUS_SHARING_VIOLATION opening remote file ... Is my only solution making sure that all users shutdown the applications that are opening the files causing this problem? Thanks. -- Brandon
Amanda always doing full backup?
I recently changed my config from
dumpcycle 4 weeks
runspercycle 20
tapecycle 25 tapes
to
dumpcycle 2 weeks
runspercycle 12
tapecycle 50 tapes
backups are working fine, but I'm confused by the following output
which is just an example of one device:
$ amadmin DailySet1 find quercus sda8
date hostdisk lv tape or file file part status
2009-04-23 quercus sda8 0 DailySet1-045 -- OK
2009-04-24 quercus sda8 1 DailySet1-055 -- OK
2009-04-25 quercus sda8 0 DailySet1-065 -- OK
2009-04-28 quercus sda8 0 DailySet1-076 -- OK
2009-04-29 quercus sda8 0 DailySet1-086 -- OK
2009-04-30 quercus sda8 0 DailySet1-096 -- OK
2009-05-01 quercus sda8 1 DailySet1-106 -- OK
2009-05-02 quercus sda8 0 DailySet1-116 -- OK
2009-05-05 quercus sda8 0 DailySet1-126 -- OK
2009-05-06 quercus sda8 1 DailySet1-136 -- OK
2009-05-07 quercus sda8 0 DailySet1-146 -- OK
2009-05-08 quercus sda8 1 DailySet1-156 -- OK
2009-05-09 quercus sda8 0 DailySet1-166 -- OK
2009-05-10 quercus sda8 0 DailySet1-176 -- OK
2009-05-12 quercus sda8 0 DailySet1-186 -- OK
2009-05-13 quercus sda8 0 DailySet1-196 -- OK
2009-05-14 quercus sda8 0 DailySet1-206 -- OK
It seems that for the last fews days Amanda is doing a full backup
every night. The filesystem on sda8 contains only the lost+found
directory and it or its contents are not changing.
However, according to the backup report, incrementals are definitely
happening:
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:22
Run Time (hrs:min) 0:57
Dump Time (hrs:min)0:32 0:23 0:10
Output Size (meg)7173.5 6151.5 1022.0
Original Size (meg) 15388.110798.0 4590.0
Avg Compressed Size (%)46.6 57.0 22.3 (level:#disks ...)
Filesystems Dumped 17 15 2 (1:2)
Avg Dump Rate (k/s) 3791.0 4639.5 1804.5
Tape Time (hrs:min)0:13 0:11 0:02
Tape Size (meg) 7173.8 6151.8 1022.1
Tape Used (%) 4.13.50.6 (level:#disks ...)
Filesystems Taped17 15 2 (1:2)
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) 9187.6 9193.1 9154.3
USAGE BY TAPE:
Label Time Size %NbNc
DailySet1-20 0:13 7346016K4.117 0
So, I guess my question has to do with interpreting the output from
'amadmin find'. Do the '0s' indicated full backups?
--
Brandon
Re: Amanda always doing full backup?
m == martin...@zmanda.com writes: m> Small disk are often promoted to try to get a balanced schedule. m> If it is a problem, you can set the 'maxpromoteday' for this disk. Yep, this makes sense based on the notes from the planner. I guess the only reason it would be a problem is if I'm running out of space on which I stored the backups? Also, I guess the planner will quit promoting full dumps once it does enough backups with the new config? -- Brandon
relabel (was Re: Amanda always doing full backup?)
d == dus...@zmanda.com writes: d> On Thu, May 14, 2009 at 10:06 AM, Brandon Metcalf d> wrote: d> > Yep, this makes sense based on the notes from the planner. ??I guess d> > the only reason it would be a problem is if I'm running out of space d> > on which I stored the backups? d> This is one of many reasons that oversubscribing your vtapes is a d> problematic configuration. If your vtapes are oversubscribed, you d> need to pay *very* close attention to what Amanda is doing, and make d> adjustments early, before Amanda finds a vtape with 0kb available. Makes sense. And I've managed to mess up my virtual tapes. Based on your email I thought I would delete some of my virtual tapes using rm(1). I did, but then wanted to add some back. I recreated them with mkdir(1), but when I try to label them I get amlabel: label DailySet1-25 already on a tape I assume this is because /etc/amanda/DailySet1/tapelist already has this label listed. Do I need to relabel the tapes I deleted and then added back? -- Brandon
Re: relabel (was Re: Amanda always doing full backup?)
b == bran...@cedar.geronimoalloys.com writes: b> d == dus...@zmanda.com writes: b> d> On Thu, May 14, 2009 at 10:06 AM, Brandon Metcalf b> d> wrote: b> d> > Yep, this makes sense based on the notes from the planner. ??I guess b> d> > the only reason it would be a problem is if I'm running out of space b> d> > on which I stored the backups? b> d> This is one of many reasons that oversubscribing your vtapes is a b> d> problematic configuration. If your vtapes are oversubscribed, you b> d> need to pay *very* close attention to what Amanda is doing, and make b> d> adjustments early, before Amanda finds a vtape with 0kb available. b> Makes sense. And I've managed to mess up my virtual tapes. Based on b> your email I thought I would delete some of my virtual tapes using b> rm(1). I did, but then wanted to add some back. I recreated them b> with mkdir(1), but when I try to label them I get b> amlabel: label DailySet1-25 already on a tape b> I assume this is because /etc/amanda/DailySet1/tapelist already has b> this label listed. b> Do I need to relabel the tapes I deleted and then added back? Figured it out. I needed to use amrmtape(8). -- Brandon
exclude lists
I have defined an exclude list in a dumptype using exclude list "/etc/amanda/DailySet1/exclude-list" with entries like $ cat /etc/amanda/DailySet1/exclude-list ./dumps ./opt/dell/srvadmin/shared/.ipc I have a couple of questions about the entries. Based on the documentation I've read and the tests I've run, the leading '.' is required which means these are relative paths to the device where these directories live. First, am I correct in thinking the '.' are required? If so, how can Amanda be told to exclude, for example, /dumps on sda1 but still backup /dumps on sda2 other than defining different dumps types with their own exclude list? Thanks. -- Brandon
Re: exclude lists
m == martin...@zmanda.com writes: m> The '.' is required, all excludes are relative to the dle you back up. m> Brandon Metcalf wrote: m> > I have defined an exclude list in a dumptype using m> > m> > exclude list "/etc/amanda/DailySet1/exclude-list" m> > m> > with entries like m> > m> > $ cat /etc/amanda/DailySet1/exclude-list m> > ./dumps m> > ./opt/dell/srvadmin/shared/.ipc m> > m> > I have a couple of questions about the entries. Based on the m> > documentation I've read and the tests I've run, the leading '.' is m> > required which means these are relative paths to the device where m> > these directories live. m> > m> > First, am I correct in thinking the '.' are required? If so, how can m> > Amanda be told to exclude, for example, /dumps on sda1 but still m> > backup /dumps on sda2 other than defining different dumps types with m> > their own exclude list? m> > m> You can't. Seems like a nice feature would be to allow the DLE to be specified in the exclude list. -- Brandon
Re: exclude lists
m == martin...@zmanda.com writes: m> Brandon Metcalf wrote: m> > m> > First, am I correct in thinking the '.' are required? If so, how can m> > m> > Amanda be told to exclude, for example, /dumps on sda1 but still m> > m> > backup /dumps on sda2 other than defining different dumps types with m> > m> > their own exclude list? m> > m> > m> > m> You can't. m> > m> > m> > Seems like a nice feature would be to allow the DLE to be specified in m> > the exclude list. m> > m> The 'exclude list' can be a relative path to the dle. So, you're saying I can specify one dumptype with exclude list "./amanda/DailySet1/exclude-list" or is it exclude list "amanda/DailySet1/exclude-list" and have this file exist on each DLE?
Re: exclude lists
P == paul.bijn...@xplanation.com writes: P> On 2009-05-19 15:35, Brandon Metcalf wrote: P> > m == martin...@zmanda.com writes: P> > P> > m> > P> > m> > P> > m> > Seems like a nice feature would be to allow the DLE to be specified in P> > m> > the exclude list. P> > m> > P> > P> > m> The 'exclude list' can be a relative path to the dle. P> > P> > P> > So, you're saying I can specify one dumptype with P> > P> > exclude list "./amanda/DailySet1/exclude-list" P> > P> > or is it P> > P> > exclude list "amanda/DailySet1/exclude-list" P> > P> > and have this file exist on each DLE? P> Specify it as: P> exclude list optional append ".exclude-list" P> And have a file (optionally) in the directory of each DLE, P> e.g. /.exclude-list (only for the root DLE) P>/var/.exclude-list (only for the /var DLE) P>/space/.exclude-list (only for the /space DLE) P> Have a look in the man page for the exact meaning of "optional" and "append". Thanks for the feedback from everyone. -- Brandon
recover when backup server dies
How do I list all images on a virtual tape using amrestore? For example, if I have the following virtual tape setup $ ls /nas1/amanda/vtapes/DailySet1/slots dataslot11 slot15 slot19 slot22 slot26 slot3 slot6 infoslot12 slot16 slot2 slot23 slot27 slot30 slot7 slot1 slot13 slot17 slot20 slot24 slot28 slot4 slot8 slot10 slot14 slot18 slot21 slot25 slot29 slot5 slot9 I can list an image with $ amrestore -p \ > /nas1/amanda/vtapes/DailySet1/slots/slot1/00013.quercus.sda11.1 blah amrestore: 0: skipping quercus.sda11.20090527.1 But how to I list all images on the virtual tape in slot1? According to the man page for amrestore I can do something like amrestore -p /dev/nrmt9 blah for a real tape. What is the equivalent for a virtual tape? Also, is amrestore the restore tool to use if an Amanda backup server is unavailable and amrecover is not possible? Thanks. -- Brandon
Re: recover when backup server dies
m == martin...@zmanda.com writes: m> Same things as a real changer, you must tell it to use slot1. m> $amtape slot 1 m> $amrestore -p file:/nas1/amanda/vtapes/DailySet1/slots blah Ah. I was leaving off 'file:'. Thanks. -- Brandon
understanding a restore
After selecting the directory I need to restore in amrecover, "list" shows me the following: TAPE DailySet1-09:9 LEVEL 0 DATE 2009-06-05 /path/to/data TAPE DailySet1-17:9 LEVEL 1 DATE 2009-06-14 /path/to/data This seems to imply that since the full backup was done on 2009-06-05 nothing was changed to cause an incremental until 2009-06-14, but I know this isn't the case. Or at least there is nothing that changed that wasn't part of the level 1 backup shown above, but this doesn't seem reasonable given that data changes daily at this location. I'm sure this is a case of me misintepreting what I'm seeing. What's the real story? -- Brandon
Re: understanding a restore
h == hoogen...@bio.umass.edu writes: h> Brandon Metcalf wrote: h> > After selecting the directory I need to restore in amrecover, "list" h> > shows me the following: h> > h> > TAPE DailySet1-09:9 LEVEL 0 DATE 2009-06-05 h> > /path/to/data h> > TAPE DailySet1-17:9 LEVEL 1 DATE 2009-06-14 h> > /path/to/data h> > h> > This seems to imply that since the full backup was done on 2009-06-05 h> > nothing was changed to cause an incremental until 2009-06-14, but I h> > know this isn't the case. Or at least there is nothing that changed h> > that wasn't part of the level 1 backup shown above, but this doesn't h> > seem reasonable given that data changes daily at this location. h> > h> > I'm sure this is a case of me misintepreting what I'm seeing. What's h> > the real story? h> Every level 1 backup backs up all changes since the last level 0. That h> means that the level 1 from 6/14 contains all the changes since 6/5. h> So, if file x changed on 6/6, it will be backed up on every level 1 h> after that until there is another level 0. If enough changes happen, h> Amanda may go to level 2. If things weren't done this way, you would h> have way too many tapes to refer to in order to accomplish a full restore. Makes sense. Thanks for the reply. -- Brandon
incremental restore
I've been looking through the docs, but can't find anything on this. Is there a way with amrecover to restore only the data in the latest level 1 backup? For example, let's say I've extracted all of the data from a level 0 backup, but now I just need to get the data from the latest level 1 since that backup. I can't seem to find a way to do this in amrecover. Thanks. -- Brandon
Re: incremental restore
m == martin...@zmanda.com writes: m> Just enter 'S' (skip) when it ask for the level 0 tape. Beautiful. Thanks. -- Brandon
starting from scratch with vtapes
I had to rebuild the disk array where our vtapes were stored. After I recreate the vtapes is there anything in /etc/amanda/DailySet1/ that I should also remove or recreate to start with a full set of backups? Thanks. -- Brandon
Re: starting from scratch with vtapes
d == dus...@zmanda.com writes: d> On Thu, Jul 16, 2009 at 2:28 PM, Brandon d> Metcalf wrote: d> > I had to rebuild the disk array where our vtapes were stored. ??After I d> > recreate the vtapes is there anything in /etc/amanda/DailySet1/ that I d> > should also remove or recreate to start with a full set of backups? d> If you lost all of your data, you should probably also 'amrmtape' all d> of the tapes Amanda still thinks it has. I already emptied out tapelist, so I guess this isn't going to work. Any manual steps I can take to get going with new level 0 backups? -- Brandon
Re: starting from scratch with vtapes
d == dus...@zmanda.com writes: d> On Thu, Jul 16, 2009 at 3:16 PM, Brandon d> Metcalf wrote: d> > I already emptied out tapelist, so I guess this isn't going to work. d> > Any manual steps I can take to get going with new level 0 backups? d> Blow away all of your old trace logs. Does this mean remove /etc/amanda/DailySet1/curinfo/*, /etc/amanda/DailySet1/curinfo.orig*, /etc/amanda/DailySet1/log.* ? Just trying to understand what is defined as a trace log. -- Brandon
Re: starting from scratch with vtapes
m == martin...@zmanda.com writes: m> amadmin CONF force '*' Thanks everyone for the help. -- Brandon
udp or tcp
I have Amanda working with the default settings for the client:
service amanda
{
disable = no
socket_type = dgram
protocol= udp
wait= yes
user= backup
group = disk
groups = yes
server = /usr/local/libexec/amandad
server_args = -auth=bsd amdump
}
Is it better to use tcp and stream since with udp there is no error
correction or does Amanda build in the error checking at the
application level? Any other settings that would be preferred over
what is shown above?
Thanks.
--
Brandon
bsd works but not bsdtcp
I've changed all of my Amanda hosts to use bsdtcp instead of bsd
authentication, but one isn't cooperating. The client works using
service amanda
{
socket_type = dgram
protocol= udp
wait= yes
user= backup
group = backup
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsd amdump amindexd amidxtaped
disable = no
}
but times out under amcheck when using
service amanda
{
socket_type = stream
protocol= tcp
wait= yes
user= backup
group = backup
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump amindexd amidxtaped
disable = no
}
The only thing interesting in the amcheck debug log is
amcheck-clients: time 30.234: security_seterror(handle=0x805ecf0,
driver=0xb7eae2e0 (BSD) error=timeout waiting for ACK)
Not sure where to look next.
--
Brandon
Re: bsd works but not bsdtcp
m == martin...@zmanda.com writes:
m> Brandon Metcalf wrote:
m> > I've changed all of my Amanda hosts to use bsdtcp instead of bsd
m> > authentication, but one isn't cooperating. The client works using
m> >
m> > service amanda
m> > {
m> > socket_type = dgram
m> > protocol= udp
m> > wait= yes
m> > user= backup
m> > group = backup
m> > groups = yes
m> > server = /usr/lib/amanda/amandad
m> > server_args = -auth=bsd amdump amindexd amidxtaped
m> > disable = no
m> > }
m> >
m> > but times out under amcheck when using
m> >
m> > service amanda
m> > {
m> > socket_type = stream
m> > protocol= tcp
m> > wait= yes
m> > user= backup
m> > group = backup
m> > groups = yes
m> > server = /usr/lib/amanda/amandad
m> > server_args = -auth=bsdtcp amdump amindexd amidxtaped
m> > disable = no
m> > }
m> >
m> wait = no
m> > The only thing interesting in the amcheck debug log is
m> >
m> > amcheck-clients: time 30.234: security_seterror(handle=0x805ecf0,
driver=0xb7eae2e0 (BSD) error=timeout waiting for ACK)
m> >
m> Change your dumptype to use bsdtcp:
m> auth "bsdtcp"
I added this to the global dumptype which all of the types I'm using
refer to and this resulted in all of my clients failing when running
amcheck:
WARNING: cedar: selfcheck request failed: recv error: Connection reset by peer
WARNING: maisrv02: selfcheck request failed: Connection refused
WARNING: quercus: selfcheck request failed: Connection refused
cedar is the host that I was originally having trouble with.
--
Brandon
Re: bsd works but not bsdtcp
m == martin...@zmanda.com writes: m> Do the clients are set for the bsdtcp auth? Yes. Changing wait from yes to no fixed the problem. -- Brandon
