Re: amanda tape use strategy
Chris Loken wrote: Apologies if this has come up before. Couldn't find anything relevant. I'm talking about level 0 dumps (not incrementals). Using ait-3 tapes, GNUTAR (not linux dump), a tape library, 104GB holding disk and setting runtapes greater than 1 (I need to dump several tapes-worth of data). NO software or hardware compression. amanda-2.4.4p1-0.3E Amanda works great for me. These are efficiency questions - not critical problems. a) When backing-up multiple disks from a single client, amanda seems to dump and then write them to tape in order of increasing size. This can be inefficient. e.g. if you have filesystems that are 40GB, 40GB, 55GB and 55GB and write to 100GB tapes, amanda will use 3 tapes when the backup would fit on 2 tapes. b) amanda doesn't seem to know that it will run out of space on a tape. E.g. if I have two 65GB filesystems to dump to a 100GB tape, amanda will write the first and then start writing the second one and keep going until it hits the end of tape. It then puts in the next tape and rewrites which is great but a lot of time was wasted writing a file which wasn't going to fit on the first tape. I'm running amtapetype for myself right now but have been using a tape entry from the amanda faq that sets length 108890 mbytes (but there are two different ones for the same drive and tape; Sony SDX-700C and AIT-3). So, my questions are 1) is what I've described above true (seems to be for me) and 2) is there anything I can do (easily) to change this behaviour? Use the parameter taperalgo largestfit. But beware of some interaction with dumporder, inparallel and the taperalgo: http://marc.theaimsgroup.com/?l=amanda-usersm=106277015010167w=2 About nr2: you can't change it, but it hurts less when the largestfit algoritm is applied. -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amanda tape use strategy
On Tue, Dec 07, 2004 at 12:35:29PM -0500, Chris Loken wrote: Apologies if this has come up before. Couldn't find anything relevant. I'm talking about level 0 dumps (not incrementals). Using ait-3 tapes, GNUTAR (not linux dump), a tape library, 104GB holding disk and setting runtapes greater than 1 (I need to dump several tapes-worth of data). NO software or hardware compression. amanda-2.4.4p1-0.3E Amanda works great for me. These are efficiency questions - not critical problems. a) When backing-up multiple disks from a single client, amanda seems to dump and then write them to tape in order of increasing size. This can be inefficient. e.g. if you have filesystems that are 40GB, 40GB, 55GB and 55GB and write to 100GB tapes, amanda will use 3 tapes when the backup would fit on 2 tapes. b) amanda doesn't seem to know that it will run out of space on a tape. E.g. if I have two 65GB filesystems to dump to a 100GB tape, amanda will write the first and then start writing the second one and keep going until it hits the end of tape. It then puts in the next tape and rewrites which is great but a lot of time was wasted writing a file which wasn't going to fit on the first tape. I'm running amtapetype for myself right now but have been using a tape entry from the amanda faq that sets length 108890 mbytes (but there are two different ones for the same drive and tape; Sony SDX-700C and AIT-3). So, my questions are 1) is what I've described above true (seems to be for me) and 2) is there anything I can do (easily) to change this behaviour? Checking the setting of your taperalgo parameter is in order. A suitable setting for what you want might be largest dump in the holding disk that should fit the remaining tape space. Probably named something like largestfit, but DHMTI. :) Note, this does not affect the order of dumping. Other parameters do that. It only affects which, of the already completed dumps, amanda will choose to tape next. It will not wait around for one that fits. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda tape use strategy
Frank Smith wrote: How difficult would it be to implement a 'best fit' strategy? The number of possibilities is extremely large, and because the tapecapacity is only an approximation, not worth the cost. My AIT-1 tapes hit end of tape between 33 and 34 Gbyte. And rarely there is one which suddenly hits EOT at 31.8 too. One that would collect all the dumps to holdingdisk and then fit them onto the tapes, the functional equivalent of running amdump without a tape and then running amflush. I realize that it would require holdingdisk size to be greater than the total size of the backup, but it would allow for the most efficient use of tape. The largestfit performs very well on my setup: 3 of my 4 AIT-1 tapes are filled to nearly 100% on my archive run. Last week one tape had even 100.6%. Also, does Amanda always try to write another dump image to tape until it hits EOT, even if the size of the dump is greater than the amount of remaining tape? For people that use hardware compression, amanda does not know what remains on the tape. And even in my config, I had to specify the tapelength a little pessimistic, because not all tapes have equal capacity. The capacity of a tape is also affected by the status of the write head and tape quality: the drive reads with one head what it just wrote, and, if it concludes that the written part is not good enough, the tapedrive writes the same block again, up to a number of times (much faster than stopping, rewinding and rewriting); this uses tapecapacity too. That's why my tapelengths are less than the real length. An improvement could be that, when nothing fits, amanda chooses the smallest, which has the best chance of fitting on the uncertain remaining inches (which in my case can be 1 Gbyte!). -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... Are you sure? ... YES ... Phew ... I'm out * ***
Re: amanda tape use strategy
Paul Jon - thanks for the tip about using taperalgo. I wasn't aware of this parameter. Will try it out but I'm not sure how well it will work for me - I'm only dumping from one client, there are a lot of filesystems that are large compared to the tape size and there isn't enough space to keep all the dumps on the holding disk (HD is size of one tape, total space to be dumped is several tapes worth). I've been using maxdumps=1 so turning that up might help (but I'm afraid of the load it will place on the client) as well as playing with dumporder. Increasing the HD size might also help :-) I was kind of hoping that the dumper would be smart enough to dump in an optimal order (the 40GB space, then one of the 55's, then the other 40GB and then the remaining 55GB in my example from below). Presumably amanda has all the relevant info since it does a planning stage first. But while this may be simple in the case of one client and no compression it could be tricky with multiple clients, differing bandwidths, compression etc. According to Paul, I can't avoid the fact that amanda will attempt to write a file to tape even when it knows there isn't enough space (if compression is off this should be very clear). A closer look at the man page confirms this. Oh well. Thanks for the responses! Chris Paul Bijnens wrote: Chris Loken wrote: Apologies if this has come up before. Couldn't find anything relevant. I'm talking about level 0 dumps (not incrementals). Using ait-3 tapes, GNUTAR (not linux dump), a tape library, 104GB holding disk and setting runtapes greater than 1 (I need to dump several tapes-worth of data). NO software or hardware compression. amanda-2.4.4p1-0.3E Amanda works great for me. These are efficiency questions - not critical problems. a) When backing-up multiple disks from a single client, amanda seems to dump and then write them to tape in order of increasing size. This can be inefficient. e.g. if you have filesystems that are 40GB, 40GB, 55GB and 55GB and write to 100GB tapes, amanda will use 3 tapes when the backup would fit on 2 tapes. b) amanda doesn't seem to know that it will run out of space on a tape. E.g. if I have two 65GB filesystems to dump to a 100GB tape, amanda will write the first and then start writing the second one and keep going until it hits the end of tape. It then puts in the next tape and rewrites which is great but a lot of time was wasted writing a file which wasn't going to fit on the first tape. I'm running amtapetype for myself right now but have been using a tape entry from the amanda faq that sets length 108890 mbytes (but there are two different ones for the same drive and tape; Sony SDX-700C and AIT-3). So, my questions are 1) is what I've described above true (seems to be for me) and 2) is there anything I can do (easily) to change this behaviour? Use the parameter taperalgo largestfit. But beware of some interaction with dumporder, inparallel and the taperalgo: http://marc.theaimsgroup.com/?l=amanda-usersm=106277015010167w=2 About nr2: you can't change it, but it hurts less when the largestfit algoritm is applied.
