Re: ye old dumpcycle/runspercycle/tapecycle/runtapes question
On Fri, Feb 28, 2014 at 1:07 AM, Jon LaBadie j...@jgcomp.com wrote: On Thu, Feb 27, 2014 at 07:19:24PM -0500, Michael Stauffer wrote: Amanda 3.3.4 Hi, Another long post from me - thanks to anyone who has time to read it. I've been reading various docs and posts about dumpcycle, runspercycle, tapecycle, runtapes online, but still can't figure out how I should set things up for my needs. I've got: - ca. 30TB of data to backup - I'll try to split this into 500GB or 1TB DLE's, but some will be much easier to leave as 2-4TB DLE's - I'm fine with a level 0 only every 30 days, especially considering a full backup will take 8-10 days. - I've got 1.5TB tapes that hold about 2TB of data with hardware compression from testing - my changer holds 35 tapes First question, will you be using more than 35 tapes. I.e. will you periodically pull some recently used and replace with less recently used tapes? If not, I think you are short on tapes. A full dump will take 15 tapes. You really want a MINIMUM of two full dumps. I prefer more. So 2x15 is two full dumps leaving only 5 tapes for incrementals. I don't think that will be enough. I was thinking of doing a periodic (at the start and then every 3 months) level 0 archive dump to a different set of tapes (probably four sets to retain 1 year's worth of dumps). Then I thought the library in the changer would be fine if it held less than two level 0 dumps at any time. I'd rather just go switch tapes once every 3 months than more often, and have offset archive too. Does that seem reasonable? I'd like to have the changer always hold a level 0 dump and then the set of subseuqent incrementals. So it should take about 15-20 tapes for a level 0 of all DLE's, and then the incrementals over a month should easily fit within 2-4 tapes, judging from experience here. When the next level 0 dump starts, I'd like amanda to use as many of the remaining 10 or so tapes before overwriting tapes from the old level 0 dump (overwriting only tapes of DLE's that have just had a new level 0, of course). (I will periodically do a level 0 dump to a different tape set for offset archiving) Do you plan to let amanda do the scheduling? Or are you going to force her kicking and screaming into the traditional schedule of one monster full dump followed by all incrementals. Then another monster. Blech. I'm fine with Amanda's scheduling. When I do my first round of amdump's though, it will be effectively a monster dump until all DLE's have a level 0. Then I presume amanda can even things out using her own scheduling? How do I setup dumpcycle, runspercycle, tapecycle, runtapes to achieve this? From the docs, it seems I'd want: dumpcycle 30 days runspercycle 15 #15, for running amdump every other day runtapes 2 # to allow for DLE's that can get up to 4TB tapecycle 34 # at least (runspercycle + 1) * runtapes - per docs suggestion # and leave one extra as a spare Is this right? Does this mean that when I run amdump, it will at most write two tapes-worth of DLE's, and then stop? Then the next run will pick up from there? I think so, but would like to make sure. I'm used to the manual paradigm of run a full backup and then do incrementals. But this seems that it will level out to be the same in the end as that? HOWEVER, I'd rather have runtapes at 3 or 4 to minimize tape waste and make it less critical to have evenly-sized DLE's, which will be difficult to maintain. But if runtapes is 3, the recommended value of tapecycle would be = 48, more than my # of tapes. But in practice, 35 should still plenty of tapes to do what I want without overwriting level 0's prematurely. It seems like tapecycle minimum should be more like '# of tapes per full backup + # of tapes for incrementals over dumpcycle + 2 * runtapes', plus one or two as a buffer. The following sentence shows you are still thinking the non-amanda way. The formula in the docs of (runspercycle + 1) * runtapes plays it very safe when you consider many incremental dumps will go to holding disk and be collected onto one tape periodically. There are not going to be a bunch of incrementls to collect onto one tape. Each amdump run will be a mix of level 0's for some DLEs and incrementals for the others. With 15 runspercycle you will AVERAGE 2TB of level 0 plus ??GB of incrementals. But remember you said some DLEs will be 4 or more TB. When they get level 0's you'll need more than 2 tapes. Can I be sure that there will always be some level 0's in a run of amdump? I guess because there's ~30TB of data and 15 runs, so that pretty much guarantees a level 0 every run? If not, or if they're tape-size, I'd like them to go to holding disk so as not to greatly underutilize a tape. I'm going to try to have my largest DLE 2TB - there will be 3 or 4 of those
Re: ye old dumpcycle/runspercycle/tapecycle/runtapes question
On Tue, Mar 04, 2014 at 05:16:05PM -0500, Michael Stauffer wrote: On Fri, Feb 28, 2014 at 1:07 AM, Jon LaBadie j...@jgcomp.com wrote: On Thu, Feb 27, 2014 at 07:19:24PM -0500, Michael Stauffer wrote: Amanda 3.3.4 Hi, Another long post from me - thanks to anyone who has time to read it. I've been reading various docs and posts about dumpcycle, runspercycle, tapecycle, runtapes online, but still can't figure out how I should set things up for my needs. I've got: - ca. 30TB of data to backup - I'll try to split this into 500GB or 1TB DLE's, but some will be much easier to leave as 2-4TB DLE's - I'm fine with a level 0 only every 30 days, especially considering a full backup will take 8-10 days. - I've got 1.5TB tapes that hold about 2TB of data with hardware compression from testing - my changer holds 35 tapes First question, will you be using more than 35 tapes. I.e. will you periodically pull some recently used and replace with less recently used tapes? If not, I think you are short on tapes. A full dump will take 15 tapes. You really want a MINIMUM of two full dumps. I prefer more. So 2x15 is two full dumps leaving only 5 tapes for incrementals. I don't think that will be enough. I was thinking of doing a periodic (at the start and then every 3 months) level 0 archive dump to a different set of tapes (probably four sets to retain 1 year's worth of dumps). Then I thought the library in the changer would be fine if it held less than two level 0 dumps at any time. I'd rather just go switch tapes once every 3 months than more often, and have offset archive too. Does that seem reasonable? I wouldn't be concerned about how many currently in the library. I'd consider how many total I have. Anyone who has administered tape backups for any significant time has a war story. Tapes that seemed to write correctly but a month later were not readable. Damage, physical and environmental. My own involves a large magnet that I did not realize was right next to my tape storage. If you only have one level 0 and it goes bad, you have no backup! I'd like to have the changer always hold a level 0 dump and then the set of subseuqent incrementals. So it should take about 15-20 tapes for a level 0 of all DLE's, and then the incrementals over a month should easily fit within 2-4 tapes, judging from experience here. When the next level 0 dump starts, I'd like amanda to use as many of the remaining 10 or so tapes before overwriting tapes from the old level 0 dump (overwriting only tapes of DLE's that have just had a new level 0, of course). (I will periodically do a level 0 dump to a different tape set for offset archiving) Do you plan to let amanda do the scheduling? Or are you going to force her kicking and screaming into the traditional schedule of one monster full dump followed by all incrementals. Then another monster. Blech. I'm fine with Amanda's scheduling. When I do my first round of amdump's though, it will be effectively a monster dump until all DLE's have a level 0. Then I presume amanda can even things out using her own scheduling? You can ease amanda into your dumpcycle. Add a couple of DLEs with each dump run until they are all added. Avoids the initial monster dump. How do I setup dumpcycle, runspercycle, tapecycle, runtapes to achieve this? From the docs, it seems I'd want: dumpcycle 30 days runspercycle 15 #15, for running amdump every other day runtapes 2 # to allow for DLE's that can get up to 4TB tapecycle 34 # at least (runspercycle + 1) * runtapes - per docs suggestion # and leave one extra as a spare Is this right? Does this mean that when I run amdump, it will at most write two tapes-worth of DLE's, and then stop? Then the next run will pick up from there? I think so, but would like to make sure. I'm used to the manual paradigm of run a full backup and then do incrementals. But this seems that it will level out to be the same in the end as that? HOWEVER, I'd rather have runtapes at 3 or 4 to minimize tape waste and make it less critical to have evenly-sized DLE's, which will be difficult to maintain. But if runtapes is 3, the recommended value of tapecycle would be = 48, more than my # of tapes. But in practice, 35 should still plenty of tapes to do what I want without overwriting level 0's prematurely. It seems like tapecycle minimum should be more like '# of tapes per full backup + # of tapes for incrementals over dumpcycle + 2 * runtapes', plus one or two as a buffer. The following sentence shows you are still thinking the non-amanda way. The formula in the docs of (runspercycle + 1) * runtapes plays it very safe when you consider many incremental dumps will go to holding disk and be
