Re: This is retarded.
On Sat, Sep 03, 2005 at 11:00:14AM -0400, Jon LaBadie wrote: > Would you point me to your other submissions so that I can check into > what happend and why they were mishandled. Just from the patches I have laying around 1. A build fix for Windows/cygwin builds so that SYSTEMROOT was part of the path. I was posting that fix to the list for 2 years which new patch releases were coming out without the fix. It didn't work in cygwin without it. 2. A patch for Solaris so that it would properly compile/install in /opt (given the right --prefix options) instead of assuming /usr/local paths. Then I remember significant questions about EOT handling and willingness to write a taper that would deal with this appropriately, as well as trying to get some insight into the transport mechanism so I could rewrite it to use SSH as a transport. Both of these issues were blown off rather than saying "well best of luck, the things you need to know are..." When you get blown off on something deep in the core of the code, you don't spend effort trying to make it work. Notice the total failure of amanda v5. They tried anyway, and failed. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Sat, Sep 03, 2005 at 09:59:53AM -0400, Jon LaBadie wrote: > Discard a perfectly valid dump? Possibly the one and only copy of some data? > It still could be used by a SysAdmin in a number of ways. For example, it > could be manually taped or stored in some manner; it could be used for amanda > restore/recovery from the holding disk. I wouldn't assume, nor want, amanda > to discard it under the conditions you encountered. How to deal with the > unexpected situation should be the SA's decision. Valid points. Then can we address why it was backed up in the first place? I do not want to have to spend a few hours every night trying to make sure every backup will fit on a tape. If a sysadmin screws up his excludes, or loads 20gb extra on his system, I want a report in the morning to deal with (get the sysadmin in question to alter his excludes or his DLEs) and not 3 blank tapes and something in the dumpspace I have to manually remove. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Fri, Sep 02, 2005 at 02:52:45PM -0700, Joe Rhett wrote: > > I would offer a patch, but every patch I've offered in the past has been > ignored. Back when I did have a good attitude about this project, and > thought it was really good. On Fri, Sep 02, 2005 at 02:45:46PM -0700, Joe Rhett wrote: > On Thu, Sep 01, 2005 at 05:18:39PM -0400, Joshua Baker-LePain wrote: > > > See above. And please, please check your attitude at the door. If you > > want to make amanda better, join the -hackers list and get hacking. > > I did. I am still on -hackers. I spent the last 4 years working on real > problems and submitting patches for them. Only to see them ignored. Not > rejected for any given reason, just ignored. So now I'm tired of being > ignored and frustrated by the long-standing and oft-reported bugs in tape > handling. What do you expect? Joe, You have made similar statements a number of times. It disturbs be that something like that would happen. Not being one of the amanda developers, I certainly was not responsible for the slight, but that should not happen and it is quite distinct from the response I encountered when I suggested any patches. Things were accepted, rejected, and one was greatly enhanced by a better coder. But none were ignored. So I tried to look into what happened to your submissions. I keep an personal archive of -users and -hackers though I can't claim they have 100% of all articles posted they have the vast majority. My -hackers archive goes back to mid-1999 and has nearly 3000 articles. Here is what I found posted under your name. Nov 2000 7 postsSuggested changes to chg-zd-mtx Jun 2003 4 postsDiscussion of Win32 client Oct 2003 1 post Question about using amrecover Jan 2004 4 postsDiscussion of chg-mtx Feb 2004 1 post Question about amanda development status I only found the one submission for enhancements to chg-zd-mtx. That submission was accepted and your contribution is attributed in the code. Would you point me to your other submissions so that I can check into what happend and why they were mishandled. Thanks, jon -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: This is retarded.
On Fri, Sep 02, 2005 at 02:38:59PM -0700, Joe Rhett wrote: > > > > You know you have a DLE that is too big to tape and that amanda > > > > does not handle it well. Isn't it time to stop wasting time > > > > and tape and split the DLE? > > > On Thu, Sep 01, 2005 at 09:51:15AM -0700, Joe Rhett wrote: > > > Ah, I'm sorry! I should not assume that amanda can do it's own math. I > > > must sit down with a calculator every night and do my own estimates and > > > make this all work? > > > Jon, this is a fairly serious problem with amanda. Why are you blaming > > > the > > > reporter instead of trying to fix the problem? > > On Thu, Sep 01, 2005 at 01:14:16PM -0400, Jon LaBadie wrote: > > No, I was commenting on the futility of trying again something that failed > > once unexpectedly, you knew it was going to fail again, and have known from > > 5 years of amanda experience that that particular situation would always > > fail. > > THAT was already done. Several days ago. What I didn't stop to do was > delete the amount to flush because I assumed amanda would do the math and > realize it couldn't back it up and then discard it. I didn't expect taper > to blindly attempt something guaranteed to fail. Discard a perfectly valid dump? Possibly the one and only copy of some data? It still could be used by a SysAdmin in a number of ways. For example, it could be manually taped or stored in some manner; it could be used for amanda restore/recovery from the holding disk. I wouldn't assume, nor want, amanda to discard it under the conditions you encountered. How to deal with the unexpected situation should be the SA's decision. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: This is retarded.
On Fri, Sep 02, 2005 at 08:50:21PM -0400, Joshua Baker-LePain wrote: > On Fri, 2 Sep 2005 at 3:00pm, Joe Rhett wrote > > > On Thu, Sep 01, 2005 at 05:06:37PM -0400, Joshua Baker-LePain wrote: > > > *You* are the only one who can really identify the problem. Are you > > > using > > > server-side or client-side estimates? If the latter, look at the > > > sendsize*debug for the 53GB dump to see what amanda thought the size > > > would > > > be at estimate time. > > > > See the attached log except. Amanda knew it was bigger than a tape, but > > backed it up anyway. > > I'd be intereseted in seeing the sendsize*debug from the client and/or > the relevant planner line from the amdump.N file. Amanda didn't do a > level 0 b/c it knew that was bigger than tapelength, but then the level 1 > turned out to be even bigger than the estimate for the level 0. ?? Here's the relevant lines from amdump's log: planner: time 0.084: setting up estimates for customer-plat1:/ customer-plat1:/ overdue 82 days for level 0 setup_estimate: customer-plat1:/: command 0, options: last_level 1 next_level0 -82 level_days 9 getting estimates 0 (10) 1 (-1) -1 (-1) planner: time 864.871: got result for host customer-plat1 disk /: 0 -> 53227860K, 1 -> 53195890K, -1 -> -1K pondering customer-plat1:/... next_level0 -82 last_level 1 (due for level 0) (picking inclevel for degraded mode) pick: size 53195890 level 1 days 9 (thresh 204800K, 1 days) INITIAL SCHEDULE (size 145819677): customer-plat1 / pri 83 lev 0 size 53227860 ...etc DELAYING DUMPS IF NEEDED, total_size 145819677, tape length 241282048 mark 32 delay: customer-plat1 / 20050825 0 [dump larger than tape, 53227860 KB, full dump delayed] now at level 1 delay: Total size now 145787707. DUMP customer-plat1 34cbfe811f0100 / 20050825 83 1 2005:5:28:8:23:38 53195890 7092785 The only remaining lines referecing it are the flush, then taper, then taper-tryagain... repeat twice more. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Fri, 2 Sep 2005 at 3:00pm, Joe Rhett wrote > On Thu, Sep 01, 2005 at 05:06:37PM -0400, Joshua Baker-LePain wrote: > > *You* are the only one who can really identify the problem. Are you using > > server-side or client-side estimates? If the latter, look at the > > sendsize*debug for the 53GB dump to see what amanda thought the size would > > be at estimate time. > > See the attached log except. Amanda knew it was bigger than a tape, but > backed it up anyway. I'd be intereseted in seeing the sendsize*debug from the client and/or the relevant planner line from the amdump.N file. Amanda didn't do a level 0 b/c it knew that was bigger than tapelength, but then the level 1 turned out to be even bigger than the estimate for the level 0. ?? -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: This is retarded.
On Thu, Sep 01, 2005 at 01:14:16PM -0400, Jon LaBadie wrote: > No, I was commenting on the futility of trying again something that failed > once unexpectedly, you knew it was going to fail again, and have known from > 5 years of amanda experience that that particular situation would always fail. Perhaps I am missing your point. Should I be going around and making sure of amanda's dependancies? Or just maybe, perhaps, you could patch amanda to check for sanity in what it is attempting to do? I would offer a patch, but every patch I've offered in the past has been ignored. Back when I did have a good attitude about this project, and thought it was really good. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Fri, Sep 02, 2005 at 10:56:55AM +1000, Jamie Wilkinson wrote: > You seem to be under the impression that free software is going to be fixed > by other people. Not at all. All the other free software projects I've worked on we submit patches and they are either accepted or rejected with a reason. Not ignored. Pretty much everyone that has at times tried to improve amanda over the years has come here, done a whole bunch of work and then watched it come to nothing because the maintainers just don't care. Notice that pretty much every original amanda maintainer is now working on other backup projects like bacula. > Rather than whinge on mailing lists, our admins actively experiment and test > changes to our backup system. No-one else is going to do it for us. Yes, > amanda has plenty of shortcomings, but complaining about it and stamping > your feet is not going to fix it. I did. I have done. The questions I have about the code segments get ignored. The patches I send to fix clear and repeatable problems are ignored. And so after 4 years of wasting my time, I get irritated. The truth is that I've done a lot of patches to amanda. Small patches to external components only, because questions about the core code modules are ignored. But I don't see your name on any significant patches, so who are you bitching about? It's easy to "whine and complain". It's even easier to stand back and call people whiners, when you aren't doing any work yourself. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Thu, Sep 01, 2005 at 05:18:39PM -0400, Joshua Baker-LePain wrote: > On Thu, 1 Sep 2005 at 10:10am, Joe Rhett wrote > > > On Thu, Sep 01, 2005 at 08:52:29AM -0400, Joshua Baker-LePain wrote: > > > This can happen for a variety of reasons. Sometimes a DLE grows between > > > estimate time and dump time. If you're using software compression and > > > it's a relatively new DLE (or the nature of the data changes), amanda's > > > estimate of the compressed size can be inaccurate. > > > > By 20gb? And no, we're not using compression anywhere so that's not it. > > I've had DLEs grow by over 100GB between estimate time and dump time. > Investigate this. Only you can figure out why that 53GB dump happened in > the first place. I have already said that, that isn't it. In fact, the size of this filesystem has only gone down over the last 10 months. > > > It's *impossible* to > > > anticipate every situation, although I believe amanda does a pretty good > > > job. > > > > Huh? Impossible to anticipate that 53gb can't be written to a 30gb tape? > > Amanda figures backups are really important, and really wants to get them > on tape. It knows that tapelengths can be estimates (especially if one is > using hardware compression), and figures it'll give it a shot even if it > doesn't look like it'll fit. By trying to write nearly twice the size of a tape to a tape? C'mon, be serious here. > See above. And please, please check your attitude at the door. If you > want to make amanda better, join the -hackers list and get hacking. I did. I am still on -hackers. I spent the last 4 years working on real problems and submitting patches for them. Only to see them ignored. Not rejected for any given reason, just ignored. So now I'm tired of being ignored and frustrated by the long-standing and oft-reported bugs in tape handling. What do you expect? -- Joe Rhett senior geek meer.net
Re: This is retarded.
> > > You know you have a DLE that is too big to tape and that amanda > > > does not handle it well. Isn't it time to stop wasting time > > > and tape and split the DLE? > On Thu, Sep 01, 2005 at 09:51:15AM -0700, Joe Rhett wrote: > > Ah, I'm sorry! I should not assume that amanda can do it's own math. I > > must sit down with a calculator every night and do my own estimates and > > make this all work? > > Jon, this is a fairly serious problem with amanda. Why are you blaming the > > reporter instead of trying to fix the problem? On Thu, Sep 01, 2005 at 01:14:16PM -0400, Jon LaBadie wrote: > No, I was commenting on the futility of trying again something that failed > once unexpectedly, you knew it was going to fail again, and have known from > 5 years of amanda experience that that particular situation would always fail. THAT was already done. Several days ago. What I didn't stop to do was delete the amount to flush because I assumed amanda would do the math and realize it couldn't back it up and then discard it. I didn't expect taper to blindly attempt something guaranteed to fail. -- Joe Rhett senior geek meer.net
Re: This is retarded.
> On Thu, 1 Sep 2005 at 9:51am, Joe Rhett wrote > > Ah, I'm sorry! I should not assume that amanda can do it's own math. I > > must sit down with a calculator every night and do my own estimates and > > make this all work? > > > > Jon, this is a fairly serious problem with amanda. Why are you blaming the > > reporter instead of trying to fix the problem? You haven't even attempted > > to identify the problem, you're just being busy shooting the messenger. On Thu, Sep 01, 2005 at 05:06:37PM -0400, Joshua Baker-LePain wrote: > *You* are the only one who can really identify the problem. Are you using > server-side or client-side estimates? If the latter, look at the > sendsize*debug for the 53GB dump to see what amanda thought the size would > be at estimate time. It knew the size would be 53gb. > > Nice approach. > > When you come in with the attitude you are, expect nothing else. Let me get this right. 1. Ignore 7 years of people telling you this doesn't work. 2. Ignore repeated good-faith attempts to fix the system 3. Then blame the messenger when repeatable bugs are found. Who exactly has the attitude problem? After years of trying in good faith to get these problems fixed, one learns that a million polite words get nowhere with the current maintainers. -- Joe Rhett senior geek meer.net
Re: This is retarded.
Thankyou for both copies of your courteous email! This one time, at band camp, Joe Rhett wrote: >On Thu, Sep 01, 2005 at 09:46:43AM +1000, Jamie Wilkinson wrote: >> There's a patch from John Stange that I don't believe has been committed to >> CVS, but it takes care of splitting dumps and spanning tapes. Grep for it >> on the amanda-hackers list. Doesn't fix #2, but should take care of #1 for >> you. > >Because it's a patch that's been floating around for 2+ years and isn't >being accepted into the mainline code for reasons unknown. Support for >amanda is really only done by like 2 people, and applying this patch would >make it just John, and there are many things John doesn't know or can't do. I do apologise for failing to read your mind properly. Had I known that you knew already about John Stange's patch and were unwilling to try it out I wouldn't have bothered mentioning it. >The things about backups is that they aren't a place to play games. No shit. >I do have a test backup system, but it's focused on software that does >recognize EOT and deal with it appropriately... You seem to be under the impression that free software is going to be fixed by other people. Rather than whinge on mailing lists, our admins actively experiment and test changes to our backup system. No-one else is going to do it for us. Yes, amanda has plenty of shortcomings, but complaining about it and stamping your feet is not going to fix it. Perhaps you already knew that, but my mindreader is acting up.
Re: This is retarded.
On Thu, 1 Sep 2005 at 10:10am, Joe Rhett wrote > On Thu, Sep 01, 2005 at 08:52:29AM -0400, Joshua Baker-LePain wrote: > > This can happen for a variety of reasons. Sometimes a DLE grows between > > estimate time and dump time. If you're using software compression and > > it's a relatively new DLE (or the nature of the data changes), amanda's > > estimate of the compressed size can be inaccurate. > > By 20gb? And no, we're not using compression anywhere so that's not it. I've had DLEs grow by over 100GB between estimate time and dump time. Investigate this. Only you can figure out why that 53GB dump happened in the first place. > > It's *impossible* to > > anticipate every situation, although I believe amanda does a pretty good > > job. > > Huh? Impossible to anticipate that 53gb can't be written to a 30gb tape? Amanda figures backups are really important, and really wants to get them on tape. It knows that tapelengths can be estimates (especially if one is using hardware compression), and figures it'll give it a shot even if it doesn't look like it'll fit. Now, I'll be the first to admit that the tape logic needs some work. In the above case, it'd be nice if there were some sort of "slop" variable. You set it to say, 10%, and that tells taper not to bother if an image is >10% bigger than the amount of tapelength left. Also, when runtapes is >1, it'd be nice if amanda knew that this doesn't equate to 1 tape of N*tapelength size, which is how I read the current code. Unfortunately, I'm not much of a coder, so I can't dig in and add these. Thus you also don't hear me complaining about lack of them. > > As an aside, I really think you need to rethink your attitude. Your > > subject line is incredibly unprofessional, as is your general attitude > > toward the amanda community. > > Comes from 4 years of having the exact same discussions over and over and > over and over again. Why can't amanda figure out EOT when everyone else > can? Why doesn't it start writing the new bit to the next tape? Don't > point me at the FAQ, when every other bit of software has nailed this > problem down just fine. Tape spanning breaks one of the long held strengths/philosophies of amanda -- that you can do a restore with nothing but mt, dd, and tar/restore. > And now the new best one -- if amanda is SO sensitive about fitting a > DLE onto a tape, how come it simply tries to do it without checking the > size first? See above. And please, please check your attitude at the door. If you want to make amanda better, join the -hackers list and get hacking. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: This is retarded.
On Thu, 1 Sep 2005 at 9:51am, Joe Rhett wrote > > On Wed, Aug 31, 2005 at 03:46:36PM -0700, Joe Rhett wrote: > > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > > > hours and 3 tapes doing absolutely NOTHING. > > On Wed, Aug 31, 2005 at 07:22:53PM -0400, Jon LaBadie wrote: > > What's the saying? > > "If you keep doing what you've been doing > > you'll keep getting what you've been getting!" > > > > You know you have a DLE that is too big to tape and that amanda > > does not handle it well. Isn't it time to stop wasting time > > and tape and split the DLE? > > Ah, I'm sorry! I should not assume that amanda can do it's own math. I > must sit down with a calculator every night and do my own estimates and > make this all work? > > Jon, this is a fairly serious problem with amanda. Why are you blaming the > reporter instead of trying to fix the problem? You haven't even attempted > to identify the problem, you're just being busy shooting the messenger. *You* are the only one who can really identify the problem. Are you using server-side or client-side estimates? If the latter, look at the sendsize*debug for the 53GB dump to see what amanda thought the size would be at estimate time. > Nice approach. When you come in with the attitude you are, expect nothing else. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: This is retarded.
On Wed, Aug 31, 2005 at 07:42:37PM -0400, Lengyel, Florian wrote: > Ah, not splitting the DLE is retarded. The whining is a most productive use > of bandwidth, however. I'd like to see a discussion of the code with the > purported error. That would have some intellectual (as opposed to emotional) > content I have gone that route many times over the last four years. One gets tired of finding and fixing a code problem, and then releasing the patch with every new update that comes out, and then mailing it off to everyone who reports the same problem. So one learns that reading the code and fixing the problem is not appreciated by the amanda maintainers, and thus one is left with the only way to get a fix implemented is to get them to fix it themselves. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Thu, Sep 01, 2005 at 09:51:15AM -0700, Joe Rhett wrote: > > On Wed, Aug 31, 2005 at 03:46:36PM -0700, Joe Rhett wrote: > > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > > > hours and 3 tapes doing absolutely NOTHING. > > On Wed, Aug 31, 2005 at 07:22:53PM -0400, Jon LaBadie wrote: > > What's the saying? > > "If you keep doing what you've been doing > > you'll keep getting what you've been getting!" > > > > You know you have a DLE that is too big to tape and that amanda > > does not handle it well. Isn't it time to stop wasting time > > and tape and split the DLE? > > Ah, I'm sorry! I should not assume that amanda can do it's own math. I > must sit down with a calculator every night and do my own estimates and > make this all work? > > Jon, this is a fairly serious problem with amanda. Why are you blaming the > reporter instead of trying to fix the problem? No, I was commenting on the futility of trying again something that failed once unexpectedly, you knew it was going to fail again, and have known from 5 years of amanda experience that that particular situation would always fail. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: This is retarded.
On Thu, Sep 01, 2005 at 09:46:43AM +1000, Jamie Wilkinson wrote: > There's a patch from John Stange that I don't believe has been committed to > CVS, but it takes care of splitting dumps and spanning tapes. Grep for it > on the amanda-hackers list. Doesn't fix #2, but should take care of #1 for > you. Because it's a patch that's been floating around for 2+ years and isn't being accepted into the mainline code for reasons unknown. Support for amanda is really only done by like 2 people, and applying this patch would make it just John, and there are many things John doesn't know or can't do. The things about backups is that they aren't a place to play games. I do have a test backup system, but it's focused on software that does recognize EOT and deal with it appropriately... -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Thu, Sep 01, 2005 at 08:52:29AM -0400, Joshua Baker-LePain wrote: > This can happen for a variety of reasons. Sometimes a DLE grows between > estimate time and dump time. If you're using software compression and > it's a relatively new DLE (or the nature of the data changes), amanda's > estimate of the compressed size can be inaccurate. By 20gb? And no, we're not using compression anywhere so that's not it. > It's *impossible* to > anticipate every situation, although I believe amanda does a pretty good > job. Huh? Impossible to anticipate that 53gb can't be written to a 30gb tape? > As an aside, I really think you need to rethink your attitude. Your > subject line is incredibly unprofessional, as is your general attitude > toward the amanda community. Comes from 4 years of having the exact same discussions over and over and over and over again. Why can't amanda figure out EOT when everyone else can? Why doesn't it start writing the new bit to the next tape? Don't point me at the FAQ, when every other bit of software has nailed this problem down just fine. And now the new best one -- if amanda is SO sensitive about fitting a DLE onto a tape, how come it simply tries to do it without checking the size first? -- Joe Rhett senior geek meer.net
Re: This is retarded.
> On Wed, Aug 31, 2005 at 03:46:36PM -0700, Joe Rhett wrote: > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > > hours and 3 tapes doing absolutely NOTHING. On Wed, Aug 31, 2005 at 07:22:53PM -0400, Jon LaBadie wrote: > What's the saying? > "If you keep doing what you've been doing > you'll keep getting what you've been getting!" > > You know you have a DLE that is too big to tape and that amanda > does not handle it well. Isn't it time to stop wasting time > and tape and split the DLE? Ah, I'm sorry! I should not assume that amanda can do it's own math. I must sit down with a calculator every night and do my own estimates and make this all work? Jon, this is a fairly serious problem with amanda. Why are you blaming the reporter instead of trying to fix the problem? You haven't even attempted to identify the problem, you're just being busy shooting the messenger. Nice approach. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Wed, Aug 31, 2005 at 06:00:39PM -0500, Frank Smith wrote: > > On Wed, Aug 31, 2005 at 12:41:43AM -0700, Joe Rhett wrote: > >> Some part of the code mistakenly decided to store 53gb to the holding disk, > >> and then tried to flush it to tape, regardless of the fairly basic math > >> involved. > > > > And well, tonight, it does it again. > > > > USAGE BY TAPE: > > Label Time Size %Nb > > svk17 0:00 0.00.0 0 > > svk18 0:00 0.00.0 0 > > svk19 1:32 15181.6 45.142 > > svk20 0:00 0.00.0 0 > > > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > > hours and 3 tapes doing absolutely NOTHING. > --On Wednesday, August 31, 2005 15:46:36 -0700 Joe Rhett <[EMAIL PROTECTED]> > wrote: > I'm still curious how a 53gb dump was even done if your tapelength was > set to 30ish GB. Was your tapelenght set to something longer at the > time the dump was originally done? Nope. The tapelength was 33gb for 4 years, and then we found a problem last year where amanda was calculating or writing wrong or whatever so we lowered the tapelength to 30gb to avoid this problem. (you can find the thread in the archives) > In the meantime, to quit burning tapes, set autoflush off (or is it false?) > to let the old dump stay on disk while continuing your current backups > and researching the problem. Or are you saying it did a new 53gb dump? I simply removed it from the spool. > Yes, I agree that Amanda shouldn't mark a tape as used if nothing was > successfully written to it. You could edit your tapelist to 'unuse' a > tape, just be aware you can screw things up if you get it wrong. I've already done this. Notice that 3 of these tapes were the same unused tapes from the previous day. Kindof hard to get it wrong ;-) Move to bottom, change date to date prior to previous tapes. That said, manually hacking on this in not the appropriate way to handle this. Why doesn't amanda recognize that it can't write this out? Why did it back it up in the first place? -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Wed, 31 Aug 2005 at 12:35am, Joe Rhett wrote > I don't think this is the answer. The real answer is that the filesystem > it is trying to flush was 53gb in size when you put all the chunks > together. That won't fit on a 33gb tape. > > 1. Why aren't we backing up chunks to different tapes yet? Amanda is the > only backup software which doesn't handle this. As has been mentioned (and as a simple search of the archives would tell you), there is a spanning patch out there being tested by various folks. You could be one of them. > -and more importantly- > > 2. Why is it trying to back up 53gb to a 33gb tape definition? > This can happen for a variety of reasons. Sometimes a DLE grows between estimate time and dump time. If you're using software compression and it's a relatively new DLE (or the nature of the data changes), amanda's estimate of the compressed size can be inaccurate. It's *impossible* to anticipate every situation, although I believe amanda does a pretty good job. I do agree, however, that the taping logic could be smarter when images on disk are too big for either the tape or how much tape is left. As an aside, I really think you need to rethink your attitude. Your subject line is incredibly unprofessional, as is your general attitude toward the amanda community. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: This is retarded.
This one time, at band camp, Joe Rhett wrote: >1. Why aren't we backing up chunks to different tapes yet? Amanda is the >only backup software which doesn't handle this. > >-and more importantly- > >2. Why is it trying to back up 53gb to a 33gb tape definition? > >#2 is clearly a bug. #1 is a feature request long overdue, but #2 is >clearly the bug. There's a patch from John Stange that I don't believe has been committed to CVS, but it takes care of splitting dumps and spanning tapes. Grep for it on the amanda-hackers list. Doesn't fix #2, but should take care of #1 for you.
RE: This is retarded.
Title: RE: This is retarded. On Wed, Aug 31, 2005 at 03:46:36PM -0700, Joe Rhett wrote: > > And well, tonight, it does it again. > > USAGE BY TAPE: > Label Time Size % Nb > svk17 0:00 0.0 0.0 0 > svk18 0:00 0.0 0.0 0 > svk19 1:32 15181.6 45.1 42 > svk20 0:00 0.0 0.0 0 > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > hours and 3 tapes doing absolutely NOTHING. What's the saying? "If you keep doing what you've been doing you'll keep getting what you've been getting!" You know you have a DLE that is too big to tape and that amanda does not handle it well. Isn't it time to stop wasting time and tape and split the DLE? jl Ah, not splitting the DLE is retarded. The whining is a most productive use of bandwidth, however. I'd like to see a discussion of the code with the purported error. That would have some intellectual (as opposed to emotional) content
Re: This is retarded.
On Wed, Aug 31, 2005 at 03:46:36PM -0700, Joe Rhett wrote: > On Wed, Aug 31, 2005 at 12:41:43AM -0700, Joe Rhett wrote: > > Some part of the code mistakenly decided to store 53gb to the holding disk, > > and then tried to flush it to tape, regardless of the fairly basic math > > involved. > > And well, tonight, it does it again. > > USAGE BY TAPE: > Label Time Size %Nb > svk17 0:00 0.00.0 0 > svk18 0:00 0.00.0 0 > svk19 1:32 15181.6 45.142 > svk20 0:00 0.00.0 0 > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > hours and 3 tapes doing absolutely NOTHING. What's the saying? "If you keep doing what you've been doing you'll keep getting what you've been getting!" You know you have a DLE that is too big to tape and that amanda does not handle it well. Isn't it time to stop wasting time and tape and split the DLE? jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: This is retarded.
--On Wednesday, August 31, 2005 15:46:36 -0700 Joe Rhett <[EMAIL PROTECTED]> wrote: > On Wed, Aug 31, 2005 at 12:41:43AM -0700, Joe Rhett wrote: >> Some part of the code mistakenly decided to store 53gb to the holding disk, >> and then tried to flush it to tape, regardless of the fairly basic math >> involved. > > And well, tonight, it does it again. > > USAGE BY TAPE: > Label Time Size %Nb > svk17 0:00 0.00.0 0 > svk18 0:00 0.00.0 0 > svk19 1:32 15181.6 45.142 > svk20 0:00 0.00.0 0 > > Yeah, I didn't need those tapes for anything else. Let's just waste 12 > hours and 3 tapes doing absolutely NOTHING. I'm still curious how a 53gb dump was even done if your tapelength was set to 30ish GB. Was your tapelenght set to something longer at the time the dump was originally done? In the meantime, to quit burning tapes, set autoflush off (or is it false?) to let the old dump stay on disk while continuing your current backups and researching the problem. Or are you saying it did a new 53gb dump? Yes, I agree that Amanda shouldn't mark a tape as used if nothing was successfully written to it. You could edit your tapelist to 'unuse' a tape, just be aware you can screw things up if you get it wrong. Frank > > -- > Joe Rhett > senior geek > meer.net -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: This is retarded.
On Wed, Aug 31, 2005 at 12:41:43AM -0700, Joe Rhett wrote: > Some part of the code mistakenly decided to store 53gb to the holding disk, > and then tried to flush it to tape, regardless of the fairly basic math > involved. And well, tonight, it does it again. USAGE BY TAPE: Label Time Size %Nb svk17 0:00 0.00.0 0 svk18 0:00 0.00.0 0 svk19 1:32 15181.6 45.142 svk20 0:00 0.00.0 0 Yeah, I didn't need those tapes for anything else. Let's just waste 12 hours and 3 tapes doing absolutely NOTHING. -- Joe Rhett senior geek meer.net
Re: This is retarded.
On Tue, Aug 30, 2005 at 06:01:07PM -0500, Frank Smith wrote: > In order to tell if Amanda is deficient in some way, we need some more > information. > According to the original post, this was an amflush run, so this DLE must be > sitting > on a holdindisk. Since normally Amanda refuses to backup a DLE that won't > fit on > a tape and give a warning, I see two possibilities: > 1. Your tapelength is set incorrectly in your config, so Amanda thinks a dump > will >fit when it won't Tapelength is actually less than real tape length. We adjusted this well below the real length of DLT tapes some time ago to avoid amanda miscalculation errors. > 2. For some reason the dump ended up larger than the estimate, perhaps due to > a >changing filesystem or using both H/W compression on already compressed > data. We don't use compression anywhere. Never. None of our backup definitions have compression enabled. > Is your tapelength set to something less than 34.2GB? Yes, it's set to 30gb. Much less than the real capacity. > How big is the dump in the holdingdisk (not how big the chunks are, if there > are > more than one, but the total of all the chunks of that DLE)? 53 1gb chunks. > With that information we could find where the problem lies and maybe find a > solution. Some part of the code mistakenly decided to store 53gb to the holding disk, and then tried to flush it to tape, regardless of the fairly basic math involved. This works fine without holding disks, or when the holding disk is too small for the backup. The logic flaw must be somewhere in the "tape not available, store to holding disk for later" logic and/or "flush this to tape" not checking the sizes involved. I am still strongly of the opinion that amanda's handling of DLEs is still its strongest failing point. We're hacking vainly at something which needs to be redesigned to work properly. -- Joe Rhett senior geek meer.net
Re: This is retarded.
> > >On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: > > >>tracking ability, but let's use 3 tapes and write not a single byte to > > >>them? On Tue, Aug 30, 2005 at 06:35:12PM -0400, Jon LaBadie wrote: > And not using samba, right Joe :)) Right. Straight amanda clients, some Solaris, some Linux, lots of Freebsd, and some Windows. But all samba native clients 2.4.4p2 or later. > >The dumps were flushed to tapes svk17, svk18, svk19. > >The next 7 tapes Amanda expects to used are: svk20, svk21, svk01, > >svk02, svk03, svk04, svk05. > > Looks like runtapes is 7. > > >Output Size (meg) 0.00.00.0 > > Nothing made it to any holding disk. This was a flush. It's already on the holding disk. > > taper: tape svk17 kb 34286880 fm 1 writing file: short write > > taper: retrying customer-plat1:/.1 on new tape: [writing file: short > >write] > > taper: tape svk18 kb 34227328 fm 1 writing file: short write > > taper: retrying customer-plat1:/.1 on new tape: [writing file: short > >write] > > taper: tape svk19 kb 0 fm 0 [OK] > > This looks like the dump was going directly to tape and was too large to > fit your 35GB tape. So it was retried and of course was still too big. > > customer- / lev 1 FAILED 20050825 [too many taper retries] Yes. The item it is trying to flush is 53 1gb tar blockfilies. du -ks /amandadump/20050825 5336478620050825 > This seems to be an improvement over what I would have expected. > I recall amanda continuing through any and all runtapes tapes. > It would have done 7 attempts in your config. At least now it > stops after a couple of attempts. > > As to whether amanda's behavior is reasonable, well ... > > It is nearly impossible, perhaps totally impossible, to tell if the taping > failed because of reaching the end of the tape or a tape or hardware error. > Is having a backup important enough to continue and try again or should the > first failure, possibly a bad/worn out tape terminate all the remaining > backups? > > I don't think there is a simple answer. What would be your recommendation? I don't think this is the answer. The real answer is that the filesystem it is trying to flush was 53gb in size when you put all the chunks together. That won't fit on a 33gb tape. 1. Why aren't we backing up chunks to different tapes yet? Amanda is the only backup software which doesn't handle this. -and more importantly- 2. Why is it trying to back up 53gb to a 33gb tape definition? #2 is clearly a bug. #1 is a feature request long overdue, but #2 is clearly the bug. -- Joe Rhett senior geek meer.net
Re: This is retarded.
--On Tuesday, August 30, 2005 18:35:12 -0400 Jon LaBadie <[EMAIL PROTECTED]> wrote: > On Tue, Aug 30, 2005 at 01:48:28PM -0700, Joe Rhett wrote: >> > On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: >> >> tracking ability, but let's use 3 tapes and write not a single byte to >> >> them? >> >> On Aug 30, 2005, at 1:18 PM, Cam wrote: >> > What's retarded? your configuration? you really didn't give enough >> > information for anyone to help you. >> >> We've been running in this configuration for 5+ years now. The >> configuration works. > > And not using samba, right Joe :)) > > >> ... . Yes, I could supply the configuration but it >> really doesn't change anything. Amanda should not use 3 tapes and >> write 0 bytes to them, no matter what the configuration is. >> > > Couple of items from your original post. > >> The dumps were flushed to tapes svk17, svk18, svk19. >> The next 7 tapes Amanda expects to used are: svk20, svk21, svk01, >> svk02, svk03, svk04, svk05. > > Looks like runtapes is 7. > >> Output Size (meg) 0.00.00.0 > > Nothing made it to any holding disk. > >> taper: tape svk17 kb 34286880 fm 1 writing file: short write >> taper: retrying customer-plat1:/.1 on new tape: [writing file: short >> write] >> taper: tape svk18 kb 34227328 fm 1 writing file: short write >> taper: retrying customer-plat1:/.1 on new tape: [writing file: short >> write] >> taper: tape svk19 kb 0 fm 0 [OK] > > This looks like the dump was going directly to tape and was too large to > fit your 35GB tape. So it was retried and of course was still too big. > >> customer- / lev 1 FAILED 20050825 [too many taper retries] > > This seems to be an improvement over what I would have expected. > I recall amanda continuing through any and all runtapes tapes. > It would have done 7 attempts in your config. At least now it > stops after a couple of attempts. > > As to whether amanda's behavior is reasonable, well ... > > It is nearly impossible, perhaps totally impossible, to tell if the taping > failed because of reaching the end of the tape or a tape or hardware error. > Is having a backup important enough to continue and try again or should the > first failure, possibly a bad/worn out tape terminate all the remaining > backups? In order to tell if Amanda is deficient in some way, we need some more information. According to the original post, this was an amflush run, so this DLE must be sitting on a holdindisk. Since normally Amanda refuses to backup a DLE that won't fit on a tape and give a warning, I see two possibilities: 1. Your tapelength is set incorrectly in your config, so Amanda thinks a dump will fit when it won't 2. For some reason the dump ended up larger than the estimate, perhaps due to a changing filesystem or using both H/W compression on already compressed data. Is your tapelength set to something less than 34.2GB? How big is the dump in the holdingdisk (not how big the chunks are, if there are more than one, but the total of all the chunks of that DLE)? With that information we could find where the problem lies and maybe find a solution. Frank > > I don't think there is a simple answer. What would be your recommendation? > > -- > Jon H. LaBadie [EMAIL PROTECTED] > JG Computing > 4455 Province Line Road(609) 252-0159 > Princeton, NJ 08540-4322 (609) 683-7220 (fax) -- Frank Smith[EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: This is retarded.
On Tue, Aug 30, 2005 at 01:48:28PM -0700, Joe Rhett wrote: > >On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: > >>tracking ability, but let's use 3 tapes and write not a single byte to > >>them? > > On Aug 30, 2005, at 1:18 PM, Cam wrote: > > What's retarded? your configuration? you really didn't give enough > >information for anyone to help you. > > We've been running in this configuration for 5+ years now. The > configuration works. And not using samba, right Joe :)) > ... . Yes, I could supply the configuration but it > really doesn't change anything. Amanda should not use 3 tapes and > write 0 bytes to them, no matter what the configuration is. > Couple of items from your original post. >The dumps were flushed to tapes svk17, svk18, svk19. >The next 7 tapes Amanda expects to used are: svk20, svk21, svk01, >svk02, svk03, svk04, svk05. Looks like runtapes is 7. >Output Size (meg) 0.00.00.0 Nothing made it to any holding disk. > taper: tape svk17 kb 34286880 fm 1 writing file: short write > taper: retrying customer-plat1:/.1 on new tape: [writing file: short >write] > taper: tape svk18 kb 34227328 fm 1 writing file: short write > taper: retrying customer-plat1:/.1 on new tape: [writing file: short >write] > taper: tape svk19 kb 0 fm 0 [OK] This looks like the dump was going directly to tape and was too large to fit your 35GB tape. So it was retried and of course was still too big. > customer- / lev 1 FAILED 20050825 [too many taper retries] This seems to be an improvement over what I would have expected. I recall amanda continuing through any and all runtapes tapes. It would have done 7 attempts in your config. At least now it stops after a couple of attempts. As to whether amanda's behavior is reasonable, well ... It is nearly impossible, perhaps totally impossible, to tell if the taping failed because of reaching the end of the tape or a tape or hardware error. Is having a backup important enough to continue and try again or should the first failure, possibly a bad/worn out tape terminate all the remaining backups? I don't think there is a simple answer. What would be your recommendation? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
RE: This is retarded.
"Should" is not a computer science term. Looks like a hardware error; are you willing to provide further details about your hardware and software setup? It's easy to provide them; I'll start: I'm using a Spectra Logic 2K here. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Joe Rhett Sent: Tuesday, August 30, 2005 4:48 PM To: Cam Cc: amanda-users@amanda.org Subject: Re: This is retarded. > On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: >> tracking ability, but let's use 3 tapes and write not a single byte to >> them? On Aug 30, 2005, at 1:18 PM, Cam wrote: > What's retarded? your configuration? you really didn't give enough > information for anyone to help you. We've been running in this configuration for 5+ years now. The configuration works. Yes, I could supply the configuration but it really doesn't change anything. Amanda should not use 3 tapes and write 0 bytes to them, no matter what the configuration is. -- Joe Rhett senior geek meer.net
RE: This is retarded.
> > On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: > >> tracking ability, but let's use 3 tapes and write not a > single byte > >> to them? > > On Aug 30, 2005, at 1:18 PM, Cam wrote: > > What's retarded? your configuration? you really didn't give enough > > information for anyone to help you. > > We've been running in this configuration for 5+ years now. The > configuration works. Yes, I could supply the configuration but it > really doesn't change anything. Amanda should not use 3 tapes and > write 0 bytes to them, no matter what the configuration is. > Joe, I find your response retarded, if we're going to name-call like we're 5. You don't go to the doctor and say "My body has worked for 34 years, and now all of a sudden, this bad thing is happening..." and not give the doctor any more information than that. My experience with computers and/or software is that anything that can happen will. And my experience with this forum is that we don't answer questions without all of the information. I'm not sure what you expect us to do to help you when you're not telling us anything. With what you've supplied, I'm saying it's hardware. Does that help you? Didn't think so. Rebecca.
Re: This is retarded.
--On August 30, 2005 2:36:45 PM -0600 Graeme Humphries <[EMAIL PROTECTED]> wrote: This looks like a hardware error with your tape drive more than an amanda problem, at least to me. Could be, or could be AMANDA's still poor behavior when a DLE exceeds the capacity of a single tape, and the tapetype is just a bit long for the tapes or maybe as someone I think hypothesized that taper ignores the tapelen and just writes to EOT, and only planner really uses the tapelen? I'm not sure, haven't tested that myself and didn't get a chance to read the rest of that thread... :S Graeme -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler
Re: This is retarded.
On 8/30/05, Joe Rhett <[EMAIL PROTECTED]> wrote: tracking ability, but let's use 3 tapes and write not a single byte to them? On Aug 30, 2005, at 1:18 PM, Cam wrote: What's retarded? your configuration? you really didn't give enough information for anyone to help you. We've been running in this configuration for 5+ years now. The configuration works. Yes, I could supply the configuration but it really doesn't change anything. Amanda should not use 3 tapes and write 0 bytes to them, no matter what the configuration is. -- Joe Rhett senior geek meer.net
Re: This is retarded.
Joe Rhett wrote: I'm sorry, but this is retarded. I know that amanda has stone-age tape tracking ability, but let's use 3 tapes and write not a single byte to them? This looks like a hardware error with your tape drive more than an amanda problem, at least to me. Graeme
