[Amanda-users] Making sure AMANDA does not run "over" tapesize
Charles Curley wrote: > On Wed, 05 Jan 2011 13:19:42 -0500 > rory_f backupcentral.com> wrote: > > > > Perhaps what i'm looking to do is guarantee the most economical way > > of filling the tapes. > > > > Which is more valuable, a few terabytes of tape space, or your time? > > -- > > Charles Curley /"\ASCII Ribbon Campaign > Looking for fine software \ /Respect for open standards > and/or writing? X No HTML/RTF in email > http://www.charlescurley.com/ \No M$ Word docs in email > > Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB I guess that depends if you ask me (as it is my time) or the boss (who pays for the tapes ;-]) +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Making sure AMANDA does not run "over" tapesize
choogendyk wrote: > On 1/5/11 12:00 PM, rory_f wrote: > > > I want to ensure tapes are filled 100% each time where possible. I've > > written a script in python to look at directory, figure out size, and > > create a disklist which will ensure a round about size for each disklist > > file - so for instance it will try to create a disklist file that contains > > entries in groups of 400gb's - the size of a tape. I know amanda will fill > > a tape to 100% where possible but sometimes, if it is using compression, > > this doesn't work, and the first two tapes will fill 500gb+ and then the > > last tape will be left with 200gb. This is a waste of 200gb - I'm trying to > > make sure all tapes are full where possible and not waste any space. > > > > Not to be rude, but that's a false economy. > > It could just as easily be said that you would be wasting tape capacity by > not using compression. > > You are asking to not allow more than 400GB per tape, and thus no more than > 1200GB on the set of 3. > Then you are complaining that the 1200GB is unevenly distributed across the 3 > tapes, because > compression allowed more than 400GB on each of the first 2 tapes. So, stated > another way, you are > asking that the "wasted" (or unused) 300GB (or so) of space be distributed > across all 3 tapes, > rather than just being on the last tape, and/or to just not use compression > so that you can imagine > that you are not wasting tape. > > 500GB per tape means that you are getting about 20% compression. If that is > consistent, have your > python script set to queue up somewhere between 1400GB to 1500GB for backup, > the choice depending on > how close you want to shave it (with a higher risk of over running the last > tape). Then you are > being economical with your tape usage, getting a couple hundred more GB on > the set of tapes than you > were originally thinking. > > Of course, compressibility varies widely. Huge directories of TIFF and JPEG > files can be essentially > uncompressible. Typical unix directories of predominantly text based stuff, > like log files or > configuration files, are highly compressible, and repetitive things like > Apache access logs can > compress as much as 10:1. So, you have to know your data to efficiently plan > what you are trying to do. > > Ok. I totally see your point - you are very correct.The majority of the files being backed up are images - dpx, exr, cin, etc - we are a VFX house. I guess with a wide range of varying file types theres no real way to 'predict' the type of compression, is there? Perhaps what i'm looking to do is guarantee the most economical way of filling the tapes.. It would be great to count on compression of 20% every time but it seems to vary too much to rely on AMANDA to work this way. Thanks for your view though - initially I didnt look at it that way. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Making sure AMANDA does not run "over" tapesize
I want to ensure tapes are filled 100% each time where possible. I've written a script in python to look at directory, figure out size, and create a disklist which will ensure a round about size for each disklist file - so for instance it will try to create a disklist file that contains entries in groups of 400gb's - the size of a tape. I know amanda will fill a tape to 100% where possible but sometimes, if it is using compression, this doesn't work, and the first two tapes will fill 500gb+ and then the last tape will be left with 200gb. This is a waste of 200gb - I'm trying to make sure all tapes are full where possible and not waste any space. I know I could take the tape that is half full and archive the contents again with added content but this is time consuming. I just want to make sure amanda is working with my script to make sure all tapes are being filled. Do you see what i'm getting at? Thanks, Brian Cuttler wrote: > Rory, > > When I am using tape compression I often...sorry. > > When using SW tape compression amanda will know what the > typical compression is for any given DLE and will (I believe) > use the expected compressed size of the data when estimating > overall tape usage. > > When I use HW compression I often lie about the tape length, > extending the actually physical size by the expected compression > amount so that I am able to utilize the full physical tape. > This is very valuable for me in the couple of cases where I > have a non-spanning DLE that is larger than the physical tape > would be without compression, else amanda would report that the > DLE was larger than the tape... > > What goal/outcome are you seeking ? > > Brian > > On Wed, Jan 05, 2011 at 11:34:47AM -0500, rory_f wrote: > > > Hey, > > > > So I've noticed that sometimes Amanda will fill up a tape with more than > > 400gb (LTO3) - I'm assuming this is down to compression? Is there another > > way to limit this from happening apart from turning hardware and software > > compression off? > > > > Thanks, > > > > +-- > > |This was sent by rory < at > mrxfx.com via Backup Central. > > |Forward SPAM to abuse < at > backupcentral.com. > > +-- > > > > > > > --- > +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Making sure AMANDA does not run "over" tapesize
Hey, So I've noticed that sometimes Amanda will fill up a tape with more than 400gb (LTO3) - I'm assuming this is down to compression? Is there another way to limit this from happening apart from turning hardware and software compression off? Thanks, +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Gene Heskett wrote: > On Tuesday 09 March 2010, rory_f wrote: > > Absolutely no, amanda ignores that. It is for your edification only. > > One other question: Did you take note of whether or not the drive was > streaming steadily, or was it 'shoe shining' occasionally? One can normally > hear such goings on by the squawk of the steppers as they wind up and slow > down. > > Hey, I couldn't tell you . I don't recall anything like that though. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Dustin J. Mitchell wrote: > On Tue, Mar 9, 2010 at 9:55 AM, rory_f > backupcentral.com> wrote: > > > One question; i don't have to run amtapetype every time i do this do i? For > > instance, say the speed problems are resolved, will amdump pick up on this > > automatically or does it stick to the tape speed given in amtapetype > > absolutely? > > > > Amdump just runs the tape as fast as it goes. The speed estimate is not used. > > Dustin > > -- > Open Source Storage Engineer > http://www.zmanda.com Thanks Dustin, good to know. Thanks to everyone else for the help. Time to return this HP drive ;-] +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Gene Heskett wrote: > On Tuesday 09 March 2010, rory_f wrote: > > If the card is wide scsi, perhaps a cabling issue or termination issue has > caused it to fall back to scsi-II width and speeds? I have read that some > cards do this, and a reboot once the problem is solved, might bring back the > full speed. Might be worth a try just to satisfy the curious cat in all of > us. > > > > Theoretically, this new drive should be as fast if not faster.. but it > > isn't. > > > > Do you recall the block size used with the previous drive? If it was also > 32k which is the default, some speed improvements can be had with larger > block sizes, but I doubt that could make a 2/1 ratio change in the drives > speed. > > > > At least it seems to *work* now. > > > > Which is nice, but I believe I would take up the speed problem with the > vendor after I had rebooted and checked to see if it persists. Perhaps there > is a spindle speed jumper that is miss-placed on this one? Possibly > automatically interlocked with the write clock speed since you did get the > exact same size, so the drive maintains the same bits per inch. > > But I'm making SWAG's here, my knowledge of drives isn't nearly as deeply > ingrained as the scsi experience is. > > Perhaps someone else here can help? > > I've rebooted the machine a few times. I'll try again, after i set the cables again. One question; i don't have to run amtapetype every time i do this do i? For instance, say the speed problems are resolved, will amdump pick up on this automatically or does it stick to the tape speed given in amtapetype absolutely? +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Gene Heskett wrote:
> On Monday 08 March 2010, rory_f wrote:
>
> Please do. It could be something I missed, or it could be a brand new
> phenomenon to file away in my trivia file.
>
>
gene,
ama...@backup tor]$ amtapetype -o -f /dev/nst0 -e 400G
Writing 1024 Mbyte compresseable data: 31 sec
Writing 1024 Mbyte uncompresseable data: 31 sec
Estimated time to write 2 * 409600 Mbyte: 24800 sec = 6 h 53 min
wrote 12845056 32 Kb blocks in 98 files in 11738 seconds (No space left on
device)
wrote 12910592 32 Kb blocks in 197 files in 11840 seconds (No space left on
device)
define tapetype NEW_IBM_DRIVE {
comment "just produced by tapetype prog (hardware compression off)"
length 402432 mbytes
filemark 0 kbytes
speed 34955 kps
}
comparing this tapetype to the last ibm drive (that broke and was replaced by
this) the speed is nearly half.
define tapetype BROKE_IBM_DRIVE {
comment "just produced by tapetype prog (hardware compression off)"
length 402432 mbytes
filemark 0 kbytes
speed 71201 kps
}
This brings me to another question - what could be causing this ?
The system itself has not changed, nor has the cabling, really. unless the
cables have become damaged (or the scsi card inside the machine is now having
problems), what else is to check?
Theoretically, this new drive should be as fast if not faster.. but it isn't.
At least it seems to *work* now.
+--
|This was sent by r...@mrxfx.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
[Amanda-users] Amtapetype not returning right tape length?
Gene Heskett wrote: > On Monday 08 March 2010, Dustin J. Mitchell wrote: > > > On Mon, Mar 8, 2010 at 9:55 AM, Gene Heskett > > verizon.net> > > > wrote: > > > > > > Be my guest if you have write privileges. > > > > > > > It's a wiki - everyone has write privileges (including, to my dismay, > > spammers .. but I try to keep up with them). > > > > Please do add this to the wiki! > > > > Dustin > > > > > I would Dustin, but 2 questions: Where do I put in, or can I make a new > category? > > And 2, I don't see any edit buttons anyplace, security by obscurity I believe > they call that. So that seems to be a show stopper. :( > > -- > Cheers, Gene > "There are four boxes to be used in defense of liberty: > soap, ballot, jury, and ammo. Please use in that order." > -Ed Howdershelt (Author) > > Feanor: u have no idea of the depth of the stupidty of american law I started an amtapetype earlier on the new drive, and it's still running now. Writing 1024 Mbyte uncompresseable data: 31 sec Estimated time to write 2 * 409600 Mbyte: 24800 sec = 6 h 53 min wrote 12845056 32 Kb blocks in 98 files in 11738 seconds (No space left on device) wrote 3473408 32 Kb blocks in 53 files that estimated time write is still high (i think) but the 'no space left on device' is roughly about double the amount of seconds as it was when the tape was only being filled 50%. So i think this is going to produce a proper tapetype, if not a slow one. So it could be a combination of things.. i'll know more tomorrow. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
[quote="Gene Heskett"]On Monday 08 March 2010, rory_f wrote: > Gene Heskett wrote: > > In your case, two things. Is it the last drive on the cable?, and is it > terminated properly? And I would certainly check the 5 volt line of the host > computer to see if its sagging in its old age. 4.85 volts won't cause the > host to reset itself, but it does put the noise margin of a scsi buss in > danger. You need every millivolt you can get there. > > It is setup as such: Overland tape library, drive inside it. Drive connects to library via one cable, then another cable connects to scsi card in linux machine - It has been setup like this for a couple of years now. It could very well be the cable degrading, or the card in the linux machine. That is something I am yet to check. Our other drive is being replaced today (For a NEW ibm lto 3 drive) so hopefully i can test that one - if i get th same issues ,we can say it is something else other than the drives. I'll let you all know what i find. Thanks. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Gene Heskett wrote: > On Saturday 06 March 2010, Jon LaBadie wrote: > > > Because the scsi buss is a transmission line, and demands a decent VSWR, > there are 2 things to remember when dealing with scsi. > > 1.A list of pre-requisites that must be met if it is to work: > > 1.a: termination power, it has to come from someplace, preferably the host > controller. > > 1.b: terminations can only be at the ends of the cable, one and exactly one > on each end. Leaving a 6" pigtail hanging out because you removed the drive > on the end of the cable, and which co-incidentally also was the terminating > device is a more common mistake than one would think. > > 1.c: when supplying scsi term power, there needs to be an isolation diode > between the supply and the term power line in the cable, pointed so that the > voltage can get to the cable ok, but cannot be fed back into an accessory > device after it has powered the terminators on that device. > > 1.d: Because this diode has a voltage drop, and the original idiots (yes, > that _is_ the right terminology) chose the resistor values based on std > values AND a 5 volt supply, so a common silicon diode virtually cannot be > used if the scsi bus is to be error free. The .65 to .7 volts of drop in an > Si diode reduces the logic one voltages noise margin by about 2/3rds. from > the target of 3.0 volts, but usually about 2.85 due to the resistors chosen, > throw in the .7 loss of the diode divided by the resistor ratio's (330 to > ground, 220 to the 5 volt line, and your logic 1 noise margin is whats left > after you subtract the 2.4 volts of a guaranteed logic 1. At 2.7 volts it > might work if the resistor tolerance are spot on. At 2.55 volts, you have > only a 150 millivolt noise margin and you may as well hold a seance, its > dead, Jim. Therefore this diode must be either a power germanium or a > schotkey rated for the current in order to get a decently low forward voltage > drop. Otherwise find a 6 volt supply for termination power. TBT, when the > psu voltage in the host machine is starting to sag in its old age, and is > already 200 millivolts low, I've been known to go get a 6 volt dc wall wart > from the shack and use that. It Just Works(TM). FWIW, I have yet to see a > Ge or Schotkey diode used on any of the major scsi vendors cards, so don't > assume that just because it says Adaptec on it, it can't be wrong. And it is > precisely those cards I've had to wall wart power several times. > > If you've done all that & it still doesn't work, have a tech with a 100+ mhz > scope look at it to see why the signal echos (aka VSWR) are so bad. Maybe > you've got one bad resistor in the termination packs, so check all active > lines, 21 of them in a 50 wire scsi-II cable IIRC. Make sure he understands > exactly what the term 'VSWR' means, a great many don't, somehow thinking it > only applies to continuous wave radio/tv transmitters. VSWR is VSWR whether > its digital, or CW, its still VSWR. > > 2: Alternatively, you can advertise for a virgin to be sacrificed. It won't > help of course. And its a terrible waste considering how hard it is to find > one these days. > > -- > Cheers, Gene > "There are four boxes to be used in defense of liberty: > soap, ballot, jury, and ammo. Please use in that order." > -Ed Howdershelt (Author) > > Reality is bad enough, why should I tell the truth? > -- Patrick Sky Well, thanks for the long explanation. I doubt i will be able to test the setup as much as you have described but it's useful information. I've tried a couple of different cables, unplugged and plugged back in, all that jazz. These are the cables i've used previously without any trouble. I'm more inclined then to think it is the drive. but who knows.. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Jon LaBadie wrote:
> On Fri, Mar 05, 2010 at 02:57:01PM -0500, rory_f wrote:
>
> >
> > Hey guys. Thanks
> >
> > I checked my library GUI and for the tape run i was just talking about,
> > where it was stopping around 230gb, it seems compression WAS on. I turned
> > it off via mt and now the gui says "off" (after stopping the run obviously)
> >
> > I was *sure* i turned it off with mt before. Im doing another amtapetype
> > now.
> >
> > And the comment regarding labeling: thanks. I'll most certainly relabel all
> > the tapes after this tapetype is done.
> >
> > Thanks again guys.
> >
> >
>
> Recalling your data in the original posting I think an amtapetype run was
> estimated
> to take about 8 hours. I think LTO drives are capable of writing a full tape
> in
> about 1.5 hrs, amtapetype should do two full writes, thus about 3+ hours
> total.
>
> As you system seems to be writing at less than 1/2 full speed, I wonder if it
> is failing to stream. If an LTO drive is not streaming, is its capacity
> affected
> by having large gaps caused by the start and stopping action?
>
> jl
> --
> Jon H. LaBadie jon < at > jgcomp.com
> JG Computing
> 12027 Creekbend Drive (703) 787-0884
> Reston, VA 20194 (703) 787-0922 (fax)
ok, so here is the latest one.
[ama...@backup tor]$ amtapetype -o -f /dev/nst0 -e 400G
Writing 1024 Mbyte compresseable data: 32 sec
Writing 1024 Mbyte uncompresseable data: 32 sec
Estimated time to write 2 * 409600 Mbyte: 25600 sec = 7 h 6 min
wrote 6815744 32 Kb blocks in 52 files in 5733 seconds (No space left on device)
wrote 6815744 32 Kb blocks in 104 files in 5959 seconds (No space left on
device)
define tapetype unknown-tapetype {
comment "just produced by tapetype prog (hardware compression off)"
length 212992 mbytes
filemark 0 kbytes
speed 37322 kps
}
compression was set OFF in the gui for the library interfacing with the drive.
so, it seems, unless these tapes i have here are all pooched, the drive is
giving me problems. ugh.
i'd try it on our other drive but ... that's out being replaced ;-). maybe
it's time to get this replaced too, while i still have the maintenance contract.
thanks for the help.. any further input on this is greatly appreciated!
+--
|This was sent by r...@mrxfx.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
[Amanda-users] Amtapetype not returning right tape length?
Jon LaBadie wrote: > On Fri, Mar 05, 2010 at 02:57:01PM -0500, rory_f wrote: > > > > > Hey guys. Thanks > > > > I checked my library GUI and for the tape run i was just talking about, > > where it was stopping around 230gb, it seems compression WAS on. I turned > > it off via mt and now the gui says "off" (after stopping the run obviously) > > > > I was *sure* i turned it off with mt before. Im doing another amtapetype > > now. > > > > And the comment regarding labeling: thanks. I'll most certainly relabel all > > the tapes after this tapetype is done. > > > > Thanks again guys. > > > > > > Recalling your data in the original posting I think an amtapetype run was > estimated > to take about 8 hours. I think LTO drives are capable of writing a full tape > in > about 1.5 hrs, amtapetype should do two full writes, thus about 3+ hours > total. > > As you system seems to be writing at less than 1/2 full speed, I wonder if it > is failing to stream. If an LTO drive is not streaming, is its capacity > affected > by having large gaps caused by the start and stopping action? > > jl > -- > Jon H. LaBadie jon < at > jgcomp.com > JG Computing > 12027 Creekbend Drive (703) 787-0884 > Reston, VA 20194 (703) 787-0922 (fax) Hey This is true - i believe on previous amtapetype runs it estimated 4hrs. I started again , compression off. still 7hrs. So this drive seems pooched? Anything else I can check? Could it be cables? There are no drive errors via the kernel or on the library. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
Brian Cuttler wrote: > Gene, > > I had no idea it was that difficult to reset. > > I apologize if I left the impression that I thought it > was an amanda issue, I was sure it was in the HW and > the driver, but I didn't express. > > In truth I seldom touch a drive for other amanda amanda > related work, only when assisting an end-user backing up > their own data or installing a new drive for an end-user use. > > On Fri, Mar 05, 2010 at 01:30:43PM -0500, Gene Heskett wrote: > > > On Friday 05 March 2010, Brian Cuttler wrote: > > > > > Rory, > > > > > > I may be wrong, nor my information obsolete for your tapedevice > > > but I seem to recall that if you want to actually turn off compression > > > you have to also relabel the tape. If it sees the amlabel was > > > written in compessed mode... > > > > > > > > Yes, back when I was using tape, that was a major problem, so much so that > > I > > finally read the first block of a new, but labeled, uncompressed tape, then > > used dd to write that first block to every tape on the premises, then > > forcibly relabeled them for my system. A certified PIMA... Then I bought > > another case of tape from ebay, they looked new, cello wrap & the whole > > maryann, but they weren't, and the whole scene had to be repeated, its a > > darned virus I think. > > > > This BTW, is not an amanda problem, its 100% the drive, it reads its own > > hidden header data while 'recognizing' or 'accepting' the tape, and sets > > the > > compression according to what it finds. So that gets reset to match the > > tape > > any time you do a rewind, so you must do the rewind, and THEN issue the > > compression off command, through mtx I think it was, and then, without > > accessing the drive, write the uncompressed first block. Never give it a > > chance to re-read that block until its done writing the new one.. > > > > > > > > > On Fri, Mar 05, 2010 at 12:21:32PM -0500, rory_f wrote: > > > > > > > Dustin J. Mitchell wrote: > > > > > > > > > On Fri, Mar 5, 2010 at 9:17 AM, rory_f > > > > > > > > > > > > > > backupcentral.com> wrote: > > > > > > > > > > > > > > > > > > > > i'm doing a tape run now, using the original tapetype definition, > > > > > > and > > > > > > it has filled a tape to around 238gb and moved onto the next one > > > > > > (well it has changed tapes,it's still waiting to write to tape) > > > > > > > > > > > > is this normal? > > > > > > > > > > > > > > > > Amanda treats any error from the tape drive as EOT. It's the nature > > > > > of UNIX kernel drivers that they don't give much more information than > > > > > that. So it's quite possible that your tape drive is failing, or the > > > > > tapes are failing, and intermittent errors are causing the early EOT > > > > > indication. > > > > > > > > > > Alternately, maybe the earlier amtapetype run was made with hardware > > > > > compression on? The size is about double.. > > > > > > > > > > Dustin > > > > > > > > > > > > > I made sure to turn compression off and also the output of the tapetype > > > > didn't mention compression was on - it normally does if it detects > > > > compression, right? > > > > > > > > Perhaps compression is turned on elsewhere - im not sure where though. > > > > So > > > > you think what is happening is the kernel thinks these tapes are only > > > > 200~gb and giving an EOT to amanda - when infact they are 400gb LTO3 > > > > tapes. > > > > > > > > I doubt the tapes are failing - but i mean it is possible, i've used the > > > > same type of tapes for over 400 tapes with amanda and never seen this > > > > issue. > > > > > > > > I'm guessing then it's the drive itself.. there's no real way to tell. > > > > This is our second tape drive, the one we have used more consistantly > > > > with Amanda is out being replaced as it died, too :/ > > > > > > > > I'll run a amtapetype again to make sure, i guess. > > > > >
[Amanda-users] Amtapetype not returning right tape length?
Dustin J. Mitchell wrote: > On Fri, Mar 5, 2010 at 9:17 AM, rory_f > backupcentral.com> wrote: > > > i'm doing a tape run now, using the original tapetype definition, and it > > has filled a tape to around 238gb and moved onto the next one (well it has > > changed tapes,it's still waiting to write to tape) > > > > is this normal? > > > > Amanda treats any error from the tape drive as EOT. It's the nature > of UNIX kernel drivers that they don't give much more information than > that. So it's quite possible that your tape drive is failing, or the > tapes are failing, and intermittent errors are causing the early EOT > indication. > > Alternately, maybe the earlier amtapetype run was made with hardware > compression on? The size is about double.. > > Dustin > > -- > Open Source Storage Engineer > http://www.zmanda.com I made sure to turn compression off and also the output of the tapetype didn't mention compression was on - it normally does if it detects compression, right? Perhaps compression is turned on elsewhere - im not sure where though. So you think what is happening is the kernel thinks these tapes are only 200~gb and giving an EOT to amanda - when infact they are 400gb LTO3 tapes. I doubt the tapes are failing - but i mean it is possible, i've used the same type of tapes for over 400 tapes with amanda and never seen this issue. I'm guessing then it's the drive itself.. there's no real way to tell. This is our second tape drive, the one we have used more consistantly with Amanda is out being replaced as it died, too :/ I'll run a amtapetype again to make sure, i guess. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
anyone? i'm doing a tape run now, using the original tapetype definition, and it has filled a tape to around 238gb and moved onto the next one (well it has changed tapes,it's still waiting to write to tape) is this normal? +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Amtapetype not returning right tape length?
[ama...@backup tor]$ amtapetype -o -f /dev/nst0 -e 400G
Writing 1024 Mbyte compresseable data: 31 sec
Writing 1024 Mbyte uncompresseable data: 32 sec
Estimated time to write 2 * 409600 Mbyte: 25600 sec = 7 h 6 min
wrote 6684672 32 Kb blocks in 51 files in 5871 seconds (No space left on device)
wrote 262144 32 Kb blocks in 4 files
wrote 7012352 32 Kb blocks in 107 files in 5952 seconds (No space left on
device)
define tapetype unknown-tapetype {
comment "just produced by tapetype prog (hardware compression off)"
length 214016 mbytes
filemark 0 kbytes
speed 37067 kps
}
I've done an amtapetype with this drive over a year ago and it returned this
tapetype (from amanda.conf)
define tapetype otherdrive {
comment "just produced by tapetype prog (hardware compression off)"
length 402432 mbytes
filemark 0 kbytes
speed 67629 kps
}
The drive is an LTO3 HP drive.
Does this point to bad hardware? I tried two different tapes to run the
tapetype - both with the same results. There are no errors in dmesg to report
scsi problems during the test ..
What else could it be?
+--
|This was sent by r...@mrxfx.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
[Amanda-users] Recommend a SCSI card for Amanda/Linux use??
Currently we are using this: 03:0c.0 SCSI storage controller: Adaptec AIC-7892A U160/m (rev 02) And it is throwing up hardware errors every now and again so we want to grab another one to put in there. I understand it could also be cabling which is causing this and we'll address that too We're using a fc7 box, 32 bit. Can anyone recommend a card they have had good use out of ? rory +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Specify DLE in amrecover, amidxtaped reports differently?
Hi, amrecover> sethost archive2.win.mrxfx.com 200 Dump host set to archive2.win.mrxfx.com. amrecover> setdisk /array/sata-1/xxx/WS039 200 Disk set to /array/sata-1/xxx/WS039. but in the log for it, amidxtaped reports this: 1241720675.702638: amidxtaped: disk = '/array/sata-1/other/SHOTS/147Ax020' Why would that be? Help ! thanks :) +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Specify DLE in amrecover, amidxtaped reports differently?
rory_f wrote: > Hi, > > amrecover> sethost archive2.win.mrxfx.com > 200 Dump host set to archive2.win.mrxfx.com. > amrecover> setdisk /array/sata-1/xxx/WS039 > 200 Disk set to /array/sata-1/xxx/WS039. > > but in the log for it, amidxtaped reports this: > > 1241720675.702638: amidxtaped: disk = > '/array/sata-1/other/SHOTS/147Ax020' > > Why would that be? > > Help ! thanks :) Just to add, i checked the log file from index/archive2.win.mrxfx.com and the gz file has the correct file listing, all pertaining to WS039.. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] amidxtaped errors
[r...@backup tor]# cat amidxtaped.20090417114001.debug 1239982801.291428: amidxtaped: pid 3896 ruid 790 euid 790: start at Fri Apr 17 11:40:01 2009 1239982801.291510: amidxtaped: amidxtaped: version 2.6.0p2 1239982806.292428: amidxtaped: > FEATURES=9ffe00 1239982806.294173: amidxtaped: > CONFIG=tor 1239982806.334057: amidxtaped: > LABEL=AmaTor-164:2 1239982806.334078: amidxtaped: append_to_tapelist(tapelist=(nil), label='AmaTor-164', file=-1, partnum=-1, isafile=0) 1239982806.334096: amidxtaped: append_to_tapelist(tapelist=0x86eaf88, label='AmaTor-164', file=2, partnum=-1, isafile=0) 1239982806.334115: amidxtaped: > FSF=2 1239982806.334129: amidxtaped: > HEADER 1239982806.334140: amidxtaped: > DEVICE=/dev/nst0 1239982806.334152: amidxtaped: > HOST=^mrx-storage3 1239982806.334166: amidxtaped: > DISK=^/array/x_ff4$ 1239982806.334178: amidxtaped: > DATESTAMP=20090327102517 1239982806.334189: amidxtaped: > END 1239982806.336511: amidxtaped: pid 3896 ruid 790 euid 790: rename at Fri Apr 17 11:40:06 2009 1239982806.336719: amidxtaped: Sending output to file descriptor 52 amidxtaped: Using tapedev /dev/nst0 "/dev/nst0" uses deprecated device naming convention; using "tape:/dev/nst0" instead. 1239982806.338169: amidxtaped: device_read_label; mode = 0 1239982808.467833: amidxtaped: device_start mode = 1 1239982808.473085: amidxtaped: device_start done; dev->access_mode = 1, result 1 Scanning volume AmaTor-164 1239982808.473119: amidxtaped: search_a_tape: desired_tape=0x86eaf88 label=AmaTor-164 1239982808.473131: amidxtaped: tape: numfiles = 1 1239982808.473141: amidxtaped: tape: files[0] = 2 1239982808.473151: amidxtaped: current tapefile_idx = 0 1239982859.274778: amidxtaped: Building type 3 (FILE) header of size 32768 using: 1239982859.274815: amidxtaped: Contents of *(dumpfile_t *)0x8705408: 1239982859.274826: amidxtaped: type = 3 (FILE) 1239982859.274837: amidxtaped: datestamp= '20090327102517' 1239982859.274847: amidxtaped: dumplevel= 0 1239982859.274857: amidxtaped: compressed = 0 1239982859.274867: amidxtaped: encrypted= 0 1239982859.274876: amidxtaped: comp_suffix = 'N' 1239982859.274886: amidxtaped: encrypt_suffix = '' 1239982859.274896: amidxtaped: name = 'mrx-storage3.win.mrxfx.com' 1239982859.274906: amidxtaped: disk = '/array/x_ff4' 1239982859.274916: amidxtaped: program = '/bin/tar' 1239982859.274940: amidxtaped: dumper = '' 1239982859.274950: amidxtaped: srvcompprog = '' 1239982859.274960: amidxtaped: clntcompprog = '' 1239982859.274969: amidxtaped: srv_encrypt = '' 1239982859.274979: amidxtaped: clnt_encrypt = '' 1239982859.274989: amidxtaped: recover_cmd = '/bin/tar -xpGf - ...' 1239982859.274999: amidxtaped: uncompress_cmd = '' 1239982859.275008: amidxtaped: encrypt_cmd = '' 1239982859.275018: amidxtaped: decrypt_cmd = '' 1239982859.275028: amidxtaped: srv_decrypt_opt = '' 1239982859.275037: amidxtaped: clnt_decrypt_opt = '' 1239982859.275047: amidxtaped: cont_filename= '' 1239982859.275056: amidxtaped: is_partial = 0 1239982859.275066: amidxtaped: partnum = 0 1239982859.275075: amidxtaped: totalparts = 0 1239982859.275085: amidxtaped: blocksize= 32768 Error reading 32768 bytes from /dev/nst0: Input/output error 1239984420.786613: amidxtaped: Restoration finished 1239984420.78: amidxtaped: pid 3896 finish time Fri Apr 17 12:07:00 2009 What should i be looking at in this instance. The original backup logs did not report any errors while taping. I'm assuming this is more hardware than software (gonna try another tape drive now) but anymore insight is appreciated. Thanks +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] best way to organise tape file listings ?
rory_f wrote: > > Dustin J. Mitchell wrote: > > On Thu, Feb 19, 2009 at 2:52 PM, rory_f > > backupcentral.com> wrote: > > > > > how hard would it be to migrate our naming convention over to match our > > > labels ? is there any way of doing it? > > > > > > > Well, the Amanda label is written on each tape, at the beginning -- so > > you'd need to do some tape-drive trickery to rewrite that > > (single-block) file without overwriting the following data. I'm not > > sure tape drives can do that. > > > > The remaining adjustments in Amanda's metadata would be tricky, but > > not impossible (basically a global search-and-replace over all of > > Amanda's metadata would do the trick). > > > > Dustin > > > > -- > > Storage Software Engineer > > http://www.zmanda.com > > > Yeah > A lot more hassle than it is worth. > > Which ever way i figure out is best i'll be sure to post here and let you > guys know.. ;) I got a script working with php... shows the disklist then the label and the resulting tape. results like so: [ama...@backup tor]$ ./figureouttapes.php |head /array/sata-1/show/SHOTS/033x010 located on: AmaTor-002 MRX157L3 /array/sata-1/show/SHOTS/033x010 located on: AmaTor-003 MRX158L3 /array/sata-1/show/SHOTS/148Ax050 located on: AmaTor-003 MRX158L3 /array/sata-1/show/SHOTS/148Ax050 located on: AmaTor-004 MRX159L3 /array/sata-1/show/SHOTS/148Ax050 located on: AmaTor-005 MRX170L3 :D +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] best way to organise tape file listings ?
Frank Smith wrote: > rory_f wrote: > > > Dustin J. Mitchell wrote: > > > > how hard would it be to migrate our naming convention over to match > > our labels ? is there any way of doing it? > > > > > > If you're talking about changing barcodes to match Amanda's tape label, > couldn't you just replace the barcodes on the tapes with ones matching > your labels (possibly with a preceding digit added to ensure uniqueness > (might have to tweak your labelstr regex to support it)? That way you > wouldn't need to make any changes to Amanda except your barcodes file > (if you configured your changer script to use one). > > Frank > > -- > Frank Smith fsmith < at > hoovers.com > Sr. Systems Administrator Voice: 512-374-4673 > Hoover's Online Fax: 512-374-4501 yeah that would work too :D +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] best way to organise tape file listings ?
Dustin J. Mitchell wrote: > On Thu, Feb 19, 2009 at 2:52 PM, rory_f > backupcentral.com> wrote: > > > how hard would it be to migrate our naming convention over to match our > > labels ? is there any way of doing it? > > > > Well, the Amanda label is written on each tape, at the beginning -- so > you'd need to do some tape-drive trickery to rewrite that > (single-block) file without overwriting the following data. I'm not > sure tape drives can do that. > > The remaining adjustments in Amanda's metadata would be tricky, but > not impossible (basically a global search-and-replace over all of > Amanda's metadata would do the trick). > > Dustin > > -- > Storage Software Engineer > http://www.zmanda.com Yeah A lot more hassle than it is worth. Which ever way i figure out is best i'll be sure to post here and let you guys know.. ;) +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] best way to organise tape file listings ?
Dustin J. Mitchell wrote: > On Thu, Feb 19, 2009 at 11:41 AM, rory_f > backupcentral.com> wrote: > > > i know i can `amadmin tor find | grep DLE` to find the amanda tape, and > > then compare that to changer-barcodes - but its a bit of a long process > > > > I think that's the quickest way to map amanda labels to paper labels, > unfortunately. > > Dustin > > -- > Storage Software Engineer > http://www.zmanda.com how hard would it be to migrate our naming convention over to match our labels ? is there any way of doing it? +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] best way to organise tape file listings ?
Dustin J. Mitchell wrote:
> On Thu, Feb 19, 2009 at 11:41 AM, rory_f
> backupcentral.com> wrote:
>
> > i know i can `amadmin tor find | grep DLE` to find the amanda tape, and
> > then compare that to changer-barcodes - but its a bit of a long process
> >
>
> I think that's the quickest way to map amanda labels to paper labels,
> unfortunately.
>
> Dustin
>
> --
> Storage Software Engineer
> http://www.zmanda.com
i have this figured out..
dle=WS260; am=(`cat /usr/local/etc/amanda/tor/am_tor_find |grep $dle | awk
'{print $6}'`); real=(`grep $am /usr/local/etc/amanda/tor/changer-barcodes|awk
'{print $2}'`); echo "$shot on tape $real($am)"
WS260 on tape MRX316L3(AmaTor-130)
but my basic bash scripting only means its outputting one line of the `am`
command while there is two.. need to figure out a way to output them all .. but
thats a start :)
+--
|This was sent by r...@mrxfx.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
[Amanda-users] best way to organise tape file listings ?
hi, we're trying to organise back up logs for our users to interface with and tell us when they need files restored from tape. i've created all the listings from the index/ directory, made a copy, gzip -d'd them to get the filelisting for each entry - but i'm trying to find a way to figure out which shot is on which tape - amanda label and actual tape label the smartest thing would have been to label the amanda tapes the same way our tapes are but .. yeah. that didn't happen for a few reasons. i know i can `amadmin tor find | grep DLE` to find the amanda tape, and then compare that to changer-barcodes - but its a bit of a long process does anyone have any ideas how to deal with such a problem? thanks. +-- |This was sent by r...@mrxfx.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
[Amanda-users] Restoring amanda tapes without amanda
sf karel wrote: > On Nov 27, 2008, at 4:49 PM, rory_f wrote: > > > > > > As amanda uses tar, you surely can restore a tape (or a portion of > > a tape, perhaps just a dle?) using the command line 'tar' command, > > right ? > > > > no, amanda doesn't always use tar. See the latter part of the AMANDA > chapter from "Backup and Recovery" for the commands to use "mt" and > "dd" to extract files from the tapes and then use tar from the > command line if that is how the backup was made. > > http://wiki.zmanda.com/index.php/Amanda_chapter_in_Backup_and_Recovery i always do my runs with gnutar and i always observe tar processes on my machines when taping is going on so im sure my backups use tar. thanks for the link +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Restoring amanda tapes without amanda
As amanda uses tar, you surely can restore a tape (or a portion of a tape, perhaps just a dle?) using the command line 'tar' command, right ? Is it just the same as a normal extract ? tar -xvf /dev/nst0 ? Or do you have to do other things to ensure this works properly. Nothing is wrong with my amanda configuration, but It's something i want to document for my own reference :) Rory +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Amanda using tapes in a very strange (and wrong) way
Dustin J. Mitchell wrote: > On Wed, Nov 12, 2008 at 10:26 PM, rory_f > backupcentral.com> wrote: > > > i have a feeling it might be hardware too. im gonna try with _brand_ new > > tapes tomorrow, to see if i cannot reproduce this.. > > > > It looks like it might be a changer problem -- I'm seeing several: > > 1226376231.524952: taper: changer: got exit: 2 str: barcode > "MRX157L3" not found in mtx status output > > you should probably check the changer logfile to see if anything else > appears there. You may also want to increase the initial_poll_delay > so that the changer waits longer for the next tape to appear. Usually > when we see this kind of problem, it occurs because the tape drive > hasn't "settled down" by the time Amanda starts writing to it. > > Dustin > > -- > Storage Software Engineer > http://www.zmanda.com Dustin, Ok i will try that. I'll update you with anything else. Thanks +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Amanda using tapes in a very strange (and wrong) way
Dustin J. Mitchell wrote: > On Wed, Nov 12, 2008 at 7:01 PM, rory_f > backupcentral.com> wrote: > > > Amanda hasnt been acting as it should. See the output of a failed dump > > below: > > > > This actually looks like two other bugs we're in the process of > chasing -- but may turn out to be totally unrelated. Which Amanda > version, and which OS, are you using? Can you send along the taper > debug log? > > Dustin > > -- > Storage Software Engineer > http://www.zmanda.com sure i can. give me an email. im on linux, fedora core 7, using amanda-2.6.0p2/ im just wondering why it is has started happening now. in my case, because of: "driver: interface-state time 17584.842 if default: free 98976 driver: hdisk-state time 17584.842 hdisk 0: free 1420454186 dumpers 1 driver: result time 17584.842 from taper: NEW-TAPE 01-00121 AmaTor-054 Got EIO on /dev/nst0, assuming end of tape. Error writing final filemark: Input/output error Error writing final filemark: Input/output error " i have a feeling it might be hardware too. im gonna try with _brand_ new tapes tomorrow, to see if i cannot reproduce this.. but give me your email, i'll send a taper. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Amanda using tapes in a very strange (and wrong) way
Hi,
Amanda hasnt been acting as it should. See the output of a failed dump below:
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:04
Run Time (hrs:min) 3:07
Dump Time (hrs:min)5:03 5:03 0:00
Output Size (meg) 618864.1 618864.10.0
Original Size (meg)618864.1 618864.10.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped 61 61 0
Avg Dump Rate (k/s) 34866.334866.3--
Tape Time (hrs:min)3:03 3:03 0:00
Tape Size (meg)446721.9 446721.90.0
Tape Used (%) 111.0 111.00.0
Filesystems Taped60 60 0
Chunks Taped 74 74 0
Avg Tp Write Rate (k/s) 41721.841721.8--
USAGE BY TAPE:
LabelTime Size %NbNc
AmaTor-040 3:05 462842042K 112.35960
AmaTor-041 0:00 2912K0.0 0 1
AmaTor-043 0:0048576K0.0 0 1
AmaTor-044 0:00 3296K0.0 0 1
AmaTor-045 0:00 3680K0.0 0 1
AmaTor-046 0:00 7840K0.0 0 1
AmaTor-047 0:0010848K0.0 0 1
AmaTor-048 0:0010048K0.0 0 1
AmaTor-049 0:00 3680K0.0 0 1
AmaTor-050 0:00 3296K0.0 0 1
AmaTor-051 0:00 3296K0.0 0 1
AmaTor-052 0:00 3296K0.0 0 1
AmaTor-053 0:00 3296K0.0 0 1
AmaTor-054 0:00 3296K0.0 0 1
AmaTor-055 0:00 3456K0.0 1 1
Why is it writing all that data to the first tape, then failing to write more
than 3-10mb on the rest of the tapes?
We have never observed this behaviour before. Our DLE's are sizes ranging from
1gb to 200gb, and we have been fine doing as such up until now.
We used root-tar on our DLE, taperalgo largestfit and dumporder "TT" - as
from the amanda wiki. We also tried reverting back to taperalgo first and
dumporder "sssS" to see if this wuold make a difference and we still saw these
errors.
The only thing i can see from the amdump is:
driver: interface-state time 17584.842 if default: free 98976
driver: hdisk-state time 17584.842 hdisk 0: free 1420454186 dumpers 1
driver: result time 17584.842 from taper: NEW-TAPE 01-00121 AmaTor-054
Got EIO on /dev/nst0, assuming end of tape.
Error writing final filemark: Input/output error
Error writing final filemark: Input/output error
what does this suggest? the tapes need to be rewound? im doing so for our next
(attempted) dump. im gonna go back to using'span' in our DLE too, as we never
had this tape writing problem previously.
Please let me know if you need more info,
thanks.
+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--
[Amanda-users] amcheckdump output - questions :)
rory_f wrote: > > Paul Bijnens wrote: > > > > Indeed, use a changer and set "runtapes" to something greater than 1. > > Amanda will write a tape until she hits End Of Tape (signaled by a write > > error actually), and then move to a new tape. The whole backup image > > will be restarted on the next tape. (With using "tape_splitsize" you > > avoid starting the whole image, but start only the last chunk again) > > > > When using a holdingdisk, the dumps are collected in the holdingdisk > > unless the estimated backup image is larger than the holdingdisk. > > When one complete backup image is finished, Amanda start to put it on tape. > > Because many dumps can be collected simultaneously to holdingdisk, > > several of them may be waiting to be written to tape. If there are > > several to choose from, then the "taperalgo" setting decides which one > > will get tried first. A good setting is "largestfit" (see the wiki url > > above). If no backupimage that is ready will fit in the estimated > > available tapespace, then amanda will try the smallest, which has > > the largest chance of fitting in the unknown free space. > > > > The free space at the end of a tape is indeed unknown, because > > tapes do not always have exactly the same length; and when using > > software compression, the remaining space is even more uncertain, > > because it depends on the compressability of the previous tape images. > > > backup image, meaning DLE? so if it runs out of space, it will right, for > instance, 35% of the DLE to the last bit of the tpae, then start again on the > new tape, thus meaning all data, in teh end, will get written to tape. and > we can use amadmin find to see what tape the whole DLE was dumped to, > right? > > we currently have our runtapes to 28. as that is what our library can hold. > > it all seems a lot clearer now. thanks just as something more to this. last night i backed up what i thought was ~340gb. it was actually a lot more than that (3 tapes worth) - i did this for DLEs: site.com /array/sata-1/aaa/020 root-tar site.com /array/sata-1/aaa/030 root-tar site.com /array/sata-1/aaa/040 root-tar site.com /array/sata-1/aaa/050 root-tar site.com /array/sata-1/aaa/001 root-tar site.com /array/sata-1/aaa/014 root-tar site.com /array/sata-1/aaa/S020 root-tar site.com /array/sata-1/aaa/S035 root-tar site.com /array/sata-1/aaa/S039 root-tar site.com /array/sata-1/aaa/S040 root-tar site.com /array/sata-1/aaa/S045 root-tar and it ended up being about 1.3tb, so it went onto 3 tapes. i got errors doing amcheckdump but i succesfully restored files from the tapes. two of the DLE's got 'PARTIAL' then 'OK" results in `amadmin tor find ` - so i am to assume they worked. am i missing anything? +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] amcheckdump output - questions :)
Paul Bijnens wrote: > > Indeed, use a changer and set "runtapes" to something greater than 1. > Amanda will write a tape until she hits End Of Tape (signaled by a write > error actually), and then move to a new tape. The whole backup image > will be restarted on the next tape. (With using "tape_splitsize" you > avoid starting the whole image, but start only the last chunk again) > > When using a holdingdisk, the dumps are collected in the holdingdisk > unless the estimated backup image is larger than the holdingdisk. > When one complete backup image is finished, Amanda start to put it on tape. > Because many dumps can be collected simultaneously to holdingdisk, > several of them may be waiting to be written to tape. If there are > several to choose from, then the "taperalgo" setting decides which one > will get tried first. A good setting is "largestfit" (see the wiki url > above). If no backupimage that is ready will fit in the estimated > available tapespace, then amanda will try the smallest, which has > the largest chance of fitting in the unknown free space. > > The free space at the end of a tape is indeed unknown, because > tapes do not always have exactly the same length; and when using > software compression, the remaining space is even more uncertain, > because it depends on the compressability of the previous tape images. backup image, meaning DLE? so if it runs out of space, it will right, for instance, 35% of the DLE to the last bit of the tpae, then start again on the new tape, thus meaning all data, in teh end, will get written to tape. and we can use amadmin find to see what tape the whole DLE was dumped to, right? we currently have our runtapes to 28. as that is what our library can hold. it all seems a lot clearer now. thanks +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] amcheckdump output - questions :)
Paul Bijnens wrote: > On 2008-11-05 17:53, rory_f wrote: > > > > Um, haha. its all on the same machine. i dont think thats the case. > > Im actually finishing up a backing up a 6tb project right now using > > tape_splitsize, and perhaps from now on i wont use it. we have a > > system in place to breakdown folder sizes for our whole file system > > here, so using just single tapes and doing root-tar whilst figuring > > out tape capacities ourself isnt actually out of the question. > > > > > I'm not sure if I misunderstood you, just to be sure you understand me... > > These two concepts are orthogonal: > - Amanda can use multiple tapes for one backup run. > - Amanda can split one backup image over multiple tapes. > > It is only the second feature using the "tape_splitsize" parameter that makes > restores more difficult. > See: http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes > > But instead of dumping one enourmous DLE in one backup image, you can use > the include/exclude options with gnutar to split the DLE into smaller ones, > each fitting on a tape. > See: http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists > > Figuring out tapecapacity and optimizing tape usage is handled with the > parameter "taperalgo largestfit", as explained here: > http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25 > > -- > Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 > Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 > http://www.xplanation.com/ email: Paul.Bijnens < at > xplanation.com > *** > * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * > * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * > * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * > * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * > * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * > * ... "Are you sure?" ... YES ... Phew ... I'm out * > *** ok, smaller DLEs is fine. we have been doing this. i just thought using span was the only way for amanda to intelligently change tapes when a tape got full. we use a changer script. is that all we need to ensure it will change tapes when a tape gets full? for instance, we just want to make sure amanda will look at the size of the next dump, and move onto a new tape if needed. right? we use a holding disk also, which im sure makes it easier for amanda to know when to change tapes. ultimately we want to setup the right DLEs, and archive a whole proejct, around 6tb at at time. no interaction in this process is key. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] amcheckdump output - questions :)
Jon LaBadie wrote: > On Wed, Nov 05, 2008 at 09:39:36AM -0500, rory_f wrote: > > > > > the question again then i guess? > > > > hi guys. > > > > we're run amcheckdump on a backup we just did and it has given us a few > > outputs we're not sure about -whether it is tar being non-understanding of > > a backup using spanned tapes, or something else? we're a bit lost so > > hopefully someone can help > > > > ps. ignore the file paths below, i just changed them to cover our > > hostname/directories. > > > > > > ..20081029235711 level 0 part 4 on tape AmaTor-007 file #1 > > "/dev/nst0" uses deprecated device naming convention; > > using "tape:/dev/nst0" instead. > > /bin/gtar: Read 2048 bytes from - > > /bin/gtar: Unexpected EOF in archive > > /bin/gtar: Error is not recoverable: exiting now > > Validation process returned 2 (full status 512) > > > > As above, a good number of the error messages were generated > by /bin/gtar itself. Is it possible that the backups were > made with one version of gnutar and amcheckdump is calling > a different one with some incompatibilities? > > Nah, that never happens ;) > > jl > > > > > using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > > > /tmp/amanda_amcheckdump'. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 5 on tape AmaTor-007 file #2 > > Continuing with previously started validation process. > > Error reading 32768 bytes from /dev/nst0: Input/output error > > Error reading device or writing data to validation command. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 1 on tape AmaTor-007 file #3 > > Could not seek to file 3 of volume AmaTor-007. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 2 on tape AmaTor-007 file #4 > > Details of dump at file 4 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 3 on tape AmaTor-007 file #5 > > Details of dump at file 5 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 4 on tape AmaTor-007 file #6 > > Details of dump at file 6 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 5 on tape AmaTor-007 file #7 > > Details of dump at file 7 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 6 on tape AmaTor-007 file #8 > > Details of dump at file 8 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 7 on tape AmaTor-007 file #9 > > Details of dump at file 9 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 8 on tape AmaTor-007 file #10 > > Details of dump at file 10 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 9 on tape AmaTor-007 file #11 > > Details of dump at file 11 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 10 on tape AmaTor-007 file #12 > > Details of dump at file 12 of volume AmaTor-007 do not match logfile. > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 11 on tape AmaTor-008 file #1 > > "/dev/nst0" uses deprecated device naming convention; > > using "tape:/dev/nst0" instead. > > using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > > > /tmp/amanda_amcheckdump'. > > /bin/gtar: This does not look like a tar archive > > /bin/gtar: Skipping to next header > > /bin/gtar: Archive contains obsolescent base-64 headers > > Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp > > 20081029235711 level 0 part 12 on tape AmaTor-008 file #2 > > Continuing with previously started validation process. > > /bin/gtar: Error exit delayed from previous errors > > Validation pro
[Amanda-users] amcheckdump output - questions :)
the question again then i guess? hi guys. we're run amcheckdump on a backup we just did and it has given us a few outputs we're not sure about -whether it is tar being non-understanding of a backup using spanned tapes, or something else? we're a bit lost so hopefully someone can help ps. ignore the file paths below, i just changed them to cover our hostname/directories. ..20081029235711 level 0 part 4 on tape AmaTor-007 file #1 "/dev/nst0" uses deprecated device naming convention; using "tape:/dev/nst0" instead. /bin/gtar: Read 2048 bytes from - /bin/gtar: Unexpected EOF in archive /bin/gtar: Error is not recoverable: exiting now Validation process returned 2 (full status 512) using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > /tmp/amanda_amcheckdump'. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 5 on tape AmaTor-007 file #2 Continuing with previously started validation process. Error reading 32768 bytes from /dev/nst0: Input/output error Error reading device or writing data to validation command. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 1 on tape AmaTor-007 file #3 Could not seek to file 3 of volume AmaTor-007. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 2 on tape AmaTor-007 file #4 Details of dump at file 4 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 3 on tape AmaTor-007 file #5 Details of dump at file 5 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 4 on tape AmaTor-007 file #6 Details of dump at file 6 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 5 on tape AmaTor-007 file #7 Details of dump at file 7 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 6 on tape AmaTor-007 file #8 Details of dump at file 8 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 7 on tape AmaTor-007 file #9 Details of dump at file 9 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 8 on tape AmaTor-007 file #10 Details of dump at file 10 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 9 on tape AmaTor-007 file #11 Details of dump at file 11 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 10 on tape AmaTor-007 file #12 Details of dump at file 12 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 11 on tape AmaTor-008 file #1 "/dev/nst0" uses deprecated device naming convention; using "tape:/dev/nst0" instead. using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > /tmp/amanda_amcheckdump'. /bin/gtar: This does not look like a tar archive /bin/gtar: Skipping to next header /bin/gtar: Archive contains obsolescent base-64 headers Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 12 on tape AmaTor-008 file #2 Continuing with previously started validation process. /bin/gtar: Error exit delayed from previous errors Validation process returned 2 (full status 512) Whats the easiest way to figure out what file(s) the error is refering to? And/Or is it really something to worry about. What about the last error, validation process returned 2? What does this mean? is there any way to run a check on the tape itself, and not the whole backup again, to save time? Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] amcheckdump output - questions :)
Hi guys, any ideas?? +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] amcheckdump output - questions :)
hi guys. we're run amcheckdump on a backup we just did and it has given us a few outputs we're not sure about -whether it is tar being non-understanding of a backup using spanned tapes, or something else? we're a bit lost so hopefully someone can help ps. ignore the file paths below, i just changed them to cover our hostname/directories. ..20081029235711 level 0 part 4 on tape AmaTor-007 file #1 "/dev/nst0" uses deprecated device naming convention; using "tape:/dev/nst0" instead. /bin/gtar: Read 2048 bytes from - /bin/gtar: Unexpected EOF in archive /bin/gtar: Error is not recoverable: exiting now Validation process returned 2 (full status 512) using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > /tmp/amanda_amcheckdump'. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 5 on tape AmaTor-007 file #2 Continuing with previously started validation process. Error reading 32768 bytes from /dev/nst0: Input/output error Error reading device or writing data to validation command. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 1 on tape AmaTor-007 file #3 Could not seek to file 3 of volume AmaTor-007. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 2 on tape AmaTor-007 file #4 Details of dump at file 4 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 3 on tape AmaTor-007 file #5 Details of dump at file 5 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 4 on tape AmaTor-007 file #6 Details of dump at file 6 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 5 on tape AmaTor-007 file #7 Details of dump at file 7 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 6 on tape AmaTor-007 file #8 Details of dump at file 8 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 7 on tape AmaTor-007 file #9 Details of dump at file 9 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 8 on tape AmaTor-007 file #10 Details of dump at file 10 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 9 on tape AmaTor-007 file #11 Details of dump at file 11 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 10 on tape AmaTor-007 file #12 Details of dump at file 12 of volume AmaTor-007 do not match logfile. Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 11 on tape AmaTor-008 file #1 "/dev/nst0" uses deprecated device naming convention; using "tape:/dev/nst0" instead. using '/bin/gtar tf - > /tmp/amanda_amcheckdump && cat > /tmp/amanda_amcheckdump'. /bin/gtar: This does not look like a tar archive /bin/gtar: Skipping to next header /bin/gtar: Archive contains obsolescent base-64 headers Validating image .com:/array/sata-1/.../SHOTS/something/ datestamp 20081029235711 level 0 part 12 on tape AmaTor-008 file #2 Continuing with previously started validation process. /bin/gtar: Error exit delayed from previous errors Validation process returned 2 (full status 512) Whats the easiest way to figure out what file(s) the error is refering to? And/Or is it really something to worry about. What about the last error, validation process returned 2? What does this mean? is there any way to run a check on the tape itself, and not the whole backup again, to save time? Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Easiest way to create a full TOC of each tape?
Hey guys, How can this (see title) be done? We need to have a full list of all directories sent to the tape for easy and quick restore process. Would a simple `dd if=/dev/nst0 bs=32k skip=1 |/bin/gtar -tvf -` output a full listing? Is there a program within amanda to do so? I know there is the `tapecat` tool but this,when i ran it, did not output a full TOC. Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Question corrupt amanda backup / tape speed / etc..help!
We actually made some changes on our system we're using for backing up(bigger holding disk, sata2 instead of sata1) and we're backing up fresh again (using the same tapes again) Asssuming the tapes themselves are not broke (they are new tapes), hopefully this will work this time. if it does give us corrupt data again, i'll post more logs. I dont have them to hand right now (i might do on aonther machine i'll check later, if my ssh session is still up:)) Thanks +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
[Amanda-users] Question corrupt amanda backup / tape speed / etc..help!
This is an edited version of a previous post
Hi,
We managed to up our speed of dumping and writing a whole lot by using a
holding disk, which is great. However, the data we wrote, around 245gb, failed
a amcheckdump and i couldnt, therefore, not amrestore anything.
After the amtapetest, i was given an output of this:
define tapetype ibm {
comment "just produced by tapetype prog (hardware compression off)"
length 402432 mbytes
filemark 0 kbytes
speed 57497 kps
}
However, it says in the amreport we wrote to tape at :
Avg Tp Write Rate (k/s) 90801.790801.7--
Here is some of our amdump output:
diver: result time 6482.541 from taper: PARTDONE 00-2 AmaTor-004 5 41943040
"[sec 449.501801 kb 41943040 kps 93310.059952]"
driver: state time 6950.318 free kps: 10 space: 719002550 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 6950.318 if default: free 10
driver: hdisk-state time 6950.318 hdisk 0: free 719002550 dumpers 0
driver: result time 6950.332 from taper: PARTDONE 00-2 AmaTor-004 6
41943040 "[sec 466.669805 kb 41943040 kps 89877.338432]"
driver: state time 7023.335 free kps: 10 space: 719002550 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 7023.335 if default: free 10
driver: hdisk-state time 7023.335 hdisk 0: free 719002550 dumpers 0
driver: result time 7023.350 from taper: PARTDONE 00-2 AmaTor-004 7 5813290
"[sec 71.901664 kb 5813290 kps 80850.562791]"
driver: state time 7023.350 free kps: 10 space: 719002550 taper: writing
idle-dumpers: 1 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 7023.350 if default: free 10
driver: hdisk-state time 7023.350 hdisk 0: free 719002550 dumpers 0
How can this be? our holding disk is a sata1, 1tb disk. That write rate is
above what sata1 can give data out at, right? is this the reason for our data
corruption?
What else could it be? Do you need anymoer info to help diagnose this? We're
gonna hopefully try with a sata2 disk soon to see if that makes any difference.
The tapespeed variable in amanda.conf, to my understanding, would not allow me
to write at 9k~. Nor would the speed of a sata1 disk.
+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--
