Re: Backup startegy help
On Tue, Dec 21, 2004 at 12:49:01PM +0200, Shai Ayal wrote: > Hi all, > > I'm trying to set up an backup system in our company. We have two major > file systems to backup - one is the homedirs of everyone, including mail > in Maildir format, and the other is data. > > Our idea was to have two computers -- one would hold the data and homes, > and the other would back it up daily to a disk. Additionally, once a > week or so we would to a tape backup to take off site. > > The data is quite huge and it is often updated but only in small > amounts, i.e. we have about 80GB of data, and we add less than 300MB > each week. ... Huge is in the eye of the beholder :) Some people on the list describe their environments in TB, not GB. > ... Clearly doing a full backup every week is very wasteful. Not necessarily. I recall an incident where I did not realize a file critical to my system had been damaged. However it did not come into play except during boot, and unix systems don't often reboot (at least mine :) When I stepped through my weekly level 0's I found when it changed, restored it, and was able to determine the cause based on what else happened at that time. > > Homedirs are standard and I think the usual amanda backup algorithm will > work with no problem. That's a problem with standard's, there are so many of them :) > > Problems: > > 1. Can I implement two different backup strategies, and write the result > to the same tape? i.e. although the strategy for home and data is > different, they would go in the same tape if they are scheduled for the > same day? Not sure you will really have two different stratagies. The simple answer is no, you can't have two amanda configs writing to the same tape. But there is a great deal of flexibility in how each backup entity (called a DiskList Entry or DLE in amanda) can be configured. Different dumpcycles, exclude lists, dump program, priority, strategy, ... > > 2. What is the recommended strategy for backing up the data? "strategy > nofull" seems appropriate, but does it work? -- the manual seems to be > ambiguous about it. Does anyone have other recommended strategy ? > I'm pretty sure some of those strategies are for those admin's who want to manually schedule full backups. Maybe there are other uses of which I'm unaware. But amanda does assume having full backups done. > 3. On the same subject, can someone help me with the technical terms of > backup level: I understand level 0 is full backup, but do level 1 > backups depend on the last level 0 backup done or on the last level 1 > backup done ? > There does seem to be some variation in the term incremental between different backup management schemes. Amanda terminology calls anything other than a full backup (aka level 0) of a DLE an incremental. The various levels are relative to the last backup of a higher (lower numerically) backup. So if if a DLE gets a level 0 on Sunday, then any level 1 done until the next level 0 is all the changes relative to the level 0. If a level 2 is done, it is the changes since the last level 1 ... > Thanks in advance for any help > > Shai Ayal > > > > > > > >>> End of included message <<< -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Flushing file: device to tape
On Wed, Dec 22, 2004 at 12:55:28PM +0100, Nicki Messerschmidt, Linksystem Muenchen GmbH wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi, > at a customer I recently switched from tape backup to file device backup > due to a tape drive failure. Now, after the real tape drive is back > again, I'd like to tape the harddisk content to tape. > > How do I do this? > Does "for i in $TAPEPATH/*; do dd if=$i of=$TAPEDEV bs=32k; done" the job? > That might get them onto the tape, but not in a form usable by amanda. I'm not certain how one goes about making it suitable for amanda. Here are the problems I see with the approach. - no tape header file identifying the tape, date, ... - no tape trailer file indicating end of tape - no amanda index info saying which tape contains which dumps - no amanda index info saying the position on the tape for each dump Did you name the virtual tapes the same as the physical tapes? If so, then maybe, note the maybe - I'm very unsure, you might be able to do the loop above to a physical tape of the same name. It would have to be sandwiched between dd's of the header and trailer. OTOH, what you propose is non-destructive to the vtape data. So you could do a sample and check it emperically. Any chance of just keeping the vtapes around for a tapecycle? Keep a copy of the amanda.conf dir tree as it stands now. Then change the working copy to physical tapes configuration. Should you need to recover from the vtapes, rename the config dirs. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Antw: Re: Filesize too big
On Wed, Dec 22, 2004 at 08:47:21AM +0100, Mathias Indermuehle wrote: > Thanks for your answer.. > > Badly it doesn't realy help.. the problem is not on the holdingdisk, > its on the "tape" - I use a harddisk to backup these files. > The setting made smaller files at the holdingdisk, but not on the tape > (destination disk). > > If there is no other posibility it would be a hint if someone can tell > me how to get these splitet files togethere.. (then I can use a script > to put them on the backup disk manualy and when I need a restore I can > put them togethere and unpack them) I, and I'm sure Gene and others, thought you were writing to tape, not virtual tapes. On the disk drive where the vtapes reside, is there sufficient space? If so, can't you just increase the virtual size in the tapetype definition? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Antw: Re: Filesize too big
On Wed, Dec 22, 2004 at 04:01:08PM +0100, Mathias Indermuehle wrote: > Im sorry if I don't make myself clear.. > > Yes im writing to vtapes (harddisk). Yes there is sufficient space on > the destination disk.. and I tried to increase the virtual size, but > that doesn't helps. > The problem is that the destination harddisk has an fat16 or something > filesystem, and its only provide 2GB max filesize. > Sorry to be blunt, put your vtapes on a different type of file system or get your individual DLE's below 2GB. I see no alternatives. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup startegy help
On Thu, Dec 23, 2004 at 03:33:24PM +0200, Shai Ayal wrote: > OK, you can all stop laughing at me calling 80GB Huge. Sorry. > > Anyway, we do have a 80/160GB DLT tape. However I do not like it for 2 > reasons: > 1. Our previous 40/80 DLT tape let us down and the tape got eaten by the > tape drive at the most critical time. > 2. Irrespective of the tape reliability, someone has to change the tape > each day, since a tape changer is clearly way out of our budget. Knowing > the people here, and me in particular, this won't happen. > > This is why a daily disk drive backup is attractive to us. We are > content with having just 14 days of history. each week we will also do a > tape backup for off site storage. > > As for the strategy -- If I have some (OK, not HUGE) body of data that > is unchanging by definition, I would like Amanda to do one level 0, and > then continue with lower levels until this becomes unpractical (i.e. > sizeof(level 1) ~ sizeof(level 0) ). I would only need to keep the level > 0 and 14 level 1's to have a 14 days day by day history. This is almost > what Amanda does, only there is a dumpcycle limit in days. Can this be > done ? > Some unorganized thoughts. Use 2 configs, they could both be amanda configs or the daily could be amanda, the second one (to tape) any backup scheme. This is similar to what I use, Daily and Archive except both of mine go to tape and my Archive config is used on a sporadic (at best) schedule. You would have to make sure they do not attempt to run at the same time. Do you really need a daily backup for both saturday and sunday? I've elected to do six daily config backups a week. One runs Sat after midnight to cover friday changes and the next on Mon after midnight to cover both weekend days. That leaves me from about 6AM Sat to 1AM Mon to do an archive if I wish. This will be old had to A-U readers. You don't want to do all your level 0's then 14 days later redo them. You will probably be reusing the storage medium, disk or tape. If so, then you are wiping out the last set of level 0's when you make the new ones. Should something happen during the dump, you may be left with NO level 0's. Amanda doesn't want to do all the level 0's at the same time and then level 1's till the next complete set of level 0's. It was designed specifically to avoid that scheme. It can be forced into such a scheme, but it is not recommended. Instead amanda maintains a collection of entities to backup, file-systems, directory trees, ... These are kept in a file called "disklist" and the entities are thus called disklist entries (DLEs). Each DLE can be customized if need be (dumpcycle, priority, ...) but in general you will specify a global dumpcycle that applies to all DLEs by default. Amanda then tries to balance the total amount of data backed up during each run. So some DLEs will get level 0s on Mon, others Tue, ... Depending on the balancing act, some DLEs will have their level 0s shifted around. But each DLE will get "at least one" level 0 during any dumpcycle period. By default amanda will then drop to level 1s and lower depending on the amount of new data. If there is very little, it will drop to level 2, if more, do another level 1, if a lot more (say you downloaded several cd's) it might even decide to do another level 0 since it would be nearly the same size as the level 1. It seems to me that the default scheme would be good for your daily config to virtual tapes (disk). But don't pick a dumpcycle of 14 and have only 14 vtapes. Either maintain a lot more vtapes on disk or shorten your dumpcycle a lot. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: New setup: problem with permissions
On Tue, Dec 28, 2004 at 05:15:30PM -0700, Dwight Tovey wrote: > I'm looking at using Amanda for my backup solution, and I've run into a > bit of a problem that I'm hoping to get some help with. > ... > > If I try to use the device instead of the mountpoint, amcheck (and > eventually amdump) fails with "Can't open disk 'hda6'" for each 'include'. > Looking at the source I found that every time it encounters an 'include', > Amanda trys to do an 'opendir()' to verify the included name. Since the > device is not a directory, the opendir() fails resulting in the displayed > error. > > So, is there any way around this short of running Amanda as 'root'? Is > there really any problem with running as root? If the amanda installation was done as root, then all the necessary set-user-id-root permissions should have been done at that time. Amanda tries to run with the minimum necessary permissions, but at times, the minimum is root. Thus several amanda programs are setuid'ed during installation. These include the "runtar" program that invokes gnutar. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: short write - could that mean that the "use" on holdingdisk is too small?
On Fri, Dec 31, 2004 at 03:02:15PM +0100, Peter Guhl wrote: > Hello all > > This error happens all the time - but sometimes it vanishes and then it > comes again. Now I have got the idea that the space allocated at the > holding disk is too small (not the holding disk itself). The only other > explanation is a hardware defect on the DLT Drive but mt and every other > diagnosis I tried don't look like that. > > Could it be that I am on the right track? > > Is it a general problem that there's less space at the holding disk than > at the tape? I guess not because in the past this was fairly common. A grep for "short write" in the source files shows it present in only two files, taper.c and tapeio.c. This confirms my notion on reading your note that it is a tape problem, not a holding disk problem. I only see that particular message when the total size of what is being taped (all DLE's total size) is greater than the capacity of a single tape. The last DLE that tried to fit on the tape does not, so it gets a "short write". In my case, with multiple tapes per dump, the DLE that encountered the end of the tape gets put first onto the next tape. Depending on your setup the same could happen or the DLE could be left on the holding disk for later flushing (automatic or manual) or the dump could fail if it were being written direct to tape. This latter situation is the only one I can think of where the holding disk would come into play. But it would have affected whether the dump went to tape or holding disk, not whether it got the short write. Have you modified the "reserve" parameter from the default 100%. That does affect how much of your holding disk is available for level 0 dumps. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: re-labelling or duplicating a tape?
On Mon, Jan 03, 2005 at 10:36:51AM -0500, [EMAIL PROTECTED] wrote: > > Hi all, > > Is there an easy way to change the amanda label of a tape that's been > written to? Short of that, is there an easy way to recover the > contents of the tape in amdump form, then re-label the tape and use > amflush to get the contents on the newly/properly labelled tape? Quick answer is sure, just overwrite the 32KB first file on the tape. Longer answer dependes on what you want to do with the tape and how much you expect amanda to remember about the tape, its contents, and its new label. To overwrite it, do a "dd if= of= bs=32k". Maybe make a copy of so if you mess up you can restore it. Next edit the file to change the tape name. Then rewind the tape and do the same dd command with 'if' and 'of' reversed. Note, after doing this amanda has an invalid concept of what is on the tape. It still thinks the other tape exists and has valid contents. Depending on what new tape name you choose, the modified tape is either not known by amanda's index (a new tape name) or does not match the index's listing of the contents. So depending on what you plan to do with the modified tape this is may or may not be a reasonable thing to do. If you don't need to use the index (i.e. use amrestore and not amrecover) then fine, modify it and amrmtape the original name. Otherwise a different action may be the wisest choice. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: re-labelling or duplicating a tape?
On Mon, Jan 03, 2005 at 03:28:03PM -0500, [EMAIL PROTECTED] wrote: > > In a message dated: Mon, 03 Jan 2005 13:14:02 EST > Jon LaBadie said: > > >Quick answer is sure, just overwrite the 32KB first file on the tape. > > > >Longer answer dependes on what you want to do with the tape and how > >much you expect amanda to remember about the tape, its contents, and > >its new label. > > ... > > Hmm, okay. Ideally I'd like amanda's knowledge of what's on the tape > to change with the tape label. > > This is the problem: I have a configuration called 'archive', the > sole purpose of which is be stored off site indefinitely. There a > no-reuse policy on these tapes. The tape labels are of the form > '-Q[1-4]-Tape-X/Y' This way, I need only know from which > quarter I need to restore from should I ever need to pull one of > these tapes from storage. Last week I ran this configuration and > labelled the tape as 1/2, but we only used 1 tape, not 2. > > What want to accomplish is changing this tape to be 1/1 instead of 1/2. > So, I can either edit the tape header and the amanda database, or > dump the tape to disk, then flush it to a new tape, which actually > sounds easier. > > So, how would I extract the contents of a tape so I could flush it to > another tape? I know of no way to accomplish what you are looking to do. Doesn't mean it can't be done, I just don't know of any and I can see lots of potential problems trying to accomplish it. Perhaps for a future version of amanda someone could write a routine to "import" a tape(s) into the index and tapelist. Anyone anxious to do some coding for the amanda project :) As an alternative, couldn't you change the human aspect of the problem? Amanda is quite happy with things the way they are. It has a label, has written data to a tape of that label, has an index of the contents, doesn't know (or care) that you expected to need two tapes. There might be an extra tape labeled ...2/2, but that can be "rmlabel'ed" and relabeled. Seems the only one with problems is the humans involved. Couldn't they find a way to note that the second tape was not used for this archive? Maybe even an empty tape box or slot with a note in it? Just my one fiftyith of a dollar. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: A question regarding backing up Window machines
On Tue, Jan 04, 2005 at 02:42:13PM -0800, Joe Rhett wrote: > On Mon, Dec 20, 2004 at 04:02:49PM +0100, Paul Bijnens wrote: > > - You can exclude only one pattern. > > Um, hello -- can you clue me in on this? I am trying (lazily) to figure > out why my excludes are ignored during backups, but work just fine when I > run the same tar command by hand. I need to add some debugging code, and > just haven't found the time. Have you clued into something I've missed? > Joe, could you give us your example exclude (note it is singlular) pattern that fails with amanda dumps but works with smbclient. Also the path to file(s) on the window machine that is excluded properly by smbclient but gets included by amanda. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: A question regarding backing up Window machines
On Wed, Jan 05, 2005 at 03:58:29PM -0800, Joe Rhett wrote: > > > On Mon, Dec 20, 2004 at 04:02:49PM +0100, Paul Bijnens wrote: > > > > - You can exclude only one pattern. > > > On Tue, Jan 04, 2005 at 02:42:13PM -0800, Joe Rhett wrote: > > > Um, hello -- can you clue me in on this? I am trying (lazily) to figure > > > out why my excludes are ignored during backups, but work just fine when I > > > run the same tar command by hand. I need to add some debugging code, and > > > just haven't found the time. Have you clued into something I've missed? > > On Tue, Jan 04, 2005 at 06:17:31PM -0500, Jon LaBadie wrote: > > could you give us your example exclude (note it is singlular) pattern > > that fails with amanda dumps but works with smbclient. Also the path > > to file(s) on the window machine that is excluded properly by smbclient > > but gets included by amanda. > > No such animal. I never said smbclient, I said tar. The system is running > the amanda client. Your subject says "Window machines". I assumed you were backing up samba shares. That uses smbclient with its "tar" option. If you had meant the much less commonly used cygwin port of amanda, which does use gnutar, I thought you would have mentioned it. Or maybe there is another possibility I haven't considered. Give me a hand here. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Client failure problem -answer
On Thu, Jan 06, 2005 at 10:46:30AM +, Keith Matthews wrote: > On Wed, 5 Jan 2005 15:15:01 -0500 > > Did you perchance increase the etimeout and dtimeout values first? > > > > Nope, partly as there is either no documentation on them or it's well > hidden. Yup, we keep the man page for amanda well hidden :)) > > ... . As long as the spindle numbers used are > > assigned to the individual drive per number, I've not even had any > > disk thrashing problems either. > > > > This was all one spindle. But did you tell amanda that? If you don't tell it, by indicating which DLE's are on the same spindle, amanda assumes each is a separate spindle and might try many simultaneous dumps from that spindle. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amcheck hang
On Thu, Jan 06, 2005 at 02:19:42PM -, Dan Tomlinson wrote: > Hi all, > > amanda has been failing during my nightly dumps: > > FAILURE AND STRANGE DUMP SUMMARY: >mymachine/var/lib/mysql/ lev 0 FAILED [mymachine NAK: amandad > busy] > > > > Trawled newsgroups for answers and it appears that this particular error > can be caused by amandad processes hanging around after a failed amanda > operation. Did a quick "ps -ef | grep amanda" and saw there were > "amandad" and "..amanda/selfcheck" processes active. Tried to kill them, > but only the amandad would die :o( Eventually managed to kill the > selfcheck with a kill -9 > > I labeled up another new tape to attempt to manually continue the > dumpcycle, and ran an "amcheck" after labelling, only to have it fail to > finish. Another "ps -ef | grep amanda" found that there was now a new > selfcheck process and a new amandad. > > So the problem seems to be that the selfcheck process is hanging during > amcheck execution and preventing the dump from finishing. As to how to > solve it? Various in the mailing lists suggest rebooting machine but a) > this is inconvienient to say the least, and b) how do I know this wont > just happen again next time amanda runs... > Feeling a little snarky here, how do I know the sun will come up tomorrow? > I am using amanda version 2.4.2p2 on debian with a 2.4 kernel, any > ideas? Do we need to upgrade our amanda? Basically Dan, you need to determine "why" selfcheck is not finishing. That will take some research at your end. Perhaps some of the debug files in /tmp/amanda may help. I'm having a similar, but totally unrelated problem at the moment, my tape drive is acting up. When taper tries to open the drive, the open call never returns and taper just sits there. Its not a bug in amanda's taper program, but for some reason the OS and its tape device driver return neither a success nor a failure to taper. So it is making a call to the system and until it returns, nothing more happens. My solution until I figure out what is wrong with the drive is to turn it off when I'm not trying to analyze it. Now when taper tries to open it in the middle of the night, an error is returned (no such device) and the backups collect in a fast shrinking holding disk space. I'm not an advocate of keeping up with the latest and greatest. If something is doing the job, no need to upgrade. While your version is pretty old, a survey done about 20 months ago showed 30-40% of servers and clients were using 2.4.2 versions. I'd expect substantially fewer today. Certainly little (aka none) effort is going into maintaining that code base. There are many good reasons for upgrading. I doubt your current problem is one of them. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrestore problem, headers ok but no data
On Fri, Jan 07, 2005 at 02:01:26PM -0500, Eric Siegerman wrote: > On Fri, Jan 07, 2005 at 11:17:21AM -0500, Brian Cuttler wrote: > >samar 170# dd of=/dev/sdlt2 obs=32k if=./scratch >64+0 records in >1+0 records out > >"bs" block size, "obs" outpub BS, (there is an IBS also, which I >am afraid of developing should this not resolve soon) > "bs" is just a short cut to saying "obs" and "ibs". So the following are the same: obs=32k ibs=32k bs=32k It is only necessary to use obs and ibs if they are different (including an unspecified one of the default 512 bytes). > > > > Oddly trying to dd if=/dev/rmt/tps... read no data > > > > samar 85# mt -f /dev/rmt/tps1d4nrns rewind > > samar 86# dd if=/dev/rmt/tps1d4nrns of=scratch > > Read error: Invalid argument > > 0+0 records in > > 0+0 records out > > These two things might well be related. That dd command, without > a "bs=" argument, is trying to read 512-byte blocks. But the > physical blocks on the tape are 32 KB -- your adjustments have > seen to that. It would be appropriate for the read() call to > fail in that situation, as indeed it did. On Solaris (whose man > pages I have access to at the moment), the error status would be > ENOMEM; perhaps on your system it's EINVAL == "Invalid argument" > instead. (The place to look that up would likely be in the man > page for the tape driver -- st(7) is where I found the Solaris > version.) That ENOMEM, reported as "insufficient memory" sometimes used to throw me for a loop. Here is the situation as I understand it. To enhance performance dd tries to do unbuffered I/O, meaning the data goes directly to memory in dd rather than through buffers in the OS and then to dd. An upshot of this is that the buffer dd reads into must be at least as big as a block on the device. As there is no OS buffering, dd must get the entire block from the device in one shot, no taking little chunks at a time. The devices do not send bytes on request, they send blocks. So if dd is left with a default 512 byte "ibs", input block size, but the device is using a larger block size, like an amanda tape of 32k, dd has allocated a 512 byte piece of memory to hold the input data. But when dd requests the first block it unexectedly gets 32k of data and has "insufficient memory" (ENOMEM). The reverse is not really a problem. Suppose you said "ibs=128k". dd would simply read sufficient device blocks until the buffer was filled, four blocks in the above example. The problems arise when the device wants to feed dd more data per read than dd is prepared to receive. On output dd can make its own adjustments. If the obs is larger, it can move multiple input buffers to the output buffer before doing the write. If the reverse is true, input block size larger than output, it can copy part of the input block to the output buffer and do multiple outputs from a single input buffer. HTH jon -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrestore problem, headers ok but no data
On Fri, Jan 07, 2005 at 03:40:12PM -0500, Brian Cuttler wrote: > > samar 24# mt -f /dev/sdlt2 blksize > > Recommended tape I/O size: 131072 bytes (256 512-byte blocks) > Minimum block size: 4 byte(s) > Maximum block size: 16777212 bytes > Current block size: Variable > > I am using /dev/sdlt2 link to /dev/rmt/tps1d4nrns - no rewind, non-byte swap > device. I am no longer using the trailing 'v' device which would normally > provide variable length blocks. > Brian, as you were using the 'v', might the data on the tapes have been written with some > 32K block size? All of your dd's seem to fail after the header. How about trying a couple of mt fsf # to some file in the middle (on the no rewind device), then do dd's with bs=128k and/or bs=1024k and/or bs=16384k. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrestore problem, headers ok but no data
On Mon, Jan 10, 2005 at 11:48:42AM -0500, Brian Cuttler wrote: > > The following (just scripted, with commentary added) is a little long > and rather un-good. > ... > > Probing the drive I find a block size of 32768 again. > > samar 25# mt -f /dev/sdlt2 setblksz 131072 > > samar 26# /usr/local/sbin/amlabel -f samar SAMAR08 > labeling tape in slot 7 (/dev/sdlt2): > rewinding, reading label, reading label: Invalid argument > rewinding, writing label SAMAR08 > amlabel: writing label: Invalid argument > > Have we hit some sort of amanda buffer size limit ? > Something that didn't error during the write but did > during the read ? > This is not too surprising to me. A tape with a blocksize of 132k will accept a write of 32k and simply pad it to the required size. No errors. But on read back, it will not allow a read of 32k when the blocksize of the taped data is 132k. That could be your read error. In recent versions, amanda can work with blocksizes other than 32k. I forget if it is a configure option needed during the build or a parameter that can be set in amanda.conf. I've never used it. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Removing / replacing tape in cycle
On Mon, Jan 10, 2005 at 07:01:34AM +0100, [EMAIL PROTECTED] wrote: > Hi, > > after trying this and this morning and getting even more confused, I just > found out that the tape was faulty. > > I inserted an new one, labeled it tape21 (my tapecycle is from tape1 to > tape20), ran amflush and everything is working fine. But know the next > question. > How do I get the faulty one replaced by the new one ? > I can´t use a scratch file from the old one, as the header is not readable > anymore. The tape gone bad was tape11. > I'm guessing that you believe the data on tape11 to still be good but the tape-label is messed up. You could remove each of the files to hard disk, create a new tape11 with amlabel -f, and copy each of the hard disk copies of the tape files back to tape. A small variation is to copy the tape label from tape21 to disk, edit the "2" to a "1" and use that as the first file to copy to a blank tape. That could eliminate the amlabel step. Totally untested (need a posix shell like bash or ksh or zsh): Dir=/tmp/tape11 mkdir $Dir # start with empty dir cd $Dir Tape=" DDcmd="dd bs=32k if=$Tape" MTcmd="mt -f $Tape" # load tape21 $MTcmd rewind $MTcmd status# just like to check $DDcmd of=file001# should be 32k # edit file01 to change "21" to "11" # I forget if a date should be changed too # rewind and unload tape21 # load the damaged tape11 $MTcmd rewind $MTcmd fsf 1 $MTcmd status typeset -Z3 n=2 while $DDcmd of=file$n do let n=n+1 done # each of your files "fileXXX" should be a # multiple of 32k The last one should be # exactly 32k and should contain a tape end label # rewind and unload the damaged tape11 # load a blank replacement tape for f in file* do dd bs=32k if=$f of=$Tape done -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup priorities and initialization of backup
On Tue, Jan 11, 2005 at 11:11:06AM -0600, Gil Naveh wrote: > Hello, I am quit new with Amanda and I don't understand an important concept > regarding Amanda. > > I realize that Amanda decides what type of backup to perform (level 0 , > level 1 etc) according to its algorithm. > > However, how does it knows which backup level to perform when a user have a > tape drive with X different tapes that are manually changed? Is there a > file that tells Amanda which backups has been performed and if so what is > this file name? > > Finally, so far I have been changing the file disklist in order to check > different backup scenarios. The file tapelist was automatically updated: > > The new tapelist file looks like: > > 20041222 DailySet105 reuse > > 20041222 DailySet104 reuse etc. > > > > At this point, I want to run the backup - which means I have to initialize > all my experiments so far - any ideas on how to do so? > > Which files do I have to modify in addition to the disklist file. > Years ago I had a script to do just the thing I think you are asking, leave the configuration along but make amanda think it is the first ever amdump of everything. Here are the things I think I had to do in the automated script. Others may add or subtract with their input. - blow away the current dump info, index info, and logs In my case these were in subdirs named in amanda.conf. The default amanda.conf keeps the logs in the top level of the config directory. I always modify it to /logs. No sense cluttering up a perfectly good config directory. So for me this was just three rm -rf /* commands. You might have to do something different about your logs. Don't remove the curinfo, index, and ?log? directories themselves, just their contents. - In the tapelist file change all the dates to a zero. This indicates unused. If you want your tapes used in sequence, sort the list with with the first tape you want used at the bottom and the last tape at the top, - If you use a changer and your changer script mantains any state files, you may need to edit or remove them. My changer script, chg-mtx, did not so I did not have to do anything there. When you do this remember that amanda must do a level 0 dump of everything disklist entry it sees the first time. After doing this, every entry will be new. So everything will get a full backup which may not be desireable. I would comment out all the entries in your disklist, then go back and uncomment one or two entries from each client. Run amdump, maybe even during the day. Uncomment a few more disklist entries and let amdump run at its normal crontab time. Next day a few more etc. You get into a balanced situation much quicker this way without having a massively huge first amdump run. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrestore problem, headers ok but no data
On Tue, Jan 11, 2005 at 03:40:47PM -0500, Brian Cuttler wrote: > > Amanda user list, > > I've spoken to Harry at Condre systems and he confirmed what we > where seeing. The jukebox robot is a narrow bus so that are two > changes we need to make. > > 1) We want to move the jukebox/SDLT to be the last device on >the daisy chain. > > 2) We want to terminate the jukebox-robot, so we need to make >certain that the live port on the box hits the SDLT before >we reach (electically) the robot. > > Hopefully this will have a positive effect on both SDLT drives. Not a scsi expert here, but if you terminate the jukebox, wouldn't that terminate only the narrow portion of the bus leaving the extra lines of the wide bus unterminated. Might there be some kind of device/cable to go from wide -> narrow terminating the unused lines? Also, I think that if both types of devices exist on the same bus, the lower performance one determines the performance of the entire bus. That narrow jukebox may hurt your sdlt performance just by being on the bus. Some of my scsi adapters have multiple channels and buses. Perhaps you could run the sdlt's on one channel, wide, and the jukebox on another, narrow or wide. I added a cheap second controller from a dead system for a very similar purpose, external devices that were narrow when the main controller was driving only wide devices. Its two channels were used for different speed devices. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrecover not listing all files in backup.
On Tue, Jan 11, 2005 at 06:12:55PM -0700, Dwight Tovey wrote: > Hello all - > > I've recently started using Amanda for my backup strategy, and I'm having > a problem with amrecover. > Frank seems to have set you on the correct path, replacing your tar version. So just a hint about using amrecover. When browsing it presents the file tree as it existed on a particular date. So suppose you accidentally go looking for a file that was deleted, by mistake, two or three days ago. If your last level 0 was before that, the file is recoverable, but when you start amrecover, it shows the file tree as of the last amdump, i.e., with the file deleted. To see and extract the file you must reset the date in amrecover to a date when the file still existed. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrestore problem, headers ok but no data
On Wed, Jan 12, 2005 at 11:18:00AM -0500, Eric Siegerman wrote: > On Tuesday 11 January 2005 16:40, Jon LaBadie wrote: > >Also, I think that if both types of devices exist on the same bus, > >the lower performance one determines the performance of the entire > >bus. > > In theory, this is *not* the case. ... > > On Tue, Jan 11, 2005 at 10:48:17PM -0500, Gene Heskett wrote: > As to the whole chain being restricted to the speed of the slowest > device, I've heard that, but it doesn't, on the face of it, make a > lot of sense to me Gene, Eric, Thanks for the correction. I'm glad I remembered to say I wasn't a scsi expert. My comments provide the proof. > > > > To many folks forget that a a scsi bus is indeed > > an rf transmission line, subject to the usual rules about vswr. > > >From the context it's pretty clear what "vswr" means, but what > does it stand for? > > Wondered too, as I'm posting anyway, I'll guess: v??? standing wave reflections jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda config for two drives
On Thu, Jan 13, 2005 at 12:06:56PM -0500, Vicki Stanfield wrote: > > I would be amazed that amcheck did not complain :-) > > It hasn't yet been given the chance to. :-) Should be the first thing after any config changes. > > Okay, so I knew that I needed to use the chg-multi changer, but I was > unsure as to how. I have found some more information, but I don't know > where to get the script. I saw reference to it in an example directory > somewhere. I can't find it though. It is an included component of an amanda install. In my case it installed in /usr/local/libexec. YMMV jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: DLT tape streaming and amanda blocksize
On Fri, Jan 14, 2005 at 11:23:19AM -0800, Dan Rich wrote: > I have a DLT IV drive that I cannot get to stream with amanda. With a > little bit of experimentation, I have found that it will stream with tar if > I set the block size to 128KB. However, amanda only seems to support block > sizes of 32kb. The constant rewind/seek motion of the tape is rough on > both the tape and the hardware (not to mention my ears if I'm still up when > the backups kick off). > > I seem to remember once upon a time that someone posted a patch that would > allow amanda to run with larger block sizes. However, now that yahoo > deletes older postings I can't find it in the archives any more. > > Does anyone have a fix for this? Any ideas if changing this is even going > to work on FreeBSD? (my original testing was on my old Linux server) It is not a patch, its a feature. But it must be specified at configure time, then compiled. Check the amanda man page for "blocksize". -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backups failing on Solaris 9 with GNU tar
On Sat, Jan 15, 2005 at 05:26:08PM -0500, Kurt Raschke wrote: > On Jan 15, 2005, at 4:38 PM, Stefan G. Weichinger wrote: > > >First thought here: > >Look for the GNUtar-version. > >1.13.25 should work. > >Newer releases are known to cause problems. > > > >Apart from that we could need more infos: > > > >Did you do any changes/updates on this client lately, has it worked > >before? > > > Well, it's a brand-new install, so I suppose you could say it has never > worked before. > > As to the tar version: > > bash-2.05$ gtar --version > tar (GNU tar) 1.13 > > Does 1.13.25 come after 1.13? If so, then I will upgrade gtar and see > if that helps. That one is DEFINITELY defective as far as amanda usage. Nothing to do with your current problem however. It is an easy build if you have the C compiler already installed. As to the other problem: amanda maintains a list of error messages from various programs that it considers innocuous. There is a config macro IGNORE_TAR_ERRORS to increase the number of guntar messages considered innocuous. Perhaps the Solaris version was compiled without this macro while the others were compiled with. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backups failing on Solaris 9 with GNU tar
On Sat, Jan 15, 2005 at 11:52:47PM +0100, Stefan G. Weichinger wrote: > Hi, Kurt, > > on Samstag, 15. Jänner 2005 at 23:26 you wrote to amanda-users: > > KR> Well, it's a brand-new install, so I suppose you could say it has never > KR> worked before. > > KR> As to the tar version: > > KR> bash-2.05$ gtar --version > KR> tar (GNU tar) 1.13 > > KR> Does 1.13.25 come after 1.13? If so, then I will upgrade gtar and see > KR> if that helps. > > As it's a new client I would say that I would not start with updating > GNUtar. 1.13 should be fine, unless there are Solaris-specialities. > In the past I've had the index problem with the Sun Freeware (including the one automatically installed in /usr/sfw/bin) and Companion CD 1.13 versions. I see that Solaris 10 will be coming with gtar 1.14 in /usr/sfw/bin. Don't know if that has/will have problems or not. IIRC some 1.14's reverted to having problems. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amrecover segfault
On Mon, Jan 17, 2005 at 02:32:24PM +0200, Shai Ayal wrote: > Hi > > amercover segfaults on me when trying to access directories with long > names. I managed to find this email: > > http://www.mail-archive.com/amanda-users@amanda.org/msg8.html > > and using addx and cdx works for me too. > > However this is quite a serious bug. Is anyone aware of this? Is it being > addressed ? > > Shai > > Gavin Henry wrote: > >Dear all, Why did you include Gavin's posting which is totally unrelated? Are you using tar or dump? I ask because with the discussion of the recent releases of gnutar I've been looking at the ChangeLog for the tar project. There are quite a few notes in there dealing with problems with long path/file names. Perhaps the problem is tar's and not directly amanda's. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: star instead of gnutar
On Mon, Jan 17, 2005 at 03:38:40PM +0100, Claus Rosenberger wrote: > hi, > > i need to use star instead of gnutar because i want to backup my acl's > too. how could i manage this? > There are several things called "star". I'm guessing that you mean the program written by Jörg Schilling. Several people have considered and/or attempted to use this archiver. I don't recall anyone reporting back on their success. The basic approach is to use a "wrapper" for the real gnutar program. The wrapper is a shell script that takes the gnutar arguments and options that amanda uses, converts them to the options and arguments star wants, and then calls star rather than gnutar. When you compile amanda you configure it with the option "--with-gnutar=" (check the actual option, my memory may falter) set to point to a copy of a normally valid gnutar (version 1.13.25 recommended). Make it a copy as it will be replaced and you don't want anything but amanda to use this "tar". I happen to use /usr/local/libexec/amgtar. In the following I'll call the copy "amgtar". Get amanda running with the amgtar copy of a real gnutar. Replace amgtar with a shell script that does nothing but call the real gnutar. I do this by renaming amgtar to amgtar.real and then naming the shell script amgtar. A first shell script can be as simple as exec "$@" Sometimes for analysis I put in a logging line before the exec. LogFile= date >> $LogFile echo "$@" >> $LogFile Should you want the arguments spread out and numbered you might replace the echo with: for arg do echo "$arg" done | cat -n >> $LogFile Make sure you do some recovers and restores as well before switching to star. Armed with the results of the above analysis and/or the argument lists from the amanda debug files in /tmp/amanda/runtar... determine what needs converting, deleting, adding, ... to make star do the samething that gnutar does. Don't worry about your acl's yet. Write a script to do the converstion that in the end "exec's star" rather than amgtar.real. Again, dumps and recovers. Then you can add your star-specific options to save and restore acl's and test again. The final item is to report back your success to the list. I'd like to use JS's star as it has the ability on Solaris, to not change the atime or the ctime during backup. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: star instead of gnutar
On Mon, Jan 17, 2005 at 12:44:01PM -0500, Gene Heskett wrote: > On Monday 17 January 2005 11:09, Jon LaBadie wrote: > > ... > > > >The final item is to report back your success to the list. I'd like > > to use JS's star as it has the ability on Solaris, to not change > > the atime or the ctime during backup. > > atime at least seems to be covered in the new tar-1.15-1 Jon, here is the > --help screen: > > [EMAIL PROTECTED] root]# tar --help > Usage: tar [OPTION...] [FILE]... > GNU `tar' saves many files together into a single tape or disk archive, and > can > restore individual files from the archive. > ... > > Handling of file attributes: > > --atime-preserve don't change access times on dumped files That option has been there for a long time. However, if you use it then the ctime gets changed. That is a normal OS feature of unix-like systems. Dump programs, as they do not work through the file system, don't affect either atime nor ctime. It would be nice if backups with tar could to that also. Shilly found an obscure ioctl that was added to Solaris to support some Sun product. It allows the resetting of atime without affecting ctime. It is not guarenteed to remain in Solaris, but has been there for over 10 years. That is the Solaris-specific capability of star to which I referred. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Data loss if configuration of amanda.conf is not right.
On Tue, Jan 18, 2005 at 09:39:56AM -0500, Vicki Stanfield wrote: > I have set up an alternate amanda.conf to run with two tape drives > instead of the one that I currently have. I am wondering whether there > is some way to rerun an incremental backup if I run it and something in > my new configuration is out of whack. Since there is stateful data > involved which presumably would not be available after the backup ran, > is there a way to get it back so that problem could be corrected and the > backup rerun? In trying to think of potential problems, I don't really come up with any which would cause amanda to think it has had a successful run when it failed unless some component was "lying" about its success; e.g. a tape system saying it was accepting the data when the tape was sitting there broken and spinning. So I don't think the stateful data would reflect a false situation. Maybe you or someone else would think of other scenarios than I've come up with. Although amdump, which starts a backup, is normally run from a cron job, there is no reason why you can't start it manually if the cron job fails. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Level clarification.
On Tue, Jan 18, 2005 at 05:44:07PM -0600, Frank Smith wrote: > --On Wednesday, January 19, 2005 00:02:36 +0100 "Erik P. Olsen" <[EMAIL > PROTECTED]> wrote: > > > Hi, a newbie question. > > > > I think I have now managed to customise amanda and I am pretty confident > > that I can now start taking back-ups of my system. > > > > One term that I thought I understood is the level set. Apparently level > > 0 is a full back-up and level 1 is incremental, but what then is level 2 > > (and up)? If it's in the docs, please tell me where I can find it. > > > > Level Description > - --- > 0 a full backup (i.e., the entire filesystem or directory). > 1 just the files that have changed since the last level 0 > 2 just the files that have changed since the last level 1 > 3 just the files that have changed since the last level 2 > > and so on through level 9, so levels 1-9 are all incrementals. > Just one trivial thing, if a level is missing, say someone did something silly like level 0, 1, 3, the level 3 is anything changed since the last "higher" (numerically lower) level. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Data loss if configuration of amanda.conf is not right.
On Wed, Jan 19, 2005 at 09:57:22AM -0500, Vicki Stanfield wrote: > > Thanks. I was unsure of whether this was a potential problem or not. As > long as Amanda tells me that there was a problem and dumps to the spool, > I am happy. Make sure your "holding disk" space is sufficiently large and that your "reserve" parameter is set to some value smaller than the default 100%. The reserve aspect is to reserve the holding disk for incrementals only when there are tape drive problems. If your holding disk is sufficiently large, you can also hold some full backups so the reserve should be decreased. I have enough holding disk for several days of normal dumps (> 1wk) and use a reserve of 20%. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Data loss if configuration of amanda.conf is not right.
On Wed, Jan 19, 2005 at 04:13:36PM -0500, Vicki Stanfield wrote: > On Wed, 2005-01-19 at 10:38 -0500, Jon LaBadie wrote: > > On Wed, Jan 19, 2005 at 09:57:22AM -0500, Vicki Stanfield wrote: > > > > > > Thanks. I was unsure of whether this was a potential problem or not. As > > > long as Amanda tells me that there was a problem and dumps to the spool, > > > I am happy. > > > > Make sure your "holding disk" space is sufficiently large and > > that your "reserve" parameter is set to some value smaller > > than the default 100%. The reserve aspect is to reserve the > > holding disk for incrementals only when there are tape drive > > problems. If your holding disk is sufficiently large, you > > can also hold some full backups so the reserve should be > > decreased. I have enough holding disk for several days > > of normal dumps (> 1wk) and use a reserve of 20%. > > > > jl > > I'm not sure that I correctly understand the reserve parameter. I have > heard that it is used to get amanda to degrade full dumps to merely > incremental dumps if the holding disk gets full. I assume the percentage > refers to how full. So how does degrading to incrementals help if the > holding disk is 100% full? What am I misunderstanding here? I can > understand saying that if the holding disk is 75% full, switch to > incrementals though. > Definite misunderstanding. I'll try to relate it as I understand. Amanda can back up each DLE directly to tape with no holding disk (HD) in use. In that case it can only do a single DLE at a time. With a HD it can be taping one from the HD and collecting another onto the HD. In fact, it can be collecting many onto the HD at the same time. What happens if the tape drive is broken, contains the wrong tape, or maybe no tape at all. Then all those DLE's collect on the HD. I have a 6 tape changer that I swap tapes from over the weekend. Occassionally I forget and leave for a 5 day business trip. Now the HD collects lots of DLE's while I'm gone. Level 0 dumps tend to be quite large and can quickly eat up the HD. Then what, no dumps at all? None to tape and none to HD either? Then I have no protection. To reduce the likelyhood of this happening the reserve parameter says "reserve this percent of the HD for just such a situation". When amanda has no tape to write to and the percent of HD left drops to this level, amanda goes into "degraded" mode and ONLY does incrementals, no level 0's. This keeps the backups going to the HD a lot longer than a mixture of level 0's and incrementals. The default reserve is 100%, i.e. no level 0's get the buffering of the HD. They always go straight to tape. That is good if your HD is small and even one or two level 0's would fill it. But most of us have pretty substantial HD, mine is about 100GB with a daily taping of about 8GB. So I can use the buffering of the HD for level 0's and even store a good number of complete backups on the HD before I hit my reserve setting of 20%. That is good for my week long trips without changing the tapes. Then I still have about 20GB for incremental only, degraded mode if I'm away a really long time. (Or like now have tape drive problems) jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Changer file for Compaq MSL5000 series
On Thu, Jan 20, 2005 at 01:22:56PM -0500, Scheuerman, James wrote: > Does anyone have a changer file for a Compaq MSL5000 series library? > I'm trying to configure Amanda 2.4.4 on Compaq Tru64 5.1b. Changer files depend on the changer script you decide to use. chg-mtx? chg-zd-mtx? chg-scsi? chg-??? Does an mtx command exist and work on your system with your changer? If so, the scripts describe pretty well what must go into the changer files. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Changer file for Compaq MSL5000 series
On Thu, Jan 20, 2005 at 10:53:26AM -0800, DK Smith wrote: > > It would be really cool if a single changer glue module was > re-written in an OOP fashion, reusing the same logic code > for every situation, but overriding the driver interface methods > for specific driver interfaces. > Ahh, you mean a "changer-api" to go along with the infamous DUMPER-API? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Fedora Core 3 - which version of tar??
On Thu, Jan 20, 2005 at 10:22:16PM +0100, Stefan G. Weichinger wrote: > > This will be added to the docs: > > - configure and make as $AMANDAUSER, make install as root > - run ldconfig afterwards. The latter in an OS-specific manner I trust. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Dump level, behaviour change
On Fri, Jan 21, 2005 at 11:30:02AM -0500, Brian Cuttler wrote: > Amanda 2.4.2p2 > Solaris 8 > SDLT 220 > > We recently make a couple of changes to our amanda conf (well, one > of our amanda server configs). > > 1) changed the config name > 2) changed from SW to HW compression, increased tape length ... > > We are seeing that amanda used to perform level 0 and 1 dumps (DUMP > type dumps) and we are now seeing that we are getting to level 2 > dumps. We where currious as to why. > ... > > It looks like there are a couple of things in play here. > > 1) From amanda.conf > bumpsize 50 MB # minimum savings (threshold) to bump level 1 -> 2 > bumpdays 1 # minimum days at each level > bumpmult 2 # threshold = bumpsize * (level-1)**bumpmult > Just guessing. Much of amanda's planing is based on how much data is "taped", not how much is backed up. I'm thinking of the balancing act it performs. Since you switched from SW to HW compression, the amount of data that is being "taped" is larger. Thus you may be hitting our bump figures if they are calculated on the taped data and not the original data. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Fedora Core 3 - which version of tar??
On Fri, Jan 21, 2005 at 06:15:28PM -0600, Frank Smith wrote: > > > --On Friday, January 21, 2005 18:18:52 -0500 Eric Siegerman <[EMAIL > PROTECTED]> wrote: > > > On Thu, Jan 20, 2005 at 10:22:16PM +0100, Stefan G. Weichinger wrote: > >> - configure and make as $AMANDAUSER > > > > I don't believe this is necessary. One should avoid building > > Amanda as root, but that's not because it'll cause problems for > > Amanda; it's for the same reason one should avoid building > > *anything* as root. > > Sorry for hijacking a thread, but playing devil's advocate here, > what difference does it really make whether you build as root > or not if you run 'make install' as root? How many people > actually go through each line of the makefile, or run make -n > first and examine all that, plus look through the code itself? >In Amanda's case, for example, if the source for runtar > contained code to 'mailx [EMAIL PROTECTED] < /etc/shadow' or > 'rm -fR /' would it really make any difference what user it > was compiled under? Would they notice even if it wasn't buried > in the source, but was just part of the 'install' target in > the makefile (or an included subdirectory makefile)? >For user programs, compiling and installing as a user > can limit the damage that can be done, but if any part of > the build, install, or execution ever has root privileges > then you really aren't increasing your security by building > as a normal user and installing as root. >All that said, I generally do build as a normal user, as > it can expose permission problems on libraries, paths, etc. > at build time instead of later when users are trying to run > the program. You have said it yourself, as ordinary user you limit the possible problems. Have you never looked at a file with an editor and inadvertently modified it? Don't you ever do little changes to the code or makefiles? How about running your builds from a script as Gene H. does. Or running my configure with a script that modifies the makefiles to allow me a target of *.i output files (post-preprocessor). I don't want to run these things as root. There doesn't have to be anything malicious to be dangerous or unwanted. Just a simple error that might cause less damage run by amanda than run by root. Or even by user jon. Suppose I had a simple "cd" in one of these scripts; like cd $AmandaBuildDir But I forgot to properly set AmandaBuildDir, or I misspelled it in the script. Now it is a "cd " meaning a cd to the home directory of the person running it. I would rather it went to amanda's, not jon's and certainly not root's home dir before continuing the script. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Problem with amrecover
On Sat, Jan 22, 2005 at 02:45:40PM +0100, Sören Edzen wrote: > Hi! > > I have problem using amrecover. I get the following output: > > amidxtaped: time 0.002: Ready to execv amrestore with: > path = /usr/local/sbin/amrestore > argv[0] = "amrestore" > argv[1] = "-p" > argv[2] = "-h" > argv[3] = "/dev/nst0" > argv[4] = "^zampo.edzen.net$" > argv[5] = "^/usr/local/cvsroot$" > argv[6] = "20050115" As the exact amrestore command is shown, can you reproduce the error by running it manually? If so, it may be time to try and pull off manually some of the tape files and see if they are recorded correctly. This will involve using mt and dd commands. Then tar or dump on the result. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Permission problems with amlabel.
On Sat, Jan 22, 2005 at 09:58:42PM +0100, Erik P. Olsen wrote: > I have labeled the tapes before during my tests without problems. Now, > when preparing amanda for production I relabeled the tapes (after having > duly removed them from the repository by amrmtape) and changed the > directory into which the tapelist would go. Then amanda wouldn't label > the tapes. The command used was: > > [EMAIL PROTECTED] full]# su amanda -c "amlabel -f full full01" > rewinding, reading label full01 > rewinding, writing label full01, checking labelamlabel: couldn't write > tapelist: Permission denied > > Well, I thought it was permissions for the file tapelist and changed > that to full permission, but still got the same error. > > I then saw syslog had the following message: > > Jan 22 21:13:39 epo su(pam_unix)[9298]: session opened for user amanda > by erik(uid=0) > Jan 22 21:13:39 epo su[9298]: Warning! Could not relabel /dev/pts/1 > with user_u:object_r:devpts_t, not relabeling.Operation not permitted > Jan 22 21:13:39 epo su(pam_unix)[9298]: session closed for user amanda > > I have no idea whatsoever what this is all about. I would be grateful if > someone would explain to me what the heck it is and how I can satisfy > amlabel? > Here is what I wrote to the list in Oct about someone who seemed to have a similar problem: == I forget the details now, and whether it was fixed, but someone == reported that the tapelist file not only had to exist, but either == had to be zero length (not even a blank line) or had to have a == blank line. Sorry I forget which. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Permission problems with amlabel.
On Sat, Jan 22, 2005 at 11:05:54PM +0100, Erik P. Olsen wrote: > On Sat, 2005-01-22 at 17:00 -0500, Jon LaBadie wrote: > > On Sat, Jan 22, 2005 at 09:58:42PM +0100, Erik P. Olsen wrote: > > > > > > I have no idea whatsoever what this is all about. I would be grateful if > > > someone would explain to me what the heck it is and how I can satisfy > > > amlabel? > > > > > > > Here is what I wrote to the list in Oct about > > someone who seemed to have a similar problem: > > > > == I forget the details now, and whether it was fixed, but someone > > == reported that the tapelist file not only had to exist, but either > > == had to be zero length (not even a blank line) or had to have a > > == blank line. Sorry I forget which. > > > Oh, tapelist exists and has a length of zero just like when I > successfully ran amlabel during the tests. As I noted, it may have been that it needed a blank line. It was something really funky. Just in case, convert it from a length of zero to a length of 1 (a blank line), as follows: echo "" > tapelist To go the other (blank line to zero) do a: > tapelist# no command -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Reiserfs?
On Wed, Jan 26, 2005 at 11:24:57AM +0100, Paul Bijnens wrote: > Nick Danger wrote: > > > >Thanks for the help with the previous problem, I have it backing up all > >the non reiserfs partitions just fine now. > > > >How do I get it to back up reiserfs partitions? I tried root-tar, and > >comp-root-tar in disklist and neither worked. > > > >FAILURE AND STRANGE DUMP SUMMARY: > > tesla.hack / lev 0 FAILED [/bin/tar returned 2] > > tesla.hack / lev 0 FAILED [dump to tape failed] > > I believe this has nothing (or very little) to do with tar. > Tar is signaling some error (exit code 2 usualy means some important > error, resulting in the output not being what you expect from tar). > > You'll have to find out why tar gives gives this error. It has been a couple of years since I looked at tar, but IIRC two things that give a return code of 2 are "file changed as we backed it up" "file disappeared after we made the list of what to backup" These conditions are common on active file systems. Both cause the error message "error exit delayed from previous errors" which is a return code of 2. The "error exit delayed" situation can be set to be considered "normal" in sendbackup-gnutar.c or it and a few others ignored by defining "IGNORE_TAR_ERRORS" in amanda.h. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Reiserfs?
On Wed, Jan 26, 2005 at 03:51:33PM +0100, Paul Bijnens wrote: > Jon LaBadie wrote: > > > >It has been a couple of years since I looked at tar, > >but IIRC two things that give a return code of 2 are > > "file changed as we backed it up" > > "file disappeared after we made the list of what to backup" > > > >These conditions are common on active file systems. > >Both cause the error message > > "error exit delayed from previous errors" > > > >which is a return code of 2. > > Consulting sources of gnutar 1.13.25: > "file changed..." would set exit(2) but is neutralized > when given the --ignore-failed-read option to tar. > I believe you are correct on a "failed read", i.e. permissions type changes, but I don't think "file changed ..."is affected by the --ignore-failed-read option. I think it still prints the message, and exits 2 with the "error exit delayed" message. > Sources of gnutar 1.15.1: > "file changed" is never triggering an exit-code different > from zero (but warning is printed nevertheless). > "file shrank ..." however still needs --ignore-failed-read flag > to be neutralized. Looks like the RE's ignored need to be revisited. > > The "file disappeared..." is diagnosed as > "Warning: Cannot stat: No such file or directory", and that > one neither sets a exitcode different from zero (in 1.13.25 > and 1.15.1). > With the --ignored-failed-read I'm sure you are correct. > > >The "error exit delayed" situation can be set to be considered > >"normal" in sendbackup-gnutar.c or it and a few others ignored > >by defining "IGNORE_TAR_ERRORS" in amanda.h. > > > > Amanda does pass the --ignore-failed-read option, and thus you > normally should not need to compile with IGNORE_TAR_ERRORS defined. > The last one classifies the printed warnings as "normal" instead > of "strange" and suppresses the STRANGE section in the amanda report. > And maybe it would ignore other more serious errors too? These are the 3 messages added by the IGNORE_TAR_ERRORS macro. "File .* shrunk by [0-9][0-9]* bytes, padding with zeros" "Cannot add file .*: No such file or directory" "Error exit delayed from previous errors" Seem to be directly related to the current discussion. > > Conclusion: the /bin/tar used is not version 1.13.25 or 1.15.1. Good possibility. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda change ctime
On Wed, Jan 26, 2005 at 06:10:19PM -0500, Joshua Baker-LePain wrote: > On Wed, 26 Jan 2005 at 2:40pm, Nina Pham wrote > > >The proplem is the ctime some how got changed when backing > > up, so the files endup archived even though it's not been touched. Any idea? > > Amanda uses dump and/or tar to actually get the bits off the disk. > Neither of those should touch ctime. > If the "ENABLE_GNUTAR_ATIME_PRESERVE" macro is defined then ctime will be modified by gnutar's restoration of atime. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda tar and star again
On Thu, Jan 27, 2005 at 02:54:34PM +0100, Claus Rosenberger wrote: > hi, > > it would be nice if amanda will be packaged with a compiled in wrapper tar > script instead of running tar from /usr/bin. perhaps it's possible to run > /usr/lib/amanda/tar-wrapper wich is calling /usr/bin/tar again. that > script could be changed then to call /usr/bin/star. > It would be nice if amanda did not use the gzip program but was packaged with a wrapper so I could replace it with zip or bzip2. It would be nice if amanda was packaged with a wrapper that called dump so I could call a different dump. Or maybe I could have it do a snapshot first. Yeah, nice, but those really are local customizations best done locally - IMHO. Related to your original post on the subject, it was suggested that you see if star supported all the arguments and options that amanda expected to see supported by gnutar and if it was compatible with the formats and error messages that gnutar generates. Were you able to do that? If star is not usable, a wrapper is of little value. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda tar and star again
On Fri, Jan 28, 2005 at 08:22:49AM +0100, Claus Rosenberger wrote: > > Related to your original post on the subject, it was suggested that > > you see if star supported all the arguments and options that amanda > > expected to see supported by gnutar and if it was compatible with > > the formats and error messages that gnutar generates. Were you > > able to do that? If star is not usable, a wrapper is of little value. > > as described star should be able to do all these things but i will check > it again. is there a doc-file where all options of tar which are used are > described or listed or do i have to look into the source code? No document that I know of. In my first email on this thread I mentioned the debug files you can use to see the arguments used and a way to have your wrapper script, during testing, record the arguments. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: archiving tapes?!
On Fri, Jan 28, 2005 at 11:22:06AM -0700, Mark Costlow wrote: > > I have a similar situation here. I recently moved to doing our daily dumps > on vtapes (I previously used a 48-tape DLT jukebox). Now periodically > (monthly > maybe -- haven't set the schedule yet) I want to take a set of off-site dumps > out of the building. > > In the old tape days, I used to force a level-0 dump when I wanted to do > that. This was a pain and never very satisfying. So I wrote a perl script > that will extract the most recent full dump for every disk partition out > of the vtapes. It copies them to removable disks which I take offsite, > but you could copy them to tapes instead if you wanted. > > The script is fairly generic (I tried not to let it depend on my local > environment too much, but there may be some gotchas). I was also lazy > about a couple of assumptions (like the fact that my dumps all used gtar). > If anyone wants it, let me know. What a neat idea, and one that works best with vtape as the daily backup. Would you consider writing up your experiences and techniques? I'm guessing that for recovery from the archived dumps you would not use amanda's indexing features. Have you had to do any recoveries from the archived dumps rather than the vtapes? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda server crashed!!!
On Mon, Jan 31, 2005 at 06:30:20AM -0500, Kevin Alford wrote: > My Amanda backup server crashed, and I had to rebuild it. I was running > Amanda 2.4.2p2 on AIX 5.1 ML6. > I desperately need to restore the data from my Amanda tapes, but I don't > have the Amanda index files. > Isn't there a way I can use native Unix to read and restore my tapes > written by Amanda? > Your assistance in this matter is greatly appreciated. If you have rebuilt your amanda installation you could use amrestore which does not depend on the index files. Alternatively the manual approach you asked about involves a few steps outlined here (remember to use the no rewind tape device): 1) Using mt's fsf option (or asf) position the tape at the beginning of a tape file (each DLE tapes to a separate tape file) 2) Using dd read the first 32KB block of the file (amanda header). Something like "dd if= bs=32k count=1". It will tell you what DLE/date/level the tape file contains and whether it used dump or tar. 3) Assuming you want these data you will then use dd to extract from tape and pipe into gzip if compressed and tar or dump as appropriate. If your tapedrive leaves you at the 32KB position after set two you can do "dd if= bs=32k | ?gzip -d? | tar or dump" I've sometimes had this fail where I attributed the second dd cmd not picking up were the first left off; in that case you need to use mt to reposition the tape to the beginning of the tape file and tell dd to "skip" the first block "dd if=... bs=32k skip=1 | ..." Remember the data is on tape relative to the root of the DLE, not the root of the system. So if the DLE were /var/spool and you want to recover the files to their original location you must issue the 3rd step commands from /var/spool. I prefer to recover to an empty directory and the move or copy to the actual location. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda server crashed!!!
On Mon, Jan 31, 2005 at 11:40:58AM -0500, Kevin Alford wrote: > I am getting the following error. I have found my data. > > [EMAIL PROTECTED] /class_saa/restore]# dd if=/dev/rmt1.1 bs=32k skip=1 | > /usr/bin/gzip -dc | usr/sbin/restore -f... - ^^ > -bash: usr/sbin/restore: A file or directory in the path name does not ^^ A leading slash would be nice. :) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Removing unwanted dumps.
On Mon, Jan 31, 2005 at 12:03:31PM -0500, Joe Konecny wrote: > If you have dumps that are on the holding disk but not > needed to be flushed to tape, what should I do? Manually > delete them from the holding disk? Maybe I'm not looking > in the right place but I don't see the answer to this question > in the docs. Don't recall hearing the question asked before. I can't see anything that would cause problems if you delete them. Others may. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda server crashed!!!
On Mon, Jan 31, 2005 at 12:11:06PM -0500, Kevin Alford wrote:
> This is the error I receive.
>
> [EMAIL PROTECTED] /class_saa/restore]# mt -f /dev/rmt1.1 rewind
> [EMAIL PROTECTED] /class_saa/restore]# mt -f /dev/rmt1.1 fsf 1
> [EMAIL PROTECTED] /class_saa/restore]# dd if=/dev/rmt1.1 bs=32k skip=1 |
> /usr/bin/gzip -dc | /usr/sbin/restore -f... -
^^^
Is this literally what you typed?
It looks like it from the messages below.
> Please mount volume 1 on
> Press the Enter key to continue.
> restore: 0511-158 Cannot open ...: A file or directory in the path name
> does not exist.
> Please mount volume 1 on
> Press the Enter key to continue.
> restore: 0511-158 Cannot open ...: A file or directory in the path name
> does not exist.
> Please mount volume 1 on
> Press the Enter key to continue.
> restore: 0511-158 Cannot open ...: A file or directory in the path name
> does not exist.
Generally an elipsis ("...") means
"and anything else you might need here".
It is not meant to be typed literally.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Removing unwanted dumps.
On Mon, Jan 31, 2005 at 12:45:14PM -0500, Eric Siegerman wrote: > On Mon, Jan 31, 2005 at 12:17:18PM -0500, Jon LaBadie wrote: > > I can't see anything that would cause problems if you delete > > [unneeded dumps that are still in holding disk]. > > Others may. > > The curinfo database will still think the dumps exist, won't it? > That might confuse amrecover until the dumps in question have > been superceded by newer ones (i.e. at same or lower dump level). > Then again, maybe these particular dumps have already been > superceded -- maybe that's why they're unneeded in the first > place. I too assumed that "unneeded" meant there are likely more recent ones available that the ones targeted for deletion. The curinfo db has very little info that would make a difference. As to particular dumps, things like date, size, level are only kept for the most recent at each level. The other info is things like the compression rate and dump rate of the last three runs. And those would still be valid for planning. > > How long would it take for this discrepancy to clear itself up? > (I know the answer for dumps that *have* been taped -- until the > tape gets overwritten one tapecycle hence. How does it work for > untaped dumps like these?) Assuming at least the most recent level 0 and subsequent incrementals, I don't see any curinfo discrepancy. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Removing unwanted dumps.
On Mon, Jan 31, 2005 at 03:54:57PM -0500, Gene Heskett wrote: > On Monday 31 January 2005 13:38, Eric Siegerman wrote: > >On Mon, Jan 31, 2005 at 01:11:48PM -0500, Jon LaBadie wrote: > >> The curinfo db has very little info that would make a difference. > >> As to particular dumps, things like date, size, level are only > >> kept for the most recent at each level. > > > >I misspoke. I should have said, "curinfo + indexes + any other > >info that Amanda keeps around". What you say applies to curinfo > >itself, but what about the rest? If nothing else, those index > >files will take up some space; will they ever be deleted, or will > >they hang around forever? > > > No, they'll be deleted as they age out of revelance by amanda. > As I recall amanda knows a dump is still on the holding disk and can do amrecovers from there. Might have been an added feature over the last year or two. When a tape is removed with amrmtape, the indexes for that tape are removed. I wonder when, or even if, amanda ever realizes something that it believes is on the holding disk is no longer there. And can the cleanup be triggered with something analogous to amrmtape. Does amcleanup do any of these things? Guess I could look it up (RTM jon) but I'm too lazy and not concerned. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: barcodes in Exabyte 230D
On Tue, Feb 01, 2005 at 03:57:54PM +0100, Wim Zwitser wrote: > Hi, > > I try to install an amanda server on a Linux Enterprise pc with > the standard redhat rpms's. The pc is connected to an Exabyte 230D > DLT jukebox with 1 drive and 31 slots. > > As far as I can see the basics are running fine, except one thing: > I can not read the barcodes on tapes in the jukebox with one of > the am*-tools. > > The only tool which gives the barcodes is mtx. After "mtx inquiry" > the "mtx status" command lists barcodes. > > But "amtape show" starts to load all tapes one by one in > order to read an amanda label. > I like to see it reading the barcodes! > > Does anyone have an idea? I don't use barcodes so this is just guessing. What changer script are you using? I think only two support barcodes. chg-scsi and chg-zd-mtx If using one of them, have you followed the steps necessary to configure the script to use barcodes? IIRC chg-zd-mtx requires uncommenting or setting of some parameters, some extra files, and/or an initial run to set up the barcode file. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda server and client
On Tue, Feb 01, 2005 at 02:48:37PM -0700, Jason Davis wrote: > On Tue, 2005-02-01 at 14:39, Nina Pham wrote: > > I'd like to setup amanda server and client on some linux machines. are > > they separate packages? > > > > well , first download the package(s) from the amanda web page ... then > count how many packages you downloaded , if there is more than one ... > chances are they "are in separate packages"; > Have a bad day Jason? Nina, Someone had written a webmin module for amanda, but it was a work-in-progress when I last saw it. Certainly none is part of the amanda distribution. Many amanda users feel that a gui interface is not a benefit for a "set it up and forget it" type of application like amanda. As to packages, sounds like you are thinking rpm type packages. I don't how any particular linux distribution packages them. If you build and install from source (recommended here), the client pieces can be built and installed separately. Remember that the server is also a client if it backs up itself. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Can Amanda use an Iomega REV drive as a tape?
On Tue, Feb 01, 2005 at 03:42:34PM -0800, Tom Simons wrote: > Can/should Amanda use an Iomeg REV drive as an output tape? > > We've got 2 servers both running RedHat AS 3.0, with 35gb & 70gb hard > drives on each, and we're intersted in running Amanda on one of the > servers to back up both (& more servers to follow). The "backup" > server has a 35gb Iomega REV drive, which RedHat sees as /cdrom2. > > Has anyone used a REV drive with Amanda? Haven't seen any mentions. I noted earlier an interesting item on their website, a 10 drive changer for the REV drives. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
AIX guru's please
Kevin Alford started a thread about losing his amanda server crashing, needing to rebuild it, not having indexes and needing to do a restore. The system details pre-crash were Amanda 2.4.2p2 on AIX 5.1 ML6 When Kevin wasn't getting very far I started an email dialog with him, but we have hit a stumbling point and could use some input from people who know AIX better than I. Working with the first dump file on the tape, it has the following information in the header: AMANDA: FILE 20041208 lightning /saa_dev lev 1 comp .gz program /usr/sbin/backup To restore, position tape at start of file and run: dd if= bs=32k skip=1 | /usr/bin/gzip -dc | usr/sbin/restore -f... - "lightning" is the amanda server, so this is a local backup. I see the "dump" program is IBM ?proprietary? program called 'backup' !! Is that program dump compatible? From my long ago AIX experience I recall that 'restore' is the appropriate recovery program for 'backup'. But what arguments should be used, i.e. what might be needed for "..."? I just love AIX! Kevin extracted the dump file, minus the amanda header. He then tried to use gzip -dc on it to create an uncompressed copy. The copy was 3.5x larger, so obviously it had been gzip'ped. However, when Kevin ran the "file" command on both the original and on the uncompressed files, AIX's file command reported both as being: data or International text Nothing about either being a gzip or a backup file. Kevin sent me the leading part of the gunzipped file and on my Solaris system, the file command reports it is a "ufsdump archive". Does AIX even have the ability to work with UFS type file systems? If so, are ufsdump/ufsrestore part of an add-on package? Kevin reports his rebuilt system has neither program. May not matter. Based on the amanda header, I suspect that backup/restore are the appropriate pair of programs, not ufsdump/ufsrestore. Do other AIX amanda installations use backup/restore? If so, what are appropriate options for a restore from an backup format file already on disk. Maybe both for TOC and for extraction. We can worry about from tape later. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: no different dumpsize between level 0 and 1
On Thu, Feb 03, 2005 at 01:17:10PM +0100, Rainer Hackel wrote: > Here an abstract of tho mail reports amanda send after a backup. The first > time it makes dumplevel 0 of backupdisk. The second time it makes dumplevel > 1 of the same disk. Why differs the size that less? Looks more like if both > were complete backups. Someone can help? > > -backupdisk 0 20166900 20166900 -- 104:513205.5 39:518434.9 > > -backupdisk 1 20223440 20223440 -- 93:013623.6 38:498683.2 First question has to be, have most of the things on 'backupdisk' changed since the level 0? Might something be causing the ctime of each file to be altered so it looks like it has changed? What is on 'backupdisk'? Is it, as the name suggests, backups? If so, are you backing up your backups? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: mounting/identifying a new tape drive
On Thu, Feb 03, 2005 at 11:53:36AM -0500, Gil Naveh wrote: > Hello, > > I have a Solaris 9 box and we bought a new tape drive model Certance LTO-2. > Currently, I am trying to identify the tape drive using amtapetype command > but it does not work. > > > Let me go a step back and describe what I did so far. > I connected the new tape drive to our Solaris 9 server and I restarted > Solaris. > Next in order to see that the server identified the new hardware I typed: > #iostat -En > and received the following output regarding the tape drive: > # rmt/0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 > # Vendor: CERTANCE Product: ULTRIUM 2Revision: 1703 Serial No: > 5150-400 > > Finally, I want to run the amtapetype to identify the tape type definition > but was enable to do so! JHL GENERAL RULE #1 Never, Never touch amanda commands until your hardware works with system commands! Make sure you can use things like mt, dd, tar, ufsdump, cat, mtx, ... with your drive/changer before adding the added complexity of the amanda layer. > When I typed: > #amtapetype: /dev/rmt > I received the following error message: > #amtapetype: /dev/rmt: rewinding tape: Inappropriate ioctl for device On Solaris, /dev/rmt is a directory, so clearly tape commands are inappropriate. > > When I typed: > # amtapetype: /dev/rmt/0 > OR > #amtapetype -f /dev/rst13 > OR > #amtapetype -f /dev/nrst13 > The output is : > # amtapetype: could not open /dev/nst13: No such file or directory And I presume the /dev/rmt directory is empty? > > Any idea why I am getting those error messages and whether I miss a step in > defining a new tape drive, or any command that describe where the new tape > drive is mounted? > For a tape drive to work on Solaris there must be an appropriate entry in the st (Scsi Tape) driver configuration file /kernel/drv/st.conf. On my Solaris 9 system (x86, but it should not matter) there are no entries for 'certance' or for 'ultrium 2' drives. Sometimes I've installed drives that had instructions to edit the file to add driver configuration information for the new drive. Your's didn't? Was it listed by the vendor as Solaris supported? I'm assuming that your new drive is the only drive on the system. If that is a bad assumption, don't do the following. Remove all the symbolic link entries in /dev/rmt and we will recreate them if the system can configure the driver for your tapedrive. rm -f /dev/rmt/* devfsadm -c tape If the directory /dev/rmt is still empty, the system doesn't know how to work with your tape drive and there is nothing amanda can do about it. You have to get it working with the system first. If the above failed, you may also get a "driver failed to attach" if you run the command "devfsadm -i st". I.e., the st driver could not find a device it could work with. > This is propably a silly question but I am stuck so any help is welcome... You did follow the Certance installation instructions for Solaris OS right? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Running multiple instances of amanda
On Thu, Feb 03, 2005 at 01:59:04PM -0500, Vicki Stanfield wrote: > I got a little further on my attempts to run backups via the new > install. It appears to backup using the new binary except that the changes to > /etc/inetd.conf seem to have caused some of the backups not to work. > For some reason, amcheck is reporting that there is no .amanda-exclude > file on each entry in the amanda disklist file (for a certain machine > at least). That machine is indeed up and that file doesn't exist there, > but it didn't when I was running 2.4.2p2-4 either and I never got these > errors. Did something significant change between those versions in this > area? > > The output from the dump last night indicated that these backups had failed. > > ERROR: morimoto: [Can't open exclude file > '/net/morimoto/client/.amanda-exclude': No such file or directory] > ERROR: morimoto: [Can't open exclude file > '/net/morimoto/project/gelato/.amanda-exclude': No such file or > directory] > ERROR: morimoto: [Can't open exclude file > '/net/morimoto/project/redhat/redhat-updates/.amanda-exclude': No such > file or directory] > > And more. However some on a different filesystem did succeed. What am I > missing here? IIRC, checking by amcheck was added and the "optional" capability to the exclude parameter was added to the config file. Basically saying - if it is there, use it, if not don't. But don't complain. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: configure for amanda server and client coordinations
On Thu, Feb 03, 2005 at 12:50:55PM -0800, Nina Pham wrote: > Please let me setup the scenerio first for 2 unix machines. The amanda > server machine(amserver) is running redhat linux 9, the amanda > client(amclient) machine is running FC2. In addition to serve the client > machine, amserver also backups some files locally. The amserver is > already setup to backup itself. I'm installing amanda on amclient now. > This is my first time to set up amanda, so it's confusing how the client > and server coordinates so that the server will backup itseft and also > backup files from amclient. > From amclient: I do ./configure --with-user=general --with-group=disk > --with-amandahosts --with-configdir=/Backup/Configs > --with-config=WeeklySet --with-index-server=amserver > --with-tape-server=amserver --prefix=/Backup/usr > --with-gnutar-listdir=/Backup/amanda/gnutar-listdir. > > Did I get it right? (-: if that is what you were trying to do :-) If you haven't already done it, you may wish to look at configure's help output. Most of the options are explained - some well, some maybe more cryptically than you would like. Also, I like the "how-to install" document fromt the source tree "docs" directory. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Problem with amrecover
On Fri, Feb 04, 2005 at 11:21:07AM -, soed2003 wrote: > > --- In [EMAIL PROTECTED], Jon LaBadie <[EMAIL PROTECTED]> wrote: > > On Sat, Jan 22, 2005 at 02:45:40PM +0100, Sören Edzen wrote: > > > Hi! > > > > > > I have problem using amrecover. I get the following output: > > > > > > amidxtaped: time 0.002: Ready to execv amrestore with: > > > path = /usr/local/sbin/amrestore > > > argv[0] = "amrestore" > > > argv[1] = "-p" > > > argv[2] = "-h" > > > argv[3] = "/dev/nst0" > > > argv[4] = "^zampo.edzen.net$" > > > argv[5] = "^/usr/local/cvsroot$" > > > argv[6] = "20050115" > > > > As the exact amrestore command is shown, > > can you reproduce the error by running it manually? > > > > If so, it may be time to try and pull off manually > > some of the tape files and see if they are recorded > > correctly. This will involve using mt and dd commands. > > Then tar or dump on the result. > > > > -- > > Jon H. LaBadie [EMAIL PROTECTED] > > JG Computing > > 4455 Province Line Road(609) 252-0159 > > Princeton, NJ 08540-4322 (609) 683-7220 (fax) > > Hi > I tried the exact amrestore command and got this: > > amrestore -p -h /dev/nst0 ^zampo.edzen.net$ ^/usr/local/cvsroot$ 20050204 > amrestore: 0: skipping start of tape: date 20050202 label zampo102 > amrestore: 1: skipping zampo.edzen.net._home_soredz_.mutt.20050202.0 > amrestore: error reading file header: Input/output error > > How to restore with dd? mt is used to rewind the tape, right? > > Any ideas? I'm getting desperate. First I had trouble getting amanda to > perform the > backups. Then when I finally made it I cant restore anything. > And of course, backups are worthless unless you can get your data back. mt is used for more than rewinding. Amanda backs up entities in its disklist file, aka DLE's, such as /home/soredz/.mutt on zampo.edzen.net. (Based on the above, I presume this is an entry in your disklist file, that it is a directory, and you are using gnutar to do the backup of this DLE, is that correct?) Each of these DLE's is backed up to an archive file and put on tape as a single tape file with 32KB of amanda info added at the front of the archive. If your disklist file has 10 DLE's, each is on the tape in a separate tape file. Plus there are two special amanda files, one each at the start and end of the tape. Using unix commands like dd, mt, tar, restore, ... you should be able to retrieve your data even with out amanda commands. That is assuming the data on the tape is good, the archive properly made, taped, and retrievable. Basically you position the tape at the beginning of the tape file you want. mt -f rewind mt -f fsf If you want to look at which DLE this tape file contains, look at the first 32KB with dd. dd if= bs=32k count=1 If you want the archive file on your hard disk, repeat the mt commands to get to the start of the tape file again and use dd. dd if= of= bs=32k skip=1 Theoretically there was no need to repeat the mt commands, you could have done the 2 dd commands one after the other (without the skip=1) dd if= of= bs=32k count=1 dd if= of= bs=32k Now you have your archive file on hard disk. It may be compresses with gzip and need uncompression (depends on your config settings). You should be able to recover what is there with gnutar or your restore command as appropriate for how the archive was made. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Getting Amanda to production
On Fri, Feb 04, 2005 at 02:43:55PM -0500, Gil Naveh wrote: > hello, > > I have a few question in mind so I'll be able to better configure it. > 1) So far I tested Amanda and backed up data on a hard drive and am quit > happy with the results. Recently we got a new tape drive and I need to start > backing up our data into taps. I am going to use the same Amanda server and > clients. Should and if so how do I initialize Amanda so it won't care about > previous backups? Additionally I'll probably have to do some more testing in > order to see how much time/bandwidth Amanda uses when it backup to our tape > drive. But after doing those tests - can I delete those files from the tape? I'd start a different config for the tape. Currently you probably do a cronjob with amdump vtapeconfig, simply comment that out entry and add an amdump realtapeconfig. That way, if you ever need to do vtape backups again, you can just uncomment it. > > 2) I understand that Amanda has its algorithm that decides when do to a full > backup, or an incremental one. >But I am a little confused with labeling tapes. (amlabel) >When I used the hard-disk I define each hard-drive 'partition' as 4GB and > then amlabel each disk e.g. >#/usr/sbin/amlabel DailySet1 HISS01 slot 1 >#/usr/sbin/amlabel DailySet1 HISS02 slot 2 ... >However, I have tapes that each can store 400 compress data, and we have > to store about 30GB - am I domed to use each tape for one backup yes > - or can I 'partition' those tapes? no What some who have a large holding disk(s) space is to let several dump runs collect on the holding disk(s) by not putting a tape in the drive. Amrecover, I'm pretty sure, works with the taped or holding disk dumps. Then insert a tape and amflush them to tape. > Do I have to put each tape in the tape drive and run amlabel? yes > > 3) We have a local and remote sites that we have to backup any thoughts, > comments on what should I keep in mine before doing so. Additionally any > recommendation for what to use to secure our data when backing up remotely? > I read that I can use TCP wrappers but how do I implement it with Amanda? Don't bring them all on line at the same time. Make all your disklist entries but put a # sign to comment them out. Uncomment a couple each dump. Run amcheck with several from each client uncommented BEFORE even thinking about a dump. Maybe use your vtape config to test drive a client before moving them to the big show. > > 4) Finally, we have one tape drive and about 20 servers that we need to > backup from. Some of those servers run on Solaris, some on Windows. Can I > back all servers on the same tape, or do I need to set separate tapes for > Win machines? > Both can share a config. Lots of ways to do the M$ machines, none fully satisfactory. Samba clients to one or several of your unix hosts, nfs shares if your M$ boxes are nfs servers (possibly from M$'s free Services For Unix product - SFU), some windows backup program that writes to a file and backup that single file/directory with amanda, install cygwin on each box and install amanda so it looks like a unix host. Many limitations to backing up M$ boxes, set your expectations low and you won't be unhappy. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Getting Amanda to production
On Fri, Feb 04, 2005 at 05:02:59PM -0500, Gil Naveh wrote: > Thanks Gavin and Jon, > Wow that was great help. > > Regarding my first question, what should I do with the 'old' labels (the > hard disk) can I delete them? Are they important to you? > if not how Amanda knows to write on the new tape drive and not on the hard > disk. How did it know to write to the vtapes? You told it where to write. How will it know where to write to the real tapes? You will tell it. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Getting Amanda to production
On Fri, Feb 04, 2005 at 05:50:58PM -0500, Joshua Baker-LePain wrote: > On Fri, 4 Feb 2005 at 5:40pm, Gil Naveh wrote > > > ok you can laugh at me :) > > but I don't understand something with amlabel: > > how does it knows which tape drive I am referring to? since the command > > looks like: > > #amlabel DailySet1 labelname > > It uses the drive specified in DailySet1's amanda.conf. The config file is the reason why nearly all amanda commands require the configuration to be specified. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Input/Output error on a HP SureStore 24/6
On Sun, Feb 06, 2005 at 06:11:05PM +0100, Markus Diesinng wrote: > Hi! > > Everything seemed to work fine until I typed amcheck, because I get the > following message for every slot the changer got: > > amcheck-server: slot 5: reading label: Input/output error > > The same happens when I try to label a new tape etc. > > I checked the FAQ and saw that the problem is more likely to be a > hardware or system one. Anyway, and I hope it's not too off topic here, > could someone pls help me find out if it's amanda, the system (an SuSE > 8.2 Linux) or the tape library itself. I would like to be sure that it#s > broken before I send it to HP. The box doesn' have warranty anymore... > > Thanks in advance for your suggestions. * bad tapes? * dirty heads? * scsi terminator problems? * scsi cable problems? * any messages in the system log files? * amanda commands are not the only ones that can use the drive, do you have problems using dd, tar, cpio, dump, cat, to write and read to the tape? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: How do I control tapedrive compression?
On Mon, Feb 07, 2005 at 10:07:50AM +0200, Shai Ayal wrote:
> Hi all,
>
> I understand that it is better to turn the tape drive's hardware
> compression off and let Amanda know the real size of the tape and gzip do
> the compression.
"better" is very subjective here. There are valid reasons for choosing
hardware or software compression depending on your situation. Many of
the regular responders on the list use SW compression (myself included)
and are vocal about their preference. But their situation is not yours.
> So... How do I turn it off, or for that matter, how do I know if it's on or
> off?
A question for which the response is hardware and OS dependent. You may
be dealing with physical hardware option switches (in the case or outside),
front panel switches, software control such as with the mt command, OS
config files, or by selecting from several choices, a specific device name.
Check your mt command man page first. Mine (Solaris) has no sub-commands
related to compression. Others, notably linux, do.
>
> My tape is a DLT 80/160 and I use the following for the tape size in
> amanda.conf (from the amanda web site):
>
> define tapetype DLTVS160 {
>comment "DLT VS160 tape drive"
>length 74529 mbytes # 72.78 Gb
>filemark 584 kbytes
>speed 2203 kbytes # 2.15 Mb/s
> }
>
> So my tape is configures with it's uncompressed length. The backups seem to
> be working as expected, but how to make sure there is no hardware
> compression ?
I suspect that when you get things sorted out that the amtapetype command
suggested by Paul B in another response will report a higher capacity.
I'm guessing the above tapetype definition may have been made by a "tapetype"
command run with hardware compression turned on.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: VXA-2 packet-loader issues and AMANDA [Fwd: hard luck with the new autoloader]
On Mon, Feb 07, 2005 at 03:55:36PM -0500, Eric Siegerman wrote: > On Fri, Feb 04, 2005 at 07:21:59PM -0500, Gene Heskett wrote: > > Aha, LVD! LVD is not compatible with the rest of the system unless > > the rest of the system is also LVD. It is two, completely seperate > > signalling methods that just happen to use the same cabling. > > Yes and no. From the SCSI FAQ: "[ANSI] specified that if an LVD > device is designed properly, it can switch to S.E. [single-ended, > i.e. "normal", SCSI] mode and operate with S.E. devices on the > same bus segment." > - http://h000625f788f5.ne.client2.attbi.com/scsi_faq/scsifaq.html#Generic099 > > So if you mix it with S.E., you lose its LVDness, e.g. you have > to stick to a S.E. bus length; but you shouldn't fry any > hardware. Can on look at the device connectors, or better yet, the external connectors, and tell if a device is LVD or SE? Or does one have to check the HW doc? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: AMLABEL
On Mon, Feb 07, 2005 at 10:06:09PM -0500, Gene Heskett wrote: > On Monday 07 February 2005 18:31, Paul Bijnens wrote: > >Gil Naveh wrote: > >> Gene - thanks for trying to help, > >> > >> Currently the only reason that I can think of is that Amanda has > >> not read correctly our tape drive so I am re-running > >> #amtapetype -f /dev/rmt/0n > > > >Add an estimate value! And it takes about 5 hours only. Without > >an estimate it takes a week or so. Like this: > > > > amtapetype -f /dev/rmt/0n -e 200g > > > >On the other hand, writing a few bytes is enough to see it the drive > >works. > > > >About the problem: you did specify that device in your amanda.conf ? > >(I believe that was what Gene was hinting about.) > > Chuckle, yup. Sometimes I get my tongue tangled up with my eyeteeth, > and can't see what I'm writing... Corrections in that case are > always welcome. :-) With the ultrium it is less important about considering HW or SW compression. But be aware that on Solaris whether HW compression is turned on or not is determined by the device you choose. You will find lots of /dev/rmt/0"xyz" devices. The "xyz" determines the properties of the device the driver will set upon opening it. Don't be fooled into assumptions about the various devices. The "c" device is listed as "compressed". But that is the conventional use. There is no certainty that it turns compression on for every device, or that devices without the "c" are no compression. You will even have a "c" device for drives that are not capable of HW compression ;-) Only way to tell is check the docs for the drive and settings for the driver. And then I'd check it if I could. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: AMLABEL - might be a bug in amtapetype
On Tue, Feb 08, 2005 at 09:47:17AM -0500, Gil Naveh wrote: > The issue I had was that the first time I run amtapetype the result was a > little off. I would not consider them different at all. capacity, differed by 0.31 percent filemark, differed by 3.41 percent (and only 0.00045% of capacity) speed,differed by 0.06 percent -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: AMLABEL
On Tue, Feb 08, 2005 at 10:04:40AM +0100, Paul Bijnens wrote: > Jon LaBadie wrote: > > > >With the ultrium it is less important about considering HW or SW > >compression. But be aware that on Solaris whether HW compression > >is turned on or not is determined by the device you choose. You > >will find lots of /dev/rmt/0"xyz" devices. The "xyz" determines > >the properties of the device the driver will set upon opening it. > > > >Don't be fooled into assumptions about the various devices. The > >"c" device is listed as "compressed". But that is the conventional > >use. There is no certainty that it turns compression on for every > >device, or that devices without the "c" are no compression. You > >will even have a "c" device for drives that are not capable of > >HW compression ;-) Only way to tell is check the docs for the > >drive and settings for the driver. And then I'd check it if I > >could. > > > > Indeed. > > And check the file st.init, where the letters get their meaning. > Paul, I'm unfamiliar with the file st.init. Nor do I find one on my system. Did you mean the st.conf file in /kernel/drv? If the st.conf file, I don't find an entries for any lto's I recognize. One poster said it was "builtin" to the driver, another said they had to edit the file and add an entry. The drive docs would probably still be needed to interpret the entry. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: AMLABEL - might be a bug in amtapetype
On Tue, Feb 08, 2005 at 04:02:29PM +0100, Paul Bijnens wrote: > > All that said, when using an LTO drive, you may have hardware > compression enabled anyway, because the hw compr algorithm in those > drives does not expand uncompressable data. Maybe the amtapetype message should be revised to something like: hardware compression off (or drive uses a dynamic compression algorithm) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: AMLABEL - might be a bug in amtapetype
On Tue, Feb 08, 2005 at 05:53:43PM +0100, Paul Bijnens wrote: > Jon LaBadie wrote: > > >On Tue, Feb 08, 2005 at 04:02:29PM +0100, Paul Bijnens wrote: > > > >>All that said, when using an LTO drive, you may have hardware > >>compression enabled anyway, because the hw compr algorithm in those > >>drives does not expand uncompressable data. > > > > > >Maybe the amtapetype message should be revised to something like: > > > > hardware compression off (or drive uses a dynamic compression algorithm) > > > > Actually the message is still correct. > amtapetype does detect if hw compr is enabled. > And you would notice the effect if you wrote without software compression. > The fact that the algorithm does not expand data is something else. My misreading of your statement. I missed the "when using an LTO drive" and thought you meant hardware compression might have been on when amtapetype was run but could have been fooled. Sorry bout dat. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda-2.4.4p4 compile error on solaris 8
On Thu, Feb 10, 2005 at 01:33:53PM +, Rolinux wrote: > > Hi, > I try to compile amanda-2.4.4p4 on solaris 8 to get the amrecover_changer > option (we run now amanda-2.4.3): > > [EMAIL PROTECTED] /root]# uname -a > SunOS backup-a01 5.8 Generic_108528-16 sun4u sparc SUNW,Ultra-250 > ... > > Some system details: > > [EMAIL PROTECTED] amanda-2.4.4p4]$ echo $PATH > /usr/bin::/usr/local/bin:/usr/ucb:/usr/ccs/bin > [EMAIL PROTECTED] amanda-2.4.4p4]$ echo $LD_LIBRARY_PATH > > [EMAIL PROTECTED] amanda-2.4.4p4]$ > [EMAIL PROTECTED] amanda-2.4.4p4]$ type ld > ld is /usr/local/bin/ld Many (most?) gcc installations on Solaris use the standard Sun provided assembler and linker. Is there a specific reason you are not using the Sun linker? Perhaps move /usr/ccs/bin in your path before /usr/local/bin. > [EMAIL PROTECTED] amanda-2.4.4p4]$ > [EMAIL PROTECTED] amanda-2.4.4p4]$ pkginfo -l SMCgcc >PKGINST: SMCgcc > NAME: gcc > CATEGORY: application > ARCH: sparc >VERSION: 3.1 >BASEDIR: /usr/local > VENDOR: Free Software Foundation ... > > Can you please help me with some ideas about how to compile this version of > amanda on solaris 8? > Many gcc installations for Solaris include a script called something like "fixheaders". Does your gcc package have such a script and has it been run? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: backup windows clients
On Thu, Feb 10, 2005 at 09:41:37AM +0100, Chantal Rosmuller wrote: > Hi, I have an amanda-windows question: > I got amanda running on a linux server with linux clients but now I want > to use it for backing up windows 2003 servers as well, > I compiled amanda with samba support: > > According to the documentation I have to put the following in the file > disklist: > > sambaservername //windowsclientname/sharename backuptype > > I dont understand why it wants a sambaserver? I thought amanda only used > smbclient to connect to a windowsshare? I have the sambaserver software > installed on the machine that runs amanda backupserver as well, but do I > have to configure something? > The amanda server does not have to even have samba software installed. It depends on an amanda client acting as the sambaserver. So it needs to know which amanda client is acting as the sambaserver for that particular windows client. Note, you could have several amanda clients acting as samba server, each for several windows clients (spread the load) and the amanda server is often an amanda client as well (backs itself up) and thus could act as an sambaserver. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda-2.4.4p4 compile error on solaris 8
On Thu, Feb 10, 2005 at 04:21:35PM +, Rolinux wrote: > On Thursday 10 February 2005 15:46, Jon LaBadie wrote: > > On Thu, Feb 10, 2005 at 01:33:53PM +, Rolinux wrote: > > > Hi, > > > I try to compile amanda-2.4.4p4 on solaris 8 to get the amrecover_changer > > > option (we run now amanda-2.4.3): > > > > > > [EMAIL PROTECTED] /root]# uname -a > > > SunOS backup-a01 5.8 Generic_108528-16 sun4u sparc SUNW,Ultra-250 > > > > ... > > > > > Some system details: > > > > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ echo $PATH > > > /usr/bin::/usr/local/bin:/usr/ucb:/usr/ccs/bin > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ echo $LD_LIBRARY_PATH > > > > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ type ld > > > ld is /usr/local/bin/ld > > > > Many (most?) gcc installations on Solaris use the standard > > Sun provided assembler and linker. Is there a specific > > reason you are not using the Sun linker? > > > > Perhaps move /usr/ccs/bin in your path before /usr/local/bin. > > Same error: > ... > ld: fatal: relocations remain against allocatable but non-writable sections > collect2: ld returned 1 exit status > *** Error code 1 > make: Fatal error: Command failed for target `libamanda.la' > Current working directory /export/home/mov04995/amanda-2.4.4p4/common-src > *** Error code 1 > make: Fatal error: Command failed for target `all-recursive' > [EMAIL PROTECTED] amanda-2.4.4p4]$ echo $PATH > /usr/bin::/usr/ccs/bin:/usr/local/bin:/usr/ucb:/usr/local/samba/bin > [EMAIL PROTECTED] amanda-2.4.4p4]$ > > Does amanda know about that. It may have recorded the location of cc, ld, ... when you did your configure. Did you reconfigure after the PATH change? BTW, before rerunning ./configure, get rid of all previous configure information. One way is to do a "make distclean". > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ > > > [EMAIL PROTECTED] amanda-2.4.4p4]$ pkginfo -l SMCgcc > > >PKGINST: SMCgcc > > > NAME: gcc > > > CATEGORY: application > > > > ... > > > > > Can you please help me with some ideas about how to compile this version > > > of amanda on solaris 8? > > > > Many gcc installations for Solaris include a script called something > > like "fixheaders". Does your gcc package have such a script and has > > it been run? > > [EMAIL PROTECTED] /]# find / -name fixheaders -print > [EMAIL PROTECTED] /]# > > No fixheaders Did you read the part about "something like". I did not look up in your documentation what it was called (if it is even there or needed). You should have checked your installation documentation. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: tuning dumpcycle, runspercycle, and tapecycle
On Sun, Feb 13, 2005 at 03:20:14PM -0800, David Newman wrote: > On Sun, 13 Feb 2005, Gene Heskett wrote: > > >On Sunday 13 February 2005 12:02, David Newman wrote: > > This isn't so good; again, I'm looking for one full dump and four > incremental dumps on each tape. (FWIW, the server in question is at a > remote location and we can't get to it every day.) > > Jon LaBadie's post on the Amanda top 10 list suggests that Amanda is not > the right tool for this. Is there some way to do this using Amanda? > > If not, any recommendations for alternatives? > Jon LaBadie's post on the top 10 list also discusses the most common alternative. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Can tapes be grouped for different types of back-up?
On Mon, Feb 14, 2005 at 01:49:53AM -0500, Gene Heskett wrote: > On Sunday 13 February 2005 16:43, Erik P. Olsen wrote: > >Until recently I had a DDS3 tape streamer. Then it broke down a few > >weeks ago and I bought a DDS4 tape streamer. I therefore have a lot > > of DDS3 tapes which I like to use for incremental back-ups because > > the new tape streamer can also handle DDS3 tapes. I will then ear > > mark the DDS4 tapes for full back-ups. Is that possible and how can > > that be accomplished? > > Not with only one config setup. And with 2, then the one you would > like to use for incrementals doesn't know anything about the fulls, > so it gets a tummy ache. > ... > > I just had an intrigueing idea though. What if there were two > identical config dirs setup, each subdir for the data amanda keeps > was lndir'ed to yet a third dir which was then the common dir for all > of amanda's data. The only real files in the two config dirs would > be the amanda.conf, and the tapelist. It seems like a bit of hassle > for the initial setup, but could anyone throw a showstopper into this > picture? For a while (about 18-24 months) I was running an archive config similar to that. It was only run sporadically, about 10 times total, and I have only done a few file recoveries. But they worked. My objective was separation of the disklists among the two configs. System files (/, /opt, /var, ...) and user+data files (/home, ...). Same tape list was used, same index, same curinfo, same logfiles, ... Main difference was amanda.conf and disklist. I kept a three directory structure, archive-sys, archive-user, and archive-common. IIRC, even the amanda.conf files were the same, though not expecting that I kept them separate. Most of the rest that joined them was not links, but entries in the amanda.conf file, like logdir, curdir, etc. pointing to the archive-common tree. tapelist might have been links as were a few others. It worked for what I wanted, the ability to archive user or system files "on demand". But it is pretty limited experience to hang your hat on. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Same problem .... Amanda runs out of tape
On Mon, Feb 14, 2005 at 10:23:59AM +0100, [EMAIL PROTECTED] wrote: > Hi, > > I am using a HP Drive 40/80 GB and I am experiencing the same problem. The > DLT has been changed twice. Backup was fine for two > days and today amanda was running again out of tape. Size of the backup is > 12,5 GB what should be no problem. Its exactly as described > the backup is running fine for some time, then SCSI errors occur, and the > backup runs out tape. > Im quite fed up with this matter at the moment, as my backup is quites > scramble up by this. I have been searching for the error for some days, > replaced tapes, replaced the DLT. > > Its frustrating. > Generally we seem to respond to questions of the form "will this drive work with amanda?" with something just short of "they all do". Only regular exception I recall were ? onStream ? drives, do I have that one right? With three similar reports about this specific HP drive, it sounds like we should add another exception. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: VXA-2 packet-loader issues -- a scsi question added
On Mon, Feb 14, 2005 at 03:55:31PM -0500, James D. Freels wrote: > I have the new scsi card (an inexpensive LSILogic LSIU80ALVDB) that > fixes the problems I was having. So, the Exabyte tech support was > correct and the great advice I got herein this mailing list also said > "get a separate scsi card" ! ... > > On Mon, 2005-02-07 at 16:20 -0500, James D. Freels wrote: > > > I am near positive I did not fry anything since everything is working > > correctly except a sustained write to the tape. I recently read an article on trouble shooting I/O performance. Most was related to disk I/O, but one tiny piece caught my eye and could be related. The writer claimed that on a SCSI bus the lower ID numbers were given precedence (priority?), the effect of which was felt during heavy usage. Further, this effect could particularly be felt by tape drives with high SCSI numbers. During heavy total I/O on the bus they would lose their ability to stream more easily than if they had lower ID's. For those intimate with Miss SCSI, any comments? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Exabyte VXA-2-Packet-Loader: new problem
On Tue, Feb 15, 2005 at 01:00:12PM -0500, James D. Freels wrote: > Recall I had a problem getting this device to work properly when it > failed to write to the tape. This problem was correct by a new and > separate scsi card dedicated to the packet loader. > > When I use the older ncr53c7xx driver in Linux, it works OK, but does > not automatically reload tapes. Here is my crontab entry: > > /usr/sbin/amdump new;/usr/bin/amtape new slot next;/usr/bin/amtape new > slot next > > When I do this at the console, it works fine. However, as a cron job, > the amtape commands never work. It is necessary to issue two amtape > commands because the first one unloads the tape, but does not load the > next one as it is supposed to do. After the first command, I get the > following error message: > > amtape: could not load slot 4: Unloading Data Transfer Element into > Storage Element 4...mtx: Request Sense: Long Report=yes > > After I issue the "amtape new slot next" command again, it loads the > tape (with the older ncr53c8xx driver). > > Then I booted linux with the newer sym53c8xx-2 driver. The amdump > command works correctly. The first amtape command also works and > unloads the tape as before. However,when I issue the second amtape > command, I get the following message to the console: > > failed to set sg timeout - Invalid argument > > In fact, any mtx command respnds the same way at this point. Checking > the /var/log/messages log, I find the following: > > Feb 15 12:41:59 fea5 kernel: sym1:3:0: ABORT operation started. > Feb 15 12:41:59 fea5 kernel: sym1:3:control msgout: 80 6. > Feb 15 12:41:59 fea5 kernel: sym1:3:0: ABORT operation complete. > Feb 15 12:42:09 fea5 kernel: sym1:3:0: ABORT operation started. > Feb 15 12:42:14 fea5 kernel: sym1:3:0: ABORT operation timed-out. > Feb 15 12:42:14 fea5 kernel: sym1:3:0: DEVICE RESET operation started. > Feb 15 12:42:19 fea5 kernel: sym1:3:0: DEVICE RESET operation timed-out. > Feb 15 12:42:19 fea5 kernel: sym1:3:0: BUS RESET operation started. > Feb 15 12:42:19 fea5 kernel: sym1: SCSI BUS reset detected. > Feb 15 12:42:19 fea5 kernel: sym1: SCSI BUS has been reset. > Feb 15 12:42:19 fea5 kernel: sym1:3:0: BUS RESET operation complete. > Feb 15 12:42:24 fea5 kernel: scsi: device set offline - not ready or > command retry failed after bus reset: host 1 channel 0 id 3 lun 0 > > > I am wondering if perhaps I need to downgrade the packet-loader option > from "native" to a compatibility mode to let amanda be compatible with > this changer ? > Well you say that "any mtx command" has the problem. From that I'd say it is not related to amanda. On my dds3 drive, when I issue some commands, but particularly a rewind, control comes back to the caller earlier than the drive is really ready. Again, particularly on rewinds, the rewind might still be in progress for quite a while. I can see it when I issue mt commands from a shell prompt. If I issue another command to the drive before it is really ready, even an "mt status", I get error messages. Thus I routinely put in delays (sleep's) in scripts that might rewind a tape or change a tape to another slot. As much as 20 or 30 second delays IIRC. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Exabyte VXA-2-Packet-Loader: new problem
On Tue, Feb 15, 2005 at 02:43:13PM -0500, James D. Freels wrote:
> I think I missed something here. The response below, where did it come
> from ? (about the sleep delays, etc.)
>
In an earlier message I noted that after a command like an mt or mtx,
my drive returned control to the caller before it was ready to accept
the next command. So a sequence like mt rewind; mtx slot 4 might fail
because the rewind was not quite finished. Thus sleep delays.
Eric suggested, quite correctly, that instead of a sleep for a fixed
time, a script, possibly called mtsync, like this would be better:
#!
TapeDev=${1:-/dev/rmt/0ln}
Limit=${2:-120}
cnt=0
while (( cnt <= Limit ))
do
mt -f $TapeDev status > /dev/null 2>&1 && break
sleep 1
(( cnt = cnt + 1 ))
done
exit $(( cnt > Limit ? 1 : 0 ))
Untested, and assumes "mt status" returns 0 if drive is ready
and non-zero if not.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amcheck is coredumping
On Tue, Feb 15, 2005 at 04:48:42PM -0500, Kevin Alford wrote: > > su amanda -c "amcheck /usr/local/etc/amanda/CLASS-WV/Full" > Segmentation fault (core dumped) > '/usr/local/etc/amanda/CLASS-WV/Full' is the strangest looking configname I've seen. Usually is it something like DailySet1. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Installing on SPARC Solaris 8 - ld.so.1 error
On Tue, Feb 15, 2005 at 03:23:27PM -0800, Steve H wrote: > Hello all, >Has anyone experienced an issue with Solaris 8? I have installed readline, > and the libreadline.so.5 is located in /usr/local/lib. When I try to start > Amandad, the system throws an error stating: > > ld.so.1: /usr/local/libexec/amandad: fatal: libreadline.so.5: open failed: No > such file or directory > > Can't quite figure out how to fix this...Any help is appreciated. > Redo your amanda compilation from the ./configure onward. First do a make distclean to get rid of the old config. Then do the new config with an addition LDFLAGS='-L/usr/local/lib -R/usr/local/lib' ./configure ... -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: forcing skipped incrementals into holding disk
On Wed, Feb 16, 2005 at 01:12:11PM +1100, Jamie Wilkinson wrote: > > Since re-reading the manpage, thanks to some advice here, I've increased > runtapes to 2, which I think will solve this. I didn't realise that it was > an upper limit rather than an absolute, although I'm still unconvinced that > Amanda will continue to use only one tape except in this extreme case; we'll > see in the next week how well it goes. Just a point of example; I've had runtapes 3 for about 2 years but very few days exceed 1 tape. Where it helps me is when I'm lazy and forget to change the tape magazine and let several, or a week's worth, of dumps build on the holding disk. When I finally change the magazine and run amflush, amanda will use up to 3 tapes. If runtapes were 1, then I'd have to do many amflushes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: forcing skipped incrementals into holding disk
On Wed, Feb 16, 2005 at 09:19:04AM +0100, Paul Bijnens wrote: > > Jean-Louis said this already: there is also the parameter "maxdumpsize" > to override the default of "runtapes * tapacapacity" upper limit. > That way you can still keep runtapes to whatever you like best, and > still collect DLE that did not fit into runtapes collect into > holdingdisk. This avoids having writen one tape to 100% and the next > one with 1%. I learn something new every month or two :) -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: invalid compressed data--crc error and other corruption on disk files
On Fri, Feb 18, 2005 at 11:36:46AM +, Thomas Charles Robinson wrote: > Hi, > > Currently I have several configurations backing up to disk (just not > using a tape in the slot leaves dump-files on the holding disk) but when > I check the dump files for consistency, about a third of them are > corrupted. > On your holding disk, are some of the files "chunks"? I.e., did you tell amanda, in amanda.conf, that it should make the pieces on the holding disk a maximum size using the chunksize parameter? If so, you may be looking at pieces that must be reassembled for verification. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: invalid compressed data--crc error and other corruption on disk files
On Fri, Feb 18, 2005 at 11:30:04AM -0500, Eric Siegerman wrote: > On Fri, Feb 18, 2005 at 11:36:46AM +, Thomas Charles Robinson wrote: > > [an excellently clear, concise, and complete [1] problem report > > -- thank you! -- which included the following:] > > > > gzip: stdin: invalid compressed data--crc error > > All of tar's varied complaints appear to stem from corrupt input, > which in turn is adequately explained by this message. > > Thus, either gzip or hardware looks like the culprit. RAM is a > good place to look, especially considering that the data being > backed up all resides on the Amanda server; you're giving that > box quite a workout. The disk and its bus (SCSI, IDE, etc.) are > possibilities too, but less likely IMO -- I'd expect the kernel > to detect and report the I/O errors in that case. > > Not to completely rule out problems with Amanda itself -- I've > learned never to rule *anything* out where computers are > concerned (or humans for that matter :-/) -- but it seems > unlikely. > ... > > gzip-1.3.3-9 > > ... is a beta. It might be worthwhile to try the latest released > version, 1.2.4. From the web page, it looks as though that > version can't handle files over 2 GB, so you'll have to split up > any larger DLEs. Or just disable them for the duration of the > test -- no loss; it's not as if you have usable backups of them > now :-( > My Solaris 9 is supplied with gzip 1.3, no extensions and dated 1999. Never experienced any problems. As to a 2GB limit, as gzip is being used in a pipeline I don't think that would apply. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Runtar error
On Fri, Feb 18, 2005 at 10:49:46AM -0600, Dege, Robert C. wrote: > > > I think it is complaining about runtar not being suid root > > like it's supposed to be. After you build amanda as the > > appropriate user, you need to do the 'make install' as root > > to get the proper permissiona and suid bits set appropriately. > > > > Frank > > Frank, > > I think so to, but when I did, ls -l, it shows the correct permissions: > > [EMAIL PROTECTED] amanda]# ls -l /usr/local/dist/libexec/ > total 288 > -rwxr-xr-x 1 amanda amanda 25853 Feb 16 10:43 amandad > -rwsr-x--- 1 root amanda 12417 Feb 16 10:43 amqde > -rwsr-x--- 1 root amanda 18015 Feb 16 10:43 calcsize > -rwsr-x--- 1 root amanda 11872 Feb 16 10:43 killpgrp > -rwxr-xr-x 1 amanda amanda 4822 Feb 16 10:43 patch-system > -rwxr-xr-x 1 root root 28452 Feb 16 10:38 rmt > -rwsr-x--- 1 root amanda 8536 Feb 16 10:43 rundump > -rwsr-x--- 1 root amanda 9947 Feb 16 10:43 runtar > Is your backup running as group "amanda"? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: A question regarding backing up Window machines
On Fri, Jan 28, 2005 at 04:09:57PM -0800, Joe Rhett wrote: > > On Wed, Jan 05, 2005 at 03:58:29PM -0800, Joe Rhett wrote: > > > > > On Mon, Dec 20, 2004 at 04:02:49PM +0100, Paul Bijnens wrote: > > > > > > - You can exclude only one pattern. > > > > > > > On Tue, Jan 04, 2005 at 02:42:13PM -0800, Joe Rhett wrote: > > > > > Um, hello -- can you clue me in on this? I am trying (lazily) to > > > > > figure > > > > > out why my excludes are ignored during backups, but work just fine > > > > > when I > > > > > run the same tar command by hand. I need to add some debugging code, > > > > > and > > > > > just haven't found the time. Have you clued into something I've > > > > > missed? > > > > > > On Tue, Jan 04, 2005 at 06:17:31PM -0500, Jon LaBadie wrote: > > > > could you give us your example exclude (note it is singlular) pattern > > > > that fails with amanda dumps but works with smbclient. Also the path > > > > to file(s) on the window machine that is excluded properly by smbclient > > > > but gets included by amanda. > > > > > > No such animal. I never said smbclient, I said tar. The system is > > > running > > > the amanda client. > > On Wed, Jan 05, 2005 at 10:06:27PM -0500, Jon LaBadie wrote: > > Your subject says "Window machines". > > > > I assumed you were backing up samba shares. > > That uses smbclient with its "tar" option. > > > > If you had meant the much less commonly used cygwin port of amanda, > > which does use gnutar, I thought you would have mentioned it. > > > > Or maybe there is another possibility I haven't considered. > > Give me a hand here. > > I'm sorry, I thought I was on the amanda mailing list, writing about the > amanda client. No, I'm not talking about using a program to back up remote > shares - I am writing about the actual amanda client. > > I'm a bit surprised by your negative response, and your assumption of an > indirect method of backup over the native client. And whatever -- this > issue has been ignored by you guys for years now, why am I bothering? Joe, In your first posting on this thread you deleted the OP's comments, starting with Paul B's response to the OP. That led me to believe, since I did not go back to the archive to examine the thread, that you were the OP. I did not realize you were jumping in on another, windows-related thread with a new, somewhat unrelated question (at least as to OS). If you'd like to continue exploring your situation I'd suggest you start a new thread and I'd like to see the same things I asked for earlier. You still could have shown them to us even given my mistaken assumption that you were dealing with a windows client. Namely things like: - disklist - filesystem organization - exclude file or patterns that amanda does not do what you want - tar command that does (including where you execute from) - ??? whatever you think might help BTW what issue has the list been ignoring? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Backup
On Mon, Feb 21, 2005 at 07:30:07AM +, Gavin Henry wrote: > On Monday 21 Feb 2005 06:05, Kaushal Shriyan wrote: > > Hi All > > > > How can I have monthly,quarterly and half yearly backup using Amanda > > Hi, > > Simply make some more amanda.conf and run them monthly, quarterly and half > yearly. > I presume you want all level 0 dumps. Set the parameters in the new amanda.conf's accordingly. dumpcycle 0, record no, runtapes ?sufficient?, ... -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda - unable to create temporary directory
On Thu, Feb 24, 2005 at 09:04:49AM -0500, Gil Naveh wrote: > > Hello, > > Amanda is successfully running on our Solaris Servers. But on Amanda's > report (the E-mail it sends after amdump) I get the following STRANGE > message: > > ? Unable to create temporary directory in any of the directories listed > below: > ? /tmp/ > ? /var/tmp/ > ? / > > But when I manually accessed the server as Amanda user - I had no problem > creating a folder under /tmp as well as /var/tmp/ ! > > Any thoughts why am I getting this message? > Additionally, can somebody tell me what files does Amada writes under /tmp > folder? I replied to your earlier post on this topic but I don't see my response in my archive of the list. That message is not coming from amanda directly, but coming from ufsdump. Now the question is why ufsdump, running for amanda, can't deal with those directories. I only had one possibility and have little confidence in it. On my system /usr/sbin/ufsdump is a symbolic link to /usr/lib/fs/ufs/ufsdump. The latter program is root-owned, set-uid. Perhaps yours has been altered. $ ls -l /usr/lib/fs/ufs/ufsdump -r-sr-xr-x 1 root bin 83820 Apr 12 2004 /usr/lib/fs/ufs/ufsdump -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda-2.4.4p4 - RedHat 7.3 - Samba-3.0
On Thu, Feb 24, 2005 at 09:19:31AM +, Tom Brown wrote: > Hi > > Using the above fine for ann Unix type hosts and of these tar and dump > based backups work fine. > > Just tried to backup my first windows partition last night and it seems > to have worked but the dump ran with 'strange' - Can anyone point me in > the direction of what might be wrong? > > FAILED AND STRANGE DUMP DETAILS: > > /-- titan //printserver/hyperion lev 0 STRANGE > sendbackup: start [titan://printserver/hyperion level 0] > sendbackup: info BACKUP=/opt/samba/bin/smbclient > sendbackup: info RECOVER_CMD=/opt/samba/bin/smbclient -f... - > sendbackup: info end > ? [2005/02/24 00:46:52, 0] client/clitar.c:process_tar(1433) > ? tar: dumped 3546 files and directories > ? [2005/02/24 00:46:52, 0] client/clitar.c:process_tar(1434) > | Total bytes written: 3155058688 > sendbackup: size 3081112 > sendbackup: end > \ > > thanks for any help! Gee, based on my experience, that would be a very "clean" report of a samba dump. :((My reports always have lots of files and/or directories that could not be backed up because of Windows various permissions and restrictions. The two messages seem to be "normal" end of smbtar/smbclient reports. Amanda maintains list of messages it ignores and considers normal. Some other types of messages it doesn't recognize but feels the backup went to some form of normal completion and considers them "strange", i.e. unrecognized by amanda so the operator must decide if they are of any consequence. If you can put up with reports like this, I'd consider them your "norm" for that DLE. Alternatively, amanda uses pattern matching (regular expression patterns) to recognize messages to ignore or report. Perhaps amanda was meant to ignore those particular messages, but in your version of samba, the message is subtly altered and the pattern no longer matches it. This has happened before. Things as simple as spacing or capitalization or punctuation could throw off the pattern match. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda's report
On Thu, Feb 24, 2005 at 12:52:17PM -0600, Karl W. Burkett wrote:
> Jon and others are correct that this message is coming from ufsdump when it
> is run as a user other than root. For instance when run as "amanda":
>
> $ /usr/sbin/ufsdump 0Ssf 1048576 - /dev/md/rdsk/d92
> Unable to create temporary directory in any of the directories listed
> below:
> /tmp/
> /var/tmp/
> /
> Please correct this problem and rerun the program.
> 43413422080
>
> and when run as root:
> /usr/sbin/ufsdump 0Ssf 1048576 - /dev/md/rdsk/d92
> 43413422080
>
> When I do "truss -o /var/tmp/truss.output /usr/sbin/ufsdump 0Ssf 1048576 -
> /dev/md/rdsk/d92" the following results:
>
> lstat64(".rlg.10aqFe", 0xFFBFE2B8) Err#2 ENOENT
> mkdir("/tmp/.rlg.10aqFe", 0)= 0
> mkdir("/tmp/.rlg.10aqFe/.rlg.10aqFe", 0)= 0
> mount("/dev/md/dsk/d92", "/tmp/.rlg.10aqFe/.rlg.10aqFe",
> MS_RDONLY|MS_DATA|MS_OP
> TIONSTR, "ufs", 0xFFBFEBBC, 4) = 0
> umount2("/dev/md/dsk/d92", 0x) = 0
> rmdir("/tmp/.rlg.10aqFe/.rlg.10aqFe") = 0
> rmdir("/tmp/.rlg.10aqFe") = 0
>
>
> and when run as "amanda":
> Truss will not work on suid applications.
>
For this situation, as root run something like:
truss -f ... su -c amanda ufsdump ...
You will be running truss as root and following (-f) any
child processes created by the primary command (su) which
will change its id and run the app you really want to trace.
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Switching to GNUTAR - RE: Amanda - unable to create temporary directory
On Thu, Feb 24, 2005 at 02:16:36PM -0500, Brian Cuttler wrote: > > Gil, > > I'm confused, why would suid be needed on ufsdump - isn't it run > beneith the amanda program (/usr/local/libexec/) rundump which is suid ? > It is not an amanda requirement, it is the way the system supplies ufsdump. I suspect it is so that things like /etc/dumpdates can be updated. Though I'm not sure /etc/dumpdates gets updated when ufsdump is run by an ordinary user. I'm pretty sure that if run by an ordinary user ufsdump would switch id's to do the actual reading of files so that the ordinary user does not get to "backup" files they are not allowed to see under normal circumstances. But there may be a few tasks that require root privleges outside of the actual dumping. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Switching to GNUTAR - RE: Amanda - unable to create temporary directory
On Thu, Feb 24, 2005 at 01:14:03PM -0500, Gil Naveh wrote: > > At this point it seems that the best way for us is to switch from ufsdump to > gnutar. > But before doing so - what will happen with the backups that I did so far? > (I backed our data on a tape drive). If you change the "program" in your dumptype from "DUMP" to "GNUTAR", you will not be able to use amrecover to get your data back from the earlier backups. Newer backups yes, older ones no. You could use amrestore and the appropriate dump program or you could temporarily reset the "program" parameter while you do your amrecover. > Does ufsdump and gnutar stores the data the same way? (Does ufsdump level > 0,1... is identical to gnutar level 0,1...) Or does it store the data on a > different format? Same way as far as dumplevels are concerned, but the storage format is totally different. > Does Gnutar is going to store the data twice or is it going to overwrite the > data that was stored by ufsdump? I don't get what you are asking by the "twice" question. But amanda does not overwrite things because you change a parameter. Only when a tape gets reused will the old dump data be overwritten. > Any other concerns that I should keep in mind before switching from ufsdump > to gnutar? Unix systems keep 3 timestamps for each file. Because a dump program avoids accessing the files via the normal filesystem calls, these time stamps are not altered by backing up a file. Tar does work with normal filesystem calls. At least one of the 3 timestames will be affected, either the last time the file was read (shows with ls -la) or the last time the file properties were changed (shows with ls -lc). If you do choose to make the switch, I suggest you do it a few DLE's at a time. Change their dumptypes and then uses amadmin to force a level 0 dump for that DLE the next run. A tar level 1 or 2 will not be much use with a ufsdump level 0. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Todd Kover > Sent: Thursday, February 24, 2005 12:37 PM > To: amanda-users@amanda.org > Subject: Re: Amanda - unable to create temporary directory > > > > Jon LaBadie said: > Still can't get you delete superfluous stuff and to bottom or inline post huh? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda indexing and multiple backup rotations
On Thu, Feb 24, 2005 at 04:44:29PM -0800, Brooks, Jason wrote: > Hello, > > A question about amanda's indexing and database > > Short version: > > Does a mechanism to share databases exist in amanda? > > Long version: (best with an example) > > I want >4 different configurations. All but one will be in offsite > storage. I want different configurations because then I will be assured > that each tapeset I have in the changer will be a complete set that I > change out each week for a new set. Before even attempting an answer I have a question. What, for your needs, are the properties of a "complete tapeset"? It sounds like you are thinking of traditional backup systems where you do a monster dump of everything on day zero and then tiny incrementals til the next monster dump. In such a scheme you need the monster dump or none of the little dumps are valid. So they form nice discreet sets. Amanda spreads the level 0 dumps across then entire dumpcycle. So there is no discreet starting point to a set of tapes. If your dumpcycle is 7 days and you do a run every days (runspercycle) then the Monday - Sunday tapes are as valid a set as the Wednesday - Tuesday. Any 7 sequential tapes is a set. > > example: > five configurations: Daily1-5 > dumpcycle = five weeks > runspercycle = 7 days worth dumpcycle is the 'maximum' time you want amanda to allow between level 0 dumps. Do you really only want a full dump every 35 days? I suspect what you want is a dumpcycle of 7 days and runspercycle 7 (note, no units). -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda indexing and multiple backup rotations
On Fri, Feb 25, 2005 at 01:47:32PM +0100, Paul Bijnens wrote: > ... > I can live with the not-so-nice-multiples-of-seven, and amanda doesn't > even care. Put them in a coloured box, a different colour for each > set, or a box labeled "SET-1", "SET-2" etc., and it's still easy > enough to switch the sets. > > I hope you do realize that for Amanda a valid set is just a sequence > of tapes of from any "dumpcycle" days, be it 1-7, or 3-10. > To be able to restore up to a certain date, you need all the preceeding > "dumpcycle" number of tapes before that day. > (That also why you should set runtapes much larger than dumpcycle.) > > You can set runtapes to e.g. 21 instead of 28, but still keep 28 > tapes in the cycle, resulting in any of the last 7 tapes being good > enough to overwrite: no you can easily remove a tape from the cycle, > without supplying a new one immediately too. In the above paragraphs, I believe Paul meant tapecycle where he typed runtapes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda report
On Fri, Feb 25, 2005 at 04:01:38PM +0100, ddaasd wrote: > > > Could you please post in text, not html. I'd like to read your messages and possibly respond. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Strange planner problem 2.4.4p3
On Fri, Feb 25, 2005 at 07:48:29PM -0700, Michael Loftis wrote: > Yes I'm aware p4+ is out...But I'm having a strange problem where my > planner promotes full dumps, but then complains about other full dumps > being delayed because the backup is too big? This makes no sense, > shouldn't it incremental these other dumps in order to keep from delaying > these other full dumps? > > Any ideas, hints, suggestions? Need/want any additional info? It just > seems pathological to do this ever. Not really, remember that the planner is trying to balance media usage over the full dumpcycle, not just today. And for a specific example, suppose your incrementals, obviously critically required, take up 90% of the available tape capacity. Further, the DLE's it would like to to do as full dumps will each take > 10%. They can't be done. But if a small DLE is promoted today, there may be more room available on the a future dump because that DLE doesn't require its full dump. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Amanda indexing and multiple backup rotations
On Fri, Feb 25, 2005 at 02:18:31PM -0800, Brooks, Jason wrote: > Jon H. LaBadie wrote: > > What, for your needs, are the properties of a "complete tapeset"? > > A complete tapeset is set of tapes that holds all completed backups > without having to determine "what tapes in a larger set have > everything". By definition, in amanda today all of the tapes in the > current defined tapeset is a "complete tapeset". Clever use of > "dumpcycle, tapecycle, runspercycle, and runtapes" will allow very fine > granularity, but it is still up to a human to maintain what tapes mean > what if he plans to use offsite storage. > ... > My point is that each configuration would maintain itself and there > wouldn't be any "bleedover" into "another set". If I were to try and > separate a tapecycle of 28 tapes into four 7-day dumpcycles, I would > have to do extra work. For that matter, what says that a new level 0 of > some host/disk won't be written amongst the other host/disk sets' > "previous" level n? As far as I can tell, there is nothing preventing > this. I could be wrong in this last though. > No, you are clearly not wrong. I don't know if you've ever looked at the output of amoverview. Here is an edited sample of my dump levels for six of my DLE's over 3 dumpcycles (18 dumps, 6 runs/cycle). butch /1 0 1 0 0 0 1 1 1 0 1 1 1 1 0 1 1 1 butch /export 0 1 1 0 1 0 1 1 2 0 0 1 1 2 2 2 0 1 butch /images 0 0 0 0 butch /var 1 0 1 0 0 0 1 1 1 0 1 1 1 1 1 0 0 1 butch //tec/D 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 0 1 1 butch //winnie/C 0 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 0 So that is three sets, but I don't see any particular grouping that I would consider complete according to your definition. But any consecutive set of 6 tapes do contain at least one level zero for each DLE and is current/complete upto the date of the last tape of the 6 selected. > My idea is simply to have a "superset" configuration: For the example > above, call this configuration "Global_daily". Very little data would > be stored in this configuration: it would merely be aware of any "child" > configurations and their relationship to each other. > Thus, I would run "amrecover Global_daily ", and set the host, disk, and > date. The superset configuration will know which config for the most > relevent backup set to use. In other words, If I need to restore some > files that are >3 weeks old, and the latest backup set was "daily4", the > superset will know to consult "daily1"'s indexes. It will be the > "daily1" configuration that will turn around and ask for the appropriate > "daily-1-xx" tapes. Is this so different than a single config? You ask for files > 3 weeks old amanda uses its single index and asks for tape-17, tape-18, and tape-19. I'm probably being dense here, but what is gained by having four separate configs for each week's worth of tapes? Unless you want to play the "level 0's on Sunday" games. Where the configs each force level 0's of everything on the first tape(s) of each set and possibly force incrementals only for the rest of the set. > > Maybe "Global_daily"'s amamda.conf file will simply read: > child_set daily1 > child_set daily2 > child_set daily3 > child_set daily4 > > It would be very easy to "retire" a backup configuration too: simply > stop using it, and put the tapes and their appropriate documentation > into deep storage. Thus, If I get a command from legal to "maintain all > backups from this point forward" due to some impending lawsuit, the only > thing I will be doing is creating a new backup set each week, and > retiring the oldest. I know I can take the oldest 7 or so tapes off of > the rotation, but I would like it to be neat and tidier. How do you envision rotating sets into the schedule? Will set daily2 just continue backing up where the last dumps of set daily1 finished. Or will it pick up where daily2 was when it was last used, three weeks ago? If each set continues where the other left off then it seems to simply be an artificial structure on top of a single config with four dumpcycles of tape. > > Perhaps in addition, rather than there being a different cronjob each > time, run: "amdump Global_daily" each time. Put another set of > configuration lines in the global config file to determine which set to > use. Perhaps something like (for dumpcycles of 7 days) "52 weeks mod > 4". Then have the amdump script, or planner simply exec "amdump > daily3". > I.e. do some fancy calculation to have amanda select the next set of tapes to be used. Something it does automatically from the tapelist anyway. ??? > > > > So why not number the tapes sequentially, daily01-daily28; > > each week you bring in a set of the 7 oldest tapes in to be > > overwritten. > > Because this would require a human do maintain this "meta" info about > the tapes. Humans can fail. I fail more often than my scripts. thus > if I
