Re: Disaster recovery.
On Friday 08 August 2003 19:16, Gene Heskett wrote: If the amanda index (and tape in my case) server is lost and all I have left is the tapes. How do I rebuild new index files for my tapes so that I can restore? I can't just restore the last set of index files from the most recent backup because they would reflect the state of the tapes from one day earlier (wouldn't they?). I can't find a tool that I can feed the collection of tapes so that it will rebuild the index files. Is there such a thing. Not that I'm aware of, however other amanda experts may well point you to the correct answer. This has been mentioned before and what I, and a number of others do, is keep a copy of the indices, config file etc. on a different disk / server / building / planet - up to you how far you choose to go. If you use a commercial offsite service, or even a homebrew offsite service, you could also put all that data on a CD-R (or CD-RW) and put it with the tapes. -- Niall
Re: The mailing list and spam
On Sunday 10 August 2003 09:35, Marty Shannon, RHCE wrote: No mas! I must unsubscribe in order to keep my job. I've already been disciplined for receiving AMANDA mail that Wow - what a pisser of a job you must have if you can be disciplined for something which is out of your control. Do you seriously expect anyone to believe that you never receive objectionable spam from ANYWHERE except the Amanda list ? includes *PORN*. I cannot believe that the list manager really wants to lose subscribers this way I'm not the list manager but TBH I'm sure he doesn't care if you (or anyone else) stays or goes. The list is a very useful technical resource for Amanda users - use it or not at your discretion. but the last discussion indicated that it was more important to have the list receive porn than to make the list useful. Well, I ignored the discussion which TBH took up more of my time than did the spam, which Spam Assassin caught for me anyway. Do not bother to respond. No bother at all. I do *NOT* need to get fired just because I subscribe to this list. No, you don't, but if that's likely to happen then you seriously need to consider your employment situation IMHO. Diety help the newbies (Just for the record, she will absolutely not do so -- in my humble opinion.) No, deities have weightier matters on their minds. Newbies should RTFM and if they're stuck then, this is the place to come. Tugba wrote: [Image] [Image] [Image] [Image] [Image] [Image] if you dont want receive mail please send blank email to [EMAIL PROTECTED] You gotta be kidding, right? This is what the AMANDA list is all about, right? No. Funny that, you being an RHCE and all - you'd think you could understand what the Amanda mailing list is all about. Unfortunately, being a public list, it's subject to spam, and the list manager doesn't choose to filter it which is his prerogative. It's 2003 - spam exists, and until capital punishment for spammers arrives, it's likely to continue to exist. There are numerous technical solutions to spam, including the free and excellent Spam Assassin (written by a fellow member of the Irish Linux Users Group) as previously mentioned. Use one of those - it seems it might well save you your job. -- Niall P.S. Or you could stop using email at all - that'll surely stop the spam.
Re: Failure because of missing tapes.
On Monday 11 August 2003 02:04, Stephen Carville wrote:
Perhaps but, AFAIK, if the tape is more than one cycle old (in this
case over seven days) it should be reusable.
I think you're getting your cycles mixed up here. You have:
from amanda.conf
dumpcycle 7 days
runspercycle 6
tapecycle 14 tapes
runtapes 3
AIUI it is the value of tapecycle which determines the reuability of a tape.
In your configuration, Amanda won't reuse a tape until at least 13 other
tapes have been used since that tape was last used.
--
Niall
Re: How-To Exclude files from a ufsdump backup
On Friday 08 August 2003 15:06, Pablo Jejcic wrote: I have Amanda working without problems on our network, and backing up many directories using ufsdump (on solaris 9). Now, we have a new project which will need a lot of static information, and I don't want to backup it everyday, but they still need a full fs for theirselfs. Therefore, how can I exclude some files/directories from a ufsdump backup? You can't exclude some files/directories from a ufsdump backup but you don't have to - amanda uses ufsdump with levels and won't backup all that static information everyday - it will only be backed up when there's a level 0 dump done, which is as a rule once per dumpcycle. -- Niall
Re: newbie problems
On Thursday 24 July 2003 15:56, Yogish wrote: 1.How can I restore just a few files from a complete backup, for testing. amrecover is the tool you want there. 2. I want to do I complete backup,once a week and the incremental backup the remaining days Wow - this must be THE most FAQ. Although you can work hard to make amanda do that, you DON'T want to. You tell amanda WHAT you want to back up, and how often a full backup should be made, and how often you will do a backup run, and a few other things, and amanda shuffles things in her own way. however when I try to backup data for the first time on tape, should I specify to amanda to do full backup or it will only do it. When you add a new entry to the disklist, amanda will try to do a full backup of that entry at the next run, provided that it fits. 3. Can I get a good documentation for Amanda? There are the man pages, and there's a large amount of stuff on the web, and there's a chapter in a backup book which is about amanda and is available free on-line - follow your nose from www.amanda.org Please help if you can The backup gods help those who help themselves, so read what's out there. amanda has a none too shallow learning curve, but it's worth it. -- Niall
Re: Backup History question
On Wednesday 23 July 2003 04:18, Jon LaBadie wrote: How can I easily and quickly get a list of the last n backup tapes which were used? I need to be able to identify the tapes for rotation to an offsite storage facility. tail -'n' tapelist ? No - that gets you the OLDEST n tapes. head -n tapelist will get you the n most recently used tapes. -- Niall
Re: How could amflush NOT flush ? :o) [OFF TOPIC]
On Wednesday 23 July 2003 15:36, Nicolas Ecarnot wrote: The doc (under FreeBSD 5.1) also says that the hard links can't be used for directories, but only for files. I tested it, and indeed, I'm stuck. A little search on google explained me that the filesystem limits that because every file needs to now who is its father (directory), and has to have only one father. This seems related to some inability to detect the recursivity in some cases (... foggy, ain't it ? :o) Well, so I'm stuck here with my problem... too bad... A little thinking here would help - in your script which runs amanda, all you have to do is hardlink every file in the last holding directory to a similarly named file in another directory and you're done. But I'm not sure what that gets you TBH, apart from a set of files. Because amanda has now flushed those files, she won't look in the holding disk when asked to recover. Two options there : 1) Recover from those backups manually 2) Also before the daily run, copy (not link) the index files (and possibly some more stuff) elsewhere. After the daily run, rename this other directory to the name of the just flushed holding directory.Then, if you need to recover from the holding disk, you just put the (1 amanda run old) set of files back in place, and amrecover, working from the old files, happily retrieves from the holding disk. Option 2) sounds messy, but once it's scripted, it'd be smooth. -- Niall
Re: I'm lost...
On Wednesday 16 July 2003 10:43, Anders Norrbring wrote: I thought (after searching the web) that amanda should provide a nice and friendly X interface, or at least a web page to do administering and job control in? Exactly what led you to think that ? Can you provide references ? The idea of a GUI or web interface has been mentioned often and as someone said yesterday, PCABM (patches cheerfully accepted by maintainers) After what I've seen so far, trying to find any documentation at all, is that it's completely .conf file manoeuvred? If I want to change anything, let's say awhat files should be backed up in this run, I need to edit a .conf file, and then run it? That's right - unfortunately, amanda isn't clairvoyant, and she needs you to tell her what to do. If that's the case, can someone please point me to what I'm looking for?? I'm sure someone can, but that someone would need to first know what it is you're looking for. If it's amanda documentation so that you know what config files you need to change, www.amanda.org would be a good place to start. If OTOH you're looking for the source of an Amanda pointy clicky interface, you're SOL. If you're looking for a backup tool which has a pointy clicky interface, there are a number of vendors of proprietary tools out there - Bru, Veritas, Arkeiea etc. As I'm not afraid of vi (I hate it, but I'm not AFRAID of it - and anyway, I use emacs) I use amanda, like most people here (here being the amanda mailing list) so I'm afraid I can't help you with that. Google is your friend. Niall
Re: Amanda don't work with Exabyte VXA-2
On Tuesday 15 July 2003 09:46, Shulov Leonid wrote: 1. amcheck don't see exabyte VXA-2 tape : amcheck DailySet1 Amanda Tape Server Host Check - Holding disk /home/amanda/tmp: 163717756 KB disk space available, that's plenty ERROR: /dev/null: rewinding tape: Inappropriate ioctl for device (expecting a new tape) You can fix that by doing this (as root) cd /dev rm null ln -s st0 null but it MIGHT be better to fix your config file. Help me please. Help yourself - RTFM. Niall
Re: amanda + gzip errors on debian?
On Tuesday 15 July 2003 16:07, Kurt Yoder wrote: they seem to go fine. However, upon verifying the backups, I notice gzip errors. I get two different kinds of errors: crc errors and format violated errors. The errors don't happen on all dump images, usually just the bigger ones (which means level 0's, which means I'm screwed for restores!). I've had them crop up from 300 MB into the image to 10 GB into it, and anywhere in between. At least one gzipped image fails on every backup. . . . Anyone else noticed this problem? And fixed it? What's your backup device ? If it's a SCSI tape then I'd say your problem is most likely SCSI cabling termination. I had this a long time ago and it drove me nuts. I eventually found that the SCSI chain wasn't terminated correctly. Just like you, I would only encounter the problems on big backups (because it was only producing occasional errors, and the bigger the file, the more likely I was to encounter it. The way to fix the problem is to make sure that all SCSI cable connections are well made and that termination is correct. Using good quality SCSI cables is a big help too. If OTOH you're not using SCSI tape then I'm afraid I'm all out of clue. Kindest regards, Niall O Broin
Re: Amanda don't work with Exabyte VXA-2
On Tuesday 15 July 2003 16:22, Jon LaBadie wrote: You can fix that by doing this (as root) cd /dev rm null ln -s st0 null Shulov, just as a check, Niall meant this as sarcasm. Well, my tongue was rather near my cheek, let's say. I guess this WOULD work, but could lead to some 'interesting' side effects. DO NOT DO IT! Probably best alright. But do follow the comments in the amanda.conf file and install docs to set up your configuration. Bottom line - amanda has to know where she is supposed to put the backups. Perhaps you know where your tape drive is, but amanda surely doesn't unless you tell her. Niall
Re: sdlt versus DLT library
On Tuesday 15 July 2003 17:01, Jeremy L. Mordkoff wrote: I'm using a DLT IV tape drive now (one tape a night), but it's not big enough. I have 70GB partitions and when they get full, I'll have problems. If I switch to a DLT tape library (8x40GB), will that solve my problem? Will Amanda split backup across two tapes? Or should I just get a SDLT tape drive (160GB)? This is a VVFAQ. Amanda will NOT split one backup across two tapes. But you can - you simply use multiple DLEs (disk list entries) for your bigger partitions, ensuring that each one is small enough to fit on a tape. Then you can decide what to do based on your backup needs, how long you want between level 0 backups etc. But if for some reason multiple DLEs doesn't suit you, an SDLT tape will help whereas a DLT library will not. Niall
Copying index records
We're happily using Amanda with a bunch of IDE disks giving us about 3 months worth of backups. But now the boss has taken a notion to do monthly archive backups of certain filesystems too. That's OK - I've a DDS3 in the backup box, and that should be just big enough to backup what I need to archive, and setting up a config for level 0 only should be OK - already do that for our offsite backups. BUT i would like, if it's not TOO troublesome, to be able to archive the files which I currently have in the regular rotation. So I'd take the backup files which are on the disks and put them on tape, and put the index records from the daily config into the archive config. But how do I do that, or is there any reasonable way ? I thought amadmin import/export might be what I wanted, but that's only the curinfo records. All thoughts welcome, Niall
amflush problem
I've been a happy amanda user for quite some time now but now I've a little problem. I backup to disk and backups occassionally stay in holding because of a shortage of disk space (I'm using 2.4.3 - want to upgrade soon to 2.4.4 to be able to use autoflush). This hasn't been a problem as I just ran amflush but now I've some dumps that I can't flush. This has happened for a while now and I get mails like this: DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-KBOUT-KB COMP% MMM:SS KB/s MMM:SS KB/s --- -- --- arda /NO FILE TO FLUSH arda /bootNO FILE TO FLUSH bree /NO FILE TO FLUSH bree /bootNO FILE TO FLUSH dale /NO FILE TO FLUSH dale /bootNO FILE TO FLUSH I get this NO FILE TO FLUSH message for every disk in my disklist, and there are dumps for those disks in the holding disk. In the last couple of days amdump doesn't seem to complete properly. Although ps shows me no relevant processes, I have to run amcleanup - then I get the above mail. I get the message about the unprocessed logfile and when I look at the logfile I see: DISK amflush arda / DISK amflush arda /boot DISK amflush bree / DISK amflush bree /boot DISK amflush dale / DISK amflush dale /boot START amflush date 20030601 START driver date 20030601 STATS driver startup time 0.015 START taper datestamp 20030601 label TIZdaily45 tape 0 INFO taper tape TIZdaily45 kb 0 fm 0 [OK] FINISH driver date 20030601 time 1.116 This is the entire logfile, except that I have removed a number of the DISK amflush lines to shorten this mail. Can anyone suggest what's going on here, and how I might flush these outstanding dumps ? Niall
Re: amflush problem
On Monday 02 June 2003 15:45, Jon LaBadie wrote: Can anyone suggest what's going on here, and how I might flush these outstanding dumps ? Gosh those straws are hard to grasp :) Config changes? switch holding disk location? names? fqdn's? disklist? Fair question Jon, and I know many people often don't give sufficient information. But I gave all the information I had available. I have changed nothing. Rebuild amanda and do an incomplete installation so part are from different version? I take it that's a question rather than a suggestion :-) And no, I didn't do that. This my backup box, which has been happily running 2.4.3 for some time now. Are they really dump files or other cruft? They're really dump files. Was a dump aborted so they are incomplete? No - just ran out of disk (tape) space, so the dumps stayed in holding. Are they possibly so old their dumps are not in the logs any longer? No - all quite recent. Do you really need them? :)) On tape? If not does amcleanup get rid of them? Well, I would like to have them - they're recent backups. What happens if I delete them ? Are they currently in the indices, or do they only go there when they get flushed from holding ? And anyway, this seems to be a problem now such that future held dumps will be unflushable too - I'd like to avoid that if I can. Niall
Re: Disabled hw compression
On Tuesday 18 February 2003 20:42, Gene Heskett wrote: mt -f /dev/st0 datcompression 0 has worked as wanted (tapeinfo is showing the change). A Side comment here in case the mt folks are watching. I would be VERY nice if mt had the same syntax across all the platforms it runs on. But there are no mt folks - that's the problem. mt is an old program, and there are many versions of it, many of which have different implementors - why, even amanda has its own version of mt now. And unfortunately, these implementors had different idea about the functionality they would add to mt, and even about the names they would give to those functionalities. Kindest regards, Niall O Broin
Re: Tape running out of space, NOT possible
On Wednesday 12 February 2003 15:27, Rebecca Pakish Crum wrote: amanda is telling me my dump is too big and degrading my level 0's to level 1! How is this possible? I run amdump from a script called by cron and have for some time. The first line in my script rewinds the tape, just in case. Tape Time (hrs:min)0:19 0:19 0:00 Tape Size (meg) 11553.111508.2 44.9 Here's your problem - you've told amanda that your tape capacity is ~ 11.5 GB - did you maybe recently upgrade from a DDS4 and forget to change your tapetype ? Regards, Niall O Broin
Re: Problems with reiserfs?
On Tuesday 11 February 2003 07:53, Rainer Hofmann wrote: Is reiserfs the problem? The amanda version installed on the backup server is amanda-2.4.1p1-29. On a different machine I'm running ext3 which is backed up properly on the same backup server. What's your backup method - dump or tar ? Dump doesn't work whatsoever with reiserfs (it undestands the ext2 FS format, and ext3 is basically the same) so that may be your problem. Also, it should be mentioned that Linus recently said that dump should NOT be used on Linux as a backup method because of the way Linux handles filesystems. He says that it may well work 99% of the time, but the 1% may be when you need to restore. Mit freundlichen Grüssen, Niall O Broin
Re: Network Attached Storage
On Thursday 09 January 2003 13:28, wab wrote: Greetings. I was wondering if anybody here uses AMANDA with Network Attached Storage, as opposed to tapes, and how it works. Thanks! The amanda server would see the NAS box as a disk drive, and then you would use the file: driver in 2.4.3. It works just fine, if a little hairy to get started with because of the lack of documentation. I plan to write such documentation Real Soon Now, but I'm not going to until my configuration is closer to optimal - for instance, I don't use a changer at the moment, and that's definitely the best way to go. Kindest regards, Niall O Broin
Re: Make amanda unload tape drive?
On Thursday 09 January 2003 20:01, John Oliver wrote: Can amanda send a signal to my DLT IV to unload the tape after backups are finished, so whoever goes back to swap tapes can just do it without waiting to manually unload? Amanda can't but the mt command can. Just add mt -f TAPEDEV rewoffl to the script you call amdump from. If you currently call amdump directly, replace that by a script. Kindest regards, Niall O Broin
Re: Tapeless Backup Documentation - call for volunteers
On Thu, Jan 02, 2003 at 12:15:26PM -0500, Jon LaBadie wrote: There are messages similar to the following, posted frequently to the amanda-users list: There is not much documentation in setting up AMANDA to be backuped up to a Hard Drive. I checked web sites, the MAN pages, the mailing lists. And three weeks of trying to do this on my own, I need help. I know some of you are using this feature. Anyone care to volunteer to write a HOW-TO that might be added to our documentation library. You would gain fame and fortune (well fame anyway, among a small elite group of appreciative readers). There is a sort of howto out there, but it's not perfect. I'd be willing to write it, perhaps in cooperation with somebody(ies) else. However, I know my method is sub optimal. I use some shell script hacking in my daily amdump script and I know the right way to do this is with a customised changer script. If anyone out there who has done that, but doesn't have the time or inclination to write the howto, would care to pass on details to me I'd be happy to oblige. Kindest regards, Niall O Broin
Re: recommendations for backing up Amanda?
On Saturday 07 December 2002 01:52, you wrote: I'm doing that too. But I should move that script to after the backup is done, as the current way means they are a day older than the tape. Thanks for pointing that detail out Niall. Doing it your way means they'll only be a day out of date if I have to pull them from the tape. No ! No ! - doing it the way I suggested (I hesitate to call it my way - sysadmins doing backup seriously have been doing something like that since forever) means not pulling it from tape at all, and having a current copy always available separate from the tape, and separate from the disk where the files normally live. I don't think you can ever really have an accurate copy of the index files on tape (*) because the files are being updated as each backup is made (or not, but then they're not much use to you). Best you can have on tape is a saved away copy of yesterday's indices. Niall (*) But, and there's always a but, there is a way of having them on tape. When amanda is one, don't have her eject the last tape, but rather seek to EOM and then put a copy of the indices in whatever format suits you on the tape. Of course Amanda won't know about this copy of the indices. In practice, this is about the same as using the method of doing a non-indexed Amanda backup at a certain time, I suppose. Bottom line - indices are small, relatively speaking. Save them somewhere convenient which is not a tape.
Re: recommendations for backing up Amanda?
On Thursday 05 December 2002 17:22, Mark Stosberg wrote: I'm curious how some you have dealt with the catch-22 of backing up Amanda's configuration files. Ideally during the restore process Amanda is available with all it's configuration files intact. However, if the Amanda server needs to be restored, you can't restore them in the usual Amanda way, because the config files to do so would be on a backup tape... This is a FFAQ (Fairly Frequently . . . .) - the sensible thing to do is to copy the amanda config files to a physically different disk, and preferrrably even to a different computer, even offsite if you want to be really paranoid. Kindest regards, Niall O Broin
Re: Tape labelling and strategy
On Friday 29 November 2002 09:16, [EMAIL PROTECTED] wrote: I would like to rotate between multiple sets of 8 tapes because my changer supports maximum 8 tapes. So I have defined the following in my amanda.conf: dumpcycle 5 days runspercycle 5 tapecycle 8 runtapes 2 When one set of 8 tapes is finished I will take it offsite and then start up with the next set, I plan to have maybe 5 sets which i will constantly rotate which means overwritting stuff. What kind of labelling should I use for that ? I was thinking of something like nameXX-Y where XX is the tape number where Y is the set letter, for example: mysite03-B that would be tape number 4 from set B This, like the people who want to do fulls on Friday and incrementals every other day, is another example of wanting to get Amanda to work YOUR way rather than HER way. You can of course, if you want, do that, but know why you're doing it. I'm guessing from what you've written that you want to have everything backed up at level 0 once per week but only do backups on weekdays (because you have runspercycle 5) but you would usually have dumpcycle 7 days in this case because your dump cycle takes up 7 calendar days. I also presume your dumps can take up to two tapes (because you have runtapes 2 :-) ) so I'd suggest the following dumpcycle 7 days runspercycle 5 tapecycle 40 (or however many tapes you have) runtapes 2 and then you simply label your tapes (physically and with amlabel) mysite01 to mysite40 (rather than 00-39 - easier for humans, and amanda doesn't care) With runspercycle 5 and runtapes 2 a complete set of tapes is anyway not 8 but rather ten. Doing it the way I suggest means that there is no confusion of sets - tapes simply have numbers. You'll have to reload the changer when it's used the last tape and the you can send that changerload of tapes offsite if desired. Personally, I'd keep the previous load of tapes in my office to cope with demands for recovery of recently accidentally erased files - in my experience by far the most common reason for restores. Kindest regards, Niall O Broin
Re: backups mirrored to remote tape (or just host)
On Thursday 28 November 2002 09:12, Paul Jakma wrote: driver: - dump of file x finished - launches script (and registers event for when it exits) ...does all its other stuff... - wait for all tapers and 'scripts' on a file to finish before unlinking a file. Hi Paul, I just had a thought - at the moment you could fudge that I think by using the FILE: device. Amanda does backups to disk by using the FILE: device in a very similar manner to a tape drive. You'd have a process which watched the FILE: devices data directory and took files from it as soon as they appeared and wrote them to two separate tape drives. OK - that's the science - the rest is just engineering. Kindest regards, Niall O Broin P.S. Of course this is a kludge, no, sorry, it's a KLUDGE but if you're in a hurry for this functionality . . . .
Re: database backup
On Wednesday 27 November 2002 15:47, Jerry wrote: I'd like to stop a mysql database for a backup runs, and start it back up after the backup. Someone else has suggested using mysqldump which is fine, though the dumped files are much bigger than the databases and depending on your database, this may or may not be an issue. Another option is to use mysqlhotcopy - from the docs mysqlhotcopy is designed to make stable copies of live MySQL databases. Here live means that the database server is running and the database may be in active use. And stable means that the copy will not have any corruptions that could occur if the table files were simply copied without first being locked and flushed from within the server. so the procedure would be that in your script which calls amdump, you first run mysqlhotcopy before doing the actual amdump. In your exclude list, exclude the mysql data directories (because copying these active files is likely to have bad results) but of course you won't include the directory to which you have hotcopied the database files. In the event of a restore, you just need to stop MySQL, copy the database files from the hotcopied area to your MySQL data directory, and restart MySQL. This method has minimal service impact (mysqlhotcopy issues a write lock on the tables whilst copying but endeavours to hold this lock for as short a time as possible) and results in fewer bytes being backed up than with mysqldump. Regards, Niall O Broin
Re: GNUTAR not found on my system
On Tuesday 26 November 2002 20:23, Ivan Zaigralin wrote: calcsize: gnutar not available on this system calcsize: gnutar not available on this system calcsize: gnutar not available on this system calcsize: gnutar not available on this system I tried to look this up in the provided documentation forums, but found nothing. What could be causing the problem? Absolute and total stab in the dark here, but perhaps gnutar is not available on this system ? If that is indeed the case, then backing up using tar is going to be rather difficult. If gnutar is available on the system, then you need to investigate why amanda isn't finding it - is it not in a standard location ? Kindest regards, Niall O Broin
Re: DAT hardware or software compression
On Friday 22 November 2002 11:01, Paul Bijnens wrote: If there is one flipped bit (read error), everything compressed with gzip after that position is lost. With HW compression you can continue restoring after that position plus some overhead. This comes up now and then, but I have my doubts about it. It assumes that on the tape are only your databits, and there is no error detection To paraphrase a best selling book - blessed are those who have not seen, and yet believe This has happened to me. My own fault in that cabling/termination on the SCSI chain was not up to scratch but it led to very occasional bad bits going to the tape which didn't give any errors when writing but when attempting to read (e.g. to verify) I got GUN tar errors from the gzipped tar file. Tar can resynchronise after bit errors but it seems that gzip cannot. This had me puzzled for a while until I redid the cabling and termination on the SCSI chain when all became sweetness and light. Kind regards, Niall O Broin
Re: DAT hardware or software compression
On Friday 22 November 2002 16:28, Gene Heskett wrote: Which should be reason enough to see if the use of bzip2 could be incorporated into amanda. AIUI, bz2 can re-synch, losing only the actual file that the error effected. Without even looking at the matter I can tell you that this is incorrect because a bzipped tar file is simply a compressed stream of data - bzip2 doesn't have any understanding of the structure of a tar file so if it could re-synch it couldn't do so to a granularity of one file in a tar file and hence lose only the actual file that the error affected. But enough talk - how about some action. I took a bzipped tar file and made a random single bit change in it. Such a corrupted bzipped tar behaves just like a corrupted gzipped tar file - ticks along fine until you hit the error and then you get tar errors like: tar: Skipping to next header tar: Archive contains obsolescent base-64 headers followed by bzip errors like: bzip2: Data integrity error when decompressing. Input file = (stdin), output file = (stdout) bzip2 tells you that: You can use the `bzip2recover' program to attempt to recover data from undamaged sections of corrupted files. so I did that - it produces a number of individual .bz2 files. Each of those can be bunzipped (except for the one which contained the error - you get the same error message, and the suggestion to use bzip2recover) and you can then use cat to glue them back together and throw them through tar like: cat rec1file.tar rec2file.tar . . . . recfile.tar | tar tf - which has about the same effect as simply running tar tjf against the original corrupt file - tar is happy until it reaches the missing corrupt section but the gap is too much for it to bridge. Food for thought, since bz2 can recover a dropped bit or several... But for our purposes, what it does is unfortunately about as much use as a rubber hacksaw blade. So if you're itching to spend some time hacking on amanda, adding bzip2 support is not going to help with bit errors on tapes. But there again, this is why you use enought tapes to always have a couple of level 0 to hand - to be sure, to be sure. The only way to recover from single bit errors is to use an ECC code, which is what decent tape drives do at a hardware level anyway. bzip2 or such a compression program could of course add in an ECC code but this would make the compressed file bigger because TANSTAAFL and as the purpose of a compression program is to make files smaller, their authors haven't felt the need to add this facility. Sorry if I've burst anyone's bubble . . . Kindest regards, Niall O Broin
Re: remote site, no changer
On Sunday 17 November 2002 22:51, Charles Sprickman wrote: getting worried. It seems that amanda wants a new tape every day if I'm doing daily backups. This presents a problem. I have a single 70GB drive, no changer, and the systems being backed up are at a remote location. At most I can have someone there once a week to change tapes. Capacity is not a problem; there will be less than a few hundred MB of change per day, and an initial full dump of all 11 machines should come in under 5-10GB. So the tape doesn't need to be changed out because it's getting full... There is no way of getting amanda to append a backup onto a tape which already contains one - the first thing she does with a tape without exception is rewind it. What you CAN do, however, is use a holding disk large enough to hold all your backups for the week - shouldn't be a problem given the sizes you've mentioned. Each week when you have the tape changed, you make sure that amanda WON'T use the new tape by e.g. zeroing out the start of it. Then when the last backup for the week is done, you amlabel the tape (with the -f flag to force labelling, because you'll be using an existing tape's name) and then do amflush. Downsides are that you'll have to do a little (very little though) tracking of what tape you need (you can read this from the tapelist, or avoid it altogether by saving the label block from the tape with dd before zeroing it, and then using dd to put it back) and more important, from a backup POV, you have a week's exposure to risk of data loss if a disk being backed up AND the holding disk both fail in the same week. If the disks are from IBM and of a certain vintage, this is a distinct possibility :-( Regards, Niall O Broin
Re: backing up to a hard drive
On Saturday 16 November 2002 12:57, jeff covey wrote: i've been over the documentation, through the faq-o-matic, etc., and i can't find the answer to what i think is a simple question: how can i use amanda to back up to space on the server's hard drive? all the documentation is about tapes, tapes, tapes. i have a set of windows machines that i just want to back up over samba to /backups on hda. Use Amanda 2.4.3 and the file: device. For further details, and you will need some :-) look back through the archives of the mailing list - this has been answered several times recently. Kindest regards, Niall O Broin
Re: Amanda Backup taking to long
On Thursday 14 November 2002 21:16, Kathy Ange wrote: I am a newbie, I have just run my fist dump on the server only, and it is taking way to long. So I figure I have something configured incorrectly. I tried looking in FAQ but didn't see anything, so here I am. I know it takes to long As I ran a manual backup (ufsdump line commands) of the same area and it only took 12 minutes,using amanda over 90 minutes. ufsdump simply does ufsdump. Amanda OTOH has to do an estimate, do the dump via gzip to compress it, and then write the dump to tape. If your holding disk is smaller than the sum of your two biggest disklist entries (I see you only have 2GB of holding space allocated) then you run the risk of having to do dumps direct to tape, which is slow particularly when using server compression as you are as you won't be able to keep the tape streaming. Then of course there's the amanda overhead i.e the time taken for amanda to do its things e.g. write indices etc. other than just the backup All these factors together are what makes Amanda slower than straight ufsdump for this simple case. If you're using amanda in a more normal configuration i.e. where she's backing up numerous clients to one tape device with sufficient holding space allocated, then she is quite fast because she has multiple clients dumping to the server and then one process on the server keeping the tape drive going at full speed. Kindest regards, Niall O Broin
Re: question about amcheck
On Wed, Nov 13, 2002 at 07:53:36PM +0100, [EMAIL PROTECTED] wrote:
I have compiled and configured my linux box with amanda 2.4.3. I need to backup
my data on big harddisks. When I test amanda with amcheck 13nov2002 i prompts:
insert tape into slot 1 and press return and this message is repeated if I press
return again and again.
A number of points:
1) If you're going to include your ENTIRE config file, then clean out the
cruft. You appended app. 550 lines of config file for Gawd's sake !
2) Please use a mail client which wraps text at a sensible line length.
3) You haven't told amanda where to backup. Your config file says, deep in
the middle of all the cruft:
tapedev no-such-device# the no-rewind tape device to be used
and amanda is valiantly trying to read a tape in a device call
no-such-device which probably doesn't exist. Your holding disk entry
specifies /DS2/dumps/amanda so I'm going to wildly assume that this is
where you want to store your backups. (this is not what the holding disk is
for BTW - RTFM for details). If this the case, then you'll need a tapedev
entry like
tapedev file:/DS2/dumps/amanda
You have this
labelstr ^13nov2002[0-9][0-9]*$ # label constraint regex: all tapes must match
for your label regex but why would you need 13nov2002 in every tape name ?
Again, you need to RTFM about this. Let's try to use the labelstr from the
example amanda.conf
labelstr ^DailySet1[0-9][0-9]*$
which would mean that you'd have tapes labelled DailySet101, DailySet102,
DailySet103 etc. Yes, you want to backup to disk, but when backing up to
disk, amanda uses a directory as a virtual tape.
So change your config as I've outlined and do the following (as user amanda,
and assuming your config is called Daily)
mkdir /DS2/dumps/amanda/data
amlabel Daily DailySet101
Assuming everything else in your config and setup is OK, amcheck should now
work. Now that if you RTFM for amcheck you will see that you can split up
the checks by using -c, -l and -t to check the various things - this is a
good idea when you're beginning.
Assuming that everything now works, you're not yet out of the woods. You've
only got one tape which is not much use.
So, you need to prepare a number of virtual tapes. What I, and others, do
is to prepare a quantity of tapes and symlink tapedev to the appropriate
one before amdump runs. So in your case, I'd change the config file again
to have
tapedev file:/DS2/dumps/amanda/tape
and then remove /DS2/dumps/amanda/* completely. Then use something like the
following to prepare your tapes: (Bourne shell syntax - if you don't use a
Bourne compatible shell, either a) figure it out or b) Run such a shell
and then do these commands)
for tape in 01 02 03 04 05 06 07 08 09
do
mkdir /DS2/dumps/amanda/DailySet1$tape
mkdir /DS2/dumps/amanda/DailySet1$tape/data
ln -s /DS2/dumps/amanda/DailySet1$tape /DS2/dumps/amanda/tape
amlabel TIZ DailySet1$tape
rm /DS2/dumps/amanda/tape
done
Now before your daily amdump run, you should do
rm /DS2/dumps/amanda/tape
ln -s /DS2/dumps/amanda/$TAPE /DS2/dumps/amanda/tape
Where you set $tape to the appropriate value. I use this in my daily dump
script (my daily config is called TIZ)
TAPE=`tail -1 /etc/amanda/TIZ/tapelist|awk '{print $2}'`
Thanks for help
You're welcome, but I think you're going to have to RTFM some more.
Regards,
Niall O Broin
Re: incomplete backups
On Wed, Nov 13, 2002 at 01:20:32PM -0500, Jon LaBadie wrote: Nothing to do with the system at all. It's just a little something added into the script which runs amanda each day - I presume at least 95% of amanda users run amanda from a cron driven script. A couple of lines in that /those (if you run more than one config) script(s) and bingo - indices safely squirelled away. I think Gene was refering to his backup on Linux missing things like amanda.conf, tapelist, disklist, ... He attributed this to tar honoring locks on these files during backup. I noted to him that my backups on Solaris do not seem to lack these files. I'm not sure that was it at all. There's no reason on earth for amanda to issue any kind of a lock on amanda.conf, tapelist, disklist during backup and anyway I don't think tar even looks at such locks - like Frank, I sometimes get tar errors against index files - no biggie. But certainly on my Linux setup, I checked with my amfind script and amanda.conf is backed up. (In fact I've a bazillion amanda.conf files in various place - must go on a cleanup). I thought Gene was bemoaning the fact that he didn't have copies of those files somewhere handy other than on his tapes (so that he first had to restore them from there before he could find out what was on what tape). But I'm sure Gene will clarify matters for us. Regards, Niall O Broin
Re: Compress the data
On Wed, Nov 13, 2002 at 01:55:33PM -0800, Mahidhar Kada wrote: How do I configure Amanda to compress the data which is backed up, so that the tape can fit with more data. RTFM. In general, using software compression, where either the backup client or server does the compression, will maximise the amount of data you get on tape. If using software compression, you must ensure that hardware compression is off on your drive. Regards, Niall O Broin
Re: smbclient question
On Wed, Nov 06, 2002 at 10:38:43AM -0500, [EMAIL PROTECTED] wrote: I have setup amanda to archive my win 2k clients, and all appeared to be well in testing. I have a client that has a 63G file, that smbclient is only showing as 170M. This is what amanda archived and if I view the file with smbclient the size of the file is reported as 170M. I think this is a restriction of smbfs which has a maximum file size of 2GB. You'll probably find that 170M = actual file size MOD 2GB. Regards, Niall O Broin
Re: smbclient question
On Wed, Nov 06, 2002 at 11:28:35AM -0500, [EMAIL PROTECTED] wrote: The interesting thing is that I restored it an it was only 170M locally on my archive host. Are you saying when I move it over it will expand? I Of course not - what did I say that makes you think that ? really do not see how that is possible, but I am no smb expert.. That's not possible, even for smb experts. A snake oil salesman might be able to help you. Regards, Niall O Broin
Re: Performance degrading over time?
On Mon, Nov 04, 2002 at 03:55:13PM +0100, Patrick M. Hausen wrote: But I need a holding disk at least as large as my largest FS to dump? So if I have one 170 GB RAID I need one 170 GB holding disk? No - you need a holding disk preferrably as big as your TWO largest disklist entries, which may be = your largest FS. From Using Amanda ] Ideally, there should be enough holding disk space for the two largest ] backup images simultaneously, so one image can be coming into the holding ] disk while the other is being written to tape. If that is not practical, any ] amount that holds at least a few of the smaller images helps. The customer won't like that ;-) Customers rarely like anything which costs them money :-) Or is this what the chunksize parameter is for - taper will start when the first chunk is completely written to the holding disk? No - chunksize merely specifies how big are the chunks the backup is split into on disk. Really only needed on older OS which have a relatively small limit (often 2GB) to the size of a single file. In this case I'm sure a holding disk will speed up things quite a bit even in my pathological case of only one big FS. No - a holding disk will only help you at all if it's at least as big as your two smallest disklist entries. Kindest regards, Niall O Broin
Re: New file tape type for backup to disk.
On Tue, Oct 29, 2002 at 10:29:32AM -0500, Brian Kennedy wrote: I've got the backups running smoothly, some scripts to change tapes (move symlinks) but I cant get a recover to work. OK - a little background on my setup so that later becomes clear - I have a partition /backup on a server called backup. In /backup I have created my tapes and I backup to backup:/backup/tape where tape is a symlink to the appropriate tape (I've a notion that a more 'elegant' olsution to this whole kludge would be a hacked mtx of some sort, but I digress) amrecover conf .. select some files and extract.. Restore files into.. continue? Y Load tape ... continue? Y and then: amrecover: error reading tape: Connection reset by peer extract_list - child returned non-zero status: 1 any clues as to what is wrong? I think amanda doesn't know where it should be looking - below an extract from our local operations manual: At this point using physical tapes, it's time to load the desired tape. But because we are using amanda's ability to use virtual tapes on disk, rather than loading a tape we must tell amanda where the tape is. The 't' option at this prompt allows us to do so Continue [?/Y/n/t]? t New tape device [?]: backup:file:/backup/TIZdaily04 Here we've told amanda where the tape is. The syntax is as in the example above backup:file:/backup/ followed by the tape name. Now here's the bit that ISN'T in our operations manual - you can't tell amanda to use a tape called file:/backup/store because then it tries to use a host called file. I'm presuming your store is a symlink analogous to my tape ? I start amrecover like this (my configuration is TIZ) amrecover TIZ -s backup -t backup and I don't even bother specifying a tape device. I have moaned about this already, and the problem is essentially the overloading of : . As it's an eminently workaroundable :-) problem, I don't suppose there's a huge incentive to fix it. Kindest regards, Niall O Broin
Re: New amanda user
On Fri, Oct 25, 2002 at 02:20:43PM -0600, Christopher C.J. Keist wrote: I have mtx working and it looks like it sees the juke box and can talk That's a good start. with it. I'm not looking at amanda to do regular backups but rather just periodic archives of certain files/directories. Well, amanda's real strength is its ability to manage the backup of a large number of clients to one server which may have a (relatively) small tape drive, and to keep indices of everything backed up to aid in restores. So it may well be that if you want to periodically archive certain projects to optical disks, some other tool like rsync with some homebrewed scripts may do the job for you. If OTOH you think Amanda IS the tool for you then version 2.4.3 does support backup to disk. It treats disk directories as tapes and has its own version of mt and AFAIR dd which support this world view (standard mt doesn't do anything sensible when asked to rewind a directory). I do daily backups with amanda to large IDE drives and I have created a large number of directories (~ tapes) which I have labelled with amlabel. My daily backup script looks in the tapelist and symlinks the appropriate directory to the directory which amanda is configured to use for backup. Now that I write this I suspect that I could have told amanda I had a changer and written a customised mtx script which would have done about the same thing. However, I suspect it would have been more work for no net gain. In your case, you could use a similar approach - you would tell amanda to use e.g. /optical1 as its backup device. Your backup script would need to look at tapelist to see what tape (disk) amanda needed next, and then it would need to use mtx to load the appropriate piece of media into the appropriate drive and mount it. I don't think that amanda will be able to make use of the 10 drives you have available, and there might thus be issues with other uses of the device i.e. if you have amanda configured to use /optical1 and at the time of a backup, some other user is using that drive. However, I think suitable imaginative use of symlinks would get around that problem. So just looking to see if anyone has configured a optical juke box to work with Amanda. I haven't, so my 2c worth may not even be worth 2c (in fact my 2c as I write are worth 1.95c to you :-) ) Kindest regards, Niall O Broin
Re: Restore from tape (not using Amanda) syntax
On Wed, Sep 25, 2002 at 09:38:55PM +1000, Gordon Cormack wrote: I'm trying to dump from an Amanda tape onto the hard-drive by not using Amanda. This is probably a you don't know how to use tar properly question but: #mt -f /dev/rmt/0n status Vendor 'SONY' Product 'SDX-300C ' tape drive: sense key(0x6)= Unit Attention residual= 0 retries= 0 file no= 0 block no= 0 #mt -f /dev/rmt/0n rewind I'm obviously doing something wrong here cause the command: #dd if=/dev/rmt/0n bs=32k skip=1 | /usr/local/bin/tar -cv -f /export/output.tar /usr/local/bin/tar: Cowardly refusing to create an empty archive You're mixing your metaphors, I'm afraid. Either of the following should work 1) dd if=/dev/rmt/0n bs=32k skip=1 of=/export/output.tar This will create the tar file for you (hopefully you haven't compressed the tar file on tape - if you have, change to of=/export/output.tar.gz) and then you can use tar [tx]f (or [tx]zf if compressed) to list|extract the contents. 2) dd if=/dev/rmt/0n bs=32k skip=1 | tar xf - This will extract the tar file under the current directory for you. Use tar xzf if it was compressed, or tar tf (or tzf if compressed) if you just want to view the contents of the backup file. The command line you gave was a bit of a mix of both of the above and it was simply wrong. Kindest regards, Niall O Broin
Amanda's tape usage
On Fri, Sep 20, 2002 at 09:19:43PM -0400, Gene Heskett wrote: seen it work in practice here, nor do I need it, but my system is only 46gigs, which amanda cheerfully uses about half a 4gig tape per nightly run to back it all up on a 1 week dumpcycle. Gene - I've seen you mention this a few times before and I'm mildly curious. You've a 1 week dumpcycle, and I'll assume runspercycle is 7. You said that amanda uses about half a 4gig tape per nightly run which is 14 gig in a week. Now even assuming that you have miniscule level 1 dumps, are you getting such good compression from gzip that you get 46G (at least) of disk into 14G of tape ? Or is the 46G the amount of disk space you're backing, but it's not actually all used ? Mind you, I have to say that I have been very impressed with the usage amanda makes of tape - I'd say that this is one of its best features, esp. now that the cost of disk storage is now only about the same as the cost of tape to back it up. of the DDS2 is the price of the tapes, they are almost a non-issue at less than 50 bucks a ten pack on ebay. Even so - $5 per 4G tape is $1.25 a tape, which is IDE disk costs now, and this price per gig doesn't seem to vary hugely as you go to bigger capacities. I think I paid about 9 EURO (a few cents less than $9 at current rates) the last time I bought DDS 3 tapes (12 G). DDS4 (20 GB) cost 18 EUR, 40 GB DLT is 63 EURO, 60 GB ADR is 60 EUR. Bottom line is tape has become relatively expensive, and amanda can get very good mileage (byteage ?) from your tapes. Regards, Niall O Broin
Re: Tape DDS-3 values
On Fri, Sep 13, 2002 at 10:25:29AM -0400, Jon LaBadie wrote: Is it a real, or theoretical, problem? I.e. has anybody experienced bit errors in a gzip'ed document? For me the incidence is low enough that I don't feel a need to use bzip2 for that extra protection. The value of your data may lead to a different conclusion. I brought this up and it's not theoretical - I've had problems with gzipped tar files coming back off tapes. I definitely have problems with my tape drive but as mentioned, such errors in a tar file would only have caused problems with some files - with the gzipped tar file, you're done for once you hit an error. Regards, Niall O Broin *** Qmail-Scanner Envelope Details Begin *** X-Qmail-Scanner-Mail-From: [EMAIL PROTECTED] via maat.reeusda.gov X-Qmail-Scanner-Rcpt-To: [EMAIL PROTECTED] X-Qmail-Scanner: 1.12 (hbedv: 2.0.3/vdf=6.14.0.3 Clear:. Processed in 1.182701 secs) *** Qmail-Scanner Envelope Details End ***
Re: Dumpcycles vs Runs
On Sat, Sep 14, 2002 at 12:57:15PM -0400, Jason Greenberg wrote: If I have a dumpcycle of 7 days, with 7 runs per cylce, I understand that I will have a full backup every 7 days. So, if that's the case, here are my questions: 1) How many tapes are used for these 7 runs? 7? or just 1? 7 x runtapes (which is the number of tapes used per run) 2) does the tape change after every dumpcycle or every run? After every run - amanda is a backup solution for people who are serious about backing up their data, and that means that you don't put a number of backups one after the other on one tape. 3) I have 15 tapes, and I would like to use 1 tape per week, with 15 weeks of total history. What would the dumpcycle / runspercycle config be? You can't do that, or at least not with the above spec. You could have dumpcycle 7 days and runspercycle 1 but you probably don't want to do that :-) (because you'd only be backing up once per week) With 15 tapes, assuming that you backup on weekdays only, and have sufficient tape capacity, you'll get only three weeks history. Regards, Niall O Broin *** Qmail-Scanner Envelope Details Begin *** X-Qmail-Scanner-Mail-From: [EMAIL PROTECTED] via maat.reeusda.gov X-Qmail-Scanner-Rcpt-To: [EMAIL PROTECTED] X-Qmail-Scanner: 1.12 (hbedv: 2.0.3/vdf=6.14.0.3 Clear:. Processed in 1.257272 secs) *** Qmail-Scanner Envelope Details End ***
Re: Can Amanda put its index on a tape?
On Wed, Sep 18, 2002 at 02:29:58PM -0800, Anthony Valentine wrote: Question: Can Amanda be setup so that it writes a copy of it's indexes to the beginning of each tape? I realize that I can add the filesystem that holds the indexes to my disklist, but then I won't know where on the tape it is. If it were always the first thing after the label I could easily find it in an emergency. That'd be a nice feature, but then a lot of things about time travel might be nice :-) I don't think it can be done in the general case, because taping will usually have started before the indices would be updated. Of course in the case of an emergency such that you need to get the indices from tape, I suppose having on on the last tape a copy of the indices correct as to the previous tape would be a whole lot better than nothing. What I, and I think many others, do is to copy the indices to a different disk after each daily Amanda run which gives you a little insurance against a disk crash. As I write this it occurs to me that the indices are not so big that it would be unreasonable to also ship them offsite somewhere also, presuming that you have access to such a place. You could even simply email them to youself at a special account on a non-company mailserver. Kindest regards, Niall O Broin
Re: Tape DDS-3 values
On Thu, Sep 12, 2002 at 05:13:53PM +0300, Conny Gyllendahl wrote: Also, tar+gnuzip gives you alot better compression than the internal hardware of the drive, at least from what I've read on this list. This is true but as a friend pointed out to me recently when I was having some tape reading problems - if you get some bit errors reading a tar file from a tape, most likely asll you will lose is the affected file. If OTOH you get bit errors reading a gzipped tar file, you most likely will not be able to read anything after the bit errors. Just something to be aware of. Kindest regards, Niall O Broin
Re: tapeless and amverify
On Mon, Sep 09, 2002 at 09:04:33PM -0400, Jean-Louis Martineau wrote:
amverify use ammt and amdd which understand the file: driver.
They must be installed at the same place as your amverify program.
'make install' do it.
If amverify dont' find them, it will use the system mt and dd program
which dosn't understand the file: driver.
Jean-Louis - please accept my apologies for implying that you would leave
such a hole. I did read in passing somewhere about amdd and ammt but I had
forgotten about them. When I started testing with the file: driver I was
using a copy of 2.4.3b4 which I built and installed in /usr/local and sure
enough there they were - /usr/local/sbin/am{mt,dd}. However, as I got
further in my testing I wanted to install 2.4.3b4 as an RPM so I built an
RPM. I used a modified version of the spec file from SuSE's delivery of
2.4.2p2 but unfortunately it wasn't modified enough - I forgot to add ammt
amd amdd into the files section of the spec file.
The result of this was that by the time I had started to use amverify I
didn't have ammt and amdd in amanda's path which led to me making a fool of
myself :-(
Before I make a fool of myself again, are there any other new programs built
between 2.42.p2 and 2.4.3b4 ?
Kindest regards,
Niall O Broin
Should I use a holding disk with the file: device ?
On Sun, Sep 08, 2002 at 07:30:34PM -0400, Jean-Louis Martineau wrote: I'm not using a holding disk at all - it doesn't seem to make sense when using the file: device. Perhaps I'm wrong there ? I don't know enough bout amanda's architecture. Perhaps it is a good idea to use a holding disk, even with the file: device, because of the ineraction between dumpers and tapers ? PARALLELISM Without holding disk, your dump will be done sequentially, with a holding disk, they will be done in parallel. OK - there's my reason to have a holding disk. But now I've a situation where the dumps are being done to disk (the holding disk) and then copied to tape (but with the file: device, actually disk) and then deleted. It would obviously be much more efficient to use the same partition for holding disk as for the file: device, then all that the taper process would have to do would be a simple mv, rather than a copy (of an often quite large file) followed by a delete. But perhaps this is rather too large a change to put into 2.4 ? Kindest regards, Niall O Broin
Re: tapeless and amverify
On Sun, Sep 08, 2002 at 07:54:29PM -0400, Jean-Louis Martineau wrote: Try this patch, tell me it works. in response to this request On Fri, Sep 06, 2002 at 03:26:56PM -0700, Adnan Olia wrote: Hi all, I have a question. Can you run amverify on a tapeless backup in Amanda. I have no tape drives or tape changes and I am backing up my machines completely on my amanda server. When I run: amverify DailySet1 I get: No tape changer... Not a character special device: file:/home/backup I've seen no response from Adnan but I'm on the same path myself i.e. starting to use the file: device. I applied the patch and it does get rid of the specific error message but then I get something like amanda@backup:/etc/amanda/TIZ amverify TIZ No tape changer... Tape device is file:/backup/a... Verify summary to [EMAIL PROTECTED] Defects file is /tmp/amanda/amverify.13907/defects amverify TIZ Tue Sep 10 01:30:30 CEST 2002 Using device file:/backup/a Waiting for device to go ready... and there it will wait, forever. Looking at the amverify script shows that it's not going to work with the file: device without a major rewrite. It uses mt extensively (in the above, it's waiting for an 'mt -f $DEVICE stat' to complete successfully. When $DEVICE is a directory, it will be left waiting :-( Later in amverify, it also waits for an 'mt -f $DEVICE rewind' to complete successfully which again will be a long wait when $DEVICE is a directory. And later, to read the tape label, it's using dd if=$DEVICE which again won't be a whole lot of use when $DEVICE is a directory. To actually read each backup file and pass it to the restore utility to see if it can create a catalog, it uses amrestore on $DEVICE - again, this isn't going to work when $DEVICE is a directory. I have to say that I'm more than a little concerned about this. I'm not so worried that amverify won't work, because I'd see the main use of amverify being to check a tape, and people probably don't want to do this quite so much when using disk (although amverify should work, or perhaps at the moment give an apropriate error message if the file: device is used) BUT I am worried about restoring from my backups. Have any of you who are using the file: device given it a good shakedown testing recovery of files or entire partitions ? Kindest regards, Niall O Broin
Re: Problems using the file: device AND Mixing Amanda versions
On Sat, Sep 07, 2002 at 08:45:59PM -0400, Jean-Louis Martineau wrote: amanda-2.4.2p2 and 2.4.3b4 are compatible. That's good to know - at least I won't have to upgrade amanda on all the clients. You must know that the file: driver is limited by the maximum file size of your filesystem. I know that and I did mention in the Problems using the file: device post that all the clients and servers are capable of handling files 2GB but I didn't mention it again in the Mixing Amanda versions post. I can't imagine how you can get the error [input: can't open:: No such file or directory]. Neither can I, hence the post :-) And I'm a little concerned that YOU can't imagine how I got it either. I now suspect that perhaps there was some clash between my new test server and the old server with two backups running at the same time. This was not intentional, and I deliberately changed the old backup's start time to avoid such an occurence, but perhaps I didn't leave enough margin. The fact that a subsequent run on the new server succeeded (when the old server was definitely NOT doing a backup) makes that explanation all the more likely. The subsequent run BTW created several files 2GB. Your other disks failed because moca:/ failed (dump to tape), you should use a larger holding disk. I'm not using a holding disk at all - it doesn't seem to make sense when using the file: device. Perhaps I'm wrong there ? I don't know enough bout amanda's architecture. Perhaps it is a good idea to use a holding disk, even with the file: device, because of the ineraction between dumpers and tapers ? Thanks for your patience on this one people - I'm quite comfortable with Amanda now, in no small part thanks to the help of this list when I was getting started, but I find myself with new questions now that I'm starting to use the file: device and I hope you can bear with me. Kindest regards, Niall O Broin
Backing up to disk
I want to start using a large IDE RAID in addition to my DAT drive with Amanda and I have a couple of questions about that. 1) Which version of Amanda should I best use ? I got 2.4.3b3 (I know b4 has been released) and 2.5.0 and I see that both of these have support for the file: backup device. So, which to use ? Is Amanda like Linux in that 2.4 is the release version, and 2.5 is the bleeding edge version ? In that case, I'll use 2.4.3b4 (and later as available) because I want stability in my backup software, thanks very much. 2) How does one restore ? I've RTFM in 2.4.3b3 and in the amanda man page I've read about usage of the file device - reasonably straightforward. In previous discussions on the list I've seen people suggesting that one scheme is to make a new directory in the backup directory each day named after the date and to symlink data to that for that day's backups - seems like a reasonable way of doing things. BUT how does the database / index files / amrecover deal with this ? Normally (using tapes) one uses amrecover, selects files for recovery, and by suitable manipulation of the index files, amrecover knows what tapes it needs to get back the files and prompts accordingly. How does this map into a configuration where the file driver is used ? Kindest regards, Niall O Broin
Re: To use software compression or not ?
On Sat, Jun 15, 2002 at 08:56:56PM -0400, Gene Heskett wrote: I think you missed the sequence there, I used the rewinding device descriptor st0 when I read the label out to a file, so when the Ah, there's the rub - that may be what you MEANT but what you posted was #!/bin/bash mt -f /dev/nst0 rewind # but don't rewind it here as you save the amanda label dd if=/dev/nst0 of=label-this-tape bs=512 count=1 which is just a little different :-) A bunch in fact, like being pregnant. But that script does work, its the one I used. I had to ssh into my office machine 30 miles up the road to get it. Now I'll have to re-check my handiwork, I hate that :-) And it begs the question - does this oft stated thing actually happen i.e. does a tape actually get locked into compressed mode ? Can anyone provide any documentation on this, one way or the other ? Regards, Niall
To use software compression or not ?
I'm trying to decide whether or not to use software compression in my amanda configuration. At the moment, I'm backing up to a DDS-3 and I've told Amanda that the tape capacity is 15GB to allow for hardware compression. Of course I don't really know what level of compression the drive is able to achieve with my mix of data. At the moment this is not an issue because the most data ever gets taped in one run is ~10GB but as I'm adding clients and disks this will increase and I'm concerned about maximising my use of the limited capacity of the DDS-3 drive. Obviously using software compression helps with this because Amanda knows exactly how big each compressed dump is and I can also tell her the truth about the tape capacity. However my big concern is restoring. I know that even with current processors gzipping several GB of data takes some time and the same applies in reverse - or does it ? Does it take significantly longer to extract one file from a gzipped tar file on a DDS-3 than it does to extract one file from an uncompressed tar file or can a reasonable CPU gunzip in a pipe as fast as the DDS-3 can deliver data ? And then of course there's the fact that I won't be restoring very often anyway, so the extra backup capacity obtained may be worth the price of slower restores. Anyway, my dilemma should be clear. I'd appreciate feedback from anyone else who's considered these issues. What did you decide, and why ? A second question that arises is the issue of existing tapes. I've read that once a DDS-3 tape has been written in hardware compressed mode, the tape is marked accordingly and will ever after be written with compression, no matter what the drive is told to do. I've also read that this mark can only be removed by using a magnetic tape eraser. Is this correct ? Regards, Niall O Broin
Re: To use software compression or not ?
There's been a little back and forth here but basically my concern was whether using software compression would slow restores. I just did a little test and it seems the converse is in fact the case, at least for my one test - not dreadfully scientific, I know. I created a tar file of my home directory on my home machine which came to 2.1G. I used gzip to compress it and got a 1.3G file. Then I put each file onto a DDS-2 tape (because that's the tape drive I have at home) using dd just as amanda does, and extracted one file from each tape with dd|tar x in the uncompressed case, and dd|gunzip|tar x in the compressed case. The restore took 59m22s elapsed time for the uncompressed file and 44m48s elapsed time for the compressed file. It would appear that there's some overhead there for gunzipping (because 59m22s x (1.3/2.1) is somewhat less than 44m48s) but in fact I don't believe there is. I also checked how long it took to dd the compressed file to the tape and it was near as damnit the exact same as the extract time. I put the difference down to the uncompressed tar file being written to the tape in compressed mode (I didn't force it, and I didn't check - I told you this wasn't a scientific test :-) ) Anyway, the bottom line is that it has convinced me and I'm going to switch to using client compression and turning hardware compression off on the tape drive. Regards, Niall O Broin
Re: To use software compression or not ?
On Sat, Jun 15, 2002 at 07:16:04PM -0400, Gene Heskett wrote: I think you missed the sequence there, I used the rewinding device descriptor st0 when I read the label out to a file, so when the Ah, there's the rub - that may be what you MEANT but what you posted was #!/bin/bash mt -f /dev/nst0 rewind # but don't rewind it here as you save the amanda label dd if=/dev/nst0 of=label-this-tape bs=512 count=1 which is just a little different :-) Regards, Niall
Re: Backing up PostgreSQL?
On Thu, Jun 13, 2002 at 10:29:33PM -0500, Kirk Strauser wrote: I would re-do the backup steps as 1) Make a snapshot 2) Use dump to back up that completely static filesystem image 3) Remove the snapshot This is NOT guaranteed to work - it may, if you're lucky. By doing this you're guaranteeing that the database files, no matter how active the database, are frozen in time via the snapshot. But the big issue that you're failing to address here is that any one point in time the database files are not internally consistent. The currently running database instance has a consistent view of the data because it has data in memory and on disk and of course it knows what's where. This is why there are programs such as pgdump etc. One means suggested to backup a database which can't be stopped is to have your database files on a mirrored disk pair (or pairs). Then to make a backup, you issue read locks on all the tables. Having got those locks, you break the disk mirror at an OS level, then relinquish the locks. Then you backup the broken part of the mirror which is not in use. I think you could do something similar with a snapshot supporting filesystem. You lock all the tables, then make a snapshot, then relinquish the locks. And you should then be able to backup a consistent view of the tables from the snapshot. The above, however, is entirely speculation and I haven't tried it. But even as speculation, it's more likely to work than simply backing up a snapshot. Bear in mind that the snapshot may often work but it won't always and your backup system should cover every eventuality. Regards, Niall O Broin
Re: why is tar running as root
On Thu, Jun 13, 2002 at 11:39:11AM -0400, Mohamed Lrhazi wrote: Although I configured with : --with-user=amnda2 --with-group=sys on both the client and the server, when run : su amanda2 -c amdump normal from the server machine, I see tar running on the client as user root, then gzip runs as user amanda2... any idea why? why doesnt tar run as amanda2 too? Tar must run as root in order to be able to backup all files with correct ownership details etc. Regards, Niall O Broin
Not enough space on tape
On a couple of occasions recently, when I added a new machine into the rotation, some incrementals didn't get done because I didn't have enough tape space. However, amanda doesn't even do these dumps and leave them in the holding disk. Is there any way of forcing that behaviour i.e. for some reason some night there isn't quite enough tape, do the backups anyway to the holding disk. I can then flush those if I so desire but probably I'd just leave them there as insurance against a failure of the disk concerned before the next tape backup (yes, I know I've a problem there in the event of a simultaneous failure of one of the not taped disks and the holding disk) One way that springs to mind is to lie about the tape capacity but it occurs to me that then amanda will feel free to use that much each night if it needs to. I'd rather that it knew of the capacity of my tape, but backed up as much as was necessary, leaving the surplus on the holding disk. I've a horrible suspicion that I'm asking for a glass of dry sweet wine here but perhaps someone can make sense of my requirements / desires. :-) Kindest regards, Niall O Broin
Re: Deleting a backup from Amanda's database
On Mon, May 27, 2002 at 11:31:57PM +0200, Bernhard R. Erdmann wrote: I just had a problem with a backup shown up by amverify - the second last filesystem wasn't dumped correctly and the last one not at all. I'd like to make amanda delete the records of those dumps from its database so I won't be offered them as choices if ever I need to restore a file from them. Is there any way of doing that ? I know I can remove a tape altogether (and hence all of its backups) with amrmtape, and amadmin delete wil remove all records of a particular disk from the database, but I can't see any way to remove the record of one disk's backup on one day. Is it possible ? Force a full dump using amadmin config force host. That doesn't do what I want - of course it makes a full backup of the affected filesystems but that's not what I want - I want to remove the record of the failed backup so that if at some future stage I look for a file which was in that backup, I won't be offered that backup as one of the recover possibilities (because it won't work). Regards, Niall O Broin
Deleting a backup from Amanda's database
I just had a problem with a backup shown up by amverify - the second last filesystem wasn't dumped correctly and the last one not at all. I'd like to make amanda delete the records of those dumps from its database so I won't be offered them as choices if ever I need to restore a file from them. Is there any way of doing that ? I know I can remove a tape altogether (and hence all of its backups) with amrmtape, and amadmin delete wil remove all records of a particular disk from the database, but I can't see any way to remove the record of one disk's backup on one day. Is it possible ? Kindest regards, Niall O Broin
Re: Reducing the amount of output from amverify
On Fri, May 24, 2002 at 03:23:02PM +0200, Ulrik Sandberg wrote: On Fri, 24 May 2002, C R Ritson wrote: I use (as a cron job) for both the backup and its following amverify run:- /usr/local/amanda/current/sbin/amdump ncl \ /usr/local/amanda/current/sbin/amverify ncl \ /tmp/amanda/amverify.debug 21 What happens if you eject and then forget to switch tapes? Like in: amdump amverify ; mt -f /dev/rmt/1n offline I think amverify prompts for a tape and then just sits there. This might affect the following amdump in a bad way. Processes already running and such. Indeed it does. I had a situation (public holiday) where the tape wasn't put in for the backup. So the dumps happened to the holding disk - no problems so far. But then amverify was run (from my backup script). It found no tape and just sat there. The next day I inserted a tape to run amflush. Of course amverify saw the tape but knew that it wasn't what it wanted so exited with an error and then the script ejected the tape. Then of course amflush failed because there was not tape in the drive when it went looking for it :-) I normally manage the office in question from 1000 km away - fortunately I was present when all this rigmarole happened, otherwise it might have been quite confusing. Next thing was to edit my script to do an 'mt status' and check the return code and only run amverify if it appears that there is a tape. Kindest regards, Niall O Broin
Re: amanda and large filesystems
On Wed, May 22, 2002 at 02:50:05PM -0400, Andre Gauthier wrote: Is there a limit to the size of a filesystem that Amanda can backup? if so what's the limit? Does Amanda support ext3? Amanda doesn't do the backup as such - dump or tar does that. So if dump or tar can handle your filesystem, amanda should be able to too. It's a question of underlying OS limits really. As to ext3, the same applies - if tar or dump can see it, amanda can back it up. Regards, Niall O Broin
Re: 24 gig Tape Settings
On Fri, May 17, 2002 at 04:08:39PM -0400, Brook Hurd wrote:
HP C5708A DDS-3 Data Cartridge, 24GB
Unfortunatly, I don't see it listed online. Anybody use this type
before, and if so, what are the correct settings?
I use a HP DDS-3 too and I came across this definition somewhere which I use
define tapetype HP-DDS3 {
comment Seagate STD224000N-SB Internal DDS-3 Drive
length 11550 mbytes
filemark 0 kbytes
speed 1075 kps
}
You must bear in mind that a DDS-3 tape holds 12GB and NOT 24 GB of data,
despite what the marketing scum might like you to think. They assume a
compression ratio of 2:1 which I've never heard of ANYONE achieving (apart
from tape drive manufacturers in their contrived tests).
From an informal survey on the Sun Managers mailing list, it seemed that the
compression ratios people achieve in real life vary between 1.2:1 and 1.6:1.
If you're saving already compressed data e.g. JPEG files, you'll get no
compression at all. If you intend to use amanda to compress, use the figures
above and make sure your drive doesn't try to compress (some variation on
the mt command - depends on your local system). If you're not going to
compress with amanda, you could up the length figure here. But by what ?
As you've been using an 8 GB drive (which I presume was a DDS-2 hence was
really a 4GB drive) you should have some good data to go on - what was the
most data Amanda ever put on a tape ? If it was for instance 5GB then you'd
be reasonably safe using length 15000 mbytes.
I hope I've shed some light on what can be a confusing subject.
Kindest regards,
Niall O Broin
Tapecycle question
What exactly does tapecycle do ? I have in my config dumpcycle 2 weeks # the number of days in the normal dump cycle runspercycle 10 # the number of amdump runs in dumpcycle days # (2 weeks * 5 amdump runs per week -- just weekdays) tapecycle 14 tapes # the number of tapes in rotation I have 22 tapes labelled and in my tapelist - last night's backup used #14 and asked for #15 for tonight. From what I've read before, amdump will use the last tape in the tapelist marked reuse (incidentally, every tape in my tapelist is marked reuse - is that correct ?) so what exactly does the tapecycle parameter do ? If there are less than tapecycle tape in the tapelist will it force amdump to ask for a new one ? A second (actually, third) before I lick the envelope :-) I've got a large holding disk so I may well change runspercycle to 14 and run amdump every day and then run amflush on Monday in order to have as many backups as possible (and people may sometimes do some work at weekends). If I do that and the amount of data in the two holding directories on Monday is small enough, is it OK to amflush ALL to one tape or am I better off flushing each day to a separate tape, which would give the same nett result as if the tapes had been put in the drive over the weekend ? Regards, Niall O Broin
Getting old mails ?
Is it just me, or have other people on the list been getting old mails again - I just got one from Michael Richard about problems with firewalls. I was sure I'd read it before and sure enough - date in header was May 5. Looking at the headers the problem (with this one anyway) appears to be on the server hongkong.com. Regards, Niall O Broin
Re: Tapecycle question
On Tue, May 14, 2002 at 11:37:51AM -0500, Dave Sherohman wrote: tapecycle parameter do ? If there are less than tapecycle tape in the tapelist will it force amdump to ask for a new one ? Yep. A tape must be at least tapecycle tapes old before amanda will agree to reuse it. So for you, with tapecycle set to 14, amanda will refuse to overwrite the last 14 tapes that have been used. Why then does my tapelist have reuse at the end of every single tape's entry ? I always amflush everything to a single tape. AFAIK, the only disadvantage to doing it that way is that, if that one tape goes bad, you're losing more backups. (And, similarly, if you do what you're talking about and your holding drive dies on Sunday night...) OK - both caveats understood. If there are people working they can change the tape. Otherwise, it's just some extra backups and if a tape or the holding disk goes bad, it's not a big loss. Thanks, Niall O Broin
Amanda dump level policy
How does Amanda decide what it's going to do, dump-level wise. From the POV of ease of restoration, level 0 every day is the easiest way to go. From the POV of minimising tape usage, bumping the dump level every day is the way to go. Between these two extremes, depending on your tape capacity and the capacity of the disks in your disklist, is the optimum strategy for amanda. I know that amanda tries to make sure that each disk in the disklist has as level 0 dump at least once in the cycle, and also that it will bump a dump of a disk to a higher level when it can save X MB (a configurable amount based on the bump variables), and also that it sometimes promotes a level 0 dump to earlier in the cycle when it has space. But exactly how does it decide this ? It would seem to me that amanda should use the tape as much as it can and promote as many dumps as necessary to do that but it doesn't seem to do so from what I've seen. Is there a minimum number of runs before amanda will repeat a level 0 for a disk ? Kindest regards, Niall O Broin
Re: Amanda dump level policy
On Mon, May 13, 2002 at 02:07:28PM -0400, Joshua Baker-LePain wrote: On Mon, 13 May 2002 at 6:55pm, Niall O Broin wrote It would seem to me that amanda should use the tape as much as it can and promote as many dumps as necessary to do that but it doesn't seem to do so from what I've seen. Is there a minimum number of runs before amanda will repeat a level 0 for a disk ? Amanda promotes and bumps dumps (within the parameters you set, e.g. dumpcycle, bumpsize, etc) with the goal of equalizing (not maximizing) tape usage. I.e., the goal is to have each night's run use the same amount of tape. So if I want to persuade amanda to use more tape I should shorten the dump cycle, correct ? But I have a DDS-3 with native capacity of 12 GB and no changer. What happens if in a dump cycle there's more to be backed up than can fit on the tape e.g. if a lot happens to change on a couple of filesystems and level 1s get large ? What falls off the edge of the world ? Regards, Niall O Broin
Re: problems with some failing backups
On Sun, May 05, 2002 at 01:36:16PM -0400, Michael Richardson wrote: Niall serv1 /boot lev 0 FAILED [Request to serv1 timed out.] serv1 / Niall lev 0 FAILED [Request to serv1 timed out.] Bingo. What is the firewall? A Linux box with 2.4.x (10, I think) and iptables (using SuSE's SuSEfirewall2 script). It's NOT a very powerful box. The firewall is a 233Mhz PII. The load on it is neglible. It has a 3Mb bridged ethernet ADSL in front of it which is pretty much busy all the time. I think my firewall is probably of that order of power, if not a bit less. Wouldn't have quite as much external traffic though. Niall amandad: dgram_recv: timeout after 10 seconds amandad: waiting for Niall ack: timeout, retrying amandad: dgram_recv: timeout after 10 Yeah... I get that as well. But not always. Wehn I've looked I've always found that. One possibility is that the state for the UDP connection is failing. I would expect to see something in the firewall logs on this, and I'd expect to see the 10080 packet on one side of the firewall and not on the other. That sounds reasonable - I haven't examined the firewall's logs (these machines are in a different country from me, and I don't at the moment have remote access to the firewall). I will test with turning off stateful inspection on the UDP stream and see what happens. If this is the case, then Amanda perhaps needs to do keepalives. Niall Like you, I've RTFM and STFW but to no avail. I didn't get to the Niall the tcpdump stage yet, mind you. Thank you for the reply. Thank you for letting me know that I'm not alone :-) As to Ulrik's suggestions, I already have etimeout set to 3000 and netusage set to 5. Niall
Re: problems with some failing backups
On Sat, May 04, 2002 at 08:02:58PM -0400, Michael Richardson wrote: I backup 7 local systems with Amanda. Three Linux boxes (1 Debian/i386, 1 RH/i386, 1 RH/Netwinder), and four NetBSD/i386 boxes. There is a NetBSD/ipf firewall between the backup server (NetBSD/i386) and some of the boxes. Some of the backups also occur over IPsec (yes, even though they are all local). Two boxes on the same wire as backup server (plus the server itself) work flawlessly. The IPsec connected ones work fine. The three behind the firewall fail frequently, but not 100% of the time. I setup backups for just those hosts, and watch with tcpdump. I've built with the appropriate port ranges, but I never seen firewall failures, yet I get failures. Speak to me brother ! I've been posting about a similar problem here but I've got no responses. Do you get messages like these in the report: serv1 /boot lev 0 FAILED [Request to serv1 timed out.] serv1 / lev 0 FAILED [Request to serv1 timed out.] My remote (to describe the machines on the other side of the firewall) backups fail nearly all the time. My boxes are all Linux with large / and small /boot partitions. Sometimes L0 backups of /boot work, and once or twice I got an L0 of / to work (of one client) but generally all that works is when I get L1 of /boot, which is of course tiny. Coincidentally, the machines that fail are all less than 300Mhz systems, (233Mhz, 350Mhz, 200Mhz), while the machines that work are 650Mhz+. The backup server itself, however is a K5-133 running NetBSD/i386, and a lot of SCSI spindles. (Yeah, it needs to be replaced) I've a different situation - my failing machines are 2 X 1.2 GHz and 1 x 250MHz. However, my firewall is quite a slow box - I can't reach it now to say exactly. I suspect that the firewall can't handle the load, although I have clients using NFS accessing servers through it. However, NFS as a protocol is good at error recovery so that's probably the answer. My impression is that the failures are because the backup time estimates take too long and the backup server gives up on them. One the clients, I don't see any errors in the /tmp/amanda output - it looks normal to me. At the end of amandad.debug on a failing client I see amandad: sending REP packet: Amanda 2.4 REP HANDLE 002-F8B30708 SEQ 1020382783 OPTIONS maxdumps=1; / 0 SIZE 6929200 /boot 0 SIZE 3600 amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, giving up! which is presumably related to the timeout in the mail reports. I've been through the documentation and the FAQs, and I've watched tcpdump's of the traffic going through... nothing obvious. Like you, I've RTFM and STFW but to no avail. I didn't get to the the tcpdump stage yet, mind you. Kindest regards, Niall O Broin
Re: Formatting output of amreport
On Fri, May 03, 2002 at 12:50:22PM -0400, Wayne Richards wrote: I seem to recall some messages regarding the format of amreport output but have been unable to locate that information. Could someone point me to the proper location? To a certain extent you can format the output by changing the field specifiers in amanda.conf. Regards, Niall O Broin
Timeout problems
I am trying to deploy Amanda to back up a small office with currently 3 Linux servers and 2 Linux desktops, which number will soon increase. We have a firewall with a fairly standard world/DMZ/internal setup and 2 of the servers (including the backup server) are on the DMZ - the other server and the desktops are on the internal LAN. I did have some problems getting the initial 2 clients working (the DMZ servers) but these were largely due to my ignorance and were resolved with the excellent help of this list. The disklist looks like this # internal LAN machines lpc1 / high-tar lpc1 /boot root-tar lpc2 / high-tar lpc2 /boot root-tar serv1 / high-tar serv1 /boot root-tar # DMZ machines pumori / high-tar pumori /boot root-tar moca / high-tar moca /boot root-tar lpc1/2 are the desktops. The /boot filesystems are small (3-4 M each) and the / filesystems vary from 2GB to 6GB, and I'm backing up to a DDS-3. However, when it came to adding the local LAN machines I had more problems. I had to reconfigure the firewall to allow amanda to work. Having done that, I ran amcheck -c TIZ and it reported 5 clients and no problems. However, amdump TIZ didn't work - for the internal machines I got errors like: lpc1 /boot lev 0 FAILED [Request to lpc1 timed out.] So I made another change to the firewall configuration (although amcheck -c TIZ reported no errors which I understand to mean that there are no communications problems between client and server) and in order to test without waiting for the next backup in the TIZ configuration I made a new configuration called test (yes, I know, it's really original) in which I only backed up /boot on the internal machines and it worked so I thought I was on the pig's back, so to speak. However, it now seems more like I'm under his backside (and you can guess where that leaves me). I modified the test configuration and lied about the tape size and forced everything to level 0. Net result - none of the internal disk were backed up. This is from the report: lpc1 /boot lev 0 FAILED [Request to lpc1 timed out.] lpc1 / lev 0 FAILED [Request to lpc1 timed out.] lpc2 /boot lev 0 FAILED [Request to lpc2 timed out.] lpc2 / lev 0 FAILED [Request to lpc2 timed out.] serv1 /boot lev 0 FAILED [Request to serv1 timed out.] serv1 / lev 0 FAILED [Request to serv1 timed out.] and when I look in the relevant amandad.2002042909XX.debug file I see: amandad: running service /usr/lib/amanda/sendsize amandad: sending REP packet: Amanda 2.4 REP HANDLE 000-20AB0708 SEQ 1020065913 OPTIONS maxdumps=1; / 0 SIZE 2795120 /boot 0 SIZE 3600 amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, retrying amandad: dgram_recv: timeout after 10 seconds amandad: waiting for ack: timeout, giving up! amandad: pid 24085 finish time Mon Apr 29 09:41:29 2002 [Ancillary question - the above is from lpc1 and I saw something similar on lpc2. However on serv1 there were no amandad.2002042909XX.debug files - only an amandad.debug file - any ideas as to why that client's not putting timestamps in the debug filenames ?] My read of this is that the amandad process, having run sendsize, cannot contact the server to report the results. But why ? I suspect the dead hand of the firewall (iptables on Linux 2.4.x) but yet I cannot be certain, because the behaviour is NOT consistent - I have had ONE run where I edited the disklist so that I backed up only /boot on lpc1, moca and pumori, and / and /boot on lpc2. This worked, including a level 0 of lpc2:/. Also, if I edit the disklist so I only backup /boot on each client and then force them all, the backup works without fail - I've tried this numerous times. Any suggestions would be very welcome as I'm at my wits' end on this one. I'm sorry to have been so longwinded but I don't think I could have explained it without giving the information I have. Kindest regards, Niall O Broin
Finding backed up files
I asked a while ago about locating a file backed up by amanda and I was pointed to the index directory and sure enough in indexdir / client / disk you find a bunch of gzipped index files named date_level.gz which is fine. However, how do I map them to backups and tapes ? I want to try to write some kind of a frontend so that I when some poor luser says I've lost VeryImportantDocument.pdf I can type something like amanda_locate VeryImportantDocument.pdf and I'll get a listing of any backups I should have of that file, where they were from, when they were made, and what tape they're on so that I can insert the right tape and get the file back with amrecover. amrecover on its own doesn't quite do the trick because it only gives me an FTP like browser, so I can't e.g. find a file without knowing what directory it was in. Regards, Niall O Broin
Re: Over large level 1 dumps
On Wed, Apr 24, 2002 at 03:56:18PM -0700, Eric Zylstra wrote: Correct. Last night, all your filesystems thought they'd never had a level 0 dump (since none had been recorded), so level 1 included everything. As each filesystem does (and records) a level 0 dump, it will start doing incremental level 1s. After a full dumpcycle, things should be behaving as you expect. But the man page says the default value for record is yes. What is up? Default value may well be yes. But when you have no in your global type . . I didn't put that no there BTW - it was in the example configuration file with my amanda installation. And things now seem to be getting on track. One server had level 1 last night and it was a smaller amount of data. The other server was scheduled for a level 0 but it died last night - heat exhaustion :-) Regards, Niall O Broin
Over large level 1 dumps
I asked about this the other day and it transpired that I didn't have record yes in my dumptype, so that explained that, or so I thought. However, I now do have record yes in my global dumptype but still level 1 dumps are of the full filesystems. amadmin TIZ find no gives me this list of backups Scanning /data/amandahold... date host disk lv tape or file file status 2002-04-16 moca / 0 TIZdaily01 3 OK 2002-04-18 moca / 1 TIZdaily02 3 OK 2002-04-20 moca / 1 TIZdaily03 3 OK 2002-04-23 moca / 1 TIZdaily04 3 OK 2002-04-24 moca / 1 TIZdaily05 3 OK 2002-04-16 moca /boot 0 TIZdaily01 1 OK 2002-04-18 moca /boot 1 TIZdaily02 4 OK 2002-04-20 moca /boot 1 TIZdaily03 4 OK 2002-04-23 moca /boot 0 TIZdaily04 4 OK 2002-04-24 moca /boot 0 TIZdaily05 4 OK 2002-04-16 pumori / 0 TIZdaily01 4 OK 2002-04-18 pumori / 0 TIZdaily02 1 OK 2002-04-20 pumori / 1 TIZdaily03 1 OK 2002-04-23 pumori / 1 TIZdaily04 1 OK 2002-04-24 pumori / 1 TIZdaily05 1 OK 2002-04-16 pumori /boot 0 TIZdaily01 2 OK 2002-04-18 pumori /boot 1 TIZdaily02 2 OK 2002-04-20 pumori /boot 0 TIZdaily03 2 OK 2002-04-23 pumori /boot 0 TIZdaily04 2 OK 2002-04-24 pumori /boot 1 TIZdaily05 2 OK moca:/etc/amandates is / 1 1019600282 /boot 0 1019600788 and pumori:/etc/amandates is / 1 1019604259 /boot 1 1019605208 Another thought just occured to me - as I wasn't recording, why did amanda ever do level 1 backups ? My tape is big enough to hold all of the filesystems (currently in disklist - I do want to add more) at level 0 so it can't have been a size issue. And is it possible for me to test this by doing more than one backup run in a day - isn't there an issue with amanda only recording certain things by date and not by time. If I had 'index no' in my dumptype would I get around that ? Regards, Niall O Broin
Over large level 1 dumps
I'm just starting using Amanda (hence the level of questions :-) ) and I currently am backing up 4 filesystems on 2 hosts though I want to increase this as soon as I'm happy with what's going on. I'm currently puzzled about the size of level 1 backups. Look at these two extracts from mail reports: Makalu TIZ AMANDA MAIL REPORT FOR April 16, 2002 DUMP SUMMARY: DUMPER STATSTAPER STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s -- - moca / 0 26593602659360 -- 13:063385.2 30:251457.5 moca /boot 04384 4384 --0:0014986.3 0:13 339.3 pumori / 0 40461764046176 -- 17:523773.5 54:451231.5 pumori /boot 03552 3552 --0:014089.4 0:05 713.0 Makalu TIZ AMANDA MAIL REPORT FOR April 23, 2002 DUMP SUMMARY: DUMPER STATSTAPER STATS HOSTNAME DISKL ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s -- - moca / 1 26712322671232 --8:435109.6 N/A N/A moca /boot 04384 4384 --0:0010109.8 N/A N/A pumori / 1 40479684047968 -- 15:514256.4 N/A N/A pumori /boot 03552 3552 --0:016062.2 N/A N/A (Note that there were intervening dumps, but 23 had both / filesystems at level 1 and I went back to 16 to get both at level 0. Why on earth are the backups at level 1 bigger than those at level 0 ? Note that these are both Linux boxes so a substantial chunk of / is the OS installation and associated files and there simply hasn't been that much changed on those boxes. I'm using GNU tar version 1.3.18 and with the recent discussion, I intend to update that today to 1.3.25 from the FSF development box. A second question, while I have your attention. Observe how the some of the figures in the report run together - is the layout of this controllable via a template ? The output is 73 characters so there could be a couple more spaces inserted and still stay 80 characters which I imagine is desired. Kindest regards, Niall O Broin
Re: What to do when tape is full?
On Tue, Apr 23, 2002 at 09:54:45AM -0500, Frank Smith wrote: Or leave the tape out of the drive and let it go to to the holding disk, then amflush whenever you have a full tapes worth. Will that work ? When I use amflush I get given a list of days on which amdump ran without a tpe for whatever reason. Then I must choose one of those days and that set of backups is flushed to the tape. To get two or more days onto one tape Amanda would have to support append :-) Regards, Niall O Broin
Re: Tape drives
On Mon, Apr 22, 2002 at 05:16:15PM +0200, Mozzi wrote: Hi all I was wondering whitch of these drives will have the best support under Linux and espetially with amanda STD2401LW-SDAT - 20 - 40Gb - Seagate Scorpion 68 pin - SCSI C5686A hp surestore DAT40i Internal Drive Either should work fine. Given my experience with DAT hardware, I'd buy whichever one you can get with the longest warranty, even if that entails paying a few denarii extra. Regards, Niall O Broin
Increasing number ot tapes used.
I'm not long using Amanda (haven't finished a full dumpcycle yet) and I've decided that I'd like to use more tapes than I initially decided. I have dumpcycle 2 weeks runspercycle 10 tapecycle 14 tapes and I'd like to bump tapecycle up to 20 tapes or even a couple more because: a) I have the tapes already :-) b) Even if my backup load expands to nearly my full capacity (tape size * runspercycle) with tapecycle = 2 * runspercycle I'll always have 2 full dumps available. Is there any problem with changing tapecycle now ? Regards, Niall O Broin
Small bug in amoverview
I found a little bug in amoverview. I curently have a couple of days dumps in the holding directory and when I ran amoverview I get: amanda@moca:~ amoverview TIZ bad date 20020418: in 20020418: found Amanda directory. bad date 20020420: in 20020420: found Amanda directory. date 04 04 04 04 04 host disk 16 17 18 19 20 A tiny bit of research showed that this was related to the output from amadmin find which beginss: amanda@moca:~ amadmin TIZ find Scanning /data/amandahold... 20020418: found Amanda directory. 20020420: found Amanda directory. date host disk lv tape or file file status 2002-04-16 moca / 0 TIZdaily01 3 OK 2002-04-18 moca / 1 /data/amandahold/20020418/moca._.1 0 OK amoverview parses the output of amadmin find and doesn't handle the found Amanda directory lines. The fix is so trivial that I'm just mentioning it inline here because that's actually smaller than the patch :-) After next if $date eq ; insert an extra line next if $host eq found; I don't know if that's the best solution - another solution is next if $date =~ /:$/; Anyway, I've patched my amoverview, and perhaps the maintainer of amoverview can either take my patch on board or do something else appropriate. Regards, Niall O Broin
Searching for backed up files.
Having just submitted a fix :-) I now don't feel so bad about looking for something bigger. It seems that the only way to recover files from a tape is either to use amrecover, which presents an FTP like interface to the database of backed up files, or amrestore, which extracts an entire backup image from a tape which you can of course pipe through whatever other program you like. Many commercial backup programs have the ability to browse the database for files matching some wildcard spec. and offer a list of files matching the spec. and then allow you to pick one or more of these files to be restored. Has anyone hacked together something like that for amanda ? Even being able to get a list of matching files would be useful to enable you to look in the right place with amrecover. Regards, Niall O Broin
Amanda user when using Gnu tar
What user should amanda run as when using tar as the dumping program ? I know that it should run as its own special user (usu. amanda) when using dump because then amanda can have full dump rights by making the user a member of the disk group which allows it to read the disks at a low level. But I don't this applies to Gnu tar which doesn't accesss the raw disk, so do I have to use root as the amanda user when using Gnu tar so that the resultant tar files can have all necessary user information in them (and also so that the dupmping program can access all files, regardless of permissions) Kindest regards, Niall O Broin
