quick help needed on changer problem
Hi, today, there have been problems with the tape drive again. Slot 7 could no longer be found in the slot list. I unloaded the module for the SCSI controller and sg and st, then turned off the device and turned it back on and reloaded the modules. Devices are reported as they should. But amtape cannot change slots: prometheus:/etc/cron.daily# cat /proc/scsi/scsi Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: ICP Model: Host Drive #00 Rev: Type: Direct-AccessANSI SCSI revision: 02 Host: scsi4 Channel: 00 Id: 00 Lun: 00 Vendor: EXABYTE Model: VXA 1x10 1U Rev: A102 Type: Medium Changer ANSI SCSI revision: 04 Host: scsi4 Channel: 00 Id: 01 Lun: 00 Vendor: EXABYTE Model: VXA-2Rev: 100E Type: Sequential-AccessANSI SCSI revision: 02 prometheus:/etc/cron.daily# su - backup [EMAIL PROTECTED]:~$ /usr/sbin/amtape condor slot 7 amtape: could not load slot /usr/lib/amanda/chg-zd-mtx:: line 605: [: 6:VolumeTag=: integer expression expected [EMAIL PROTECTED]:~$ /usr/sbin/amtape condor slot 1 amtape: could not load slot /usr/lib/amanda/chg-zd-mtx:: line 605: [: 6:VolumeTag=: integer expression expected [EMAIL PROTECTED]:~$ /usr/sbin/amtape condor slot next amtape: could not load slot /usr/lib/amanda/chg-zd-mtx:: line 605: [: 6:VolumeTag=: integer expression expected Last time I turned the tape off and back on, it helped. Is the above something that might be recovered from by restarting the server, or does it mean that the tape changer is now completely broken? The counting of SCSI devices seems to increase each time I unload and reload the modules, so it's now scsi4 and was scsi3 before. Is that normal? GH
Re: Amanda 2.5.0 upgrade weirdness
Brad, Upgrade to 2.5.0p2 The 'amtape update' was missing from 2.5.0 and barcode will not work with this release. Jean-Louis Brad Willson wrote: Hi, More information from the log chg-zd-mtx.20060515215442.debug chg-zd-mtx: debug 1 pid 3280 ruid 34 euid 34: start at Mon May 15 21:54:42 2006 21:54:42 Using config file /etc/amanda/DailySet1/changer.conf 21:54:42 Arg info: $# = 2 $0 = /usr/lib/amanda/chg-zd-mtx $1 = -search $2 = B003 21:54:42 Running: /usr/sbin/mtx status 21:54:42 Exit code: 0 Stdout: Storage Changer /dev/sg0:1 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Full (Storage Element 3 Loaded):VolumeTag = B003 Storage Element 1:Full :VolumeTag=B001 Storage Element 2:Full :VolumeTag=B002 Storage Element 3:Empty:VolumeTag= Storage Element 4:Full :VolumeTag=B004 Storage Element 5:Full :VolumeTag=B005 Storage Element 6:Full :VolumeTag=B006 Storage Element 7:Full :VolumeTag=B007 Storage Element 8:Full :VolumeTag=B008 Storage Element 9:Full :VolumeTag=B009 Storage Element 10:Full :VolumeTag=B010 21:54:42 SLOTLIST - lastslot set to 9 21:54:42 Config info: firstslot = 1 lastslot = 9 cleanslot = 10 cleancycle = 120 offline_before_unload = 0 unloadpause = 0 autoclean = 0 autocleancount = 99 havereader = 1 driveslot = 0 poll_drive_ready = 30 initial_poll_delay = 0 max_drive_wait = 360 21:54:42 SEARCH - Hunting for label B003 - !!! label B003 not found in /etc/amanda/DailySet1/changer-barcodes !!! - Remove /etc/amanda/DailySet1/changer-barcodes and run /usr/sbin/amtape DailySet1 update 21:54:42 Exit (2) - none B003: label B003 not found in /etc/amanda/DailySet1/changer-barcodes chg-zd-mtx: pid 3411 finish time Mon May 15 21:54:42 2006 So I followed instructions, deleting /etc/amanda/DailySet1/changer-barcodes, then tried running /usr/sbin/amtape DailySet1 update which informed me there was no such option update. The amtape man page is apparently out of sync with the current release. Finding no information about the changer-barcodes file format, I copied the current tapelist to changer-barcodes which stopped the complaints about Hunting for label X. amflush DailySet1 A and ALL both yield the same effect; the changer cycles through the tapes but never stops to write anything. Finally the amflush report header reads as follows: *** A TAPE ERROR OCCURRED: [No writable valid tape found]. Some dumps may have been left in the holding disk. Run amflush again to flush them to tape. The next 10 tapes Amanda expects to use are: B003, B004, B005, B006, B007, B008, B009, B010, C001, C002. I've tweaked the changer.conf file to adjust the cleaning slot to -1 and will be giving that a try. Everything that should be set to UID/GID backup is set so, the tapes are functional, there have been no issues with the previous release. Any ideas now?
Re: quick help needed on changer problem
listrcv wrote: [EMAIL PROTECTED]:~$ /usr/sbin/amtape condor slot 7 amtape: could not load slot /usr/lib/amanda/chg-zd-mtx:: line 605: [: 6:VolumeTag=: integer expression expected Could you do a `mtx status' and send us the output? Alex -- Alexander Jolk / BUF Compagnie tel +33-1 42 68 18 28 / fax +33-1 42 68 18 29
Re: Amanda 2.5.0 upgrade weirdness
Jean-Louis, Thank you! Your answer hit the mark! Brad Jean-Louis Martineau wrote: Brad, Upgrade to 2.5.0p2 The 'amtape update' was missing from 2.5.0 and barcode will not work with this release. Jean-Louis Brad Willson wrote: Hi, More information from the log chg-zd-mtx.20060515215442.debug chg-zd-mtx: debug 1 pid 3280 ruid 34 euid 34: start at Mon May 15 21:54:42 2006 21:54:42 Using config file /etc/amanda/DailySet1/changer.conf 21:54:42 Arg info: $# = 2 $0 = /usr/lib/amanda/chg-zd-mtx $1 = -search $2 = B003 21:54:42 Running: /usr/sbin/mtx status 21:54:42 Exit code: 0 Stdout: Storage Changer /dev/sg0:1 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Full (Storage Element 3 Loaded):VolumeTag = B003 Storage Element 1:Full :VolumeTag=B001 Storage Element 2:Full :VolumeTag=B002 Storage Element 3:Empty:VolumeTag= Storage Element 4:Full :VolumeTag=B004 Storage Element 5:Full :VolumeTag=B005 Storage Element 6:Full :VolumeTag=B006 Storage Element 7:Full :VolumeTag=B007 Storage Element 8:Full :VolumeTag=B008 Storage Element 9:Full :VolumeTag=B009 Storage Element 10:Full :VolumeTag=B010 21:54:42 SLOTLIST - lastslot set to 9 21:54:42 Config info: firstslot = 1 lastslot = 9 cleanslot = 10 cleancycle = 120 offline_before_unload = 0 unloadpause = 0 autoclean = 0 autocleancount = 99 havereader = 1 driveslot = 0 poll_drive_ready = 30 initial_poll_delay = 0 max_drive_wait = 360 21:54:42 SEARCH - Hunting for label B003 - !!! label B003 not found in /etc/amanda/DailySet1/changer-barcodes !!! - Remove /etc/amanda/DailySet1/changer-barcodes and run /usr/sbin/amtape DailySet1 update 21:54:42 Exit (2) - none B003: label B003 not found in /etc/amanda/DailySet1/changer-barcodes chg-zd-mtx: pid 3411 finish time Mon May 15 21:54:42 2006 So I followed instructions, deleting /etc/amanda/DailySet1/changer-barcodes, then tried running /usr/sbin/amtape DailySet1 update which informed me there was no such option update. The amtape man page is apparently out of sync with the current release. Finding no information about the changer-barcodes file format, I copied the current tapelist to changer-barcodes which stopped the complaints about Hunting for label X. amflush DailySet1 A and ALL both yield the same effect; the changer cycles through the tapes but never stops to write anything. Finally the amflush report header reads as follows: *** A TAPE ERROR OCCURRED: [No writable valid tape found]. Some dumps may have been left in the holding disk. Run amflush again to flush them to tape. The next 10 tapes Amanda expects to use are: B003, B004, B005, B006, B007, B008, B009, B010, C001, C002. I've tweaked the changer.conf file to adjust the cleaning slot to -1 and will be giving that a try. Everything that should be set to UID/GID backup is set so, the tapes are functional, there have been no issues with the previous release. Any ideas now? -- Brad Willson, Sr. Computer Specialist UW GeneTests, UW Box: 358735 EM: [EMAIL PROTECTED] W: 206.221.4674, C: 425.891.2732 http://www.genetests.org
Re: quick help needed on changer problem
On Tue, May 16, 2006 at 12:09:29PM +0200, listrcv wrote: ... The counting of SCSI devices seems to increase each time I unload and reload the modules, so it's now scsi4 and was scsi3 before. Is that normal? I'm not familiar enough with Linux to know the correct commands. But in some other OSs devices can be looked for as look for new, or start over again and look for everything. If your system isn't doing a startoveragain, then some small change to your tape could cause it to appear as a new device and get a different name. It has been a while, but ISTR that Fedora has a way of assigning a UUID (Unique U??? ID) to devices, at least to USB devices and I think SCSI as well. Once assigned a table could be constructed of the UUID's and device names that you choose. Then whenever the UUID was detected, regardless of the /dev device automatically assigned, the names you chose were also created. When I used it, it was for several USB external disks that moved around. But whenever any of them were connected, I wanted it mounted in the same place. If your moving scsi problem persists, you might look into the UUID. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda on Mac OS X
Good to know, Michael, I am still using the xtar trick, didn't even check, , and amanda works on Intel chipped Mac just fine. Chen Hello Karel, Hello list, FYI: I'm happily running amanda-2.5.0 on Mac OS X 10.4.6. No compile problems whatsoever and as a special treat, Apple has updated their UNIX utilities in 10.4 to properly work with HFS+ and resource forks. So no hfstar/xtar is needed any more. I've tested this quite a bit and found it to work nicely, although I haven't tested a full system recovery. An update to https://webserver.brandeis.edu/pages/view/Bio/AmandaMacOSXCompileNotes or http://www.amanda.org/docs/systemnotes.html#id2526824 to reflect this change might be nice. Since Apple has retired init and xinetd and replaced them by launchd, I've written a proper LaunchDaemon-File for it. It's attached to this mail. I also have small scripts enableamanda and disableamanda to switch it on and off at runtime using launchctl using a command like: echo load -w /System/Library/LaunchDaemons/org.amanda.amandad.plist | launchctl and echo unload -w /System/Library/LaunchDaemons/org.amanda.amandad.plist | launchctl Thanks and keep up the great work! === Yu Chen Howard Hughes Medical Institute Chemistry Building, Rm 182 University of Maryland at Baltimore County 1000 Hilltop Circle Baltimore, MD 21250 phone: (410)455-6347 (primary) (410)455-2718 (secondary) fax:(410)455-1174 email: [EMAIL PROTECTED] ===
Re: quick help needed on changer problem
Alexander Jolk wrote: listrcv wrote: [EMAIL PROTECTED]:~$ /usr/sbin/amtape condor slot 7 amtape: could not load slot /usr/lib/amanda/chg-zd-mtx:: line 605: [: 6:VolumeTag=: integer expression expected Could you do a `mtx status' and send us the output? Ah, mtx was the program whose name I couldn't remember! prometheus:/home/hwilmer# mtx -f /dev/sg1 status Storage Changer /dev/sg1:1 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Full (Storage Element 7 Loaded):VolumeTag = Storage Element 1:Full :VolumeTag= Storage Element 2:Full :VolumeTag= Storage Element 3:Full :VolumeTag= Storage Element 4:Full :VolumeTag= Storage Element 5:Full :VolumeTag= Storage Element 6:Full :VolumeTag= Storage Element 7:Empty:VolumeTag= Storage Element 8:Full :VolumeTag= Storage Element 9:Full :VolumeTag= Storage Element 10:Full :VolumeTag= prometheus:/home/hwilmer# Slot 7 can actually be empty because the tape is in the drive. I made the changer load the tape from slot 7 into the drive via the control panel on the device. Then I started amflush, and to my surprise, it unloaded a tape and loaded one, and it's now writing to the tape. There's a barcode reader or so in the device, but I don't use it. I think the VolumeTag would say what the bar code is if it were used. I'm still waiting for the vendors support to call back to get the device fixed. They already managed to piss me off. Avoid buying exabyte or other stuff from Fujitsu/Siemens. The laptops they make are slow, the tape changer fails after 2 years, and their support sucks. GH
windows vshadow
Has anyone used a utility from M$ called vshadow? It is a .exe that comes in a free SDK called VSS. As far as I can tell it seems to be an attempt to do the equivalent of snapshots so that backups can be done to otherwise uncopyable files. Sound like a familiar problem? If I understand it correctly, it is a bit wierd in that the snapshot only exists while vshadow program is running. So you have to provide it with a script to execute on the snapshot, when that ends, so does the snapshot. I stumbled across a blog entry that provided a script using vshadow to enable the copying of a single file. Thus far it has not worked for me. Hopefully I type the URL correctly. http://blogs.msdn.com/adioltean/archive/2005/01/05/346793.aspx The vshadow.exe is only available for WinXP and 2003 server. So it is not a general solution, but maybe a start? -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: quick help needed on changer problem
Jon LaBadie wrote: On Tue, May 16, 2006 at 12:09:29PM +0200, listrcv wrote: The counting of SCSI devices seems to increase each time I unload and reload the modules, so it's now scsi4 and was scsi3 before. Is that normal? I'm not familiar enough with Linux to know the correct commands. But in some other OSs devices can be looked for as look for new, or start over again and look for everything. There's some way to make the OS looking for SCSI devices like 'cat scsi-add-single-device ... /proc/scsi/scsi' or so. But I didn't want to use that. If your system isn't doing a startoveragain, then some small change to your tape could cause it to appear as a new device and get a different name. As far as I get it, the available SCSI hosts are numbered starting by 0. In case there are several SCSI hosts (like several SCSI controllers or multi-channel controllers or SATA devices that are considered as SCSI besides real SCSI), each host gets another number, counting upwards from 0. Since only the modules were unloaded and reloaded and the physical hosts not changed, I would expect it to get the same number it had before. The total number of hosts was in no way increased. It looks pretty weird in the syslog: May 16 11:52:58 prometheus kernel: st: Unloaded. May 16 11:54:56 prometheus kernel: PCI: Enabling device :02:02.0 (0016 - 0017) May 16 11:54:56 prometheus kernel: scsi3 : Adaptec AIC79XX PCI-X SCSI HBA DRIVER, Rev 1.3.11 May 16 11:54:56 prometheus kernel: Adaptec 29320A Ultra320 SCSI adapter May 16 11:54:56 prometheus kernel: aic7901: Ultra320 Wide Channel A, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs May 16 11:54:56 prometheus kernel: May 16 11:56:46 prometheus kernel: (scsi3:A:0): 80.000MB/s transfers (40.000MHz, 16bit) May 16 11:56:46 prometheus kernel: (scsi3:A:1): 80.000MB/s transfers (40.000MHz, 16bit) May 16 11:56:49 prometheus kernel: Vendor: EXABYTE Model: VXA 1x10 1U Rev: A102 May 16 11:56:49 prometheus kernel: Type: Medium Changer ANSI SCSI revision: 04 May 16 11:56:49 prometheus kernel: Vendor: EXABYTE Model: VXA-2 Rev: 100E May 16 11:56:49 prometheus kernel: Type: Sequential-Access ANSI SCSI revision: 02 May 16 11:57:49 prometheus kernel: Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0 May 16 11:57:49 prometheus kernel: Attached scsi generic sg1 at scsi3, channel 0, id 0, lun 0, type 8 May 16 11:57:49 prometheus kernel: Attached scsi generic sg2 at scsi3, channel 0, id 1, lun 0, type 1 May 16 11:57:53 prometheus kernel: st: Version 20040403, fixed bufsize 32768, s/g segs 256 May 16 11:57:53 prometheus kernel: Attached scsi tape st0 at scsi3, channel 0, id 1, lun 0 May 16 11:57:53 prometheus kernel: st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 134217727 May 16 11:59:17 prometheus kernel: st: Unloaded. May 16 11:59:33 prometheus kernel: PCI: Enabling device :02:02.0 (0016 - 0017) May 16 11:59:33 prometheus kernel: scsi4 : Adaptec AIC79XX PCI-X SCSI HBA DRIVER, Rev 1.3.11 May 16 11:59:33 prometheus kernel: Adaptec 29320A Ultra320 SCSI adapter May 16 11:59:33 prometheus kernel: aic7901: Ultra320 Wide Channel A, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs May 16 11:59:33 prometheus kernel: May 16 11:59:48 prometheus kernel: (scsi4:A:0): 80.000MB/s transfers (40.000MHz, 16bit) May 16 11:59:49 prometheus kernel: (scsi4:A:1): 80.000MB/s transfers (40.000MHz, 16bit) May 16 11:59:52 prometheus kernel: Vendor: EXABYTE Model: VXA 1x10 1U Rev: A102 May 16 11:59:52 prometheus kernel: Type: Medium Changer ANSI SCSI revision: 04 May 16 11:59:52 prometheus kernel: Vendor: EXABYTE Model: VXA-2 Rev: 100E May 16 11:59:52 prometheus kernel: Type: Sequential-Access ANSI SCSI revision: 02 May 16 12:00:15 prometheus kernel: Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0 May 16 12:00:15 prometheus kernel: Attached scsi generic sg1 at scsi4, channel 0, id 0, lun 0, type 8 May 16 12:00:15 prometheus kernel: Attached scsi generic sg2 at scsi4, channel 0, id 1, lun 0, type 1 At some time, reloading the module will fail because some maximum number of SCSI hosts will be reached (32?), and I would be forced to reboot. That's not good ... Is there some way to renumber the scsi hosts or to reuse them? Then whenever the UUID was detected, regardless of the /dev device automatically assigned, the names you chose were also created. Fortunately, devices are assigned as they should :) Where would I find UIDs on Debian amd64? Hm, maybe I should better post this to the debian-amd64 list? GH
Re: Problems with amreport and postfix
On (05/15/06 19:29), Jon LaBadie wrote: I've never heard of it using sendmail by default. grep'ping the source found no instance of sendmail. But the configure script has lines like: for ac_prog in Mail mailx mail case $MAILER in # Let the user override the test with a path. error: Set MAILER to some program that accepts -s subject user message_file. echo WARNING: Amanda cannot send mail reports without these programs. Which suggest to me that only MUA's are looked for. That's true, I wasn't able to find any mention of sendmail in the source either. But if I don't set MAILER, it seems to find that anyway. I've checked my backup user's environment, and it doesn't set MAILER either. Perhaps I have a link somewhere to sendmail that shouldn't be there. I suppose I'll just go ahead and keep setting MAILER=/usr/bin/Mail. One more small dependency is not too bad, I guess. Thanks for spending the time to help with such a minor problem. Josh
RE: Would like to get list's impression on (amanda 4TB backup)
Josef, here are some of my comments about TSM that I sent to a lister off list. I think in a later email I will try to list the things that I liked about TSM that I would like to see in Amanda. Some may not be possible. At my previous job we used TSM and it is a very nice backup solution. We had a tape library that held about 250 tapes and up to 8 drives (We had 5 drives in it). It seems to excel at backing up large amounts of data. One unique thing it does is that it uses and incremental forever backup strategy. It backs up a machine fully only once (it can probably be setup other ways, but that's the way we used it)--(I could be wrong about that). There is a backup server (similar to the way Amanda is setup) and it has a database that keeps track of everything that has been backed up (all files). It knows what files are on what tapes (Also similar to Amanda). So instead of a traditional setup where you would backup fulls on Sunday night and differentials or incrementals the rest of the week (or something similar) and keep 4 sets of fulls for 4 weeks of history it has some different settings. You set some parameters for each machine that is backed up (along with the system default values). You set a maximum time that it will keep old copies of files. (ex. 1 year) You set the maximum number of copies it will keep of any single file. (ex. 30 copies). You set the minimum number of copies it will keep of any single file. (ex. 5 copies) (Actually I am not certain about this setting) There are many other settings too, for example how many copies of a deleted file does it keep. With the above parameters it will keep up to 1 year of file versions, but not more than 30 copies and not less than 5. If the file changes every day you will have 30 copies of it but only 1 month of history. If it changes once a month you will have 12 copies/1 years worth. It also keeps track of each tape's fragmentation. Say you set it to 70%. As files expire they will still be on a tape until overwritten. If a tape has too many expired files on it, it will exceed the fragmentation threshold. Once that happens the system will load the tape(s) and extract the good data from it and write it to a new tape. Then the old tape(s) will be put back into rotation for subsequent backups. It can also be setup to account for offsite backups. Our tape library had a hopper in it that was about 10 tapes large. It would figure out what files were new and not yet at the offsite facility and write copies of all new files to new tapes. It would put those in the hopper for the morning person to collect. It would also print out a paper that listed the tape numbers going offsite and which ones to bring back from offsite. Those would get put back into the library. With this method we always had a complete copy of everything ever backed up in the offsite location. It also had a copy of everything it ever backed up in the library. Obviously this can only work if you have a library large enough to hold all of the data it will accumulate over a year or so. There are 2 interfaces for the system, a web/java interface and a regular application interface. The nice thing about the web interface is that you can log on from any machine and tell it to restore a file to any client using only a web browser. When restoring it can list all available versions of a file or you can just set a date and tell it to restore to that date. Or just take the default of the newest copies of all. It can be setup to backup over whatever network you have. Ours had a 2Gb optical SAN network for the IBM database servers and the diskdrives were connected directly to that SAN. For the Windows clients (the ones I was managing) they just backed up over the regular network (100Mb). It would (similar to Amanda) backup over the network to a holding disk on the server and then later push it out to tape. Very nice. I believe the total cost of the system we had was around $500,000.00. Pretty expensive, but a heck of a system! That included the tape library, the backup server (I think it was an IBM AIX R6000 server), And the software. I think I may have shocked the initial person requesting info on this with this value. Please keep in mind that this was a lot of hardware and that was the majority of the cost. The cost of the software is in line with other commercial products IMHO. After dealing with several different backup solutions, TSM is by far the most impressive I've seen. When I started looking into Amanda at my new job it seemed to have some of the features of TSM or at least was going in the direction of the approach that TSM takes. For example it determines when to make fulls, etc and all machines are not backed up as fulls on the same day. Certainly an impressive system for a GPL project. And since it is used mainly by people without mega-hardware, it probably wouldn't benefit the users that much to take the approach of TSM. I know there must be settings/etc for a much smaller
Re: Would like to get list's impression on (amanda 4TB backup)
Hope it's ok if I jump in here. I work at a University on our backups. I agree with everything Gordon said about TSM. We have both Amanda and TSM here. Our TSM setup is pretty much what Gordon described. We use Amanda to only backup what would be required to restore the system on a server. Like /, /var, and /opt (and any special files the user would require). We back up everything else with TSM. So if there were a disaster, the Admin for the server would use Amanda to restore the server, and then TSM to restore everything else. If an Admin just wants to restore a file, or some files, they use the TSM GUI like Gordon mentioned and do their own restores. Larry McMahon Gordon J. Mills III writes: Josef, here are some of my comments about TSM that I sent to a lister off list. I think in a later email I will try to list the things that I liked about TSM that I would like to see in Amanda. Some may not be possible. At my previous job we used TSM and it is a very nice backup solution. We had a tape library that held about 250 tapes and up to 8 drives (We had 5 drives in it). It seems to excel at backing up large amounts of data. One unique thing it does is that it uses and incremental forever backup strategy. It backs up a machine fully only once (it can probably be setup other ways, but that's the way we used it)--(I could be wrong about that). There is a backup server (similar to the way Amanda is setup) and it has a database that keeps track of everything that has been backed up (all files). It knows what files are on what tapes (Also similar to Amanda). So instead of a traditional setup where you would backup fulls on Sunday night and differentials or incrementals the rest of the week (or something similar) and keep 4 sets of fulls for 4 weeks of history it has some different settings. You set some parameters for each machine that is backed up (along with the system default values). You set a maximum time that it will keep old copies of files. (ex. 1 year) You set the maximum number of copies it will keep of any single file. (ex. 30 copies). You set the minimum number of copies it will keep of any single file. (ex. 5 copies) (Actually I am not certain about this setting) There are many other settings too, for example how many copies of a deleted file does it keep. With the above parameters it will keep up to 1 year of file versions, but not more than 30 copies and not less than 5. If the file changes every day you will have 30 copies of it but only 1 month of history. If it changes once a month you will have 12 copies/1 years worth. It also keeps track of each tape's fragmentation. Say you set it to 70%. As files expire they will still be on a tape until overwritten. If a tape has too many expired files on it, it will exceed the fragmentation threshold. Once that happens the system will load the tape(s) and extract the good data from it and write it to a new tape. Then the old tape(s) will be put back into rotation for subsequent backups. It can also be setup to account for offsite backups. Our tape library had a hopper in it that was about 10 tapes large. It would figure out what files were new and not yet at the offsite facility and write copies of all new files to new tapes. It would put those in the hopper for the morning person to collect. It would also print out a paper that listed the tape numbers going offsite and which ones to bring back from offsite. Those would get put back into the library. With this method we always had a complete copy of everything ever backed up in the offsite location. It also had a copy of everything it ever backed up in the library. Obviously this can only work if you have a library large enough to hold all of the data it will accumulate over a year or so. There are 2 interfaces for the system, a web/java interface and a regular application interface. The nice thing about the web interface is that you can log on from any machine and tell it to restore a file to any client using only a web browser. When restoring it can list all available versions of a file or you can just set a date and tell it to restore to that date. Or just take the default of the newest copies of all. It can be setup to backup over whatever network you have. Ours had a 2Gb optical SAN network for the IBM database servers and the diskdrives were connected directly to that SAN. For the Windows clients (the ones I was managing) they just backed up over the regular network (100Mb). It would (similar to Amanda) backup over the network to a holding disk on the server and then later push it out to tape. Very nice. I believe the total cost of the system we had was around $500,000.00. Pretty expensive, but a heck of a system! That included the tape library, the backup server (I think it was an IBM AIX R6000 server), And the software. I think I may have shocked the initial person requesting info on this with this value. Please keep in mind that this was a lot
Re: windows vshadow
On Tue, May 16, 2006 at 10:05:22AM -0400, Jon LaBadie wrote: The vshadow.exe is only available for WinXP and 2003 server. So it is not a general solution, but maybe a start? Is this related to their Volume Management Services? IIRC, this interacts with a backgroup service like LVM or Veritas. LEts you create virtual volumes and the like. Does it enable you to take snapshots of existing filesystems without rebuilding them into managed virtual volumes (or whatever the MS verbiage for these things is)? For what it's worth, if it is the same thing that I have heard of, I've heard it works ok. -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Re: Would like to get list's impression on (amanda 4TB backup)
On Tue, May 16, 2006 at 11:17:34AM -0400, Jon LaBadie wrote: On Tue, May 16, 2006 at 09:53:13AM -0500, Gordon J. Mills III wrote: here are some of my comments about TSM incremental forever is a feature I've heard of in some other backup systems. I wouldn't expect that to fit the mold of amanda as recovery would then truly demand amanda software and indexes to be present. I've been thinking about a new vtape changer that might make it easy to do this kind of incremental forever backup. One issue that I run into: Most of my system have 18-20 virtual tapes. runtapes is 2, but I really try hard to keep things at one tape. We need to kep to weeks of backups. Suppose some event happens and someone needs to make an emergency backup of what's on a server, right now. They run amdump, it clears the oldest tape. If this happens too many times in a dumpcycle, oops, we don't have our two weeks of backups. I've been thinking about a tape changer that uses a timestamp for the label. No tape is ever reused. When amdump runs, something creates a new tape, labels it, and loads it. The backup is done to this tape. You then have a cronjob that runs nightly to cull old backups. Give it a timeframe, it deletes any tapes that are older than the timeframe. If you had a model like this, where tapes could stick around forever if nothing deleted them, you'd just need a flag that a given timestamp is a full backup and should never be thrown out. With many people doing amanda backups to vtapes, it would be nice to be able to archive desired parts to offsite ptapes. Shouldn't this be easy if you plan to make your vtapes the same size as your ptapes? -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Error in building 2.5.0p2 on Solaris 9
Greetings all; I'm attempting to compile and install amanda-2.5.0p2 on a 2-way Sunfire V210 running Solaris 9 w/ 2gb RAM. Compiler is gcc, v3.4.2, linker is Sun's supplied /usr/ccs/bin/ld. Worthy of note is that this same package configured and compiled flawlessly on a 2-way Dell 1850 server running RHES 3.0. Here are my configure parameters: ./configure --prefix=/usr/local/prdamanda --with-index-server=amanda2 --with-user=prdamanda --with-group=disk --with-config=prodbackup --with-tmpdir=/tmp/prdamanda --with-gnutar=/usr/local/bin/tar The configure runs to completion with no warnings or errors save one, GNUplot not being installed. No biggie there. The make produces the following warnings as it proceeds through the building the code: [...] ../common-src/amanda-int.h:339:1: warning: UINT8_MAX redefined In file included from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/syslimits.h:41, from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/limits.h:11, from ../common-src/amanda.h:148, from uparse.y:32: /usr/include/sys/int_limits.h:79:1: warning: this is the location of the previous definition In file included from ../common-src/amanda.h:245, from uparse.y:32: ../common-src/amanda-int.h:340:1: warning: UINT16_MAX redefined In file included from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/syslimits.h:41, from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/limits.h:11, from ../common-src/amanda.h:148, from uparse.y:32: /usr/include/sys/int_limits.h:80:1: warning: this is the location of the previous definition In file included from ../common-src/amanda.h:245, from uparse.y:32: ../common-src/amanda-int.h:342:1: warning: UINT64_MAX redefined In file included from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/syslimits.h:41, from /usr/local/lib/gcc/sparc-sun-solaris2.9/3.4.2/include/limits.h:11, from ../common-src/amanda.h:148, from uparse.y:32: [...] = Despite the warnings, everything goes smoothly until the build process reaches the building the man pages section, when it errors out completely: [...] creating amrecover Making all in man /usr/local/libxslt/sparcv9/bin/xsltproc --path ./xslt/ --xinclude --stringparam latex.imagebasedir amanda.conf.5/ --stringparam noreference 1 --output xml-source/amanda.conf.5.proc.xml ./xslt/expand-sambadoc.xsl xml-source/amanda.conf.5.xml /usr/local/libxslt/sparcv9/bin/xsltproc --path ./xslt/ --output amanda.conf.5 man.xsl xml-source/amanda.conf.5.proc.xml warning: failed to load external entity http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl compilation error: file ./xslt//man.xsl line 8 element import xsl:import : unable to load http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl *** Error code 5 make: Fatal error: Command failed for target `amanda.conf.5' Current working directory /export/home/gkofoed/amanda-2.5.0p2/man *** Error code 1 make: Fatal error: Command failed for target `all-recursive' == Not sure where to go from here, I've never seen this when building amanda from source before. If I've missed supplying any necessary information to help trouble-shoot, please let me know. Any help/guidance appreciated. Thanks. George
Re: windows vshadow
On Tue, May 16, 2006 at 11:48:06AM -0400, Ross Vandegrift wrote: On Tue, May 16, 2006 at 10:05:22AM -0400, Jon LaBadie wrote: The vshadow.exe is only available for WinXP and 2003 server. So it is not a general solution, but maybe a start? Is this related to their Volume Management Services? IIRC, this interacts with a backgroup service like LVM or Veritas. LEts you create virtual volumes and the like. Does it enable you to take snapshots of existing filesystems without rebuilding them into managed virtual volumes (or whatever the MS verbiage for these things is)? For what it's worth, if it is the same thing that I have heard of, I've heard it works ok. Dunno. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Would like to get list's impression on (amanda 4TB backup)
On Tue, May 16, 2006 at 11:58:55AM -0400, Ross Vandegrift wrote: On Tue, May 16, 2006 at 11:17:34AM -0400, Jon LaBadie wrote: On Tue, May 16, 2006 at 09:53:13AM -0500, Gordon J. Mills III wrote: here are some of my comments about TSM incremental forever is a feature I've heard of in some other backup systems. I wouldn't expect that to fit the mold of amanda as recovery would then truly demand amanda software and indexes to be present. I've been thinking about a new vtape changer that might make it easy to do this kind of incremental forever backup. One issue that I run into: Most of my system have 18-20 virtual tapes. runtapes is 2, but I really try hard to keep things at one tape. We need to kep to weeks of backups. Suppose some event happens and someone needs to make an emergency backup of what's on a server, right now. They run amdump, it clears the oldest tape. If this happens too many times in a dumpcycle, oops, we don't have our two weeks of backups. I've been thinking about a tape changer that uses a timestamp for the label. No tape is ever reused. When amdump runs, something creates a new tape, labels it, and loads it. The backup is done to this tape. Couldn't this be done now with the autolabel feature and a V.large tapecycle that always called for a new tape? You then have a cronjob that runs nightly to cull old backups. Give it a timeframe, it deletes any tapes that are older than the timeframe. If you had a model like this, where tapes could stick around forever if nothing deleted them, you'd just need a flag that a given timestamp is a full backup and should never be thrown out. With many people doing amanda backups to vtapes, it would be nice to be able to archive desired parts to offsite ptapes. Shouldn't this be easy if you plan to make your vtapes the same size as your ptapes? The logs+indexes might be a problem if it was going to a different config. And I was thinking of maybe selecting what I wanted, like all level 0's of a particular host:disk. Or as of a specific date, one level 0 plus incrementals up to that date. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Would like to get list's impression on (amanda 4TB backup)
Gordon J. Mills III wrote: Josef, here are some of my comments about TSM that I sent to a lister off list. I think in a later email I will try to list the things that I liked about TSM that I would like to see in Amanda. Some may not be possible. At my previous job we used TSM and it is a very nice backup solution. We had a tape library that held about 250 tapes and up to 8 drives (We had 5 drives in it). It seems to excel at backing up large amounts of data. One unique thing it does is that it uses and incremental forever backup strategy. It backs up a machine fully only once (it can probably be setup other ways, but that's the way we used it)--(I could be wrong about that). There is a backup server (similar to the way Amanda is setup) and it has a database that keeps track of everything that has been backed up (all files). It knows what files are on what tapes (Also similar to Amanda). One possible downside of the TSM approach is this database. If it is somehow lost or corrupted, all you have is a bunch of tapes with random chunks of data that may not be recoverable into its original structure. While it is a robust database, things can happen (such as failing RAID or SAN controllers wiping out all of your redundant disks), so you need to make sure that you have backups of the database that are outside of TSM (somewhat similar to what can happen if you lose your Amanda indexes, except that with Amanda you can use native tools to restore everything on your tapes. Frank -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
RE: Would like to get list's impression on (amanda 4TB backup)
On Tue, May 16, 2006 at 09:53:13AM -0500, Gordon J. Mills III wrote: here are some of my comments about TSM Thanks Gordon. Tivoli sounds like an interesting enterprise system. incremental forever is a feature I've heard of in some other backup systems. I wouldn't expect that to fit the mold of amanda as recovery would then truly demand amanda software and indexes to be present. One feature you describe I would like to see added to amanda. It really is not a change to the operation, but a new utility. That is copying all or portions of a set of backups to new tapes or a new config. With many people doing amanda backups to vtapes, it would be nice to be able to archive desired parts to offsite ptapes. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax) It is true that the incremental forever brings up other issues. I think a discussion of some of the features/differences between Amanda and TSM may bring out some additional feature requests to improve Amanda. The Inc Forever method does demand a lot more overhead, not to mention the ability to append to tapes. I guess it doesn't need that, but it sure cuts down on the number of tapes required in the library. I think if you look at a traditional backup scheme vs TSM, Amanda falls somewhere in the middle. It is a very nice approach. I guess if I had 80 tapes with a 7 or 10 day dumpcycle I would have a lot of versions of files to choose from :-). I think that one of the things I liked the most about TSM was the saving of multiple versions of a file and being able to choose which one I wanted. In a way it had a lot to do with the way the recovery interface was done. This can sort of be done in Amanda now, but would be better if the amrecover interface listed the versions and dates available. The more I think about it, the more I think Amanda can come closer to achieving this simply by having a lot of tapes (which would be necessary anyway for the TSM approach). As you pointed out, the offsite feature would also be a nice addition. All of this stuff works best with a tape library that has more than 1 drive, so I suspect it only applies to a small percentage of Amanda users. The offsite feature would require a database to keep track of what is offsite and what is not. What new files need to be written to a tape to go offsite and what tape(s) need to come back, etc. Probably most important to me though would be the ability to handle Windows backups better. I know that it is not necessarily Amanda's fault, but the file permissions (ACLs) cannot currently be backed up with Amanda. Also, it does not allow exclude lists for windows clients. This should be possible since now smbclient has this feature. Regards, Gordon
RE: Would like to get list's impression on (amanda 4TB backup)
Gordon J. Mills III wrote: Josef, here are some of my comments about TSM that I sent to a lister off list. I think in a later email I will try to list the things that I liked about TSM that I would like to see in Amanda. Some may not be possible. At my previous job we used TSM and it is a very nice backup solution. We had a tape library that held about 250 tapes and up to 8 drives (We had 5 drives in it). It seems to excel at backing up large amounts of data. One unique thing it does is that it uses and incremental forever backup strategy. It backs up a machine fully only once (it can probably be setup other ways, but that's the way we used it)--(I could be wrong about that). There is a backup server (similar to the way Amanda is setup) and it has a database that keeps track of everything that has been backed up (all files). It knows what files are on what tapes (Also similar to Amanda). One possible downside of the TSM approach is this database. If it is somehow lost or corrupted, all you have is a bunch of tapes with random chunks of data that may not be recoverable into its original structure. While it is a robust database, things can happen (such as failing RAID or SAN controllers wiping out all of your redundant disks), so you need to make sure that you have backups of the database that are outside of TSM (somewhat similar to what can happen if you lose your Amanda indexes, except that with Amanda you can use native tools to restore everything on your tapes. Frank -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501 Hello Frank. That is true, as you point out the DB becomes VERY important in this type of backup scheme. I believe the way we handled it was to use their BMR (Bare Metal Recovery) on the backup server (and other critical servers). I suspect it backs up everything necessary to rebuild the database in case of failure. Of course this all goes towards the more overhead necessary for such a scheme. Regards, Gordon
UNIGROUP 18-MAY-2006 (Thu): AMANDA Enterprise Backup
Hello all, I will be presenting Amanda in New York this Thursday, at the regular UNIGROUP meeting. See the attached message for all the details; reservations are requested but not required. --Ian -- Zmanda: Open Source Data Protection and Archiving. http://www.zmanda.com ---BeginMessage--- UNIGROUP OF NEW YORK - UNIX USERS GROUP - MAY 2006 ANNOUNCEMENTS 1. UNIGROUP'S MAY 2006 GENERAL MEETING ANNOUNCEMENT When: THURSDAY, May 18th, 2006(3rd Thursday) Where: Alliance for Downtown NY Conference Facility Downtown Center 104 Washington Street South West Corner of Wall Street Area Downtown, New York City ** Please RSVP (not mandatory) ** Time: 6:15 PM - 6:25 PM Registration 6:25 PM - 6:45 PM Ask the Wizard, Questions, Answers and Current Events 6:45 PM - 7:00 PM Unigroup Business and Announcements 7:00 PM - 9:30 PM Main Presentation --- Topic: The AMANDA Open Source Enterprise Backup System --- Speaker: Ian Turner Zmanda http://www.zmanda.com INTRODUCTION: - AMANDA, the Advanced Maryland Automatic Network Disk Archiver, is an Enterprise-Class Network Backup System developed by the University of Maryland. --- SPECIAL INSTRUCTIONS: - To REGISTER for this event, please RSVP by using the Unigroup Registration Page: http://www.unigroup.org/unigroup-rsvp.html This will allow us to automate the registration process. (Registration will also add you to our mailing list.) Please avoid emailed RSVPs, if at all possible. Please continue to check the Unigroup web site and meeting page, for any last minute updates concerning this meeting. If you registered for this meeting, please check your email for any last minute announcements as the meeting approaches. Also make sure any anti-spam white-lists are updated to _ALLOW_ Unigroup traffic! Please try to RSVP as soon as possible. Note: RSVP is not mandatory for this location, but it does help us to properly plan the meeting (food, drinks, handouts, seating, etc.). --- MAIN PRESENTATION OUTLINE: -- Topic: AMANDA -- AMANDA is a popular open-source backup and archiving package. AMANDA uses native tools, and can back up a large number of machines running various versions of the Linux, Unix, or Windows operating systems. This talk will discuss the state of AMANDA, and also new project developments. For more information, please see: http://wiki.zmanda.com/. The main topics to be covered are: - What is Amanda? - Amanda features - Configuration - Administration - Future work - More information - Amanda Installation Demo (as time permits) --- Web Resources: -- Amanda Homepage: http://www.amanda.org Amanda Wiki: http://wiki.zmanda.com Amanda Forums: http://forums.zmanda.com Zmanda Homepage: http://www.zmanda.com --- Speaker Biography: -- Ian Turner has been involved with Amanda since 1998, when he installed it on his home computers after the sixth hard-disk crash in three years. Amanda served his needs quite well, but there were also itches to scratch, so in 2000, Ian started making contributions to the Amanda code base. In that year he was accepted as a member of the Amanda core team. Today, Ian develops Amanda full-time, as a Zmanda engineer. --- Company Biography: -- Zmanda offers proven and cost effective open source data protection solutions. By leveraging the open source development model, we are able to provide our customers with the very best quality software and significant cost savings. Our mission is to enable leading data protection solutions such as Amanda, the world's most popular open source backup and recovery solution, for the enterprise. These solutions will be available at a fraction of the cost of incumbent vendors, providing Zmanda customers with a low cost of ownership and high return on investment.
Re: Would like to get list's impression on (amanda 4TB backup)
On Tue, May 16, 2006 at 12:31:57PM -0400, Jon LaBadie wrote: On Tue, May 16, 2006 at 11:58:55AM -0400, Ross Vandegrift wrote: I've been thinking about a tape changer that uses a timestamp for the label. No tape is ever reused. When amdump runs, something creates a new tape, labels it, and loads it. The backup is done to this tape. Couldn't this be done now with the autolabel feature and a V.large tapecycle that always called for a new tape? Ooo! That's exciting, I didn't know that there was such a think as autolabel. Ironically, the first google link when searching for amanda autolabel is about TSM. I checked the stuff on www.amanda.org but I didn't find that option. Could I get some more info? Shouldn't this be easy if you plan to make your vtapes the same size as your ptapes? The logs+indexes might be a problem if it was going to a different config. And I was thinking of maybe selecting what I wanted, like all level 0's of a particular host:disk. Or as of a specific date, one level 0 plus incrementals up to that date. Oh that would be a cool tool. It would solve a lot of problems for smaller installations too. Say I want to backup my workstation and have the files accessible most of the time for easy restore. However, I do want occasional archival copies to tape/DVD/whatever. -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Re: Would like to get list's impression on (amanda 4TB backup)
On 5/16/06, Ross Vandegrift [EMAIL PROTECTED] wrote: On Tue, May 16, 2006 at 12:31:57PM -0400, Jon LaBadie wrote: On Tue, May 16, 2006 at 11:58:55AM -0400, Ross Vandegrift wrote: I've been thinking about a tape changer that uses a timestamp for the label. No tape is ever reused. When amdump runs, something creates a new tape, labels it, and loads it. The backup is done to this tape. Couldn't this be done now with the autolabel feature and a V.large tapecycle that always called for a new tape? Ooo! That's exciting, I didn't know that there was such a think as autolabel. Ironically, the first google link when searching for amanda autolabel is about TSM. I checked the stuff on www.amanda.org but I didn't find that option. Could I get some more info? See http://wiki.zmanda.com/index.php/Amanda.conf#PARAMETERS. Look for label_new_tapes parameter. This functionality is available in 2.5.0. Search for label_new_tapes in the wiki to see examples on how to use the parameter. Paddy
Re: Amanda 2.5.0 upgrade weirdness
Hello again, Good feeling gone... ~$ amcheck DailySet1 Amanda Tape Server Host Check - Holding disk /backup/amanda: 165817748 kB disk space available, using 164769172 kB slot 3: not an amanda tape (Invalid argument) slot 4: not an amanda tape (Invalid argument) slot 5: not an amanda tape (Invalid argument) slot 6: not an amanda tape (Invalid argument) slot 7: not an amanda tape (Invalid argument) slot 8: not an amanda tape (Invalid argument) slot 9: not an amanda tape (Invalid argument) slot 10: not an amanda tape (Invalid argument) slot 1: not an amanda tape (Invalid argument) slot 2: not an amanda tape (Invalid argument) (expecting tape B003 or a new tape) client check snipped (brought to you by Amanda 2.5.0p2) And ~$ amtape DailySet1 update changer: got exit: 0 str: 2 10 1 1 changer_query: changer return was 10 1 1 changer_query: searchable = 1 amtape: scanning all 10 slots in tape-changer rack: changer_find: looking for NULL changer is searchable = 1 changer: got exit: 0 str: 2 /dev/nst0 slot 2: not an amanda tape (Invalid argument) changer: got exit: 0 str: 3 /dev/nst0 slot 3: not an amanda tape (Invalid argument) Where is NULL changer defined? mtx status reports the tape barcode labels accurately read. The compile options were copied from the previous version and updated as required to match the new options. I'm using chg-zd-mtx with an Exabyte VXA PacketLoader 1x10 1U. Should I be using something else? TIA -- Brad Willson, Sr. Computer Specialist UW GeneTests, UW Box: 358735 EM: [EMAIL PROTECTED] W: 206.221.4674, C: 425.891.2732 http://www.genetests.org
