Re: amrecover from 2.5.0p2 using tape_splitsize
The two times I've tried to move to current releases I've run into trouble and had to back out. Bringing all 24 clients and server at once is a bit daunting. I think I'll have to scrounge up another system and tape drive to practice. I'll try amrestore for the immediate restore problem. Charlie Reitsma x6642 Quoting "Dustin J. Mitchell" : On Tue, May 25, 2010 at 9:48 AM, Charlie Reitsma wrote: I saw a thread that recommended trying amindexd, amidxtaped, amoldrecover from 2.5.1b2. That was reported successful with no configuration changes. I wonder if anyone could flesh that out a bit more for me? Yikes, I'm not sure that's a good idea! I compiled 2.5.1b2 and installed in /usr/local. Am I copying amindexd and amidxtaped from /usr/local/sbin into /usr/sbin on the server? Do I repeat the compile on each client and use amoldrecover instead of amrecover? I would recommend upgrading the entire client-side or server-side all at once, rather than piecemeal. There were some protocol changes between 2.5.0 and 2.5.1, so you may run into some problems there. See amanda-compatibility(7) for details. Dustin -- Open Source Storage Engineer http://www.zmanda.com
amrecover from 2.5.0p2 using tape_splitsize
I set up 2.5.0p2 amanda with tape_splitsize and am satisfied with the result (98% tape utilization on large runs). But we want to restore something and are seeing "tar: Unexpected EOF in archive" using amrecover. I saw a thread that recommended trying amindexd, amidxtaped, amoldrecover from 2.5.1b2. That was reported successful with no configuration changes. I wonder if anyone could flesh that out a bit more for me? I compiled 2.5.1b2 and installed in /usr/local. Am I copying amindexd and amidxtaped from /usr/local/sbin into /usr/sbin on the server? Do I repeat the compile on each client and use amoldrecover instead of amrecover? Charlie Reitsma
Re: broadcast traffic
So, perhaps, the ISP is mistaking the backup traffic as a broadcast storm. It would appear as a sudden spike of traffic leaving the server. If so, a conversation with the ISP should clear up the issue. Charlie Reitsma Quoting Jean-Louis Martineau : Steve, Amanda doesn't use broadcast. Jean-Louis Steve Wray wrote: Hi there, We have a server hosted with our ISP and we have recently started to run amanda backups from this server to our main backup server. The ISP has contacted us about broadcast storms emanating from this server. These storms coincide with backup runs. I wasn't aware that amanda used broadcast traffic but there is nothing else on this server which could be causing this. If amanda does use broadcast, is there a way to turn it off or otherwise contain it? Thanks
Re: [Amanda-users] need advices on install/configure amanda on solaris 10
Are you running into the change in Solaris 10 from using (x)inetd to smf? I always had trouble with initializing that configuration. If that's what you need then I may have a few notes on getting that going with Amanda on Solaris 10. Quoting conandor <[EMAIL PROTECTED]>: This is an edited version of a previous post i would like to have advices on how to install/configure on solaris 10. thank you. i facing this problen when following the 15 min tutorial on amaddclient. bash-3.00$ amaddclient --config DailySet1 --client 192.168.1.205 --diskdev /export/home/bk2 --dumptype comp-user-tar Logging to /var/log/amanda/amaddclient.20081020162002.debug /etc/amanda/DailySet1/disklist has '192.168.1.205 /export/home/bk2 ...' entry, file not updated updating /var/lib/amanda/.amandahosts on servername /var/lib/amanda/.amandahosts contains 192.168.1.205 root, file not updated Attempting to update /var/lib/amanda/.amandahosts on 192.168.1.205 command-line: line 0: Bad configuration option: ConnectTimeout lost connection WARNING: scp from 192.168.1.205 not successful. Check 192.168.1.205:/var/lib/amanda/.amandahosts file. If entry 'servername amandabackup' is not present, append the entry to the file manually. Creating amanda-client.conf for 192.168.1.205 Creating /etc/amanda/DailySet1 on 192.168.1.205 Password: mkdir: Failed to make directory "/etc/amanda/DailySet1"; File exists WARNING: Cannot create /etc/amanda/DailySet1 on 192.168.1.205 Please copy /var/lib/amanda/amanda-client.conf-192.168.1.205 to 192.168.1.205 manually File /var/lib/amanda/example/xinetd.amandaclient contains the latest Amanda client daemon configuration. Please merge it to /etc/xinetd.d/amandaclient. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
Re: strange planner problems
Looks like the default route or gateway is gone. Are you saying you can actually ssh from this box to another? Quoting Craig Dewick <[EMAIL PROTECTED]>: Hi to help those of you who know about the internals of Amanda's components, I've attached last night's amanda report which shows all the network unreachable messages. You'll note that they occur even for the actual tape host machine along with two other machines which are all here in the same room with no firewall of any sort. My theory at the moment is that the packets are for some reason ending up at my ADSL router which doesn't know what to do since the port numbers would get automatically blocked by it's default firewall settings. This could mean that I have the basic network config for my main Cisco router set up incorrectly but this problem which Amanda is displaying has never occured with any other software or port that I know of before. I'm not sure if it's a router config issue or an Amanda issue at the moment. Craig. -- Post by Craig Dewick (tm). Web @ "http://lios.apana.org.au/~cdewick";. Email 2 "[EMAIL PROTECTED]". SunShack @ "http://www.sunshack.org"; Galleries @ "http://www.sunshack.org/gallery2";. Also lots of tech data, etc. Sun Microsystems webring at "http://n.webring.com/hub?ring=sunmicrosystemsu";. Charlie Reitsma x6642
Re: encrypting data on amanda
try a newer version. look at http://amanda.zmanda.com/quick-backup-setup.html Quoting Norman Coder <[EMAIL PROTECTED]>: I have been asked to encrypt the data I backup using Amanda. We are storing our tapes off site. According to chapter 16 of the official Amanda manual (http://www.amanda.org/docs/howto-gpg.html) encryption does not seem to be ready for production systems. Note THIS IS *NOT* YET INTENDED FOR PRODUCTION SERVERS !!! Anybody have experience encrypting data using Amanda? I am using version 2.4.4p2. Also I don't seem to have amcrypt installed. How would I go about installing this piece of amanda? Is it included in later versions? I couldn't seem to find anything useful when I tried google. Norman Charlie Reitsma x6642
Re: suggestion for a disk-to-disk backup server
In addition to doing everything to make sure a backup disk failure does not destroy all your backups you need to take into consideration the length of your cycle. My experience so far has been with tape and I am just beginning in the disk-to-disk with virtual tape. How long before Amanda can reuse one of your virtual tapes? The initial backup is going to be all level 0. After that the planner is going to try for a balanced mix of fulls and incrementals. The size of the incrementals depends on the amount of activity on a filesystem. Some will change a lot; some won't change. I would count on 30% of your backup size being backed up every day. For disk-to-disk your backup server is definitely going to contain multiples of your full backup size rather than a fraction of your full backup size. Can we get to the point of making a formula? I'll go out on a limb and put some figures down. So much of it depends on amount of compression, activity of users, and growth: (days per cycle) * (size of full backup) * (average daily fraction of full) let's say your 2 TB compresses to 1.5TB: 30 days * 1.5 TB * 1/3 = 15 TB for a full cycle. If you've been backing up the same servers to tape already you can plug in more accurate numbers for your site. Quoting Matt Hyclak <[EMAIL PROTECTED]>: On Sun, Jun 24, 2007 at 11:43:26PM -0700, Rudy Setiawan enlightened us: I am trying to create backup servers out of the following specs: P4 3.0Ghz 1GB RAM RAID0 - 750GB x 2 (SATA) Hosts to backup: about 20 hosts with roughly 100GB each host. What will be the complications and the limitations? And also what are the recommendations? You realize that by using RAID 0, you are doubling the chance of losing all of your backups, because there is NO redundancy - so if either of your disks has a problem, you lose the entire array. If it is only temporary and you are then moving the data to something else, that might be all right, but I wouldn't feel comfortable with my backups (or anything other than scratch space, really) being on RAID 0. I would use RAID 1, or if you have disks to spare RAID 10. Matt -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263 Charlie Reitsma x6642
Re: Virtual tape to Real tape cycles
Quoting "Dustin J. Mitchell" <[EMAIL PROTECTED]>: On Fri, Jun 15, 2007 at 04:23:57PM -0400, Charlie Reitsma wrote: As I read through the man pages I am finding hints that it has been thought of. In the amdd man page "This may be used for debugging or to convert from one format to another." I began looking at the contents of various directories and files based on comments found in Gene's Amanda Helper scripts. The index is just a list of files contained in the images contained on the the tape. It wouldn't change at all as the images aren't being modified. I couldn't find anything that maps an image file to a particular location on a tape. It appears that the contents of a virtual tape need no further processing before moving them to a real tape. Copying those images onto a tape and moving the index, curinfo and tapelist date to another cycle seems like it might be straightforward. Since all the image sizes are known in advance they might even be reordered more optimally to fit on tape. I didn't like Gene's suggestion that the virtual tape size and the real tape size should be kept the same. We might even choose to put different file systems on different tapes as they may go to different offsite locations for security reasons. Restores appear to collect the images associated with a particular file system and give them back to the appropriate program to extract the file(s) requested. That process wouldn't change just because the media changed. I'll take a look at the Developer Documentation to continue my exploration. Charlie -- This sounds like a great project, and I'm excited you're working on it. Please feel free to consult amanda-hackers with any further technical questions or musings you might have. The functionality you're thinking about is a form of hierarchical storage management, with data migrating between storage media based on enterprise policies and needs. It's a kind of functionality that would be welcome in Amanda, and that has been in the back of most developers' minds for some time. The new Device API will be an important step in this direction, since (among other benefits) it will allow us to treat all storage media the same way. With a little reworking, taper can then become a generic data migration tool. The major unresolved issues are: - policy (how does Amanda know when to migrate what, and where) - metadata updates (which you mentioned above) Device API: http://wiki.zmanda.com/index.php/Device_API Once the Device API is released, the immediate next step is to treat holding disk as a device. This represents Amanda's existing functionality -- dumping to holding, then spooling to tape -- as a data migration, opening the door to more complex migrations down the road. Dustin -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/ I can report that my basic expectation of success has been met. I can achieve what I want "manually" as follows: 1) copy the image files (new terminology: collection) from the tape to the holding directory. I achieved this by manually copying it from the virtual tape to a directory I created on the holding disk named as the date of the original backup. I had to strip off the leading filenum from each file name. I copied the most recent level 0, level 1 and level 2 this way. amrestore -r is expected to provide the same result. 2) I created a new config with new virtual tapes and told amflush to use this config to write the image files to tape. 3) I changed the stats lines in the curinfo file of the new config to match the history lines for those image files from the original config. I think if I had copied these history lines to the new config's curinfo file before the flush then amflush would have written the stats lines properly. 4) I copied the index information from the original config for each of the image files to the new config's index. End result is amrecover is able to recover from any of the three images on the new tape. So, it can be done. Now to make it happen in a less manual way. Charlie Reitsma x6642
Re: Virtual tape to Real tape cycles
As I read through the man pages I am finding hints that it has been thought of. In the amdd man page "This may be used for debugging or to convert from one format to another." I began looking at the contents of various directories and files based on comments found in Gene's Amanda Helper scripts. The index is just a list of files contained in the images contained on the the tape. It wouldn't change at all as the images aren't being modified. I couldn't find anything that maps an image file to a particular location on a tape. It appears that the contents of a virtual tape need no further processing before moving them to a real tape. Copying those images onto a tape and moving the index, curinfo and tapelist date to another cycle seems like it might be straightforward. Since all the image sizes are known in advance they might even be reordered more optimally to fit on tape. I didn't like Gene's suggestion that the virtual tape size and the real tape size should be kept the same. We might even choose to put different file systems on different tapes as they may go to different offsite locations for security reasons. Restores appear to collect the images associated with a particular file system and give them back to the appropriate program to extract the file(s) requested. That process wouldn't change just because the media changed. I'll take a look at the Developer Documentation to continue my exploration. Quoting "Dustin J. Mitchell" <[EMAIL PROTECTED]>: On Fri, Jun 15, 2007 at 02:14:43PM -0400, Jon LaBadie wrote: On Tue, Jun 12, 2007 at 04:27:32PM -0400, Charlie Reitsma wrote: > Has there been any work on amanda being aware of backups going to > virtual tape and then the virtual tape being archived to real tape? > I'd love to be able to keep as much as possible on virtual tape but at > some point I need to archive the data to an offsite location. But in > order to do a restore I would need the index of the virtual tape > associated with the real tape. Very likely, multiple real tapes would > be required to hold a single virtual tape so the index might need to > be rewritten to reflect the media change. I see the move from virtual > tape to real tape more of a tape duplication - almost as if the > virtual disk were looked at as the holding disk being flushed to tape. To Charlie's initial question -- this is a *hard* problem, and one for which a complete answer is a long, long way off. However, we are looking in that direction. In fact, the Device API is a step along the way, as it gives us a consistent way to talk about different devices (e.g., vtape and tape) and moving data between them. Related to potential projects such as Charlie suggests: Does an documentation exist that might be called "Amanda Internals"? Things that describe the backup process, authentication procedures, network communications and particularly for this project, amanda data location, layout, and parsing info? Obviously several amanda utilities can access these data. For example, amoverview can lookup the dates and levels for each active DLE backup and amrecover can determine on which tapes they are located. But aside from reading the sources for these utilities is the location and procedure for accessing these data documented somewhere? We created the "Developer Documentation" section on wiki.zmanda.com a few weeks ago. It's far from comprehensive, but it contains links to all of the developer-targetted documentation we could find, which includes descriptions of many of the APIs, authentication methods, and data storage formats. The (small) parts of that section that I wrote myself are based on things I learned while exploring new areas of the codebase. If someone wanted to take on a significant project in Amanda, I think it would make sense to leverage the exploration that would take place to generate some additional developer documentation. Dustin -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/ Charlie Reitsma x6642
Virtual tape to Real tape cycles
Has there been any work on amanda being aware of backups going to virtual tape and then the virtual tape being archived to real tape? I'd love to be able to keep as much as possible on virtual tape but at some point I need to archive the data to an offsite location. But in order to do a restore I would need the index of the virtual tape associated with the real tape. Very likely, multiple real tapes would be required to hold a single virtual tape so the index might need to be rewritten to reflect the media change. I see the move from virtual tape to real tape more of a tape duplication - almost as if the virtual disk were looked at as the holding disk being flushed to tape. Charlie Reitsma
