Re: Not found in archive
On Wed, Jun 30, 2010 at 5:53 PM, Stefan G. Weichinger s...@amanda.org wrote: Non-IT-people might say QUOTE yes --- ;-) ? Yes, we IT people do love our N-negatives, since we can just look at the rightmost bit of N to determine its interpretation. The name is based on the underlying gnu-tar parameter, and gnutar has both --unquote and --no-unquote, but --unquote is the default. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: ZWC and exclude list not behaving as expected
On Wed, Jun 30, 2010 at 12:58 PM, Chris Nighswonger cnighswon...@foundations.edu wrote: So... am I missing something that I don't know about or is this a bug? Replying on Paddy's behalf: There's a 3.1.1 that's currently in internal testing that might improve the situation. It should be ready in a few days. Send another email if you don't hear anything about it soon? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Migrating from DLT to LTO
On Tue, Jun 15, 2010 at 2:48 AM, Marc Muehlfeld marc.muehlf...@medizinische-genetik.de wrote: Does amanda meanwhile provides a way to migrate the data to the new tapes without copying 1:1 and lossing much free space (DLT has 40 GB, LTO has 400 GB space). Not yet, no. Amvault will be able to do this, I just need to get it written :) Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: ZWC and exclude list question (another one)
On Wed, Jun 23, 2010 at 8:22 AM, Chris Nighswonger cnighswon...@foundations.edu wrote: So is the only way to do this to have a dumptype setup using 'exclude' and 'exclude append' to catch each of potentially 300 some multimedia file extensions? This would mean 300 some 'exclude'/'exclude append' directives. It seems there must be a better way. Surely you could get 90% of the space-saving effects with only 10 extensions.. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: 3202?
On Thu, Jun 24, 2010 at 10:07 AM, Gene Heskett gene.hesk...@gmail.com wrote: Then in the reports listing: coyote /bin 1 10 10 -- 0:01 15.4 0:01 10.0 Nothing I know of changed, but that does seem small. Well, it's an incremental, and as you say nothing's changed.. or did I miss your point? Dustin -- Open Source Storage Engineer http://www.zmanda.com
3.1.1 is coming up
We're looking to release 3.1.1 early next week. As I've mentioned before, the third number in Amanda version numbering is now a patch, so you can think of this as 3.1p1 if you'd like. That means it doesn't have any new functionality, only bugfixes, documentation updates, and so on. Jean-Louis and I would like to hear soon if there are any outstanding failures out there, so we can get the fixes in. This would be a great time for the Platform Experts to download the latest snapshot from the 3.1.0 branch, currently amanda-3.1.0.svn.3202.tar.gz from http://www.zmanda.com/community-builds.php and give it a run? Thanks! Dustin P.S. For those looking for other ways to help, all of the Amanda projects I mentioned a while ago are still open. There's always docs to write, too -- for example, http://wiki.zmanda.com/index.php/How_To:Backup_to_Amazon_S3 needs to be updated to use the new chg-multi. It'd also be great to see an external continuous integration system set up and hosted (I'm partial to Buildbot, since I'm the maintainer). We have 3363 tests right now, with more being added every day. -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda-3.1.0 mail report: tape write rate is off
On Mon, Jun 21, 2010 at 9:07 AM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: I haven't looked too hard on this but it seems to me that amreport for amanda-3.1.0 doesn't correctly calculate the tape write rate. At a brief look, that looks reasonable - 884G in 5 hours is about 50M/s, which is what Amanda is showing. Perhaps you can point more directly to what seems to be in error? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda-3.1.0 mail report: tape write rate is off
On Mon, Jun 21, 2010 at 10:21 AM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: 5hrs is the run time: Estimate Time (hrs:min) 0:44 Run Time (hrs:min) 5:36 Dump Time (hrs:min) 2:50 2:32 0:18 Avg Tp Write Rate (k/s) 51715.5 51345.9 98233.1 USAGE BY TAPE: Label Time Size % Nb Nc av48-2_down_Q00033 2:21 884163M 114.5 14 98 it took 2:21 for the taper to write 884163M which is more like ~100MBis as the per DLEs stats show. I believe there is an simple arithmetic error somewhere... Actually, the numbers I was looking at were: On Mon, Jun 21, 2010 at 9:07 AM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Tape Time (hrs:min) 4:52 4:49 0:02 ... Avg Tp Write Rate (k/s) 51715.5 51345.9 98233.1 ... av48-2_down_Q00033 2:21 884163M 114.5 14 98 And the difference between 4:52 and 2:21 is that the 4:52 is the 2:21 is the sum of the times for individual parts, while the 4:52 is the sum of the time for each entire dump - thus the 4:52 includes time to write filemarks and (if necessary, but not in this case) change tapes. That said, the difference between these two numbers suggests something is being added twice. Can you send the trace log from which this was generated so that I can take a closer look? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda-3.1.0 mail report: tape write rate is off
On Mon, Jun 21, 2010 at 1:40 PM, Jon LaBadie j...@jgcomp.com wrote: This time (4:52) is the sum of the total time taken in writing each individual DLE (DONE lines in the log). There is overhead, particularly when writing multi-part DLEs that is reflected in this number. .. This number (2:21) is the sum of the times actually writing each part of the DLEs (PART lines in the log). It does not include any overhead time between each part writing. This is completely correct. To my knowledge, this is how Amanda has always calculated these statistics (we are pretty careful to run the old and new amreports on a bunch of identical real-world logfiles and check that the results are the same). So the question is: should the report be updated to make this clearer? And if so, how? This might be a good place for someone to do a bit of hacking on the new Amanda::Report::human module.. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda-3.1.0 mail report: tape write rate is off
On Mon, Jun 21, 2010 at 6:44 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: I guess bumping up tape_splitsize should speed up things. I'll try that tonight. You may want to check out the taper debug logs, too - it's a very common misconfiguration to set tape_splitsize but not split_diskbuffer, and thus fall back to the default in-memory splitsize of 10M, which is fantastically small. We're working on a fix for this in 3.2 - and probably also moving the splitting configuration to the tapetype. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Many Zombie processes with Amanda 2.6.1p2 during backup
On Fri, Jun 18, 2010 at 6:43 AM, Andreas Haumer andr...@xss.co.at wrote: I recently upgraded one machine from Amanda v2.5.2p1 to v2.6.1p2 and noticed that with the new version amandad does not collect exit status from forked child processes anymore, so all terminated child processes are left as zombies until the main amandad process terminates. 2.6.1 isn't really new anymore. Does this still occur in 3.1.0? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amcheck does not check runtapes parameter
On Thu, Jun 17, 2010 at 4:12 AM, Dennis Benndorf dennis.bennd...@gmx.de wrote: I stumbled about that amcheck does not seem to recognize how much tapes per run is set in amanda.conf. So if I need 5 tapes, it would say that everything is ok if only one tape is available. Might this something to fix in further releases? Well, it's hard to check that without cycling through all 5 tapes, which of course causes wear on the changer and leaves it positioned on a different tape than where it started. Also, runtapes is a *maximum*, and many folks set it to something larger than they use for a typical dump, so that they can use extra tapes on particularly heavy days. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: perl scripting and the new API with amanda-3.1.0
First, I'm excited you're using the modules, and thanks for the feedback! On Wed, Jun 16, 2010 at 11:50 AM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Using Amanda::Config.pm as an example I tried: config_init($CONFIG_INIT_EXPLICIT_NAME, Config_Name); $ss = lookup_changer_config(av48); $slot_range = changer_config_getconf($ss, use-slots); but that obviously fails. Notice my lack of perl-fu :) I think the only missing aspect in the above is that instead of use-slots, you'll need to use $CHANGER_CONFIG_USE_SLOTS, which is available with use Amanda::Config qw( :constants ); I didn't document these constants in the POD because there's no automatic way to make sure that conffile.c/h and the POD will stay in sync, and we're fabulously bad at remembering to update the same thing in two places. They're all available and pretty readable in common-src/conffile.h. It occurs to me that the necessary string tables exist to also make things work the way you've expected them to work. Unless there's s SWIG expert in the crowd (where's Dave Beazley when you need him?) who wants to take a swing at that, I'll see what I can cook up. There seems to be a few typos in the pod for Config.pm btw. Patches and/or specific pointers would be welcome! http://wiki.zmanda.com/index.php/Fork_Amanda_on_Github Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: perl scripting and the new API with amanda-3.1.0
On Wed, Jun 16, 2010 at 2:48 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Are you sure about that? I don't see any $CHANGER_CONFIG_USE_SLOTS in 3.1.0... In the mean time I resorted to slurp all the changer properties with $tpc_prop = changer_config_getconf($ss, $CHANGER_CONFIG_PROPERTY); and parse the hash until I hit use-slots = values Not pretty but I'm in a hurry to update my scripts to work again with amanda 3.1.0 As Jean-Louis said, that's correct. I answered too quickly. You can see the format of the properties described in the DATA FORMATS section, as you've probably already discovered. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Identifying what file is at the start of a tape for audit purposes
On Tue, Jun 15, 2010 at 8:53 AM, Mark Adams m...@campbell-lange.net wrote: Is there any way to identify (via the index or other means) what file is at the start of what tape or what split part? You can use 'amadmin find' I suppose.. although it doesn't let you search by tape. I need to audit my tape sets to ensure data has been backed up correctly, I was anticipating trying to retrieve a single file from each tape to confirm this. Does anyone have any suggestions on how best to do this or is there any automated way to do this with Amanda? amcheckdump is probably what you want. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Identifying what file is at the start of a tape for audit purposes
On Tue, Jun 15, 2010 at 10:55 AM, Mark Adams m...@campbell-lange.net wrote: Hi Dustin, Thanks for this pointer. I assume amcheckdump takes just as long as the backup takes in the first place? Within an order of magnitude, yes. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
On Mon, Jun 14, 2010 at 1:02 AM, Marc Muehlfeld marc.muehlf...@medizinische-genetik.de wrote: This second patch fixes the problem. Thanks. Great - thanks! Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: auth question
On Mon, Jun 14, 2010 at 10:54 AM, McGraw, Robert P rmcg...@purdue.edu wrote: If I compile with --with-bsdtcp-security and bsd, can my 2.6.1p2 server use bsd security for some clients and bsdtcp for other clients and would I set this in the define dumptype? Absolutely. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda 3.1
On Mon, Jun 7, 2010 at 6:39 AM, Wojciech Giel wojciech.g...@cimr.cam.ac.uk wrote: Could you provide package for ubuntu server 10.4 LTS amd64 and x86? You should be able to run packaging/deb/buildpkg on such a system, to create the packages yourself. At the moment, we don't have a 10.4 build system set up. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re:
On Mon, Jun 14, 2010 at 3:09 PM, McGraw, Robert P rmcg...@purdue.edu wrote: I am not quite sure how amanda uses the --with-tcpportrange=low/high. Does this parameter have to be defined in both the client and server software or does the server pass the ranges to the client? It needs to be defined separately in both, but note that it is now runtime-configurable, so the ./configure option is only useful for setting defaults. Dustion -- Open Source Storage Engineer http://www.zmanda.com
Re: Recovering from a dead vtape drive
On Mon, Jun 14, 2010 at 5:49 PM, Charles Curley charlescur...@charlescurley.com wrote: This morning, the disk drive that holds my vtapes dies on me. I was able to recover. My notes are at http://www.charlescurley.com/blog/archives/2010/06/14/offsite_backups_arent_just_for_disaster_recovery/index.html That sounds like an un-fun change of plans for your day, but I'm glad you made lemonade out of those lemons. This would be a great story to add to the wiki - maybe we should have a Success Stories section? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: The canary just died again
On Sat, Jun 12, 2010 at 12:05 PM, Gene Heskett gene.hesk...@gmail.com wrote: And 3161 is still confused. ;-) In the final mix, how is this supposed to work?, and I'll change my 2 scripts accordingly. I think you're just pulling my chain now, but I'll go again: Right, I haven't fixed it yet. The patch is in review - there's some lingering discussion of the proper permissions for amrecover and oldamrecover. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: The canary just died again
On Sat, Jun 12, 2010 at 1:48 PM, Gene Heskett gene.hesk...@gmail.com wrote: Is it intentional now that amcheck, amplanner and a couple others are now owned by root:root? No, they should be owned by root:disk (or root:backup, depending on your system), and setuid. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: The canary just died again
On Fri, Jun 11, 2010 at 12:10 AM, Gene Heskett gene.hesk...@gmail.com wrote: At the end of the install, it falls over with: make[1]: Leaving directory `/home/amanda/amanda-3.2.0alpha.svn.3147' bash: /usr/local/sbin/amcheck: Permission denied And sure enough, when I go check, amcheck is owned by root, not amanda. Wow, I did *not* mean to commit that patch yet. Oops! That said, I wanted to commit it *eventually*, so I'll take a look at exactly what's gone wrong. We've historically chown'd just about everything to the amanda user, and this isn't really necessary, and complicates packaging. So I'm trying to strip down the permissions to the bare essentials. Obviously I got it wrong. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
On Fri, Jun 11, 2010 at 3:49 AM, Marc Muehlfeld marc.muehlf...@medizinische-genetik.de wrote: header on 'KDD174' file 29 does not match expectations: got partnum '0'; expected '1' Got no header and data from server, check in amidxtaped.*.debug and amandad.*.debug files on server Ah, this is completely different error, but you've given me enough info to replicate and fix it. I've attached a source-level patch that you can apply, but the better fix is at the C level (and will be in 3.1.1). FYI, what's happened here is that, in Amanda-3.1, we no longer produce dumps with the DUMPFILE header - every header is a SPLIT_FILE header, even if it only has one part. However, Amanda-2.6.* wrote DUMPFILEs, and when those are read back in it looks like they have partnum 0, which is not what amidxtaped is expecting. Let me know if this patch works? And *thanks* for your patience and testing! Dustin -- Open Source Storage Engineer http://www.zmanda.com marc2.patch Description: Binary data
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
On Fri, Jun 11, 2010 at 1:41 PM, Dustin J. Mitchell dus...@zmanda.com wrote: Ah, this is completely different error, but you've given me enough info to replicate and fix it. I've attached a source-level patch that you can apply, but the better fix is at the C level (and will be in 3.1.1). For those following along at home (you *are* watching all of the commits to Amanda, right??), the better patch is here, replete with a regression test: http://github.com/djmitche/amanda/commit/94e2697e8fdb22c4452bb0ac6e2fd7bd74cf983b Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: The canary just died again
On Fri, Jun 11, 2010 at 2:25 PM, Gene Heskett gene.hesk...@gmail.com wrote: Ok, but todays 3156 just fell over, same perms problem. Right, I haven't fixed it yet. Are you sure you want to change how that works? Yes - chowning everything to amanda is actually *less* secure (since it means that someone who compromises the Amanda account could modify those executables) But it would be great if something can be made to work under an rpm or dpkg install without throwing the baby out with the bathwater because rpm (don't know about the deb scene) can't do the ownership and suid bits needed. I don't know about that - Amanda's been successfully packaged in RPMs for a while now. If you'd like to start up another thread on that topic, that'd be fine. Or put the distro maintainers in touch with Dan Locks. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
OK, this was apparently a failure of my Perl-fu. I had written something like
my $check = sub {
# determine if $_ is suitable
};
@list = grep $check, @other_list;
Thinking that Perl would helpfully invoke $check. It won't - it just
looks at the boolean value of $check, and it's a nonzero scalar so
that's true.
Attached is a patch - can you try it out? If you need a fresh copy of
Planner.pm, you can get it here:
http://github.com/zmanda/amanda/raw/tags/3_1_0/perl/Amanda/Recovery/Planner.pm
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
marc.patch
Description: Binary data
Re: Cleanup vtape space Amanda-2.6.1
On Wed, Jun 9, 2010 at 12:09 AM, Dimiter Maznekov d.mazne...@optixco.com wrote: After long long holliday I'm returning in company and see this message: FAILURE DUMP SUMMARY: www.site.org /var/www/htdocs/ISU lev 1 FAILED [Not enough holding disk space] www.site.org /var/www/htdocs/ISU lev 1 FAILED [out of holding space in degraded mode] Partition vtapes are full of dumps. I'd not run for a long time amflush to flush it on the tape drive due to 2 month holliday. My question is - how to clean up old and unecessery data in vtapes ? I think you mean how to clean up old and unnecessary data in holding disk, right? You can use amadmin holding delete, as you suggested. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
On Wed, Jun 9, 2010 at 9:50 AM, Marc Muehlfeld marc.muehlf...@medizinische-genetik.de wrote: Am 09.06.2010 16:43, schrieb Dustin J. Mitchell: The messages will have gone into amidxtaped.*.debug.. But just the following files were created, until the amrecover - can't talk to tape server: service amidxtaped: message came up: Ah, there must be a typo in the amidxtaped changes. Try running amidxtaped directly from the command line - it will probably give you a syntax error of some sort, and hopefully something with a fairly obvious fix? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amanda-backup_client conflicts with amanda-backup_server-3.1.0-1.rhel5.i386
I *think* this is the same issue being addressed here: http://forums.zmanda.com/showthread.php?t=2800 Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: After upgrading to 3.1.0: ETIMEOUT must be positive
On Tue, Jun 8, 2010 at 3:58 PM, Marc Muehlfeld marc.muehlf...@medizinische-genetik.de wrote: ETIMEOUT must be positive But the manpage says: etimeout int . A negative value will be interpreted as a total amount of time to wait per client instead of per disk. Who's wrong? Manpage or program code? Well, it was changed in this commit: commit 1a2051ece2e27fea65b9bf05dac235a7e358f52a Author: Jean-Louis Martineau martin...@zmanda.com Date: Thu Dec 17 17:41:01 2009 + * common-src/conffile.c: etimeout must be positive. * common-src/protocol.c: Fix possible timeout of 0. git-svn-id: https://amanda.svn.sourceforge.net/svnroot/amanda/amanda/tr...@2358 a8d146d6-cc15-0410-8900-af154a0219e0 I don't remember the motivation for the change, though - maybe JLM does. But the upshot is that it looks like the manpage was not properly updated. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Can't recover with 2.6.1p2 client from 3.1.0 server
On Tue, Jun 8, 2010 at 3:58 PM, Marc Muehlfeld
marc.muehlf...@medizinische-genetik.de wrote:
Tue Jun 8 16:39:14 2010: amidxtaped: CTL LABEL=KDD174:29
..
Tue Jun 8 16:39:18 2010: amidxtaped: CTL MESSAGE Specified file list
matches multiple dumps; cannot continue recovery
So what's happening here is that amidxtaped is looking for the
dumpfile at that particular label:filenum, and then making sure it
finds all of the required parts of that dump. The error occurs when
the label:filenum pairs specified cover more than one dumpfile. With
only one filenum specified, it shouldn't be able to find more than one
dump.
Do you happen to have multiple volumes with that label? That would be
weird, but worth asking.
Could you edit up perl/Amanda/Recovery/Planner.pm as follows:
400 my @filelist = @{$params{'filelist'}};
401 while (@filelist) {
402 my $label = shift @filelist;
Amanda::Debug::debug(LABEL $label);
403 push @labels, $label;
404 $files{$label} = shift @filelist;
Amanda::Debug::debug(FILES . join( , @{$files{$label}}));
405 }
...
419 my %dumps = map { my $d = $_-{'dump'}; ($d, $d) } @parts;
use Data::Dumper;
Amanda::Debug::debug(.Dumper([ %dumps ]);
420 my @dumps = values %dumps;
and try again?
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: timeout waiting for REP
On Tue, Jun 1, 2010 at 9:39 AM, McGraw, Robert P rmcg...@purdue.edu wrote:
I am seeing several of the following dumper: [request failed: timeout
waiting for REP](too) errors when I run the amstatus command. Below is a
snippet of one of the errors.
Is the (too) really part of the error?
Can someone tell me what might be causing this error?
Timeouts?
1275368274.080161: dumper: (sockaddr_in *)fef206c4 = { 2, 10080,
128.210.3.201 }
1275368334.089824: dumper: security_seterror(handle=807bc10, driver=fef0ad20
(BSD) error=timeout waiting for REP)
That's 60 seconds.. as I've said *many* times, the BSD auth is from
the stone age. Use something at least slightly newer, like BSDTCP!
Dustin
P.S. Despite a lot of voiced interest, nobody is working on the ssl
auth at the moment. Anyone want to pick that project up?
--
Open Source Storage Engineer
http://www.zmanda.com
Re: perl -pthreads questions
On Mon, Jun 7, 2010 at 4:30 PM, McGraw, Robert P rmcg...@purdue.edu wrote: I have search for this information but cannot find any information. Is this an error? If so what is the resolution? How do I make perl link with -pthreads? Nope, it's just Make showing you the command it would invoke if there were an error :) Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: I see a file core20100603
On Thu, Jun 3, 2010 at 10:23 AM, McGraw, Robert P rmcg...@purdue.edu wrote: In /tmp/amanda I see for the first time a file core20100603 what is this file? It's a core file. Depending on your OS configuration, it can be created when a process is killed by a fatal error like a segfault. They're usually not particularly helpful, but can be used locally with a debugger to explore the state of the process just before it died. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: I see a file core20100603
On Thu, Jun 3, 2010 at 12:42 PM, McGraw, Robert P rmcg...@purdue.edu wrote: Usually core files have just the name core I have not seen any core files with the date attached and the fact that it was in /tmp/amanda and owned by amanda I thought it might be some file created by amanda. Yeah, Amanda has some (presumably ages-old) functionality to do that in common-src/file.c. I've always just ignored that code, but I doubt it's doing anyone any good and I'd be happy to remove it if you think it's problematic. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amgtar question
On Thu, Jun 3, 2010 at 3:25 PM, McGraw, Robert P rmcg...@purdue.edu wrote: Which seem to resolve the problem. I'm not sure what the question was .. but amgtar in 2.6.1 didn't have an ACLS property: http://wiki.zmanda.com/man/2.6.1/amgtar.8.html Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Using multiple changers
Spam detection software, running on the system becherovka.cz.omniscient.com, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see postmaster for details. Content preview: On Wed, Jun 2, 2010 at 11:06 AM, Douglas K. Rand r...@meridian-enviro.com wrote: I have 3 AIT changers (30 slot SpectraLogic TreeFrogs) each with 2 tape drives that right now I use with Amanda 2.6 with 3 different amanda configurations to be able to use all 3 changers in parallel. (And I'm currently only using 1 tape drive per changer.) With Amanda 3.1 is it possible for me to setup a single Amanda configuration to use all 3 changers in parallel in a round robin fashion? (I'd rather avoid striping with RAIT.) That would avoid the hassle of having to tune by hand which DLEs are in which Amanda configuration, and make it easier for me to not miss a DLE. I think I see how to use multiple tape drives on a single changer with the DRIVE-CHOICE property. But I don't see how to use multiple changers with a single Amanda config. [...] Content analysis details: (5.6 points, 5.0 required) pts rule name description -- -- 2.4 DNS_FROM_OPENWHOIS RBL: Envelope sender listed in bl.open-whois.org. 3.2 PERCENT_RANDOM Message has a random macro in it ---BeginMessage--- On Wed, Jun 2, 2010 at 11:06 AM, Douglas K. Rand r...@meridian-enviro.com wrote: I have 3 AIT changers (30 slot SpectraLogic TreeFrogs) each with 2 tape drives that right now I use with Amanda 2.6 with 3 different amanda configurations to be able to use all 3 changers in parallel. (And I'm currently only using 1 tape drive per changer.) With Amanda 3.1 is it possible for me to setup a single Amanda configuration to use all 3 changers in parallel in a round robin fashion? (I'd rather avoid striping with RAIT.) That would avoid the hassle of having to tune by hand which DLEs are in which Amanda configuration, and make it easier for me to not miss a DLE. I think I see how to use multiple tape drives on a single changer with the DRIVE-CHOICE property. But I don't see how to use multiple changers with a single Amanda config. This is currently not supported, but a round-robin changer would be an interesting addition to the suite of changers we already have. It would be similar to the current chg-rait, but of course not producing RAIT devices. Perhaps you'd be interesting in adding such a thing? Dustin -- Open Source Storage Engineer http://www.zmanda.com ---End Message---
Re: New options autolabel Amanda 3.1.0bet2
On Fri, May 28, 2010 at 4:19 AM, Gunnarsson, Gunnar gunnar.gunnars...@svk.se wrote: I got the fatal error see below while testing the new option autolabel, the configuration was empty and amcheck -s was ok. Does the tapelist file exist? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: New options autolabel Amanda 3.1.0bet2
On Fri, May 28, 2010 at 9:12 AM, Dustin J. Mitchell dus...@zmanda.com wrote: Does the tapelist file exist? Ah! A while back, we made a missing tapelist a warning in amcheck, but the taper was not updated to automatically create it in this case. So you'll need to create your tapelist first - then things will work. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amrecover from 2.5.0p2 using tape_splitsize
On Tue, May 25, 2010 at 9:48 AM, Charlie Reitsma reit...@denison.edu wrote: I saw a thread that recommended trying amindexd, amidxtaped, amoldrecover from 2.5.1b2. That was reported successful with no configuration changes. I wonder if anyone could flesh that out a bit more for me? Yikes, I'm not sure that's a good idea! I compiled 2.5.1b2 and installed in /usr/local. Am I copying amindexd and amidxtaped from /usr/local/sbin into /usr/sbin on the server? Do I repeat the compile on each client and use amoldrecover instead of amrecover? I would recommend upgrading the entire client-side or server-side all at once, rather than piecemeal. There were some protocol changes between 2.5.0 and 2.5.1, so you may run into some problems there. See amanda-compatibility(7) for details. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Tue, May 25, 2010 at 11:33 AM, Gunnarsson, Gunnar gunnar.gunnars...@svk.se wrote: I've tested the tape drivers and they seems to be working fine, how does drivers detect end of tape. This seems to be a specific Solaris issue ? Amanda detects EOT by counting any error as an EOT indication. What's happening in the current version of Amanda is that that error condition (internal to the Amanda code, not in the kernel) is not being cleared, so the device refuses to do anything else, such as rewinding and rewriting the tape label. I've fixed it up, and run a complete test on one of my tape drives. The fix is here: http://github.com/djmitche/amanda/commit/amtapetype and will (I hope) be in the much-delayed 3.1 release. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amanda backup of virtual machine under citrix
On Tue, May 25, 2010 at 12:05 PM, Brian Cuttler br...@wadsworth.org wrote: We seem to be missing something in our config but I'm not finding the key in the wiki. What should I be looking for ? I'm guessing that amanda/tcp isn't in your services db? On most systems, that's /etc/services, but on Sun it might be via NIS or something. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amanda backup of virtual machine under citrix
On Tue, May 25, 2010 at 12:44 PM, Chris Hoogendyk hoogen...@bio.umass.edu wrote: I don't mean to question Dustin I rather enjoy being questioned, so don't worry about that! This was a guess, and apparently I was correct this time around. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amrecover from 2.5.0p2 using tape_splitsize
On Tue, May 25, 2010 at 2:28 PM, Charlie Reitsma reit...@denison.edu wrote: The two times I've tried to move to current releases I've run into trouble and had to back out. Bringing all 24 clients and server at once is a bit daunting. I think I'll have to scrounge up another system and tape drive to practice. It's generally best to upgrade the server first, then migrate clients one by one. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Sat, May 22, 2010 at 5:46 AM, Gunnarsson, Gunnar gunnar.gunnars...@svk.se wrote: Wrote 406883762176 bytes at 7552 kb/sec Writing smaller files (4068835328 bytes) to determine filemark. amtapetype: Error writing label 'amtapetype-2046333855': Error writing block: Mysterious short write on tape device: Tried 32768, got 0 at /opt/sfw/sbin/amtapetype line 89. Ah, it's a similar problem as in the previous case, then: the device goes into an error state, and that state sticks. Probably the better solution is to re-create the device as necessary. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
Ah - after the device has encountered an error, it won't give
properties anymore (case doesn't matter). Try moving the block_size
up in the script:
diff --git a/device-src/amtapetype.pl b/device-src/amtapetype.pl
index 5bd7411..80c0ad7 100644
--- a/device-src/amtapetype.pl
+++ b/device-src/amtapetype.pl
@@ -117,6 +117,9 @@ sub write_one_file(%) {
my $pattern = $options{'PATTERN'} || 'FIXED';
my $max_time = $options{'MAX_TIME'} || 0;
+# get the block size now, while the device is still working
+my $block_size = $device-property_get(block_size);
+
# start the device
my $hdr = Amanda::Header-new();
$hdr-{type} = $Amanda::Header::F_DUMPFILE;
@@ -182,7 +185,6 @@ sub write_one_file(%) {
# OK, we finished, update statistics (even if we saw an error)
my $blocks_written = $device-block();
-my $block_size = $device-property_get(block_size);
$stats-{$pattern}-{BYTES} += $blocks_written * $block_size;
$stats-{$pattern}-{FILES} += 1;
$stats-{$pattern}-{TIME} += $duration;
Let me know if that helps? Thanks for your patience!
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Thu, May 20, 2010 at 3:00 AM, Gunnarsson, Gunnar gunnar.gunnars...@svk.se wrote: Applying heuristic check for compression. Wrote random (uncompressible) data at 7492244.64516129 bytes/sec Wrote fixed (compressible) data at 35732243.6923077 bytes/sec Compression: enabled Writing one file to fill the volume. File 1, block 6910315 amtapetype: Wrote less than 1MB to the device: Error writing block: Mysterious short write on tape device: Tried 32768, got 0 That's odd.. it looks like it wrote 6910315 blocks, yet when it multiplied that by the block size, it got something less than 1MB. Did this take a long time, or happen immediately? Can you add some print's around line 186 of amtapetype to print the blocksize and number of blocks? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Wed, May 19, 2010 at 7:27 AM, Gunnarsson, Gunnar gunnar.gunnars...@svk.se wrote: Thanks Dustin the patch is working fine, good work. Great! Compression enabled, does it mean that it is using compression ? Yep. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Tue, May 18, 2010 at 5:26 AM, Mark Adams m...@campbell-lange.net wrote: You need to specify -c with pbzip2 to have it compress to stdout I believe. If you just run it by itself; Ah! Then, for the moment at least, you'll need to write a wrapper script that doesn't require any arguments. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Tue, May 18, 2010 at 7:53 AM, Gunnarsson, Gunnar
gunnar.gunnars...@svk.se wrote:
Tue May 18 13:22:08 2010: amtapetype: Device /dev/rmt/0n error = 'Could not
seek to file 2 correctly; got 1'
Hmm.. can you send me the syscall trace (truss or strace) of that?
Here's the comment above that error message:
1277 /* double-check that we're on the right fileno, if possible.
This is most
1278 * likely a programming error if it occurs, but could also be
due to a weird
1279 * tape drive or driver (and that would *never* happen, right?) */
1280 got_file = tape_fileno(self-fd);
1281 if (got_file = 0 (guint)got_file != file) {
1282 device_set_error(d_self,
1283 vstrallocf(_(Could not seek to file %d
correctly; got %d),
1284 file, got_file),
1285 DEVICE_STATUS_DEVICE_ERROR);
1286 d_self-file = (guint)got_file;
1287 return NULL;
1288 }
So this might be an odd behavior of the tape device.
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: amtapetype Amanda 3.1.0beta2
On Tue, May 18, 2010 at 9:37 AM, Dustin J. Mitchell dus...@zmanda.com wrote: Hmm.. can you send me the syscall trace (truss or strace) of that? (truss now in hand..) Oh, silly me - the code detects that the tape drive needs the FSF_AFTER_FILEMARK property, by virtue of getting an error from the device .. but then fails on that very error. This should be pretty easy to fix - try this patch: http://github.com/djmitche/amanda/commit/fsf_after_filemark.patch You can probably apply that in-place without too much trouble. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Wed, May 12, 2010 at 12:54 PM, Mark Adams m...@campbell-lange.net wrote: 1273659791.836703: sendbackup: critical (fatal): index tee cannot write [Broken pipe] This means that the index tee (which splits off the 'tar' output to generate the index) cannot write to its output, which is the client-side compression. Since the error is EPIPE, this means either that pbzip2 exited, or that it closed its standard input prematurely. The next step would be to figure out why pbzip2 would do that. Does it automatically compress its stdin and pipe it to stdout? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Wed, May 12, 2010 at 4:39 AM, Mark Adams m...@campbell-lange.net wrote: This was the problem. Using gzip it retrieves from the 2nd tape just fine. It takes twice as long to run the backup though! Is anyone using pigz and is successfully retrieving data from the 2nd tape on? or has anyone used pbzip2? Interesting! We're hoping to implement official support for this in the next (3.2) release, so I'm curious to know more about why pigz failed. In theory, it's a drop-in gzip replacement, right? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Wed, May 12, 2010 at 11:17 AM, Mark Adams m...@campbell-lange.net wrote: In the mean time I can't get pbzip2 to work at all. It just crashes out with the following: Please attach the whole sendbackup log. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: taper segfault (BUG: Bad page map in process taper)
On Tue, May 11, 2010 at 4:55 PM, Dustin C. Hatch dha...@gosupertechs.com wrote: I update to Kernel version 2.6.32-r7 last night. I was able to do a full backup to tape earlier today, using tapes that I know failed before. I guess there could have been a driver problem in 2.6.31. I'll try a larger DLE later and see how it goes. If you can narrow it to a range of kernel versions, I'd appreciate it. If you haven't already, please open a gentoo bug and copy me (dus...@v.igoro.us) on it. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Fri, May 7, 2010 at 4:22 AM, Mark Adams m...@campbell-lange.net wrote: Does this help at all? as it read the 2nd tape does this mean the data is on the tapes and it's a problem with amrecover? Basically. It could mean that the data on the tapes is corrupt, although the basic structure of the data is intact (that is, the filemarks are in the right place and the headers are right). Also for future ref, do I actually need around 1.5* the space of the DLE to do amfetchdump? (to include the merging..) I don't recall which version you're using. This is no longer the case in 3.1. I'm going to try now using plain gzip instead of pigz, just incase this is causing my issues. It's quite possible.. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: how to eject a rait volume?
On Fri, May 7, 2010 at 12:03 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Dustin, educate me please! How do I proceed with the git patches and how do I patch the source on my side? First, if you click on each of those commits in github, you'll have URLs like http://github.com/djmitche/amanda/commit/29f7aee9b0ecb9cb7ec6d0b2faa17ecde3168e95 and you can add a .patch to the end to get a regular old patch file. Fetch that with wget or whatever you'd like. I just re-pushed to that branch with a single patch, so that should make things easier for you. I don't recall what OS you're using, or how you're installing Amanda. If it's from a source tarball, you should be able to just apply the patches: patch -p1 /path/to/patch.patch if you need to rebuild a package, things are a little bit trickier. Basically, you should get the beta2 tarball, apply the patch as above, and then run ./packaging/$flavor/buildpkg, where $flavor is the appropriate flavor for your operating system (rpm, deb, ..). This should Just Work, but of course it's tricky and won't always be effective. The trickiest, but in the end easiest, solution is to patch the robot.pm and rait.pm files in place. This omits the manpage update and the changes to the RAIT device, but neither of those are critical to fixing the bugs you've encountered. Edit out the appropriate sections of the patch for each of those two files, and then use patch /path/to/robot.pm.patch patch should then say it can't find robot.pm, and you can give the path to it (/usr/lib/amanda/perl/Amanda/Changer/robot.pm, maybe - depends on your OS). If you're on Solaris, you may want to use gpatch - Sun's patch is awful. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: how to eject a rait volume?
On Fri, May 7, 2010 at 1:13 PM, Jean-Francois Malouin
ma...@bic.mni.mcgill.ca wrote:
I see a lot of warning in the debug file:
Fri May 7 13:54:34 2010: amtape: warning: Use of uninitialized value in
bitwise or (|) at
/opt/amanda-3.1.0beta2/perl/Amanda/Changer/rait.pm line 596.
These are harmless and already fixed:
http://github.com/zmanda/amanda/commit/9861a8362a481f0625ee6e6be2fa352e82690ee9
~# su amanda -c /opt/amanda-3.1.0beta2/sbin/amtape rait eject {0,1}
ERROR: av48-half-2: this changer instance is not configured to access drive 0
I hate to say this, but shell expansion is rewriting that to
# su amanda -c /opt/amanda-3.1.0beta2/sbin/amtape rait eject 0 1
and Amanda is trying to eject drive 0, equivalent to {0,0}, so the
second half of your RAIT is complaining. Use single-quotes around the
{..} and you should be fine.
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: request: script for generating DLEs
On Fri, May 7, 2010 at 5:35 PM, Stefan G. Weichinger s...@amanda.org wrote: It's a DRAFT! Looks good so far, though! Will you be adding this to contrib/ when it's ready? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: how to eject a rait volume?
On Thu, May 6, 2010 at 12:00 PM, Jean-Francois Malouin
ma...@bic.mni.mcgill.ca wrote:
How do you specify a drive number with a mirror rait using chg-robot?
~# su amanda -c /opt/amanda-3.1.0beta2/sbin/amtape rait eject 1
You would specify it as '{1,12}', but it looks like you've uncovered a
bug there all the same, since '1' is treated as '{1,1}'.
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: client timeout issue
On Thu, May 6, 2010 at 1:44 PM, Brian Cuttler br...@wadsworth.org wrote: Amanda is now failing differently. tar exited with signal 11. On Solaris, over a ZFS filesystem. We just saw this error a few days ago on the list! Check the archives - there was a workaround. -- Open Source Storage Engineer http://www.zmanda.com
Re: how to eject a rait volume?
On Thu, May 6, 2010 at 1:28 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: I must have left my brain somewhere but it's still not clear to me how to specify a `drive' with a mirror/rait: No, this is a good question that didn't have an answer! I've answered it, and my patch is pending JLM's review (so it will probably be committed tomorrow). Here are the (3) patches: http://github.com/djmitche/amanda/commits/raitfixes Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Wed, May 5, 2010 at 3:55 AM, Mark Adams m...@campbell-lange.net wrote: Why would the file not be in the archive if it is showing in the index? Is there anything else I can try before I try to retrieve the whole DLE? So this is a new dump? No, I can't see any reason it would do that. Hopefully retrieving the entire DLE will offer some clues (and get you access to your data). Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: client timeout issue
What is the error? I suspect this: 1273071472.718696: dumper: security_seterror(handle=8074f08, driver=fef09394 (BSD) error=timeout waiting for REP) which likely means that you're exceeding the dtimeout for that DLE for whatever reason. Try increasing that? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: client timeout issue
On Wed, May 5, 2010 at 1:38 PM, Brian Cuttler br...@wadsworth.org wrote: Also, the issues seem to be specific to certain DLE, in both cases the busiest of the DLEs, not necessarily the largest but the most active. It's clearly timing-related, then. Keep in mind that hte dtimeout is a timeout without *any* data, so I suspect that this is some kind of ugly interaction between ZFS busily doing copy-on-write to implement its snapshot while trying to feed data to the syscall-hungry gnutar. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: mirror rait, one changer
On Wed, May 5, 2010 at 2:10 PM, Jean-Francois Malouin
ma...@bic.mni.mcgill.ca wrote:
I've corrected this by using the same state file in both
child-changer. I'm now experiencing other problems, maybe due to some
mis-config from my part:
~# su amanda -c /opt/amanda-3.1.0beta2/sbin/amtape rait taper
slot ?: from av48-half-1: slot 14 not in use-slots (1-12)
slot {1,13}: volume 'RAIT-001.L4' is still active and cannot be overwritten
slot {2,14}: volume 'RAIT-002.L4'
Will write to volume RAIT-002.L4 in slot {2,14}.
What's the 'slot ?:' thingie?
The slot ?: itself is an artifact of the way the output is printed -
it's a general error not corresponding to any particular slot.
Wed May 5 14:12:02 2010: amtape: warning: Use of uninitialized value in
bitwise or (|) at
/opt/amanda-3.1.0beta2/perl/Amanda/Changer/rait.pm line 572.
Harmless, but I'll fix it.
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: runtar error that I do not understand
On Tue, May 4, 2010 at 9:33 AM, McGraw, Robert P rmcg...@purdue.edu wrote: Is there someone in the zmanda world that has the following amanda setup. 1) Solaris x86 Solaris 10 2) /usr/sfw/bin/gtar (v1.23) 3) amanda 2.6.1p2 4) amanda backup of a zfs snapshot like /path/.zfs/snapshot/path/to/dir I am getting a seg fault when gtar tries to get the size of the zfs snapshot directory. I believe it has to do with .zfs as a hidden directory and I am not sure how to get around this. (since you mentioned Zmanda specifically..) We don't have a ZFS setup at the moment, and in fact our Sparc systems are fairly limited since they can't be fully virtualized. I'm borrowing time on some Solaris systems from a friend today, so if I get a chance and can set up ZFS, I'll give it a whirl. That said, I suspect that if the bug were as large as gtar on a ZFS snapshot segfaults, it would be fixed already, so the circumstances are likely more narrow. Have you checked the Solaris bug tracker? Do your system logs give any indication of what's going on? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: runtar error that I do not understand
On Tue, May 4, 2010 at 12:27 PM, McGraw, Robert P rmcg...@purdue.edu wrote: I setup a lookback for the snapshot and now gtar seem to be working. It sounds like you have a fairly good understanding of this problem now. Could you write up either a troubleshooting or How-To article on the wiki? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: mirror rait, one changer
On Tue, May 4, 2010 at 1:13 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Is it possible with amanda-3.0 and the new changer API to setup a mirror/rait using 2 physical tape drives and only one physical changer? Yes! I tested this lightly at one point, and it should work just fine. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: mirror rait, one changer
On Tue, May 4, 2010 at 4:44 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: One question: should the changerfile parameter for chg-robot point to the same file for both child-changers: Yes, it should - that will allow the changers to avoid stepping on each other. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda 3.1.0beta2 released
On Tue, Apr 27, 2010 at 1:59 PM, Dustin J. Mitchell dus...@zmanda.com wrote: Please download and test it out - with luck, this will be the last beta before the final release! We haven't heard of any showstopper bugs, although we have discovered and fixed a number of nuisances since the beta2 release. We're nearing the official release, and that means two things: 1. This is very nearly a finished' release, so we consider it fairly safe to run 2. We need a lot of testing in real situations to flush out the last few bugs I've heard privately from a few people who are testing (thanks!), but I won't feel comfortable until I hear from a few more. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: runtar error that I do not understand
On Mon, May 3, 2010 at 2:01 PM, McGraw, Robert P rmcg...@purdue.edu wrote: hertz /export/jumpstart lev 0 FAILED [/local/amanda/amanda-2.6.1p2/libexec/amanda/runtar terminated with signal 11: see /tmp/amanda/client/archive/sendsize.20100503144020.debug] signal 11 is a segfault. The last line of the 'runtar' logfile shows that it invokes dbclose, and the very next statement is an execve, which is failing. I'm not sure exactly why this would be happening, unless GNUTAR is undefined in your build? Anyway, the segfault is either coming from the execve, or from the debugging code that follows it. However, I just forced an error from execve, and the debugging code worked fine. And I don't immediately see anything wrong with it. Try invoking runtar the same way that you see in the sendbackup logfile (with suitable redirection of stdout), and see if it does the same thing. If so, try running it under 'truss' to see what is wrong with the execve call. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amrecover 3.1.0beta2 with dumps spanning tapes
On Mon, May 3, 2010 at 3:45 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Looks good to me now! I'va attached the debug files for amidxtaped amindexd and amrecover as requested. Actually, Jean-Louis managed to solve this while I was waiting for the debug info. You can (at your option) remove the debugging prints I had you add to amidxtaped.pl. I'm glad we chased this down! Jean-Louis has already committed the corresponding patch (r2978). Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Tape library with 24 slots
On Mon, May 3, 2010 at 4:06 PM, Angelo Höngens a.hong...@netmatch.nl wrote: Just for your info, this is how it looks on my bsd box. Here, it's called portal 0 instead of slot 24.. Hah, I like the term portal! If you look at the underlying SCSI protocol, drives, i/e slots, regular slots, and robot arms are all numbered in the same namespace, generally with different offsets -- so the drives may start at 512, the slots at 1024, the robot arm at 0, and the i/e slots at 1536, for example. Mtx remaps these consistently to count drives starting at 0 and, separately, slots starting at 1. It puts any i/e slots after the regular slots. Chio does about the same thing, but its mapping is different. As to the original question, I suspect it's a bug in chg-zd-mtx - maybe try changing lastslot to 25? Amanda-2.5.0 is very old, and we've patched chg-zd-mt many times since then - in fact, it's deprecated in 3.1 and will probably get the axe in 3.2. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: runtar error that I do not understand
On Mon, May 3, 2010 at 4:31 PM, McGraw, Robert P rmcg...@purdue.edu wrote: If I manually run the following gtar command I get a segmentation fault /usr/sfw/bin/gtar --create Uh-oh .. time to open a Sun^WOracle support case? It looks like a bug either in ZFS or GNU Tar.. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: amrecover 3.1.0beta2 with dumps spanning tapes
On Fri, Apr 30, 2010 at 5:49 AM, Jean-Francois Malouin
ma...@bic.mni.mcgill.ca wrote:
no, the dump used 2 vtapes, vtape-1 and vtape-2, and there is only one
`drive'. I did 2 run with amrecover just to convince me that it really
asked for vtape-2 *before* vtape-1. Since I have `amrecover_changer'
set I just hit return to see what would happen and vtape-1 got loaded
first.
Actually, looking at the amrecover output:
amrecover list
TAPE vtape-2:3 LEVEL 0 DATE 2010-04-28-17-12-43
and the amidxtaped logfile:
Thu Apr 29 16:13:31 2010: amidxtaped: CTL LABEL=vtape-2:3
the particular dump you're looking for is only on vtape-2, filenum 3.
However, what's odd is that amidxtaped then goes on to look for vtape-1:
Thu Apr 29 16:13:31 2010: amidxtaped: find_volume labeled 'vtape-1'
It looks like there is some disagreement between amindexd and
amdixtaped as to which volumes contain this file. Can you apply this
temporary patch:
--- a/server-src/amidxtaped.pl
+++ b/server-src/amidxtaped.pl
@@ -428,6 +428,8 @@ sub plan_cb {
my $self = shift;
my ($err, $plan) = @_;
+use Data::Dumper;
+Amanda::Debug::debug(Dumper($plan));
if ($err) {
$self-sendmessage($err);
return $self-quit();
and re-run, pasting the new amidxtaped debug log. Also, please send
the output of
amadmin test find gustav /raid/hrrt/SCS_SCANS
Thanks!
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: amrecover 3.1.0beta2 with dumps spanning tapes
On Thu, Apr 29, 2010 at 1:37 PM, Jean-Francois Malouin ma...@bic.mni.mcgill.ca wrote: Using amrecover to extract a multi virtual tape dump I see something confusing even though in the end the dump is successfully extracted: amrecover wants the second tape (vtape-2) loaded before the first one (vtape-1) but it seems that amanda does load the first tape correctly as while having typed yes at the prompt `Load tape vtape-2 now' I see that the `drive' is loaded with vtape-1: Is there a second drive (drive1) open here? The amrecover transcript you pasted seems to only need vtape-2, so I'm not sure where vtape-1 is involved. Perhaps in a previous recovery? It's amidxtaped which is manipulating the tapes, so you'll want to look in its debug log to see what's really going on. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Plans for 3.2
Amanda-3.1.0 isn't *quite* out the door yet, but like all developers at the end of a release cycle, we have turned our thoughts to the next feature release, Amanda-3.2.0. This wistfulness happened to coincide with a trip to Sunnyvale for all of us telecommuters at Zmanda, making it a perfect time to put our heads together and decide what to do next -- and more importantly, what to leave undone for the moment. We're looking to make the 3.2.0 release soon - perhaps as soon as four months (August). So we've selected a number of well-contained projects that are manageable in that timeframe. This plan is, of course, subject to revision, particularly if we see additional developers come onboard. 1. Smoother configuration and handling of filtering (compression and encryption), making it easier to set up customized filters, and to use different filters for different DLEs. 2. Logical EOM handling, allowing most devices to write data all the way to the end of a volume without losing any data. This allows Amanda to use volumes completely, and avoids the need to use those pesky split buffers. 3. Vaulting support for splitting and reassembly (for real this time, I swear) 4. Recovery and taper scan improvements, including more configurable searches for tapes, ability to chain multiple changers for recoveries, improvements in interactivity (feed me volume ___), and just generally a lot more flexibility and predictability in Amanda's behavior toward volumes. 5. Support for simultaneous writes to multiple tape drives. The technical support for this is in place, so the configuration and management is the hard part: how does Amanda decide which tape to put a DLE on? Can the DLE jump from tape to tape? How does Amanda know how many drives it can use? What does it do if one of them is in use? Yeah, it's complicated! 6. A DirectTCP-based media server daemon. This is basically a changer and its associated devices (tapes, vtapes, cloud, whatever) on a machine *other* than that running the Amanda server. Amanda uses DirectTCP to send data directly from the client tot he media server daemon, completely bypassing the Amanda server machine. Combine this with point #5, above, and Amanda gets a *lot* more scalable! Of course, we will have bugfixes to think about (those will end up in patch releases numbered 3.1.x, x0), and a number of loose ends to clean up and long-term projects to push forward: - consistent treatment of include/exclude lists across applications - perl rewrites on the server side - tear out some old, unused code Likewise, if time gets tight, we may implement some of these items in a limited fashion - in particular, I can think of some dumbed down implementations of #5 and #6 that would be much easier to write while still providing significant benefit to Amanda's users. I'd love to see comments on this list, and I'd especially like to hear from anyone who wants to add a project *and* work that project through to completion! Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Wed, Apr 28, 2010 at 6:57 AM, Mark Adams m...@campbell-lange.net wrote: I assume this will require space the size of the DLE? (in my case 2T) Yes, it will. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Amanda 3.1.0beta2 released
Hello, The Amanda core team is pleased to announce the second beta release of Amanda 3.1.0, the 3.1.0beta2 release. It can be downloaded from http://www.amanda.org (not yet available) or https://sourceforge.net/project/showfiles.php?group_id=120 Please download and test it out - with luck, this will be the last beta before the final release! Changes since beta1: - several fixes to amcleanup - fixes to planner for bad-schedule errors - fix to taper for assertion failure - amreport reports total time spent dumping (including changer movement) - numerous other small bugfixes - see ChangeLog Changes since 2.6.1: * Deprecated old changers; see amanda-changers(7) for replacements. These changers are still available in the distribution, but will be removed in a future release. o chg-null o chg-zd-mtx o chg-rait o chg-disk o chg-multi * Although chg-zd-mtx is still supported, we recommend that all users upgrade to the more efficient chg-robot. See contrib/convert-zd-mtx-to-robot.sh for a useful conversion script. * Amanda server configuration file changes (amanda.conf) o deprecated configuration keywords: + label_new_tapes o keywords deprecated in 2.6.1: + rawtapedev + tapebufs + file-pad o new configuration keyword + autolabel -- replace label_new_tapes + columnspec -- can specify a precision. + order -- in script, to specify script order execution. + client_port -- in dumptype, to specify which port to connect on the client. + estimate-- in dumptype, can specify multiple estimate method. o accept 'define' keyword for defining an holdingdisk * Amanda client configuration file changes (amanda-client.conf) o new configuration keyword + debug_days -- how many days to keep debug files. + client_port -- use by amrecover, specify which port to connect on the server. * Removed compile-time default --with-changer-device: specify a device explicitly in amanda.conf instead * amtape behavior has changed: o 'device' subcommand removed o 'slot advance' subcommand removed o 'update' subcommand no longer displays each slot as it is updated, and is not supported by all changers o taperscan output has changed o new 'inventory' subcommand * amrmtape rewrite o use long option + --changer-- Specify which changer to use + --cleanup-- Remove logs and indexes associated with label + --dryrun -- do not update the original copies. + --erase -- Attempt to erase the data contained on the volume + --keep-label -- Do not remove label from the tapelist + --quiet -- Opposite of --verbose + --verbose-- List backups of hosts and disks that are being discarded. * amdevcheck o new --label option. o new --properties option. * Device API o changed wildcard setting for S3_BUCKET_LOCATION from to * o new 'ndmp:' device to write to a tape on an ndmp server. o new 'dvdrw:' device to write to a dvd drive. * Application API o new properties to many applications o amgtar + new ACLS, SELINUX and XATTRS properties o amsamba + Allow '\' in diskname and amandapass. + new ALLOW-ANONYMOUS property. o new applications: + ampgsql -- Backup PostgreSQL using continuous WAL archiving + amsuntar -- Backup filesytem with the SUN tar + amraw-- Backup only one directory entry * New taper, with DirectTCP support, changed tape and catalog format: o all dumpfiles are now F_SPLIT_DUMPFILE o all on-tape dumpfiles have numparts=-1, since it's no longer possible to calculate this value in advance o there is no logging or reporting of zero-byte, successful parts (this may cause gaps in filenums in the catalog) * Redesigned amreport o much more natural command-line interface (just run 'amreport $config') o experimental XML output * configuration override for dumptype works with inheritance. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda 2.6.1p2 man page problem
On Fri, Apr 23, 2010 at 6:26 AM, McGraw, Robert P rmcg...@purdue.edu wrote: If I remove --enable-manpage-build and try to rebuild the man pages I get the following. [37][ama...@hertz]:amanda/amanda/man% make all WARNING: Not building needed amcheck.8 because building manpages is disabled. You may have to re-extract the pre-built manpages from the tarball. If I add --enable-manpage-build then when I try to rebuild the man pages the make cd to man and gets to config-status and hangs. That's an odd place for it to hang! The requirements for a manpage build are here: http://wiki.zmanda.com/index.php/Development_Dependencies#Documentation although I've never installed them on Solaris, so I don't have any additional detail for you. It's usually more complicated than just installing the corresponding packages, because there's a system-global XML catalog that needs to be customized. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Fri, Apr 23, 2010 at 3:29 AM, Mark Adams m...@campbell-lange.net wrote: Sorry for the stupid questions, but I've no idea how to identify *which* file needs to be selected in amrecover in order for it to restore from the start of the 10th chunk. I had been wondering why you were so concerned with which part (chunks are different) was on which tape. What is the task you're tring to accomplish here? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Fri, Apr 23, 2010 at 9:59 AM, Mark Adams m...@campbell-lange.net wrote: I've determined that when I try to retrieve files from the 2nd tape in my set it won't work. Tar crashes out with the helpful error due to previous errors -- no errors show before this apart from tar skipping to the next tape. Retrieves from the first tape work fine. Hmm, that's odd, because as far as tar is concerned, there aren't separate tapes - just a single datastream. Even when recovering a file only on the first tape, amrecover will still read the entire dumpfile (and thus require the second tape). I assume that you've seen files recovered successfully before you swapped to the second tape. In that case, did you kill amrecover after getting the desired files? If not, did it finish reading the second tape without error? I would recommend using 'amfetchdump' for the same dumpfile, and examining the resulting tarfile. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda refuses to use new tapes
On Fri, Apr 23, 2010 at 12:43 PM, Chris Hoogendyk hoogen...@bio.umass.edu wrote: I don't get it. Why is Amanda refusing to use the new tapes (or tape 3)? See the note in this manpage: http://wiki.zmanda.com/man/amanda-taperscan.7.html Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda refuses to use new tapes
On Fri, Apr 23, 2010 at 1:38 PM, Chris Hoogendyk hoogen...@bio.umass.edu wrote: The algorithm just seems a bit weird. I never thought that fast-searchable would jump over the new tapes, especially since they are marked reuse. I always assumed it would check the tapes in order until it found one that was suitable. It's not just a bit weird, it's very weird. That's why I wrote up the manpage :) Hopefully we'll be adding some new, less insane taperscan algorithms in 3.2. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Thu, Apr 22, 2010 at 6:49 AM, Mark Adams m...@campbell-lange.net wrote: 2010-04-09 16:55:11 localhost /upbackup 0 UPSNAPSHOT-02 1 10/-1 OK This is part 10 of the dump, at filemark 1 on UPSNAPSHOT-02. is that what you were looking for? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda 2.6.1p2 man page problem
On Thu, Apr 22, 2010 at 8:53 AM, McGraw, Robert P rmcg...@purdue.edu wrote: **BLANK LINES** I recall someone else having this problem, too. I don't know enough nroff to figure out what's wrong, but I suspect it's a docbook XSLT bug of some sort. As a workaround, the 2.6.1 manpages are available here: http://wiki.zmanda.com/man/2.6.1/amanda.8.html Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Amanda 2.6.1p2 man page problem
On Thu, Apr 22, 2010 at 10:09 AM, Darin Perusich darin.perus...@cognigencorp.com wrote: It's docbook for sure and the man pages likely didn't build because it's not on your system or it's an old version, so --enable-manpage-build isn't going to help. The tarball ships with pre-built manpages, but they are built on whatever system generated the tarball, using its version of docbook. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Still the puzzle, and I'm feeling ignored, why?
On Thu, Apr 22, 2010 at 11:00 AM, Gene Heskett gene.hesk...@verizon.net wrote: 500 No dumps available on or before date 2010-04-08 Actually, this means there are no catalog entries for that date. I think I had asked last time, what amadmin Conf find will show for that particular DLE? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: Issue with amrecover from 2nd tape
On Thu, Apr 22, 2010 at 12:00 PM, Paul Yeatman pyeat...@zmanda.com wrote: Thus the first file on the second tape, file number=0. Tape file numbers are one-based, because they reflect the number of filemarks that precede the file data, and the tape always begins with a tape header and a filemark: tape header FILEMARK first data file FILEMARK second data file ... Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: question about new changer API in 3.0.1beta1
On Thu, Apr 22, 2010 at 11:59 AM, Jean-Francois Malouin
ma...@bic.mni.mcgill.ca wrote:
Slowly going through the new stuff in 3.0.1beta1 and I came across the
following in the amanda-changers.7 manpage:
define changer hp-robot {
tapedev chg-robot:/dev/sg1
property tape-device 0=tape:/dev/nst0
property append tape-device 1=tape:/dev/nst0
device_property BLOCK_SIZE 512k
}
What the point of defining 2 tape devices that points to the same
physical drive? Or Am I missing (once more) something obvious?
Nope, it's a typo - the second should say ..nst1 :)
I'll fix it up. Thanks!
Dustin
--
Open Source Storage Engineer
http://www.zmanda.com
Re: Minor installation bug: /etc/amanda not made
On Wed, Apr 21, 2010 at 6:29 PM, Charles Curley charlescur...@charlescurley.com wrote: That resulted in: Configuration: /etc/amanda but the directory wasn't built. It would be nice if the installation process built it. I'm not entirely convinced of this -- I consider it the package's job to create stuff like that, and my impression is that's the pattern followed by most applications. I'm open to being persuaded :) Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: SV: FAILED [spawn /bin/gzip: dup2 out: Bad file descriptor] on 2.6.1p2
On Wed, Apr 21, 2010 at 11:04 AM, Volker Pallas ama...@sqmail.de wrote: Is auth=bsdtcp mandatory? If you want to switch to bsdtcp, then yes. You'll also need to change your (x)inetd configuration accordingly. The amanda-auth(7) manpage may be of use to you in figuring the whole thing out. DUstin -- Open Source Storage Engineer http://www.zmanda.com
Re: REF: compile issue on solaris X/x86_64
On Tue, Apr 20, 2010 at 1:22 PM, McGraw, Robert P rmcg...@purdue.edu wrote: My problem seem to be that by default the SunStudio uses /usr/ccs/bin/ld rather than /usr/sfw/i386-sun-solaris2.10/bin/ld. I cannot figure out how to set the sfw ld to be the default. That sounds like the correct analysis. Perhaps add LD=/usr/sfw/... to the configure invocation? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: windows backup using amanda
On Tue, Apr 20, 2010 at 1:10 PM, Brian Cuttler br...@wadsworth.org wrote: Unfortunately what we really wanted was to backup the list of directories in the backup template defined on the server. What's a backup template? Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: error timeout waiting for ACK using amrecover
On Mon, Apr 19, 2010 at 8:50 AM, Craig Constantine cr...@blkbx.com wrote: AMRECOVER Version 2.6.1p2. Contacting server on localhost ... I'd recommend just using auth local if you're communicating within the localhost. BSDTCP is just too annoying to get right. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: trying to restore an F10 insttall
On Sat, Apr 17, 2010 at 11:58 PM, Gene Heskett gene.hesk...@verizon.net wrote: 500 No dumps available on or before date 2010-04-18 No index records for disk for specified date I didn't really understand how you shuffled your machines, but it sounds like the catalog (the trace logs) is not where it should be. They should be in the directory pointed to by the 'logdir' parameter on the server. Dustin -- Open Source Storage Engineer http://www.zmanda.com
Re: trying to restore an F10 insttall
On Sat, Apr 17, 2010 at 10:49 PM, Gene Heskett gene.hesk...@verizon.net wrote: What am I doing wrong? Thanks. Ping! Why are you running amgtar directly? Dustin -- Open Source Storage Engineer http://www.zmanda.com
