Re: amanda upgrade questions
On Wed, May 09, 2007 at 09:38:20PM -0400, Guy Dallaire wrote: 2.5 seems pretty scary. I see a lot of problems on the list. Is it stable, or is it experimental ? 2.5.2 is a stable release. A lot of the recent problems have been small bugs that occurred in configurations not tested during the beta phase. That is to say, the release itself is in no way experimental, but the testing process could use some work -- that's a discussion I would be happy to have here, but in another thread. As for upgrading -- you're contemplating a rather significant version bump, and I personally hesitate to answer because I'm not very confident that I'll be right[1]. I suspect others may be reticent for similar reasons. I think that things will go smoothly for you in an upgrade, as Amanda development remains fairly conservative in maintaining backward compatiblity. The port-range options still exist and do the same thing. I *think* that the on-disk formats and the on-the-wire network protocols are also compatible; hopefully someone will reply loudly to this email if I'm wrong about that. Probably your best bet is to just try it out in as safe a manner as you can muster. Dustin [1] I was absent from the Amanda scene from somewhere in 2.4.4 to about 2.5.1, so I'm not familiar with the transition. -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
Re: Hardware suggestion
On Thu, May 10, 2007 at 08:59:40AM +0700, Olivier Nicole wrote: Could anyone advise on some disk tray solution that is stong and reliable and will handle a lot of manipulation. idealy I wouldbe looking at one tray bay and 5 or 6 trays for the disks. I think your best bet may be a set of Firewire enclosures. If there is hardware out there rugged enough to hot-swap drives in trays daily (e.g,. with a SCSI or SATA backplane), I expect it will be at a much higher price-point (both for the hot-swap equipment, and for the flavor of drives it requires) than equivalent commodity Firewire hardware. Dustin -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
Re: Hardware suggestion
Hi Olivier, Normal hot swap bay for hard disk are not designed for daily use, they are designed for maintenance only, and would break soon if I swaped the disk every day. You might want to try some (external) eSata devices. The connectors seem more reliable than the internal ones - and probably are cheap to replace... best, Kai
Re: amanda upgrade questions
Upgrading amanda is always easy if you keep the same config, we always maintains compatibility. Only the new features (amrecover use security-api, auth (ssh), ...) require config change. If you use the 2.4 amrecover, no config change is needed. 2.5.2 is stable on linux, all problem seen on other OS (irix, OSF/1, solaris8, ...) are already fixed in the latest snapshot. Jean-Louis Guy Dallaire wrote: Replying to myself: It's my second attempt at trying to get any input as to how difficult it may be to upgrade amanda. I have not received any reply on both occasions. Are my questions too dumb ? I would like to read some sort of upgrade guide, but I can't find anything useful in the doc or in the amanda wiki. 2.5 seems pretty scary. I see a lot of problems on the list. Is it stable, or is it experimental ? Thanks 2007/5/7, Guy Dallaire [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: I understand that amanda 2.5 is compatible with amanda 2.4 clients. This means I could upgrade my server and do the clients later ? I would like to upgrade to 2.5.2 for security reason (ability to encrypt backups, ssh) I compiled my server and client with restriction on the tcp and udp ports used for ease of firewall configuration: --with-tcpportrange=5,50100' '--with-udpportrange=850,859 Do these options still apply to the new amanda version ? Will the new version be able to restore old tapes that were written by the amanda 2.4 tapeserver ? Will amrecover on a 2.4 client be able to use the 2.5 amanda server ? Are the new encryption and ssh features easy to use ? Thanks !
Re: no backups since a week ago (data write: Connection reset by peer)
From amdump log:
This similar flow happens every time it fails. At first I was worried
about the not enough diskspace message, but this is just because my
holding disk filled up. I see the same thing happening in successful runs.
driver: state time 8459.717 free kps: 37761 space: 55584 taper: idle
idle-dumpers: 7 qlen tapeq: 0 runq: 18 roomq: 2 wakeup: 0 d
river-idle: no-diskspace
driver: interface-state time 8459.717 if : free 37761
driver: hdisk-state time 8459.717 hdisk 0: free 55584 dumpers 1
driver: result time 8459.727 from chunker3: RQ-MORE-DISK 03-4
find diskspace: not enough diskspace. Left with 508960 K
find diskspace: not enough diskspace. Left with 71872 K
find diskspace: not enough diskspace. Left with 17664 K
find diskspace: not enough diskspace. Left with 508960 K
driver: Don't know how to send ABORT command to chunker
taper: DONE [idle wait: 7182.514 secs]
chunker: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7311.263: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7311.263: pid 1129 finish time Thu May 10 02:06:02 2007
chunker: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7888.517: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7888.517: pid 31879 finish time Thu May 10 02:06:02 2007
chunker: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7888.485: error [bad command after RQ-MORE-DISK: QUIT]
chunker: time 7888.485: pid 31880 finish time Thu May 10 02:06:02 2007
taper: writing end marker. [VOL14 OK kb 2300352 fm 28]
dumper: kill index command
dumper: kill index command
dumper: kill index command
amdump: end at Thu May 10 02:06:02 EDT 2007
Scanning /holding/amanda...
20070509234502: found Amanda directory.
Gene Heskett wrote:
On Wednesday 09 May 2007, Steven Settlemyre wrote:
Can someone please help me?
Steven Settlemyre wrote:
I haven't changed my configs for months and things were running great
until last week. Since last tues, none of my dailies have finished,
and last night a monthly failed.
Looking through the logs I see the problem always seems to start with
data write: Connection reset by peer and Don't know how to send
ABORT command to chunker. I'm having a hard time interpreting the
logs and can't seem to find too much in the archives about this. Was
wondering if someone could walk me through an explanation of the
problem and how to avoid it in the future.
My monthlies run tape spanning on 3 40G tapes.
Here is the email output generated:
*** THE DUMPS DID NOT FINISH PROPERLY!
These dumps were to tape Monthly21.
The next 3 tapes Amanda expects to use are: Monthly01, Monthly02,
Monthly03.
The next 3 new tapes already labelled are: Monthly19, Monthly20,
Monthly22.
FAILURE AND STRANGE DUMP SUMMARY:
wagstaff /usr/locallev 1 FAILED [data
write: Connection reset by peer]
lollipop /files1 lev 0 FAILED [data
write: Connection reset by peer]
helios /files3 lev 1 FAILED [data write:
Connection reset by peer]
helios / RESULTS MISSING
helios /files2 RESULTS MISSING
helios /usr RESULTS MISSING
helios /usr/localRESULTS MISSING
helios /var RESULTS MISSING
lollipop / RESULTS MISSING
lollipop /usr RESULTS MISSING
lollipop /usr/localRESULTS MISSING
wagstaff /files3 RESULTS MISSING
wagstaff /files4 RESULTS MISSING
wagstaff /files5 RESULTS MISSING
wagstaff /files6/vol/Voiceware RESULTS MISSING
wizard/files2 RESULTS MISSING
snapserver /hd/vol_mnt0/shares/TermLab RESULTS MISSING
snapserver /hd/vol_mnt0/shares/bcl RESULTS MISSING
snapserver /hd/vol_mnt0/shares/biochem RESULTS MISSING
snapserver /hd/vol_mnt0/shares/confocal RESULTS MISSING
driver: FATAL Don't know how to send ABORT command to chunker
chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT]
chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT]
chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT]
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:08
Run Time (hrs:min) 1:01
Dump Time (hrs:min)1:55 1:40 0:16
Output Size (meg)8519.7 7729.7 790.1
Original Size (meg) 13146.311595.5 1550.8
Avg Compressed Size (%)64.8 66.7 50.9 (level:#disks
...)
Filesystems Dumped 35 12 23 (1:23)
Avg Dump Rate (k/s) 1261.0 1323.3 863.1
Tape Time (hrs:min)0:53 0:44 0:09
Tape Size (meg) 8521.6 7730.3 791.3
Tape Used (%)
Re: amanda upgrade questions
In config file you are referring to the .config file that is home to all the switches like --with- ssh and etc? __ Lewis [EMAIL PROTECTED] Cell: (734) 323-8776 - Original Message - From: [EMAIL PROTECTED] [EMAIL PROTECTED] To: Guy Dallaire [EMAIL PROTECTED] Cc: AMANDA users amanda-users@amanda.org Sent: Thu May 10 07:16:01 2007 Subject: Re: amanda upgrade questions Upgrading amanda is always easy if you keep the same config, we always maintains compatibility. Only the new features (amrecover use security-api, auth (ssh), ...) require config change. If you use the 2.4 amrecover, no config change is needed. 2.5.2 is stable on linux, all problem seen on other OS (irix, OSF/1, solaris8, ...) are already fixed in the latest snapshot. Jean-Louis Guy Dallaire wrote: Replying to myself: It's my second attempt at trying to get any input as to how difficult it may be to upgrade amanda. I have not received any reply on both occasions. Are my questions too dumb ? I would like to read some sort of upgrade guide, but I can't find anything useful in the doc or in the amanda wiki. 2.5 seems pretty scary. I see a lot of problems on the list. Is it stable, or is it experimental ? Thanks 2007/5/7, Guy Dallaire [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: I understand that amanda 2.5 is compatible with amanda 2.4 clients. This means I could upgrade my server and do the clients later ? I would like to upgrade to 2.5.2 for security reason (ability to encrypt backups, ssh) I compiled my server and client with restriction on the tcp and udp ports used for ease of firewall configuration: --with-tcpportrange=5,50100' '--with-udpportrange=850,859 Do these options still apply to the new amanda version ? Will the new version be able to restore old tapes that were written by the amanda 2.4 tapeserver ? Will amrecover on a 2.4 client be able to use the 2.5 amanda server ? Are the new encryption and ssh features easy to use ? Thanks !
Re: amanda upgrade questions
On Thursday 10 May 2007, Donofrio, Lewis wrote: In config file you are referring to the .config file that is home to all the switches like --with- ssh and etc? __ Lewis [EMAIL PROTECTED] Cell: (734) 323-8776 Yup, that one. Here is mine. #!/bin/sh # since I'm always forgetting to su amanda... if [ `whoami` != 'amanda' ]; then echo echo Warning echo Amanda needs to be configured and built by the user amanda, echo but must be installed by user root. echo exit 1 fi make clean rm -f config.status config.cache ./configure --with-user=amanda \ --with-group=disk \ --with-owner=amanda \ --with-gnu-ld \ --prefix=/usr/local \ --with-tapedev=FILE:/amandatapes \ --with-debugging=/tmp/amanda-dbg/ \ --with-tape-server=coyote \ --with-bsdtcp-security --with-amandahosts \ --with-configdir=/usr/local/etc/amanda \ --with-config=Daily \ --with-gnutar=/bin/tar make You will also need to add/modify the ~/.amandahosts file IIRC. -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) I just know I'm a better manager when I have Joe DiMaggio in center field. -- Casey Stengel
Re: no backups since a week ago (data write: Connection reset by peer)
On Thursday 10 May 2007, Steven Settlemyre wrote: From amdump log: This similar flow happens every time it fails. At first I was worried about the not enough diskspace message, but this is just because my holding disk filled up. I see the same thing happening in successful runs. driver: state time 8459.717 free kps: 37761 space: 55584 taper: idle idle-dumpers: 7 qlen tapeq: 0 runq: 18 roomq: 2 wakeup: 0 d river-idle: no-diskspace driver: interface-state time 8459.717 if : free 37761 driver: hdisk-state time 8459.717 hdisk 0: free 55584 dumpers 1 driver: result time 8459.727 from chunker3: RQ-MORE-DISK 03-4 find diskspace: not enough diskspace. Left with 508960 K find diskspace: not enough diskspace. Left with 71872 K find diskspace: not enough diskspace. Left with 17664 K find diskspace: not enough diskspace. Left with 508960 K driver: Don't know how to send ABORT command to chunker taper: DONE [idle wait: 7182.514 secs] chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7311.263: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7311.263: pid 1129 finish time Thu May 10 02:06:02 2007 chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.517: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.517: pid 31879 finish time Thu May 10 02:06:02 2007 chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.485: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.485: pid 31880 finish time Thu May 10 02:06:02 2007 taper: writing end marker. [VOL14 OK kb 2300352 fm 28] dumper: kill index command dumper: kill index command dumper: kill index command amdump: end at Thu May 10 02:06:02 EDT 2007 Scanning /holding/amanda... 20070509234502: found Amanda directory. Humm, I see no such diskspace related messages in last nights amdump.1 file here. How much holding disk do you have, and how much of it is reserved for emergency incrementals? By default, its 100%, so if you don't have that reserved set at all... I think one should have enough holding disk that is not reserved to cover the two largest disklist entries you have. In my case that would be around 15GB of non-reserved space, but then I have about 80GB available here, more than adequate till I fill it up. Again.. :-) Gene Heskett wrote: On Wednesday 09 May 2007, Steven Settlemyre wrote: Can someone please help me? Steven Settlemyre wrote: I haven't changed my configs for months and things were running great until last week. Since last tues, none of my dailies have finished, and last night a monthly failed. Looking through the logs I see the problem always seems to start with data write: Connection reset by peer and Don't know how to send ABORT command to chunker. I'm having a hard time interpreting the logs and can't seem to find too much in the archives about this. Was wondering if someone could walk me through an explanation of the problem and how to avoid it in the future. My monthlies run tape spanning on 3 40G tapes. Here is the email output generated: *** THE DUMPS DID NOT FINISH PROPERLY! These dumps were to tape Monthly21. The next 3 tapes Amanda expects to use are: Monthly01, Monthly02, Monthly03. The next 3 new tapes already labelled are: Monthly19, Monthly20, Monthly22. FAILURE AND STRANGE DUMP SUMMARY: wagstaff /usr/locallev 1 FAILED [data write: Connection reset by peer] lollipop /files1 lev 0 FAILED [data write: Connection reset by peer] helios /files3 lev 1 FAILED [data write: Connection reset by peer] helios / RESULTS MISSING helios /files2 RESULTS MISSING helios /usr RESULTS MISSING helios /usr/localRESULTS MISSING helios /var RESULTS MISSING lollipop / RESULTS MISSING lollipop /usr RESULTS MISSING lollipop /usr/localRESULTS MISSING wagstaff /files3 RESULTS MISSING wagstaff /files4 RESULTS MISSING wagstaff /files5 RESULTS MISSING wagstaff /files6/vol/Voiceware RESULTS MISSING wizard/files2 RESULTS MISSING snapserver /hd/vol_mnt0/shares/TermLab RESULTS MISSING snapserver /hd/vol_mnt0/shares/bcl RESULTS MISSING snapserver /hd/vol_mnt0/shares/biochem RESULTS MISSING snapserver /hd/vol_mnt0/shares/confocal RESULTS MISSING driver: FATAL Don't know how to send ABORT command to chunker chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT] chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT] chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT] STATISTICS: Total Full
Re: no backups since a week ago (data write: Connection reset by peer)
Steven Settlemyre wrote: From amdump log: This similar flow happens every time it fails. At first I was worried about the not enough diskspace message, but this is just because my holding disk filled up. I see the same thing happening in successful runs. In some case, this error is fatal, this bug is fixed in newer release. driver: state time 8459.717 free kps: 37761 space: 55584 taper: idle idle-dumpers: 7 qlen tapeq: 0 runq: 18 roomq: 2 wakeup: 0 d river-idle: no-diskspace driver: interface-state time 8459.717 if : free 37761 driver: hdisk-state time 8459.717 hdisk 0: free 55584 dumpers 1 driver: result time 8459.727 from chunker3: RQ-MORE-DISK 03-4 find diskspace: not enough diskspace. Left with 508960 K find diskspace: not enough diskspace. Left with 71872 K find diskspace: not enough diskspace. Left with 17664 K find diskspace: not enough diskspace. Left with 508960 K driver: Don't know how to send ABORT command to chunker taper: DONE [idle wait: 7182.514 secs] chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7311.263: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7311.263: pid 1129 finish time Thu May 10 02:06:02 2007 chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.517: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.517: pid 31879 finish time Thu May 10 02:06:02 2007 chunker: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.485: error [bad command after RQ-MORE-DISK: QUIT] chunker: time 7888.485: pid 31880 finish time Thu May 10 02:06:02 2007 taper: writing end marker. [VOL14 OK kb 2300352 fm 28] dumper: kill index command dumper: kill index command dumper: kill index command amdump: end at Thu May 10 02:06:02 EDT 2007 Scanning /holding/amanda... 20070509234502: found Amanda directory.
Ooops
Greetings; I just tried to build 2.5.2-20070510 got this gcc -DHAVE_CONFIG_H -I. -I. -I../config -I../gnulib -Wall -Wextra -Wparentheses -Wdeclaration-after-statement -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wformat -Wsign-compare -Wfloat-equal -Wold-style-definition -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -MT util.lo -MD -MP -MF .deps/util.Tpo -c util.c -fPIC -DPIC -o .libs/util.o util.c: In function 'cmp_sockaddr': util.c:701: error: 'ss_in' undeclared (first use in this function) util.c:701: error: (Each undeclared identifier is reported only once util.c:701: error: for each function it appears in.) util.c:706: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:707: warning: assignment from incompatible pointer type util.c:717: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:718: warning: assignment from incompatible pointer type make[1]: *** [util.lo] Error 1 Not quite ready :) -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) Make a wish, it might come true.
Re: Hardware suggestion
On Thu, May 10, 2007 at 08:59:40AM +0700, Olivier Nicole wrote: Hi, I am considering replacement for my Amanda server, the guy has been in production for about 6 years now and I want to be ready for the time when the tape drive will die. ... Could anyone advise on some disk tray solution that is stong and reliable and will handle a lot of manipulation. idealy I wouldbe looking at one tray bay and 5 or 6 trays for the disks. This is not a recommendation, only an info point. There is one company, I think Iomega who made the zip drives, who has a cartridge format that came out about 3 yrs ago. In their product line is a 10 cartridge changer, much like a tape library. One concern I've had with vtapes is the loss of a harddrive could mean the loss of ALL your backups where the lost of a tape means the loss of a single backup and the loss of a drive/library means the loss of zero data. When I set up my vtape solution I chose to have several harddrives with my vtapes spread across them, a few on one drive, the next group on the next drive, etc. My slots directory is symlinks to the actual vtapes. As I chose to use external usb drives, and the ones I got power down when not in use, most consume little energy for days on end. I only used 3 large drives. Perhaps you might consider a large collection of smaller drives and a substantial usb hub to connect them. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Ooops
I build a new snapshot with the compilation problem fixed. Thanks for reporting the problem. Jean-Louis Gene Heskett wrote: Greetings; I just tried to build 2.5.2-20070510 got this gcc -DHAVE_CONFIG_H -I. -I. -I../config -I../gnulib -Wall -Wextra -Wparentheses -Wdeclaration-after-statement -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wformat -Wsign-compare -Wfloat-equal -Wold-style-definition -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -MT util.lo -MD -MP -MF .deps/util.Tpo -c util.c -fPIC -DPIC -o .libs/util.o util.c: In function 'cmp_sockaddr': util.c:701: error: 'ss_in' undeclared (first use in this function) util.c:701: error: (Each undeclared identifier is reported only once util.c:701: error: for each function it appears in.) util.c:706: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:707: warning: assignment from incompatible pointer type util.c:717: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:718: warning: assignment from incompatible pointer type make[1]: *** [util.lo] Error 1 Not quite ready :)
Re: Ooops
i am curious. how many platforms does zmanda have and use for test builds and amanda testing? OSs and hardware? thanks, chris On Thu, May 10, 2007 at 12:03:14PM -0400, Jean-Louis Martineau wrote: I build a new snapshot with the compilation problem fixed. Thanks for reporting the problem. Jean-Louis Gene Heskett wrote: Greetings; I just tried to build 2.5.2-20070510 got this gcc -DHAVE_CONFIG_H -I. -I. -I../config -I../gnulib -Wall -Wextra -Wparentheses -Wdeclaration-after-statement -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wformat -Wsign-compare -Wfloat-equal -Wold-style-definition -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -MT util.lo -MD -MP -MF .deps/util.Tpo -c util.c -fPIC -DPIC -o .libs/util.o util.c: In function 'cmp_sockaddr': util.c:701: error: 'ss_in' undeclared (first use in this function) util.c:701: error: (Each undeclared identifier is reported only once util.c:701: error: for each function it appears in.) util.c:706: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:707: warning: assignment from incompatible pointer type util.c:717: error: 'struct sockaddr_in' has no member named 'ss_family' util.c:718: warning: assignment from incompatible pointer type make[1]: *** [util.lo] Error 1 Not quite ready :) -- Christopher Linn celinn at mtu.edu | By no means shall either the CEC System Administrator II | or MTU be held in any way liable Center for Experimental Computation | for any opinions or conjecture I Michigan Technological University | hold to or imply to hold herein.
amanda-client version
How do I find out which version of amanda-client is running on my hosts? I found the debian ones through the package manager, but wonder if there's a command-line switch to use? Steve
Re: Ooops
On Thursday 10 May 2007, Jean-Louis Martineau wrote: I build a new snapshot with the compilation problem fixed. Thanks for reporting the problem. Jean-Louis And thank you for the rapid fix! -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) Beware of a dark-haired man with a loud tie.
Re: amanda-client version
On Thu, May 10, 2007 at 01:35:46PM -0400, Steven Settlemyre wrote: How do I find out which version of amanda-client is running on my hosts? I found the debian ones through the package manager, but wonder if there's a command-line switch to use? Use amgetconf build.VERSION if the box has 'amgetconf'; otherwise you can look in your sendbackup or sendsize debug logs. -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
Re: Ooops
On Thu, May 10, 2007 at 12:17:51PM -0400, Christopher Linn wrote: i am curious. how many platforms does zmanda have and use for test builds and amanda testing? OSs and hardware? One of my current projects is to put a more thorough testing process in place -- mostly because it's been *my* patches which have turned out to be non-portable! For the community version, we currently focus our testing on the platforms listed on http://zmanda.com/downloads.html: RHEL 3-5, FC 3-6, OpenSUSE 10, SLES 9-10, and Windows (cygwin). That testing takes place on releases, *not* on the snapshots. The bug which started this thread was in last night's snapshot. For other platforms, we rely on users to test betas, since the Amanda user community has a much wider array of hardware and configurations than we could *ever* hope to test in-house. I have a few nascent ideas of how we could improve this process, but I would prefer to hear others' suggestions and proposals before I bias the conversation with my thoughts. Dustin P.S. As for hardware -- well, virtualization is a wonderful thing ;) -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
first attempt with tape spanning...nogo.
My first attempt at using tape spanning just failed.
Anyone can offer some guidance?
Server is an sgi with 28 cpus 28G of ram and
client is a debian Xeon quad-core with 4GB of ram.
I've setup an amanda test config for a DLE of ~200GB
supposed to be stuffed in 100GB LTO1 tapes. The dumptype
for the DLE reads:
define dumptype tar-span {
global
program GNUTAR
priority high
auth bsdtcp
tape_splitsize 10 Gb
compress none
record no
strategy noinc
fallback_splitsize 2 Gb
split_diskbuffer /holddisk/conf9/split_diskbuffer
comment tape-spanning with tar
}
Makes sense or am I using crazy values? Not sure where
split_diskbuffer lives: on the client or on the server?
The amdump reports say:
FAILURE AND STRANGE DUMP SUMMARY:
gaspar concussion2-span lev 0 FAILED [dump to tape failed]
taper: FATAL [EMAIL PROTECTED]: memory allocation failed (2147483648 bytes
requested)
taper: FATAL writer: Syncpipe failure before start
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:02
Run Time (hrs:min) 0:07
Dump Time (hrs:min)0:00 0:00 0:00
Output Size (meg) 0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min)0:00 0:00 0:00
Tape Size (meg) 0.00.00.0
Tape Used (%) 0.00.00.0
Filesystems Taped 0 0 0
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
USAGE BY TAPE:
Label Time Size %NbNc
span-test-09 0:000M0.0 0 0
NOTES:
planner: Adding new disk gaspar:concussion2-span.
taper: mmap failed (No such device or address): using fallback split size of
2097152kb to buffer gaspar:concussion2-span.0 in-memory
driver: taper pid 13744170 exited with signal 6
DUMP SUMMARY:
DUMPER STATSTAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SSKB/s MMM:SSKB/s
- -- --
gaspar -cussion2-span 0 FAILED --
(brought to you by Amanda version 2.5.1p2)
jf
--
°
Re: first attempt with tape spanning...nogo.
You can't mmap a 10 Gb and alloc a 2GB buffer.
I don't know what are the limit on irix.
Compile in 64 bits mode and/or use lower value for tape_splitsize and
fallback_splitsize
split_diskbuffer is on the server.
Jean-Louis
Jean-Francois Malouin wrote:
My first attempt at using tape spanning just failed.
Anyone can offer some guidance?
Server is an sgi with 28 cpus 28G of ram and
client is a debian Xeon quad-core with 4GB of ram.
I've setup an amanda test config for a DLE of ~200GB
supposed to be stuffed in 100GB LTO1 tapes. The dumptype
for the DLE reads:
define dumptype tar-span {
global
program GNUTAR
priority high
auth bsdtcp
tape_splitsize 10 Gb
compress none
record no
strategy noinc
fallback_splitsize 2 Gb
split_diskbuffer /holddisk/conf9/split_diskbuffer
comment tape-spanning with tar
}
Makes sense or am I using crazy values? Not sure where
split_diskbuffer lives: on the client or on the server?
The amdump reports say:
FAILURE AND STRANGE DUMP SUMMARY:
gaspar concussion2-span lev 0 FAILED [dump to tape failed]
taper: FATAL [EMAIL PROTECTED]: memory allocation failed (2147483648 bytes
requested)
taper: FATAL writer: Syncpipe failure before start
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:02
Run Time (hrs:min) 0:07
Dump Time (hrs:min)0:00 0:00 0:00
Output Size (meg) 0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0 0 0
Avg Dump Rate (k/s) -- -- --
Tape Time (hrs:min)0:00 0:00 0:00
Tape Size (meg) 0.00.00.0
Tape Used (%) 0.00.00.0
Filesystems Taped 0 0 0
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --
USAGE BY TAPE:
Label Time Size %NbNc
span-test-09 0:000M0.0 0 0
NOTES:
planner: Adding new disk gaspar:concussion2-span.
taper: mmap failed (No such device or address): using fallback split size of
2097152kb to buffer gaspar:concussion2-span.0 in-memory
driver: taper pid 13744170 exited with signal 6
DUMP SUMMARY:
DUMPER STATSTAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SSKB/s MMM:SSKB/s
- -- --
gaspar -cussion2-span 0 FAILED --
(brought to you by Amanda version 2.5.1p2)
jf
Re: first attempt with tape spanning...nogo.
On Thu, May 10, 2007 at 02:40:22PM -0400, Jean-Francois Malouin wrote:
My first attempt at using tape spanning just failed.
Anyone can offer some guidance?
Server is an sgi with 28 cpus 28G of ram and
client is a debian Xeon quad-core with 4GB of ram.
I've setup an amanda test config for a DLE of ~200GB
supposed to be stuffed in 100GB LTO1 tapes. The dumptype
for the DLE reads:
define dumptype tar-span {
global
program GNUTAR
priority high
auth bsdtcp
tape_splitsize 10 Gb
compress none
record no
strategy noinc
fallback_splitsize 2 Gb
split_diskbuffer /holddisk/conf9/split_diskbuffer
comment tape-spanning with tar
}
Makes sense or am I using crazy values? Not sure where
split_diskbuffer lives: on the client or on the server?
The amdump reports say:
client
|via network
V
server holding disk, possibly chunked
|
V
diskbuffer(s)
split1, split2, ... likely one filling, one already filled taping
|
V
tape
NOTES:
planner: Adding new disk gaspar:concussion2-span.
taper: mmap failed (No such device or address): using fallback split size
of 2097152kb to buffer gaspar:concussion2-span.0 in-memory
driver: taper pid 13744170 exited with signal 6
On my fedora system I had a problem with mmap. It would not handle
buffers of the size I wanted, 2GB. It did finally accept 1GB sized
split_diskbuffers. For me, an OS thing, not an amanda thing.
jl
--
Jon H. LaBadie [EMAIL PROTECTED]
JG Computing
4455 Province Line Road(609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: first attempt with tape spanning...nogo.
On Thu, May 10, 2007 at 03:00:52PM -0400, Jean-Louis Martineau wrote: You can't mmap a 10 Gb and alloc a 2GB buffer. I don't know what are the limit on irix. Compile in 64 bits mode and/or use lower value for tape_splitsize and fallback_splitsize I just added a troubleshooting entry on the wiki; Jean-Francois, please feel free to update with your findings. http://wiki.zmanda.com/index.php/Memory_allocation_failed Dustin -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/
Re: amanda-client version
You can also use amadmin xx version See http://wiki.zmanda.com/index.php/Quick_start#Collecting_information_for_configuration Paddy On 5/10/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Thu, May 10, 2007 at 01:35:46PM -0400, Steven Settlemyre wrote: How do I find out which version of amanda-client is running on my hosts? I found the debian ones through the package manager, but wonder if there's a command-line switch to use? Use amgetconf build.VERSION if the box has 'amgetconf'; otherwise you can look in your sendbackup or sendsize debug logs. -- Dustin J. Mitchell Storage Software Engineer, Zmanda, Inc. http://www.zmanda.com/ -- Amanda documentation: http://wiki.zmanda.com Amanda forums: http://forums.zmanda.com
Re: amanda-client version
Paddy Sreenivasan wrote: You can also use amadmin xx version That only works on the server. If '--without-server' was used to build the software for some of your clients, there won't be an amadmin command to run, so you have to check the *.debug files. See http://wiki.zmanda.com/index.php/Quick_start#Collecting_information_for_configuration Paddy On 5/10/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Thu, May 10, 2007 at 01:35:46PM -0400, Steven Settlemyre wrote: How do I find out which version of amanda-client is running on my hosts? I found the debian ones through the package manager, but wonder if there's a command-line switch to use? Use amgetconf build.VERSION if the box has 'amgetconf'; otherwise you can look in your sendbackup or sendsize debug logs. -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: Ooops
On Thu, May 10, 2007 at 01:31:28PM -0500, [EMAIL PROTECTED] wrote: On Thu, May 10, 2007 at 12:17:51PM -0400, Christopher Linn wrote: i am curious. how many platforms does zmanda have and use for test builds and amanda testing? OSs and hardware? One of my current projects is to put a more thorough testing process in place -- mostly because it's been *my* patches which have turned out to be non-portable! For the community version, we currently focus our testing on the platforms listed on http://zmanda.com/downloads.html: RHEL 3-5, FC 3-6, OpenSUSE 10, SLES 9-10, and Windows (cygwin). That testing takes place on releases, *not* on the snapshots. The bug which started this thread was in last night's snapshot. For other platforms, we rely on users to test betas, since the Amanda user community has a much wider array of hardware and configurations than we could *ever* hope to test in-house. I have a few nascent ideas of how we could improve this process, but I would prefer to hear others' suggestions and proposals before I bias the conversation with my thoughts. Dustin P.S. As for hardware -- well, virtualization is a wonderful thing ;) ok, thanks alot. cheers.. chris -- Christopher Linn celinn at mtu.edu | By no means shall either the CEC System Administrator II | or MTU be held in any way liable Center for Experimental Computation | for any opinions or conjecture I Michigan Technological University | hold to or imply to hold herein.
to disk to tape?
I have gotten so far as getting my first vtape written and have lots of questions brewing. The web docs suggest splitting up your vtape space into vtapes of the same size. Is this required; doe Amanda intelligently handle vtapes of various sizes? Be well, rir
Re: to disk to tape?
You need to keep vtape as close to the size of your physical tapes as possable - as far as I know...and going from data to holding to vtapes to physical tapes will require two jobs. One for data to holding to vtapes then one for vtapes to holding disk to archival media. --someone corrent me if I'm mistaken. __ Lewis [EMAIL PROTECTED] Cell: (734) 323-8776 - Original Message - From: [EMAIL PROTECTED] [EMAIL PROTECTED] To: amanda-users@amanda.org amanda-users@amanda.org Sent: Thu May 10 16:47:41 2007 Subject: to disk to tape? I have gotten so far as getting my first vtape written and have lots of questions brewing. The web docs suggest splitting up your vtape space into vtapes of the same size. Is this required; doe Amanda intelligently handle vtapes of various sizes? Be well, rir
RegEx not working?
Hi there, I have 3 separate configs running on one server (bak13, bak14, bak15). In the past I have made sure that they did not use each other's tapes by use of the label regex. Here is the regex for bak14; labelstr ^VOL[14][0-9] # label constraint regex: all tapes must match The other 2 are similar, replacing only the 14 with 13 or 15. However, when I run amcheck bak14 with the wrong tape in the drive, I get this message; == Subject: BAK-05 /bak14 AMANDA PROBLEM: FIX BEFORE RUN, IF POSSIBLE Amanda Tape Server Host Check - read label `VOL132', date `20070510005900' label VOL132 match labelstr but it not listed in the tapelist file. (expecting a new tape) Server check took 61.588 seconds Amanda Backup Client Hosts Check Client check: 1 host checked in 0.183 seconds, 0 problems found (brought to you by Amanda 2.5.1p3) == Why is the above regex matching the label for bak14 which should only accept VOL140 through Vol149? This exact same regex appears to work properly in earlier versions of amanda. TIA for any enlightenment. - Richard
Re: RegEx not working?
It was a bug if it worked before. The regex works correctly. [14] is 1 or 4 You want ^VOL14[0-9]. Jean-Louis Richard Stockton wrote: Hi there, I have 3 separate configs running on one server (bak13, bak14, bak15). In the past I have made sure that they did not use each other's tapes by use of the label regex. Here is the regex for bak14; labelstr ^VOL[14][0-9] # label constraint regex: all tapes must match The other 2 are similar, replacing only the 14 with 13 or 15. However, when I run amcheck bak14 with the wrong tape in the drive, I get this message; == Subject: BAK-05 /bak14 AMANDA PROBLEM: FIX BEFORE RUN, IF POSSIBLE Amanda Tape Server Host Check - read label `VOL132', date `20070510005900' label VOL132 match labelstr but it not listed in the tapelist file. (expecting a new tape) Server check took 61.588 seconds Amanda Backup Client Hosts Check Client check: 1 host checked in 0.183 seconds, 0 problems found (brought to you by Amanda 2.5.1p3) == Why is the above regex matching the label for bak14 which should only accept VOL140 through Vol149? This exact same regex appears to work properly in earlier versions of amanda. TIA for any enlightenment. - Richard
Re: RegEx not working?
Richard Stockton wrote: Hi there, I have 3 separate configs running on one server (bak13, bak14, bak15). In the past I have made sure that they did not use each other's tapes by use of the label regex. Here is the regex for bak14; labelstr ^VOL[14][0-9] # label constraint regex: all tapes must match The other 2 are similar, replacing only the 14 with 13 or 15. However, when I run amcheck bak14 with the wrong tape in the drive, I get this message; == Subject: BAK-05 /bak14 AMANDA PROBLEM: FIX BEFORE RUN, IF POSSIBLE Amanda Tape Server Host Check - read label `VOL132', date `20070510005900' label VOL132 match labelstr but it not listed in the tapelist file. (expecting a new tape) Server check took 61.588 seconds Amanda Backup Client Hosts Check Client check: 1 host checked in 0.183 seconds, 0 problems found (brought to you by Amanda 2.5.1p3) == Why is the above regex matching the label for bak14 which should only accept VOL140 through Vol149? This exact same regex appears to work properly in earlier versions of amanda. Did id? A regex of ^VOL[14][0-9] would match two ranges, both VOL10 to VOL19 and VOL40 to VOL49. A [] with a list of characters is an 'either' unless it is two characters separated by a - which makes it a range, so [14] matches either a 1 or a 4. That makes your label VOL14 match because it starts with a 1, but it would also match VOL12 or VOL19, as well as VOL40. Don't you need something like: ^VOL14[0-9] to match VOL140 to VOL149 or ^VOL14[0-9][0-9] to match VOL1400 to VOL1499 (actually either should match longer strings as well that start with 140-149). Frank TIA for any enlightenment. - Richard -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: to disk to tape?
On Thursday 10 May 2007, R Ransbottom wrote: I have gotten so far as getting my first vtape written and have lots of questions brewing. The web docs suggest splitting up your vtape space into vtapes of the same size. Is this required; doe Amanda intelligently handle vtapes of various sizes? Be well, rir Amanda handles vtape with the same intelligence it uses on a regular fixed size tape. Amanda will promote a full backup that isn't quite due in order to help maintain an approximately equal tape usage every night. In fact, for those that come into the amanda arena, expecting to be able to do incremental's monday-thursday, and a complete full on friday nights are in for a somewhat rude shock because you have to bend over backwards, and write scripts that beat her about the brow in order to achieve that hugely tape wasting schedule. That simply is not in the amanda view of how things work from the original conception well over a decade ago. amanda, when left to her own devices, will figure out a schedule (provided your settings in amanda.conf are sane) that will backup a 50GB system to a 4GB tape, using one tape per nightly run, every night of a 7 day dumpcycle, filling each tape to 95% of its capacity, and do it without ever hitting an EOT _if_ your data is fairly stable. Admittedly that's a little extreme, but I have done it. This does mean that it might take amanda a week or more to figure out a suitable schedule, but it should only be adjusted if amanda is falling farther and farther behind when looked at over a 2 or 3 dumpcycle period. You cannot, and should not, ever figure on using (v)tape number-007 on a certain night of the week, again, amanda is managing the re-use of the tapes, and a sneeze sometime that requires you to do a catchup flush will advance that usage schedule by a day, and before you know it, the (v)tape named Daily-tuesday will be actually used on monday or earlier. -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) lp1 on fire (One of the more obfuscated kernel messages)
Re: RegEx not working?
At 03:34 PM 5/10/2007, Jean-Louis Martineau wrote: It was a bug if it worked before. The regex works correctly. [14] is 1 or 4 You want ^VOL14[0-9]. Foolish me, of course that is correct. My thanks again for all your help. - Richard Jean-Louis Richard Stockton wrote: Hi there, I have 3 separate configs running on one server (bak13, bak14, bak15). In the past I have made sure that they did not use each other's tapes by use of the label regex. Here is the regex for bak14; labelstr ^VOL[14][0-9] # label constraint regex: all tapes must match The other 2 are similar, replacing only the 14 with 13 or 15. However, when I run amcheck bak14 with the wrong tape in the drive, I get this message; == Subject: BAK-05 /bak14 AMANDA PROBLEM: FIX BEFORE RUN, IF POSSIBLE Amanda Tape Server Host Check - read label `VOL132', date `20070510005900' label VOL132 match labelstr but it not listed in the tapelist file. (expecting a new tape) Server check took 61.588 seconds Amanda Backup Client Hosts Check Client check: 1 host checked in 0.183 seconds, 0 problems found (brought to you by Amanda 2.5.1p3) == Why is the above regex matching the label for bak14 which should only accept VOL140 through Vol149? This exact same regex appears to work properly in earlier versions of amanda. TIA for any enlightenment. - Richard
Re: RegEx not working?
It was a bug if it worked before. The regex works correctly. [14] is 1 or 4 You want ^VOL14[0-9]. Actually wouldn't it be better to use ^VOL14[0-9]$ to avoid any trailing character after the third digit? olivier
Re: Hardware suggestion
Hi, Normal hot swap bay for hard disk are not designed for daily use, they are designed for maintenance only, and would break soon if I swaped the disk every day. You might want to try some (external) eSata devices. The connectors seem more reliable than the internal ones - and probably are cheap to replace... I was going through the reports of various amanda run, my actual tape is reported to give about 5MB/s, so any USB2 interface would be plenty enough. People with huge Amanda experience, please correct me if I am wrong. It seems to me that most ofthe time is taken by the dump process, and as soon as a dump is finished, a taping takes place, in parallel with other dump continuing. So the speed of the (v)tape only matters when the last dump finishes and there is no more parallelism, just taping. At same time, Amanda can be configured to make the big dumps first, so what is left at the end should be minimal and take almost no time. We are talking batch processing here, so that the last taping takes 5 minutes or 20 minutes is of very little concerns. What aveage tap write rate do you see in your reports (in the statistics)? I get about 5000 k/s for SLR100 taep drive. bests, olivier
