index tee cannot write problem with 2.5.1
I upgraded to the new 2.5.1 yesterday, bleeding edge, that's me. All but one of my clients got backed up. The amanda report says: ambiance.med.utah.edu sdc1 lev 1 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdc1 lev 1 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdc1 lev 1 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] ambiance.med.utah.edu sdb5 lev 2 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu / lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdb5 lev 2 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu sdb5 lev 2 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] ambiance.med.utah.edu / lev 0 FAILED [cannot read header: got 0 instead of 32768] ambiance.med.utah.edu / lev 0 FAILED [too many dumper retry: "[request failed: timeout waiting for ACK]"] the sendbackup debug files on the client all say: sendbackup: time 0.054: started backup sendbackup: time 50.042: index tee cannot write [Broken pipe] sendbackup: time 50.042: pid 5989 finish time Tue Sep 5 20:01:43 2006 There is no firewall between the server and client. They are both running the latest Red Hat Linux AS. I have these settings in my amanda.conf: etimeout2400 dtimeout1800 ctimeout30 I've searched through the archives and found nothing, everything worked fine with 2.4.5p1. Ideas? Thanks, Steve -- Steven J. BackusComputer Specialist University of Utah E-Mail: [EMAIL PROTECTED] Biomedical Informatics Alternate: [EMAIL PROTECTED] 391 Chipeta Way -- Suite D150 Office: 801.587.9308 Salt Lake City, UT 84108-1266 http://www.math.utah.edu/~backus
Re: Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)
On Wed, 6 Sep 2006, Ian Turner wrote:
> On Wednesday 06 September 2006 04:23, Geert Uytterhoeven wrote:
> > So my ideal backup solution would be Amanda, with support for incrementally
> > storing backups at a remote location :-)
>
> Well, Amanda does that, via incremental backups. What it doesn't do (because
> of tool support) is incremental backups of individual files -- mostly because
> we don't have (I'm not aware of) any tool that does that.
Except that from time to time you need a level 0, which is big. Switching to
pure-incremental doesn't help, since then you (a) need to keep the initial
level 0 forever and (b) restore will be painful since you have to go throughall
incrementals.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [EMAIL PROTECTED]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)
On Wednesday 06 September 2006 04:23, Geert Uytterhoeven wrote: > So my ideal backup solution would be Amanda, with support for incrementally > storing backups at a remote location :-) Well, Amanda does that, via incremental backups. What it doesn't do (because of tool support) is incremental backups of individual files -- mostly because we don't have (I'm not aware of) any tool that does that. I am aware of how to create such a tool, however, and it's something we might do once Application API (or maybe Filter API; I'm not quite sure how to fit it in) lands. Cheers, --Ian -- Wiki for Amanda documentation: http://wiki.zmanda.com/
Re: amanda.conf include file question
On Wed, Sep 06, 2006 at 09:10:30AM -0400, McGraw, Robert P. wrote: > > > I am looking at using the includefile feature in the amanda configuration > file. > > I am running 2.4.5P1. > > 1) Is the includefile available in 2.4.5P1. Its been around a long time, so yes. > > 2) What would happen if I have the same configuration parameters in both the > include file and the amanda.conf file? Which one would have precedence? Does > order matter? In one of my older setups I broke lots of common things out of amanda.conf to be read in with "includefile". Things like dumptypes, tapetypes, templates, holding disks, network interfaces. Sure made amanda.conf much more reasonable size. I did run into an order-dependency situation that I'm unsure has been changed or not in more recent releases. JLM explained my observations and I'm paraphrasing from memory here, so if any errors are mine. Some parameters have both global and per-DLE values. If not set in a DLE, the global value is used. Global parameters have default values, but if ever set to a value, that value becomes the global value. Dump- type definitions use the global values in effect when the dumptype is defined, and are unaffected by any later changes to the global values. In my case, one of the includefiles was setting a parameter before any setting in the main amanda.conf. It thus became the global value while all my "includefile" dumptypes were defined and the later setting of the same parameter in the the main file did not affect the dumptype definitions. The solution in my case was simple, move the includefiles to the end of amanda.conf. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: amanda.conf include file question
On Wednesday 06 September 2006 09:10, McGraw, Robert P. wrote: >I am looking at using the includefile feature in the amanda configuration >file. > >I am running 2.4.5P1. > >1) Is the includefile available in 2.4.5P1. > >2) What would happen if I have the same configuration parameters in both > the include file and the amanda.conf file? Which one would have > precedence? Does order matter? That last one read takes precedence, Robert, so AIUI, it would be in the order of amanda.conf, then the individual include/exclude in that dumptype, or if files are named in the dumptype, then whatever is in that file will be the final control. >Thanks > >Robert > > >_ >Robert P. McGraw, Jr. >Manager, Computer System EMAIL: [EMAIL PROTECTED] >Purdue University ROOM: MATH-807 >Department of MathematicsPHONE: (765) 494-6055 >150 N. University Street FAX: (419) 821-0540 >West Lafayette, IN 47907-2067 -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)
On Wednesday 06 September 2006 04:23, Geert Uytterhoeven wrote:
>On Tue, 5 Sep 2006, Phil Howard wrote:
>> If you want all those benefits of restore, and don't mind having a disk
>> with a filesystem already on it, then why not use something like rsync
>> to make backups? As long as you aren't working with over about a
>> million individual files, it works great. It makes a replica of a
>> filesystem or multi-filesystem tree, and gives you direct access to
>> every individual file for restore purpose. Use multiple disks to make
>> multiple backups. When backing up to a disk previously used, rsync
>> avoids the writing work for files not changed (according to matching
>> meta data, though this can be turned off). And rsync works well over a
>> network via ssh.
>>
>> So I can't really understand your argument. What you seem to
>> specifically want that dismisses raw disk might well be better served
>> with rsync instead of Amanda. I might want Amanda, though, for huge
>> volume and speed.
>
>Now it starts to become interesting :-)
>
>This is actually what I've been in mind to post since a long time...
>First, let's say I use Amanda and vtapes to backup my home systems.
>
>I like Amanda, because it's simple to set up, robust, ease of recovery,
> ... However, storing backups offsite over the Internet (say, on a remote
> disk at a friend's place) is not an option, due to the monthly upload
> quota enforced by all ISPs here (in Belgium).
>
>I like rsync, since it only transfers what needs to be transfered. But it
>doesn't keep multiple days of backups and hard links can be tricky.
Writing a nearly identical rsync line for crontab, to be exec'd only on x
day of the week, such that rsync uses a different directory on the raid
for each (active) day of the week is one way to handle this problem.
We've been doing that at the tv station for about 4 years now.
We've had to build a bigger raid of course, at least twice, starting at
320GB but the last rebuild took it over the terrabyte marker by quite a
bit.
Its been very handy. We can lose a drive in a very important machine,
replace it, re-install the os, then rsync its data from the raid, and have
that machine back in service as if nothing ever happened in less than a
day's elapsed time & with only an hour or 2 of actual, on the machine
work. And thats getting faster as gigabit cards and switches are being
cycled into to replace the now aging 100base-T stuffs.
>I tried rdiff-backup, which keeps reverse-incrementals, but it can take
> lots of memory on the client side (i.e. not suitable to backup old
> machines) and doesn't work well with hard links.
>
>I also use duplicity, which keeps reverse-incrementals and supports
> encryption and authentication (nice for offsite backups of my digital
> pictures on a big scratch disk at work :-), but it can take lots of
> space on $TMPDIR on the client side, and it doesn't support hard links.
>
>So my ideal backup solution would be Amanda, with support for
> incrementally storing backups at a remote location :-)
>
>In theory, it should be possible to write a tool to take the tar archives
> as created by Amanda and calculate differentials, and reassemble the tar
> archives at the other end of the network pipe, right? Or are there
> better solutions?
>
One idea might be to have another drive located remotely, set it up
similarly to the vtape lashup amanda is using, with a pair of crontab
entries, one to re-cycle the 'data' link on the remote drive in a round
robin fashion, and then rsync /path/to/data to the remotes /path/to/data
sometime later in the morning after amanda has finished. I've thought of
doing that from here to my shops machine, but that mobo doesn't like 2
drives on the same pata cable even if they are the same brand of drives.
Of course, looking at the bigger picture, if a fire took this house, but
left the shop standing, I'd have a hell of a lot more important problems
than recovering this machine...
>Gr{oetje,eeting}s,
>
> Geert
>
>--
>Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
> [EMAIL PROTECTED]
>
>In personal conversations with technical people, I call myself a hacker.
> But when I'm talking to journalists I just say "programmer" or something
> like that. -- Linus Torvalds
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
amanda.conf include file question
I am looking at using the includefile feature in the amanda configuration file. I am running 2.4.5P1. 1) Is the includefile available in 2.4.5P1. 2) What would happen if I have the same configuration parameters in both the include file and the amanda.conf file? Which one would have precedence? Does order matter? Thanks Robert _ Robert P. McGraw, Jr. Manager, Computer System EMAIL: [EMAIL PROTECTED] Purdue University ROOM: MATH-807 Department of MathematicsPHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067 smime.p7s Description: S/MIME cryptographic signature
Re: Can I change the /tmp/amanda dir somewhere on the client host to another mount point as I have disabled executables in /tmp dir
Hi List sorry I had a typo " No Amanda Admin errors" Just to let you know solved my issue it was my SuSE Default Firewall I ran tcpdump -X -s 1500 udp and port 10080 0 packets captured 0 packets received by filter 0 packets dropped by kernel Thus via Yast turned the Firewall off and after running my daily/monthly script file on the tape server I got "No" Amanda Admin Errors. Cheers List.On Wed, 2006-09-06 at 11:57 +0100, Chuck Amadi Systems Administrator wrote: > Hi List > > Just to let you know solved my issue it was my SuSE Default Firewall I > ran > > tcpdump -X -s 1500 udp and port 10080 > > 0 packets captured > 0 packets received by filter > 0 packets dropped by kernel > > Thus via Yast turned the Firewall off and after running my > daily/monthly > script file on the tape server I got Amanda Admin Errors. > > Cheers List. -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
Re: Can I change the /tmp/amanda dir somewhere on the client host to another mount point as I have disabled executables in /tmp dir
On 2006-09-06 12:26, Chuck Amadi Systems Administrator wrote: [...] server#/tmp/amanda # less amandad.20060906111651.debug [...] amandad: time 29.997: dgram_recv: timeout after 30 seconds amandad: error receiving message: timeout amandad: time 29.998: error receiving message: timeout amandad: time 29.998: pid 498 finish time Wed Sep 6 11:17:21 2006 This proves that the client amandad has no problems with /tmp or how it is mounted. Good. Thus is there any tell tale signs on why Amanda Client process amanadad is not working properly and causing my tape server to "selfcheck request timed out message" Now run the amcheck on the server again, and see what messages you got in /tmp/amanda. If nothing at all got created, than ythe check request didn't even reach the host/port, or (x)inetd couldn't start amandad. Please note that I have added ports 10080 10082 and 10083 within SuSE SLES 9 Default Firewall and I also disbaled it and it didn't make any difference. Plus it's on my side of the firewall. Firewall is still not to be ruled out... Check (x)inetd settings too. (like "disable = yes" :-) ) -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: Can I change the /tmp/amanda dir somewhere on the client host to another mount point as I have disabled executables in /tmp dir
Hi List I have removed /tmp/amanda directory. Thus cd /usr/lib/amanda/ and run ./amandad This re created the amanda directory and the debug file: server#/tmp/amanda # less amandad.20060906111651.debug amandad: debug 1 pid 498 ruid 37 euid 37: start at Wed Sep 6 11:16:51 2006 amandad: version 2.4.4p2 amandad: build: VERSION="Amanda-2.4.4p2" amandad:BUILT_DATE="Wed Jun 30 23:45:53 UTC 2004" amandad:BUILT_MACH="Linux eisenstein 2.6.5 #1 Thu Nov 14 12:14:04 UTC 2002 i686 i686 i386 GNU/Linux" amandad:CC="gcc" amandad:CONFIGURE_COMMAND="'./configure' '--mandir=/usr/share/man' '--prefix=/usr' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--libdir=/usr/ lib' '--libexecdir=/usr/lib/amanda' '--localstatedir=/var/lib' '--with-index-server=localhost' '--with-gnutar-listdir=/var/lib/amanda/gnutar-lists' '--wit h-smbclient=/usr/bin/smbclient' '--with-amandahosts' '--with-user=amanda' '--with-group=disk' '--with-gnutar=/bin/tar' '--disable-libtool' '--disable-shar ed' '--disable-static'" amandad: paths: bindir="/usr/bin" sbindir="/usr/sbin" amandad:libexecdir="/usr/lib/amanda" mandir="/usr/share/man" amandad:AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda" amandad:CONFIG_DIR="/etc/amanda" DEV_PREFIX="/dev/" amandad:RDEV_PREFIX="/dev/r" DUMP="/sbin/dump" amandad:RESTORE="/sbin/restore" VDUMP=UNDEF VRESTORE=UNDEF amandad:XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF amandad:SAMBA_CLIENT="/usr/bin/smbclient" GNUTAR="/bin/tar" amandad:COMPRESS_PATH="/usr/bin/gzip" amandad:UNCOMPRESS_PATH="/usr/bin/gzip" LPRCMD="/usr/bin/lpr" amandad:MAILER="/usr/bin/Mail" amandad:listed_incr_dir="/var/lib/amanda/gnutar-lists" amandad: defs: DEFAULT_SERVER="localhost" DEFAULT_CONFIG="DailySet1" amandad:DEFAULT_TAPE_SERVER="localhost" amandad:DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM amandad:LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE amandad:AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS amandad:CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP amandad:COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast" amandad:COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc" amandad: time 29.997: dgram_recv: timeout after 30 seconds amandad: error receiving message: timeout amandad: time 29.998: error receiving message: timeout amandad: time 29.998: pid 498 finish time Wed Sep 6 11:17:21 2006 Thus is there any tell tale signs on why Amanda Client process amanadad is not working properly and causing my tape server to "selfcheck request timed out message" Please note that I have added ports 10080 10082 and 10083 within SuSE SLES 9 Default Firewall and I also disbaled it and it didn't make any difference. Plus it's on my side of the firewall. Cheers List. On Tue, 2006-09-05 at 12:19 -0400, Ian Turner wrote: > On Tuesday 05 September 2006 10:18, Chuck Amadi Systems Administrator wrote: > > Is it possible to set the AMANDA_DBGDIR directory (usually /tmp/amanda) > > to some thing else example /usr/tmp/amanda . > > Yes, but you have to compile from source. When you configure the compilation, > use --with-debugging=/path/to/amanda/debug/files. > > Cheers, > > --Ian > -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)
On Tue, 5 Sep 2006, Phil Howard wrote:
> If you want all those benefits of restore, and don't mind having a disk
> with a filesystem already on it, then why not use something like rsync
> to make backups? As long as you aren't working with over about a million
> individual files, it works great. It makes a replica of a filesystem or
> multi-filesystem tree, and gives you direct access to every individual
> file for restore purpose. Use multiple disks to make multiple backups.
> When backing up to a disk previously used, rsync avoids the writing work
> for files not changed (according to matching meta data, though this can
> be turned off). And rsync works well over a network via ssh.
>
> So I can't really understand your argument. What you seem to specifically
> want that dismisses raw disk might well be better served with rsync instead
> of Amanda. I might want Amanda, though, for huge volume and speed.
Now it starts to become interesting :-)
This is actually what I've been in mind to post since a long time...
First, let's say I use Amanda and vtapes to backup my home systems.
I like Amanda, because it's simple to set up, robust, ease of recovery, ...
However, storing backups offsite over the Internet (say, on a remote disk at a
friend's place) is not an option, due to the monthly upload quota enforced by
all ISPs here (in Belgium).
I like rsync, since it only transfers what needs to be transfered. But it
doesn't keep multiple days of backups and hard links can be tricky.
I tried rdiff-backup, which keeps reverse-incrementals, but it can take lots of
memory on the client side (i.e. not suitable to backup old machines) and
doesn't work well with hard links.
I also use duplicity, which keeps reverse-incrementals and supports encryption
and authentication (nice for offsite backups of my digital pictures on a big
scratch disk at work :-), but it can take lots of space on $TMPDIR on the
client side, and it doesn't support hard links.
So my ideal backup solution would be Amanda, with support for incrementally
storing backups at a remote location :-)
In theory, it should be possible to write a tool to take the tar archives as
created by Amanda and calculate differentials, and reassemble the tar archives
at the other end of the network pipe, right? Or are there better solutions?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [EMAIL PROTECTED]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: Can I change the /tmp/amanda dir somewhere on the client host to another mount point as I have disabled executables in /tmp dir
Hi List Thanks for your help and direction. As a previous post from Paul stated my /etc/fstab entries for the mounted /tmp shouldn't prevent amanda from writing the debug logs So I am going go back to basics and trouble shoot why "amcheck' report `selfcheck request timed out" for my web server. Cheers On Tue, 2006-09-05 at 10:50 -0400, Gene Heskett wrote: > On Tuesday 05 September 2006 10:18, Chuck Amadi Systems Administrator > wrote: > >Hi List > > > >Is it possible to set the AMANDA_DBGDIR directory (usually /tmp/amanda) > >to some thing else example /usr/tmp/amanda . > > Yes you can Chuck, but its in the configuration as you prepare to build > amanda, not in the amanda.conf. So to move it, you'll need to build the > tarball. 2.4.2 is now very long in the tooth anyway. > > Here is an example of my ./configure drivers script: > > #!/bin/sh > # since I'm always forgetting to su amanda... > if [ `whoami` != 'amanda' ]; then > echo > echo " Warning " > echo "Amanda needs to be configured and built by the user amanda," > echo "but must be installed by user root." > echo > exit 1 > fi > make clean > rm -f config.status config.cache > ./configure --with-user=amanda \ > --with-group=disk \ > --with-owner=amanda \ > --with-gnu-ld \ > --prefix=/usr/local \ > --with-tapedev="FILE:/amandatapes" \ > --with-debugging=/tmp/amanda-dbg/ \ > --with-tape-server=coyote \ > --with-bsdtcp-security --with-amandahosts \ > --with-configdir=/usr/local/etc/amanda \ > --with-config=Daily \ > --with-gnutar=/usr/local/bin/tar > > make > -- > > Its possible that the SuSe src rpm could be rebuilt after the appropriate > edits and effect the change that way, but I've never tried that, mainly > because most of the rpms are also configured to accept localhost, a huge > security hole but its the only way to make an rpm that will run anyplace. > > I can build and install a new tarball snapshot and amcheck it, on two > machines in less than 20 minutes. The above script is of course for the > server, the clients script is slightly simpler. > > >I am using Amanda 2.4.2 that came with SuSE Linux ES 9 server. > >Which was installed using Yast. > > > >Cheers > > > >Chuck > > > > Tue, 2006-09-05 at 13:49 +0100, Chuck Amadi Systems Administrator > > >wrote: > >> Hi List > >> > >> Got a good one here! > >> > >> Mi boss decode to tighten up are new web server which is due to be > >> commissioned soon thus when I re partitioned the hard disk I created > >> new disks and thus create a /tmp dir mounted on /dev/hda9 > >> > >> Thus here is my /etc/ftsab thus I stopped the possibilities of runing > >> any executables from the /tmp directory but thus amanda cant create > >> selfchecks in /tmp/amanda > >> > >> /dev/hda9/tmp reiserfs noexec,auto,nouser,rw,async,nosuid 1 2 > >> > >> Thus I am using amanda default program using Yast2 So is there a > >> parameter I can change on the client to use and create another > >> directory for example /usr/tmp/amanda which /usr directory has fstab > >> entry as below > >> > >> /dev/hda8 /usr reiserfs acl,user_xattr 1 2 > >> > >> Cheers > -- Unix/ Linux Systems Administrator Chuck Amadi The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Email chuck.smtl.co.uk Tel: +44 1656 752820 Fax: +44 1656 752830
