RE: amflush during amdump
> > Will there be a conflict if I run amflush while an amdump > is running? > > > > I'll be running amdump like this so that I can accumulate enough > > multiple dumps in my holding disk before I amflush it to tape: > > > > amdump DailySet1 -o tapedev=/dev/null -o tpchanger="" > > (thanks to Jean-Louis for the taper options patch) > > > > > > Then I'll have a cron job that periodically checks the > amount of space > > used in the holding disk. If the size is close to the > capacity of one > > of my tapes, then amflush will run. > > > > So my question is, if amflush runs while amdump is running, > will it try > > to flush the dump that amdump is currently working on and cause any > > problems? > > > > It won't run, it will complain there is another process > already running. > What I would do is set autoflush=yes in amanda.conf, then > instead of running > a separate cron job just write a wrapper around your amdump > job. If the data > is less than your required level, call it with your -o > options, if it is > above, call as usual and it will flush everything (including > that run) to > tape. > > Matt > Ah, great idea. Thanks, Matt! > -- > Matt Hyclak > Department of Mathematics > Department of Social Work > Ohio University > (740) 593-1263 > This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: Question on bandwidth limits
Joel Coltoff wrote: > A recent thread on Loong backups got me to look at my configuration. It > always seemed to me that it should run a bit faster than it does. I wasn't too > concerned given that it started at 9:00 PM and was done by 2:00 AM. It never > got in the way of things. We are moving to a much larger server and I'd like > to resolve this for the flood of new files we'll have. We are running 2.4.4p2. > > I've been trying different numbers in my interface setup and they don't seem > to have any effect. This is what I have in my amanda.conf file. Most of the > DLEs are on the host that runs amanda. I don't have a setting for "netusage" > so I get the default. When the dump is done amstatus reports. > > network free kps: 10700 > > define interface local { > comment "a local disk" > use 1 kbps > } > > define interface ethernet { > comment "10 Mbps ethernet" > use 400 kbps You're limiting yourself quite a bit here. Amanda won't start another dumper if the current dumpers are using more than 4% of your bandwidth, so you'll probably never see two remote dumps happening simultaneously. I would suggest setting it much higher (like maybe 7 or 8 Mbps), and if other apps are impacted too much during the dumps, lower it a little bit each day until the other apps are no longer too affected. >} > > My disklist looks like this > > phoenix.wmi.com/export/cdburn project-files 2 local > phoenix.wmi.com/export/cac project-files 2 local > phoenix.wmi.com/export/opt project-files 2 local > phoenix.wmi.com/export/project project-files 2 local > > goliath.wmi.com /users /users { > user-files > exclude "./jc517/lightvr/*/*.o" > exclude append "./jc517/lightvr/*/*.bz2" > } 1 ethernet > > goliath.wmi.com /export /export { > user-files > include "./wumpus" "./plover" "./uclibc" "./vendor" > } 1 ethernet > > Finally, here is the tail of amstatus > > network free kps: 6700 > holding space : 10119040k ( 99.92%) > dumper0 busy : 4:20:38 ( 94.20%) > dumper1 busy : 0:02:16 ( 0.82%) >taper busy : 3:31:28 ( 76.43%) > 0 dumpers busy : 0:14:44 ( 5.33%)no-diskspace: 0:14:44 (100.00%) > 1 dumper busy : 4:19:40 ( 93.85%)no-bandwidth: 2:45:21 ( 63.68%) > not-idle: 1:34:19 ( 36.32%) > 2 dumpers busy : 0:02:16 ( 0.82%)no-bandwidth: 0:02:16 (100.00%) > > > If I run amstatus I'll see "no-bandwidth" associated with 1 dumper busy more > often than not. > What's a reasonable number to use so that I have more than 1 dumper running > at a > time? I guess the real question is should a single dump saturate connections > to the localhost? Unless you have other things running on your network while the backups are running, and they can't tolerate the slowdown (such as VOIP traffic), you should raise the bandwidth limit fairly high. A single dump may not be able to stream more than a few Mb/sec, due to factors such as disk speed, CPU, and network architecture. As for your 'local' speed (where the client and server are the same host), that interface has practically infinite bandwidth (not really, but from a backup perspective it does), your limiting factor is bus, disk controller, and disk I/O, and you're better off using a combination of spindle numbers and inparallel to control the local backups. Frank > > Thanks > - Joel > > -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: amflush during amdump
On Fri, Oct 27, 2006 at 01:10:57PM -0500, Lee, Raymond enlightened us: > Will there be a conflict if I run amflush while an amdump is running? > > I'll be running amdump like this so that I can accumulate enough > multiple dumps in my holding disk before I amflush it to tape: > > amdump DailySet1 -o tapedev=/dev/null -o tpchanger="" > (thanks to Jean-Louis for the taper options patch) > > > Then I'll have a cron job that periodically checks the amount of space > used in the holding disk. If the size is close to the capacity of one > of my tapes, then amflush will run. > > So my question is, if amflush runs while amdump is running, will it try > to flush the dump that amdump is currently working on and cause any > problems? > It won't run, it will complain there is another process already running. What I would do is set autoflush=yes in amanda.conf, then instead of running a separate cron job just write a wrapper around your amdump job. If the data is less than your required level, call it with your -o options, if it is above, call as usual and it will flush everything (including that run) to tape. Matt -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263
amflush during amdump
Hi List, Will there be a conflict if I run amflush while an amdump is running? I'll be running amdump like this so that I can accumulate enough multiple dumps in my holding disk before I amflush it to tape: amdump DailySet1 -o tapedev=/dev/null -o tpchanger="" (thanks to Jean-Louis for the taper options patch) Then I'll have a cron job that periodically checks the amount of space used in the holding disk. If the size is close to the capacity of one of my tapes, then amflush will run. So my question is, if amflush runs while amdump is running, will it try to flush the dump that amdump is currently working on and cause any problems? Thanks, Ray This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
results missing
I finally got past the gnutar problem I was having. I ran amcheck and everything looked good, but amdump gives me a bunch of results missing errors (even for hosts that worked fine before). Attached is the output from amdump. Thanks, Steve *** THE DUMPS DID NOT FINISH PROPERLY! These dumps were to tape VOL24. The next tape Amanda expects to use is: VOL01. FAILURE AND STRANGE DUMP SUMMARY: helios.medsci.domain /var lev 2 STRANGE snapserver.medsci.domain /hd/vol_mnt0/shares/biochem lev 1 FAILED [data write: Broken pipe] wagstaff.asel.domain /usr/local lev 1 FAILED [data write: Connection reset by peer] helios.medsci.domain /usr/local RESULTS MISSING lollipop.asel.domain /files1 RESULTS MISSING wagstaff.asel.domain /RESULTS MISSING wagstaff.asel.domain /files1 RESULTS MISSING wagstaff.asel.domain /files3 RESULTS MISSING wagstaff.asel.domain /files4 RESULTS MISSING wagstaff.asel.domain /files5 RESULTS MISSING wagstaff.asel.domain /files6/vol/SynthRESULTS MISSING wagstaff.asel.domain /files6/vol/VoicewareRESULTS MISSING wagstaff.asel.domain /files6/vol/bvd RESULTS MISSING wagstaff.asel.domain /files6/vol/spdata1 RESULTS MISSING wagstaff.asel.domain /files6/vol/spdata2 RESULTS MISSING wagstaff.asel.domain /files6/vol/speech7 RESULTS MISSING wagstaff.asel.domain /usr RESULTS MISSING wizard.asel.domain/var/mailRESULTS MISSING wizard.asel.domain/files2 RESULTS MISSING snapserver.medsci.domain /hd/vol_mnt0/shares/bcl RESULTS MISSING driver: FATAL Don't know how to send ABORT command to chunker chunker: FATAL error [bad command after RQ-MORE-DISK: "QUIT"] chunker: FATAL error [bad command after RQ-MORE-DISK: "QUIT"] STATISTICS: Total Full Incr. Estimate Time (hrs:min)0:10 Run Time (hrs:min) 0:14 Dump Time (hrs:min)0:14 0:00 0:14 Output Size (meg) 765.70.0 765.7 Original Size (meg) 1948.10.0 1948.1 Avg Compressed Size (%)39.3--39.3 (level:#disks ...) Filesystems Dumped 36 0 36 (1:35 2:1) Avg Dump Rate (k/s) 966.6-- 966.6 Tape Time (hrs:min)0:04 0:00 0:04 Tape Size (meg) 767.70.0 767.7 Tape Used (%) 2.10.02.1 (level:#disks ...) Filesystems Taped36 0 36 (1:35 2:1) (level:#chunks ...) Chunks Taped 36 0 36 (1:35 2:1) Avg Tp Write Rate (k/s) 3038.4-- 3038.4 USAGE BY TAPE: Label Time Size %NbNc VOL24 0:04 786080k2.13636 FAILED AND STRANGE DUMP DETAILS: /-- helios.medsci.domain /var lev 2 STRANGE sendbackup: start [helios.medsci.domain:/var level 2] sendbackup: info BACKUP=/bin/tar sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... - sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end ? gtar: ./lib/amanda/asel/index/professor.asel.domain/_usr_local/20061026_1.gz.tmp: Warning: Cannot stat: No such file or directory ? gtar: ./lib/amanda/asel/index/snapserver.medsci.domain/_hd_vol__mnt0_shares_NeuroGen/20061026_1.gz.tmp: Warning: Cannot stat: No such file or directory | gtar: ./run/postgresql/.s.PGSQL.5432: socket ignored | Total bytes written: 1115033600 (1.1GiB, 6.5MiB/s) sendbackup: size 1088900 sendbackup: end \ /-- snapserver.medsci.domain /hd/vol_mnt0/shares/biochem lev 1 FAILED [data write: Broken pipe] sendbackup: start [snapserver.medsci.domain:/hd/vol_mnt0/shares/biochem level 1] sendbackup: info BACKUP=/bin/tar sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... - sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end \ /-- wagstaff.asel.domain /usr/local lev 1 FAILED [data write: Connection reset by peer] sendbackup: start [wagstaff.asel.domain:/usr/local level 1] sendbackup: info BACKUP=/usr/sbin/ufsdump sendbackup: info RECOVER_CMD=/usr/local/bin/gzip -dc |/usr/sbin/ufsrestore -f... - sendbackup: info COMPRESS_SUFFIX=.gz sendbackup: info end | DUMP: Writing 32 Kilobyte records | DUMP: Date of this level 1 dump: Thu Oct 26 23:55:10 2006 | DUMP: Date of last level 0 dump: Mon Oct 23 23:55:43 2006 | DUMP: Dumping /dev/rdsk/c0t0d0s7 (wagstaff:/usr/local) to standard output. | DUMP: Mapping (Pass I) [regular files] | DUMP: Mapping (Pass II) [directories] | DUMP: Mapping (Pass II) [directories] | DUMP: Mapping (Pass II) [directories] | DUMP: Estimated 11706368
Amstatus report question.
I get this when I run amstatus --config daily --date. 8 dumpers busy : 0:00:39 ( 0.19%) no-dumpers: 0:00:35 (89.89%) start-wait: 0:00:03 ( 10.11%) 9 dumpers busy : 0:21:16 ( 6.13%) no-dumpers: 0:19:34 (92.00%) start-wait: 0:01:42 (8.00%) 10 dumpers busy : 1:02:29 ( 18.01%) no-dumpers: 1:02:25 (99.89%) start-wait: 0:00:04 ( What does the "no-dumpers" and "start-wait" mean. Thanks Robert _ Robert P. McGraw, Jr. Manager, Computer System EMAIL: [EMAIL PROTECTED] Purdue University ROOM: MATH-807 Department of MathematicsPHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067 smime.p7s Description: S/MIME cryptographic signature
Question on bandwidth limits
A recent thread on Loong backups got me to look at my configuration. It always seemed to me that it should run a bit faster than it does. I wasn't too concerned given that it started at 9:00 PM and was done by 2:00 AM. It never got in the way of things. We are moving to a much larger server and I'd like to resolve this for the flood of new files we'll have. We are running 2.4.4p2. I've been trying different numbers in my interface setup and they don't seem to have any effect. This is what I have in my amanda.conf file. Most of the DLEs are on the host that runs amanda. I don't have a setting for "netusage" so I get the default. When the dump is done amstatus reports. network free kps: 10700 define interface local { comment "a local disk" use 1 kbps } define interface ethernet { comment "10 Mbps ethernet" use 400 kbps } My disklist looks like this phoenix.wmi.com/export/cdburn project-files 2 local phoenix.wmi.com/export/cac project-files 2 local phoenix.wmi.com/export/opt project-files 2 local phoenix.wmi.com/export/project project-files 2 local goliath.wmi.com /users /users { user-files exclude "./jc517/lightvr/*/*.o" exclude append "./jc517/lightvr/*/*.bz2" } 1 ethernet goliath.wmi.com /export /export { user-files include "./wumpus" "./plover" "./uclibc" "./vendor" } 1 ethernet Finally, here is the tail of amstatus network free kps: 6700 holding space : 10119040k ( 99.92%) dumper0 busy : 4:20:38 ( 94.20%) dumper1 busy : 0:02:16 ( 0.82%) taper busy : 3:31:28 ( 76.43%) 0 dumpers busy : 0:14:44 ( 5.33%)no-diskspace: 0:14:44 (100.00%) 1 dumper busy : 4:19:40 ( 93.85%)no-bandwidth: 2:45:21 ( 63.68%) not-idle: 1:34:19 ( 36.32%) 2 dumpers busy : 0:02:16 ( 0.82%)no-bandwidth: 0:02:16 (100.00%) If I run amstatus I'll see "no-bandwidth" associated with 1 dumper busy more often than not. What's a reasonable number to use so that I have more than 1 dumper running at a time? I guess the real question is should a single dump saturate connections to the localhost? Thanks - Joel
tar 1.16 (was: Re: Problems with gtar)
On Fri, 27 Oct 2006, Steve Wray wrote: > Paul Yeatman wrote: > > ->>In response to your message<<- > > --received from Jean-Louis Martineau-- > >> > >>> I have been seeing the same thing for about two weeks. > >>> > [snip] > > This is all with Debian Etch. > > > > Now I'm experiencing similar problems again as explained in this thread > > but with Fedora 5 boxes. > > > > I find it a bit bothersome that not just one but two linux > > distributions are releasing packages of incompatible versions of tar > > and amanda. I suppose this is reasonable considering the complexity of > > the issue and that both are progressive versions of the distributions? > > It may be a Good Idea to make certain that this issue is brought to the > attention of the Debian package maintainer for amanda, hence the CC to > Bdale Garbee. > > If this *isn't* fixed before Etch freezes (some time in December?) its > unlikely that it will *ever* be fixed (since its not a security issue) > and will remain broken until the next release of Debian. tar 1.16 has just entered Debian sid/unstable. It's supposed to fix the problems with incrementals, but I haven't tried it yet. Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [EMAIL PROTECTED] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds