Question on bandwidth limits
A recent thread on Loong backups got me to look at my configuration. It
always seemed to me that it should run a bit faster than it does. I wasn't too
concerned given that it started at 9:00 PM and was done by 2:00 AM. It never
got in the way of things. We are moving to a much larger server and I'd like
to resolve this for the flood of new files we'll have. We are running 2.4.4p2.
I've been trying different numbers in my interface setup and they don't seem
to have any effect. This is what I have in my amanda.conf file. Most of the
DLEs are on the host that runs amanda. I don't have a setting for netusage
so I get the default. When the dump is done amstatus reports.
network free kps: 10700
define interface local {
comment a local disk
use 1 kbps
}
define interface ethernet {
comment 10 Mbps ethernet
use 400 kbps
}
My disklist looks like this
phoenix.wmi.com/export/cdburn project-files 2 local
phoenix.wmi.com/export/cac project-files 2 local
phoenix.wmi.com/export/opt project-files 2 local
phoenix.wmi.com/export/project project-files 2 local
goliath.wmi.com /users /users {
user-files
exclude ./jc517/lightvr/*/*.o
exclude append ./jc517/lightvr/*/*.bz2
} 1 ethernet
goliath.wmi.com /export /export {
user-files
include ./wumpus ./plover ./uclibc ./vendor
} 1 ethernet
Finally, here is the tail of amstatus
network free kps: 6700
holding space : 10119040k ( 99.92%)
dumper0 busy : 4:20:38 ( 94.20%)
dumper1 busy : 0:02:16 ( 0.82%)
taper busy : 3:31:28 ( 76.43%)
0 dumpers busy : 0:14:44 ( 5.33%)no-diskspace: 0:14:44 (100.00%)
1 dumper busy : 4:19:40 ( 93.85%)no-bandwidth: 2:45:21 ( 63.68%)
not-idle: 1:34:19 ( 36.32%)
2 dumpers busy : 0:02:16 ( 0.82%)no-bandwidth: 0:02:16 (100.00%)
If I run amstatus I'll see no-bandwidth associated with 1 dumper busy more
often than not.
What's a reasonable number to use so that I have more than 1 dumper running at a
time? I guess the real question is should a single dump saturate connections
to the localhost?
Thanks
- Joel
Amstatus report question.
I get this when I run amstatus --config daily --date. 8 dumpers busy : 0:00:39 ( 0.19%) no-dumpers: 0:00:35 (89.89%) start-wait: 0:00:03 ( 10.11%) 9 dumpers busy : 0:21:16 ( 6.13%) no-dumpers: 0:19:34 (92.00%) start-wait: 0:01:42 (8.00%) 10 dumpers busy : 1:02:29 ( 18.01%) no-dumpers: 1:02:25 (99.89%) start-wait: 0:00:04 ( What does the no-dumpers and start-wait mean. Thanks Robert _ Robert P. McGraw, Jr. Manager, Computer System EMAIL: [EMAIL PROTECTED] Purdue University ROOM: MATH-807 Department of MathematicsPHONE: (765) 494-6055 150 N. University Street FAX: (419) 821-0540 West Lafayette, IN 47907-2067 smime.p7s Description: S/MIME cryptographic signature
results missing
I finally got past the gnutar problem I was having. I ran amcheck and
everything looked good, but amdump gives me a bunch of results missing
errors (even for hosts that worked fine before). Attached is the output
from amdump.
Thanks,
Steve
*** THE DUMPS DID NOT FINISH PROPERLY!
These dumps were to tape VOL24.
The next tape Amanda expects to use is: VOL01.
FAILURE AND STRANGE DUMP SUMMARY:
helios.medsci.domain /var lev 2 STRANGE
snapserver.medsci.domain /hd/vol_mnt0/shares/biochem lev 1 FAILED
[data write: Broken pipe]
wagstaff.asel.domain /usr/local lev 1 FAILED
[data write: Connection reset by peer]
helios.medsci.domain /usr/local RESULTS MISSING
lollipop.asel.domain /files1 RESULTS MISSING
wagstaff.asel.domain /RESULTS MISSING
wagstaff.asel.domain /files1 RESULTS MISSING
wagstaff.asel.domain /files3 RESULTS MISSING
wagstaff.asel.domain /files4 RESULTS MISSING
wagstaff.asel.domain /files5 RESULTS MISSING
wagstaff.asel.domain /files6/vol/SynthRESULTS MISSING
wagstaff.asel.domain /files6/vol/VoicewareRESULTS MISSING
wagstaff.asel.domain /files6/vol/bvd RESULTS MISSING
wagstaff.asel.domain /files6/vol/spdata1 RESULTS MISSING
wagstaff.asel.domain /files6/vol/spdata2 RESULTS MISSING
wagstaff.asel.domain /files6/vol/speech7 RESULTS MISSING
wagstaff.asel.domain /usr RESULTS MISSING
wizard.asel.domain/var/mailRESULTS MISSING
wizard.asel.domain/files2 RESULTS MISSING
snapserver.medsci.domain /hd/vol_mnt0/shares/bcl RESULTS MISSING
driver: FATAL Don't know how to send ABORT command to chunker
chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT]
chunker: FATAL error [bad command after RQ-MORE-DISK: QUIT]
STATISTICS:
Total Full Incr.
Estimate Time (hrs:min)0:10
Run Time (hrs:min) 0:14
Dump Time (hrs:min)0:14 0:00 0:14
Output Size (meg) 765.70.0 765.7
Original Size (meg) 1948.10.0 1948.1
Avg Compressed Size (%)39.3--39.3 (level:#disks ...)
Filesystems Dumped 36 0 36 (1:35 2:1)
Avg Dump Rate (k/s) 966.6-- 966.6
Tape Time (hrs:min)0:04 0:00 0:04
Tape Size (meg) 767.70.0 767.7
Tape Used (%) 2.10.02.1 (level:#disks ...)
Filesystems Taped36 0 36 (1:35 2:1)
(level:#chunks ...)
Chunks Taped 36 0 36 (1:35 2:1)
Avg Tp Write Rate (k/s) 3038.4-- 3038.4
USAGE BY TAPE:
Label Time Size %NbNc
VOL24 0:04 786080k2.13636
FAILED AND STRANGE DUMP DETAILS:
/-- helios.medsci.domain /var lev 2 STRANGE
sendbackup: start [helios.medsci.domain:/var level 2]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
? gtar:
./lib/amanda/asel/index/professor.asel.domain/_usr_local/20061026_1.gz.tmp:
Warning: Cannot stat: No such file or directory
? gtar:
./lib/amanda/asel/index/snapserver.medsci.domain/_hd_vol__mnt0_shares_NeuroGen/20061026_1.gz.tmp:
Warning: Cannot stat: No such file or directory
| gtar: ./run/postgresql/.s.PGSQL.5432: socket ignored
| Total bytes written: 1115033600 (1.1GiB, 6.5MiB/s)
sendbackup: size 1088900
sendbackup: end
\
/-- snapserver.medsci.domain /hd/vol_mnt0/shares/biochem lev 1 FAILED
[data write: Broken pipe]
sendbackup: start [snapserver.medsci.domain:/hd/vol_mnt0/shares/biochem
level 1]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/gzip -dc |/bin/tar -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
\
/-- wagstaff.asel.domain /usr/local lev 1 FAILED [data write:
Connection reset by peer]
sendbackup: start [wagstaff.asel.domain:/usr/local level 1]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/local/bin/gzip -dc
|/usr/sbin/ufsrestore -f... -
sendbackup: info COMPRESS_SUFFIX=.gz
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 1 dump: Thu Oct 26 23:55:10 2006
| DUMP: Date of last level 0 dump: Mon Oct 23 23:55:43 2006
| DUMP: Dumping /dev/rdsk/c0t0d0s7 (wagstaff:/usr/local) to standard
output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 11706368
amflush during amdump
Hi List, Will there be a conflict if I run amflush while an amdump is running? I'll be running amdump like this so that I can accumulate enough multiple dumps in my holding disk before I amflush it to tape: amdump DailySet1 -o tapedev=/dev/null -o tpchanger= (thanks to Jean-Louis for the taper options patch) Then I'll have a cron job that periodically checks the amount of space used in the holding disk. If the size is close to the capacity of one of my tapes, then amflush will run. So my question is, if amflush runs while amdump is running, will it try to flush the dump that amdump is currently working on and cause any problems? Thanks, Ray This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: amflush during amdump
On Fri, Oct 27, 2006 at 01:10:57PM -0500, Lee, Raymond enlightened us: Will there be a conflict if I run amflush while an amdump is running? I'll be running amdump like this so that I can accumulate enough multiple dumps in my holding disk before I amflush it to tape: amdump DailySet1 -o tapedev=/dev/null -o tpchanger= (thanks to Jean-Louis for the taper options patch) Then I'll have a cron job that periodically checks the amount of space used in the holding disk. If the size is close to the capacity of one of my tapes, then amflush will run. So my question is, if amflush runs while amdump is running, will it try to flush the dump that amdump is currently working on and cause any problems? It won't run, it will complain there is another process already running. What I would do is set autoflush=yes in amanda.conf, then instead of running a separate cron job just write a wrapper around your amdump job. If the data is less than your required level, call it with your -o options, if it is above, call as usual and it will flush everything (including that run) to tape. Matt -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263
Re: Question on bandwidth limits
Joel Coltoff wrote:
A recent thread on Loong backups got me to look at my configuration. It
always seemed to me that it should run a bit faster than it does. I wasn't too
concerned given that it started at 9:00 PM and was done by 2:00 AM. It never
got in the way of things. We are moving to a much larger server and I'd like
to resolve this for the flood of new files we'll have. We are running 2.4.4p2.
I've been trying different numbers in my interface setup and they don't seem
to have any effect. This is what I have in my amanda.conf file. Most of the
DLEs are on the host that runs amanda. I don't have a setting for netusage
so I get the default. When the dump is done amstatus reports.
network free kps: 10700
define interface local {
comment a local disk
use 1 kbps
}
define interface ethernet {
comment 10 Mbps ethernet
use 400 kbps
You're limiting yourself quite a bit here. Amanda won't
start another dumper if the current dumpers are using more
than 4% of your bandwidth, so you'll probably never see
two remote dumps happening simultaneously. I would suggest
setting it much higher (like maybe 7 or 8 Mbps), and if
other apps are impacted too much during the dumps, lower it
a little bit each day until the other apps are no longer
too affected.
}
My disklist looks like this
phoenix.wmi.com/export/cdburn project-files 2 local
phoenix.wmi.com/export/cac project-files 2 local
phoenix.wmi.com/export/opt project-files 2 local
phoenix.wmi.com/export/project project-files 2 local
goliath.wmi.com /users /users {
user-files
exclude ./jc517/lightvr/*/*.o
exclude append ./jc517/lightvr/*/*.bz2
} 1 ethernet
goliath.wmi.com /export /export {
user-files
include ./wumpus ./plover ./uclibc ./vendor
} 1 ethernet
Finally, here is the tail of amstatus
network free kps: 6700
holding space : 10119040k ( 99.92%)
dumper0 busy : 4:20:38 ( 94.20%)
dumper1 busy : 0:02:16 ( 0.82%)
taper busy : 3:31:28 ( 76.43%)
0 dumpers busy : 0:14:44 ( 5.33%)no-diskspace: 0:14:44 (100.00%)
1 dumper busy : 4:19:40 ( 93.85%)no-bandwidth: 2:45:21 ( 63.68%)
not-idle: 1:34:19 ( 36.32%)
2 dumpers busy : 0:02:16 ( 0.82%)no-bandwidth: 0:02:16 (100.00%)
If I run amstatus I'll see no-bandwidth associated with 1 dumper busy more
often than not.
What's a reasonable number to use so that I have more than 1 dumper running
at a
time? I guess the real question is should a single dump saturate connections
to the localhost?
Unless you have other things running on your network while the
backups are running, and they can't tolerate the slowdown (such
as VOIP traffic), you should raise the bandwidth limit fairly high.
A single dump may not be able to stream more than a few Mb/sec,
due to factors such as disk speed, CPU, and network architecture.
As for your 'local' speed (where the client and server are the same
host), that interface has practically infinite bandwidth (not really,
but from a backup perspective it does), your limiting factor is bus,
disk controller, and disk I/O, and you're better off using a
combination of spindle numbers and inparallel to control the local
backups.
Frank
Thanks
- Joel
--
Frank Smith [EMAIL PROTECTED]
Sr. Systems Administrator Voice: 512-374-4673
Hoover's Online Fax: 512-374-4501
RE: amflush during amdump
Will there be a conflict if I run amflush while an amdump is running? I'll be running amdump like this so that I can accumulate enough multiple dumps in my holding disk before I amflush it to tape: amdump DailySet1 -o tapedev=/dev/null -o tpchanger= (thanks to Jean-Louis for the taper options patch) Then I'll have a cron job that periodically checks the amount of space used in the holding disk. If the size is close to the capacity of one of my tapes, then amflush will run. So my question is, if amflush runs while amdump is running, will it try to flush the dump that amdump is currently working on and cause any problems? It won't run, it will complain there is another process already running. What I would do is set autoflush=yes in amanda.conf, then instead of running a separate cron job just write a wrapper around your amdump job. If the data is less than your required level, call it with your -o options, if it is above, call as usual and it will flush everything (including that run) to tape. Matt Ah, great idea. Thanks, Matt! -- Matt Hyclak Department of Mathematics Department of Social Work Ohio University (740) 593-1263 This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
