Re: dumporder
Am 06.11.18 um 19:18 schrieb Chris Nighswonger: This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Did you succeed? ;-) I currently try to use amplot on Debian 11.5 and get a ps file where the margins don't match the text etc I also think of generating a png-file after every amdump run and mailing it as well.
Re: dumporder
On Tue, Nov 06, 2018 at 01:18:15PM -0500, Chris Nighswonger wrote: > This seems to work: > > amplot /var/backups/campus/log/amdump.1 > > Running under the amanda user. > > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. > You have the permissions on your X screen restrictive (as they should probably be). Thus if you logged in a nathan and ran a program that needs the screen it cannot. The "xhost" command controls the access permissions. jl -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
RE: dumporder
Yah, the plot is a very helpful tool. You can get it to work harder if you can enlarge the work area or add an additional work area. Also you might want to check your chunk size, tendency, at least when I started out, was for many smaller files, I increased the size of the chunks in the holding area and believe it help to improve performance as there were few files and fewer file creates/accesses/deletes later on. From: Chris Nighswonger Sent: Tuesday, November 6, 2018 2:38 PM To: Cuttler, Brian R (HEALTH) Cc: ned.danie...@duke.edu; amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Setting the output to postscript (-p) and then converting to pdf (ps2pdf) worked the trick. It looks like my holding disk maxes out. [20181106020001.jpg] Christopher Nighswonger Faculty Member Network & Systems Director Foundations Bible College & Seminary www.foundations.edu<https://protect2.fireeye.com/url?k=5fb1533cbfe0ffe5.5fb3aa09-e5969cc601a99a3a=http://www.foundations.edu> www.fbcradio.org<https://protect2.fireeye.com/url?k=7e89f7371429ff69.7e8b0e02-990af0384ddad8fd=http://www.fbcradio.org> - NOTICE: The information contained in this electronic mail message is intended only for the use of the intended recipient, and may also be protected by the Electronic Communications Privacy Act, 18 USC Sections 2510-2521. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the sender, and delete the original message. Thank you. On Tue, Nov 6, 2018 at 2:26 PM Cuttler, Brian R (HEALTH) mailto:brian.cutt...@health.ny.gov>> wrote: Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org<mailto:owner-amanda-us...@amanda.org> mailto:owner-amanda-us...@amanda.org>> On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu<mailto:ned.danie...@duke.edu> Cc: amanda-users@amanda.org<mailto:amanda-users@amanda.org> Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley mailto:ned.danie...@duke.edu>> wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu<mailto:ned.danie...@duke.edu>) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
On Tue, Nov 06, 2018 at 13:18:15 -0500, Chris Nighswonger wrote: > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. Not sure if this is more useful than getting it to write directly to a PNG file, but the amplot man page lists an existing "-p" option to generate a PostScript file in place of trying to display directly on the screen via X11. Nathan Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
Re: dumporder
This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs > > load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my system, I > have to run amplot as user 'amanda'. that means that I have to be in a > directory where amanda has write permission, otherwise title can't be > generated. my home directory doesn't work, but a temp dir that's chmod 777 > does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
AW: dumporder
Hello Chris, Just a guess without access to a amanda box currently: maybe the file is somewhere in the distribution and a) not installed correctly or b) is expected to be in the current directory, so a 'cd' to another directory might help. Regards, Ingo Originalnachricht Von: Chris Nighswonger Gesendet: Dienstag, 6. November 2018 17:51 An: brian.cutt...@health.ny.gov Cc: amanda-users@amanda.org Betreff: Re: dumporder Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot "run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org On > > Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > > 5: 0:00:09 ( 22.36%) > > 4: 0:00:08 ( 18.89%) > > 1: 0:00:01 ( 2.37%) > > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > > 5: 0:00:02 ( 7.13%) > > 4: 0:00:01 ( 3.09%) > > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > > 12 dumpers busy : 0
Re: dumporder
On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > Digging around a bit, it appears that it might be a reference to a > file which is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs > load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P looks like the awk script is supposed to generate 'title'. on my system, I have to run amplot as user 'amanda'. that means that I have to be in a directory where amanda has write permission, otherwise title can't be generated. my home directory doesn't work, but a temp dir that's chmod 777 does. -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
My bad here. I should have inferred this from the man page for amplot, but I've been so wound up in the debug logs that I forgot there are others. This seems to work: amplot /var/backups/campus/log/amdump.1 On Tue, Nov 6, 2018 at 11:54 AM Cuttler, Brian R (HEALTH) wrote: > > I think you need to provide the name of the amconfig. I believe it reads > amanda.conf. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:51 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > Digging around a bit, it appears that it might be a reference to a file which > is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P > > On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) > wrote: > > > > It has been a while, title might be the org string from amanda.conf? > > I'm sorry, it has been years since I ran it regularly. > > > > -Original Message- > > From: Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:42 AM > > To: Cuttler, Brian R (HEALTH) > > Cc: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > The amplot utility is new to me. Here is what it says when I attempt to run > > it: > > > > root@scriptor:~# amplot > > /var/log/amanda/server/campus/amdump.20181106020001.debug > > Displaying graph on the screen, for next graph : MISSING SPACE > > DECLARATION "title", line 62: Cannot open script file 'title' > > > > I have both gnuplot and gawk installed on the backup server. > > > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > > wrote: > > > > > > Chris, > > > > > > There is an amplot utility that I used to run a lot, it would show me > > > what I was constrained on, often holding space, which if you are running > > > a bunch of large dumps to start off with could constrain dumping later on. > > > > > > > > > -Original Message- > > > From: owner-amanda-us...@amanda.org > > > On Behalf Of Chris Nighswonger > > > Sent: Tuesday, November 6, 2018 11:02 AM > > > To: amanda-users@amanda.org > > > Subject: Re: dumporder > > > > > > ATTENTION: This email came from an external source. Do not open > > > attachments or click on links from unknown senders or unexpected emails. > > > > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > > wrote: > > > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > > > inparallel 15 > > > dumporder "STs" > > > > > > Which resulted in a reduced time over what I have been seeing. Here are > > > some stats from amstatus for this config. It looks like I could drop > > > about half of the dumpers and still be fine. Any idea what causes the > > > dumpers over the first five to be utilized less? > > > > > > dumper0 busy : 5:13:42 ( 97.98%) > > > dumper1 busy : 1:12:09 ( 22.54%) > > > dumper2 busy : 0:40:30 ( 12.65%) > > > dumper3 busy : 0:33:44 ( 10.54%) > > > dumper4 busy : 0:03:47 ( 1.19%) > > > dumper5 busy : 0:37:32 ( 11.73%) > > > dumper6 busy : 0:02:00 ( 0.62%) > > > dumper7 busy : 0:00:57 ( 0.30%) > > > dumper8 busy
Re: dumporder
On Tue, Nov 06, 2018 at 11:42:16AM -0500, Chris Nighswonger wrote: > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION > "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. I have the same problem. the script 'amplot.g' says # file title has the parameters that this program needs load 'title' is the file 'title' supposed to be part of the distribution? -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
I think you need to provide the name of the amconfig. I believe it reads amanda.conf. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:51 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org > > On Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%)
Re: dumporder
Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org On > > Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( > > 99.98%) > > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( > > 99.59%) > > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( > > 56.35%) > > 5: 0:00:09 ( > > 22.36%) > > 4: 0
RE: dumporder
It has been a while, title might be the org string from amanda.conf? I'm sorry, it has been years since I ran it regularly. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:42 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On > Behalf Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
Re: dumporder
The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On Behalf > Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
RE: dumporder
Note on that - you want to maximize throughput, not maximize the number of running dumpers. More dumpers moving data is usually good, but more isn't always better. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Cuttler, Brian R (HEALTH) Sent: Tuesday, November 6, 2018 11:11 AM To: Chris Nighswonger ; amanda-users@amanda.org Subject: RE: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
RE: dumporder
Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On 11/5/2018 1:31 PM, Chris Nighswonger wrote: Is there any wisdom available on optimization of dumporder? This is personal experience only, but I find that in general, if all your dumps run at about the same speed and you don't have to worry about bandwidth, using something like the following generally gets reasonably good behavior: 'ssSS' In essence, it ensures that the smallest dumps complete fast, while still making sure the big ones get started early. Where I work, we've got a couple of slow systems, and I find that this works a bit better under those circumstances: 'ssST' Similar to the above, except it makes sure that the long backups get started early too (I would use 'ssTT', except that we only run one DLE at a time for any given host).
RE: dumporder
Depends on how many dumpers you are using, I like of like TSTSTSts, or something like that. Also very much depends on the size/length of the relative dumps, but I do like to kick off some of the longest ones early. Caveat – I haven’t done enough experimentation to know that what I’m doing is actually reasonable. From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Monday, November 5, 2018 1:31 PM To: amanda-users Subject: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Is there any wisdom available on optimization of dumporder? Kind regards, Chris
dumporder
Is there any wisdom available on optimization of dumporder? Kind regards, Chris
RE: dumporder parameter
Another question:
How can I verify that those DLE entries were in fact only dumped after
05:00? In the Amanda email report I can see when the backups completed,
and can more or less work out from that when writing to tape started for
them. But I'd like to check when they actually started dumping.
I checked through the logs in /var/log/amanda/server/daily, but the only
relevant thing that may be what I'm looking for is from
dumper.20080722235901.debug (my job started at 23:59, and I set up those
disklist entries with starttime 0500):
1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2
feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0
GNUTAR X X X
|;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list;
OPTIONS features=9ffe00;hostname=server2;
1216790552.535954: dumper: name = 'server2'
Any ideas on where I can find this information?
Thank you.
Johan
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen
Sent: 22 July 2008 17:07
To: amanda-users@amanda.org
Subject: RE: dumporder parameter
Hi,
No worries - I thought maybe my question was obvious to the point of not
being worthy a reply... :)
Paul Bijns did point out to me that the starttime parameter would be the
one to try, so I will do that. Here's what he said:
The dumpers are not spread in order of the disklist. You cannot map
the third letter to the third server.
There is the disklist parameter starttime:
starttime int
Default: none. Backups will not start until after this time of day.
The
value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as
1830.
e.g.
Server2/dir4/backups/backup1 {
comp-user-tar
dumpcycle 0 # force a full backup each time
starttime 0700
}
Server3 /dir5/backups/backup2 {
comp-user-tar
dumpcycle 0
starttime 0705
}
So I've implemented his suggestions and will see what happens on the
next Amanda run.
Thanks!
Johan
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Jon LaBadie
Sent: 22 July 2008 15:10
To: amanda-users@amanda.org
Subject: Re: dumporder parameter
On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote:
Hi,
I got no reply so apologies if it was a stupid question. I'm now
simply
using dumporder Ssss, and Amanda seems to be dumping disklist
entries
starting from largest to smallest, which will work just fine for me.
I'm doing level 0 backups each night, and know how long the largest
DLE
takes, so can reasonably accurately work out when to start the backup
job.
Anyways - I'll keep an eye on it just to make sure it keeps on doing
that.
John,
I didn't reply because I've never used the feature.
As no one else has, I'll suggest you look at the starttime
parameter. If specified in a dumptype the dump of DLEs using
that dumptype will not begin until after the indicated time.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen
Sent: 18 July 2008 21:20
To: amanda-users@amanda.org
Subject: dumporder parameter
I have a question about the dumporder parameter in amanda.conf.
I want to back up 4 servers. The volumes of data on disk are as
follows:
Server1
/dir1 298G
/dir2 74G
/dir3 66G
Server2
/dir4/backups/backup1 57G
Server3
/dir5/backups/backup2 28G
Server4
/dir6/backups/backup3 9.5G
My problem is that all the data to back up on Server2, Server3, and
Server4 isn't necessarily ready to be backed up until around 7am each
morning (this could possibly be moved to as early as 5am, but not by
much more than that). Server1 can be backed up at any time I wish
during the night. The sizes of the data to be backed up are fairly
static - I don't anticipate massive sudden increases (touch wood). I
also now have the capacity to force full backups each night. The data
on Servers 2, 3, and 4 will change completely each night anyway, so I
know I'll be backing up the amounts as above, even if I used Amanda
the
classic way of level 0s and level 1s.
How can I configure the dumporder parameter to first back up Server1
(I
know how long a level 0 of that will take to dump) before doing the
other servers? Then I can plan more accurately when to actually start
the backup job.
Something like SSsS, if I want to back up Server1 first, then
Server2,
then Server4, and then Server3 (Server3 takes the longest to create
the
data to back up, so should come last)?
No sooner had I gotten the backup server to work with two tape drives,
than one tape drive died. Now I have a fancy new tape drive with lots
of capacity, so I'm trying to consolidate backups that previously were
done seperately
Re: dumporder parameter
Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the TZ=UTC on the local computer to have the time in local time there). btw, the word PORT-DUMP indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
RE: dumporder parameter
Thanks very much. Makes perfect sense now. I did not expect that dump to bypass the holding disk, but I'm a bit low on holding disk space as things stand, so will look into that. Thanks for pointing that out, though. Much appreciated. Johan -Original Message- From: Paul Bijnens [mailto:[EMAIL PROTECTED] Sent: 23 July 2008 12:35 To: Johan Booysen Cc: amanda-users@amanda.org Subject: Re: dumporder parameter Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the TZ=UTC on the local computer to have the time in local time there). btw, the word PORT-DUMP indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
Re: dumporder parameter
On 2008-07-22 10:37, Johan Booysen wrote:
Hi,
I got no reply so apologies if it was a stupid question. I'm now simply
using dumporder Ssss, and Amanda seems to be dumping disklist entries
starting from largest to smallest, which will work just fine for me.
I'm doing level 0 backups each night, and know how long the largest DLE
takes, so can reasonably accurately work out when to start the backup
job.
Correction:
dumporder Ssss
actually means that the first dumper will choose the the largest
dump available from the queue, the second to the fourth dumper will
chose the smallest from the queue.
If you have only 1 dumper running (inparallel 1, or without holdingdisk,
or client constrained) then, and only then the dumps go from large to small
with that parameter. Use to be more certain about the largest
to smallest.
My problem is that all the data to back up on Server2, Server3, and
Server4 isn't necessarily ready to be backed up until around 7am each
morning (this could possibly be moved to as early as 5am, but not by
much more than that). Server1 can be backed up at any time I wish
during the night. The sizes of the data to be backed up are fairly
static - I don't anticipate massive sudden increases (touch wood). I
also now have the capacity to force full backups each night. The data
on Servers 2, 3, and 4 will change completely each night anyway, so I
know I'll be backing up the amounts as above, even if I used Amanda the
classic way of level 0s and level 1s.
How can I configure the dumporder parameter to first back up Server1 (I
know how long a level 0 of that will take to dump) before doing the
other servers? Then I can plan more accurately when to actually start
the backup job.
Something like SSsS, if I want to back up Server1 first, then Server2,
then Server4, and then Server3 (Server3 takes the longest to create the
data to back up, so should come last)?
The dumpers are not spread in order of the disklist. You cannot map
the third letter to the third server.
There is the disklist parameter starttime:
starttime int
Default: none. Backups will not start until after this time of day. The
value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830.
e.g.
Server2/dir4/backups/backup1 {
comp-user-tar
dumpcycle 0 # force a full backup each time
starttime 0700
}
Server3 /dir5/backups/backup2 {
comp-user-tar
dumpcycle 0
starttime 0705
}
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: dumporder parameter
On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote: Hi, I got no reply so apologies if it was a stupid question. I'm now simply using dumporder Ssss, and Amanda seems to be dumping disklist entries starting from largest to smallest, which will work just fine for me. I'm doing level 0 backups each night, and know how long the largest DLE takes, so can reasonably accurately work out when to start the backup job. Anyways - I'll keep an eye on it just to make sure it keeps on doing that. John, I didn't reply because I've never used the feature. As no one else has, I'll suggest you look at the starttime parameter. If specified in a dumptype the dump of DLEs using that dumptype will not begin until after the indicated time. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen Sent: 18 July 2008 21:20 To: amanda-users@amanda.org Subject: dumporder parameter I have a question about the dumporder parameter in amanda.conf. I want to back up 4 servers. The volumes of data on disk are as follows: Server1 /dir1 298G /dir2 74G /dir3 66G Server2 /dir4/backups/backup1 57G Server3 /dir5/backups/backup2 28G Server4 /dir6/backups/backup3 9.5G My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like SSsS, if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? No sooner had I gotten the backup server to work with two tape drives, than one tape drive died. Now I have a fancy new tape drive with lots of capacity, so I'm trying to consolidate backups that previously were done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the Amanda backup routine at all. I'd appreciate any advice. Johan End of included message -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
Another dumporder parameter question.
When I backup I would like all my larger size files to be put on the first part of the tape and work down to the smaller size files toward the end of the tape. This I believe and hope will provide better tape usage. If large size files are dumped last to tape there is a good chance that it will hit EOT and will then have to start over and a large chunk of tape will have been wasted and so will time. If smaller size files are sent at the end then if it does hit EOT not much tape is wasted. I have been using SS. So my question: 1) dose Amanda get the size of all the backup files and then figure the order that it will write to tape? 2) will Amanda get the size of the tape and then figure the order that it will write files to tape. 3) will Amanda do the dumps in the order that it needs to write to tape. Robert smime.p7s Description: S/MIME cryptographic signature
Re: Another dumporder parameter question.
McGraw, Robert P wrote: When I backup I would like all my larger size files to be put on the first part of the tape and work down to the smaller size files toward the end of the tape. This I believe and hope will provide better tape usage. Yes, it does. If large size files are dumped last to tape there is a good chance that it will hit EOT and will then have to start over and a large chunk of tape will have been wasted and so will time. If smaller size files are sent at the end then if it does hit EOT not much tape is wasted. I have been using SS. So my question: 1) dose Amanda get the size of all the backup files and then figure the order that it will write to tape? Amanda does an estimate of each DLE, and from those figures decides on which dumps to choose first (dumporder). 2) will Amanda get the size of the tape and then figure the order that it will write files to tape. When taper is free to write the next dumpimage from holdingdisk to tape, then the taperalgo parameter decides which image from holdingdisk (only those that are completely done) is the next onej to write to tape. 3) will Amanda do the dumps in the order that it needs to write to tape. No. Dumporder is independent of tapeorder. But tapeorder is influenced by the availability of finished dumps in the holdingdisk. Dumps that bypass holdingdisk are always done last, even when, and mostly because, they are (too) large (to fit in holdingdisk space). See: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25
dumporder parameter
I have a question about the dumporder parameter in amanda.conf. I want to back up 4 servers. The volumes of data on disk are as follows: Server1 /dir1 298G /dir2 74G /dir3 66G Server2 /dir4/backups/backup1 57G Server3 /dir5/backups/backup2 28G Server4 /dir6/backups/backup3 9.5G My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like SSsS, if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? No sooner had I gotten the backup server to work with two tape drives, than one tape drive died. Now I have a fancy new tape drive with lots of capacity, so I'm trying to consolidate backups that previously were done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the Amanda backup routine at all. I'd appreciate any advice. Johan
Question about dumporder
Hello, I have a question about the dumporder parameter. To what are the two bandwidth options (B and b) related? Client/Server connection (e. g. 1Gbit)? Or the real bandwith between server and client? I have a client that is connected by a 1Gbit LAN-card to it's subnet in our branch office. This subnet uses a 4 Mbit connection to exchange data with the subnet in our head office. The amanda server (in the head office subnet) is also connected by a 1Gbit LAN-card. I wanna do some optimizations to ensure that the DLE's of the branch office are backuped first, because a full dump of all DLE's is running a long time (because of the slow connection) and I just want to make sure, that the dump of the client is done as early as possible Regards Marc -- Marc Muehlfeld (Leitung Systemadministration) Zentrum fuer Humangenetik und Laboratoriumsmedizin Dr. Klein und Dr. Rost Lochhamer Str. 29 - D-82152 Martinsried Telefon: +49(0)89/895578-0 - Fax: +49(0)89/895578-78 http://www.medizinische-genetik.de
Re: Question about dumporder
Jean-Louis Martineau schrieb: bandwidth estimation is done using the previous backup of the DLE, You can get the 'dump rates' with: amadmin config info Would be cool if this short info could be added to the dumporder option of the amanda.conf manpage. I didn't found this information. Thanks Regards Marc -- Marc Muehlfeld (Leitung Systemadministration) Zentrum fuer Humangenetik und Laboratoriumsmedizin Dr. Klein und Dr. Rost Lochhamer Str. 29 - D-82152 Martinsried Telefon: +49(0)89/895578-0 - Fax: +49(0)89/895578-78 http://www.medizinische-genetik.de
Re: Question about dumporder
bandwidth estimation is done using the previous backup of the DLE, You can get the 'dump rates' with: amadmin config info Jean-Louis Marc Muehlfeld wrote: Hello, I have a question about the dumporder parameter. To what are the two bandwidth options (B and b) related? Client/Server connection (e. g. 1Gbit)? Or the real bandwith between server and client? I have a client that is connected by a 1Gbit LAN-card to it's subnet in our branch office. This subnet uses a 4 Mbit connection to exchange data with the subnet in our head office. The amanda server (in the head office subnet) is also connected by a 1Gbit LAN-card. I wanna do some optimizations to ensure that the DLE's of the branch office are backuped first, because a full dump of all DLE's is running a long time (because of the slow connection) and I just want to make sure, that the dump of the client is done as early as possible Regards Marc
Good option for dumporder
I'm trying to set up the options in amanda.conf to improve the performance (this equates to shortest time to backup) of these five Windows NT/2K servers: server1 C$ 2.19GB server1 D$ 35.6GB server2 C$ 1.58GB server2 D$ 12.9GB server3 C$ 2.18GB server3 D$ 11.6GB server4 C$ 1.3GB server4 D$ 20.4GB server5 C$ 2.97GB server5 D$ 13.7GB If you run the backup on these servers single threaded it takes about 10 hours to backup. I've changed inparallel to 8 and maxdumps to 8 also. The dumporder is sssSsssS. In the latest test, the backup completed in 8 hours and 41 minutes. I'm using amanda with gnutar and Samba to do the backup. Does anyone have any ideas on how I may improve the performance of this backup? Thanks. Bob... -- Robert Zahn UNIX Systems Administrator Oklahoma City Community College S. May Avenue Oklahoma City, Ok 73159 [EMAIL PROTECTED][EMAIL PROTECTED] --
Re: Good option for dumporder
On Tue, 8 Jul 2003 at 11:21am, Bob Zahn wrote I'm trying to set up the options in amanda.conf to improve the performance (this equates to shortest time to backup) of these five Windows NT/2K servers: server1 C$ 2.19GB server1 D$ 35.6GB server2 C$ 1.58GB server2 D$ 12.9GB server3 C$ 2.18GB server3 D$ 11.6GB server4 C$ 1.3GB server4 D$ 20.4GB server5 C$ 2.97GB server5 D$ 13.7GB How many samba servers are you using? If you run the backup on these servers single threaded it takes about 10 hours to backup. I've changed inparallel to 8 and maxdumps to 8 If you have one samba server, and maxdumps 8, you may see a slowdown of your dumprates offsetting your increased parallelism. How do your dumprates compare? -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Good option for dumporder
On Tuesday 08 July 2003 12:21, Bob Zahn wrote: I'm trying to set up the options in amanda.conf to improve the performance (this equates to shortest time to backup) of these five Windows NT/2K servers: server1 C$ 2.19GB server1 D$ 35.6GB server2 C$ 1.58GB server2 D$ 12.9GB server3 C$ 2.18GB server3 D$ 11.6GB server4 C$ 1.3GB server4 D$ 20.4GB server5 C$ 2.97GB server5 D$ 13.7GB If you run the backup on these servers single threaded it takes about 10 hours to backup. I've changed inparallel to 8 and maxdumps to 8 also. The dumporder is sssSsssS. In the latest test, the backup completed in 8 hours and 41 minutes. I'm using amanda with gnutar and Samba to do the backup. Does anyone have any ideas on how I may improve the performance of this backup? Thanks. Bob... You didn't post your disklist Bob. And while you can use samba, if you can get cygwin installed on the m$ boxes so you can build and run an amanda client on each box, then you have the possibility of doing as show below. One of the things that will speed things up is to make sure that each disklist entry that is in fact on a different hard drive has a different, unique to that disk, 'spindle number', usually the last item on each line of the disklist. This will allow amanda to run in parallel, a client for each spindle, speeding things up considerably. I have no idea how this will play out in a samba environment however as after the first dozen runs, I learned to avoid that at all costs. This is due to the broken way windows and samba handle the files dateing, which results in an incremental being made into a full in most cases. Since that is never going to be fixed, its time to walk away Jim, the horse is dead. That, combined with making the clients do the compression so that many clients can all be working on the compression of their own stuff at the same as the others, and a 10 hour backup can become a 2 hour backup, depending of course on the speed of the tape drive itself. This is dependant on making cygwin run on your M$ boxes I think. Somebody correct me please if this in in-accurate due to recent progress on the windows front. My windows are all made of glass here, a 100% linux house. :) Robert Zahn UNIX Systems Administrator Oklahoma City Community College S. May Avenue Oklahoma City, Ok 73159 [EMAIL PROTECTED][EMAIL PROTECTED] -- -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.26% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Good option for dumporder
This is in response to Gene's and Joshua's additional quesitons:
1. I'm only running one Samba server and that's on the amanda host server.
2. Sorry, disklist looks like:
amanda1 //server1/C$ pc-tar 3 ge0
amanda1 //server1/D$ pc-tar 4 ge0
amanda1 //server2/C$ pc-tar 5 ge0
amanda1 //server2/D$ pc-tar 6 ge0
amanda1 //server3/C$ pc-tar 11 ge0
amanda1 //server3/D$ pc-tar 12 ge0
amanda1 //server4/C$ pc-tar 44 ge0
amanda1 //server4/D$ pc-tar 45 ge0
amanda1 //server5/C$ pc-tar 47 ge0
amanda1 //server5/D$ pc-tar 48 ge0
From amanda.conf
define dumptype pc-tar {
comment Full dump of this filesystem always
program GNUTAR
maxdumps 8
holdingdisk yes
compress none
index yes
priority high
strategy noinc
dumpcycle 0
}
Also, I have 2 holding disks, 20GB each with chunksize of 1GB.
3. I thought that the general opinion of this listserv was that cygwin was not a good
option and samba was better for backing up M$ servers. Have I got this backwards?
Bob...
-- Original Message --
From: Gene Heskett [EMAIL PROTECTED]
Reply-To: [EMAIL PROTECTED]
Date: Tue, 8 Jul 2003 13:28:20 -0400
On Tuesday 08 July 2003 12:21, Bob Zahn wrote:
I'm trying to set up the options in amanda.conf to improve the
performance (this equates to shortest time to backup) of these five
Windows NT/2K servers:
server1 C$ 2.19GB
server1 D$ 35.6GB
server2 C$ 1.58GB
server2 D$ 12.9GB
server3 C$ 2.18GB
server3 D$ 11.6GB
server4 C$ 1.3GB
server4 D$ 20.4GB
server5 C$ 2.97GB
server5 D$ 13.7GB
If you run the backup on these servers single threaded it takes
about 10 hours to backup. I've changed inparallel to 8 and maxdumps
to 8 also. The dumporder is sssSsssS. In the latest test, the
backup completed in 8 hours and 41 minutes. I'm using amanda with
gnutar and Samba to do the backup. Does anyone have any ideas on
how I may improve the performance of this backup? Thanks. Bob...
You didn't post your disklist Bob. And while you can use samba, if
you can get cygwin installed on the m$ boxes so you can build and run
an amanda client on each box, then you have the possibility of doing
as show below.
One of the things that will speed things up is to make sure that each
disklist entry that is in fact on a different hard drive has a
different, unique to that disk, 'spindle number', usually the last
item on each line of the disklist. This will allow amanda to run in
parallel, a client for each spindle, speeding things up considerably.
I have no idea how this will play out in a samba environment however
as after the first dozen runs, I learned to avoid that at all costs.
This is due to the broken way windows and samba handle the files
dateing, which results in an incremental being made into a full in
most cases. Since that is never going to be fixed, its time to walk
away Jim, the horse is dead.
That, combined with making the clients do the compression so that many
clients can all be working on the compression of their own stuff at
the same as the others, and a 10 hour backup can become a 2 hour
backup, depending of course on the speed of the tape drive itself.
This is dependant on making cygwin run on your M$ boxes I think.
Somebody correct me please if this in in-accurate due to recent
progress on the windows front. My windows are all made of glass
here, a 100% linux house. :)
Robert Zahn UNIX Systems Administrator
Oklahoma City Community College
S. May Avenue
Oklahoma City, Ok 73159
[EMAIL PROTECTED][EMAIL PROTECTED]
--
--
Cheers, Gene
AMD [EMAIL PROTECTED] 320M
[EMAIL PROTECTED] 512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
--
Robert Zahn UNIX Systems Administrator
Oklahoma City Community College
S. May Avenue
Oklahoma City, Ok 73159
[EMAIL PROTECTED][EMAIL PROTECTED]
--
Re: Good option for dumporder
On Tue, 8 Jul 2003 at 1:46pm, Bob Zahn wrote This is in response to Gene's and Joshua's additional quesitons: 1. I'm only running one Samba server and that's on the amanda host server. You'll want to look at your dumprates before and after increasing maxdumps. 8 may lead to too much contention (disk and net), slowing you down disproportionately. As for samba vs. cygwin, I still use samba here, although I don't do whole disk dumps. I just get user data -- I'm never going to recover a whole system from tape only. The only way to know what works best for you is to try it out. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Good option for dumporder
On Tuesday 08 July 2003 14:46, Bob Zahn wrote:
This is in response to Gene's and Joshua's additional quesitons:
1. I'm only running one Samba server and that's on the amanda host
server.
2. Sorry, disklist looks like:
amanda1 //server1/C$ pc-tar 3 ge0
amanda1 //server1/D$ pc-tar 4 ge0
amanda1 //server2/C$ pc-tar 5 ge0
amanda1 //server2/D$ pc-tar 6 ge0
amanda1 //server3/C$ pc-tar 11 ge0
amanda1 //server3/D$ pc-tar 12 ge0
amanda1 //server4/C$ pc-tar 44 ge0
amanda1 //server4/D$ pc-tar 45 ge0
amanda1 //server5/C$ pc-tar 47 ge0
amanda1 //server5/D$ pc-tar 48 ge0
From amanda.conf
define dumptype pc-tar {
comment Full dump of this filesystem always
program GNUTAR
maxdumps 8
holdingdisk yes
compress none
index yes
priority high
strategy noinc
dumpcycle 0
}
Also, I have 2 holding disks, 20GB each with chunksize of 1GB.
3. I thought that the general opinion of this listserv was that
cygwin was not a good option and samba was better for backing up M$
servers. Have I got this backwards?
I'm not the 'expert' in that arena, so I'll let those that have first
hand knowledge testify. But I'd suspect that success or failure is
more in the hands of cygwin than anything else. Personal opinion of
course. My 'samba' bad experience was with linux, not windows, on
the other end of the cableing.
--
Cheers, Gene
AMD [EMAIL PROTECTED] 320M
[EMAIL PROTECTED] 512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: amanda-2.4.3b1 / dumporder warning
Hello Roland,
Try this patch.
Jean-Louis
On Sat, Dec 08, 2001 at 12:03:29AM +, Roland E. Lipovits wrote:
Roland E. Lipovits [EMAIL PROTECTED] wrote:
Int the logfile and the report, there are many lines saying
| WARNING driver Unknown dumporder character '
The dumporder line in amanda.conf is:
| dumporder sssS # specify the priority order of each dumper
Any ideas why this warning is shown?
Grmbl. May be this happens because I have 'inparallel 5' and so
'dumporder' should be 5 chars long. I'll try that tomorrow.
Regards,
Lipo
--
They that can give up essential liberty | Roland E. Lipovits
to obtain a little temporary safety | Vienna, Austria
deserve neither liberty nor safety. | DSA-KeyID: 0xDA153FAB
- Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
--
Jean-Louis Martineau email: [EMAIL PROTECTED]
Departement IRO, Universite de Montreal
C.P. 6128, Succ. CENTRE-VILLETel: (514) 343-6111 ext. 3529
Montreal, Canada, H3C 3J7Fax: (514) 343-5834
--- server-src/driver.c.1 Sat Dec 8 10:48:36 2001
+++ server-src/driver.c Sat Dec 8 10:52:29 2001
@@ -584,7 +584,7 @@
if(!accept) {
char dumptype;
char *dumporder = getconf_str(CNF_DUMPORDER);
- if((strlen(dumporder)+1) = (dumper-dmptable)) {
+ if(strlen(dumporder) = (dumper-dmptable)) {
if(dumper-dmptable 3)
dumptype = 's';
else
Re: amanda-2.4.3b1 / dumporder warning
On Sat, Dec 08, 2001 at 10:55:27AM -0500, Jean-Louis Martineau wrote: Try this patch. I guess I should try this patch with the too short dumporder-string again, right? Regards, Lipo -- They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety. | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
Re: amanda-2.4.3b1 / dumporder warning
On Sat, Dec 08, 2001 at 01:51:21PM -0500, Jean-Louis Martineau wrote: I guess I should try this patch with the too short dumporder-string again, right? Right, it will work with a long dumporder-string but I will appreciate if you can test my patch. Ok, tested with too short dumporder-string and no warnings in logfile. Thanks, Lipo -- They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety. | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
amanda-2.4.3b1 / dumporder warning
Hi, I've tried the new amanda-2.4.3b1, with a amanda.conf file which had the new keywords (dumporder autoflush) in it, taken from example/amanda.conf. Int the logfile and the report, there are many lines saying | WARNING driver Unknown dumporder character ' The dumporder line in amanda.conf is: | dumporder sssS # specify the priority order of each dumper Any ideas why this warning is shown? Regards, Lipo -- They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety. | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
Re: amanda-2.4.3b1 / dumporder warning
Roland E. Lipovits [EMAIL PROTECTED] wrote: Int the logfile and the report, there are many lines saying | WARNING driver Unknown dumporder character ' The dumporder line in amanda.conf is: | dumporder sssS # specify the priority order of each dumper Any ideas why this warning is shown? Grmbl. May be this happens because I have 'inparallel 5' and so 'dumporder' should be 5 chars long. I'll try that tomorrow. Regards, Lipo -- They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety. | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
