Re: dumporder
Am 06.11.18 um 19:18 schrieb Chris Nighswonger: This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Did you succeed? ;-) I currently try to use amplot on Debian 11.5 and get a ps file where the margins don't match the text etc I also think of generating a png-file after every amdump run and mailing it as well.
Re: dumporder
On Tue, Nov 06, 2018 at 01:18:15PM -0500, Chris Nighswonger wrote: > This seems to work: > > amplot /var/backups/campus/log/amdump.1 > > Running under the amanda user. > > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. > You have the permissions on your X screen restrictive (as they should probably be). Thus if you logged in a nathan and ran a program that needs the screen it cannot. The "xhost" command controls the access permissions. jl -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
RE: dumporder
Yah, the plot is a very helpful tool. You can get it to work harder if you can enlarge the work area or add an additional work area. Also you might want to check your chunk size, tendency, at least when I started out, was for many smaller files, I increased the size of the chunks in the holding area and believe it help to improve performance as there were few files and fewer file creates/accesses/deletes later on. From: Chris Nighswonger Sent: Tuesday, November 6, 2018 2:38 PM To: Cuttler, Brian R (HEALTH) Cc: ned.danie...@duke.edu; amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Setting the output to postscript (-p) and then converting to pdf (ps2pdf) worked the trick. It looks like my holding disk maxes out. [20181106020001.jpg] Christopher Nighswonger Faculty Member Network & Systems Director Foundations Bible College & Seminary www.foundations.edu<https://protect2.fireeye.com/url?k=5fb1533cbfe0ffe5.5fb3aa09-e5969cc601a99a3a=http://www.foundations.edu> www.fbcradio.org<https://protect2.fireeye.com/url?k=7e89f7371429ff69.7e8b0e02-990af0384ddad8fd=http://www.fbcradio.org> - NOTICE: The information contained in this electronic mail message is intended only for the use of the intended recipient, and may also be protected by the Electronic Communications Privacy Act, 18 USC Sections 2510-2521. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the sender, and delete the original message. Thank you. On Tue, Nov 6, 2018 at 2:26 PM Cuttler, Brian R (HEALTH) mailto:brian.cutt...@health.ny.gov>> wrote: Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org<mailto:owner-amanda-us...@amanda.org> mailto:owner-amanda-us...@amanda.org>> On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu<mailto:ned.danie...@duke.edu> Cc: amanda-users@amanda.org<mailto:amanda-users@amanda.org> Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley mailto:ned.danie...@duke.edu>> wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu<mailto:ned.danie...@duke.edu>) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
On Tue, Nov 06, 2018 at 13:18:15 -0500, Chris Nighswonger wrote: > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. Not sure if this is more useful than getting it to write directly to a PNG file, but the amplot man page lists an existing "-p" option to generate a PostScript file in place of trying to display directly on the screen via X11. Nathan Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
Re: dumporder
This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs > > load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my system, I > have to run amplot as user 'amanda'. that means that I have to be in a > directory where amanda has write permission, otherwise title can't be > generated. my home directory doesn't work, but a temp dir that's chmod 777 > does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > Digging around a bit, it appears that it might be a reference to a > file which is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs > load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P looks like the awk script is supposed to generate 'title'. on my system, I have to run amplot as user 'amanda'. that means that I have to be in a directory where amanda has write permission, otherwise title can't be generated. my home directory doesn't work, but a temp dir that's chmod 777 does. -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
My bad here. I should have inferred this from the man page for amplot, but I've been so wound up in the debug logs that I forgot there are others. This seems to work: amplot /var/backups/campus/log/amdump.1 On Tue, Nov 6, 2018 at 11:54 AM Cuttler, Brian R (HEALTH) wrote: > > I think you need to provide the name of the amconfig. I believe it reads > amanda.conf. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:51 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > Digging around a bit, it appears that it might be a reference to a file which > is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P > > On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) > wrote: > > > > It has been a while, title might be the org string from amanda.conf? > > I'm sorry, it has been years since I ran it regularly. > > > > -Original Message- > > From: Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:42 AM > > To: Cuttler, Brian R (HEALTH) > > Cc: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > The amplot utility is new to me. Here is what it says when I attempt to run > > it: > > > > root@scriptor:~# amplot > > /var/log/amanda/server/campus/amdump.20181106020001.debug > > Displaying graph on the screen, for next graph : MISSING SPACE > > DECLARATION "title", line 62: Cannot open script file 'title' > > > > I have both gnuplot and gawk installed on the backup server. > > > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > > wrote: > > > > > > Chris, > > > > > > There is an amplot utility that I used to run a lot, it would show me > > > what I was constrained on, often holding space, which if you are running > > > a bunch of large dumps to start off with could constrain dumping later on. > > > > > > > > > -Original Message- > > > From: owner-amanda-us...@amanda.org > > > On Behalf Of Chris Nighswonger > > > Sent: Tuesday, November 6, 2018 11:02 AM > > > To: amanda-users@amanda.org > > > Subject: Re: dumporder > > > > > > ATTENTION: This email came from an external source. Do not open > > > attachments or click on links from unknown senders or unexpected emails. > > > > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > > wrote: > > > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > > > inparallel 15 > > > dumporder "STs" > > > > > > Which resulted in a reduced time over what I have been seeing. Here are > > > some stats from amstatus for this config. It looks like I could drop > > > about half of the dumpers and still be fine. Any idea what causes the > > > dumpers over the first five to be utilized less? > > > > > > dumper0 busy : 5:13:42 ( 97.98%) > > > dumper1 busy : 1:12:09 ( 22.54%) > > > dumper2 busy : 0:40:30 ( 12.65%) > > > dumper3 busy : 0:33:44 ( 10.54%) > > > dumper4 busy : 0:03:47 ( 1.19%) > > > dumper5 busy : 0:37:32 ( 11.73%) > > > dumper6 busy : 0:02:00 ( 0.62%) > > > dumper7 busy : 0:00:57 ( 0.30%) > > > dumper8 busy
Re: dumporder
On Tue, Nov 06, 2018 at 11:42:16AM -0500, Chris Nighswonger wrote: > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION > "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. I have the same problem. the script 'amplot.g' says # file title has the parameters that this program needs load 'title' is the file 'title' supposed to be part of the distribution? -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
I think you need to provide the name of the amconfig. I believe it reads amanda.conf. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:51 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org > > On Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%)
Re: dumporder
Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org On > > Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( > > 99.98%) > > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( > > 99.59%) > > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( > > 56.35%) > > 5: 0:00:09 ( > > 22.36%) > > 4: 0
RE: dumporder
It has been a while, title might be the org string from amanda.conf? I'm sorry, it has been years since I ran it regularly. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:42 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On > Behalf Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
Re: dumporder
The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On Behalf > Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
RE: dumporder
Note on that - you want to maximize throughput, not maximize the number of running dumpers. More dumpers moving data is usually good, but more isn't always better. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Cuttler, Brian R (HEALTH) Sent: Tuesday, November 6, 2018 11:11 AM To: Chris Nighswonger ; amanda-users@amanda.org Subject: RE: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
RE: dumporder
Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On 11/5/2018 1:31 PM, Chris Nighswonger wrote: Is there any wisdom available on optimization of dumporder? This is personal experience only, but I find that in general, if all your dumps run at about the same speed and you don't have to worry about bandwidth, using something like the following generally gets reasonably good behavior: 'ssSS' In essence, it ensures that the smallest dumps complete fast, while still making sure the big ones get started early. Where I work, we've got a couple of slow systems, and I find that this works a bit better under those circumstances: 'ssST' Similar to the above, except it makes sure that the long backups get started early too (I would use 'ssTT', except that we only run one DLE at a time for any given host).
RE: dumporder
Depends on how many dumpers you are using, I like of like TSTSTSts, or something like that. Also very much depends on the size/length of the relative dumps, but I do like to kick off some of the longest ones early. Caveat – I haven’t done enough experimentation to know that what I’m doing is actually reasonable. From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Monday, November 5, 2018 1:31 PM To: amanda-users Subject: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Is there any wisdom available on optimization of dumporder? Kind regards, Chris
RE: dumporder parameter
Another question:
How can I verify that those DLE entries were in fact only dumped after
05:00? In the Amanda email report I can see when the backups completed,
and can more or less work out from that when writing to tape started for
them. But I'd like to check when they actually started dumping.
I checked through the logs in /var/log/amanda/server/daily, but the only
relevant thing that may be what I'm looking for is from
dumper.20080722235901.debug (my job started at 23:59, and I set up those
disklist entries with starttime 0500):
1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2
feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0
GNUTAR X X X
|;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list;
OPTIONS features=9ffe00;hostname=server2;
1216790552.535954: dumper: name = 'server2'
Any ideas on where I can find this information?
Thank you.
Johan
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen
Sent: 22 July 2008 17:07
To: amanda-users@amanda.org
Subject: RE: dumporder parameter
Hi,
No worries - I thought maybe my question was obvious to the point of not
being worthy a reply... :)
Paul Bijns did point out to me that the starttime parameter would be the
one to try, so I will do that. Here's what he said:
The dumpers are not spread in order of the disklist. You cannot map
the third letter to the third server.
There is the disklist parameter starttime:
starttime int
Default: none. Backups will not start until after this time of day.
The
value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as
1830.
e.g.
Server2/dir4/backups/backup1 {
comp-user-tar
dumpcycle 0 # force a full backup each time
starttime 0700
}
Server3 /dir5/backups/backup2 {
comp-user-tar
dumpcycle 0
starttime 0705
}
So I've implemented his suggestions and will see what happens on the
next Amanda run.
Thanks!
Johan
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Jon LaBadie
Sent: 22 July 2008 15:10
To: amanda-users@amanda.org
Subject: Re: dumporder parameter
On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote:
Hi,
I got no reply so apologies if it was a stupid question. I'm now
simply
using dumporder Ssss, and Amanda seems to be dumping disklist
entries
starting from largest to smallest, which will work just fine for me.
I'm doing level 0 backups each night, and know how long the largest
DLE
takes, so can reasonably accurately work out when to start the backup
job.
Anyways - I'll keep an eye on it just to make sure it keeps on doing
that.
John,
I didn't reply because I've never used the feature.
As no one else has, I'll suggest you look at the starttime
parameter. If specified in a dumptype the dump of DLEs using
that dumptype will not begin until after the indicated time.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen
Sent: 18 July 2008 21:20
To: amanda-users@amanda.org
Subject: dumporder parameter
I have a question about the dumporder parameter in amanda.conf.
I want to back up 4 servers. The volumes of data on disk are as
follows:
Server1
/dir1 298G
/dir2 74G
/dir3 66G
Server2
/dir4/backups/backup1 57G
Server3
/dir5/backups/backup2 28G
Server4
/dir6/backups/backup3 9.5G
My problem is that all the data to back up on Server2, Server3, and
Server4 isn't necessarily ready to be backed up until around 7am each
morning (this could possibly be moved to as early as 5am, but not by
much more than that). Server1 can be backed up at any time I wish
during the night. The sizes of the data to be backed up are fairly
static - I don't anticipate massive sudden increases (touch wood). I
also now have the capacity to force full backups each night. The data
on Servers 2, 3, and 4 will change completely each night anyway, so I
know I'll be backing up the amounts as above, even if I used Amanda
the
classic way of level 0s and level 1s.
How can I configure the dumporder parameter to first back up Server1
(I
know how long a level 0 of that will take to dump) before doing the
other servers? Then I can plan more accurately when to actually start
the backup job.
Something like SSsS, if I want to back up Server1 first, then
Server2,
then Server4, and then Server3 (Server3 takes the longest to create
the
data to back up, so should come last)?
No sooner had I gotten the backup server to work with two tape drives,
than one tape drive died. Now I have a fancy new tape drive with lots
of capacity, so I'm trying to consolidate backups that previously were
done seperately
Re: dumporder parameter
Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the TZ=UTC on the local computer to have the time in local time there). btw, the word PORT-DUMP indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
RE: dumporder parameter
Thanks very much. Makes perfect sense now. I did not expect that dump to bypass the holding disk, but I'm a bit low on holding disk space as things stand, so will look into that. Thanks for pointing that out, though. Much appreciated. Johan -Original Message- From: Paul Bijnens [mailto:[EMAIL PROTECTED] Sent: 23 July 2008 12:35 To: Johan Booysen Cc: amanda-users@amanda.org Subject: Re: dumporder parameter Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the TZ=UTC on the local computer to have the time in local time there). btw, the word PORT-DUMP indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
Re: dumporder parameter
On 2008-07-22 10:37, Johan Booysen wrote:
Hi,
I got no reply so apologies if it was a stupid question. I'm now simply
using dumporder Ssss, and Amanda seems to be dumping disklist entries
starting from largest to smallest, which will work just fine for me.
I'm doing level 0 backups each night, and know how long the largest DLE
takes, so can reasonably accurately work out when to start the backup
job.
Correction:
dumporder Ssss
actually means that the first dumper will choose the the largest
dump available from the queue, the second to the fourth dumper will
chose the smallest from the queue.
If you have only 1 dumper running (inparallel 1, or without holdingdisk,
or client constrained) then, and only then the dumps go from large to small
with that parameter. Use to be more certain about the largest
to smallest.
My problem is that all the data to back up on Server2, Server3, and
Server4 isn't necessarily ready to be backed up until around 7am each
morning (this could possibly be moved to as early as 5am, but not by
much more than that). Server1 can be backed up at any time I wish
during the night. The sizes of the data to be backed up are fairly
static - I don't anticipate massive sudden increases (touch wood). I
also now have the capacity to force full backups each night. The data
on Servers 2, 3, and 4 will change completely each night anyway, so I
know I'll be backing up the amounts as above, even if I used Amanda the
classic way of level 0s and level 1s.
How can I configure the dumporder parameter to first back up Server1 (I
know how long a level 0 of that will take to dump) before doing the
other servers? Then I can plan more accurately when to actually start
the backup job.
Something like SSsS, if I want to back up Server1 first, then Server2,
then Server4, and then Server3 (Server3 takes the longest to create the
data to back up, so should come last)?
The dumpers are not spread in order of the disklist. You cannot map
the third letter to the third server.
There is the disklist parameter starttime:
starttime int
Default: none. Backups will not start until after this time of day. The
value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830.
e.g.
Server2/dir4/backups/backup1 {
comp-user-tar
dumpcycle 0 # force a full backup each time
starttime 0700
}
Server3 /dir5/backups/backup2 {
comp-user-tar
dumpcycle 0
starttime 0705
}
--
Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512
http://www.xplanation.com/ email: [EMAIL PROTECTED]
***
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, *
* F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... Are you sure? ... YES ... Phew ... I'm out *
***
Re: dumporder parameter
On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote: Hi, I got no reply so apologies if it was a stupid question. I'm now simply using dumporder Ssss, and Amanda seems to be dumping disklist entries starting from largest to smallest, which will work just fine for me. I'm doing level 0 backups each night, and know how long the largest DLE takes, so can reasonably accurately work out when to start the backup job. Anyways - I'll keep an eye on it just to make sure it keeps on doing that. John, I didn't reply because I've never used the feature. As no one else has, I'll suggest you look at the starttime parameter. If specified in a dumptype the dump of DLEs using that dumptype will not begin until after the indicated time. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen Sent: 18 July 2008 21:20 To: amanda-users@amanda.org Subject: dumporder parameter I have a question about the dumporder parameter in amanda.conf. I want to back up 4 servers. The volumes of data on disk are as follows: Server1 /dir1 298G /dir2 74G /dir3 66G Server2 /dir4/backups/backup1 57G Server3 /dir5/backups/backup2 28G Server4 /dir6/backups/backup3 9.5G My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like SSsS, if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? No sooner had I gotten the backup server to work with two tape drives, than one tape drive died. Now I have a fancy new tape drive with lots of capacity, so I'm trying to consolidate backups that previously were done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the Amanda backup routine at all. I'd appreciate any advice. Johan End of included message -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
