Re: amanda-2.6.0p2: a few remarks and questions
amreport and amstatus outputs seem ok as the toc files. Thanks Jean-Louis! * Jean-Louis Martineau <[EMAIL PROTECTED]> [20081114 10:36]: > The patch was for 2.6.1. > > The attached patch is for 2.6.0. > > Jean-Louis > > Jean-Francois Malouin wrote: > >Jean-Louis, > > > >your new patch doesn't apply cleanly. Do I apply it on top of the > >first one on only the second one on a fresh tarball? > > > > > -- <° >< Jean-François Malouin McConnell Brain Imaging Centre System/Network AdministratorMontréal Neurological Institute 3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada
Re: amanda-2.6.0p2: a few remarks and questions
Try this newest patch.
Some fix will works only if the dump is done with the patched version.
Jean-Louis
Jean-Francois Malouin wrote:
* Jean-Louis Martineau <[EMAIL PROTECTED]> [2008 20:12]:
Hi Jean-François,
Can you try the attached patch?
Tried it but something is still missing. Mind you, I just applied
the patch and recompiled: this is from a previous run.
-amreport: patch works: it reports tape rate above 100MBs
Avg Tp Write Rate (k/s) 109575 109575--
-amstatus: I still don't get the lines 'tape X', like:
taped : 25422328m423178m ( 99.80%) ( 99.80%)
tape 1: 25422328m423178m (109.40%) av24_U00061L3 (79 chunks)
-amtoc: I was doing my own edit of the amtoc perl script when you sent
me the patch. If I run the patched amtoc on a previous log file I get
something like:
# Server:/partition date lev size[Kb]
0 av48-1_top_R00012L4:200811101 - -
1 gustav:/raid/prefrontal 200811101 0 *16777216
2 gustav:/raid/prefrontal 200811101 0 *16777216
...
21 gustav:/raid/prefrontal 200811101 0 *16777216
22 gustav:/raid/prefrontal 200811101 0 367489930
Notice the '*'. There is something wrong around line 228 of amtoc:
if(!/^CHUNK/){
# this case should never happend:
$strange=1;
$note = "*";
}
It inserts the '*' on the chunk size. Also the total:on_tape is missing.
This is simple perl stuff, I should be able to do something about it!
Thanks a lot.
jf
Jean-Louis
Jean-Francois Malouin wrote:
Hello,
I've finally got to install amanda-2.6.0p2 on a system going soon into
production and while getting used to the new features and I noticed a
few things that have changed since 2.5.x :
- I use amtoc after a dump ends to get stats on what was written on
tape and noticed that it doesn't parse correctly the log files and the
toc files end up not showing what was written on tape. Looking at the
source I believe that this is due to the fact that amtoc doesn't parse
lines that start with 'PART' as the new log format seems to use
compared to 'CHUNK' in pre-2.6.x.
- amstatus doesn't show which tape(s) is being used and how much data
been written to it so far, a useful feature when I want to check that
a flush is being done correctly when using a changer with 'runtapes>1'
- As I posted a few days ago, amreport doesn't report correctly the
aggragated transfer rate to tape beyond 99.99MiBs...
- The new version features 'device_output_buffer_size' that replaces
'tapebufs'. Any hint on a ballpark figure for this? Right now I have
set it to 8192k (I use a blocksize=2048k) but I really don't know if I
overshooting with this...the server has 32GB of memory btw.
Thank you for the amazing work you guys are doing!
jf
Index: server-src/amstatus.pl
===
--- server-src/amstatus.pl (revision 14198)
+++ server-src/amstatus.pl (working copy)
@@ -722,6 +722,16 @@
$ntsize{$nb_tape} = 0;
$ntesize{$nb_tape} = 0;
}
+ elsif($line[1] eq "using") {
+ #1:"using" #2:"label" #3:`label' #4:date #5 `timestamp'
+ $nb_tape++;
+ $lineX =~ /using label `(\S*)'/;
+ $label = $1;
+ $ntlabel{$nb_tape} = $label;
+ $ntpartition{$nb_tape} = 0;
+ $ntsize{$nb_tape} = 0;
+ $ntesize{$nb_tape} = 0;
+ }
}
elsif($line[0] eq "splitting" &&
$line[1] eq "chunk" &&
Index: server-src/driver.c
===
--- server-src/driver.c (revision 14198)
+++ server-src/driver.c (working copy)
@@ -303,6 +303,7 @@
conf_runtapes = getconf_int(CNF_RUNTAPES);
tape = lookup_tapetype(conf_tapetype);
tape_length = tapetype_get_length(tape);
+g_printf("driver: tape size %lld\n", (long long)tape_length);
conf_flush_threshold_dumped = getconf_int(CNF_FLUSH_THRESHOLD_DUMPED);
conf_flush_threshold_scheduled = getconf_int(CNF_FLUSH_THRESHOLD_SCHEDULED);
conf_taperflush = getconf_int(CNF_TAPERFLUSH);
Index: server-src/amtoc.pl
===
--- server-src/amtoc.pl (revision 14198)
+++ server-src/amtoc.pl (working copy)
@@ -162,11 +162,26 @@
}
if ( ! /^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+)/) { next;}
# $_ = $1;
- $host = $2;
- $disk = $3;
- $date = $4;
- $chunk = $5;
- $level = $6;
+ if (/PART taper/) {
+/^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+)/;
+$filenum = $3;
+$host = $4;
+$disk = $5;
+$date = $6;
+$chunk = $7;
+$level = $8;
+if ($filenum != $filenumber) {
+ # This should not be possible */
+ $filenumber = $filenum;
+}
+ } else {
+/^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+)/;
+$host = $2;
+$disk = $3;
+$date = $4;
+$chunk = $5;
+$level
Re: amanda-2.6.0p2: a few remarks and questions
* Jean-Louis Martineau <[EMAIL PROTECTED]> [2008 20:12]:
> Hi Jean-François,
>
> Can you try the attached patch?
Tried it but something is still missing. Mind you, I just applied
the patch and recompiled: this is from a previous run.
-amreport: patch works: it reports tape rate above 100MBs
Avg Tp Write Rate (k/s) 109575 109575--
-amstatus: I still don't get the lines 'tape X', like:
taped : 25422328m423178m ( 99.80%) ( 99.80%)
tape 1: 25422328m423178m (109.40%) av24_U00061L3 (79 chunks)
-amtoc: I was doing my own edit of the amtoc perl script when you sent
me the patch. If I run the patched amtoc on a previous log file I get
something like:
# Server:/partition date lev size[Kb]
0 av48-1_top_R00012L4:200811101 - -
1 gustav:/raid/prefrontal 200811101 0 *16777216
2 gustav:/raid/prefrontal 200811101 0 *16777216
...
21 gustav:/raid/prefrontal 200811101 0 *16777216
22 gustav:/raid/prefrontal 200811101 0 367489930
Notice the '*'. There is something wrong around line 228 of amtoc:
if(!/^CHUNK/){
# this case should never happend:
$strange=1;
$note = "*";
}
It inserts the '*' on the chunk size. Also the total:on_tape is missing.
This is simple perl stuff, I should be able to do something about it!
Thanks a lot.
jf
>
> Jean-Louis
>
> Jean-Francois Malouin wrote:
> >Hello,
> >
> >I've finally got to install amanda-2.6.0p2 on a system going soon into
> >production and while getting used to the new features and I noticed a
> >few things that have changed since 2.5.x :
> >
> >- I use amtoc after a dump ends to get stats on what was written on
> >tape and noticed that it doesn't parse correctly the log files and the
> >toc files end up not showing what was written on tape. Looking at the
> >source I believe that this is due to the fact that amtoc doesn't parse
> >lines that start with 'PART' as the new log format seems to use
> >compared to 'CHUNK' in pre-2.6.x.
> >
> >- amstatus doesn't show which tape(s) is being used and how much data
> >been written to it so far, a useful feature when I want to check that
> >a flush is being done correctly when using a changer with 'runtapes>1'
> >
> >- As I posted a few days ago, amreport doesn't report correctly the
> >aggragated transfer rate to tape beyond 99.99MiBs...
> >
> >- The new version features 'device_output_buffer_size' that replaces
> >'tapebufs'. Any hint on a ballpark figure for this? Right now I have
> >set it to 8192k (I use a blocksize=2048k) but I really don't know if I
> >overshooting with this...the server has 32GB of memory btw.
> >
> >Thank you for the amazing work you guys are doing!
> >jf
> >
>
--
<° >< Jean-François Malouin McConnell Brain Imaging Centre
System/Network AdministratorMontréal Neurological Institute
3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada
Re: amanda-2.6.0p2: a few remarks and questions
Hi Jean-François,
Can you try the attached patch?
Jean-Louis
Jean-Francois Malouin wrote:
Hello,
I've finally got to install amanda-2.6.0p2 on a system going soon into
production and while getting used to the new features and I noticed a
few things that have changed since 2.5.x :
- I use amtoc after a dump ends to get stats on what was written on
tape and noticed that it doesn't parse correctly the log files and the
toc files end up not showing what was written on tape. Looking at the
source I believe that this is due to the fact that amtoc doesn't parse
lines that start with 'PART' as the new log format seems to use
compared to 'CHUNK' in pre-2.6.x.
- amstatus doesn't show which tape(s) is being used and how much data
been written to it so far, a useful feature when I want to check that
a flush is being done correctly when using a changer with 'runtapes>1'
- As I posted a few days ago, amreport doesn't report correctly the
aggragated transfer rate to tape beyond 99.99MiBs...
- The new version features 'device_output_buffer_size' that replaces
'tapebufs'. Any hint on a ballpark figure for this? Right now I have
set it to 8192k (I use a blocksize=2048k) but I really don't know if I
overshooting with this...the server has 32GB of memory btw.
Thank you for the amazing work you guys are doing!
jf
Index: server-src/amstatus.pl
===
--- server-src/amstatus.pl (revision 14148)
+++ server-src/amstatus.pl (working copy)
@@ -722,6 +722,16 @@
$ntsize{$nb_tape} = 0;
$ntesize{$nb_tape} = 0;
}
+ elsif($line[1] eq "using") {
+ #1:"using" #2:"label" #3:`label' #4:date #5 `timestamp'
+ $nb_tape++;
+ $lineX =~ /using label `(\S*)'/;
+ $label = $1;
+ $ntlabel{$nb_tape} = $label;
+ $ntpartition{$nb_tape} = 0;
+ $ntsize{$nb_tape} = 0;
+ $ntesize{$nb_tape} = 0;
+ }
}
elsif($line[0] eq "splitting" &&
$line[1] eq "chunk" &&
Index: server-src/driver.c
===
--- server-src/driver.c (revision 14148)
+++ server-src/driver.c (working copy)
@@ -303,6 +303,7 @@
conf_runtapes = getconf_int(CNF_RUNTAPES);
tape = lookup_tapetype(conf_tapetype);
tape_length = tapetype_get_length(tape);
+g_printf("driver: tape size %lld\n", (long long)tape_length);
conf_flush_threshold_dumped = getconf_int(CNF_FLUSH_THRESHOLD_DUMPED);
conf_flush_threshold_scheduled = getconf_int(CNF_FLUSH_THRESHOLD_SCHEDULED);
conf_taperflush = getconf_int(CNF_TAPERFLUSH);
Index: server-src/amtoc.pl
===
--- server-src/amtoc.pl (revision 14148)
+++ server-src/amtoc.pl (working copy)
@@ -162,11 +162,26 @@
}
if ( ! /^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+)/) { next;}
# $_ = $1;
- $host = $2;
- $disk = $3;
- $date = $4;
- $chunk = $5;
- $level = $6;
+ if (/PART taper/) {
+/^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+)/;
+$filenum = $3;
+$host = $4;
+$disk = $5;
+$date = $6;
+$chunk = $7;
+$level = $8;
+if ($filenum != $filenumber) {
+ # This should not be possible */
+ $filenumber = $filenum;
+}
+ } else {
+/^([A-Z]+) taper (\S+) (\S+) (\S+) (\S+) (\S+)/;
+$host = $2;
+$disk = $3;
+$date = $4;
+$chunk = $5;
+$level = $6;
+ }
switch: {
/START taper/ && do {
$tocfilename=&tfn($chunk) if ($#subs >= 0);
@@ -191,7 +206,7 @@
&pr("#","Server","/partition","date", "level","size[Kb]","part");
&pr("$filenumber","$chunk","","$disk","-","-","-");
last switch; };
-/^(?:SUCCESS|CHUNK) taper/ && do {
+/^(?:SUCCESS|CHUNK|PART) taper/ && do {
if(/SUCCESS/){
$level = $chunk;
$chunk = "-";
@@ -239,6 +254,9 @@
}
last switch; };
/FAIL taper/ && do { next; };
+/DONE taper/ && do {
+ $filenumber -= 1;
+ last switch; };
}
$filenumber += 1;
}
Index: server-src/reporter.c
===
--- server-src/reporter.c (revision 14148)
+++ server-src/reporter.c (working copy)
@@ -826,20 +826,24 @@
double q = (b); \
if (!isnormal(q)) \
g_fprintf((fp)," -- "); \
- else if ((q = (a)/q) >= 999.95) \
- g_fprintf((fp), "###.#"); \
+ else if ((q = (a)/q) >= 9.95) \
+ g_fprintf((fp), "#"); \
+ else if (q >= 999.95) \
+ g_fprintf((fp), "%5.0lf",q);\
else\
- g_fprintf((fp), "%5.1lf",q); \
+ g_fprintf((fp), "%5.1lf",q);\
} while(0)
#define divzero_wide(fp,a,b) \
do { \
double q = (b); \
if (!isnormal(q)) \
g_fprintf((fp),"-- "); \
- else if ((q = (a)/q) >= 9.95) \
- g_fprintf((fp), "#.#"); \
+ else if ((q = (a)/q) >= 999.95) \
+ g_fprintf((fp), "###"); \
+ else if (q >= 9.95) \
+
Re: amanda-2.6.0p2: a few remarks and questions
On Mon, 2008-11-10 at 17:53 -0500, Jean-Francois Malouin wrote: > * Jean-Francois Malouin <[EMAIL PROTECTED]> [20081110 12:27]: > > Hello, > > > > I've finally got to install amanda-2.6.0p2 on a system going soon into > > production and while getting used to the new features and I noticed a > > few things that have changed since 2.5.x : > > Since no-one is answering, I'll continue my lonely tirade :) > > What happened to the `CONFIGURE_COMMAND' line in the amandad debug files? > > This is very useful when testing different config setups. For > instance, it took me a while to discover that I had made a typo while > running configure with `--with-maxtapeblocksize'. I would had spotted > this right away with that line in the amandad debug file. After a lot > of greping through the log files I discovered, buried in the taper > debug files, the only place afaik where one can find the tapetype > blocksize numerical value in lines like: > > : taper: Building type 3 (FILE) header of size 2097152 using: > > cheers, > jf > > > > > - I use amtoc after a dump ends to get stats on what was written on > > tape and noticed that it doesn't parse correctly the log files and the > > toc files end up not showing what was written on tape. Looking at the > > source I believe that this is due to the fact that amtoc doesn't parse > > lines that start with 'PART' as the new log format seems to use > > compared to 'CHUNK' in pre-2.6.x. > > > > - amstatus doesn't show which tape(s) is being used and how much data > > been written to it so far, a useful feature when I want to check that > > a flush is being done correctly when using a changer with 'runtapes>1' Well, if it makes you feel any better, I'll confess that I have also missed amstatus showing which tape is being used. I appreciated the confirmation that the active dump *was* using a tape *and* the tape I expected to be used. I've saved time by killing early on and restarting many an amdump run because, after due time, the tape I expected didn't show up with an amstatus. I shifted to running grep - /etc/amanda//amdump' instead to verify use of a tape. Paul > > > > - As I posted a few days ago, amreport doesn't report correctly the > > aggragated transfer rate to tape beyond 99.99MiBs... > > > > - The new version features 'device_output_buffer_size' that replaces > > 'tapebufs'. Any hint on a ballpark figure for this? Right now I have > > set it to 8192k (I use a blocksize=2048k) but I really don't know if I > > overshooting with this...the server has 32GB of memory btw. > > > > Thank you for the amazing work you guys are doing! > > jf > > -- > > <° >< Jean-François Malouin McConnell Brain Imaging Centre > > System/Network AdministratorMontréal Neurological Institute > > 3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada > -- Design Engineer Zmanda, Inc.
Re: amanda-2.6.0p2: a few remarks and questions
* Jean-Francois Malouin <[EMAIL PROTECTED]> [20081110 12:27]: > Hello, > > I've finally got to install amanda-2.6.0p2 on a system going soon into > production and while getting used to the new features and I noticed a > few things that have changed since 2.5.x : Since no-one is answering, I'll continue my lonely tirade :) What happened to the `CONFIGURE_COMMAND' line in the amandad debug files? This is very useful when testing different config setups. For instance, it took me a while to discover that I had made a typo while running configure with `--with-maxtapeblocksize'. I would had spotted this right away with that line in the amandad debug file. After a lot of greping through the log files I discovered, buried in the taper debug files, the only place afaik where one can find the tapetype blocksize numerical value in lines like: : taper: Building type 3 (FILE) header of size 2097152 using: cheers, jf > > - I use amtoc after a dump ends to get stats on what was written on > tape and noticed that it doesn't parse correctly the log files and the > toc files end up not showing what was written on tape. Looking at the > source I believe that this is due to the fact that amtoc doesn't parse > lines that start with 'PART' as the new log format seems to use > compared to 'CHUNK' in pre-2.6.x. > > - amstatus doesn't show which tape(s) is being used and how much data > been written to it so far, a useful feature when I want to check that > a flush is being done correctly when using a changer with 'runtapes>1' > > - As I posted a few days ago, amreport doesn't report correctly the > aggragated transfer rate to tape beyond 99.99MiBs... > > - The new version features 'device_output_buffer_size' that replaces > 'tapebufs'. Any hint on a ballpark figure for this? Right now I have > set it to 8192k (I use a blocksize=2048k) but I really don't know if I > overshooting with this...the server has 32GB of memory btw. > > Thank you for the amazing work you guys are doing! > jf > -- > <° >< Jean-François Malouin McConnell Brain Imaging Centre > System/Network AdministratorMontréal Neurological Institute > 3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada -- <° >< Jean-François Malouin McConnell Brain Imaging Centre System/Network AdministratorMontréal Neurological Institute 3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada
amanda-2.6.0p2: a few remarks and questions
Hello, I've finally got to install amanda-2.6.0p2 on a system going soon into production and while getting used to the new features and I noticed a few things that have changed since 2.5.x : - I use amtoc after a dump ends to get stats on what was written on tape and noticed that it doesn't parse correctly the log files and the toc files end up not showing what was written on tape. Looking at the source I believe that this is due to the fact that amtoc doesn't parse lines that start with 'PART' as the new log format seems to use compared to 'CHUNK' in pre-2.6.x. - amstatus doesn't show which tape(s) is being used and how much data been written to it so far, a useful feature when I want to check that a flush is being done correctly when using a changer with 'runtapes>1' - As I posted a few days ago, amreport doesn't report correctly the aggragated transfer rate to tape beyond 99.99MiBs... - The new version features 'device_output_buffer_size' that replaces 'tapebufs'. Any hint on a ballpark figure for this? Right now I have set it to 8192k (I use a blocksize=2048k) but I really don't know if I overshooting with this...the server has 32GB of memory btw. Thank you for the amazing work you guys are doing! jf -- <° >< Jean-François Malouin McConnell Brain Imaging Centre System/Network AdministratorMontréal Neurological Institute 3801 Rue University, Suite WB219, Montréal, Québec, H3A 2B4, Canada
