amreport reports wrong results

[René Kanters]

Hi,

I just tried the amreport command and it looks like it is not  
providing the proper result.
The email it generated was that the results were missing for every  
client and in the summary the line I kept is was repeated for every  
client too.
I checked and indeed there is no log file in the location it is  
looking for. There are files of the format log.20061206004500.0, so  
could this have something to do with the fact that I am using  
'usetimestamps' in amanda.conf?


*** THE DUMPS DID NOT FINISH PROPERLY!

The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: DAILY-1.

FAILURE AND STRANGE DUMP SUMMARY:
  werner.richmond.edu   /home/reneRESULTS MISSING
[snip]
  hartree.richmond.edu  /home/amanda  RESULTS MISSING
  amreport: ERROR could not open log /usr/local/etc/amanda/daily/ 
log: No such file or directory



STATISTICS:
  Total   Full  Incr.
      
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min)0:00   0:00   0:00
Output Size (meg)   0.00.00.0
Original Size (meg) 0.00.00.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped0  0  0
Avg Dump Rate (k/s) -- -- --

Tape Time (hrs:min)0:00   0:00   0:00
Tape Size (meg) 0.00.00.0
Tape Used (%)   0.00.00.0
Filesystems Taped 0  0  0

Chunks Taped  0  0  0
Avg Tp Write Rate (k/s) -- -- --

DUMP SUMMARY:
   DUMPER STATS
TAPER STATS
HOSTNAME DISKL ORIG-kB  OUT-kB  COMP%  MMM:SS   KB/s  
MMM:SS   KB/s
-- -  
-
albert.richm -ome/amanda   MISSING  
---

[snip]
werner.richm /home/reneMISSING  
---


(brought to you by Amanda version 2.5.1p1)

I assume that the dump itself worked properly since amdump sent me  
the email (with similar snipping):

These dumps were to tape DAILY-14.
The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: DAILY-1.


STATISTICS:
  Total   Full  Incr.
      
Estimate Time (hrs:min)0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min)0:00   0:00   0:00
Output Size (meg)  33.3   33.30.0
Original Size (meg)   110.9  110.90.0
Avg Compressed Size (%)30.0   30.0--
Filesystems Dumped   11 11  0
Avg Dump Rate (k/s)  2366.8 2366.8--

Tape Time (hrs:min)0:00   0:00   0:00
Tape Size (meg)33.3   33.30.0
Tape Used (%)  33.3   33.30.0
Filesystems Taped11 11  0

Chunks Taped  0  0  0
Avg Tp Write Rate (k/s)  1738.2 1738.2--

USAGE BY TAPE:
  Label  Time  Size  %NbNc
  DAILY-14   0:0034080k   33.311 0

NOTES:
  planner: Adding new disk werner.richmond.edu:/home/rene.
[snip]
  taper: tape DAILY-14 kb 34080 fm 11 [OK]

DUMP SUMMARY:
   DUMPER STATS
TAPER STATS
HOSTNAME DISKL ORIG-kB  OUT-kB  COMP%  MMM:SS   KB/s  
MMM:SS   KB/s
-- -  
-
albert.richm -ome/amanda 0   107702624   24.40:00 5554.3
0:00 5498.7

[snip]
werner.richm /home/rene  019701920   97.50:00 9078.8
0:05  367.7


(brought to you by Amanda version 2.5.1p1)

Is this a bug in amreport, or does it have to do with some new  
features (e.g. usetimestamp) whose ramifications is not implemented  
throughout the suite of programs?


Thanks,
René

Re: How to force amanda to serialize dumper/taper

[Evan Harris]

On Thu, 7 Dec 2006, Brian Cuttler wrote:


I've assumed that the spindle the amanda work area is on was dedicated
to the work area and not split, I know nothing about the architecture


That is correct.  That disk is only used by amanda.


of the specific system bus. If you are able to get through put with
amdump than I'd guess its not a bus issue to either the tape nor the
drive. That leaves perhaps CPU or sharing the bus access to the disk
drive ?


It is almost certainly an issue (as I think I stated previously) with the 
dumper writes and drive seeks between reading from and writing to the same 
spindle degrading the holding disk throughput enough to cause the disk to be 
unable to have sufficient bandwidth to keep the tape drive streaming.



But I'm going to have to step back at this point, since it sounds like
a problem specific to a system that I'm not readily able to visualize.


Here's an attempt to explain the issue better:

Let's say my holding disk has a throughput of about 20MB/sec.  I think 
everyone can agree that with 20MB/sec of throuhput from the drive and 
11MB/sec to the tape, there should be no problem keeping the tape drive 
streaming, assuming there isn't a CPU or bus bottleneck, and that there is 
no additional disk contention for the same spindle.  We've got 9MB/sec of 
left-over bandwidth, which is plenty.


Now, the tech specs on drive used for my holding disk say that the average 
seek time is 9ms.  That means for every seek done (on average), I lose 184KB 
of bandwidth.


Ok, so my disk has a total of 20MB/sec bandwidth.  For the purposes of this 
discussion, lets assume that the read and write bandwidths are equal.  Lets 
also say that the dumper (for the purposes of this discussion, we have 
limited amanda to only allow one dumper at a time, not several running 
concurrently, in order to avoid additional complications) is consuming about 
6MB/sec on average (reasonable assumption for dumps coming from a remote 
machine on a 100mbit network backbone).  That leaves 14MB/sec left.  To keep 
the tape drive streaming (from your own specs for uncompressed writes) 
requres 11MB/sec.  That leaves us with 3MB/sec.


Now say that interleaving the reads and writes to the disk is causing 25 
seeks per second.  25 seeks * 184KB/sec is 4.6MB/sec of throughput lost to 
seek overhead, and you can see that we are now have a deficit of 1.6MB/sec. 
From this example, you should be able to see that it is impossible to keep 
the tape drive streaming unless either the number of seeks or the amount of 
data being written by the dumper (or both) is reduced.


Does that make the issue clear?  The only way to reliably fix the shoe-shine 
problem (absent replacement of hardware) is to keep the dumper from using 
the disk while taper is using it, or being able to throttle the dumper's use 
of the disk drive to leave enough bandwidth to keep the taper from starving. 
And it is appears that neither method of limiting is currently supported by 
Amanda, which seems very surprising to me.


Evan


Evan

On Thu, 7 Dec 2006, Brian Cuttler wrote:


On Wed, Dec 06, 2006 at 04:33:16PM -0600, Evan Harris wrote:


Debian testing, SDLT 110/220, Intel P4 2.8Ghz 3gig RAM.  Amanda 2.5.1p1-2.

The holding disk is a dedicated PATA IDE drive (master) on its own
cable/bus (no slave).  The tape drive is on its own SCSI bus (no other
devices). Bonnie tests on the IDE drive give roughly 20MB/sec, and I have
no trouble keeping the tape drive streaming using dd from the IDE drive to
the tape drive.  Amanda tapetest speed on the tapedrive came in right at
10MB/sec.


I have an SDLT 220 also, mine being set for high density but without
HW compression, I specifically perform SW compression (client side though
in this case the client==server). I am also peaking about 10MB/sec per
the amdump report.

I have to look at the tape specs... Sun online docs show a sustained
transfer rate of 11 MB/Sec, so you and I are both doing pretty well
in that dept.

Tell me again why you feel your drive is shoe-shining ?
Are the specs for your particular drive substantially different ?



I'm currently testing on a standalone SDLT drive first.  But the final
config will be the same type of drive in an ADIC changer.

Evan

On Wed, 6 Dec 2006, Brian Cuttler wrote:


Evan,

What OS, type of tape, type of drive, HW platform ?

I'm unfamiliar with a parameter to do what your asking
for for good measure, what version of Amanda ?

Actually, how did you determine that a drive within a changer
was shoe-shining ? What else shares the bus with the tape and
with the amanda work area drive(s) ?

On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:


I'm having a problem with my tape drive shoe-shining because the holding
disk can't keep up with the tape drive if it is also being written to
by a
dumper.  Without the extra disk seek overhead of dumpers writing to the
holding disk at the same time, the holding disk should be plenty fast
enough to keep th

Re: 2.5.1p1 - Error connecting to windows clients

[Matt Hyclak]

On Thu, Dec 07, 2006 at 02:20:40PM -0700, Ryan Castleberry enlightened us:
> I am running Debian testing. Upon initial upgrade to 2.5.1p1, amanda 
> would not get ACKs from any machines. I ended up downloading a new 
> amanda 2.5.1p1 debian package and reinstalling. I now have no problems 
> getting to Linux machines, but am unable to reach windows machines via 
> samba.
> 
> As user backup, I am able to /usr/bin/smbclient //winpc/share.
> 
> Any help would be appreciated. Thanks!
> 
> 
> Information that may be pertinent follows:
> 
> $ amcheck -c daily
> Amanda Backup Client Hosts Check
> 
> WARNING: winpc.mydomain.com: selfcheck request failed: timeout waiting 
> for ACK
> Client check: 1 host checked in 30.013 seconds, 1 problem found
> 
> (brought to you by Amanda 2.5.1p1)
> 
< snip <--< snip <--< snip <
> 
> ---
> amandapass:
> ---
> //winpc/sharebackup%password
> 
> -
> disklist:
> -
> winpc.mydomain.com//winpc/sharesamba
>

Shouldn't that be someunixboxwithsamba.mydomain.org?

Matt

-- 
Matt Hyclak
Department of Mathematics 
Department of Social Work
Ohio University
(740) 593-1263

Re: How to force amanda to serialize dumper/taper

[Brian Cuttler]

Evan,

On Thu, Dec 07, 2006 at 03:39:05PM -0600, Evan Harris wrote:
> 
> I am NOT peaking at 10MB/sec from the amdump report, the 10MB/sec figure is 
> from several amtapetest runs, as well as my own manual testing with dd.
> 
> I KNOW the tape is shoe-shining, both because I can HEAR it consistently 
> having to reposition the tape every few seconds, and because I'm only 
> getting 2-3MB/sec average throughput when writing to tape while a dumper is 
> running.  But I get close to the 10MB/sec when using amflush after all the 
> dumpers have completed (under a stripped down testing config where all the 
> data fits on and is dumped to the holding disk before the tape is brought 
> online).

Oh, that's different. Sorry, I may have missed something in an
earlier email but by the time I started to reply to you this info
was not clear in the thread. Yes, I'd say that was pretty much the
standard diagnostic for shoe-shining.

I've assumed that the spindle the amanda work area is on was dedicated
to the work area and not split, I know nothing about the architecture
of the specific system bus. If you are able to get through put with
amdump than I'd guess its not a bus issue to either the tape nor the
drive. That leaves perhaps CPU or sharing the bus access to the disk
drive ?

But I'm going to have to step back at this point, since it sounds like
a problem specific to a system that I'm not readily able to visualize.
Nor for that matter did I envision CPU or bus activity being sufficiently
intensive that it was blocking the disk to tape IO and I don't know what
tool to suggest monitoring with.



> Evan
> 
> On Thu, 7 Dec 2006, Brian Cuttler wrote:
> 
> >On Wed, Dec 06, 2006 at 04:33:16PM -0600, Evan Harris wrote:
> >>
> >>Debian testing, SDLT 110/220, Intel P4 2.8Ghz 3gig RAM.  Amanda 2.5.1p1-2.
> >>
> >>The holding disk is a dedicated PATA IDE drive (master) on its own
> >>cable/bus (no slave).  The tape drive is on its own SCSI bus (no other
> >>devices). Bonnie tests on the IDE drive give roughly 20MB/sec, and I have
> >>no trouble keeping the tape drive streaming using dd from the IDE drive to
> >>the tape drive.  Amanda tapetest speed on the tapedrive came in right at
> >>10MB/sec.
> >
> >I have an SDLT 220 also, mine being set for high density but without
> >HW compression, I specifically perform SW compression (client side though
> >in this case the client==server). I am also peaking about 10MB/sec per
> >the amdump report.
> >
> >I have to look at the tape specs... Sun online docs show a sustained
> >transfer rate of 11 MB/Sec, so you and I are both doing pretty well
> >in that dept.
> >
> >Tell me again why you feel your drive is shoe-shining ?
> >Are the specs for your particular drive substantially different ?
> >
> >
> >>I'm currently testing on a standalone SDLT drive first.  But the final
> >>config will be the same type of drive in an ADIC changer.
> >>
> >>Evan
> >>
> >>On Wed, 6 Dec 2006, Brian Cuttler wrote:
> >>
> >>>Evan,
> >>>
> >>>What OS, type of tape, type of drive, HW platform ?
> >>>
> >>>I'm unfamiliar with a parameter to do what your asking
> >>>for for good measure, what version of Amanda ?
> >>>
> >>>Actually, how did you determine that a drive within a changer
> >>>was shoe-shining ? What else shares the bus with the tape and
> >>>with the amanda work area drive(s) ?
> >>>
> >>>On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:
> 
> I'm having a problem with my tape drive shoe-shining because the holding
> disk can't keep up with the tape drive if it is also being written to 
> by a
> dumper.  Without the extra disk seek overhead of dumpers writing to the
> holding disk at the same time, the holding disk should be plenty fast
> enough to keep the tape drive streaming.
> 
> Is there any way I can force amanda to serialize the dumper/taper so 
> that
> they are never run concurrently?  I've already set inparallel to 1, but
> that only affects how many dumpers can run, not the taper.
> 
> I've also tried increasing the tapebufs parameter to 8000 (256MiB) to 
> see
> if that would at least let the drive stay streaming for longer periods,
> but
> if it made any difference, it wasn't significant.  What thresholds does
> the
> taper use to decide when the tapebufs are filled enough to start tape
> motion?  There doesn't seem to be any docs on that, or settings to
> customize.
> 
> I did get a suggestion that I should just leave the tape out of the 
> drive,
> let the dumpers fill the holding disk and then load the tape and run
> amflush, but that doesn't really work when using a changer, plus the
> holding disk isn't large enough for the total size of all the backups,
> though it can fit them one-by-one.
> 
> Seems like there should be a "speed" config option for holding disks 
> like
> there is for network interfaces and tape drives, so that amanda

2.5.1p1 - Error connecting to windows clients

[Ryan Castleberry]

Hello!

I am running Debian testing. Upon initial upgrade to 2.5.1p1, amanda 
would not get ACKs from any machines. I ended up downloading a new 
amanda 2.5.1p1 debian package and reinstalling. I now have no problems 
getting to Linux machines, but am unable to reach windows machines via 
samba.


As user backup, I am able to /usr/bin/smbclient //winpc/share.

Any help would be appreciated. Thanks!


Information that may be pertinent follows:

$ amcheck -c daily
Amanda Backup Client Hosts Check

WARNING: winpc.mydomain.com: selfcheck request failed: timeout waiting 
for ACK

Client check: 1 host checked in 30.013 seconds, 1 problem found

(brought to you by Amanda 2.5.1p1)

--
amcheck.debug:
--
amcheck: debug 1 pid 18122 ruid 34 euid 0: start at Thu Dec  7 14:02:18 2006
amcheck: debug 1 pid 18122 ruid 34 euid 34: rename at Thu Dec  7 
14:02:18 2006

security_getdriver(name=BSD) returns 0xa7f110e0
security_handleinit(handle=0x8057380, driver=0xa7f110e0 (BSD))
amcheck-clients: time 0.003: bind_portrange2: Try  port 852: Available   
- Success

amcheck-clients: time 0.004: dgram_bind: socket bound to 0.0.0.0.852
amcheck-clients: dgram_send_addr(addr=0xafcc65e0, dgram=0xa7f12084)
amcheck-clients: time 0.006: (sockaddr_in *)0xafcc65e0 = { 2, 10080, 
192.168.1.3 }

amcheck-clients: dgram_send_addr: 0xa7f12084->socket = 3
amcheck-clients: dgram_send_addr(addr=0xafcc64d0, dgram=0xa7f12084)
amcheck-clients: time 10.010: (sockaddr_in *)0xafcc64d0 = { 2, 10080, 
192.168.1.3 }

amcheck-clients: dgram_send_addr: 0xa7f12084->socket = 3
amcheck-clients: dgram_send_addr(addr=0xafcc64d0, dgram=0xa7f12084)
amcheck-clients: time 20.014: (sockaddr_in *)0xafcc64d0 = { 2, 10080, 
192.168.1.3 }

amcheck-clients: dgram_send_addr: 0xa7f12084->socket = 3
security_seterror(handle=0x8057380, driver=0xa7f110e0 (BSD) 
error=timeout waiting for ACK)

security_close(handle=0x8057380, driver=0xa7f110e0 (BSD))
amcheck: pid 18122 finish time Thu Dec  7 14:02:48 2006

--
amandad.debug:
--
amandad: debug 1 pid 14843 ruid 34 euid 34: start at Thu Dec  7 03:07:18 
2006

security_getdriver(name=BSD) returns 0xa7fb20e0
amandad: version 2.5.1p1
amandad: build: VERSION="Amanda-2.5.1p1"
amandad:BUILT_DATE="Sun Oct 29 11:43:25 MST 2006"
amandad:BUILT_MACH="Linux rover 2.6.18-1-686 #1 SMP Fri Sep 29 
16:25:40 UTC 2006 i686 GNU/Linux"

amandad:CC="gcc" CONFIGURE_COMMAND="'./configure' 'linux' 'gnu'"
amandad: paths: bindir="/usr/sbin" sbindir="/usr/sbin"
amandad:libexecdir="/usr/lib/amanda" mandir="/usr/share/man"
amandad:AMANDA_TMPDIR="/tmp/amanda"
amandad:AMANDA_DBGDIR="/var/log/amanda" CONFIG_DIR="/etc/amanda"
amandad:DEV_PREFIX="/dev/" RDEV_PREFIX="/dev/" DUMP="/sbin/dump"
amandad:RESTORE="/sbin/restore" VDUMP=UNDEF VRESTORE=UNDEF
amandad:XFSDUMP="/sbin/xfsdump" XFSRESTORE="/sbin/xfsrestore"
amandad:VXDUMP=UNDEF VXRESTORE=UNDEF
amandad:SAMBA_CLIENT="/usr/bin/smbclient" GNUTAR="/bin/tar"
amandad:COMPRESS_PATH="/bin/gzip" UNCOMPRESS_PATH="/bin/gzip"
amandad:LPRCMD="/usr/bin/lpr" MAILER="/usr/bin/Mail"
amandad:listed_incr_dir="/var/lib/amanda/gnutar-lists"
amandad: defs:  DEFAULT_SERVER="localhost" DEFAULT_CONFIG="DailySet1"
amandad:DEFAULT_TAPE_SERVER="localhost" HAVE_MMAP HAVE_SYSVSHM
amandad:LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
amandad:AMANDA_DEBUG_DAYS=4 BSD_SECURITY RSH_SECURITY 
USE_AMANDAHOSTS

amandad:CLIENT_LOGIN="backup" FORCE_USERID HAVE_GZIP
amandad:COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
amandad:COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"


---
amandapass:
---
//winpc/sharebackup%password

-
disklist:
-
winpc.mydomain.com//winpc/sharesamba


amanda.conf:

#
# Amanda configuration file for daily backups.
#

org "daily" # your organization name for reports
mailto "[EMAIL PROTECTED]"
dumpuser "backup"

dumpcycle 7 days# the number of days in the normal dump cycle
runspercycle 5  # the number of amdump runs in dumpcycle days
   # (7 days * 1 amdump per day -- just weekdays)
tapecycle 10 tapes  # the number of tapes in rotation
   # 7 days (dumpcycle) plus a few to handle errors 
that

   # need amflush and so we do not overwrite the full
   # backups performed at the beginning of the previous
   # cycle

bumpsize 20 Mb  # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1  # minimum days at each level
bumpmult 4  # threshold = bumpsize * bumpmult^(level-1)

ctimeout 15

tpchanger "chg-disk"
changerfile "/etc/amanda/chg-disk-status"
tapedev "file:/backups"
rawtapedev "/dev/null"  # the raw device to be used (Linux ftape only)
tapetype HARD-DISK

labels

Re: How to force amanda to serialize dumper/taper

[Evan Harris]

I am NOT peaking at 10MB/sec from the amdump report, the 10MB/sec figure is 
from several amtapetest runs, as well as my own manual testing with dd.


I KNOW the tape is shoe-shining, both because I can HEAR it consistently 
having to reposition the tape every few seconds, and because I'm only 
getting 2-3MB/sec average throughput when writing to tape while a dumper is 
running.  But I get close to the 10MB/sec when using amflush after all the 
dumpers have completed (under a stripped down testing config where all the 
data fits on and is dumped to the holding disk before the tape is brought 
online).


Evan

On Thu, 7 Dec 2006, Brian Cuttler wrote:


On Wed, Dec 06, 2006 at 04:33:16PM -0600, Evan Harris wrote:


Debian testing, SDLT 110/220, Intel P4 2.8Ghz 3gig RAM.  Amanda 2.5.1p1-2.

The holding disk is a dedicated PATA IDE drive (master) on its own
cable/bus (no slave).  The tape drive is on its own SCSI bus (no other
devices). Bonnie tests on the IDE drive give roughly 20MB/sec, and I have
no trouble keeping the tape drive streaming using dd from the IDE drive to
the tape drive.  Amanda tapetest speed on the tapedrive came in right at
10MB/sec.


I have an SDLT 220 also, mine being set for high density but without
HW compression, I specifically perform SW compression (client side though
in this case the client==server). I am also peaking about 10MB/sec per
the amdump report.

I have to look at the tape specs... Sun online docs show a sustained
transfer rate of 11 MB/Sec, so you and I are both doing pretty well
in that dept.

Tell me again why you feel your drive is shoe-shining ?
Are the specs for your particular drive substantially different ?



I'm currently testing on a standalone SDLT drive first.  But the final
config will be the same type of drive in an ADIC changer.

Evan

On Wed, 6 Dec 2006, Brian Cuttler wrote:


Evan,

What OS, type of tape, type of drive, HW platform ?

I'm unfamiliar with a parameter to do what your asking
for for good measure, what version of Amanda ?

Actually, how did you determine that a drive within a changer
was shoe-shining ? What else shares the bus with the tape and
with the amanda work area drive(s) ?

On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:


I'm having a problem with my tape drive shoe-shining because the holding
disk can't keep up with the tape drive if it is also being written to by a
dumper.  Without the extra disk seek overhead of dumpers writing to the
holding disk at the same time, the holding disk should be plenty fast
enough to keep the tape drive streaming.

Is there any way I can force amanda to serialize the dumper/taper so that
they are never run concurrently?  I've already set inparallel to 1, but
that only affects how many dumpers can run, not the taper.

I've also tried increasing the tapebufs parameter to 8000 (256MiB) to see
if that would at least let the drive stay streaming for longer periods,
but
if it made any difference, it wasn't significant.  What thresholds does
the
taper use to decide when the tapebufs are filled enough to start tape
motion?  There doesn't seem to be any docs on that, or settings to
customize.

I did get a suggestion that I should just leave the tape out of the drive,
let the dumpers fill the holding disk and then load the tape and run
amflush, but that doesn't really work when using a changer, plus the
holding disk isn't large enough for the total size of all the backups,
though it can fit them one-by-one.

Seems like there should be a "speed" config option for holding disks like
there is for network interfaces and tape drives, so that amanda could test
to see if the holding disk can't handle dumpers using the holding disk at
the same time a taper is running.  That seems like it'd solve the problem
nicely, and even seems to fit with the scheme amanda uses for network
interfaces.

Thanks.

Evan

---
 Brian R Cuttler [EMAIL PROTECTED]
 Computer Systems Support(v) 518 486-1697
 Wadsworth Center(f) 518 473-6384
 NYS Department of HealthHelp Desk 518 473-0773


---
  Brian R Cuttler [EMAIL PROTECTED]
  Computer Systems Support(v) 518 486-1697
  Wadsworth Center(f) 518 473-6384
  NYS Department of HealthHelp Desk 518 473-0773

"archive" level 0 backups

[Don Murray]

Hello list - thanks for your help in the past - I now have my amanda 
configuration running stably on a subset of my partitions with my 8-tape 
changer for a few weeks without trouble.  I've tested file recovery with 
amrecover and all is good.


I only have one last hurdle to overcome before I can declare complete 
success.  One of the missions of our backups is to provide an archival 
backup for offsite storage at least once per month.  I was hoping to get 
some advice on this issue.


From the zmanda wiki I have gleaned that the typical way to do this is 
create an additional amanda configuration called "archive" that runs 
once per month (or whatever) and forces a level 0 on all things you want 
to archive.


This is fine, except that a full level 0 on all systems is going to 
cause some strain on the infrastructure (network and disk activity for 
example).  I was wondering if it is possible instead to cache the latest 
level0's from the daily backups and have them written to tape.  My 
holding disk is big enough to hold a copy of all level 0's, and it is 
not necessary for my once-a-month backup to have level 0's all from the 
same day.  Since I am already paying the price to get a full set of 
level 0's once per week with my daily tape cycle, is it possible to 
leverage this into a easier monthly backup?


I hope this makes some sense to people and would really appreciate any 
advice on this issue.


Thanks in advance,
Don

Re: How to force amanda to serialize dumper/taper

[Brian Cuttler]

On Wed, Dec 06, 2006 at 04:33:16PM -0600, Evan Harris wrote:
> 
> Debian testing, SDLT 110/220, Intel P4 2.8Ghz 3gig RAM.  Amanda 2.5.1p1-2.
> 
> The holding disk is a dedicated PATA IDE drive (master) on its own 
> cable/bus (no slave).  The tape drive is on its own SCSI bus (no other 
> devices). Bonnie tests on the IDE drive give roughly 20MB/sec, and I have 
> no trouble keeping the tape drive streaming using dd from the IDE drive to 
> the tape drive.  Amanda tapetest speed on the tapedrive came in right at 
> 10MB/sec.

I have an SDLT 220 also, mine being set for high density but without
HW compression, I specifically perform SW compression (client side though
in this case the client==server). I am also peaking about 10MB/sec per
the amdump report.

I have to look at the tape specs... Sun online docs show a sustained
transfer rate of 11 MB/Sec, so you and I are both doing pretty well
in that dept.

Tell me again why you feel your drive is shoe-shining ?
Are the specs for your particular drive substantially different ?


> I'm currently testing on a standalone SDLT drive first.  But the final 
> config will be the same type of drive in an ADIC changer.
> 
> Evan
> 
> On Wed, 6 Dec 2006, Brian Cuttler wrote:
> 
> >Evan,
> >
> >What OS, type of tape, type of drive, HW platform ?
> >
> >I'm unfamiliar with a parameter to do what your asking
> >for for good measure, what version of Amanda ?
> >
> >Actually, how did you determine that a drive within a changer
> >was shoe-shining ? What else shares the bus with the tape and
> >with the amanda work area drive(s) ?
> >
> >On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:
> >>
> >>I'm having a problem with my tape drive shoe-shining because the holding
> >>disk can't keep up with the tape drive if it is also being written to by a
> >>dumper.  Without the extra disk seek overhead of dumpers writing to the
> >>holding disk at the same time, the holding disk should be plenty fast
> >>enough to keep the tape drive streaming.
> >>
> >>Is there any way I can force amanda to serialize the dumper/taper so that
> >>they are never run concurrently?  I've already set inparallel to 1, but
> >>that only affects how many dumpers can run, not the taper.
> >>
> >>I've also tried increasing the tapebufs parameter to 8000 (256MiB) to see
> >>if that would at least let the drive stay streaming for longer periods, 
> >>but
> >>if it made any difference, it wasn't significant.  What thresholds does 
> >>the
> >>taper use to decide when the tapebufs are filled enough to start tape
> >>motion?  There doesn't seem to be any docs on that, or settings to
> >>customize.
> >>
> >>I did get a suggestion that I should just leave the tape out of the drive,
> >>let the dumpers fill the holding disk and then load the tape and run
> >>amflush, but that doesn't really work when using a changer, plus the
> >>holding disk isn't large enough for the total size of all the backups,
> >>though it can fit them one-by-one.
> >>
> >>Seems like there should be a "speed" config option for holding disks like
> >>there is for network interfaces and tape drives, so that amanda could test
> >>to see if the holding disk can't handle dumpers using the holding disk at
> >>the same time a taper is running.  That seems like it'd solve the problem
> >>nicely, and even seems to fit with the scheme amanda uses for network
> >>interfaces.
> >>
> >>Thanks.
> >>
> >>Evan
> >---
> >  Brian R Cuttler [EMAIL PROTECTED]
> >  Computer Systems Support(v) 518 486-1697
> >  Wadsworth Center(f) 518 473-6384
> >  NYS Department of HealthHelp Desk 518 473-0773
> >
---
   Brian R Cuttler [EMAIL PROTECTED]
   Computer Systems Support(v) 518 486-1697
   Wadsworth Center(f) 518 473-6384
   NYS Department of HealthHelp Desk 518 473-0773

Re: runtapes < {total size of dump}?

[Alexander Jolk]

Hi,


So, if I have a dump where every item is the same priority, and the
complete dump takes 4 tapes, what happens if I set runtapes to '2'?

Will it fail gracefully, filling 2 tapes with half(-ish) the data one
pass and the other half the next, or will it fail in such a way that I
need to amflush after every run and/or start from the 1st disklisted
partition every time?


As long as your individual dumps are smaller than your tape size (or
total tape size for one run if you use splitting), it will work out
after two runs.  On the first run, you'll get a few MISSING dumps
because your dump size was too big; those will be done on the next
run.  Depending on your holding disk size and reserve parameter, a few
dumps may also end up on the holding disk after the first run; with
autoflush, they'll be flushed on the next run.

Alex


--
Alexander Jolk * +33-1 42 62 31 95 * +33-6 89 65 36 58
[EMAIL PROTECTED]

runtapes < {total size of dump}?

[JB Segal]

So, if I have a dump where every item is the same priority, and the
complete dump takes 4 tapes, what happens if I set runtapes to '2'?

Will it fail gracefully, filling 2 tapes with half(-ish) the data one
pass and the other half the next, or will it fail in such a way that I
need to amflush after every run and/or start from the 1st disklisted
partition every time?

(2.5.1p1, RHEL4u4)

Thanks!
JB, trying to avoid having to maintain 2 disklist files and swapping
them back and forth, or some such.

Thanks!
JB
--
JB Segal617-886-5575www.smartertravel.com
Systems/Network Admin.  465 Medford St. Ste 400 www.bookingbuddy.com
Smarter Living, Inc.Boston, MA 02129www.tripmania.com