Re: SATA hardware raid/multiple disk controllers/pci bus performance (was Re: slow client)
On Tuesday 12 April 2005 14:23, Eric Dantan Rzewnicki wrote: >On Tue, Apr 12, 2005 at 01:50:15PM -0400, Gene Heskett wrote: >> On Tuesday 12 April 2005 10:50, Salada, Duncan S. wrote: >> >Actually, I'm not using a holding disk at all. I had no idea >> > that I could be shortening the life of the tape drive by not >> > using one. So, that's probably the source of my problem then? >> >> The idea behind the holding disk (which really shouldn't be on the >> same controller as the disk drive because of bus contention >> issues) is that if the drive is waiting on data and stops, it will >> not restart until the next file to be written is wholely in the >> holding disk, at which point the copy can then take place at the >> drives natural speed. > >My current amanda server lives on one big 1.4TB partition on an SATA >hardware raid controller. Copying from the holding disk to > file-tapes on the same device has so far been limited to about > 15MB/s. Considering that the data has to go both ways up and down the cable, that may be about the practical limit. > This seems quite slow to me. Does amanda add any overhead > above a simple copy on the same device? An hdparm -t on this system > when it's idle gives around 58MB/s. > >I want to add a second SATA controller, fill up the remaining 4 > drive bays and move / and the holding disk to this separate raid. I > expect this set up will provide better performance. But, I want to > ask if anyone here has experience with a similar settup. Are there > pci bus limitations that would get in the way of the potential > benefits? other issues to consider? The stock pci bus is pretty well tapped out at 132 mhz, pci-x can quadruple that. -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.34% setiathome rank, not too shabby for a WV hillbilly Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Re: SATA hardware raid/multiple disk controllers/pci bus performance (was Re: slow client)
On Tue, Apr 12, 2005 at 08:56:46PM +0200, Geert Uytterhoeven wrote: > On Tue, 12 Apr 2005, Eric Dantan Rzewnicki wrote: > > My current amanda server lives on one big 1.4TB partition on an SATA > > hardware raid controller. Copying from the holding disk to file-tapes on > > the same device has so far been limited to about 15MB/s. This seems > > quite slow to me. Does amanda add any overhead above a simple copy on > > the same device? An hdparm -t on this system when it's idle gives around > > 58MB/s. > Just for comparison... > I don't use a holding disk, but my vtapes are on the same (single) SATA disk > as > the other data on my backup machine. > While hdparm -t claims ca. 55 MB/s for the raw disk, level 0 dumps (actually > tars) from my digital picture collection (ca 1.3 MB/file) to the vtapes are > performed at 10-13 MB/s. Ok. so maybe I don't actually have a problem. -- Eric Dantan Rzewnicki | Systems Administrator Technical Operations Division | Radio Free Asia 2025 M Street, NW | Washington, DC 20036 | 202-530-4900 CONFIDENTIAL COMMUNICATION This e-mail message is intended only for the use of the addressee and may contain information that is privileged and confidential. Any unauthorized dissemination, distribution, or copying is strictly prohibited. If you receive this transmission in error, please contact [EMAIL PROTECTED]
Re: SATA hardware raid/multiple disk controllers/pci bus performance (was Re: slow client)
On Tue, 12 Apr 2005, Eric Dantan Rzewnicki wrote:
> My current amanda server lives on one big 1.4TB partition on an SATA
> hardware raid controller. Copying from the holding disk to file-tapes on
> the same device has so far been limited to about 15MB/s. This seems
> quite slow to me. Does amanda add any overhead above a simple copy on
> the same device? An hdparm -t on this system when it's idle gives around
> 58MB/s.
Just for comparison...
I don't use a holding disk, but my vtapes are on the same (single) SATA disk as
the other data on my backup machine.
While hdparm -t claims ca. 55 MB/s for the raw disk, level 0 dumps (actually
tars) from my digital picture collection (ca 1.3 MB/file) to the vtapes are
performed at 10-13 MB/s.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [EMAIL PROTECTED]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Recommendations on combined backup to HD/DVD
Hey, all! I've got a client who was running backups to tape using amanda for several years and the backup server finally died. After much discussion, the decision was reached to replace it with a new box mounting a pair of 160G hard drives (in a RAID1 mirror) and a pair of external DVD burners. I figure the hard drives will comfortably handle 3 weeks' worth of backups (amanda was previously averaging a little over 7G/night, M-F, so rounding that up to 8G/run * 5 runs/week * 3 weeks = 120G, leaving 40G for formatting info, the OS, etc.), plus they want the backups burned to DVD (presumably nightly, as 7+G will fill most of 2 DVDs) and, on top of that, they want to see how often it's feasible to send copies of the backups to another facility via T-1 (either nightly or weekly). My amanda experience to date has been entirely with standard runtapes=1, changer=manual setups, so I'm not entirely sure of the best way to do configure this. I expect that the backups to HD should be pretty straightforward using the vtape driver and I assume that doing the offsite with a script run from cron independently of amanda should be even easier. The DVDs I'm not so sure about, though. Would it be possible (and advisable...) to get amanda to write the backups to DVD and vtape concurrently, or will I need to set up another script to run after amanda is done, sort the night's backups into two DVD-sized chunks, and burn them? (Or, for that matter, would it be best to set up a weekly archival config for amanda and just let it spill over onto >2 DVDs, since that version is intended for long-term archiving anyhow?)
SATA hardware raid/multiple disk controllers/pci bus performance (was Re: slow client)
On Tue, Apr 12, 2005 at 01:50:15PM -0400, Gene Heskett wrote: > On Tuesday 12 April 2005 10:50, Salada, Duncan S. wrote: > >Actually, I'm not using a holding disk at all. I had no idea that I > > could be shortening the life of the tape drive by not using one. > > So, that's probably the source of my problem then? > The idea behind the holding disk (which really shouldn't be on the > same controller as the disk drive because of bus contention issues) > is that if the drive is waiting on data and stops, it will not > restart until the next file to be written is wholely in the holding > disk, at which point the copy can then take place at the drives > natural speed. My current amanda server lives on one big 1.4TB partition on an SATA hardware raid controller. Copying from the holding disk to file-tapes on the same device has so far been limited to about 15MB/s. This seems quite slow to me. Does amanda add any overhead above a simple copy on the same device? An hdparm -t on this system when it's idle gives around 58MB/s. I want to add a second SATA controller, fill up the remaining 4 drive bays and move / and the holding disk to this separate raid. I expect this set up will provide better performance. But, I want to ask if anyone here has experience with a similar settup. Are there pci bus limitations that would get in the way of the potential benefits? other issues to consider? -- Eric Dantan Rzewnicki | Systems Administrator Technical Operations Division | Radio Free Asia 2025 M Street, NW | Washington, DC 20036 | 202-530-4900 CONFIDENTIAL COMMUNICATION This e-mail message is intended only for the use of the addressee and may contain information that is privileged and confidential. Any unauthorized dissemination, distribution, or copying is strictly prohibited. If you receive this transmission in error, please contact [EMAIL PROTECTED]
RE: slow client
Thanks for the info, Gene. That will be very helpful when telling folks why the disk is needed and when I set it up. Duncan --- Duncan Salada Titan Corporation 301-925-3222 x375 > -Original Message- > From: Gene Heskett [mailto:[EMAIL PROTECTED] > Sent: Tuesday, April 12, 2005 1:50 PM > To: amanda-users@amanda.org > Subject: Re: slow client > > > On Tuesday 12 April 2005 10:50, Salada, Duncan S. wrote: > > >Actually, I'm not using a holding disk at all. I had no idea that I > > could be shortening the life of the tape drive by not using one. > > So, that's probably the source of my problem then? > > I expect it is, Duncan. One really should have a holding disk area > that is capable of holding the 2 or 3 largest disklist entries, > preferably lots more, and thats easy when todays commodity drive is a > 120GB or so drive. I have about a 25GB area, free space on a > partition and this, for the dle's I have constructed, is only used to > maybe the 22GB free at its peak use. But then I've tried to follow > my own advice in that most of my disklist entries (nearly 50 of them) > are all fairly smallish, giving amanda as much help in her balanceing > action as possible. > > The idea behind the holding disk (which really shouldn't be on the > same controller as the disk drive because of bus contention issues) > is that if the drive is waiting on data and stops, it will not > restart until the next file to be written is wholely in the holding > disk, at which point the copy can then take place at the drives > natural speed. > > By carefull choice of the priority string in your > amanda.conf, one can > often hit a point where once the drive starts, it will run non-stop > until the nightly run is done. I try to put the biggest ones on the > front of the tape, and by the time the first one is cached and > written, many of the rest are waiting, so the drive never stops > between files. > > Shoe-shining the drive because its running out of data, making it > stop, backup to the last good block written, & repeat at 2 (or less) > second intervals, is very hard on both the drives mechanics and the > heads, but is also making a one pass write into a many pass write, > with a severe reduction in the life of the tape. > > When you do setup the holding disk, be sure and add a line in your > amanda.conf thats says something like 'reserved 30%', otherwise it > can only be used in the event of a tape problem for incrementals, no > fulls. With it in there, then fulls can proceed as normal, piling up > in the holding area until its down to 30% of its original capacity > remaining. It might give you an extra day or 2 to solve a tape drive > problem in that event. > > [...] > > -- > Cheers Duncan, Gene > "There are four boxes to be used in defense of liberty: > soap, ballot, jury, and ammo. Please use in that order." > -Ed Howdershelt (Author) > 99.34% setiathome rank, not too shabby for a WV hillbilly > Yahoo.com and AOL/TW attorneys please note, additions to the above > message by Gene Heskett are: > Copyright 2005 by Maurice Eugene Heskett, all rights reserved. >
Re: slow client
On Tuesday 12 April 2005 10:50, Salada, Duncan S. wrote: >Actually, I'm not using a holding disk at all. I had no idea that I > could be shortening the life of the tape drive by not using one. > So, that's probably the source of my problem then? I expect it is, Duncan. One really should have a holding disk area that is capable of holding the 2 or 3 largest disklist entries, preferably lots more, and thats easy when todays commodity drive is a 120GB or so drive. I have about a 25GB area, free space on a partition and this, for the dle's I have constructed, is only used to maybe the 22GB free at its peak use. But then I've tried to follow my own advice in that most of my disklist entries (nearly 50 of them) are all fairly smallish, giving amanda as much help in her balanceing action as possible. The idea behind the holding disk (which really shouldn't be on the same controller as the disk drive because of bus contention issues) is that if the drive is waiting on data and stops, it will not restart until the next file to be written is wholely in the holding disk, at which point the copy can then take place at the drives natural speed. By carefull choice of the priority string in your amanda.conf, one can often hit a point where once the drive starts, it will run non-stop until the nightly run is done. I try to put the biggest ones on the front of the tape, and by the time the first one is cached and written, many of the rest are waiting, so the drive never stops between files. Shoe-shining the drive because its running out of data, making it stop, backup to the last good block written, & repeat at 2 (or less) second intervals, is very hard on both the drives mechanics and the heads, but is also making a one pass write into a many pass write, with a severe reduction in the life of the tape. When you do setup the holding disk, be sure and add a line in your amanda.conf thats says something like 'reserved 30%', otherwise it can only be used in the event of a tape problem for incrementals, no fulls. With it in there, then fulls can proceed as normal, piling up in the holding area until its down to 30% of its original capacity remaining. It might give you an extra day or 2 to solve a tape drive problem in that event. [...] -- Cheers Duncan, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.34% setiathome rank, not too shabby for a WV hillbilly Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Re: splitting DLE's
On Tue, Apr 12, 2005 at 09:03:16AM -0400, Jon LaBadie wrote: > On Tue, Apr 12, 2005 at 01:45:15PM +0200, Paul Bijnens wrote: > > I believe you need 2.4.3 or later to specify a "diskname" and > > "diskdevice". In 2.4.2 the syntax of a DLE would allow only a > > "diskname". > > Because the diskname should be unique in a disklist file, the > > workaround in that time was to add mulitple "/." to the end of > > a diskname, like this: > > some.host.org /my/disknamedumptype-with-excludes > > some.host.org /my/diskname/. dumptype-with-other-excludes > > some.host.org /my/diskname/./.dumptype-with-yet-other-excludes > > That is, is you don't want/can't upgrade the client to a recent > > amanda version. > I would not have remembered that on my own, but as soon as you said it I did > :) > An aside, I've always wondered about the selection of column names there, > "diskname" and "diskdevice". Very /sbin/dump and whole-file-system centric. > As an alternative, might "DLEtag" and "DataLocation" be more descriptive? > And the pairing of host and DLEtag be known as the DLEname? > Ahh, nevermind, it is just semantics. just semantics, but now I know what is meant. So thanks everyone for discussing this. -- Eric Dantan Rzewnicki | Systems Administrator Technical Operations Division | Radio Free Asia 2025 M Street, NW | Washington, DC 20036 | 202-530-4900 CONFIDENTIAL COMMUNICATION This e-mail message is intended only for the use of the addressee and may contain information that is privileged and confidential. Any unauthorized dissemination, distribution, or copying is strictly prohibited. If you receive this transmission in error, please contact [EMAIL PROTECTED]
Re: splitting DLE's
On Tue, Apr 12, 2005 at 01:45:15PM +0200, Paul Bijnens wrote: > I can't remember if this was mentioned, but what is the amanda > version of the client giving the error message? > I believe you need 2.4.3 or later to specify a "diskname" and > "diskdevice". In 2.4.2 the syntax of a DLE would allow only a > "diskname". ahha! the client is a debian woody (stable) box with amanda 2.4.2p2-4. That makes perfect sense. > Because the diskname should be unique in a disklist file, the > workaround in that time was to add mulitple "/." to the end of > a diskname, like this: > > some.host.org /my/disknamedumptype-with-excludes > some.host.org /my/diskname/. dumptype-with-other-excludes > some.host.org /my/diskname/./.dumptype-with-yet-other-excludes > > That is, is you don't want/can't upgrade the client to a recent > amanda version. -- Eric Dantan Rzewnicki | Systems Administrator Technical Operations Division | Radio Free Asia 2025 M Street, NW | Washington, DC 20036 | 202-530-4900 CONFIDENTIAL COMMUNICATION This e-mail message is intended only for the use of the addressee and may contain information that is privileged and confidential. Any unauthorized dissemination, distribution, or copying is strictly prohibited. If you receive this transmission in error, please contact [EMAIL PROTECTED]
Re: Append Files using tar
On Tue, Apr 12, 2005 at 04:15:51PM +0200, Paul Bijnens wrote:
> Most modern tapedevices are not random access.
> I don't know how to "rewind a little", and neither does the gnutar
> programmer which states in the beginning of the "update.c" source
> file: "... if [the files] are on raw tape or something like that,
> it will probably lose... */ ".
That was probably me, lo those many years ago.
> I guess that if you "dd" the first file, you find a nice tar file,
> and if you "dd" the next file, which seems to be written to
> tape too, you'll find another file beginning with the last block
> again, now having the trailer replaced with the next archive members.
> Useless actually.
The fact that tar didn't give you an error is clearly a bug. You
should report it. Tar should have noticed that it's attempt to
backspace the archive failed.
It is theoretically possible to make this work on a theoretical tape
drive, if the drive in question supports the MTBSR command in the
MTIOCTOP ioctl. Back when I was working on Gnu Tar, I considered
attempting to use said command to backspace a tape drive when
attempting to append to an archive on a tape. Unfortunatly, if I
wrote such a thing, someone would eventually attempt to use it on
hardware that was incapable of doing what they wanted, and would
become very irate when their real-world tape drive behaved like a
mechanical device and destroyed their data.
Back when I was working on Gnu Tar, I had access to two different
kinds of tape drives: 9-track reel-to-reel tapes, and cartridge
"streaming" tapes. The 9-track drives a capable of backspacing a
single record (aka "tape blocks"), but reading a tape block,
backspacing, and attempting to write over the tape block will cause
the new tape block to be written to a slightly different physical
location on the tape. Eventually (how soon depends on the inaccuracy
of the paticular tape drive) the new tape block will be written on top
of the previous tape block, destroying the data on that section of the
tape.
streaming tape drives are (were) incapable of backspacing a single
record, so there is simply no way to append to a tape file.
Tape drives that use timing tracks for aligning data on the tape may
be capable of backspacing a tape block and rewriting it safely, but I
didn't have any such devices, so they were strictly theoretical.
In any case, tar has no way to tell what kind of tape drive the
archive is on, and relying on the user to know the capabilities of
their tape drive is foolish. So tar doesn't even try.
Here's how to append files to a tape archive:
1: free up a chunk of disk space on the machine with the tape drive
that is at least as large as the capacity of the tape.
2: wind the tape to the start of the archive you wish to append to ("mt
rewind" or "mt fsf N" as appropriate)
3: use dd to copy the tape archive into the free disk space ("dd
bs={size of tape blocks} < /dev/ntape > /free/space/path/tmp.tar")
4: append the files you want to the temporary copy of the tape archive
("tar rvbf {size of tape blocks} /free/space/path/tmp.tar ...")
5: backspace the tape to the start of the archive you want to append
to ("mt bsf 1" or "mt rewind", etc.)
5: Copy the new archive back to the tape with dd
("dd bs={size of tape blocks} < /free/space/path/tmp.tar >
/dev/ntape" )
Note that some tape drives will only allow you to either append to the
tape or start writing at the beginning, in which case you will have to
copy off any tape archives before the one you wish to append to, and
rewrite the tape from the beginning.
-- JF
RE: slow client
Actually, I'm not using a holding disk at all. I had no idea that I could be shortening the life of the tape drive by not using one. So, that's probably the source of my problem then? I'd say the error count is insignificant... # netstat -ni Name Mtu Net/Dest AddressIpkts Ierrs Opkts Oerrs Collis Queue lo0 8232 127.0.0.0 127.0.0.1 1492823 0 1492823 0 0 0 dmfe1 1500 xxx.xxx.xxx.0 xxx.xxx.xxx.xx 275677299 0 402788989 1 0 0 Duncan --- Duncan Salada Titan Corporation 301-925-3222 x375 > -Original Message- > From: Paul Bijnens [mailto:[EMAIL PROTECTED] > Sent: Tuesday, April 12, 2005 10:27 AM > To: Salada, Duncan S.; amanda-users@amanda.org > Subject: Re: slow client > > > Salada, Duncan S. wrote: > > > > The slow client is a Sun V100 running Solaris 8. While it is > > certainly a low-end system, it is also the newest. According to > > Amanda's reports, it's consistently transferring dumps at around 425 > > to 475 KB/s. By contrast, I have another client (a Sun Ultra > > Enterprise I running 2.6, arguably a dinosaur) that > consisently dumps > > a 750 - 800 KB/s. > > > > One thing I've thought of is the partition size. The V100 is using > > the default partitions that the system was shipped with, the biggest > > of which is a 67G partition. The Ultra I has a 16G > partition as it's > > biggest. The traffic on the network isn't bad when I'm doing the > > backups. I'm just not sure what I should be looking at or for? Any > > help or tips would be greatly appreciated. > > > Are you using the holdingdisk for that 67 Gbyte partition or > is the holdingdisk too small? If not, your tapedrive probably stops > streaming, resulting in poor throughput, and a broken tapedrive in > a few weeks/months due to the wear/tear on the mechanics. > To verify, grep the amdump.X output for "PORT-DUMP" (dump to > a tcp-port, where the taper program is listening, instead of > the normal word "FILE-DUMP" when dumping to holdingdisk file). > > Also watch out for ethernet duplex mismatches (although in that case > I wouldn't even expect a speed of 40 KB/s.). Is the errorcount > in "netstat -ni" significant? > > -- > Paul Bijnens, XplanationTel +32 > 16 397.511 > Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 > 16 397.512 > http://www.xplanation.com/ email: > [EMAIL PROTECTED] > ** > * > * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, > ^Q, F6, * > * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, > bye, /bye, * > * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, > hangup, * > * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, > shutdown, * > * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, > ...* > * ... "Are you sure?" ... YES ... Phew ... I'm out > * > ** > * > >
Re: slow client
Salada, Duncan S. wrote: The slow client is a Sun V100 running Solaris 8. While it is certainly a low-end system, it is also the newest. According to Amanda's reports, it's consistently transferring dumps at around 425 to 475 KB/s. By contrast, I have another client (a Sun Ultra Enterprise I running 2.6, arguably a dinosaur) that consisently dumps a 750 - 800 KB/s. One thing I've thought of is the partition size. The V100 is using the default partitions that the system was shipped with, the biggest of which is a 67G partition. The Ultra I has a 16G partition as it's biggest. The traffic on the network isn't bad when I'm doing the backups. I'm just not sure what I should be looking at or for? Any help or tips would be greatly appreciated. Are you using the holdingdisk for that 67 Gbyte partition or is the holdingdisk too small? If not, your tapedrive probably stops streaming, resulting in poor throughput, and a broken tapedrive in a few weeks/months due to the wear/tear on the mechanics. To verify, grep the amdump.X output for "PORT-DUMP" (dump to a tcp-port, where the taper program is listening, instead of the normal word "FILE-DUMP" when dumping to holdingdisk file). Also watch out for ethernet duplex mismatches (although in that case I wouldn't even expect a speed of 40 KB/s.). Is the errorcount in "netstat -ni" significant? -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
GFS Rotation for amanda.conf
Hi has anyone got a GFS(Grand Father Son rotation setup for amanda.conf file. As I going to use this commonly used backup cycle for dumpcycle, runspercycle and tapecycle of 24. 4 daily, 4/5 for weekly and 12 monthly two for error/amflush and 1 for tape cleaning. Cheers Chuck -- Unix/ Linux Systems Administrator The Surgical Material Testing Laboratory (SMTL), Princess of Wales Hospital Coity Road Bridgend, United Kingdom, CF31 1RQ. Tel: +44 1656 752820 Fax: +44 1656 752830
Re: Append Files using tar
Kaushal Shriyan wrote: [...] [EMAIL PROTECTED] tartest]# mt -f /dev/nst0 rewind [EMAIL PROTECTED] tartest]# tar -tvf /dev/nst0 drwxr-xr-x root/root 0 2005-04-01 15:54:53 folderA/ -rw-r--r-- root/root 7 2005-04-01 15:54:48 folderA/t1 -rw-r--r-- root/root10 2005-04-01 15:54:55 folderA/t2 [EMAIL PROTECTED] tartest]# mt -f /dev/nst0 rewind [EMAIL PROTECTED] tartest]# tar -rvf /dev/nst0 folderB folderB/ folderB/t3 folderB/t4 [EMAIL PROTECTED] tartest]# mt -f /dev/nst0 rewind [EMAIL PROTECTED] tartest]# tar -tvf /dev/nst0 drwxr-xr-x root/root 0 2005-04-01 15:54:53 folderA/ -rw-r--r-- root/root 7 2005-04-01 15:54:48 folderA/t1 -rw-r--r-- root/root10 2005-04-01 15:54:55 folderA/t2 [EMAIL PROTECTED] tartest]# As you can see, in the end when i do tar -tvf /dev/nst0, only the contents of folder A are displayed and that of folderB are not displayed. This does not meet our requirement. We want the contents of both the folders. This has actually nothing to do with amanda. What you're trying to do is to use tar to append to an existing archive. This works when the archive is on a disk, but it does not work on most tapedrives. (Actually, I do not know any modern tape drive where it does work! I did use 9-inch reels, but that was on an IBM mainframe running MVS; and I guess it can be done on such devices -- but never tried myself. I do not have access anymore to such tapedrives either.) A tapedevice has no "inode" to indicate the size of a file to know where it ends. Instead it writes a marker after a file to signal the end of file. When reading a tapefile, and the program reads this marker, it returns 0 from the next read(), the common way to indicate EOF in programs. When tar has to update an archive on disk, it can read to the end of the file, keep the last block in memory, overwrite the trailer in that block with the file-announcement and bytes of the next file, and overwrite that block again. When tar has to update a tape, it would have to read to the end of the file (by reading up to and including that marker), *rewind* *a* *little*, and rewrite that updated last block. Most modern tapedevices are not random access. I don't know how to "rewind a little", and neither does the gnutar programmer which states in the beginning of the "update.c" source file: "... if [the files] are on raw tape or something like that, it will probably lose... */ ". I guess that if you "dd" the first file, you find a nice tar file, and if you "dd" the next file, which seems to be written to tape too, you'll find another file beginning with the last block again, now having the trailer replaced with the next archive members. Useless actually. -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
slow client
Hi, I'm currently using Amanda to backup 4 servers (three Solaris boxes and one Fedora Core 3). They are all running Amanda 2.4.4p4. I have a problem with one of the clients being MUCH slower than the others. I was wondering if anyone could give me any pointers about where to begin poking around. The slow client is a Sun V100 running Solaris 8. While it is certainly a low-end system, it is also the newest. According to Amanda's reports, it's consistently transferring dumps at around 425 to 475 KB/s. By contrast, I have another client (a Sun Ultra Enterprise I running 2.6, arguably a dinosaur) that consisently dumps a 750 - 800 KB/s. One thing I've thought of is the partition size. The V100 is using the default partitions that the system was shipped with, the biggest of which is a 67G partition. The Ultra I has a 16G partition as it's biggest. The traffic on the network isn't bad when I'm doing the backups. I'm just not sure what I should be looking at or for? Any help or tips would be greatly appreciated. Duncan --- Duncan Salada Titan Corporation 301-925-3222 x375
Re: splitting DLE's
On Tue, Apr 12, 2005 at 01:45:15PM +0200, Paul Bijnens wrote: > > I believe you need 2.4.3 or later to specify a "diskname" and > "diskdevice". In 2.4.2 the syntax of a DLE would allow only a > "diskname". > > Because the diskname should be unique in a disklist file, the > workaround in that time was to add mulitple "/." to the end of > a diskname, like this: > > some.host.org /my/disknamedumptype-with-excludes > some.host.org /my/diskname/. dumptype-with-other-excludes > some.host.org /my/diskname/./.dumptype-with-yet-other-excludes > > That is, is you don't want/can't upgrade the client to a recent > amanda version. I would not have remembered that on my own, but as soon as you said it I did :) An aside, I've always wondered about the selection of column names there, "diskname" and "diskdevice". Very /sbin/dump and whole-file-system centric. As an alternative, might "DLEtag" and "DataLocation" be more descriptive? And the pairing of host and DLEtag be known as the DLEname? Ahh, nevermind, it is just semantics. -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: Newbie: amlabel: could not load slot "current": changerfile must be specified in amanda.conf
On Tue, Apr 12, 2005 at 01:19:52PM +1000, Jesus Salvo Jr. wrote: > On Tuesday 12 April 2005 13:07, Jesus Salvo Jr. wrote: > > I think I know what the problem maybe, though I do not know the solution: > > > > The test that I did with chg-zd-mtx works only if I run it in > > the /usr/local/etc/amanda/Daily directory. > > > > If I do it anywhere else, I get the exact same error as amtape does: > > > > bash-2.05$ pwd > > /export/home/amanda > > bash-2.05$ /usr/local/libexec/chg-zd-mtx -reset > > changerfile must be specified in amanda.conf > > > > bash-2.05$ set | grep PATH > > LD_LIBRARY_PATH=:/usr/local/lib > > MANPATH=:/usr/local/man > > PATH=/usr/bin::/usr/local/bin:/usr/local/sbin:/usr/local/libexec > > > > Based on changer.c, amtape simply exec / calls chg-zd-mtx, so the error is > > the same. > > > > What must I do so that chg-zd-mtx will work from any directory ? > > I must have missed a very simple step. > > > > More info: > > I changed chg-zd-mtx so that I added an echo line just before it calls > amgetconf: > > echo "amgetconf$SUF changerfile" > changerfile=`amgetconf$SUF changerfile 2>/dev/null | grep -v BUGGY` > if [ -z "$changerfile" ]; then > Exit 2 \ > "" \ > "changerfile must be specified in amanda.conf" > fi > > ... and here is the output: > > bash-2.05$ /usr/local/libexec/chg-zd-mtx -reset > amgetconf changerfile > changerfile must be specified in amanda.conf > > So chg-zd-mtx script does not pass the parameter to any of the am* > executables, including amgetconf. Thus, amgetconf called from within > chg-zd-mtx only works from within the directory containing the amanda.conf > file. > > Anyway to resolve this without modifying the changer script ? > Interesting. An aside, as I understand it, changer scripts generally don't work from any directory except the config directory. I don't think they are passed the config name, so chg-zd-mtx assumes it is in the config directory and the directory name is the same as the config name. The code doing that seems to be: 314 config=`pwd 2>/dev/null` 315 config=`expr "$config" : '.*/\(.*\)'` So your early observation on this point is "as expected". For amgetconf the config argument is optional. If given, it uses the appropriate amanda.conf, but if not given, it uses the amanda.conf in the current directory. So the behavior of chg-zd-mtx in not giving the config name is understandable, it expects to be run from the config directory and amgetconf does not need the config name then. That leads me to question from what directory is chg-zd-mtx being executed when run by amtape or amcheck? Perhaps a few debugging statements to report the current directory in the script would be informative. Alternatively, on Solaris, there are the "proc" command (in /usr/proc/bin on Solaris 8 and 9) and the /proc file system. Using either you could interogate the running commands to determine the current directories of amtape and its child processes. Perhaps using pstop to pause them if they are too fast. BTW you gave a thorough answer to lots of my questions earlier. But stopped at the last couple of paragraphs. One thing I'd like to hear is whether a POSIX shell, probably /bin/ksh on your Solaris, sees to work around the problem It has for others though the reason has not been determined. Change #!/bin/sh to #!/bin/ksh at line 1. jl -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 4455 Province Line Road(609) 252-0159 Princeton, NJ 08540-4322 (609) 683-7220 (fax)
Re: splitting DLE's
I can't remember if this was mentioned, but what is the amanda version of the client giving the error message? I believe you need 2.4.3 or later to specify a "diskname" and "diskdevice". In 2.4.2 the syntax of a DLE would allow only a "diskname". Because the diskname should be unique in a disklist file, the workaround in that time was to add mulitple "/." to the end of a diskname, like this: some.host.org /my/disknamedumptype-with-excludes some.host.org /my/diskname/. dumptype-with-other-excludes some.host.org /my/diskname/./.dumptype-with-yet-other-excludes That is, is you don't want/can't upgrade the client to a recent amanda version. -- Paul Bijnens, XplanationTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
Re: splitting DLE's
On Mon, 11 Apr 2005 at 9:10pm, Jon LaBadie wrote > out of the second field with a 4 field entry. I.e. raidion_HOME_REST > rather than /raidion... I've never used the diskname in a 4 part'er > with slashes. I'm thinking, maybe the leading slash makes some part > of amanda think it is a pathname to the device/directory. Perhaps > if there is no leading slash it will pick up the 3rd field as the device. My 4 part DLEs have slashes in fields 2 and 3, so it should work. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
