Re: GNUTAR problem
Dana Bourgeois wrote: I am running all of these for the first time. Amanda is scheduling 35G (one tape) but only about 3G is actually getting written for one reason or another (some is compression, some is weird tar errors). At this rate it Is the "compression" problem because amanda actually compresses better than the default 50%, you can change that parameter too. See "comprate" in the manpage. The weird tar errors should be sorted out of course. will take weeks to have this running. What's the fastest way to get this going? Should I run amdump a couple of times per day adding clients each run? Perhaps run a different set of clients each night? That would take 4 Do not run multiples times on a day. If you have a large holdingdisk, you could instead set the "maxdumpsize" to a much larger value for one or a few runs. Amanda will schedule that amount, and fill one tape of data. In the morning you flush the rest of the holdingdisk manually. If you have a changer, you could increase runtapes too, which similar effect. But then you need to set reserve to something else than 100 (probably 0). -- Paul @ Home
Re: GNUTAR problem
--On Saturday, November 08, 2003 12:43:12 -0800 Dana Bourgeois <[EMAIL PROTECTED]> wrote: > I am building a new Amanda server and have to use GNUTAR to split up some > very large DLEs. The result is only 128G but 62 DLEs. This is not my > question. > > I am running all of these for the first time. Amanda is scheduling 35G (one > tape) but only about 3G is actually getting written for one reason or > another (some is compression, some is weird tar errors). At this rate it > will take weeks to have this running. What's the fastest way to get this > going? First, you need to fix whatever it is that causes your backups to stop after 3 GB when you have a 35GB tape. Once you solve that Amanda will sort out the rest of it for you relatively quickly (although with four tapes you won't have much room for incrementals [4*35=140 - 128 = 12GB, or about 10% of your data can change each week, but that might be OK for your case. >Should I run amdump a couple of times per day adding clients each > run? No, planner gets confused when run more than once a day. > Perhaps run a different set of clients each night? That would take 4 > nights since my cycle is 4 tapes. Problem is the cycle is also one week so > it will take several weeks to get this going. Should I tell Amanda it has 3 > or 4 tapes to write to at first so it will schedule more stuff? It sounds like Amanda is already scheduling 35GB worth but it is failing for some reason. > > Half of the clients are on a slow hub system with the rest on a fast switch. > So I have potential timeout problems as well. Fiddling with spindle numbers in the disklist might help here, but I wouldn't worry about it until after you get everything up and running. You can also increase the timeouts in amanda.conf, but is not generally needed. If you're not seeing timeout errors don't worry about it. > What is the general rule of thumb in getting a relatively large disklist > going quickly? Planner is not queuing up enough DLEs and I'm getting only > about 10% of the data I should in one run. Some people add a few DLEs per day (useful if you want to make sure certain ones get backed up first), others just toss in the whole batch and ignore the 'no space' errors while planner sorts things out for a few days. > At this rate, it'll take a > months to get everything right. Since I'm using ftape, I'm going to > increase the size of my tapes so planner will get more scheduled up. I > figure later, I'll drop the tape size down when planner has better history. At the start of your email you said Amanda scheduled 35GB but only wrote 3GB, so increasing your tape size won't help, only confuse the planner and increase the chance of it trying to write something too big at the end of your tape and wasting that space. Post your entire daily report (change the hostnames if you want) and this group can give you more detailed help. We might then want you you to look at and/or post some of your debug files and config files depending on the errors and warnings in your daily report. Your first problem to solve is why only 3GB is being written and the daily report will give some clues. Frank > > > Dana Bourgeois > -- Frank Smith [EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
GNUTAR problem
I am building a new Amanda server and have to use GNUTAR to split up some very large DLEs. The result is only 128G but 62 DLEs. This is not my question. I am running all of these for the first time. Amanda is scheduling 35G (one tape) but only about 3G is actually getting written for one reason or another (some is compression, some is weird tar errors). At this rate it will take weeks to have this running. What's the fastest way to get this going? Should I run amdump a couple of times per day adding clients each run? Perhaps run a different set of clients each night? That would take 4 nights since my cycle is 4 tapes. Problem is the cycle is also one week so it will take several weeks to get this going. Should I tell Amanda it has 3 or 4 tapes to write to at first so it will schedule more stuff? Half of the clients are on a slow hub system with the rest on a fast switch. So I have potential timeout problems as well. What is the general rule of thumb in getting a relatively large disklist going quickly? Planner is not queuing up enough DLEs and I'm getting only about 10% of the data I should in one run. At this rate, it'll take a months to get everything right. Since I'm using ftape, I'm going to increase the size of my tapes so planner will get more scheduled up. I figure later, I'll drop the tape size down when planner has better history. Dana Bourgeois
Re: MacOSX and gnutar problem
arrghhg the one at freeforums doesn't work and the the itunes site insists you're on a Mac. I'll do it later:-( -- martin [EMAIL PROTECTED] wrote: > On Mon, 1 Jul 2002, Martin Hepworth wrote: > > >>OK I'll give it a go - got a copy hfspax I can download without having >>to muck around with sea.hqx files etc ie a tar.gz file so I can to this >>remotely with an ssh connection. > > > Trick for dealing with .hqx / .sit files; use the command Open (captial O) > from the cmd line and in the directory where the .sit file is. This > generally creates a folder locally - and you can then continue from > thereon. Same trick with .dmgl see man hdutils. > > >
Re: newbie: gnutar problem?
At 09:22 AM 12/5/2001, Hussain Ali wrote: >Hello, >I have been using amanda without any problems for a while. I used to use >dump but have switched to gnutar for dumps that are larger then the tape >(ie using a more granular technique of direcotory partitioning ) > >Now, it dumps fine, but i cannot recover the data that was archived via >dump or gtar . I get the following error (for both gtarred and sumped >disks): > >[root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore >ivbf 32 - >amrestore: short file header block: 2048 bytes >amrestore: WARNING: not at start of tape, file numbers will be offset >amrestore: 0: reached end of tape: date Q >Verify tape and initialize maps >End-of-tape encountered >Tape is not a dump tape Note your amrestore line. You're piping the image from tape into "restore". As in dump's counterpart. As in NOT tar. You need to use restore to restore dump images, and tar to restore tar images. [...] >-Hussain [...] Cheers. Darin Dugan System Administrator Iowa State University Extension [EMAIL PROTECTED] http://www.extension.iastate.edu
Re: amrecover/GNUTAR problem
Big numbers at the beginning indicate a buggy version of gnutar.Switch to 1.13.19 i think it is. The indexes to your files are messed up, but the data should actually be there. So the experts keep repeating.I haven't used TAR yet, just dump. Deb Baddorf > 7420253310 7365112753 /./files/html/Presentation/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/images/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/mailing lists/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/video/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/website/ > 7420253310 7365112752 /./files/html/Presentation/Sheena/website/images/ > 7420253310 7365112752 /./files/html/Presentation/Templates/ > 7420253310 7365112753 /./files/html/Presentation/_notes/ > 7420253310 7365112753 /./files/html/Presentation/angels/ > 7420253310 7365112753 /./files/html/Presentation/angels/_notes/ > >Could this cause the problem? It worked two months ago and I don't >believe I changed anything. I am running GNUTAR 1.13 >Has anyone ever seen this before? > >Josh --- Deb Baddorf [EMAIL PROTECTED] 840-2289 "You can't help getting older, but you don't have to get old." - George Burns <
amrecover/GNUTAR problem
I am trying to run amrecover to restore some files but I keep getting: Load tape DailySet121 now Continue? [Y/n]: y tar: ./a0016969: Not found in archive tar: Error exit delayed from previous errors extract_list - child returned non-zero status: 2 Continue? [Y/n]: I've checked all the debug logs and everything seems fine so I amrestore'd the level 0 and did a tar -tpGvf hans.sd1a.20020112.0 and this is the format I get: 7420253310 7365112753 /./files/html/Presentation/ 7420253310 7365112752 /./files/html/Presentation/Sheena/ 7420253310 7365112752 /./files/html/Presentation/Sheena/images/ 7420253310 7365112752 /./files/html/Presentation/Sheena/mailing lists/ 7420253310 7365112752 /./files/html/Presentation/Sheena/video/ 7420253310 7365112752 /./files/html/Presentation/Sheena/website/ 7420253310 7365112752 /./files/html/Presentation/Sheena/website/images/ 7420253310 7365112752 /./files/html/Presentation/Templates/ 7420253310 7365112753 /./files/html/Presentation/_notes/ 7420253310 7365112753 /./files/html/Presentation/angels/ 7420253310 7365112753 /./files/html/Presentation/angels/_notes/ Could this cause the problem? It worked two months ago and I don't believe I changed anything. I am running GNUTAR 1.13 Has anyone ever seen this before? Josh
Re: newbie: gnutar problem?
At 09:22 AM 12/5/2001, Hussain Ali wrote: >Hello, >I have been using amanda without any problems for a while. I used to use >dump but have switched to gnutar for dumps that are larger then the tape >(ie using a more granular technique of direcotory partitioning ) > >Now, it dumps fine, but i cannot recover the data that was archived via >dump or gtar . I get the following error (for both gtarred and sumped >disks): > >[root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore >ivbf 32 - >amrestore: short file header block: 2048 bytes >amrestore: WARNING: not at start of tape, file numbers will be offset >amrestore: 0: reached end of tape: date Q >Verify tape and initialize maps >End-of-tape encountered >Tape is not a dump tape Note your amrestore line. You're piping the image from tape into "restore". As in dump's counterpart. As in NOT tar. You need to use restore to restore dump images, and tar to restore tar images. [...] >-Hussain [...] Cheers. Darin Dugan System Administrator Iowa State University Extension [EMAIL PROTECTED] http://www.extension.iastate.edu
Re: newbie: gnutar problem?
Use tar not restore.
amrestore -p /dev/nrst0 fije /u3/mgmt | tar xfp -
Andrew
On Wed, 5 Dec 2001, Hussain Ali wrote:
>
> Hello,
>
> I have been using amanda without any problems for a while. I used to use
> dump but have switched to gnutar for dumps that are larger then the tape
> (ie using a more granular technique of direcotory partitioning )
>
> Now, it dumps fine, but i cannot recover the data that was archived via
> dump or gtar . I get the following error (for both gtarred and sumped
> disks):
>
> [root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore
> ivbf 32 -
> amrestore: short file header block: 2048 bytes
> amrestore: WARNING: not at start of tape, file numbers will be offset
> amrestore: 0: reached end of tape: date Q
> Verify tape and initialize maps
> End-of-tape encountered
> Tape is not a dump tape
>
> note the tape is rewinded and "dd if=/dev/nst0 bs=32k count=1"
>
> AMANDA: TAPESTART DATE 20011130 TAPE normal-012
>
> the tape is good, ie it worked before i started using tar. from what i
> gather from the archives alot there are those who encountered these errors
> with gtar, but the seemed solved... now how do i go about it?
>
> thanks, in advance for any help...
>
> -Hussain
>
>
> * my environment:
>
> netbsd1.5
> gtar 1.12
> amanda2.4.2.p2
>
> * relevent amanda.conf:
>
> define dumptype mydir-nocompress {
> comment "Mydir compress dump type"
> compress none
> program "GNUTAR"
> index yes
> }
>
> relevant inetd.conf
>
> amanda dgram udp waitbackup /usr/pkg/libexec/amandad amandad
> amandaidx stream tcp nowait backup /usr/pkg/libexec/amindexd amindexd
> amidxtape stream tcp nowait backup /usr/pkg/libexec/amidxtaped
>amidxtaped
>
> * screen shot (trying to to recover file that was "dump"ed not
> * "tar"ed)
> amrecover> add dmesg.boot
> Added /var/run/dmesg.boot
> amrecover> extract
>
> Extracting files using tape drive /dev/st0 on host fiji.agiservices.com.
> The following tapes are needed: normal-012
>
> Restoring files into directory /tmp/tar
> Continue? [Y/n]: y
>
> Load tape normal-012 now
> Continue? [Y/n]: y
> Dec 3 11:19:47 fiji inetd[14799]: connection from fiji.agiservices.com,
> service amidxtape (tcp)
> Dec 3 11:19:48 fiji /netbsd: isp0: Bus 1 Target 3 at 10MHz Max Offset 8
> EOF, check amidxtaped.debug file on fiji.agiservices.com.
> amrecover: short block 0 bytes
> UNKNOWN file
> amrecover: Can't read file header
> extract_list - child returned non-zero status: 1
> Continue? [Y/n]: n
> amrecover> quit
>
> ** /tmp/amanda/amidxtaped
> amidxtaped: debug 1 pid 14799 ruid 32770 euid 32770 start time Mon Dec 3
> 11:19:47 2001
> amidxtaped: version 2.4.2.p2
> > SECURITY USER root
> bsd security: remote host fiji.agiservices.com user root local user backup
> amandahosts security check passed
> > 6
> amrestore_nargs=6
> > -h
> > -p
> > /dev/st0
> > fiji
> > ^wd0a$
> > 20011130
> Ready to execv amrestore with:
> path = /usr/pkg/sbin/amrestore
> argv[0] = "amrestore"
> argv[1] = "-h"
> argv[2] = "-p"
> argv[3] = "/dev/st0"
> argv[4] = "fiji"
> argv[5] = "^wd0a$"
> argv[6] = "20011130"
> amrestore: 0: skipping start of tape: date 20011130 label normal-012
> amrestore: missing file header block
> amrestore: 1: reached end of tape: date 20011130
> amidxtaped: amrestore terminated normally with status: 1
> Rewinding tape: done
> amidxtaped: pid 14799 finish time Mon Dec 3 11:19:53 2001
>
>
>
>
>
>
>
>
>
>
>
>
Re: newbie: gnutar problem?
On Wed, 5 Dec 2001 at 10:22am, Hussain Ali wrote > [root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore > ivbf 32 - > amrestore: short file header block: 2048 bytes > amrestore: WARNING: not at start of tape, file numbers will be offset > amrestore: 0: reached end of tape: date Q > Verify tape and initialize maps > End-of-tape encountered > Tape is not a dump tape > > note the tape is rewinded and "dd if=/dev/nst0 bs=32k count=1" > > AMANDA: TAPESTART DATE 20011130 TAPE normal-012 If you're going to use amrestore, then don't dd the tape past the header! amrestore uses the header to identify the dump file. You only need to use dd to get past the header if you intend on using restore or 'tar x' directly. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
newbie: gnutar problem?
Hello,
I have been using amanda without any problems for a while. I used to use
dump but have switched to gnutar for dumps that are larger then the tape
(ie using a more granular technique of direcotory partitioning )
Now, it dumps fine, but i cannot recover the data that was archived via
dump or gtar . I get the following error (for both gtarred and sumped
disks):
[root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore
ivbf 32 -
amrestore: short file header block: 2048 bytes
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore: 0: reached end of tape: date Q
Verify tape and initialize maps
End-of-tape encountered
Tape is not a dump tape
note the tape is rewinded and "dd if=/dev/nst0 bs=32k count=1"
AMANDA: TAPESTART DATE 20011130 TAPE normal-012
the tape is good, ie it worked before i started using tar. from what i
gather from the archives alot there are those who encountered these errors
with gtar, but the seemed solved... now how do i go about it?
thanks, in advance for any help...
-Hussain
* my environment:
netbsd 1.5
gtar1.12
amanda 2.4.2.p2
* relevent amanda.conf:
define dumptype mydir-nocompress {
comment "Mydir compress dump type"
compress none
program "GNUTAR"
index yes
}
relevant inetd.conf
amanda dgram udp waitbackup /usr/pkg/libexec/amandad amandad
amandaidx stream tcp nowait backup /usr/pkg/libexec/amindexd amindexd
amidxtape stream tcp nowait backup /usr/pkg/libexec/amidxtaped amidxtaped
* screen shot (trying to to recover file that was "dump"ed not
* "tar"ed)
amrecover> add dmesg.boot
Added /var/run/dmesg.boot
amrecover> extract
Extracting files using tape drive /dev/st0 on host fiji.agiservices.com.
The following tapes are needed: normal-012
Restoring files into directory /tmp/tar
Continue? [Y/n]: y
Load tape normal-012 now
Continue? [Y/n]: y
Dec 3 11:19:47 fiji inetd[14799]: connection from fiji.agiservices.com,
service amidxtape (tcp)
Dec 3 11:19:48 fiji /netbsd: isp0: Bus 1 Target 3 at 10MHz Max Offset 8
EOF, check amidxtaped.debug file on fiji.agiservices.com.
amrecover: short block 0 bytes
UNKNOWN file
amrecover: Can't read file header
extract_list - child returned non-zero status: 1
Continue? [Y/n]: n
amrecover> quit
** /tmp/amanda/amidxtaped
amidxtaped: debug 1 pid 14799 ruid 32770 euid 32770 start time Mon Dec 3
11:19:47 2001
amidxtaped: version 2.4.2.p2
> SECURITY USER root
bsd security: remote host fiji.agiservices.com user root local user backup
amandahosts security check passed
> 6
amrestore_nargs=6
> -h
> -p
> /dev/st0
> fiji
> ^wd0a$
> 20011130
Ready to execv amrestore with:
path = /usr/pkg/sbin/amrestore
argv[0] = "amrestore"
argv[1] = "-h"
argv[2] = "-p"
argv[3] = "/dev/st0"
argv[4] = "fiji"
argv[5] = "^wd0a$"
argv[6] = "20011130"
amrestore: 0: skipping start of tape: date 20011130 label normal-012
amrestore: missing file header block
amrestore: 1: reached end of tape: date 20011130
amidxtaped: amrestore terminated normally with status: 1
Rewinding tape: done
amidxtaped: pid 14799 finish time Mon Dec 3 11:19:53 2001
newbie: gnutar problem?
Hello,
I have been using amanda without any problems for a while. I used to use
dump but have switched to gnutar for dumps that are larger then the tape
(ie using a more granular technique of direcotory partitioning )
Now, it dumps fine, but i cannot recover the data that was archived via
dump or gtar . I get the following error (for both gtarred and sumped
disks):
[root@fiji /tmp/tar 899]$ amrestore -p /dev/nrst0 fiji /u3/mgmt | restore
ivbf 32 -
amrestore: short file header block: 2048 bytes
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore: 0: reached end of tape: date Q
Verify tape and initialize maps
End-of-tape encountered
Tape is not a dump tape
note the tape is rewinded and "dd if=/dev/nst0 bs=32k count=1"
AMANDA: TAPESTART DATE 20011130 TAPE normal-012
the tape is good, ie it worked before i started using tar. from what i
gather from the archives alot there are those who encountered these errors
with gtar, but the seemed solved... now how do i go about it?
thanks, in advance for any help...
-Hussain
* my environment:
netbsd 1.5
gtar1.12
amanda 2.4.2.p2
* relevent amanda.conf:
define dumptype mydir-nocompress {
comment "Mydir compress dump type"
compress none
program "GNUTAR"
index yes
}
relevant inetd.conf
amanda dgram udp waitbackup /usr/pkg/libexec/amandad amandad
amandaidx stream tcp nowait backup /usr/pkg/libexec/amindexd amindexd
amidxtape stream tcp nowait backup /usr/pkg/libexec/amidxtaped amidxtaped
* screen shot (trying to to recover file that was "dump"ed not
* "tar"ed)
amrecover> add dmesg.boot
Added /var/run/dmesg.boot
amrecover> extract
Extracting files using tape drive /dev/st0 on host fiji.agiservices.com.
The following tapes are needed: normal-012
Restoring files into directory /tmp/tar
Continue? [Y/n]: y
Load tape normal-012 now
Continue? [Y/n]: y
Dec 3 11:19:47 fiji inetd[14799]: connection from fiji.agiservices.com,
service amidxtape (tcp)
Dec 3 11:19:48 fiji /netbsd: isp0: Bus 1 Target 3 at 10MHz Max Offset 8
EOF, check amidxtaped.debug file on fiji.agiservices.com.
amrecover: short block 0 bytes
UNKNOWN file
amrecover: Can't read file header
extract_list - child returned non-zero status: 1
Continue? [Y/n]: n
amrecover> quit
** /tmp/amanda/amidxtaped
amidxtaped: debug 1 pid 14799 ruid 32770 euid 32770 start time Mon Dec 3
11:19:47 2001
amidxtaped: version 2.4.2.p2
> SECURITY USER root
bsd security: remote host fiji.agiservices.com user root local user backup
amandahosts security check passed
> 6
amrestore_nargs=6
> -h
> -p
> /dev/st0
> fiji
> ^wd0a$
> 20011130
Ready to execv amrestore with:
path = /usr/pkg/sbin/amrestore
argv[0] = "amrestore"
argv[1] = "-h"
argv[2] = "-p"
argv[3] = "/dev/st0"
argv[4] = "fiji"
argv[5] = "^wd0a$"
argv[6] = "20011130"
amrestore: 0: skipping start of tape: date 20011130 label normal-012
amrestore: missing file header block
amrestore: 1: reached end of tape: date 20011130
amidxtaped: amrestore terminated normally with status: 1
Rewinding tape: done
amidxtaped: pid 14799 finish time Mon Dec 3 11:19:53 2001
