Re: amrecover cannot find index for host

[John R. Jackson]

>... A 160GB hard drive is about $260. That's much lower cost than
>10 DDS-2 tapes and the changer hardware.  ...

Agreed, but there is a fundamental difference here and that's the
critical failure path.  If one of those 10 tapes fails, you should still
be able to recover the majority of your data.  If that single disk fails,
you're screwed.

I'm not disagreeing with the idea of using disk for backup.  Just saying
it takes some thought to make it a safe setup (tape does too, of course).

>Jeremy Wadsack

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: amrecover cannot find index for host

[Jeremy Wadsack]

Kirk Strauser ([EMAIL PROTECTED]):


> At 2002-01-25T19:16:22Z, "Jeremy Wadsack" <[EMAIL PROTECTED]> writes:

>> We switch from tape- to hdd-based backup because hard drives are MUCH
>> cheaper than tapes these days. I need to figure out the 2GB limit on the
>> drives before I completely restart the system (separate problem.)

> Ehhh?  The good tape drives aren't particularly cheap, but the media is
> practically throw-away money, especially for any shop larger than one
> person.  I mean, DDS-3 tapes are about $15, which translates to $0.625 per
> GB.  Add the fact that tapes are, by their nature, hot-swappable, and I
> think you'd be hard-pressed to find a less-expensive and more-featureful HD
> setup.

Well, $0.625 / GB does not include the cost of the hardware. With a
ten tape changer you *might* get 180GB of storage on "24GB" DDS-3
tapes. A 160GB hard drive is about $260. That's much lower cost than
10 DDS-2 tapes and the changer hardware. And it's significantly
faster. It became impossible for us to backup 120GB of data in the
off-hours of the server and the gtar process was using far too much
resources on live web servers.


-- 

Jeremy Wadsack
Wadsack-Allen Digital Group

Re: amrecover cannot find index for host

[Kirk Strauser]

At 2002-01-25T19:16:22Z, "Jeremy Wadsack" <[EMAIL PROTECTED]> writes:

> We switch from tape- to hdd-based backup because hard drives are MUCH
> cheaper than tapes these days. I need to figure out the 2GB limit on the
> drives before I completely restart the system (separate problem.)

Ehhh?  The good tape drives aren't particularly cheap, but the media is
practically throw-away money, especially for any shop larger than one
person.  I mean, DDS-3 tapes are about $15, which translates to $0.625 per
GB.  Add the fact that tapes are, by their nature, hot-swappable, and I
think you'd be hard-pressed to find a less-expensive and more-featureful HD
setup.
-- 
Kirk Strauser

Re: amrecover cannot find index for host

[John R. Jackson]

>I had to run amrecover twice. Once to find out which 'tape' it wanted.
>Next to run it with the right '-d' pointing to that tape. Is it
>possible to redesign the disk-backup thing to allow the tape to be
>selected during the amrecover process without running it twice?

As one of my other posts said, I think you can trick settape into working.

However, I'd probably use symlinks for this.  Point amrecover (-d)
at a made up name of your choice (e.g. /tmp/amr.$$).  When you figure
out what tape is needed, symlink that to the "right" name.  If you need
multiple tapes to do the restore, which is common, remove and reset the
link whenever amrecover asks for a new tape.

>Jeremy Wadsack

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: amrecover cannot find index for host

[Jeremy Wadsack]

John R. Jackson ([EMAIL PROTECTED]):

>>There is no tape device and I can't give it 'file:/home/amanda/dumps/'
>>because it tries to parse that as host:device!

> Huh?  Did you actually try "-d file:/home/amanda/dumps/" on the command
> line?  What did it say?  I just looked through the code and I don't see
> it parsing that (although I could easily have missed it).

No, I tried it from the 'settape' command. From the command-line it
works .. sort of.

I had to run amrecover twice. Once to find out which 'tape' it wanted.
Next to run it with the right '-d' pointing to that tape. Is it
possible to redesign the disk-backup thing to allow the tape to be
selected during the amrecover process without running it twice?

Thanks,

-- 

Jeremy Wadsack
Wadsack-Allen Digital Group

Re: amrecover cannot find index for host

[John R. Jackson]

Upon even further review :-), it looks like you should be able to put
the device name on the command line with just "-d file:/whatever".
The place you'll have trouble is with the settape command when inside
amrecover, and in that case should use "settape server:file:/whatever".

I think.  I'm just looking at the code, not actually trying any of this.

JJ

Re: amrecover cannot find index for host

[John R. Jackson]

Never mind.  I see where it's happening now.

Try "-d your-tape-server:file:/home/amanda/dumps/".

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: amrecover cannot find index for host

[John R. Jackson]

>There is no tape device and I can't give it 'file:/home/amanda/dumps/'
>because it tries to parse that as host:device!

Huh?  Did you actually try "-d file:/home/amanda/dumps/" on the command
line?  What did it say?  I just looked through the code and I don't see
it parsing that (although I could easily have missed it).

>Jeremy Wadsack

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: amrecover cannot find index for host

[Jeremy Wadsack]

John R. Jackson ([EMAIL PROTECTED]):

>>Does that mean that all the backups are no good, even though they
>>recovered on some servers before?  ...

> As I recall, the backups are OK, it's just the index files that are bogus.
> And as I wrote in private mail, I *think* the bad index files can be
> mangled back into something Amanda will be happy with.

The first command that you gave seems to have worked. At least
amrecover doesn't complain about any missing index.


Next question, though. With 'AMRECOVER Version 2.4.2p2-tapeio' how do
I tell it what tape device to use?

amrecover> extract
amrecover: warning: using /dev/null as the tape device will not work

There is no tape device and I can't give it 'file:/home/amanda/dumps/'
because it tries to parse that as host:device!

Thank very much for all the help.

-- 

Jeremy Wadsack
Wadsack-Allen Digital Group

Re: amrecover cannot find index for host

[Joshua Baker-LePain]

On Fri, 25 Jan 2002 at 12:44pm, Jeremy Wadsack wrote

> > Bingo.  Those numbers at the beginning of each line indicate a bad version 
> > of tar on that client.  You need at least 1.13.17.
> 
> Uh, is there some way Amanad could *detect* this at backup time and
> warn about it? Does that mean that all the backups are no good, even
> though they recovered on some servers before? (I didn't test all
> servers, I admit, I thought they were similar enough...)

IIRC, the backups are fine, it's just the indices that are hosed.  Once 
you get a good version of tar, it's possible to rebuild the indices.

> I specifically installed this version of tar in October to work with
> Amanda. It's version 1.13, which is the latest version available on
> any of the GNU mirrors. If I need a newer version can you tell me
> where to find it?
> 
docs/INSTALL for amanda-2.4.2p2 actually tells you to use either 1.12 plus 
patches or 1.13.19, which is available from 
.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: amrecover cannot find index for host

[John R. Jackson]

>Uh, is there some way Amanad could *detect* this at backup time and
>warn about it?  ...

As much trouble as it's caused, we probably should.

>Does that mean that all the backups are no good, even
>though they recovered on some servers before?  ...

As I recall, the backups are OK, it's just the index files that are bogus.
And as I wrote in private mail, I *think* the bad index files can be
mangled back into something Amanda will be happy with.

>I specifically installed this version of tar in October to work with
>Amanda. It's version 1.13, which is the latest version available on
>any of the GNU mirrors. If I need a newer version can you tell me
>where to find it?

It's at alpha.gnu.org.  No, we have **no** idea why the GNU folks are
still offering that bad version on the normal mirrors.  It's not very
nice of them.

>Jeremy Wadsack

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: amrecover cannot find index for host

[Jeremy Wadsack]

Joshua Baker-LePain ([EMAIL PROTECTED]):

> On Fri, 25 Jan 2002 at 12:16pm, Jeremy Wadsack wrote

>> > What do you get if you do this:
>> 
>> >   cd /var/amanda/daily/index/10.0.0.17/hda1
>> >   /usr/bin/gzip -dc 20020121_1.gz | sort | head
>> 
>> Something that looks really normal to me:
>> 
>> 07042765301/./dev/modem
>> 07042765301/./dev/ttyS0
>> 07422570751/./dev/ttyp0
>> 07422670637/./etc/logrotate.onboot/site27
>> 07422670637/./etc/named.boot
>> 07422670641/./root/.bash_history
>> 07422670641/./root/.ssh/known_hosts
>> ...
>> 
> Bingo.  Those numbers at the beginning of each line indicate a bad version 
> of tar on that client.  You need at least 1.13.17.

Uh, is there some way Amanad could *detect* this at backup time and
warn about it? Does that mean that all the backups are no good, even
though they recovered on some servers before? (I didn't test all
servers, I admit, I thought they were similar enough...)

I specifically installed this version of tar in October to work with
Amanda. It's version 1.13, which is the latest version available on
any of the GNU mirrors. If I need a newer version can you tell me
where to find it?


-- 

Jeremy Wadsack
Wadsack-Allen Digital Group

Re: amrecover cannot find index for host

[Joshua Baker-LePain]

On Fri, 25 Jan 2002 at 12:16pm, Jeremy Wadsack wrote

> > What do you get if you do this:
> 
> >   cd /var/amanda/daily/index/10.0.0.17/hda1
> >   /usr/bin/gzip -dc 20020121_1.gz | sort | head
> 
> Something that looks really normal to me:
> 
> 07042765301/./dev/modem
> 07042765301/./dev/ttyS0
> 07422570751/./dev/ttyp0
> 07422670637/./etc/logrotate.onboot/site27
> 07422670637/./etc/named.boot
> 07422670641/./root/.bash_history
> 07422670641/./root/.ssh/known_hosts
> ...
> 
Bingo.  Those numbers at the beginning of each line indicate a bad version 
of tar on that client.  You need at least 1.13.17.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: amrecover cannot find index for host

[Jeremy Wadsack]

John R. Jackson ([EMAIL PROTECTED]):

>>No index records for disk for specified date
>>...
>>> DISK hda1
>>- 2002-01-22 1 daily2 16
>>- 2002-01-21 1 daily1 19
>>- 2002-01-20 1 daily8 8
>>- 2002-01-19 1 daily7 9

> The first thing that's odd about this is that there are no level 0's.
> That *might* be OK, but it's strange.

We switch from tape- to hdd-based backup because hard drives are MUCH
cheaper than tapes these days. I need to figure out the 2GB limit on
the drives before I completely restart the system (separate problem.)


>>Uncompress command: /usr/bin/gzip -dc '/var/amanda/daily/index/10.0.0.17/hda1/
>>20020121_1.gz' 2>/dev/null | sort > '/var/amanda/daily/index/10.0.0.17/hda1/20
>>020121_1'
>>f /var/amanda/daily/index/10.0.0.17/hda1/20020121_1
>>< 500 "/" is an invalid directory

> What do you get if you do this:

>   cd /var/amanda/daily/index/10.0.0.17/hda1
>   /usr/bin/gzip -dc 20020121_1.gz | sort | head

Something that looks really normal to me:

07042765301/./dev/modem
07042765301/./dev/ttyS0
07422570751/./dev/ttyp0
07422670637/./etc/logrotate.onboot/site27
07422670637/./etc/named.boot
07422670641/./root/.bash_history
07422670641/./root/.ssh/known_hosts
...


Thanks for the help,

-- 

Jeremy Wadsack
Wadsack-Allen Digital Group

Re: amrecover cannot find index for host

[John R. Jackson]

>No index records for disk for specified date
>...
>> DISK hda1
>- 2002-01-22 1 daily2 16
>- 2002-01-21 1 daily1 19
>- 2002-01-20 1 daily8 8
>- 2002-01-19 1 daily7 9

The first thing that's odd about this is that there are no level 0's.
That *might* be OK, but it's strange.

>Uncompress command: /usr/bin/gzip -dc '/var/amanda/daily/index/10.0.0.17/hda1/
>20020121_1.gz' 2>/dev/null | sort > '/var/amanda/daily/index/10.0.0.17/hda1/20
>020121_1'
>f /var/amanda/daily/index/10.0.0.17/hda1/20020121_1
>< 500 "/" is an invalid directory

What do you get if you do this:

  cd /var/amanda/daily/index/10.0.0.17/hda1
  /usr/bin/gzip -dc 20020121_1.gz | sort | head

>Jeremy Wadsack

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]