Re: amrecover as root on client NAK'd

[Gene Heskett]

On Monday 01 July 2019 16:45:51 Nathan Stratton Treadway wrote:

> On Mon, Jul 01, 2019 at 15:28:53 -0400, Gene Heskett wrote:
> > Explain to me, what lcd vs cd does?  What filesystem does each
> > represent?
>
> "cd" changes the working directory in the virtual filesystem built
> from Amanda's index catalog.  You will definitely need to use this
> command as part of your amrecover session (except in the case where
> you actually do want to restore an entire dump in full, anyway).
>
> "lcd" changes the "local" working directory, i.e. the one amrecover
> will write the recoverd files into.  As long as you change into your
> recovery directory (e.g. using the "cd" command from your shell
> prompt, as shown in my sample transcript) before entering amrecover,
> you can completely ignore the "lcd" command.
>
> > And why is there not an lls to list whats at the point of a
> > successful lcd,  like an ls does for a cd?
>
> I can't tell you for sure why amrecover was originally written the way
> it was, but I suspect it might have been modeled after the standard
> Unix "ftp" client program, which was probably a very familiar
> file-transfer mechanism when amrecover was first written...
>
> (I'm sure there are many versions of that floating around, but I think
> the "traditional" version has an "lcd" command with the same meaning
> as the one in amrecover, and does not have "lls".)
>
> You can of course always just check the contents of the directory
> listed in the output of the amrecover "lpwd" command using a shell in
> another terminal window... but in general I usually restore into an
> empty recovery directory (as shown in my example transcript), and as a
> side-effect of that I never have to wonder what currently exists in
> that directory :) .
>
> > One or the other didn't work, I've forgotten which and not being
> > able see where I was, and verify which filesystem was what got so
> > confusing I
>
> In the transcript you posted, it was the "lcd" command that failed;
> that was because the /home/pi did not exist on coyote (where you were
> running amrecover).  As long as you are recovering on the server side,
> I think you will just want to restore into a recovery directory (and
> thus avoid the need for using "lcd")  (Note that if "/home/pi" DID
> actually exist on coyote, you would have dumped copies of the files
> from picnc on top of it -- presumably not what you would have
> intended)
>
> > gave up and used a variation of the first blocks command line to
> > extract the whole 7+gigs of picnc's /, then I walked thru it with mc
> > and copied
>
> (Exactly; if you use amrecover to select just the few subdirectories
> you want to recover, you avoid recovering all the 7GB of other
> unneeded stuff...)
>
> > And FWIW, the commandline in block 1 of the file is bogus, eats cpu
> > like they were M, and 20 minutes of bouncing from core to core
> > didn't output a single byte, so I turned the last tar command in
> > that pipelined chain around to 'tar x -', and had
>
> (The GNUTAR program has used a few different sets of options for "tar"
> over the course of various versions, and I'm not sure which you were
> seeing  but in your case, since you were recovering the files into
> an empty directory and then manually moving them, including setting
> file ownership/permissions, onto the client machine, I suspect that
> any additional options listed in the header command line wouldn't have
> added any useful functionality on top of the basic "tar x -" line you
> used
>
> [Also, while I won't try to claim that amrecover always works without
> a hitch, in general if you use amrecover it takes care of figuring out
> the correct recovery command to use, thus saving you the trouble :)
> .])
>
>   Nathan
>
I've got the lcd problem sussed I think, I was trying, from the server, 
this machine, to set the recovery (the "to" directory) 
to /sshnet/picnc//home/pi. But thats an sshfs mount done as me. But 
amrecover was running as root, and to "root" the /sshnet/picnc directory 
is empty. Part of my paranoia, root is locked away from the rest of the 
local network.

An nfs mount may not have had that problem. But over the years I've had 
so much file corruption with nfs that its probably been a decade since 
the last time I set up an nfs share here at the coyote.den.  For me, 
sshfs Just Works.
>
> --
>-- Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic
> region Ray Ontko & Co.  -  Software consulting services  -  
> http://www.ontko.com/ GPG Key:
> http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239 Key
> fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239



Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt 

Re: amrecover as root on client NAK'd

[Nathan Stratton Treadway]

On Mon, Jul 01, 2019 at 15:28:53 -0400, Gene Heskett wrote:
> Explain to me, what lcd vs cd does?  What filesystem does each represent?

"cd" changes the working directory in the virtual filesystem built from
Amanda's index catalog.  You will definitely need to use this command as
part of your amrecover session (except in the case where you actually
do want to restore an entire dump in full, anyway).

"lcd" changes the "local" working directory, i.e. the one amrecover will
write the recoverd files into.  As long as you change into your recovery
directory (e.g. using the "cd" command from your shell prompt, as shown
in my sample transcript) before entering amrecover, you can completely
ignore the "lcd" command.

> And why is there not an lls to list whats at the point of a successful 
> lcd,  like an ls does for a cd?

I can't tell you for sure why amrecover was originally written the way
it was, but I suspect it might have been modeled after the standard Unix
"ftp" client program, which was probably a very familiar file-transfer
mechanism when amrecover was first written...

(I'm sure there are many versions of that floating around, but I think
the "traditional" version has an "lcd" command with the same meaning as
the one in amrecover, and does not have "lls".)

You can of course always just check the contents of the directory listed
in the output of the amrecover "lpwd" command using a shell in another
terminal window... but in general I usually restore into an empty
recovery directory (as shown in my example transcript), and as a
side-effect of that I never have to wonder what currently exists in that
directory :) .


> One or the other didn't work, I've forgotten which and not being able see 
> where I was, and verify which filesystem was what got so confusing I 

In the transcript you posted, it was the "lcd" command that failed;
that was because the /home/pi did not exist on coyote (where you were
running amrecover).  As long as you are recovering on the server side, I
think you will just want to restore into a recovery directory (and thus
avoid the need for using "lcd")  (Note that if "/home/pi" DID
actually exist on coyote, you would have dumped copies of the files from
picnc on top of it -- presumably not what you would have intended)



> gave up and used a variation of the first blocks command line to extract 
> the whole 7+gigs of picnc's /, then I walked thru it with mc and copied 

(Exactly; if you use amrecover to select just the few subdirectories you
want to recover, you avoid recovering all the 7GB of other unneeded
stuff...)


> And FWIW, the commandline in block 1 of the file is bogus, eats cpu
> like they were M, and 20 minutes of bouncing from core to core
> didn't output a single byte, so I turned the last tar command in that
> pipelined chain around to 'tar x -', and had

(The GNUTAR program has used a few different sets of options for "tar"
over the course of various versions, and I'm not sure which you were
seeing  but in your case, since you were recovering the files into
an empty directory and then manually moving them, including setting file
ownership/permissions, onto the client machine, I suspect that any
additional options listed in the header command line wouldn't have added
any useful functionality on top of the basic "tar x -" line you used

[Also, while I won't try to claim that amrecover always works without a
hitch, in general if you use amrecover it takes care of figuring out
the correct recovery command to use, thus saving you the trouble :) .])

Nathan



Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Monday 01 July 2019 08:30:20 Nathan Stratton Treadway wrote:

> On Sun, Jun 30, 2019 at 21:49:16 -0400, Gene Heskett wrote:
> > On Sunday 30 June 2019 19:40:19 Nathan Stratton Treadway wrote:
> > > amrecover> sethost picnc
> > > 200 Dump host set to picnc.
> > > amrecover> setdisk /
> > > 200 Disk set to /.
> > > amrecover> setdate --06-08
> > > 200 Working date set to 2019-06-08.
> > > amrecover> cd /home/pi/linuxcnc
> > > /home/pi/linuxcnc
> >
> > failed here, no pi to be found.
>
> In the transcript you posted in your message on Mon, 24 Jun 2019
> 15:03:35 -0400, you show a sucessful "cd /home/pi", so if it didn't
> work this time perhaps you didn't set the host to picnc?
>
> (I came up with the path "/home/pi/linuxcnc" based your dd/tar-based
> recovery instructions in a later email, so that could be a bit wrong,
> but you should at least be able to "cd /home/pi" and then use "ls" to
> figured out what to "cd" into below that.)
>
>   Nathan
Explain to me, what lcd vs cd does?  What filesystem does each represent?

And why is there not an lls to list whats at the point of a successful 
lcd,  like an ls does for a cd?

One or the other didn't work, I've forgotten which and not being able see 
where I was, and verify which filesystem was what got so confusing I 
gave up and used a variation of the first blocks command line to extract 
the whole 7+gigs of picnc's /, then I walked thru it with mc and copied 
what I needed.  That took half an hour, and just worked whereas I'd been 
screwing with amrecover for 2 days. And FWIW, the commandline in block 1 
of the file is bogus, eats cpu like they were M, and 20 minutes of 
bouncing from core to core didn't output a single byte, so I turned the 
last tar command in that pipelined chain around to 'tar x -', and had 
the whole thing in a navigatible filesystem in the scratch dir of the 
same drive. In under 20 minutes.
>
> --
>-- Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic
> region Ray Ontko & Co.  -  Software consulting services  -  
> http://www.ontko.com/ GPG Key:
> http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239 Key
> fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239



Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: amrecover as root on client NAK'd

[Nathan Stratton Treadway]

On Sun, Jun 30, 2019 at 21:49:16 -0400, Gene Heskett wrote:
> On Sunday 30 June 2019 19:40:19 Nathan Stratton Treadway wrote:
> > amrecover> sethost picnc
> > 200 Dump host set to picnc.
> > amrecover> setdisk /
> > 200 Disk set to /.
> > amrecover> setdate --06-08
> > 200 Working date set to 2019-06-08.
> > amrecover> cd /home/pi/linuxcnc
> > /home/pi/linuxcnc
> 
> failed here, no pi to be found.

In the transcript you posted in your message on Mon, 24 Jun 2019
15:03:35 -0400, you show a sucessful "cd /home/pi", so if it didn't work 
this time perhaps you didn't set the host to picnc?

(I came up with the path "/home/pi/linuxcnc" based your dd/tar-based
recovery instructions in a later email, so that could be a bit wrong,
but you should at least be able to "cd /home/pi" and then use "ls" to
figured out what to "cd" into below that.)

Nathan 


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Sunday 30 June 2019 19:40:19 Nathan Stratton Treadway wrote:

> On Thu, Jun 27, 2019 at 16:38:25 -0400, Gene Heskett wrote:
> > On Thursday 27 June 2019 14:22:05 Debra S Baddorf wrote:
> > > Amrecover  on picnc didn???t work, so you went straight to dd.
> > > (Well, I frequently use dd too, but:  )
> > > Did you try amrecover  on the server node?   Start yourself in a
> > > temp area, amrecover  ,
> > > set host  picnc
> > > setdisk (  /  was it?)
> > > cd  one dir at a time.
> > > extract
> > > edit and copy files to client as needed
> >
> > I expect, but wasn't thinking of that, but I think amrecover could
> > probably have dumped the whole dle to some scratch space right on
> > the same drive, doing essentially what I did.
>
> Deb didn't mention the "add" step explicitly in her overview, but she
> hinted at one big benefit of using amrecover: it lets you select just
> the specific files you want to restore (baseed on the files-dumped
> index it keeps, as made available by the "index server").
>
> Presumably decovering these particular files is water under the bridge
> now, but just to give a more concrete example, you could done
> something along these lines:
>
> =
> gene@coyote:~$ mkdir recoverytemp
> gene@coyote:~$ cd recoverytemp
> gene@coyote:~/recoverytemp$ sudo amrecover Daily
> [sudo] password for gene:
> AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote ...
> 220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
> Setting restore date to today (2019-06-24)
> 200 Working date set to 2019-06-24.
> 200 Config set to Daily.
> 200 Dump host set to coyote.
> Use the setdisk command to choose dump disk to recover
> amrecover> sethost picnc
> 200 Dump host set to picnc.
> amrecover> setdisk /
> 200 Disk set to /.
> amrecover> setdate --06-08
> 200 Working date set to 2019-06-08.
> amrecover> cd /home/pi/linuxcnc
> /home/pi/linuxcnc

failed here, no pi to be found.

> amrecover> ls
> [...]
> amrecover> add configs
> Added dir /home/pi/linuxcnc/configs/ at date 2019-06-08-hh-mm-ss
> amrecover> add nc_files
> Added dir /home/pi/linuxcnc/nc_files/ at date 2019-06-08-hh-mm-ss
> amrecover> list
> TAPE Daily:: LEVEL 0 DATE 2019-06-08-hh-mm-ss
> /home/pi/linuxcnc/configs/
> /home/pi/linuxcnc/nc_files/
> amrecover> extract
> [...]
> =
>
> At that point amrecover should (after prompting you to make sure the
> specific "tape" is available) proceed to find the vtape with the
> desired label and then extract just the two requested subdirectories
> (leaving the recovered directories trees underneath the current local
> directory, i.e. ~/recoverytemp).
>
> Note that because I "cd" into the recoverytemp directory before I
> start amrecover, and because I know that is an empty directory created
> just for this purpose, I don't have to worry about "lcd" or anything
> once inside amrecover; I only have to worry about finding/adding the
> files that I want to recover.
>
> Also note that "ls" shows the files found in the current "virtual"
> directory (based on the index), while "list" shows the
> files/subdirectories which have been queued for extraction.
>
> (Obviously I contructed the top and bottom parts of of that transcript
> by hand, based on the contents of your earlier emails; they might not
> be exactly correct, but hopefully it is close enough to see what I
> mean.)
>
> > And I still would have had to copy what I wanted to an intermediate
> > location where I could change the perms back to the target machine,
> > then put then back where they belonged.
>
> (Right, the transfer-the-recovered-files-to-the-client-machine work
> would be exactly the same as what you did this time around.)
>
>   Nathan
>
> --
>-- Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic
> region Ray Ontko & Co.  -  Software consulting services  -  
> http://www.ontko.com/ GPG Key:
> http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239 Key
> fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239



Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: amrecover as root on client NAK'd

[Nathan Stratton Treadway]

On Thu, Jun 27, 2019 at 16:38:25 -0400, Gene Heskett wrote:
> On Thursday 27 June 2019 14:22:05 Debra S Baddorf wrote:
> > Amrecover  on picnc didn???t work, so you went straight to dd.
> > (Well, I frequently use dd too, but:  )
> > Did you try amrecover  on the server node?   Start yourself in a temp
> > area, amrecover  ,
> > set host  picnc
> > setdisk (  /  was it?)
> > cd  one dir at a time.
> > extract
> > edit and copy files to client as needed


> I expect, but wasn't thinking of that, but I think amrecover could 
> probably have dumped the whole dle to some scratch space right on the 
> same drive, doing essentially what I did. 

Deb didn't mention the "add" step explicitly in her overview, but she
hinted at one big benefit of using amrecover: it lets you select just
the specific files you want to restore (baseed on the files-dumped index
it keeps, as made available by the "index server").

Presumably decovering these particular files is water under the bridge
now, but just to give a more concrete example, you could done something
along these lines:

=
gene@coyote:~$ mkdir recoverytemp
gene@coyote:~$ cd recoverytemp
gene@coyote:~/recoverytemp$ sudo amrecover Daily
[sudo] password for gene:
AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote ...
220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
Setting restore date to today (2019-06-24)
200 Working date set to 2019-06-24.
200 Config set to Daily.
200 Dump host set to coyote.
Use the setdisk command to choose dump disk to recover
amrecover> sethost picnc
200 Dump host set to picnc.
amrecover> setdisk /
200 Disk set to /.
amrecover> setdate --06-08
200 Working date set to 2019-06-08.
amrecover> cd /home/pi/linuxcnc
/home/pi/linuxcnc
amrecover> ls
[...]
amrecover> add configs
Added dir /home/pi/linuxcnc/configs/ at date 2019-06-08-hh-mm-ss
amrecover> add nc_files
Added dir /home/pi/linuxcnc/nc_files/ at date 2019-06-08-hh-mm-ss
amrecover> list
TAPE Daily:: LEVEL 0 DATE 2019-06-08-hh-mm-ss
/home/pi/linuxcnc/configs/
/home/pi/linuxcnc/nc_files/
amrecover> extract
[...]
=

At that point amrecover should (after prompting you to make sure the
specific "tape" is available) proceed to find the vtape with the desired
label and then extract just the two requested subdirectories (leaving
the recovered directories trees underneath the current local directory,
i.e. ~/recoverytemp).

Note that because I "cd" into the recoverytemp directory before I start
amrecover, and because I know that is an empty directory created just
for this purpose, I don't have to worry about "lcd" or anything once
inside amrecover; I only have to worry about finding/adding the files
that I want to recover.

Also note that "ls" shows the files found in the current "virtual"
directory (based on the index), while "list" shows the
files/subdirectories which have been queued for extraction.

(Obviously I contructed the top and bottom parts of of that transcript
by hand, based on the contents of your earlier emails; they might not be
exactly correct, but hopefully it is close enough to see what I mean.)


> And I still would have had to copy what I wanted to an intermediate
> location where I could change the perms back to the target machine,
> then put then back where they belonged.

(Right, the transfer-the-recovered-files-to-the-client-machine work
would be exactly the same as what you did this time around.)

Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Thursday 27 June 2019 14:22:05 Debra S Baddorf wrote:

> > On Jun 26, 2019, at 6:33 PM, Gene Heskett 
> > wrote:
> >
> > On Wednesday 26 June 2019 17:54:12 Nathan Stratton Treadway wrote:
> >> On Tue, Jun 25, 2019 at 03:45:15 -0400, Gene Heskett wrote:
> >>> I tried to run amrecover on picnc, but was NAK'd at every step.
> >>> This even though I have added the root line to
> >>> /var/backups/.amandahosts.
> >>
> >> (Like so many aspects of Amanda, it's very likely possible to get
> >> amrecover working on picnc -- but that's a whole separate
> >> discussion, and since you have already retrieved the files you
> >> needed, I don't know how worthwhile it is to go down that road at
> >> this point)
> >>
> >>> So I located the last full which was on the 16th, and using the
> >>> usual dd if=/path/to/file._.0 bs=32k skip-1 | gzip -d | tar x -
> >>> command, unpacked it to some spare space on the /amandatapes
> >>> drive, then copied what I needed from that tree to
> >>> /home/gene/linuxcnc/transfer-dir,
> >>
> >> The goal of my earlier message was just to point out that running
> >> amrecover into a temporary working directory on the server is a
> >> middle ground.  With that approach, you let Amanda (via
> >> "amrecover") manage all the inventory-of-dumps tracking and
> >> unpacking of the files (including the possibility of limiting the
> >> unpacking to only the particular subdirectories you care about)...
> >> but without the need to worry about setting anything up on the
> >> client side.
> >>
> >> (Obviously since you are skipping the client side setup you do
> >> still need to copy the recovered files from the server over to the
> >> client, but that process would be identical to what you just did
> >> above, just starting with the directory tree created my amrecover
> >> instead of the one created by "tar x".)
> >>
> >> But hey, if you don't mind building the dd/gzip/tar command
> >> pipeline yourself, that certainly gets the job done too :)
> >>
> >>Nathan
> >
> > Just one big horsefly sized bug in following the directions in that
> > files header. The last argument to tar was a 4 char string I've
> > never seen before, and I watched it grind away at 100% of a random
> > core for about an hour before I gave it a ctrl-c. Quit instantly and
> > did no damage, so I up-arrowed and edited that string to just an x
> > >picnic-slash.
> >
> > Since it was a good sized file, that also took a while, but gave me
> > a filesystem I could walk thru to what I needed, and copy it out.
> > And once I had the output, the rest was think, faster.  But while I
> > did get it, I feel like I should have been able to do it with
> > amrecover. And I'm less than pleased.  That _is what amrecover is
> > for_  is it not?
> >
> > Thanks Nathan.
>
> Amrecover  on picnc didn’t work, so you went straight to dd.
> (Well, I frequently use dd too, but:  )
> Did you try amrecover  on the server node?   Start yourself in a temp
> area, amrecover  ,
> set host  picnc
> setdisk (  /  was it?)
> cd  one dir at a time.
> extract
> edit and copy files to client as needed
>
> If client is using tar,  this should work fine.
> If client is using dump,  then server can only do this if server is
> the same flavor of machine,  using  compatible restore  program.   But
> I think you were on tar, so no sweat.   This gets you the result 
> (minus final copying)  entirely from amanda.
>
> Deb Baddorf
> Fermilab

I expect, but wasn't thinking of that, but I think amrecover could 
probably have dumped the whole dle to some scratch space right on the 
same drive, doing essentially what I did. And I still would have had to 
copy what I wanted to an intermediate location where I could change the 
perms back to the target machine, then put then back where they 
belonged. But due to the poor choice of words in the man pages, I have 
difficulty tracking what a cd vs an lcd does, and pwm/lpwm suffer a 
similar problem.  Some of that confusion could be alleviated
by orthogonality between ls's, allowing one to see where you are in both 
the clients file structure and in the dle's structure.  But it has no 
lls to match the lpwd. So you need to type blind, sort of a cuss and cry 
operation. Not at all conducive to achieving an "insitu" recovery.

I realize the man pages are somewhat archival and make the assumption 
everone uses tape. But vtapes on a big hd are still about 1000 times 
faster and are random access. I suppose one could have more than one 
drive, and cycle them between local and offsite storage.

And I see the planner is gradually adjusting things, it hasn't surprised 
me and actually used 2 "tapes" in a couple weeks now.  So some progress 
is being made.

I hope its a little cooler at your location, it 90F here and I think my 
AC might be needing a recharge.

Take care Deb.

Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of 

Re: amrecover as root on client NAK'd

[Debra S Baddorf]

> On Jun 26, 2019, at 6:33 PM, Gene Heskett  wrote:
> 
> On Wednesday 26 June 2019 17:54:12 Nathan Stratton Treadway wrote:
> 
>> On Tue, Jun 25, 2019 at 03:45:15 -0400, Gene Heskett wrote:
>>> I tried to run amrecover on picnc, but was NAK'd at every step. 
>>> This even though I have added the root line to
>>> /var/backups/.amandahosts.
>> 
>> (Like so many aspects of Amanda, it's very likely possible to get
>> amrecover working on picnc -- but that's a whole separate discussion,
>> and since you have already retrieved the files you needed, I don't
>> know how worthwhile it is to go down that road at this point)
>> 
>>> So I located the last full which was on the 16th, and using the
>>> usual dd if=/path/to/file._.0 bs=32k skip-1 | gzip -d | tar x -
>>> command, unpacked it to some spare space on the /amandatapes drive,
>>> then copied what I needed from that tree to
>>> /home/gene/linuxcnc/transfer-dir,
>> 
>> The goal of my earlier message was just to point out that running
>> amrecover into a temporary working directory on the server is a middle
>> ground.  With that approach, you let Amanda (via "amrecover") manage
>> all the inventory-of-dumps tracking and unpacking of the files
>> (including the possibility of limiting the unpacking to only the
>> particular subdirectories you care about)... but without the need to
>> worry about setting anything up on the client side.
>> 
>> (Obviously since you are skipping the client side setup you do still
>> need to copy the recovered files from the server over to the client,
>> but that process would be identical to what you just did above, just
>> starting with the directory tree created my amrecover instead of the
>> one created by "tar x".)
>> 
>> But hey, if you don't mind building the dd/gzip/tar command pipeline
>> yourself, that certainly gets the job done too :)
>> 
>>  Nathan
> Just one big horsefly sized bug in following the directions in that files 
> header. The last argument to tar was a 4 char string I've never seen 
> before, and I watched it grind away at 100% of a random core for about 
> an hour before I gave it a ctrl-c. Quit instantly and did no damage, so 
> I up-arrowed and edited that string to just an x >picnic-slash.
> 
> Since it was a good sized file, that also took a while, but gave me a 
> filesystem I could walk thru to what I needed, and copy it out. And once 
> I had the output, the rest was think, faster.  But while I did get it, I 
> feel like I should have been able to do it with amrecover. And I'm less 
> than pleased.  That _is what amrecover is for_  is it not?
> 
> Thanks Nathan.


Amrecover  on picnc didn’t work, so you went straight to dd.
(Well, I frequently use dd too, but:  )
Did you try amrecover  on the server node?   Start yourself in a temp area,
amrecover  ,
set host  picnc
setdisk (  /  was it?)
cd  one dir at a time.
extract
edit and copy files to client as needed

If client is using tar,  this should work fine.
If client is using dump,  then server can only do this if server is the same 
flavor
of machine,  using  compatible restore  program.   But I think you were on tar,
so no sweat.   This gets you the result  (minus final copying)  entirely from 
amanda.

Deb Baddorf
Fermilab

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Wednesday 26 June 2019 17:54:12 Nathan Stratton Treadway wrote:

> On Tue, Jun 25, 2019 at 03:45:15 -0400, Gene Heskett wrote:
> > I tried to run amrecover on picnc, but was NAK'd at every step. 
> > This even though I have added the root line to
> > /var/backups/.amandahosts.
>
> (Like so many aspects of Amanda, it's very likely possible to get
> amrecover working on picnc -- but that's a whole separate discussion,
> and since you have already retrieved the files you needed, I don't
> know how worthwhile it is to go down that road at this point)
>
> > So I located the last full which was on the 16th, and using the
> > usual dd if=/path/to/file._.0 bs=32k skip-1 | gzip -d | tar x -
> > command, unpacked it to some spare space on the /amandatapes drive,
> > then copied what I needed from that tree to
> > /home/gene/linuxcnc/transfer-dir,
>
> The goal of my earlier message was just to point out that running
> amrecover into a temporary working directory on the server is a middle
> ground.  With that approach, you let Amanda (via "amrecover") manage
> all the inventory-of-dumps tracking and unpacking of the files
> (including the possibility of limiting the unpacking to only the
> particular subdirectories you care about)... but without the need to
> worry about setting anything up on the client side.
>
> (Obviously since you are skipping the client side setup you do still
> need to copy the recovered files from the server over to the client,
> but that process would be identical to what you just did above, just
> starting with the directory tree created my amrecover instead of the
> one created by "tar x".)
>
> But hey, if you don't mind building the dd/gzip/tar command pipeline
> yourself, that certainly gets the job done too :)
>
>   Nathan
Just one big horsefly sized bug in following the directions in that files 
header. The last argument to tar was a 4 char string I've never seen 
before, and I watched it grind away at 100% of a random core for about 
an hour before I gave it a ctrl-c. Quit instantly and did no damage, so 
I up-arrowed and edited that string to just an x >picnic-slash.

Since it was a good sized file, that also took a while, but gave me a 
filesystem I could walk thru to what I needed, and copy it out. And once 
I had the output, the rest was think, faster.  But while I did get it, I 
feel like I should have been able to do it with amrecover. And I'm less 
than pleased.  That _is what amrecover is for_  is it not?

Thanks Nathan.

>
> --
>-- Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic
> region Ray Ontko & Co.  -  Software consulting services  -  
> http://www.ontko.com/ GPG Key:
> http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239 Key
> fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239



Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: amrecover as root on client NAK'd

[Nathan Stratton Treadway]

On Tue, Jun 25, 2019 at 03:45:15 -0400, Gene Heskett wrote:
> I tried to run amrecover on picnc, but was NAK'd at every step.  This 
> even though I have added the root line to /var/backups/.amandahosts.

(Like so many aspects of Amanda, it's very likely possible to get
amrecover working on picnc -- but that's a whole separate discussion,
and since you have already retrieved the files you needed, I don't know
how worthwhile it is to go down that road at this point)


> So I located the last full which was on the 16th, and using the usual dd 
> if=/path/to/file._.0 bs=32k skip-1 | gzip -d | tar x -
> command, unpacked it to some spare space on the /amandatapes drive, then 
> copied what I needed from that tree to /home/gene/linuxcnc/transfer-dir, 

The goal of my earlier message was just to point out that running
amrecover into a temporary working directory on the server is a middle
ground.  With that approach, you let Amanda (via "amrecover") manage all
the inventory-of-dumps tracking and unpacking of the files (including
the possibility of limiting the unpacking to only the particular
subdirectories you care about)... but without the need to worry about
setting anything up on the client side.

(Obviously since you are skipping the client side setup you do still
need to copy the recovered files from the server over to the client, but
that process would be identical to what you just did above, just
starting with the directory tree created my amrecover instead of the one
created by "tar x".)

But hey, if you don't mind building the dd/gzip/tar command pipeline
yourself, that certainly gets the job done too :)

Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Tuesday 25 June 2019 00:42:38 Nathan Stratton Treadway wrote:

> On Mon, Jun 24, 2019 at 15:03:35 -0400, Gene Heskett wrote:
> > But right now I need to reach back about 10 days and recover
> > my /home/pi/linuxcnc directory which contains the configs and
> > nc_files to run this 1500 lb lathe.
> >
> > But it won't let me run it on the client.  On this host, it won't
> > let me setdisk to /home/pi
> > Session:
> > gene@coyote:~$ sudo amrecover Daily
> > [sudo] password for gene:
> > AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote
> > ... 220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
> > Setting restore date to today (2019-06-24)
> > 200 Working date set to 2019-06-24.
> > 200 Config set to Daily.
> > 200 Dump host set to coyote.
> > Use the setdisk command to choose dump disk to recover
> > amrecover> sethost picnc
> > 200 Dump host set to picnc.
> > amrecover> setdisk /
> > 200 Disk set to /.
> > amrecover> setdate --06-08
> > 200 Working date set to 2019-06-08.
> > amrecover> cd /home/pi
> > /home/pi
> > amrecover> lcd /home/pi
> > /home/pi: No such file or directory
> > amrecover>
> >
> >
> > so that both lpwd and pwd show the same paths.
> > which should be /home/pi
> > but I cannot
> > amrecover> lcd pi
> > pi: No such file or directory
> >
> > What I want is to extract the /home/pi/linuxcnc directory insitu
> > I've setdate to the date of the last full of this particular file.
> > That was on --06-08, and there were about 4 incrementals since.
> >
> > How do I do this?  Thank you.
>
> Looks like you have already retrieved the files you needed, but to
> (attempt to) answer your original question:
>
> From the above transcript you appear to be running amrecover on
> coyote, but trying to change the local directory to "/home/pi" but
> I assume that directory only exists on picnc, right?
>
> (Note that you actually do seem to have successfully set the amrecover
> "extraction source directory" to /home/pi on the "picnc /" disk; the
> error came when you tried to use the "lcd" command to set the
> extraction destination directory.)
>
> If you were running amrecover on picnc it might make sense to recover
> directly in-place, but presumably since you are extracting on coyote
> you would want to create a temporary working directory to hold the
> recovered directory tree, run the amrecover session from within that
> temporary directory, then copy/move the files you need from that
> directory tree over to the appropriate locations on picnc
>
>
>   Nathan
I tried to run amrecover on picnc, but was NAK'd at every step.  This 
even though I have added the root line to /var/backups/.amandahosts.

So I located the last full which was on the 16th, and using the usual dd 
if=/path/to/file._.0 bs=32k skip-1 | gzip -d | tar x -
command, unpacked it to some spare space on the /amandatapes drive, then 
copied what I needed from that tree to /home/gene/linuxcnc/transfer-dir, 
where I chowned it to match the picnc stuffs, then mc'd it to 
an /sshnet/picnc//home/pi/linuxcnc/configs, and nc_files dirs.  And I 
had it done in 20 minutes, not the 2 days I had been battling amrecover 
to try and get it done.

I have no clue what I was doing wrong, but I do consider the man pages 
wordings as less than helpfull at schooling me on how to do it.  
Likewise the online help sucks, And while I can and have amrecovered 
insitu on the server itself, I'd have to say that my inability to do the 
same for a client machine is a serious bug. One that really ought to be 
fixed.  Since amanda uses a different rights image than /sshnet, which 
is strictly a user 1000 access path, the whole /sshnet thing is blocked 
for root as a security measure in case someone actually gains root thru 
my router. That has only happened once in 20 years and I had to give Jim 
the credentials to do it. I needed help with an internal networking 
problem and it took Jim less than a minute to fix it once in. dd-wrt is 
the best guard dog I've never had to feed or clean up after.  If you 
don't have it flashed into your router between your local net and the 
world on the other side of your modem, you simply are leaving your 
systems open for the worlds black hats to exploit. It blocks out the 
world noise so well that I don't have any iptables or other security 
things running on this side of the router except portsentry and 
fail2ban. They only run on this machine, the other 6 are running naked, 
and that includes a win10 home edition machine.

>
> --
>-- Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic
> region Ray Ontko & Co.  - Software consulting services -
> http://www.ontko.com/ GPG Key:
> http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239 Key
> fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239



Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 

Re: amrecover as root on client NAK'd

[Nathan Stratton Treadway]

On Mon, Jun 24, 2019 at 15:03:35 -0400, Gene Heskett wrote:
> But right now I need to reach back about 10 days and recover 
> my /home/pi/linuxcnc directory which contains the configs and nc_files 
> to run this 1500 lb lathe.
> 
> But it won't let me run it on the client.  On this host, it won't let me 
> setdisk to /home/pi
> Session:
> gene@coyote:~$ sudo amrecover Daily
> [sudo] password for gene:
> AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote ...
> 220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
> Setting restore date to today (2019-06-24)
> 200 Working date set to 2019-06-24.
> 200 Config set to Daily.
> 200 Dump host set to coyote.
> Use the setdisk command to choose dump disk to recover
> amrecover> sethost picnc
> 200 Dump host set to picnc.
> amrecover> setdisk /
> 200 Disk set to /.
> amrecover> setdate --06-08
> 200 Working date set to 2019-06-08.
> amrecover> cd /home/pi
> /home/pi
> amrecover> lcd /home/pi
> /home/pi: No such file or directory
> amrecover> 
> 
>
> so that both lpwd and pwd show the same paths.
> which should be /home/pi
> but I cannot 
> amrecover> lcd pi
> pi: No such file or directory
> 
> What I want is to extract the /home/pi/linuxcnc directory insitu
> I've setdate to the date of the last full of this particular file.
> That was on --06-08, and there were about 4 incrementals since.
> 
> How do I do this?  Thank you. 

Looks like you have already retrieved the files you needed, but to
(attempt to) answer your original question: 

>From the above transcript you appear to be running amrecover on coyote,
but trying to change the local directory to "/home/pi" but I assume
that directory only exists on picnc, right?

(Note that you actually do seem to have successfully set the amrecover
"extraction source directory" to /home/pi on the "picnc /" disk; the
error came when you tried to use the "lcd" command to set the extraction
destination directory.)

If you were running amrecover on picnc it might make sense to recover
directly in-place, but presumably since you are extracting on coyote you
would want to create a temporary working directory to hold the recovered
directory tree, run the amrecover session from within that temporary
directory, then copy/move the files you need from that directory tree
over to the appropriate locations on picnc


Nathan



Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray
Ontko & Co.  - Software consulting services - http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: amrecover as root on client NAK'd

[Jon LaBadie]

On Mon, Jun 24, 2019 at 03:03:35PM -0400, Gene Heskett wrote:
> Greetings all;
> 
> I have a pi running stretch with an rt-preempt kernel so it can run 
> linuxcnc, which according to my tests so far it should do it nicely, its 
> realtime enough to handle servo machines with a 1 kilohertz update rate, 
> jitter is about 29 microseconds.
> 
> But I've followed the output of amcheck and it now reports no errors, 
> although the excludes file is empty, something I'll need to fix.

I thought that was the norm.  The file is there so amcheck is happy.
It doesn't check the contents.  That is so you can define an exclude
file for all DLEs and if unneeded, leave empty.

-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: amrecover as root on client NAK'd

[Gene Heskett]

On Monday 24 June 2019 16:16:55 Debra S Baddorf wrote:

> > On Jun 24, 2019, at 2:03 PM, Gene Heskett 
> > wrote:
> >
> > Greetings all;
> >
> > I have a pi running stretch with an rt-preempt kernel so it can run
> > linuxcnc, which according to my tests so far it should do it nicely,
> > its realtime enough to handle servo machines with a 1 kilohertz
> > update rate, jitter is about 29 microseconds.
> >
> > But I've followed the output of amcheck and it now reports no
> > errors, although the excludes file is empty, something I'll need to
> > fix.
> >
> > But right now I need to reach back about 10 days and recover
> > my /home/pi/linuxcnc directory which contains the configs and
> > nc_files to run this 1500 lb lathe.
> >
> > But it won't let me run it on the client.  On this host, it won't
> > let me setdisk to /home/pi
> > Session:
> > gene@coyote:~$ sudo amrecover Daily
> > [sudo] password for gene:
> > AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote
> > ... 220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
> > Setting restore date to today (2019-06-24)
> > 200 Working date set to 2019-06-24.
> > 200 Config set to Daily.
> > 200 Dump host set to coyote.
> > Use the setdisk command to choose dump disk to recover
> > amrecover> sethost picnc
> > 200 Dump host set to picnc.
> > amrecover> setdisk /
> > 200 Disk set to /.
> > amrecover> setdate --06-08
> > 200 Working date set to 2019-06-08.
> > amrecover> cd /home/pi
> > /home/pi
> > amrecover> lcd /home/pi
> > /home/pi: No such file or directory
> > amrecover>
> >
> >
> > so that both lpwd and pwd show the same paths.
> > which should be /home/pi
> > but I cannot
> > amrecover> lcd pi
> > pi: No such file or directory
> >
> > What I want is to extract the /home/pi/linuxcnc directory insitu
> > I've setdate to the date of the last full of this particular file.
> > That was on --06-08, and there were about 4 incrementals since.
> >
> > How do I do this?  Thank you.
> >
> > Cheers, Gene Heskett
>
> I *think*   I’ve had to   “cd  /home”  then  “cd pi”  …. i.e.  do one
> directory at a time. Actually, since you are already in   “setdisk / “
>then the first command is  “cd home”   with no slash.
>
> Worth a try, anyway…
>
> Deb Baddorf
> Fermilab
dead in the water Deb.

At this point, I think I'll use some of /amandatapes spare space to 
extract the last full, then go find what I need, and copy it back by 
hand. AIR use dd to feed starting at 32k, to tar xv
better yet use dd to get the instructions from the files header 

which I did, but the final tar's argument was wrong and it errored 
without creating an output file.  So I just fed it to tar x -, and it 
worked perfectly. Then I used a root mc to copy the 2 trees I needed to 
a transfer dir in my /home/gene/linuxcnc dir, did a chown -R gene:gene 
on both of those dirs.  Then found /sshnet/picnc was empty, so dropped 
out of root, cd, then ran bin/mount-machines answering with my pw, which 
brought that link to life, fired up mc as me, and copied those 2 dirs 
to /sshnet/picnc/home/pi/linucnc.shut that down and ran linuxcnc picking 
the normal config, and 20 seconds later I had the same gui I've been 
looking at for a couple years.  So I think I am in business. Just one 
problem, the grin has pushed my ears a good half an inch to the 
rear. :-)

Thanks for the hand holding, Deb, its both appreciated, and helps keep my 
train of thought from getting derailed in the minutia. At 84 & still 
counting, that is very easy to do.  Theres's no one else here that both 
understands what I'm doing, or cares if I do it. Nobody else to talk to 
IOW.

Its past dinnertime here, so I'd better go do something about that. 

Take care now.

Copyright 2019 by Maurice E. Heskett
Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: amrecover as root on client NAK'd

[Debra S Baddorf]

> On Jun 24, 2019, at 2:03 PM, Gene Heskett  wrote:
> 
> Greetings all;
> 
> I have a pi running stretch with an rt-preempt kernel so it can run 
> linuxcnc, which according to my tests so far it should do it nicely, its 
> realtime enough to handle servo machines with a 1 kilohertz update rate, 
> jitter is about 29 microseconds.
> 
> But I've followed the output of amcheck and it now reports no errors, 
> although the excludes file is empty, something I'll need to fix.
> 
> But right now I need to reach back about 10 days and recover 
> my /home/pi/linuxcnc directory which contains the configs and nc_files 
> to run this 1500 lb lathe.
> 
> But it won't let me run it on the client.  On this host, it won't let me 
> setdisk to /home/pi
> Session:
> gene@coyote:~$ sudo amrecover Daily
> [sudo] password for gene:
> AMRECOVER Version 3.5.1.git.19364c7b. Contacting server on coyote ...
> 220 coyote AMANDA index server (3.5.1.git.19364c7b) ready.
> Setting restore date to today (2019-06-24)
> 200 Working date set to 2019-06-24.
> 200 Config set to Daily.
> 200 Dump host set to coyote.
> Use the setdisk command to choose dump disk to recover
> amrecover> sethost picnc
> 200 Dump host set to picnc.
> amrecover> setdisk /
> 200 Disk set to /.
> amrecover> setdate --06-08
> 200 Working date set to 2019-06-08.
> amrecover> cd /home/pi
> /home/pi
> amrecover> lcd /home/pi
> /home/pi: No such file or directory
> amrecover> 
> 
> 
> so that both lpwd and pwd show the same paths.
> which should be /home/pi
> but I cannot 
> amrecover> lcd pi
> pi: No such file or directory
> 
> What I want is to extract the /home/pi/linuxcnc directory insitu
> I've setdate to the date of the last full of this particular file.
> That was on --06-08, and there were about 4 incrementals since.
> 
> How do I do this?  Thank you. 
> 
> Cheers, Gene Heskett

I *think*   I’ve had to   “cd  /home”  then  “cd pi”  …. i.e.  do one directory 
at a time.
Actually, since you are already in   “setdisk / “then the first command is  
“cd home”   with no slash.

Worth a try, anyway…

Deb Baddorf
Fermilab