Re: Confused by Amanda

[Gene Heskett]

On Monday 03 September 2018 01:24:35 Mark Fletcher wrote:

> On Sun, Sep 02, 2018 at 11:46:44AM -0400, Gene Heskett wrote:
> > On Sunday 02 September 2018 06:27:01 Dan Ritter wrote:
> > > Amanda is not good for the situation you describe.
> >
> > No its not ideal in some cases,, which is why I wrote a wrapper
> > script for the make a backup portions of amanda. With the resources
> > and configs that existed at the time that backup was made actually
> > appended to the end of the vtape, pretty much an empty drive
> > recovery is possible. It appends the /usr/local/etc/amana/$config,
> > and /usr/local/var/amanda/record_of_backups to each vtape it makes.
> > So those 2 files can be recovered with tar and gzip, put back on a
> > freshly installed linux of your favorite flavor, and a restore made
> > that will be a duplicate of what your had last night when backup.sh
> > was ran.
>
> Thanks Gene, I was hoping you would pipe up but didn't want to throw
> the spotlight on you if you weren't inclined to. This is exactly what
> I'm after so I will definitely check it out.
>
> Thanks also to Dan and Jose, I can see what you mean and it makes much
> of the Amanda documentation make more sense now. But as I mentioned,
> my configuration currently isn't an end state and I'm planning to
> expand it to cover other machines on my network, at which point Amanda
> will make more sense. I get the concept of two Amandas, one to backup
> the Amanda server of the first, but then you're into a "turtles all
> the way down" scenario, aren't you? Just seems overkill when one
> Amanda can look after its own server as well, albeit with some
> jiggerypokery which Gene has kindly cast light on.
>
> So I think we can agree, Amanda's expected usage model is ideally for
> situations where there are multiple machines to back up, you designate
> one machine the Amanda server (presumably the one with the easiest /
> fastest access to the backup media) and accept that that machine needs
> special, usually separate, arrangements for _its_ backup. But it's
> _possible_ with attention to the right details such as things Gene has
> pointed out, to include the Amanda server machine itself in the
> backup.
>
> Thanks all, especially Gene for, I suspect, saving me a lot of work.
>
> Mark
I don't have anything special in terms of backup up this server machine, 
its just another set of entries in the disklist. But I'm failing at 
makeing disklist entries that are reasonable sized, which defeats 
amanda's scheduleing to try and use the same amount of backup media 
every night. So I recently separated one of the subdirs in my /home dir 
that has over 50 GB in it, and now I need to subdivide that even further 
to distribute the load better yet. So as not to disturb amanda greatly 
I'll break out one of those subdirs in another 3 or 4 days, which means 
that subdir gets an entry in the excludes file for that dumptype, and a 
new dumptype created just for that subdir, breaking out the biggest one 
remaining each time until I get all disklist entries down to not more 
than 10 to 15 GB. Biggest problem is install iso's as I keep all of them 
for all 3 architectures here on this machine, where the best dvd burner 
lives.

After the next install on this machine, several x86 iso's can go away. 
But thats future plans. This morning, after I feed the missus, and put a 
can of R134 in my pickups AC which finally gave up yesterday, its peaked 
at 97 on my thermometers, then its up to WHAW, our local radio station 
and see if I can find the intermittent loss of 70% of the oscillator 
assemblies output thats putting him back on his 50 watt night time 
transmitter.

And the owner is afraid of mods, which are unavoidable because the OEM 
parts are not available. Its a 1959 Gates BC1T. 59 years old. I've 
finally convinced him its on borrowed time if for no other reason that 
every time he buys a new set of 833 final tubes, thats 4 less of them 
that exist on the planet and whats he gonna do when he calls up for 
another set and there aren't any left, even the Chinese have shut down 
that production line years ago.  Hell, its time he retired anyway.

BTDT, used all of the planets remaining 4-1000's for the tv transmitter 
at WDTV when I was the CE there for the last 19 years of my working 
life. I should have built a new box for the pair of 4-1000's it used, 
but with a single 4CX3000 tube, a 30 year newer design I could still get 
but was too busy with other stuff at the studio to really attack that 
project. Now its finally turned off for good with the digital conversion 
at midnight, June 30, 2008. It was good while it lasted, that 
transmitter was built in 1953. So it was 55 yo when shut down for the 
last time.

Gotta get to it, coffee s/b ready. :)

-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: Confused by Amanda

[Mark Fletcher]

On Sun, Sep 02, 2018 at 11:46:44AM -0400, Gene Heskett wrote:
> On Sunday 02 September 2018 06:27:01 Dan Ritter wrote:
> 
> > Amanda is not good for the situation you describe.
> 
> No its not ideal in some cases,, which is why I wrote a wrapper script 
> for the make a backup portions of amanda. With the resources and configs 
> that existed at the time that backup was made actually appended to the 
> end of the vtape, pretty much an empty drive recovery is possible. It 
> appends the /usr/local/etc/amana/$config, 
> and /usr/local/var/amanda/record_of_backups to each vtape it makes. So 
> those 2 files can be recovered with tar and gzip, put back on a freshly 
> installed linux of your favorite flavor, and a restore made that will be 
> a duplicate of what your had last night when backup.sh was ran.
> 

Thanks Gene, I was hoping you would pipe up but didn't want to throw the 
spotlight on you if you weren't inclined to. This is exactly what I'm 
after so I will definitely check it out.

Thanks also to Dan and Jose, I can see what you mean and it makes much 
of the Amanda documentation make more sense now. But as I mentioned, my 
configuration currently isn't an end state and I'm planning to expand it 
to cover other machines on my network, at which point Amanda will make 
more sense. I get the concept of two Amandas, one to backup the Amanda 
server of the first, but then you're into a "turtles all the way down" 
scenario, aren't you? Just seems overkill when one Amanda can look after 
its own server as well, albeit with some jiggerypokery which Gene has 
kindly cast light on.

So I think we can agree, Amanda's expected usage model is ideally for 
situations where there are multiple machines to back up, you designate 
one machine the Amanda server (presumably the one with the easiest / 
fastest access to the backup media) and accept that that machine needs 
special, usually separate, arrangements for _its_ backup. But it's 
_possible_ with attention to the right details such as things Gene has 
pointed out, to include the Amanda server machine itself in the backup.

Thanks all, especially Gene for, I suspect, saving me a lot of work.

Mark

Re: Confused by Amanda

[Jose M Calhariz]

On Sun, Sep 02, 2018 at 06:27:01AM -0400, Dan Ritter wrote:
> On Sun, Sep 02, 2018 at 04:09:22PM +0900, Mark Fletcher wrote:
> > I recently messed up some files and decided to resort to the backup to 
> > recover them. I was able to do so, but the process left me wondering if 
> > I would really be in a position to do so in all cases. For example, 
> > Amanda configuration is in /etc/amanda -- what if /etc was what I needed 
> > to restore? Similarly, I gather there are files under /var/lib/amanda -- 
> > what happens if /var is damaged?
> > 
> > I have not been able to understand from the Amanda documentation really 
> > all that I need to have in place to be able to expect to recover from, 
> > say, a disk replacement after catastrophic failure. I'm imagining, main 
> > disk goes to data heaven, I buy a new one, install Stretch again fresh, 
> > and now I want to re-install packages and restore their backed-up 
> > configuration as well as restore my data in /home etc. I know there are 
> > a few experienced users of Amanda on this list -- can anyone help me, or 
> > perhaps point me to a good resource that explains it, or even if there's 
> > a section in the documentation I've missed that makes it clear?
> > 
> > I guess a key point is, in my configuration, the same machine is both 
> > Amanda server and Amanda client. I guess I may expand this in the future 
> > to have this machine manage backups for other machines, but at the 
> > moment that is not happening. Of course, the disk that houses the Amanda 
> > virtual tapes is off-machine. 
> > 
> > What I'm looking for is along the lines of "your nightly backup routine 
> > needs to be run amdump, then rsync this, this and this directory 
> > somewhere safe" or whatever it is. Or alternatively "don't be an idiot 
> > you don't need to do any of that, amanda is magic in this, this and this 
> > way".
> 
> Amanda is not good for the situation you describe.

I can confirm that.  I manage 3 installations of amanda server and in
2 of them I have a secondary backup system of the server using
storeBackup.  My home installation of amanda is a machine with 3 disks
of 4TB.  The first have all the system data and my personal files, the
others 2 disks contains the vTapes and the backup files of
storeBackup.  With this setup I can survive the loss of a single
disk.

Being the 3 disks always online, I am not protected against a virus
or a sysadmin error that can wipe the data in the 3 disks with one
single command.

> 
> Tell us your scenario, and we can offer better advice.

Indeed.

> 
> -dsr-
> 
> 

Kind regards
Jose M Calhariz


-- 
--
Em todo casal há pelo menos um tolo.
-- Henry Fielding


signature.asc
Description: PGP signature

Re: Confused by Amanda

[Gene Heskett]

On Sunday 02 September 2018 06:27:01 Dan Ritter wrote:

> On Sun, Sep 02, 2018 at 04:09:22PM +0900, Mark Fletcher wrote:
> > I recently messed up some files and decided to resort to the backup
> > to recover them. I was able to do so, but the process left me
> > wondering if I would really be in a position to do so in all cases.
> > For example, Amanda configuration is in /etc/amanda -- what if /etc
> > was what I needed to restore? Similarly, I gather there are files
> > under /var/lib/amanda -- what happens if /var is damaged?
> >
> > I have not been able to understand from the Amanda documentation
> > really all that I need to have in place to be able to expect to
> > recover from, say, a disk replacement after catastrophic failure.
> > I'm imagining, main disk goes to data heaven, I buy a new one,
> > install Stretch again fresh, and now I want to re-install packages
> > and restore their backed-up configuration as well as restore my data
> > in /home etc. I know there are a few experienced users of Amanda on
> > this list -- can anyone help me, or perhaps point me to a good
> > resource that explains it, or even if there's a section in the
> > documentation I've missed that makes it clear?
> >
> > I guess a key point is, in my configuration, the same machine is
> > both Amanda server and Amanda client. I guess I may expand this in
> > the future to have this machine manage backups for other machines,
> > but at the moment that is not happening. Of course, the disk that
> > houses the Amanda virtual tapes is off-machine.
> >
> > What I'm looking for is along the lines of "your nightly backup
> > routine needs to be run amdump, then rsync this, this and this
> > directory somewhere safe" or whatever it is. Or alternatively "don't
> > be an idiot you don't need to do any of that, amanda is magic in
> > this, this and this way".
>
> Amanda is not good for the situation you describe.

No its not ideal in some cases,, which is why I wrote a wrapper script 
for the make a backup portions of amanda. With the resources and configs 
that existed at the time that backup was made actually appended to the 
end of the vtape, pretty much an empty drive recovery is possible. It 
appends the /usr/local/etc/amana/$config, 
and /usr/local/var/amanda/record_of_backups to each vtape it makes. So 
those 2 files can be recovered with tar and gzip, put back on a freshly 
installed linux of your favorite flavor, and a restore made that will be 
a duplicate of what your had last night when backup.sh was ran.

Like all such scripts, its subject to being modified the instant it's 
been published, and likely as not, won't fit your situation. But the 
outline is there, and fairly well commented, and should serve as a 
guildline to customize it for your own use. Look on my web site if 
interested, and can hack this old copy to do this for you, see my 
website in the sig, adding Genes-os9-stf to the path, and its the 1st 
two files at the top of the list. I am currently backing up this machine 
and 4 of my cnc machines in the garage and an outbuilding with todays 
version of it. To vtapes on a separate 2T drive. All in one session 
every night in the wee hours.
>
> Amanda expects that the server is doing backups for a number of
> clients, and that the server itself is backed up from either
> a different Amanda server or via a different method.
>
> If you say what your actual configuration is, we can recommend
> more appropriate systems. For example:
>
> Let's say you have a standalone system with a 4TB hard disk for
> /home  and a 250GB SSD which is root and /var. Right now df -h
> says you have used 27GB in /, 8GB in /var, and 1.2TB in /home.
> You don't expect a single massive data usage change, so much as
> slow growth over the course of the next few years.
>
> In that scenario, I would first suggest buying a 3-5TB disk and
> putting it in an external case - eSATA is excellent, USB3 is pretty
> good. Mount it as /backup and use rsnapshot to make ongoing
> backups via the standard cron job it will set up. An rsnapshot
> backup is entirely filesystem-visible, so when you try to
> recover a few files (because you accidentally deleted or
> overwrote them) you can just copy them back in to place; when
> you need to restore to new disks, you can install a minimal
> system then copy over on top of it.
>
> Tell us your scenario, and we can offer better advice.
>
> -dsr-



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: Confused by Amanda

[Dan Ritter]

On Sun, Sep 02, 2018 at 04:09:22PM +0900, Mark Fletcher wrote:
> I recently messed up some files and decided to resort to the backup to 
> recover them. I was able to do so, but the process left me wondering if 
> I would really be in a position to do so in all cases. For example, 
> Amanda configuration is in /etc/amanda -- what if /etc was what I needed 
> to restore? Similarly, I gather there are files under /var/lib/amanda -- 
> what happens if /var is damaged?
> 
> I have not been able to understand from the Amanda documentation really 
> all that I need to have in place to be able to expect to recover from, 
> say, a disk replacement after catastrophic failure. I'm imagining, main 
> disk goes to data heaven, I buy a new one, install Stretch again fresh, 
> and now I want to re-install packages and restore their backed-up 
> configuration as well as restore my data in /home etc. I know there are 
> a few experienced users of Amanda on this list -- can anyone help me, or 
> perhaps point me to a good resource that explains it, or even if there's 
> a section in the documentation I've missed that makes it clear?
> 
> I guess a key point is, in my configuration, the same machine is both 
> Amanda server and Amanda client. I guess I may expand this in the future 
> to have this machine manage backups for other machines, but at the 
> moment that is not happening. Of course, the disk that houses the Amanda 
> virtual tapes is off-machine. 
> 
> What I'm looking for is along the lines of "your nightly backup routine 
> needs to be run amdump, then rsync this, this and this directory 
> somewhere safe" or whatever it is. Or alternatively "don't be an idiot 
> you don't need to do any of that, amanda is magic in this, this and this 
> way".

Amanda is not good for the situation you describe.

Amanda expects that the server is doing backups for a number of
clients, and that the server itself is backed up from either
a different Amanda server or via a different method.

If you say what your actual configuration is, we can recommend
more appropriate systems. For example:

Let's say you have a standalone system with a 4TB hard disk for
/home  and a 250GB SSD which is root and /var. Right now df -h
says you have used 27GB in /, 8GB in /var, and 1.2TB in /home.
You don't expect a single massive data usage change, so much as
slow growth over the course of the next few years.

In that scenario, I would first suggest buying a 3-5TB disk and putting
it in an external case - eSATA is excellent, USB3 is pretty
good. Mount it as /backup and use rsnapshot to make ongoing
backups via the standard cron job it will set up. An rsnapshot
backup is entirely filesystem-visible, so when you try to
recover a few files (because you accidentally deleted or
overwrote them) you can just copy them back in to place; when
you need to restore to new disks, you can install a minimal
system then copy over on top of it.

Tell us your scenario, and we can offer better advice.

-dsr-

Confused by Amanda

[Mark Fletcher]

Hello

I use Amanda for daily backups on Stretch. I found it not too difficult 
to set up once I got my head around its virtual tape concept.

Recently, prompted by not very much, I have started to question whether 
having these backups really put me in a position to restore the machine 
if I need to.

I recently messed up some files and decided to resort to the backup to 
recover them. I was able to do so, but the process left me wondering if 
I would really be in a position to do so in all cases. For example, 
Amanda configuration is in /etc/amanda -- what if /etc was what I needed 
to restore? Similarly, I gather there are files under /var/lib/amanda -- 
what happens if /var is damaged?

I have not been able to understand from the Amanda documentation really 
all that I need to have in place to be able to expect to recover from, 
say, a disk replacement after catastrophic failure. I'm imagining, main 
disk goes to data heaven, I buy a new one, install Stretch again fresh, 
and now I want to re-install packages and restore their backed-up 
configuration as well as restore my data in /home etc. I know there are 
a few experienced users of Amanda on this list -- can anyone help me, or 
perhaps point me to a good resource that explains it, or even if there's 
a section in the documentation I've missed that makes it clear?

I guess a key point is, in my configuration, the same machine is both 
Amanda server and Amanda client. I guess I may expand this in the future 
to have this machine manage backups for other machines, but at the 
moment that is not happening. Of course, the disk that houses the Amanda 
virtual tapes is off-machine. 

What I'm looking for is along the lines of "your nightly backup routine 
needs to be run amdump, then rsync this, this and this directory 
somewhere safe" or whatever it is. Or alternatively "don't be an idiot 
you don't need to do any of that, amanda is magic in this, this and this 
way".

Thanks

Mark