Potential user - more questions
Hi again, I've got another batch of questions while I consider switching to Amanda: 1) catalog (indecies) It seems the main catalog/database is stored in the index files. Is it straightforward to back these up? This doc (http://www.zmanda.com/protecting-amanda-server.html) sugests backing up these dirs/files to be able to restore an amanda configuration (and presumably the backup catalog): /etc/amandates, /etc/dumpdates, /etc/amanda, /var/lib/amanda. 2) Spanning and parts Say I split my 32TB of data into DLE's of 2-3TB. a) If I set a 'part' size of 150GB (10% of native tape capacity is what I saw recommended), what is the format of each part as it's written? Is each part its own tarfile? Seems that would make it easier to restore things manually. b) If a part spans two volumes, what's the format of that? Is it a single tarfile that's split in two? c) What's the manual restore process for such a spanned part? cat the two parts together and pipe to tar for extraction? 3) Restoring w/out Amanda I thought data was written to tape as tar files. But this page suggests a dumpfile is only readable by Amanda apps. Is a dumpfile something else? http://wiki.zmanda.com/index.php/Dumpfile 4) holding disk and flushing I see how flushing can be forced when the holding disk has a certain % of tape size. Can a flush be forced every N days? The idea here would be to get data to tape at a min of every week or so, should successive incrementals be small. 5) alerting Is there a provision for email and/or other alerts on job completion or error, etc? Thanks! -M
Re: Potential user - more questions
Michael, > 1) catalog (indecies) > It seems the main catalog/database is stored in the index files. Is it > straightforward to back these up? > This doc (http://www.zmanda.com/protecting-amanda-server.html) sugests > backing up these dirs/files to be able to restore an amanda > configuration (and presumably the backup catalog): /etc/amandates, > /etc/dumpdates, /etc/amanda, /var/lib/amanda. Many years ago, I wrote a Perl script that will rsync that to a different machine an email me the contents of amadmin export. Knowing that the mail is automatically being replicated on a mail backup machine (and forwarded to my gmail account). I am pretty sure the information is safe. > 2) Spanning and parts > Say I split my 32TB of data into DLE's of 2-3TB. > > a) If I set a 'part' size of 150GB (10% of native tape capacity is > what I saw recommended), what is the format of each part as it's > written? Is each part its own tarfile? Seems that would make it easier > to restore things manually. Nothing is written as a tar file, but as an Amanda backup file (there is a few bytes header before the tar file). I am not sure about splited backup. > b) If a part spans two volumes, what's the format of that? Is it a > single tarfile that's split in two? > > c) What's the manual restore process for such a spanned part? cat the > two parts together and pipe to tar for extraction? > > 3) Restoring w/out Amanda > I thought data was written to tape as tar files. But this page > suggests a dumpfile is only readable by Amanda apps. Is a dumpfile > something else? > http://wiki.zmanda.com/index.php/Dumpfile For a single file DLE, you have to use dd to remove the few bytes header and you are left with a pure tar/dump file. > 4) holding disk and flushing > I see how flushing can be forced when the holding disk has a certain % > of tape size. > Can a flush be forced every N days? The idea here would be to get data > to tape at a min of every week or so, should successive incrementals > be small. > > 5) alerting > Is there a provision for email and/or other alerts on job completion > or error, etc? Yes, Amanda sends nice summary emails at the end of a dump. Best regards, Olivier > Thanks! > > -M
Re: Potential user - more questions
On Wed, Oct 09, 2013 at 06:27:48PM -0400, Michael Stauffer wrote: > Hi again, > > I've got another batch of questions while I consider switching to Amanda: > > 1) catalog (indecies) > It seems the main catalog/database is stored in the index files. Is it > straightforward to back these up? > This doc (http://www.zmanda.com/protecting-amanda-server.html) sugests > backing up these dirs/files to be able to restore an amanda > configuration (and presumably the backup catalog): /etc/amandates, > /etc/dumpdates, /etc/amanda, /var/lib/amanda. There is no built-in way to do this in amanda. The problems are they are not complete, and changing, until the backup is done. Several members of this list have described their home-grown techniques. > > 2) Spanning and parts > Say I split my 32TB of data into DLE's of 2-3TB. > > a) If I set a 'part' size of 150GB (10% of native tape capacity is > what I saw recommended), what is the format of each part as it's > written? Is each part its own tarfile? Seems that would make it easier > to restore things manually. Traditional amanda tape files, holding the complete tar or dump archive, are a 32KB header followed by the archive. Manual restoration is done with dd to skip the header and pipe the rest to the appropriate command line to restore the data. The header contains information identifying the contents, how they were created, and when. Parts alter this scheme only slightly. Each part still has a header. The header now includes info on which sequential part it is. The part name also identifies it location in the sequence. The data is simply a chunk of the complete archive. Manual restoration again is strip the headers and pipe to the restore command. > > b) If a part spans two volumes, what's the format of that? Is it a > single tarfile that's split in two? A part will NOT span two volumes. If the end of the media is reached, the part is restarted on the next volume. > > c) What's the manual restore process for such a spanned part? cat the > two parts together and pipe to tar for extraction? > > 3) Restoring w/out Amanda > I thought data was written to tape as tar files. But this page > suggests a dumpfile is only readable by Amanda apps. Is a dumpfile > something else? > http://wiki.zmanda.com/index.php/Dumpfile I think the author meant there are no "standard unix/linux" commands that know the header + data layout. The dumpfiles can be handled with amanda commands or as described above, the operator can use standard commands when armed with knowledge of the layout. > > 4) holding disk and flushing > I see how flushing can be forced when the holding disk has a certain % > of tape size. > Can a flush be forced every N days? The idea here would be to get data > to tape at a min of every week or so, should successive incrementals > be small. Dumping to holding disk without taping can be done. Then have a crontable entry to flush when you want. This can done with a separate amflush command, or by varying amdump options. > > 5) alerting > Is there a provision for email and/or other alerts on job completion > or error, etc? > Most amanda admins have an amreport emailed to them at amdump or amflush completion. As the cron entry can be a shell script, you could customize greatly. Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (609) 477-8330 (C)
Re: Potential user - more questions
Olivier and Jon, thanks for the helpful answers. I'm going to setup my redeployed backup system with Amanda. It seems enough easier than Bacula to make it worth while to make the switch, and I especially like the simple format of the dump files and the simple text indecies for cataloging backups. I'm sure you'll hear from me more while I get things going! -M On Thu, Oct 10, 2013 at 12:45 AM, Jon LaBadie wrote: > On Wed, Oct 09, 2013 at 06:27:48PM -0400, Michael Stauffer wrote: > > Hi again, > > > > I've got another batch of questions while I consider switching to Amanda: > > > > 1) catalog (indecies) > > It seems the main catalog/database is stored in the index files. Is it > > straightforward to back these up? > > This doc (http://www.zmanda.com/protecting-amanda-server.html) sugests > > backing up these dirs/files to be able to restore an amanda > > configuration (and presumably the backup catalog): /etc/amandates, > > /etc/dumpdates, /etc/amanda, /var/lib/amanda. > > There is no built-in way to do this in amanda. The problems are they > are not complete, and changing, until the backup is done. Several > members of this list have described their home-grown techniques. > > > > 2) Spanning and parts > > Say I split my 32TB of data into DLE's of 2-3TB. > > > > a) If I set a 'part' size of 150GB (10% of native tape capacity is > > what I saw recommended), what is the format of each part as it's > > written? Is each part its own tarfile? Seems that would make it easier > > to restore things manually. > > Traditional amanda tape files, holding the complete tar or dump archive, > are a 32KB header followed by the archive. Manual restoration is done > with dd to skip the header and pipe the rest to the appropriate command > line to restore the data. > > The header contains information identifying the contents, how they > were created, and when. > > Parts alter this scheme only slightly. Each part still has a header. > The header now includes info on which sequential part it is. The part > name also identifies it location in the sequence. The data is simply > a chunk of the complete archive. Manual restoration again is strip > the headers and pipe to the restore command. > > > > > b) If a part spans two volumes, what's the format of that? Is it a > > single tarfile that's split in two? > > A part will NOT span two volumes. If the end of the media is reached, > the part is restarted on the next volume. > > > > > c) What's the manual restore process for such a spanned part? cat the > > two parts together and pipe to tar for extraction? > > > > 3) Restoring w/out Amanda > > I thought data was written to tape as tar files. But this page > > suggests a dumpfile is only readable by Amanda apps. Is a dumpfile > > something else? > > http://wiki.zmanda.com/index.php/Dumpfile > > I think the author meant there are no "standard unix/linux" commands > that know the header + data layout. The dumpfiles can be handled > with amanda commands or as described above, the operator can use > standard commands when armed with knowledge of the layout. > > > > > 4) holding disk and flushing > > I see how flushing can be forced when the holding disk has a certain % > > of tape size. > > Can a flush be forced every N days? The idea here would be to get data > > to tape at a min of every week or so, should successive incrementals > > be small. > > Dumping to holding disk without taping can be done. Then have a > crontable entry to flush when you want. This can done with a > separate amflush command, or by varying amdump options. > > > > 5) alerting > > Is there a provision for email and/or other alerts on job completion > > or error, etc? > > > Most amanda admins have an amreport emailed to them at amdump or amflush > completion. As the cron entry can be a shell script, you could > customize greatly. > > Jon > -- > Jon H. LaBadie j...@jgcomp.com > 11226 South Shore Rd. (703) 787-0688 (H) > Reston, VA 20190 (609) 477-8330 (C) >
Re: Potential user - more questions
Michael, You may have noticed my answer to question 2a was not complete. I answered for a not parted DLE. On a second thought, as each part is exactly the size you configure (exact to the byte), I assume that each file cannot be an independent tar, so the DLE must be tar'ed into a big file first, that is then cut into chunks. I never had to manually restore a parted DLE (I would manually restore / and maybe /usr, that would give me a running Amanda and then use Amanda to restore the data; my / or /usr are smaller that the size of parts); manually restoring a DLE is easy, even with a live-cd, you really only need dd and tar (or dump/restore). It takes some precautions, but is pretty doable. Best regards, Olivier On Fri, Oct 11, 2013 at 4:16 AM, Michael Stauffer wrote: > Olivier and Jon, thanks for the helpful answers. > I'm going to setup my redeployed backup system with Amanda. It seems enough > easier than Bacula to make it worth while to make the switch, and I > especially like the simple format of the dump files and the simple text > indecies for cataloging backups. > > I'm sure you'll hear from me more while I get things going! > > -M > > > On Thu, Oct 10, 2013 at 12:45 AM, Jon LaBadie wrote: >> >> On Wed, Oct 09, 2013 at 06:27:48PM -0400, Michael Stauffer wrote: >> > Hi again, >> > >> > I've got another batch of questions while I consider switching to >> > Amanda: >> > >> > 1) catalog (indecies) >> > It seems the main catalog/database is stored in the index files. Is it >> > straightforward to back these up? >> > This doc (http://www.zmanda.com/protecting-amanda-server.html) sugests >> > backing up these dirs/files to be able to restore an amanda >> > configuration (and presumably the backup catalog): /etc/amandates, >> > /etc/dumpdates, /etc/amanda, /var/lib/amanda. >> >> There is no built-in way to do this in amanda. The problems are they >> are not complete, and changing, until the backup is done. Several >> members of this list have described their home-grown techniques. >> > >> > 2) Spanning and parts >> > Say I split my 32TB of data into DLE's of 2-3TB. >> > >> > a) If I set a 'part' size of 150GB (10% of native tape capacity is >> > what I saw recommended), what is the format of each part as it's >> > written? Is each part its own tarfile? Seems that would make it easier >> > to restore things manually. >> >> Traditional amanda tape files, holding the complete tar or dump archive, >> are a 32KB header followed by the archive. Manual restoration is done >> with dd to skip the header and pipe the rest to the appropriate command >> line to restore the data. >> >> The header contains information identifying the contents, how they >> were created, and when. >> >> Parts alter this scheme only slightly. Each part still has a header. >> The header now includes info on which sequential part it is. The part >> name also identifies it location in the sequence. The data is simply >> a chunk of the complete archive. Manual restoration again is strip >> the headers and pipe to the restore command. >> >> > >> > b) If a part spans two volumes, what's the format of that? Is it a >> > single tarfile that's split in two? >> >> A part will NOT span two volumes. If the end of the media is reached, >> the part is restarted on the next volume. >> >> > >> > c) What's the manual restore process for such a spanned part? cat the >> > two parts together and pipe to tar for extraction? >> > >> > 3) Restoring w/out Amanda >> > I thought data was written to tape as tar files. But this page >> > suggests a dumpfile is only readable by Amanda apps. Is a dumpfile >> > something else? >> > http://wiki.zmanda.com/index.php/Dumpfile >> >> I think the author meant there are no "standard unix/linux" commands >> that know the header + data layout. The dumpfiles can be handled >> with amanda commands or as described above, the operator can use >> standard commands when armed with knowledge of the layout. >> >> > >> > 4) holding disk and flushing >> > I see how flushing can be forced when the holding disk has a certain % >> > of tape size. >> > Can a flush be forced every N days? The idea here would be to get data >> > to tape at a min of every week or so, should successive incrementals >> > be small. >> >> Dumping to holding disk without taping can be done. Then have a >> crontable entry to flush when you want. This can done with a >> separate amflush command, or by varying amdump options. >> > >> > 5) alerting >> > Is there a provision for email and/or other alerts on job completion >> > or error, etc? >> > >> Most amanda admins have an amreport emailed to them at amdump or amflush >> completion. As the cron entry can be a shell script, you could >> customize greatly. >> >> Jon >> -- >> Jon H. LaBadie j...@jgcomp.com >> 11226 South Shore Rd. (703) 787-0688 (H) >> Reston, VA 20190 (609) 477-8330 (C) > >
