Re: filemark & HP LTO-2 drive
On Thu, Mar 03, 2011 at 08:22:21PM +0100, Gour wrote: > On Thu, 03 Mar 2011 10:48:29 -0500 > Jon LaBadie wrote: > > > 207KB for each filemark? > > > > Wow, what a huge waste!! > > Indeed. > I meant that with sarcasm, hope it came through. > > > > But consider the difference from a 0KB filemark to Amanda's > > calculations. If it were to write 1000 separate tape files, > > "wasting" 207MB in filemarks, its calculations of how much > > dump data it could fit on the tape would differ by 0.1%. > > That's right...Still, I wonder whether to try using 0kb for filemark, > run the test again or something else? I'd wager you wouldn't be able to differentiate using an LTO-2 tapetype definition using 0KB vs 207KB filemarks. Unless things have changed, amanda uses the filemark in calculations like: 1) during planning it might determine if your DLEs, at their scheduled backup levels, will fit on the allowed number of tapes. So unless you only allow 1 tape and each amdump run gets right up to the 200GB tape capacity, 0.000207 GB per tape file will have little impact on its plans. 2) while taping, the actual amount of data taped, plus each 32KB tape file header, plus each 207KB file mark, is tracked. When deciding which completed DLE dump (sitting on the hold- ing disk) should be taped next, the remaining tape capacity may be considered depending on the taper algorithim you have selected (ex the default FIFO vs Largest that Fits). But those considerations would only come into play if you have "runtapes" > 1 and are using a changer (even chg-manual). In my limited experience with various tape formats and tapetype, I found my LTO-1 drive to give virtually identical results each run across different individual tapes and 2 brands of tape. The filemark was always 0. In contrast, my tapetype results for DAT (DAT-2 and DAT-3) were a bit inconsistant and showed definite differences between brands. If my amtapetype results showed a significant filemark while others generally report zero filemark for LTO formats, I'd ponder, but wouldn't worry too much, why I got my results. I'd certainly rerun amtapetype. It not that big a hassle. You could have done that before asking and had your result before any replies to your query. I'd also look at the reported throughput. Amanda makes no use of the speed results; those are for your info only. I'd compare my observed results with other reports and with the drive manu- facturer's specified speed. If your system is unable to feed your drive data at a rate sufficient to keep it streaming, I'm pretty sure that apparent capacity and filemark can be affected. Jon -- Jon H. LaBadie j...@jgcomp.com JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
Re: Bacula --> Amanda migration
On Thu, 3 Mar 2011 09:48:22 -0700 John Hein wrote: > Amanda should not have problems. If you hit a snag, ask the list. Cool. > You may hit issues like the mt(1) syntax is different (use mt for > things like setting blocksize and disabling hardware compression > before amanda starts up). That's OK and we believe we will be able to find out our way... Sincerely, Gour -- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu) http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA signature.asc Description: PGP signature
Re: Bacula --> Amanda migration
On Thu, 03 Mar 2011 11:00:01 -0500 Chris Hoogendyk wrote: > I saw almost exactly the same question on the Bacula list recently, > with respect to migrating from netbackup to Bacula. The short answer > was that there is no way except recovering and then backing up with > the new backup program. :-( > If you had two tape drives and a decent sized holding disk, and you > wanted to get really creative, you could write a module that recovers > from Bacula tapes and streams directly to the Amanda backup. ;-) > But, that's *way* over the top. Much easier to just recover and back > up again. Neither I've two tape drives, not enough creativity to delve into such project. :-) > I've found that using amadmin to force full and then marking the > tapes as no-reuse works for archive and ends up keeping the same > configuration for DLEs, etc. Good. > If you want more than one copy, you can always run that again, or > just copy the tape if you have two drives. Bacula has Copy Job. What about Amanda's amvault? Can it serve the same purpose? Sincerely, Gour -- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu) http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA signature.asc Description: PGP signature
Re: filemark & HP LTO-2 drive
On Thu, 03 Mar 2011 10:48:29 -0500 Jon LaBadie wrote: > 207KB for each filemark? > > Wow, what a huge waste!! Indeed. > > But consider the difference from a 0KB filemark to Amanda's > calculations. If it were to write 1000 separate tape files, > "wasting" 207MB in filemarks, its calculations of how much > dump data it could fit on the tape would differ by 0.1%. That's right...Still, I wonder whether to try using 0kb for filemark, run the test again or something else? Sincerely, Gour -- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu) http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA signature.asc Description: PGP signature
Re: Bacula --> Amanda migration
Gour wrote at 16:23 +0100 on Mar 3, 2011: > Is there any concern when migrating Amanda from Linux to FreeBSD? Amanda should not have problems. If you hit a snag, ask the list. You may hit issues like the mt(1) syntax is different (use mt for things like setting blocksize and disabling hardware compression before amanda starts up).
Re: Bacula --> Amanda migration
I saw almost exactly the same question on the Bacula list recently, with respect to migrating from netbackup to Bacula. The short answer was that there is no way except recovering and then backing up with the new backup program. Of course, netbackup and Bacula both have their own unique formats that no one else reads. Bacula's is at least open, so . . . If you had two tape drives and a decent sized holding disk, and you wanted to get really creative, you could write a module that recovers from Bacula tapes and streams directly to the Amanda backup. ;-) But, that's *way* over the top. Much easier to just recover and back up again. I've found that using amadmin to force full and then marking the tapes as no-reuse works for archive and ends up keeping the same configuration for DLEs, etc. If you want more than one copy, you can always run that again, or just copy the tape if you have two drives. On 3/3/11 10:23 AM, Gour wrote: Hello! I've 8 LTO-2 tapes filled with raw video (DV) and 4 with raw high-resolution (35mm) scans stored in Bacula& catalog in Postgres database. Now, I'd like to put this data (~1.7TB of video& 700GBs of scans) on the tape backed up by Amanda. I'd also like to do this migration while my desktop machine is still running Linux (with ext4 fs) 'cause I plan to put Free(PC)BSD on my desktop with ZFS filesystem soon, in the attempt to avoid migrating Postgres database from one OS to the other. As far as I can see, there is no way to avoid restoring all this data and then do backup with Amanda. The problem is that I'm a bit tight on the HD - my desktop has 2x1TBs hard disks running in raid1 mirror, and two smaller external USB disks (320GB + 750GB), so the only solution (without buying more hardware) seems to be to remove one disk from raid1 array and used it along with the other disks to restore everything on those and then backup? Let me add that video files (8 tapes) are stored in separate catalog then the slides, iow. they use different pools of tapes. Any idea how to do this Bacula --> Amanda migration in a most simple way? Is there any concern when migrating Amanda from Linux to FreeBSD? Can you recommend some appropriate strategy to do this long-term archiving of video& slides (besides http://wiki.zmanda.com/index.php/FAQ:How_can_I_configure_Amanda_for_long-term_archiving%3F) in order to have it easy way when doing recovery/restore without having all the eggs in one basket? -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology& Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst --- Erdös 4
Re: filemark & HP LTO-2 drive
On Thu, Mar 03, 2011 at 02:13:00PM +0100, Gour wrote:
> Hello,
>
> after performing testing of my HP Ultrium 448 drive with:
>
> amtapetype -f /dev/nst0
>
> I got the following result:
>
> Checking for FSF_AFTER_FILEMARK requirement
> Applying heuristic check for compression.
> Wrote random (uncompressible) data at 21864712.2580645 bytes/sec
> Wrote fixed (compressible) data at 50207857.778 bytes/sec
> Compression: enabled
> Writing one file to fill the volume.
> Wrote 203994431488 bytes at 22522 kb/sec
> Got LEOM indication, so drive and kernel together support LEOM
> Writing smaller files (2039939072 bytes) to determine filemark.
> define tapetype unknown-tapetype {
> comment "Created by amtapetype; compression enabled"
> length 199213312 kbytes
> filemark 207 kbytes
> speed 22522 kps
> blocksize 32 kbytes
> }
> # for this drive and kernel, LEOM is supported; add
> # device-property "LEOM" "TRUE"
> # for this device.
>
>
> However, when comparing the above result with the entries at
> http://wiki.zmanda.com/index.php/Tapetype_definitions#Hewlett-Packard_Ultrium_448
> I see that all of 'em have: "filemark 0 kybtes", while I got "filemark
> 207 kbytes"
>
> What do you think, iow. shall I use the one arrived after amtapetype
> testing?
207KB for each filemark?
Wow, what a huge waste!!
But consider the difference from a 0KB filemark to Amanda's
calculations. If it were to write 1000 separate tape files,
"wasting" 207MB in filemarks, its calculations of how much
dump data it could fit on the tape would differ by 0.1%.
--
Jon H. LaBadie j...@jgcomp.com
JG Computing
12027 Creekbend Drive (703) 787-0884
Reston, VA 20194 (703) 787-0922 (fax)
Bacula --> Amanda migration
Hello! I've 8 LTO-2 tapes filled with raw video (DV) and 4 with raw high-resolution (35mm) scans stored in Bacula & catalog in Postgres database. Now, I'd like to put this data (~1.7TB of video & 700GBs of scans) on the tape backed up by Amanda. I'd also like to do this migration while my desktop machine is still running Linux (with ext4 fs) 'cause I plan to put Free(PC)BSD on my desktop with ZFS filesystem soon, in the attempt to avoid migrating Postgres database from one OS to the other. As far as I can see, there is no way to avoid restoring all this data and then do backup with Amanda. The problem is that I'm a bit tight on the HD - my desktop has 2x1TBs hard disks running in raid1 mirror, and two smaller external USB disks (320GB + 750GB), so the only solution (without buying more hardware) seems to be to remove one disk from raid1 array and used it along with the other disks to restore everything on those and then backup? Let me add that video files (8 tapes) are stored in separate catalog then the slides, iow. they use different pools of tapes. Any idea how to do this Bacula --> Amanda migration in a most simple way? Is there any concern when migrating Amanda from Linux to FreeBSD? Can you recommend some appropriate strategy to do this long-term archiving of video & slides (besides http://wiki.zmanda.com/index.php/FAQ:How_can_I_configure_Amanda_for_long-term_archiving%3F) in order to have it easy way when doing recovery/restore without having all the eggs in one basket? Sincerely, Gour -- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu) http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA signature.asc Description: PGP signature
amrecover fails
Running Ubuntu server 10.10 with Amanda 3.1.0 both new installs. Tried to
amrecover
for a test from a remote machine. The machines are connected with GbE. Here
is what
happened Any tips?
amrecover> extract
Extracting files using tape drive changer on host rmt170.rmtohio.com.
The following tapes are needed: Daily08
Restoring files into directory /usr/home/jkonecn
Continue [?/Y/n]?
Extracting files using tape drive changer on host rmt170.rmtohio.com.
Load tape Daily08 now
Continue [?/Y/n/s/t]?
timeout waiting for restore
increase READ_TIMEOUT in recover-src/extract_list.c if your tape is slow
amrecover: error reading header (Operation timed out), check amidxtaped.*.debug
on server
amidxtaped.*.debug is 162MB but here is the start of it if it helps...
Wed Mar 2 16:04:50 2011: amidxtaped: pid 9974 ruid 34 euid 34 version 3.1.0:
start at Wed Mar 2 16:04:50 2011
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << FEATURES=feff9ffeff7f
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << CONFIG=Daily
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << LABEL=Daily08:2
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << FSF=2
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << HEADER
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << DEVICE=changer
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << HOST=^r4p17$
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << DISK=^/usr$
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << DATESTAMP=20110302075132
Wed Mar 2 16:04:50 2011: amidxtaped: CTL << END
Wed Mar 2 16:04:50 2011: amidxtaped: pid 9974 ruid 34 euid 34 version 3.1.0:
rename at Wed Mar 2 16:04:50 2011
Wed Mar 2 16:04:50 2011: amidxtaped: ignoring specified DISK, as it may be
badly quoted
Warning: no log files found for tape Daily03 written 2011-03-01 13:29:28
Warning: no log files found for tape Daily02 written 2011-03-01 13:16:52
Warning: no log files found for tape Daily01 written 2011-03-01 13:11:48
Wed Mar 2 16:04:50 2011: amidxtaped: warning: Use of uninitialized value $part{"status"} in string eq at
/usr/lib/amanda/perl/Amanda/DB/Catalog.pm line 632.
Warning: no log files found for tape Daily03 written 2011-03-01 13:29:28
Warning: no log files found for tape Daily02 written 2011-03-01 13:16:52
Warning: no log files found for tape Daily01 written 2011-03-01 13:11:48
Wed Mar 2 16:04:51 2011: amidxtaped: warning: Use of uninitialized value $part{"status"} in string eq at
/usr/lib/amanda/perl/Amanda/DB/Catalog.pm line 632.
Wed Mar 2 16:04:53 2011: amidxtaped: warning: Use of uninitialized value $next_label in concatenation (.) or string at
/usr/lib/amanda/perl/Amanda/Recovery/Clerk.pm line 46
7.
Wed Mar 2 16:04:53 2011: amidxtaped: warning: Use of uninitialized value $label in concatenation (.) or string at
/usr/lib/amanda/perl/Amanda/Recovery/Scan.pm line 196.
Wed Mar 2 16:04:53 2011: amidxtaped: find_volume labeled ''
Wed Mar 2 16:04:54 2011: amidxtaped: warning: Use of uninitialized value $label in string eq at
/usr/lib/amanda/perl/Amanda/Recovery/Scan.pm line 259.
Wed Mar 2 16:04:54 2011: amidxtaped: warning: Use of uninitialized value $label in string eq at
/usr/lib/amanda/perl/Amanda/Recovery/Scan.pm line 259.
Wed Mar 2 16:04:54 2011: amidxtaped: warning: Use of uninitialized value $label in string eq at
/usr/lib/amanda/perl/Amanda/Recovery/Scan.pm line 259.
Wed Mar 2 16:04:54 2011: amidxtaped: warning: Use of uninitialized value $label in string eq at
/usr/lib/amanda/perl/Amanda/Recovery/Scan.pm line 259.
filemark & HP LTO-2 drive
Hello,
after performing testing of my HP Ultrium 448 drive with:
amtapetype -f /dev/nst0
I got the following result:
Checking for FSF_AFTER_FILEMARK requirement
Applying heuristic check for compression.
Wrote random (uncompressible) data at 21864712.2580645 bytes/sec
Wrote fixed (compressible) data at 50207857.778 bytes/sec
Compression: enabled
Writing one file to fill the volume.
Wrote 203994431488 bytes at 22522 kb/sec
Got LEOM indication, so drive and kernel together support LEOM
Writing smaller files (2039939072 bytes) to determine filemark.
define tapetype unknown-tapetype {
comment "Created by amtapetype; compression enabled"
length 199213312 kbytes
filemark 207 kbytes
speed 22522 kps
blocksize 32 kbytes
}
# for this drive and kernel, LEOM is supported; add
# device-property "LEOM" "TRUE"
# for this device.
However, when comparing the above result with the entries at
http://wiki.zmanda.com/index.php/Tapetype_definitions#Hewlett-Packard_Ultrium_448
I see that all of 'em have: "filemark 0 kybtes", while I got "filemark
207 kbytes"
What do you think, iow. shall I use the one arrived after amtapetype
testing?
Sincerely,
Gour
--
“In the material world, conceptions of good and bad are
all mental speculations…” (Sri Caitanya Mahaprabhu)
http://atmarama.net | Hlapicina (Croatia) | GPG: CDBF17CA
signature.asc
Description: PGP signature
