Re: [Bacula-users] Travan tape - very slow
On Mon, 27 Nov 2006, Peter Crighton wrote: I did read that and yes I understand it says don't use 512 byte blocks, but that is the only way that I've managed to get it to work. http://www.arkeia.com/archives_indexed/2003/08/msg00167.html makes comments about the extreme slowness of the drives under Arkeia, so this isn't a unique problem. Did you use a blank tape when you tested bacula for the first time? More importantly, was btape used? Yes (and before I'd waited for that to finish (I wanted to sleep and can't with a tape drive running in the next room) cancelled it and tried a Bacula backup with the same throughput) IDE-based tape drives have always difficult... And of course I have an IDE drive... This is due to ide-scsi emulation. You really need to run btape and get solid answers. Have you thought about starting btape in the morning and lettng it run all day? Ideally you can ssh into the machine remotely... - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Crighton wrote: On Fri, 24 Nov 2006, John Drescher wrote: IDE-based tape drives have always difficult... And of course I have an IDE drive... Hello, Chiming in late here (and I admit that I didn't read the entire conversation), but have you tried the setup for OnStream drives mentioned in the manual? http://www.bacula.org/rel-manual/Testing_Your_Tape_Drive.html#SECTION000373900 I once had to work with those IDE drives and they had the same issue with not accepting variable block sizes, so I had to set them to fixed block size as described in that section. (The other stuff in the SD definition was basically found through trial and error, and nagging KErn here on the list ;) and it may or may not apply to your drive.) Greetings, Michel -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (MingW32) - GPGrelay v0.959 iD8DBQFFbCcW2Vs+MkscAyURAkNBAKC0DOxs/yJsTu5N8WvLDG0nQFmIywCeP4We kni5aK7MKP2n+gUKC0MbUwE= =NRCA -END PGP SIGNATURE- - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Tue, 28 Nov 2006 10:55:15 + (GMT), you wrote: You really need to run btape and get solid answers. Have you thought about starting btape in the morning and lettng it run all day? I did - it managed about 3GB (of 10 uncompressed) before I went to bed from 7am to 11pm or so. -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Fri, 24 Nov 2006, John Drescher wrote: I did read that and yes I understand it says don't use 512 byte blocks, but that is the only way that I've managed to get it to work. Did you use a blank tape when you tested bacula for the first time? More importantly, was btape used? IIRC this attempts to discover the optimum fixed block setting, should this mode be necessary IDE-based tape drives have always difficult... AB - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Sat, 25 Nov 2006, Peter Crighton wrote: What are the best alternative type of drive (it's for a home office so only requires a modest capability) Define modest With many home computers now holding up to 1Tb of local disk, that's an awful lot of DATs to be stuffing into the drive for one full backup. Looking on Ebay DDS-3/DDS-4 drives and tapes are available at a reasonable cost, giving a comparable capacity to my current drive and probably almost a no-cost swap if I sell my existing drive and tapes on Ebay. Avoid 4mm tape systems - they are prone to failure and in my experience are not reliable for more than short-term storage This is no reflection on the technology, the basic problem is the tape base polymer itself. It's just not wide enough to cope well with flaws. 8mm helical scan systems (AIT, etc) are fine. There's still only one applicable maxim for tape backup systems : Good, fast, cheap: pick any 2 - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Mon, 27 Nov 2006 12:15:19 + (GMT), you wrote: On Sat, 25 Nov 2006, Peter Crighton wrote: What are the best alternative type of drive (it's for a home office so only requires a modest capability) Define modest Circa 20GB (yes I have more data but don't back up ripped CDs) and some data (photos) is held on two computers so not backed up to tape. I will also have RAID as soon as I rebuild my server so there'll be two copies of everything important plus a tape backup of the most important. I don't think 1TB is the norm for home use (yet - but I can see it going that way in the future with more photos). -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Mon, 27 Nov 2006 12:07:59 + (GMT), you wrote: On Fri, 24 Nov 2006, John Drescher wrote: I did read that and yes I understand it says don't use 512 byte blocks, but that is the only way that I've managed to get it to work. Did you use a blank tape when you tested bacula for the first time? More importantly, was btape used? Yes (and before I'd waited for that to finish (I wanted to sleep and can't with a tape drive running in the next room) cancelled it and tried a Bacula backup with the same throughput) IDE-based tape drives have always difficult... And of course I have an IDE drive... -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Fri, 24 Nov 2006 18:01:27 -0500, you wrote: Did you use a blank tape when you tested bacula for the first time? If not this is your problem. I do not believe the block size can be changed once there is data on the tape. you can fix this by rewinding the tape and writing an eof at the beginning of the tape using mt -f /dev/nst0 rewind mt -f /dev/nst0 weof then I believe bacula can use the tape and set the block size. Tried that - no difference. I tried using mt setblk to set the block size and all values I tried other than 512 gave an error. I suspect that the only valid block size is 512. Also tried mt -f /dev/nst0 stsetoptions no-blklimits scsi2logical that I found suggested on a web page, but again no improvement. I tried tar cvf /dev/nst0 file, with a 32MB file and the tape drive ran continuously for 40s, which is about 800kb/s. Although btape runs the tape drive in a bursty mode, will bacula in fact run it in a continuous write mode or does it use it exactly like btape? Not having completed the testing of the tape drive I am not sure how bacula will actually write. -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Saturday 25 November 2006 12:13, Peter Crighton wrote: On Fri, 24 Nov 2006 18:01:27 -0500, you wrote: Did you use a blank tape when you tested bacula for the first time? If not this is your problem. I do not believe the block size can be changed once there is data on the tape. you can fix this by rewinding the tape and writing an eof at the beginning of the tape using mt -f /dev/nst0 rewind mt -f /dev/nst0 weof then I believe bacula can use the tape and set the block size. Tried that - no difference. I tried using mt setblk to set the block size and all values I tried other than 512 gave an error. I suspect that the only valid block size is 512. In one sense you are right, the buffer size must be a multiple of 512, but any tape drive that requires all blocks to be written as 512 bytes should be immediately thrown out as it will never perform well on Unix systems, and will be even worse with Bacula which gains a lot of efficiency by blocking, and there is a certain overhead in blocking, which becomes important with small block sizes like 512 bytes, but is insignificant in 64K blocks. I don't believe this is the case for you i.e. I see no reason the tape drive will not accept blocks of larger than 512 byets. It is just a matter of getting all the blocking factors correct. I would be *very* surprised if you cannot use a minimum buffer size of 32K or more. Also tried mt -f /dev/nst0 stsetoptions no-blklimits scsi2logical that I found suggested on a web page, but again no improvement. I tried tar cvf /dev/nst0 file, with a 32MB file and the tape drive ran continuously for 40s, which is about 800kb/s. Read the tape back with dd, which should allow you to determine the block size that tar used. In fact, if you do an strace on tar, you will see *exactly* what it is doing, and you can simply duplicate it in Bacula. It is possible, but I will be very surprised if it is writing in 512 byte chunks. Although btape runs the tape drive in a bursty mode, will bacula in fact run it in a continuous write mode or does it use it exactly like btape? Not having completed the testing of the tape drive I am not sure how bacula will actually write. -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Sat, 25 Nov 2006 13:00:00 +0100, you wrote: On Saturday 25 November 2006 12:13, Peter Crighton wrote: I tried tar cvf /dev/nst0 file, with a 32MB file and the tape drive ran continuously for 40s, which is about 800kb/s. Read the tape back with dd, which should allow you to determine the block size that tar used. In fact, if you do an strace on tar, you will see *exactly* what it is doing, and you can simply duplicate it in Bacula. It is possible, but I will be very surprised if it is writing in 512 byte chunks. OK, with the command dd of=/home/root/test/tape count=1 /dev/nst0 I end up with a file called tape of size 512, so I believe that tar is writing 512 byte blocks, but doing so continuously rather than repeatedly stopping. Out of interest and despite not having completed the fill test in btape, I have initiated a backup using bconsole. 35MB were backed up in approx 8 minutes, equating to the 70kb/s that I see with btape (and confirming to me that the tape is accessed in a bursty fashion within bacula as I expected). The size is confirmed by using scanblocks in btape. Unless anyone can suggest an alternative setup for my existing tape drive, I guess now that my options are to accept the performance of this tape drive, use an alternative backup program, or buy a new (well second hand off Ebay) drive. What are the best alternative type of drive (it's for a home office so only requires a modest capability). Looking on Ebay DDS-3/DDS-4 drives and tapes are available at a reasonable cost, giving a comparable capacity to my current drive and probably almost a no-cost swap if I sell my existing drive and tapes on Ebay. -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On Saturday 25 November 2006 15:45, Peter Crighton wrote: On Sat, 25 Nov 2006 13:00:00 +0100, you wrote: On Saturday 25 November 2006 12:13, Peter Crighton wrote: I tried tar cvf /dev/nst0 file, with a 32MB file and the tape drive ran continuously for 40s, which is about 800kb/s. Read the tape back with dd, which should allow you to determine the block size that tar used. In fact, if you do an strace on tar, you will see *exactly* what it is doing, and you can simply duplicate it in Bacula. It is possible, but I will be very surprised if it is writing in 512 byte chunks. OK, with the command dd of=/home/root/test/tape count=1 /dev/nst0 I end up with a file called tape of size 512, so I believe that tar is writing 512 byte blocks, but doing so continuously rather than repeatedly stopping. Out of interest and despite not having completed the fill test in btape, I have initiated a backup using bconsole. 35MB were backed up in approx 8 minutes, equating to the 70kb/s that I see with btape (and confirming to me that the tape is accessed in a bursty fashion within bacula as I expected). The size is confirmed by using scanblocks in btape. Unless anyone can suggest an alternative setup for my existing tape drive, I guess now that my options are to accept the performance of this tape drive, use an alternative backup program, or buy a new (well second hand off Ebay) drive. What are the best alternative type of drive (it's for a home office so only requires a modest capability). Looking on Ebay DDS-3/DDS-4 drives and tapes are available at a reasonable cost, giving a comparable capacity to my current drive and probably almost a no-cost swap if I sell my existing drive and tapes on Ebay. I would strongly recommend against DDS-3. DDS-4 is OK, barely so. You will be much better off with a DLT-8000, which is probably more expensive, but if your data is important, it is worth it, if for no other reason than any DDS technology or older DLTs are likely to cause you problems as they have about a 90% chance of failing at some point, either the drives or the tapes. -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Travan tape - very slow
I now have found btape so I set my tape testing on fill this morning.
After 12 hours it's only managed 3GB. It is reporting about 70kb/s,
which is consistent with the rate it's reporting.
With other programs (Windows 98/Veritas backup and Linux/Amanda) it
can fill a 10GB tape in that time (it's spec is 60MB/min and as I
recall but didn't note Amanda was not far off - from memory I think it
was a about 40MB/min).
I have had to set 512 byte blocks (it didn't work with variable size
and mt reports that it has 512 byte blocks). Others have also reported
this requirement. The drive is on an IDE cable shared with the CDROM
(not in use) so it should have full access to the IDE port.
With other programs when writing data the tape whirrs continuously in
one direction. With btape it is winding one way then silent the
rewinding then going forward (that's what it sounds like anyway). That
will clearly hit the performance if it doesn't write continuously.
The drive is a Travan 20 (10GB native, 20GB compressed) Seagate
STT22A.
Clearly the performance is much reduced from what the drive is capable
of. Has anyone suggestions (ideally first hand experience of this or a
very similar model, e.g the STT28000A 8GB/4GB version) that could
help?
The drive may be a few years old, but it's capacity and speed (when
it's going at it's flat out speed) suits my requirements, plus I have
a second drive in another machine which gives me some redundancy in
the event of a failure).
This is the definition that I added to bacula-sd.conf:
Device {
Name = Travan
Description = Travan 10/20GB
Media Type = TR-5
Archive Device = /dev/nst0
AutomaticMount = yes;
AlwaysOpen = yes
Offline On Unmount = no
Fast Forward Space File = no
Hardware End of Medium = no
Minimum Block Size = 512
Maximum Block Size = 512
}
--
Peter Crighton
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
Hello,
It generally helps to read the manual in cases like this. Here is a short
except from the manual that should pretty much explain it:
Bacula's preferred method of working with tape drives (sequential devices) is
to run in variable block mode, and this is what is set by default. You should
first ensure that your tape drive is set for variable block mode (see below).
If your tape drive is in fixed block mode and you have told Bacula to use
different fixed block sizes or variable block sizes (default), you will get
errors when Bacula attempts to forward space to the correct block (the kernel
driver's idea of tape blocks will not correspond to Bacula's).
All modern tape drives support variable tape blocks, but some older drives
(in particular the QIC drives) as well as the ATAPI ide-scsi driver run only
in fixed block mode. The Travan tape drives also apparently must run in fixed
block mode (to be confirmed).
Even in variable block mode, with the exception of the first record on the
second or subsequent volume of a multi-volume backup, Bacula will write
blocks of a fixed size. However, in reading a tape, Bacula will assume that
for each read request, exactly one block from the tape will be transferred.
This the most common way that tape drives work and is well supported by
Bacula.
Drives that run in fixed block mode can cause serious problems for Bacula if
the drive's block size does not correspond exactly to Bacula's block size. In
fixed block size mode, drivers may transmit a partial block or multiple
blocks for a single read request. From Bacula's point of view, this destroys
the concept of tape blocks. It is much better to run in variable block mode,
and almost all modern drives (the OnStream is an exception) run in variable
block mode. In order for Bacula to run in fixed block mode, you must include
the following records in the Storage daemon's Device resource definition:
Minimum Block Size = nnn
Maximum Block Size = nnn
where nnn must be the same for both records and must be identical to the
driver's fixed block size.
We recommend that you avoid this configuration if at all possible by using
variable block sizes.
If you must run with fixed size blocks, make sure they are not 512 bytes.
This is too small and the overhead that Bacula has with each record will
become excessive. If at all possible set any fixed block size to something
like 64,512 bytes or possibly 32,768 if 64,512 is too large for your drive.
See below for the details on checking and setting the default drive block
size.
To recover files from tapes written in fixed block mode, see below.
On Friday 24 November 2006 21:06, Peter Crighton wrote:
I now have found btape so I set my tape testing on fill this morning.
After 12 hours it's only managed 3GB. It is reporting about 70kb/s,
which is consistent with the rate it's reporting.
With other programs (Windows 98/Veritas backup and Linux/Amanda) it
can fill a 10GB tape in that time (it's spec is 60MB/min and as I
recall but didn't note Amanda was not far off - from memory I think it
was a about 40MB/min).
I have had to set 512 byte blocks (it didn't work with variable size
and mt reports that it has 512 byte blocks). Others have also reported
this requirement. The drive is on an IDE cable shared with the CDROM
(not in use) so it should have full access to the IDE port.
With other programs when writing data the tape whirrs continuously in
one direction. With btape it is winding one way then silent the
rewinding then going forward (that's what it sounds like anyway). That
will clearly hit the performance if it doesn't write continuously.
The drive is a Travan 20 (10GB native, 20GB compressed) Seagate
STT22A.
Clearly the performance is much reduced from what the drive is capable
of. Has anyone suggestions (ideally first hand experience of this or a
very similar model, e.g the STT28000A 8GB/4GB version) that could
help?
The drive may be a few years old, but it's capacity and speed (when
it's going at it's flat out speed) suits my requirements, plus I have
a second drive in another machine which gives me some redundancy in
the event of a failure).
This is the definition that I added to bacula-sd.conf:
Device {
Name = Travan
Description = Travan 10/20GB
Media Type = TR-5
Archive Device = /dev/nst0
AutomaticMount = yes;
AlwaysOpen = yes
Offline On Unmount = no
Fast Forward Space File = no
Hardware End of Medium = no
Minimum Block Size = 512
Maximum Block Size = 512
}
--
Peter Crighton
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
Re: [Bacula-users] Travan tape - very slow
On Fri, 24 Nov 2006 22:42:30 +0100, you wrote: Hello, It generally helps to read the manual in cases like this. Here is a short except from the manual that should pretty much explain it: Hi, I did read that and yes I understand it says don't use 512 byte blocks, but that is the only way that I've managed to get it to work. Variable blocks didn't work (as I said). It says that Travan must run in fixed block mode. The issue is that the Travan tape is very slow. Now maybe one of my settings is wrong, but I don't know what and was looking for some help as I'd like to get it working. Alternatively, maybe someone who has this going well can help me. [I looked at using Amanda. I got it going but it requires a new tape for every backup. Bacula overcomes that limitation but at a price of very slow tape writing at the moment for me.]] -- Peter Crighton - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Travan tape - very slow
On 11/24/06, Peter Crighton [EMAIL PROTECTED] wrote: On Fri, 24 Nov 2006 22:42:30 +0100, you wrote: Hello, It generally helps to read the manual in cases like this. Here is a short except from the manual that should pretty much explain it: Hi, I did read that and yes I understand it says don't use 512 byte blocks, but that is the only way that I've managed to get it to work. Did you use a blank tape when you tested bacula for the first time? If not this is your problem. I do not believe the block size can be changed once there is data on the tape. you can fix this by rewinding the tape and writing an eof at the beginning of the tape using mt -f /dev/nst0 rewind mt -f /dev/nst0 weof then I believe bacula can use the tape and set the block size. John - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
