Hello,
Today is my deadline in the project to increase the speed of writing to
disk. But I failed..... :-(
Iam just asking the same question again, If anybody can give me a lead,
it will be very helpful.
I will give the complete detail.
SYSTEM : Power edge 2650.
CPU - 1
RAM - 4GB.
OS - SLES 9 SP3.
Bacula - 2.2.7.
Tape Drive - SCSI LVD Ultrium LTO 3 Autchanger.
Database - mysql 4.1
Controller in host - Mega raid controller (perc3)
Maximum writing speed with bacula - 7.8 Megabytes/second.
Maximum Writing speed with vdump and 1024 block size - 32
Megabytes/second.
The tape library device - /dev/sg1
The tape drive - /dev/sg0 ( But we use /dev/nst0 for taking backup )
The Autochanger company claims the following speed of writing.
Data Transfer Rate (native): 80 MBps ( 22.8 GBph )
Data Transfer Rate (compressed): 160 MBps ( 45.5 GBph )
I feel the above is in Megabits/sec? am I correct, please tell me if
anybody is sure on this. As I get a speed of 7.8 Megabytes / sec with
compression and without compression.
The whole bacula set up works fine with our environment, like Storage
daemon,file daemon etc. And the backup process is absolutely fine except
for the speed.
I have changed the daisy chain scsi cable 3 times. No improvement in
speed.
I have tried changing the terminator also today. The same speed results.
The funny thing is I tried without terminator also and everything works
fine with same speed.
I have tried with enabling spooling ( no raid ). The spooling speed
differs from 2 Megabytes/sec to 100 Megabytes/sec but the despooling
speed lies exactly at 7.8 megabytes/second.
The despooling speed is at 7.8 Megabytes/sec for local file system,
mounted file system(tru 64 filesystem mountes in bacula server ) and any
file system of bacula client .
Does anyone find anything fishy???? Kindly help...
The config iam using are attached.
Thanks & Regards,
Bhaskaran
-----Original Message-----
From: Rathinasamy, Bhaskaran (ext)
Sent: Wednesday, February 20, 2008 5:19 PM
To: 'John Drescher'
Cc: bacula-users@lists.sourceforge.net
Subject: RE: [Bacula-users] Increase DeSpooling Speed.
Hello John,
Many Thanks .
I have tested the raw drive speed with vdump and bs=1024, The drive
speed is around 35MB/s during vdump.
The tape drive is SCSI LVD.I had tried replacing the cable twice.but no
improvement in speed through bacula.
Does a spool file on raid0 increases the despooling speed significantly.
Because my spooling directory is /var/backup and it is not raided and no
spool size limit is set.
We use a single processor machine with 4 GB memory and the spool folder
is ext3.
Is your despooling speed with Compression or not? Either software or
hardware compression?
Thanks & Regards,
Bhaskaran
-----Original Message-----
From: John Drescher [mailto:[EMAIL PROTECTED]
Sent: Wednesday, February 20, 2008 4:55 PM
To: Rathinasamy, Bhaskaran (ext)
Cc: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Increase DeSpooling Speed.
On Feb 20, 2008 10:38 AM, Rathinasamy, Bhaskaran (ext)
<[EMAIL PROTECTED]> wrote:
> Hello,
>
> We have a LTO-3 Ultrium Tape Library.Bacula 2.2.7 is implemented in
SLES
> 9.
>
> The despooling speed to the tape is 7.8 MB/sec always, even with
> hardware compression and no hardware compression.
>
Is your tape drive scsi? Possiblty the problem is a scsi problem.
You can test raw drive speeds with dd something like:
time dd if=/dev/sda of=/dev/nst0 bs=1G count=10
If you have hardware compression on do not use /dev/zero because you
will get a very unrealistic result.
>
> Is it possible to increase the despooling speed.
>
I get despooling speeds of 30 to 45MB/s on a LTO2 autochanger when
using a 2GB spool file. My spool folder is currently on its own xfs
filesystem and using software raid0. My bacula-sd machine (different
machine from the director) is a 2 processor opteron 248 with 4GB of
memory.
John
#
# Default Bacula File Daemon Configuration file
#
# For Bacula release 2.2.7 (24 December 2007) -- suse 9
#
# There is not much to change here except perhaps the
# File daemon Name to
#
#
# List Directors who are permitted to contact this File daemon
#
Director {
Name = bw1lib1-dir
Password = "HLpR5YG1GvgK+90EPvVjN4G2UWI7ff6+qZe80En4SLEI"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the file daemon
#
Director {
Name = bw1lib1-mon
Password = "i5p7IZaavhq0TshzlUmloXf/dw7jLdUAgReoe11tVG/y"
Monitor = yes
}
#
# "Global" File daemon configuration specifications
#
FileDaemon { # this is me
Name = bw1lib1-fd
FDport = 9102 # where we listen for the director
WorkingDirectory = /home/bw1ratb0/bacula/working
Pid Directory = /home/bw1ratb0/bacula/bin/working
Maximum Concurrent Jobs = 20
}
# Send all messages except skipped files back to Director
Messages {
Name = Standard
director = bw1lib1-dir = all, !skipped, !restored
}
#
# Default Bacula Storage Daemon Configuration file
#
# For Bacula release 2.2.7 (24 December 2007) -- suse 9
#
# You may need to change the name of your tape drive
# on the "Archive Device" directive in the Device
# resource. If you change the Name and/or the
# "Media Type" in the Device resource, please ensure
# that dird.conf has corresponding changes.
#
Storage { # definition of myself
Name = bw1lib1-sd
SDPort = 9103 # Director's port
WorkingDirectory = "/home/bw1ratb0/bacula/working"
Pid Directory = "/home/bw1ratb0/bacula/bin/working"
Maximum Concurrent Jobs = 20
}
#
# List Directors who are permitted to contact Storage daemon
#
Director {
Name = bw1lib1-dir
Password = "IxyTEGNzxjcdfg7Ovn7KByqg1fFou1KbcUjv/hqqJqHz"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = bw1lib1-mon
Password = "FLnjCXbL08CnER9lzUATqord0/Zj8N6F4V58EiOcuHeB"
Monitor = yes
}
# Devices supported by this Storage daemon
# To connect, the Director's bacula-dir.conf must have the
# same Name and MediaType.
#
#Device {
# Name = FileStorage
# Media Type = LTO-3
# Archive Device = /dev/nst0
# LabelMedia = yes; # lets Bacula label unlabeled media
# Random Access = Yes;
# AutomaticMount = yes; # when device opened, read it
# RemovableMedia = no;
# AlwaysOpen = no;
#}
#
# An autochanger device with two drives
#
Autochanger {
Name = Autochanger
Device = Drive-1
# Device = Drive-2
Changer Command = "/home/bw1ratb0/bacula/bin/mtx-changer %c %o %S %a %d"
Changer Device = /dev/sg1
}
Device {
Name = Drive-1 #
Drive Index = 0
Media Type = LTO-3
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
AutoChanger = yes
Maximum Spool Size = 6G
Maximum Job Spool Size = 6G
SpoolDirectory = /var/backup;
# #
# # Enable the Alert command only if you have the mtx package loaded
# # Note, apparently on some systems, tapeinfo resets the SCSI controller
# # thus if you turn this on, make sure it does not reset your SCSI
# # controller. I have never had any problems, and smartctl does
# # not seem to cause such problems.
# #
# Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
# If you have smartctl, enable this, it has more info than tapeinfo
# Alert Command = "sh -c 'smartctl -H -l error %c'"
}
# Send all messages to the Director,
# mount messages also are sent to the email address
#
Messages {
Name = Standard
director = bw1lib1-dir = all
}
#
# Default Bacula Director Configuration file
#
# The only thing that MUST be changed is to add one or more
# file or directory names in the Include directive of the
# FileSet resource.
#
# For Bacula release 2.2.7 (24 December 2007) -- suse 9
#
# You might also want to change the default email address
# from root to your address. See the "mail" and "operator"
# directives in the Messages resource.
#
Director { # define myself
Name = bw1lib1-dir
DIRport = 9101 # where we listen for UA connections
QueryFile = "/home/bw1ratb0/bacula/bin/query.sql"
WorkingDirectory = "/home/bw1ratb0/bacula/working"
PidDirectory = "/home/bw1ratb0/bacula/bin/working"
Maximum Concurrent Jobs = 1
Password = "m3nRsYwfI/XMd1tky+UtuWror1FCf37ROMd/A2QWW4Ol" # Console
password
Messages = Daemon
}
JobDefs {
Name = "DefaultJob"
Type = Backup
# Level = Incremental
Level = Full
Client = bw1lib1-fd
FileSet = "Full Set"
Schedule = "WeeklyCycle"
Storage = File
Messages = Standard
Pool = Default
Priority = 10
SpoolData=yes
}
Job {
Name = "TestslaveJob"
Type = Backup
# Level = Incremental
Level = Full
Client = testslave-fd
FileSet = "Full Set"
Schedule = "WeeklyCycle"
Storage = File
Messages = Standard
Pool = Default
Priority = 10
SpoolData=no
Write Bootstrap = "/home/bw1ratb0/bacula/working/testslave.bsr"
}
#
# Define the main nightly save backup job
# By default, this job will back up to disk in
Job {
Name = "Client1"
JobDefs = "DefaultJob"
Write Bootstrap = "/home/bw1ratb0/bacula/working/Client1.bsr"
}
# Backup the catalog database (after the nightly save)
Job {
Name = "BackupCatalog"
JobDefs = "DefaultJob"
Level = Full
FileSet="Catalog"
Schedule = "WeeklyCycleAfterBackup"
# This creates an ASCII copy of the catalog
RunBeforeJob = "/home/bw1ratb0/bacula/bin/make_catalog_backup bacula bacula"
# This deletes the copy of the catalog
RunAfterJob = "/home/bw1ratb0/bacula/bin/delete_catalog_backup"
Write Bootstrap = "/home/bw1ratb0/bacula/working/BackupCatalog.bsr"
Priority = 11 # run after main backup
}
#
# Standard Restore template, to be changed by Console program
# Only one such job is needed for all Jobs/Clients/Storage ...
#
Job {
Name = "RestoreFiles"
Type = Restore
Client=bw1lib1-fd
FileSet="Full Set"
Storage = File
Pool = Default
Messages = Standard
Where = /bacula-restores
}
# List of files to be backed up
FileSet {
Name = "Full Set"
Include {
Options {
signature = MD5
# compression=gzip9
}
#
# Put your list of files here, preceded by 'File =', one per line
# or include an external list with:
#
# File = <file-name
#
# Note: / backs up everything on the root partition.
# if you have other partitons such as /usr or /home
# you will probably want to add them too.
# By default this is defined to point to the Bacula build
# directory to give a reasonable FileSet to backup to
# disk storage during initial testing.
#
# File = /home/bw1ratb0/bacula/bacula-2.2.7
File = /opt
}
#
# If you backup the root directory, the following two excluded
# files can be useful
#
Exclude {
File = /proc
File = /tmp
File = /.journal
File = /.fsck
}
}
#
# When to do the backups, full backup on first sunday of the month,
# differential (i.e. incremental since full) every other sunday,
# and incremental backups other days
Schedule {
Name = "WeeklyCycle"
Run = Full fri at 13:05
Run = Differential 2nd-5th sun at 23:05
Run = Incremental fri at 13:05
}
# This schedule does the catalog. It starts after the WeeklyCycle
Schedule {
Name = "WeeklyCycleAfterBackup"
Run = Full sun-sat at 23:10
}
# This is the backup of the catalog
FileSet {
Name = "Catalog"
Include {
Options {
signature = MD5
}
File = /home/bw1ratb0/bacula/working/bacula.sql
}
}
# Client (File Services) to backup
Client {
Name = bw1lib1-fd
Address = bw1lib1
FDPort = 9102
Catalog = MyCatalog
Password = "HLpR5YG1GvgK+90EPvVjN4G2UWI7ff6+qZe80En4SLEI" # password
for FileDaemon
File Retention = 30 days # 30 days
Job Retention = 6 months # six months
AutoPrune = yes # Prune expired Jobs/Files
}
Client {
Name = testslave-fd
Address = testslave
FDPort = 9102
Catalog = MyCatalog
# Password = "E8MVOq4At5F9gKpUqDSDEmvtvvr3wZ/M5mS5hIoxPlND" #
password for FileDaemon
Password = "HLpR5YG1GvgK+90EPvVjN4G2UWI7ff6+qZe80En4SLEI" # password
for FileDaemon
File Retention = 30 days # 30 days
Job Retention = 6 months # six months
AutoPrune = yes # Prune expired Jobs/Files
}
# Definition of file storage device
Storage {
Name = File
Autochanger = "yes"
# Do not use "localhost" here
Address = bw1lib1 # N.B. Use a fully qualified name here
SDPort = 9103
Password = "IxyTEGNzxjcdfg7Ovn7KByqg1fFou1KbcUjv/hqqJqHz"
Device = Autochanger
Media Type = LTO-3
}
# Generic catalog service
Catalog {
Name = MyCatalog
dbname = bacula; user = bacula; password = ""
}
# Reasonable message delivery -- send most everything to email address
# and to the console
Messages {
Name = Standard
#
# NOTE! If you send to two email or more email addresses, you will need
# to replace the %r in the from field (-f part) with a single valid
#
mailcommand = "/home/bw1ratb0/bacula/bin/bsmtp -h localhost -f \"\(Bacula\)
\<%r\>\" -s \"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/home/bw1ratb0/bacula/bin/bsmtp -h localhost -f
\"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"
mail = [EMAIL PROTECTED] = all, !skipped
operator = [EMAIL PROTECTED] = mount
console = all, !skipped, !saved
#
# WARNING! the following will create a file that you must cycle from
# time to time as it will grow indefinitely. However, it will
# also keep all your messages if they scroll off the console.
#
append = "/home/bw1ratb0/bacula/working/log" = all, !skipped
}
#
# Message delivery for daemon messages (no job).
Messages {
Name = Daemon
mailcommand = "/home/bw1ratb0/bacula/bin/bsmtp -h localhost -f \"\(Bacula\)
\<%r\>\" -s \"Bacula daemon message\" %r"
mail = [EMAIL PROTECTED] = all, !skipped
console = all, !skipped, !saved
append = "/home/bw1ratb0/bacula/working/log" = all, !skipped
}
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
}
# Scratch pool definition
Pool {
Name = Scratch
Pool Type = Backup
}
#
# Restricted console used by tray-monitor to get the status of the director
#
Console {
Name = bw1lib1-mon
Password = "wEKqNN6XiEiD5LVCYeemFsUP9uc29SWTi08gVNSCpfMS"
CommandACL = status, .status
}
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
