Re: [Bacula-users] Postgres vs SQLite

2013-05-01 Thread Tim Gustafson
 If you are backing up one machine with a small number of files then sqlite
 might be okay, but otherwise you'll probably find it will be a performance
 bottleneck for anything bigger.

 I recommend you go with postgresql (or mysql).

I've used MySQL in the past, and Bacula is just apparently not
optimized for it (or vice-versa, I'm not sure which).  We run a fairly
beefy MySQL server and we have hundreds of apps and web sites that all
use that server and all of them work extremely well but when we used
it for Bacula, the query that it used to build a list of files to
restore took *ages* - in some cases more than 24 hours, and in some
cases it never finished at all - for our data set.  When we switched
to Postgres, that query went down to a few minutes.  Our backup load
has changed significantly since then - we now use ZFS snapshots for
our multi-terabyte, multi-million-inode file systems and use Bacula
for our smaller VMs, none of which have more than a few tens of
thousands of files each.  So maybe it's time to revisit using MySQL.

I just really hate maintaining a whole database server for one
application, especially one as unwieldy as Postgres.  Postgres
requires a fair amount of memory, and has some compatibility issues
with FreeBSD Jails (it requires you enable sysvipc for all jails,
which is something of a security concern).  It's also one more thing
that I have to monitor.

-- 

Tim Gustafson
t...@ucsc.edu
831-459-5354
Baskin Engineering, Room 313A

--
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with 2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Postgres vs SQLite

2013-04-30 Thread Tim Gustafson
Hi,

I was wondering if there was any information about the performance
difference between running Bacula with a Postgres database vs an
SQLite database.  I don't have any other need for a Postgres server,
so if I can get Bacula to perform as well with SQLite as it does with
Postgres, then I'd prefer to drop Postgres altogether.

--

Tim Gustafson
t...@ucsc.edu
831-459-5354
Baskin Engineering, Room 313A

--
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with 2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Backup-To-Files / Multiple Concurrent Jobs

2011-06-10 Thread Tim Gustafson
Hi,

We have Bacula configured to backup to file.  I'm attaching the relevant 
snippets of my director, storage daemon and file daemon configuration files.

When we run jobs that are configured to go to the same storage device, the 
first job runs and then all the subsequent jobs queue up with a message like 
this:

www-01.2011-06-10_08.56.15_06 is waiting on Storage bacula-02-bsoe-02

I'm sure that I'm just missing some switch somewhere - what am I doing wrong 
that is preventing more than one job from running at once?

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Backup-To-Files / Multiple Concurrent Jobs

2011-06-10 Thread Tim Gustafson
Whoops, I forgot to attach the config files.  Here they are!

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Client {
  Name = www-01-fd
  Address = www-01
  FDPort = 9102
  Catalog = MyCatalog
  Password = password
  File Retention = 2 Months
  Job Retention = 2 Months
  Maximum Concurrent Jobs = 10
}

FileSet {
  Name = www-01-files
  Include {
Options {
  Accurate = mcs
  Verify = mcs
  Onefs = Yes
  Sparse = Yes
}
File = /
  }
  Exclude {
File = /tmp
File = /var/tmp
File = /usr/src
File = /usr/obj
File = /usr/ports
  }
}

Pool {
  Name = www-01-pool
  Pool Type = Backup
  Volume Retention = 2 Months
  Maximum Volume Jobs = 1
  Maximum Volumes = 50
  Label Format = www-01-
}

Job {
  JobDefs = DefaultJob
  Name = www-01
  Client = www-01-fd
  FileSet = www-01-files
  Messages = www-01-messages
  Schedule = default
  Pool = www-01-pool
  Storage = bacula-02-bsoe-02
  Maximum Concurrent Jobs = 10
}

Storage {
  Name = bacula-02-bsoe-02
  Address = bacula-02
  SDPort = 9103
  Password = password
  Device = bsoe-02
  Media Type = File
  Maximum Concurrent Jobs = 10
  AllowCompression = Yes
}
Storage {
  Name = bacula-02-dir
  WorkingDirectory = /var/db/bacula
  Pid Directory = /var/run
  Maximum Concurrent Jobs = 10
}

Device {
  Name = bsoe-02
  Media Type = File
  Archive Device = /bacula/bsoe
  LabelMedia = Yes
  Random Access = Yes
  AutomaticMount = Yes
  RemovableMedia = No
  AlwaysOpen = No
  Maximum Concurrent Jobs = 10
}

Director {
  Name = bacula-dir-dir
  Password = password
}

FileDaemon {
  Name = www-01-fd
  FDport = 9102
  WorkingDirectory = /var/db/bacula
  Pid Directory = /var/run
  Maximum Concurrent Jobs = 10
  Maximum Network Buffer Size = 65536
}

--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Backup-To-Files / Multiple Concurrent Jobs

2011-06-10 Thread Tim Gustafson
 You have Maximum Volume Jobs = 1 in your pool. If you intend to
 use a single storage device and this pool you can only run 1 job 
 at a time on this pool since only 1 job can be written to a volume
 a storage device can only load 1 volume at a time.

I think we had Maximum Volume Jobs = 1 so that we get a new file on the 
storage daemon for each job that runs.

Is there any way to keep the one-file-per-job feature and also allow multiple 
concurrent jobs to the device?

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Director Control Protocol

2011-06-07 Thread Tim Gustafson
Hi,

I was wondering if there is any documentation anywhere on the protocol that 
bconsole uses to connect to the director and issue commands?

I've built a web interface to the Bacula configuration files, and I would like 
to add the ability to reload the configuration and/or start or cancel jobs from 
that web interface as well.

If it's not well documented, I'll just tcpdump the connection between bconsole 
and the director itself and try to reverse-engineer it, but I was hoping that 
it was actually documented somewhere.

BTW: I don't need comments about re-inventing the wheel - I know there are 
other web-based Bacula tools out there; I built this one with specific 
organizational needs in mind.  All I want to know is if the protocol between 
bconsole and the director is documented somewhere.  I did find some 
documentation on the protocols for director-fd, direcdtor-sd and fd-sd, but 
nothing about bconsole-director.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Director Control Protocol

2011-06-07 Thread Tim Gustafson
 Can't you spawn bconsole from the web application? That is what most
 of the other web apps do.

I thought of that, but the interaction to get a simple bit of data out that way 
seems to be rather difficult to parse.  Consider getting the status of a given 
client:

1. Connect to the bacula-dir server via SSH
2. Run bconsole
3. Wait for a prompt
4. Enter status
5. Wait for a prompt
6. Enter 3
7. Parse the list of client presented and figure out which option represents 
the client you want
8. Enter the number you parsed from step 7
9. Parse the output of that command into a program-usable form

I'm hoping that there might be a protocol that cuts out all the screen parsing 
and instead lets me just do something like:

1. Connect to the bacula-dir daemon
2. Authenticate
3. Send a command like show status client blah.foo.bar-fd

and have that return machine-parse-able status information, rather than 
human-readable information.

Really, it's just a matter of cleaner code.  I -could- do it by running 
bconsole and using a send/expect type system, but it seems like there ought to 
be a better way.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Fwd: Director Control Protocol

2011-06-07 Thread Tim Gustafson
 What about
 echo show status client=ClientName | bconsole

Yes, I know I can do that.

I was asking if there was any documentation on the protocol that bconsole uses 
to talk to the director daemon.  It would appear that the answer is no.  
Thanks anyhow!

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Fwd: Director Control Protocol

2011-06-07 Thread Tim Gustafson
 http://bacula.org/5.0.x-manuals/en/developers/developers/Protocol_Used_Between_Direc.html#SECTION0065

That appears to be the protocol between the director and the file daemon, not 
bconsole.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] bacula over long-haul connections

2011-06-02 Thread Tim Gustafson
Hi,

We have two storage directors: one locally and one located at an off-site data 
center.  Our connection to the data center is a 10G link (but it's shared with 
other UC universities).  Using programs like BBFTP, we've been able to achieve 
actual data throughput of something like 600Mbit/s to the remote storage daemon.

The storage daemon boxes are FreeBSD 8.2 with 16TB (local) and 12TB (remote) 
ZFS file systems, and the file daemon we're testing with is CentOS with an ext3 
file system.

When we back up to the local storage daemon, we're getting about 500Mbit/s.

When we back up to the remote storage daemon, we're getting about 66Mbit/s.

So, my question is: are there any compile-time options or configuration tricks 
for Bacula that would make it perform better on this sort of long-haul 
connection?  I'm guessing that the answer is probably you need to tweak your 
OS TCP/IP stack, but I thought I'd ask here if anyone else has this sort of 
setup and could maybe offer advice specific to Bacula that might help, at least 
to some degree.

Thanks!

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today. 
http://p.sf.net/sfu/quest-dev2dev2 
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Understanding Encryption

2011-05-27 Thread Tim Gustafson
 one master key for each client wouldn't make that much sense, since you
 could just the client keys in a safe place. I have one master key for
 everything. But I don't keep the private key on the director. I have it
 on a pen drive and (to be extra sure) printed out in a safe on site and
 on an encrypted pen drive that I always carry with me.

So, the master key is a second key that can be used to decrypt the backup then. 
 The people whose severs I'm backing up might not want me to have access to 
their data, so those users would have to manage their own master keys, correct?

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery, 
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now. 
http://p.sf.net/sfu/quest-d2dcopy1
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Understanding Encryption

2011-05-26 Thread Tim Gustafson
Hi there,

I was just looking at the following documentation page:

http://www.bacula.org/en/dev-manual/main/main/Data_Encryption.html

That page contains information about generating a master key and then also a 
set of client keys.  However, the page is not clear whether you're supposed 
to use the same master key for all your clients, or if you should have a 
different master key for each client.  Should I be sharing the master.cert file 
with each client and keeping the master.key file on my bacula-dir server, or 
does each client need its own master.cert and master.key file?

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tim Gustafsont...@soe.ucsc.edu
Baskin School of Engineering 831-459-5354
UC Santa Cruz Baskin Engineering 317B
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


--
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery, 
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now. 
http://p.sf.net/sfu/quest-d2dcopy1
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tuning for large (millions of files) backups?

2010-10-08 Thread Tim Gustafson
 This is an interesting observation.  How does one
 determine/set the InnoDB block size?

Sorry for butting in here, but I've been following this thread.

You can't change the InnoDB block size unless you recompile from source, from 
what I understand...but that's besides the point.

Using InnoDB adds quite a bit of overhead to most database operations; 
shouldn't Bacula be using MyISAM tables, which are much faster?  My thinking is 
that there is not a lot of concurrency with database reads and writes, and 
probably not much need for referential integrity...or am I missing something?

Tim Gustafson
Baskin School of Engineering
UC Santa Cruz
t...@soe.ucsc.edu
831-459-5354

--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2  L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tuning Bacula

2010-10-07 Thread Tim Gustafson
I'm going to try to reply to all the responses I got together.

 Have you tried backing up other hosts on your network? What are
 the speeds with these hosts? I've noticed that different host
 respond with varying speeds despite being on the same network.
 Wondering if this has to do the client OS doing some throttling
 based on work load.

I am backing up the Bacula server itself, my workstation (with is a FreeBSD box 
as well), and our main file server, which is a SunOS server.  We aren't doing 
any throttling intentionally, but I also see a large variation in throughput 
depending on the client in question, but none of them - not even the local 
server backing up itself - are all that impressive right now.

 I would start by turning off software compression and do
 performance tests with full backups. A second thing to try is to
 enable attribute spooling so the database does not slow down the
 backup. This can be useful if you have millions of files.

We do not have software compression enabled, as far as I can tell.  I've turned 
on the Spool Attributes option in my job definition, and we'll see if that 
helps.

 Compare against a stock, non tuned, Bacula install. Are you
 going between building where you get the slow transfer speed?
 UCSC has 1 Gb links between buildings from my recollection. The
 link to the outside world is not much more than that. Bacula
 also has a batch mode which you can twiddle around with.

For the slowest backup job, the two servers are sitting in the same rack on the 
same gigabit switch.  The fastest client actually is in a different building.  
Yes, we have 1Gb between buildings here, but out Internet connection was 
recently upgraded to 10Gb (not that it really applies to this situation anyhow).

I found some Google hits that talked about batch mode, but no documentation 
that tells me how to enable it.  Can you provide a link?

 Is the MySQL database storage on the same RAID array you are
 writing backups to?

Yes and no.  Currently, in our dev environment, they are both on the same 
physical RAID array, but Bacula operates in a separate jail from mySQL.  When 
we move to production, the director will probably run on one server and the 
storage daemon on another, so maybe that will help?

 It may be useful to run iftop on the network interfaces of the
 Bacula server to see what the network IO is like, and then compare
 that to iotop to see what the disk IO is like.

We actually run Cacti against all our servers.  Disk throughput for the Bacula 
server can hit as much as 240Mb/s during a backup, whereas the network 
throughput at the same time is around 80Mb/s, with a few spikes to 96Mb/s.  For 
what it's worth, iperf can hit about 780Mb/s between these hosts.

I just twiddled some ZFS parameters last night (turning off the primary and 
seconday caches) and reconfigured the zpool to let ZFS handle the striping 
(rather than the Adaptec controller handling the RAID array), so we'll see what 
numbers we come back with tomorrow.  I've also added some other different 
hardware/OS combination clients to see if we can work out a pattern.

Tim Gustafson
Baskin School of Engineering
UC Santa Cruz
t...@soe.ucsc.edu
831-459-5354

--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2  L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tuning Bacula

2010-10-07 Thread Tim Gustafson
 Without attribute spooling or batch (not sure if that
 is postgres only) after each file is read the database
 needs to add records.

We have attribute spooling activated right now.

Tim Gustafson
Baskin School of Engineering
UC Santa Cruz
t...@soe.ucsc.edu
831-459-5354

--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2  L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Tuning Bacula

2010-10-04 Thread Tim Gustafson
We have recently installed Bacula onto a FreeBSD server and several Linux, 
SunOS and FreeBSD clients.  The Bacula director and storage daemon run on a box 
with about 6 terabytes of RAID6 storage (SATA 300 drives, 1TB each, Adaptec 
RAID controller with 512MB cache).  The box has 16GB of RAM and is not really 
doing much else right now. We're using mySQL for our database back-end, and we 
have MD5 hashing of files turned off (Accurate = mcs and Verify = mcs are 
set in bacula-dir.conf).

However, we're getting pretty pitiful throughput numbers.  When I scp a file 
from my workstation to the Bacula server, I get something like 40MB/s 
(320Mb/s).  When Bacula runs, we're lucky to get 20MB/s (160Mb/s), and we often 
get numbers closer to 10MB/s (80Mb/s).

I Googled tuning bacula and came up with primarily stuff related to tuning 
Postgres as it relates to Bacula, but nothing about tuning the file daemon or 
the storage daemon.  Can anyone point me to some leads as far as what I can do 
to bump up the throughput?  We have a data set that is several terabytes large 
to back up, and it will never complete in a reasonable amount of time at 
10MB/s.  I need to achieve something closer to 40MB/s to make this a workable 
option.

Tim Gustafson
Baskin School of Engineering
UC Santa Cruz
t...@soe.ucsc.edu
831-459-5354


--
Virtualization is moving to the mainstream and overtaking non-virtualized
environment for deploying applications. Does it make network security 
easier or more difficult to achieve? Read this whitepaper to separate the 
two and get a better understanding.
http://p.sf.net/sfu/hp-phase2-d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Newbie

2010-07-30 Thread Tim Gustafson
Hi all,

I just installed Bacula 5.0.2 on a FreeBSD 8.1 system with about 24TB of disk 
space with the intention of doing my backups straight to disk.  The storage 
array is configured using ZFS, and I've configured Bacula to make backups to 
/bacula/FileStorage.

My question is this: the other day, I re-started the bacula-dir service in the 
middle of a backup, and it seems to have corrupted the FileStorage file.  This 
wasn't a big deal during testing as I could just whack the existing file and 
create a new one, but it got me wondering about the resiliency of FileStorage 
devices during a backup server crash.

Is there anything special I should do to make FileStorage devices more robust?

Also, is it a good idea to label a separate volume for each backup job?

Tim Gustafson
Baskin School of Engineering
UC Santa Cruz
t...@soe.ucsc.edu
831-459-5354


--
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users