Re: [Bacula-users] disable batch inserts in 2.2.4

2007-12-10 Thread Dane Miller 
Chris Howells wrote:
 Are you running out of space on /tmp, the default location that MySQL 
 stores temporary tables (until they are copied into their final table)? 
 This can happen if you have a reasonably small / (which is pretty common).

No, I don't think this is the problem in my case.  Mysql's tmpdir
is /var/tmp, and has 34G free.

 I have so far used bacula 2.2.5 to backup 15TB and about 100 million 
 files (10GB catalog) without a single batch table problem.

I realize there are people who back up more than I do, and do it without
a problem.  But we all have different backup ecosystems, and *something*
in mine is causing mysql to drop temporary tables.  I see this in the
binlog:

#071207 15:13:04
DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `bacula`.`batch`/*!*/;

So I'd really like to test bacula without the temporary tables feature,
and was hoping for some advice on doing so within the limits of the
FreeBSD port sysutils/bacula-server.

Thanks,

-- 
Dane Miller
Systems Administartor
Greatschools, Inc
http://www.greatschools.net


-
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] disable batch inserts in 2.2.4

2007-12-07 Thread Dane Miller 
Hi,

How can I disable batch inserts when building bacula from FreeBSD ports
sysutils/bacula-server?  make configure didn't give me any choices.

The bacula.batch table consistently disappears during long-running
jobs, which generates lots of sql errors (seems and oft reported
problem).  So I'd like to test without the batch insert feature.

Thanks,
Dane
-- 
Dane Miller
Systems Administartor
Greatschools, Inc
http://www.greatschools.net


-
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Wanted to create Volume, but it already exists

2007-11-26 Thread Dane Miller 
On Fri, 2007-11-16 at 11:38 -0800, Dane Miller wrote:
 I'm doing disk-based backups and having a problem getting bacula to
 recycle my purged volumes.  It looks like bacula moves the purged
 volumes out of their configured pool.  Then when bacula tries to
 recycle
 a volume, it chokes with the warning 'Wanted to create Volume
 clone-Diff-0005, but it already exists. Trying again'.   I'm not
 using
 RecyclePool in my configs. 

Follow-up: I think I found the problem and either fixed it or found a
work-around.  I think I inadvertently moved the volumes out of their
native pool when I used bweb.pl (web interface) to manually purge
volumes.  Somehow the volumes got moved to the 'Default' pool, instead
of their job-specific pool... maybe because the JobDefs specified 'Pool
= Default'... not sure.

So, to be safe, I added Recycle Pool = job-specific-pool to every
pool definition.  Now bacula recycles expired volumes correctly.  Here's
an example.  Hope this helps...

Job {
  Name = clone
  Client = clone-fd
  Storage = cloneFile
  Schedule = WednesdaySchedule
  JobDefs = DefaultJob
  FileSet = RootFS-clone
  Full Backup Pool = clone-Full
  Differential Backup Pool = clone-Diff
}
Pool {
# weekly full backups
  Name = clone-Full
  RecyclePool = clone-Full ==
  Pool Type = Backup
  Maximum Volume Jobs = 1
  Volume Retention = 1 weeks 5 days
  Recycle Oldest Volume = yes
  Label Format = clone-Full-
}
Pool {
# daily differential backups
  Name = clone-Diff
  RecyclePool = clone-Diff ==
  Pool Type = Backup
  Maximum Volume Jobs = 1;
  Volume Retention = 5 days
  Recycle Oldest Volume = yes
  Label Format = clone-Diff-
}
JobDefs {
  Name = DefaultJob
  Type = Backup
  Level = Incremental
  Pool = Default
  FileSet = RootFS
  Schedule = DefaultSchedule
  Messages = Standard
  Write Bootstrap = /var/db/bacula/%c_%n.bsr
  Priority = 10
}

-- 
Dane Miller
Systems Administartor
Greatschools, Inc
http://www.greatschools.net


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Wanted to create Volume, but it already exists

2007-11-16 Thread Dane Miller 
Hi,

I'm doing disk-based backups and having a problem getting bacula to
recycle my purged volumes.  It looks like bacula moves the purged
volumes out of their configured pool.  Then when bacula tries to recycle
a volume, it chokes with the warning 'Wanted to create Volume
clone-Diff-0005, but it already exists. Trying again'.   I'm not using
RecyclePool in my configs.

How do I configure bacula to keep purged volumes in their original pool
so they get reused after the retention period expires?

Here's an example:
zeus-dir: Start Backup JobId 268, Job=clone.2007-11-16_05.05.00
zeus-dir: There are no more Jobs associated with Volume clone-Diff-0005. 
Marking it purged.
zeus-dir: New Pool is: Default
zeus-dir: All records pruned from Volume clone-Diff-0005; marking it Purged
zeus-dir: clone.2007-11-16_05.05.00 Warning: Wanted to create Volume 
clone-Diff-0005, but it already exists. Trying again.

And here's the job definition:
Job {
  Name = clone
  Client = clone-fd
  Storage = cloneFile
  Schedule = WednesdaySchedule
  JobDefs = DefaultJob
  FileSet = RootFS-clone
  Full Backup Pool = clone-Full
  Differential Backup Pool = clone-Diff
}
JobDefs {
  Name = DefaultJob
  Type = Backup
  Level = Incremental
  Pool = Default
  FileSet = RootFS
  Schedule = DefaultSchedule
  Messages = Standard
  Write Bootstrap = /var/db/bacula/%c_%n.bsr
  Priority = 10
}
Pool {
# daily differential backups
  Name = clone-Diff
  Pool Type = Backup
  Maximum Volume Jobs = 1;
  Volume Retention = 5 days
  Recycle Oldest Volume = yes
  Label Format = clone-Diff-
}

And some software details:
Director OS: FreeBSD 6.2-RELEASE
File Daemon OS: FreeBSD 6.1-RELEASE
bacula-dir/sd: 2.2.4
bacula-fd: 2.2.4
catalog: MySQL 5.0

Thanks,
Dane
-- 
Dane Miller
Systems Administartor
Greatschools, Inc
http://www.greatschools.net


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bacula-fd hangs on ClientRunAfterJob

2007-11-01 Thread Dane Miller 
Eric Bollengier wrote:
 IHMO, i think that the script doesn't close stderr, stdout and stdin
 properly.

Hey you're right!  stderr is the culprit here.  The following Client Run
After Job script works around the problem:

#!/bin/sh
# AfterScript.sh
/usr/local/etc/rc.d/mysql-server start 2 /dev/null

It might be an issue with mysql's launcher script mysqld_safe. I'll look
into it a bit more.

But why does this cause bacula-fd to hang?

Dane


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now  http://get.splunk.com/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] bacula-fd hangs on ClientRunAfterJob

2007-10-30 Thread Dane Miller 
Hi,

I added Client Run Before/After Job scripts to two backup jobs in
order to stop/start mysql.  But the After script seems to hang the
bacula-fd. Using 'status dir', the bacula console shows a terminated
status for these jobs, and the command 'list jobs' shows their status as
'R'.

The script being called is the FreeBSD rc script used to start/stop
mysql: /usr/local/etc/rc.d/mysql-server [start|stop].  I've ensured that
this script works when run manually, and that it returns 0.

When I restart the offending bacula-fd's (kill doesn't work; requires
kill -9), the offending jobs finish with errors and the rest of my
queued jobs begin to run.

Any ideas what's causing this?  Suggestions for fixing it?

Here are some details:

Director OS: FreeBSD 6.2-RELEASE
File Daemon OS: FreeBSD 6.1-RELEASE
bacula-dir/sd: 2.2.4
bacula-fd: 2.2.4
catalog: MySQL 5.0

Total # of jobs: 13, of which...
   7 Priority = 10
   5 Priority = 12
   1 Priority = 20

bacula-dir.conf: Director{Maximum Concurrent Jobs=10;...}
bacula-sd.conf: Storage{Maximum Concurrent Jobs=20;...}

Offending Client Run Before/After Job scripts:
  Client Run Before Job = /usr/local/etc/rc.d/mysql-server stop
  Client Run After Job = /usr/local/etc/rc.d/mysql-server start

logged output on the director while job is hung:
   zeus-dir: sql_find.c:134 No Job record found: ERR=
   CMD=SELECT StartTime FROM Job WHERE JobStatus='T' AND Type='B' AND Level='F' 
AND Name='ritin' AND ClientId=5 AND FileSetId=5 ORDER BY StartTime DESC LIMIT 1
   zeus-dir: No prior or suitable Full backup found in catalog. Doing FULL 
backup.
   zeus-dir: Start Backup JobId 31, Job=ritin.2007-10-30_05.05.04
   zeus-dir: Created new Volume ritin-Full-0002 in catalog.
   zeus-dir: Using Device ritinFileStorage
   ritin-fd: ClientRunBeforeJob: run command /usr/local/etc/rc.d/mysql-server 
stop
   ritin-fd: ClientRunBeforeJob: Stopping mysql.
   ritin-fd: ClientRunBeforeJob: Waiting for PIDS: 96261.
   zeus-sd: Labeled new Volume ritin-Full-0002 on device ritinFileStorage 
(/bacula/disk2).
   zeus-sd: Wrote label to prelabeled Volume ritin-Full-0002 on device 
ritinFileStorage (/bacula/disk2)
   zeus-dir: Max Volume jobs exceeded. Marking Volume ritin-Full-0002 as Used.
   ritin-fd:  Disallowed filesystem. Will not descend from / into /dev
   zeus-sd: Job write elapsed time = 00:57:03, Transfer rate = 9.499 M 
bytes/second
   ritin-fd: ClientAfterJob: run command /usr/local/etc/rc.d/mysql-server 
start
   ritin-fd: ClientAfterJob: Starting mysql.


'status dir' output while job on 'ritin' is hung (truncated):

Running Jobs:
 JobId Level   Name   Status
==
31 Fullritin.2007-10-30_05.05.04 has terminated
34 Fullcomdev.2007-10-30_05.05.07 is waiting for higher priority jobs 
to finish
35 Fullcomstag.2007-10-30_05.05.08 is waiting execution


Dane
-- 
Dane Miller
Systems Administrator
Great Schools, Inc
http://greatschools.net


-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now  http://get.splunk.com/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bacula-fd hangs on ClientRunAfterJob

2007-10-30 Thread Dane Miller 
Michael Lewinger wrote:
 Is bacula running as root or bacula user ? bacula user cannot restart 
 mysql.

Thanks Michael.  bacula-fd is running as root.  Also note that it *does*
stop and start mysql, but hangs after-wards (see the job output from my
o.p. below)

Dane

 On 10/30/07, Dane Miller [EMAIL PROTECTED] wrote:
 
  logged output on the director while job is hung:
 zeus-dir: sql_find.c:134 No Job record found: ERR=
 CMD=SELECT StartTime FROM Job WHERE JobStatus='T' AND Type='B' AND 
  Level='F' AND Name='ritin' AND ClientId=5 AND FileSetId=5 ORDER BY 
  StartTime DESC LIMIT 1
 zeus-dir: No prior or suitable Full backup found in catalog. Doing FULL 
  backup.
 zeus-dir: Start Backup JobId 31, Job=ritin.2007-10-30_05.05.04
 zeus-dir: Created new Volume ritin-Full-0002 in catalog.
 zeus-dir: Using Device ritinFileStorage
 ritin-fd: ClientRunBeforeJob: run command 
  /usr/local/etc/rc.d/mysql-server stop
 ritin-fd: ClientRunBeforeJob: Stopping mysql.
 ritin-fd: ClientRunBeforeJob: Waiting for PIDS: 96261.
 zeus-sd: Labeled new Volume ritin-Full-0002 on device 
  ritinFileStorage (/bacula/disk2).
 zeus-sd: Wrote label to prelabeled Volume ritin-Full-0002 on device 
  ritinFileStorage (/bacula/disk2)
 zeus-dir: Max Volume jobs exceeded. Marking Volume ritin-Full-0002 as 
  Used.
 ritin-fd:  Disallowed filesystem. Will not descend from / into /dev
 zeus-sd: Job write elapsed time = 00:57:03, Transfer rate = 9.499 M 
  bytes/second
 ritin-fd: ClientAfterJob: run command /usr/local/etc/rc.d/mysql-server 
  start
 ritin-fd: ClientAfterJob: Starting mysql.



-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now  http://get.splunk.com/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bacula-fd hangs on ClientRunAfterJob

2007-10-30 Thread Dane Miller 
Interesting... I've been testing this manually and have more info.  When
bacula-fd is hung, I can fix it by stopping mysql.  As soon as I stop
mysql, the bacula job completes successfully.  

Also, it doesn't matter if my fileset includes mysql's data directory or
not.  For example, if I set my fileset to only include File
= /tmp/empty.txt, the job still hangs after starting mysql.

What the heck is bacula-fd doing during/after the Client Run After
script executes?

Is anyone else using a Client Run script to stop/start mysql on FreeBSD?


Dane




-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now  http://get.splunk.com/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] howt to split the bacula-dir.conf

2007-10-02 Thread Dane Miller 
John Drescher wrote:
  however, i would prefer to have all these files on the director host,
  and include them based on the job running.
 
  something like:
 
File = /etc/bacula/${jobname}_exclude.conf
 
 Yes, that works and it is in the manual.

I don't think variable expansion works in this context (which is how I
interpreted the example).  Bacula complains:

Fatal error: Cannot open included
file: /usr/local/etc/bacula/filesets/${jobname}_exclude.conf. ERR=No
such file or directory

As you can see, bacula thinks ${jobname} is the literal file name.  The
same thing happens for the % character substitution [1] and the volume
label variables [2].

Using variables to include external files would be a huge convenience.

[1]
http://bacula.org/rel-manual/Configuring_Director.html#character_substitution
[2] http://bacula.org/rel-manual/Variable_Expansion.html
Dane
-- 
Dane Miller
Systems Administrator
Great Schools, Inc
http://greatschools.net


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] concurrent jobs on FreeBSD

2007-10-02 Thread Dane Miller 
Dane Miller wrote:
 Is this what you're suggesting...
 
   bacula-dir.conf:
  Director { Maximum Concurrent Jobs = 10; ... }
  # include jobs from separate files
  @/usr/local/etc/bacula/jobs/clientA.job
  @/usr/local/etc/bacula/jobs/clientB.job
   
   bacula-sd.conf:
  Storage { Maximum Concurrent Jobs = 20; ... }
Device { Name = FileStorageClientA; ... }
Device { Name = FileStorageClientB; ... }
 
   clientX.job:
  Job { Name = clientX; Maximum Concurrent Jobs = 3; ... }
  Client { Name = clientX-fd; Maximum Concurrent Jobs = 3; ... }
  Pool { Name = clientX-Full; Maximum Volume Jobs = 1; ... }
  Pool { Name = clientX-Diff; Maximum Volume Jobs = 1; ... }
Storage { Name = FileClientX; Device = FileStorageClientX; ... }

FYI, this worked.  Jobs with the same priority ran simultaneously this
morning.  Thanks Arno and John for pointing me in the right direction.

Dane


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] concurrent jobs on FreeBSD

2007-10-01 Thread Dane Miller 
I'm trying to backup multiple clients to disk and would like the jobs to
run simultaneously.  Each job writes to a unique volume name (e.g.
clientA-Full-0001, clientB-Diff-0001).

There are many posts on this subject, so I'm sorry for bringing it up
again.  But I'm still having problems after reading the list archives.
My jobs run one-at-a-time, not concurrently.  What am I doing wrong?

Here's what I've got so far...

Director OS: FreeBSD 6.2-RELEASE i386
bacula-dir/bacula-sd: 2.2.4 (built from ports sysutils/bacula-server)
client OS: FreeBSD 6.1-RELEASE i386
catalog: MySQL 5.0.37

bacula-dir.conf:
   Director { Maximum Concurrent Jobs = 10; ... }
   Storage { Maximum Concurrent Jobs = 3; ... }
   # include jobs from separate files
   @/usr/local/etc/bacula/jobs/clientA.job
   @/usr/local/etc/bacula/jobs/clientB.job

bacula-sd.conf:
   Storage { Maximum Concurrent Jobs = 20; ... }

clientX.job:
   Job { Name = clientX; Maximum Concurrent Jobs = 3; ... }
   Client { Name = clientX-fd; Maximum Concurrent Jobs = 3; ... }
   Pool { Name = clientX-Full; Maximum Volume Jobs = 1; ... }
   Pool { Name = clientX-Diff; Maximum Volume Jobs = 1; ... }

bacula-fd.conf (on each client):
   FileDaemon {  Name = clientX-fd; Maximum Concurrent Jobs = 20; ... }



-- 
Dane Miller
Systems Administrator
Great Schools, Inc
http://greatschools.net


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] concurrent jobs on FreeBSD

2007-10-01 Thread Dane Miller 
Arno Lehmann wrote:
 You need one storage device per job.
 
 Think of storage devices as tapes: One drive can only write to one 
 tape at a time. And tape is what Bacula was designed to use.
 
 You can probably easily integrate the necessary definitions into your 
 client-specific settings.

So to get multiple jobs to write to disk simultaneously, I need one
Storage{} resource for each job, each with a unique Device =
parameter?  I thought that was the meaning of 'Maximum Concurrent Jobs'
in the Director's Storage{} resource.

Is this what you're suggesting...

  bacula-dir.conf:
 Director { Maximum Concurrent Jobs = 10; ... }
 # include jobs from separate files
 @/usr/local/etc/bacula/jobs/clientA.job
 @/usr/local/etc/bacula/jobs/clientB.job
  
  bacula-sd.conf:
 Storage { Maximum Concurrent Jobs = 20; ... }
   Device { Name = FileStorageClientA; ... }
   Device { Name = FileStorageClientB; ... }

  clientX.job:
 Job { Name = clientX; Maximum Concurrent Jobs = 3; ... }
 Client { Name = clientX-fd; Maximum Concurrent Jobs = 3; ... }
 Pool { Name = clientX-Full; Maximum Volume Jobs = 1; ... }
 Pool { Name = clientX-Diff; Maximum Volume Jobs = 1; ... }
   Storage { Name = FileClientX; Device = FileStorageClientX; ... }

Thanks for helping,
Dane

-- 
Dane Miller
Systems Administrator
Great Schools, Inc
http://greatschools.net


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users