[Bacula-users] Tape automounts but still requires mount command
Hi there, Im using Bacula 2.2.5 on centos 4.3 (2.6.9-34.ELsmp) with an LTO4 drive and tapes. We upgraded recently from an older bacula (1.36.3) and LTO1. One thing I have noticed now is that when I label or relabel a tape it auto-mounts, reports that the tape has been mounted, requests that the tape be mounted, and then sits there until I manually issue a mount command. Is this a known issue? If so what is the cause / resolution? Any advice would be appreciated! Thanks, Kris -- View this message in context: http://www.nabble.com/Tape-automounts-but-still-requires-mount-command-tp15630725p15630725.html Sent from the Bacula - Users mailing list archive at Nabble.com. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How does bacula determine files to backup in incremental mode
Rainer Koenig wrote: > Hi Arno, > > Am Donnerstag, 21. Februar 2008 12:03 schrieb Arno Lehmann: > >> What it comes down to is that you need to know why the access times >> of the directories are modified. Virus scanners are known to do this >> sometimes, but actually, any process could. > > Yes, I had also clamav installed due to a dependency. I removed it now > and tomorrow I'll see if this solves the problem. > > Thanks for your hint. > > Rainer I think beagle or some other indexing tools could do same "wrong" thing ? I've not tested this. -- Bruno Friedmann [EMAIL PROTECTED] Ioda-Net Sàrl - www.ioda-net.ch 2830 Vellerat - Switzerland Tél : ++41 32 435 7171 Fax : ++41 32 435 7172 gsm : ++41 78 802 6760 C'est Facile et Cool d'Évoluer en ligne : www.cfcel.com - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Tape automounts but still requires mount command
Kris Monstad wrote: > > > Hi there, > > Im using Bacula 2.2.5 on centos 4.3 (2.6.9-34.ELsmp) with an LTO4 drive > and tapes. We upgraded recently from an older bacula (1.36.3) and LTO1. > > > One thing I have noticed now is that when I label or relabel a tape it > auto-mounts, reports that the tape has been mounted, requests that the > tape be mounted, and then sits there until I manually issue a mount > command. > > Is this a known issue? If so what is the cause / resolution? > > Any advice would be appreciated! > > Thanks, > Kris > I should have mentioned - the auto-mount behavior is what I want. The issue is with the automount seemingly failing. Thanks, Kris -- View this message in context: http://www.nabble.com/Tape-automounts-but-still-requires-mount-command-tp15630725p15631024.html Sent from the Bacula - Users mailing list archive at Nabble.com. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Feature Request: Backup and Restore of Windows Encrypted Files through raw encryption functions
Item 1: Backup and Restore of Windows Encrypted Files through raw encryption functions Origin: Michael Mohr, SAG Date: 22 February 2008 Status: What: Make it possible to backup and restore Encypted Files from and to Windows systems without the need to decrypt it by using the raw encryption functions API (see: http://msdn2.microsoft.com/en-us/library/aa363783.aspx) that is provided for that reason by Microsoft. If a file ist encrypted could be examined by evaluating the FILE_ATTRIBUTE_ENCRYTED flag of the GetFileAttributes function. Why:Without the usage of this interface the fd-daemon running under the system account can't read encypted Files because the key needed for the decrytion is missed by them. As a result actually encrypted files are not backed up by bacula and also no error is shown while missing these files. Notes: ./. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Increase the rate of despooling.
Hello, Today is my deadline in the project to increase the speed of writing to disk. But I failed. :-( Iam just asking the same question again, If anybody can give me a lead, it will be very helpful. I will give the complete detail. SYSTEM : Power edge 2650. CPU - 1 RAM - 4GB. OS - SLES 9 SP3. Bacula - 2.2.7. Tape Drive - SCSI LVD Ultrium LTO 3 Autchanger. Database - mysql 4.1 Controller in host -Mega raid controller (perc3) Maximum writing speed with bacula - 7.8 Megabytes/second. Maximum Writing speed with vdump and 1024 block size - 32 Megabytes/second. The tape library device - /dev/sg1 The tape drive - /dev/sg0( But we use /dev/nst0 for taking backup ) The Autochanger company claims the following speed of writing. Data Transfer Rate (native): 80 MBps ( 22.8 GBph ) Data Transfer Rate (compressed): 160 MBps ( 45.5 GBph ) I feel the above is in Megabits/sec? am I correct, please tell me if anybody is sure on this. As I get a speed of 7.8 Megabytes / sec with compression and without compression. The whole bacula set up works fine with our environment, like Storage daemon,file daemon etc. And the backup process is absolutely fine except for the speed. I have changed the daisy chain scsi cable 3 times. No improvement in speed. I have tried changing the terminator also today. The same speed results. The funny thing is I tried without terminator also and everything works fine with same speed. I have tried with enabling spooling ( no raid ). The spooling speed differs from 2 Megabytes/sec to 100 Megabytes/sec but the despooling speed lies exactly at 7.8 megabytes/second. The despooling speed is at 7.8 Megabytes/sec for local file system, mounted file system(tru 64 filesystem mountes in bacula server ) and any file system of bacula client . Does anyone find anything fishy Kindly help... The config iam using are attached. Thanks & Regards, Bhaskaran -Original Message- From: Rathinasamy, Bhaskaran (ext) Sent: Wednesday, February 20, 2008 5:19 PM To: 'John Drescher' Cc: bacula-users@lists.sourceforge.net Subject: RE: [Bacula-users] Increase DeSpooling Speed. Hello John, Many Thanks . I have tested the raw drive speed with vdump and bs=1024, The drive speed is around 35MB/s during vdump. The tape drive is SCSI LVD.I had tried replacing the cable twice.but no improvement in speed through bacula. Does a spool file on raid0 increases the despooling speed significantly. Because my spooling directory is /var/backup and it is not raided and no spool size limit is set. We use a single processor machine with 4 GB memory and the spool folder is ext3. Is your despooling speed with Compression or not? Either software or hardware compression? Thanks & Regards, Bhaskaran -Original Message- From: John Drescher [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 20, 2008 4:55 PM To: Rathinasamy, Bhaskaran (ext) Cc: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Increase DeSpooling Speed. On Feb 20, 2008 10:38 AM, Rathinasamy, Bhaskaran (ext) <[EMAIL PROTECTED]> wrote: > Hello, > > We have a LTO-3 Ultrium Tape Library.Bacula 2.2.7 is implemented in SLES > 9. > > The despooling speed to the tape is 7.8 MB/sec always, even with > hardware compression and no hardware compression. > Is your tape drive scsi? Possiblty the problem is a scsi problem. You can test raw drive speeds with dd something like: time dd if=/dev/sda of=/dev/nst0 bs=1G count=10 If you have hardware compression on do not use /dev/zero because you will get a very unrealistic result. > > Is it possible to increase the despooling speed. > I get despooling speeds of 30 to 45MB/s on a LTO2 autochanger when using a 2GB spool file. My spool folder is currently on its own xfs filesystem and using software raid0. My bacula-sd machine (different machine from the director) is a 2 processor opteron 248 with 4GB of memory. John # # Default Bacula File Daemon Configuration file # # For Bacula release 2.2.7 (24 December 2007) -- suse 9 # # There is not much to change here except perhaps the # File daemon Name to # # # List Directors who are permitted to contact this File daemon # Director { Name = bw1lib1-dir Password = "HLpR5YG1GvgK+90EPvVjN4G2UWI7ff6+qZe80En4SLEI" } # # Restricted Director, used by tray-monitor to get the # status of the file daemon # Director { Name = bw1lib1-mon Password = "i5p7IZaavhq0TshzlUmloXf/dw7jLdUAgReoe11tVG/y" Monitor = yes } # # "Global" File daemon configuration specifications # FileDaemon { # this is me Name = bw1lib1-fd FDport = 9102 # where we listen for the director WorkingDirectory = /home/bw1ratb0/bacula/working Pid Directory = /home/bw1ratb0/bacula/bin/working Maximum Concurrent Jobs = 20 } # Send all messages except skipped files back to Director Messages { Name = Standard director = bw1lib1-dir = all, !skipped, !restored } # # Default Bacula Storage Daemon Configuration file #
Re: [Bacula-users] Bacula Console Not installed properly
> On Thu, 21 Feb 2008 22:09:54 -0700, Glen Vickers said: > > K I think I might have read the strace wrong. > > I found the correct bconsole located at /sbin/bconsole and did the > /sbin/bconsole -c /root/bacula/bin/bconsole.conf > > Same message. The message "bconsole not properly installed" means that you are running the bconsole shell script, but somehow it cannot find the bconsole executable. It looks like you have configured Bacula with sbindir and sysconfdir set to the same directory. Am I correct? That setting works for me, but running make install will still install both bconsole files: the first install is the script and the second is the executable. What is the output of make install for you? __Martin - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Increase the rate of despooling.
You might made these changes and see if your better off. /etc/modprobe.conf options st buffer_kbs=1024 try_direct_io=1 /etc/bacula/bacula-fd.conf FileDaemon { # this is me . Maximum Network Buffer Size = 65536 } /etc/bacula/bacula-sd.conf Device { ... Maximum Block Size = 1048576; Maximum Network Buffer Size = 65536 } On Fri, 2008-02-22 at 13:14 +0100, Rathinasamy, Bhaskaran (ext) wrote: > Hello, > > Today is my deadline in the project to increase the speed of writing to > disk. But I failed. :-( > > Iam just asking the same question again, If anybody can give me a lead, > it will be very helpful. > > I will give the complete detail. > > SYSTEM : Power edge 2650. > CPU - 1 > RAM - 4GB. > OS - SLES 9 SP3. > Bacula - 2.2.7. > Tape Drive - SCSI LVD Ultrium LTO 3 Autchanger. > Database - mysql 4.1 > Controller in host - Mega raid controller (perc3) > Maximum writing speed with bacula - 7.8 Megabytes/second. > Maximum Writing speed with vdump and 1024 block size - 32 > Megabytes/second. > The tape library device - /dev/sg1 > The tape drive - /dev/sg0( But we use /dev/nst0 for taking backup ) > > The Autochanger company claims the following speed of writing. > > Data Transfer Rate (native): 80 MBps ( 22.8 GBph ) > > Data Transfer Rate (compressed): 160 MBps ( 45.5 GBph ) > > I feel the above is in Megabits/sec? am I correct, please tell me if > anybody is sure on this. As I get a speed of 7.8 Megabytes / sec with > compression and without compression. > > The whole bacula set up works fine with our environment, like Storage > daemon,file daemon etc. And the backup process is absolutely fine except > for the speed. > > I have changed the daisy chain scsi cable 3 times. No improvement in > speed. > > I have tried changing the terminator also today. The same speed results. > The funny thing is I tried without terminator also and everything works > fine with same speed. > > I have tried with enabling spooling ( no raid ). The spooling speed > differs from 2 Megabytes/sec to 100 Megabytes/sec but the despooling > speed lies exactly at 7.8 megabytes/second. > > The despooling speed is at 7.8 Megabytes/sec for local file system, > mounted file system(tru 64 filesystem mountes in bacula server ) and any > file system of bacula client . > > Does anyone find anything fishy Kindly help... > > The config iam using are attached. > > Thanks & Regards, > Bhaskaran > > > > > -Original Message- > From: Rathinasamy, Bhaskaran (ext) > Sent: Wednesday, February 20, 2008 5:19 PM > To: 'John Drescher' > Cc: bacula-users@lists.sourceforge.net > Subject: RE: [Bacula-users] Increase DeSpooling Speed. > > Hello John, > Many Thanks . > > I have tested the raw drive speed with vdump and bs=1024, The drive > speed is around 35MB/s during vdump. > > The tape drive is SCSI LVD.I had tried replacing the cable twice.but no > improvement in speed through bacula. > > Does a spool file on raid0 increases the despooling speed significantly. > Because my spooling directory is /var/backup and it is not raided and no > spool size limit is set. > > We use a single processor machine with 4 GB memory and the spool folder > is ext3. > > Is your despooling speed with Compression or not? Either software or > hardware compression? > > > > Thanks & Regards, > Bhaskaran > > -Original Message- > From: John Drescher [mailto:[EMAIL PROTECTED] > Sent: Wednesday, February 20, 2008 4:55 PM > To: Rathinasamy, Bhaskaran (ext) > Cc: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Increase DeSpooling Speed. > > On Feb 20, 2008 10:38 AM, Rathinasamy, Bhaskaran (ext) > <[EMAIL PROTECTED]> wrote: > > Hello, > > > > We have a LTO-3 Ultrium Tape Library.Bacula 2.2.7 is implemented in > SLES > > 9. > > > > The despooling speed to the tape is 7.8 MB/sec always, even with > > hardware compression and no hardware compression. > > > Is your tape drive scsi? Possiblty the problem is a scsi problem. > > You can test raw drive speeds with dd something like: > > time dd if=/dev/sda of=/dev/nst0 bs=1G count=10 > > If you have hardware compression on do not use /dev/zero because you > will get a very unrealistic result. > > > > > Is it possible to increase the despooling speed. > > > I get despooling speeds of 30 to 45MB/s on a LTO2 autochanger when > using a 2GB spool file. My spool folder is currently on its own xfs > filesystem and using software raid0. My bacula-sd machine (different > machine from the director) is a 2 processor opteron 248 with 4GB of > memory. > > John > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ > ___ Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge
Re: [Bacula-users] [Bacula-devel] Feature Request: Backup and Restore of Windows Encrypted Files through raw encryption functions
Thanks, I have added this to the projects file. Kern On Friday 22 February 2008 11.54:43 [EMAIL PROTECTED] wrote: > Item 1: Backup and Restore of Windows Encrypted Files through raw > encryption functions > > Origin: Michael Mohr, SAG > > Date: 22 February 2008 > > Status: > > What: Make it possible to backup and restore Encypted Files from and > to Windows systems > without the need to decrypt it by using the raw encryption > functions API > (see: http://msdn2.microsoft.com/en-us/library/aa363783.aspx) > > that is provided for that reason by Microsoft. > If a file ist encrypted could be examined by evaluating the > FILE_ATTRIBUTE_ENCRYTED flag of the GetFileAttributes > function. > > Why:Without the usage of this interface the fd-daemon running > under the system account can't read encypted Files because > the key needed for the decrytion is missed by them. As a > result > actually encrypted files are not backed up > by bacula and also no error is shown while missing these > files. > > Notes: ./. > > > > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ > ___ > Bacula-devel mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/bacula-devel - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Increase the rate of despooling.
On Fri, 22 Feb 2008, Rathinasamy, Bhaskaran (ext) wrote: > The Autochanger company claims the following speed of writing. > > Data Transfer Rate (native): 80 MBps ( 22.8 GBph ) > > Data Transfer Rate (compressed): 160 MBps ( 45.5 GBph ) > > I feel the above is in Megabits/sec? No, that's MegaBytes/second and it is accurate for LTO4 If you are only achieving 7.8 MBytes/second then there is soemthing seriously wrong with your setup. Have you tried benchmarking tape speeds using btape? LTO has very good hardware compression and I wouldn't evenn bother with software compression as the general experience is that it will slow things up - a LOT. AB - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Increase the rate of despooling.
> Maximum Writing speed with vdump and 1024 block size - 32 > Megabytes/second. > I am not sure what vdump is but if it dumps data to the tape similar to dd you should be getting over 60MB/s for LTO3. With dd I get over 35MB/s with LTO2 and my root partition as the source data so this number still is a problem. John - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How does bacula determine files to backup in incremental mode
On Thu, Feb 21, 2008 at 6:45 AM, Rainer Koenig <[EMAIL PROTECTED]> wrote: > Hi Arno, > > Am Donnerstag, 21. Februar 2008 12:03 schrieb Arno Lehmann: > > > > What it comes down to is that you need to know why the access times > > of the directories are modified. Virus scanners are known to do this > > sometimes, but actually, any process could. > > Yes, I had also clamav installed due to a dependency. I removed it now > and tomorrow I'll see if this solves the problem. > You may need to add the noatime or nodiratime to your mount for your filesystem if you do not already have this. John - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] LTO questions - and answers.
There is a Wikipedia page covering LTO specifications, tape speeds and history. It should probably be in the Bacula FAQ and documentation. http://en.wikipedia.org/wiki/Linear_Tape-Open Note that the quoted native (uncompressed) speeds for LTO3 and LTO4 are 80MBytes/sec and 120MBytes/sec - this is significantly faster than most disk drives can throw data at them and the Wikipedia article points this out. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Redundant Run in Schedule
Hi, On Wednesday 20 February 2008, Bastian Friedrich wrote: > when configuring multiple "Run"s in a schedule that occur at the same time, > they are run sequentially: > Run = Full Pool = PoolSpecial w01 mon at 8:00 > Run = Full Pool = Pool mon at 8:00 > results in two executions of the job that refers to the schedule. > [...] > I'd have a need for exclusiveness in this case: run the job only in case it > is not referenced before. Would you also regard this as a sensible > modification? despite the fact that we found workarounds during our discussion (on bacula-users), I wrote a small patch: With the attached patch, Schedule statements can include an optional "QueueOnce" flag (i.e. QueueOnce=true, with QueueOnce=false being default). If this flag is set, a job will only be queued once per minute. With this enabled, one can e.g. schedule a job "first of month" as well as "mondays"; the job will only be queued once if a monday is a first of a month as well, and both events were queued for the same minute. This modifies the current behavior, where jobs were queued twice. Because of Bacula's internal logic, the _last_ "Run" statement in a schedule wins; we can understand this as "overwriting" statements. What do you think about this idea/code? Oh, btw: the attached patch is against SVN trunk. Thx, Best regards, Bastian -- Collax GmbH . Burkheimer Straße 3 . 79111 Freiburg . Germany p: +49 (0) 761-45684-24 f: +49 (0) 761-45684-10www.collax.com Geschäftsführer: William K. Hite / Boris Nalbach AG München HRB 158898 . Ust.-IdNr: DE 814464942 \ Machine-independent: does not run on any existing machine. Index: src/dird/scheduler.c === --- src/dird/scheduler.c (revision 6461) +++ src/dird/scheduler.c (working copy) @@ -67,7 +67,7 @@ /* Forward referenced subroutines */ static void find_runs(); -static void add_job(JOB *job, RUN *run, time_t now, time_t runtime); +static void add_job(JOB *job, RUN *run, SCHED *sched, time_t now, time_t runtime); static void dump_job(job_item *ji, const char *msg); /* Imported subroutines */ @@ -364,11 +364,11 @@ tm.tm_sec = 0; /* zero secs */ runtime = mktime(&tm); if (run_now) { - add_job(job, run, now, runtime); + add_job(job, run, sched, now, runtime); } /* If job is to be run in the next hour schedule it */ if (run_nh) { - add_job(job, run, now, runtime + 3600); + add_job(job, run, sched, now, runtime + 3600); } } } @@ -377,7 +377,7 @@ Dmsg0(dbglvl, "Leave find_runs()\n"); } -static void add_job(JOB *job, RUN *run, time_t now, time_t runtime) +static void add_job(JOB *job, RUN *run, SCHED *sched, time_t now, time_t runtime) { job_item *ji; bool inserted = false; @@ -397,6 +397,32 @@ #endif return; } + +/* + * If schedule permits only one execution per minute ("QueueOnce" set), + * search for existing queue entries within one minute. Drop if found. + */ +if (sched->queueonce) { + foreach_dlist(ji, jobs_to_run) { + if (!strcmp(ji->job->hdr.name, job->hdr.name) && +(abs(ji->runtime - runtime) < 60)) { + + char dt[50], dt1[50], dt2[50]; + bstrftime_nc(dt, sizeof(dt), ji->runtime); + bstrftime_nc(dt1, sizeof(dt1), runtime); + Dmsg5(dbglvl, "Duplicate queue entry for job %s, QueueOnce enabled. " + "Queued runtime=%s(%x), dropped runtime = %s(%x).\n", + job->hdr.name, + dt, + ji->runtime, + dt1, + runtime); + fflush(stdout); + return; + } + } +} + #ifdef SCHED_DEBUG Dmsg4(000, "Add: Job=\"%s\" run=%x last_run=%x now=%x\n", job->hdr.name, runtime, run->last_run, now); Index: src/dird/dird_conf.c === --- src/dird/dird_conf.c (revision 6454) +++ src/dird/dird_conf.c (working copy) @@ -347,6 +347,7 @@ {"name", store_name, ITEM(res_sch.hdr.name), 0, ITEM_REQUIRED, 0}, {"description", store_str, ITEM(res_sch.hdr.desc), 0, 0, 0}, {"run", store_run, ITEM(res_sch.run), 0, 0, 0}, + {"queueonce", store_bool, ITEM(res_sch.queueonce), 0, ITEM_DEFAULT, false}, {NULL, NULL, {0}, 0, 0, 0} }; Index: src/dird/dird_conf.h === --- src/dird/dird_conf.h (revision 6454) +++ src/dird/dird_conf.h (working copy) @@ -492,6 +492,8 @@ RES hdr; RUN *run; + + bool queueonce; }; /* - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforg
[Bacula-users] Incremental promotion to full for queued jobs
I have the following schedule at a site I deal with: Full on the first saturday of every month. Incremental on non-full saturdays plus sun-fri. The problem: When the first incremental backup is run, Bacula promotes it to a full as expected since there is no prior full. However, this backup is roughly 1TB being compressed and written to an NFS share at approx. 1.5MB/s. Thus, the full backup takes over 48 hours to complete. While that is running, the Bacula scheduler kicks-in on the next day and the day after, queueing-up those day's backups. However, because the full that those backups depend on is still running and has not been committed to the catalog, those jobs are likewise promoted to fulls. When the initial backup completes, it is immediately followed by 2 more full backups which exceeds the space available. Although this can be worked around by temporarily disabling the incremental schedules, it is not ideal since the problem will only occur again when the 50-day retention period is exceeded. Is it possible for Bacula to determine whether or not an incremental job should be promoted to a full at the time the job starts running rather than at the time the job gets queued by the scheduler? That would solve my problem since any incremental jobs that were queued while their dependent full was still running would not get promoted to fulls themselves. I suspect this is a corner-case since backups, even fulls, that take longer than 24 hours (let alone 48 hours) are rare. --PLB - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Increase the rate of despooling.
Hello All, Hurrah!Completed my job right at the deadline I tried changing the SCSI adapter to a new one... And bingo the despooling speed is 60 MB/sec to 80 MB/sec. Many thanks to all who replied, and special thanks to Eric And John, who made me think of changing the terminator and scsi card. And iam very sorry for doubting a lot on bacula for past 3 days. Thanks a lot. Bhaskaran -Original Message- From: Jason A. Kates [mailto:[EMAIL PROTECTED] Sent: Friday, February 22, 2008 3:02 PM To: Rathinasamy, Bhaskaran (ext) Cc: John Drescher; bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Increase the rate of despooling. You might made these changes and see if your better off. /etc/modprobe.conf options st buffer_kbs=1024 try_direct_io=1 /etc/bacula/bacula-fd.conf FileDaemon { # this is me . Maximum Network Buffer Size = 65536 } /etc/bacula/bacula-sd.conf Device { ... Maximum Block Size = 1048576; Maximum Network Buffer Size = 65536 } On Fri, 2008-02-22 at 13:14 +0100, Rathinasamy, Bhaskaran (ext) wrote: > Hello, > > Today is my deadline in the project to increase the speed of writing to > disk. But I failed. :-( > > Iam just asking the same question again, If anybody can give me a lead, > it will be very helpful. > > I will give the complete detail. > > SYSTEM : Power edge 2650. > CPU - 1 > RAM - 4GB. > OS - SLES 9 SP3. > Bacula - 2.2.7. > Tape Drive - SCSI LVD Ultrium LTO 3 Autchanger. > Database - mysql 4.1 > Controller in host - Mega raid controller (perc3) > Maximum writing speed with bacula - 7.8 Megabytes/second. > Maximum Writing speed with vdump and 1024 block size - 32 > Megabytes/second. > The tape library device - /dev/sg1 > The tape drive - /dev/sg0( But we use /dev/nst0 for taking backup ) > > The Autochanger company claims the following speed of writing. > > Data Transfer Rate (native): 80 MBps ( 22.8 GBph ) > > Data Transfer Rate (compressed): 160 MBps ( 45.5 GBph ) > > I feel the above is in Megabits/sec? am I correct, please tell me if > anybody is sure on this. As I get a speed of 7.8 Megabytes / sec with > compression and without compression. > > The whole bacula set up works fine with our environment, like Storage > daemon,file daemon etc. And the backup process is absolutely fine except > for the speed. > > I have changed the daisy chain scsi cable 3 times. No improvement in > speed. > > I have tried changing the terminator also today. The same speed results. > The funny thing is I tried without terminator also and everything works > fine with same speed. > > I have tried with enabling spooling ( no raid ). The spooling speed > differs from 2 Megabytes/sec to 100 Megabytes/sec but the despooling > speed lies exactly at 7.8 megabytes/second. > > The despooling speed is at 7.8 Megabytes/sec for local file system, > mounted file system(tru 64 filesystem mountes in bacula server ) and any > file system of bacula client . > > Does anyone find anything fishy Kindly help... > > The config iam using are attached. > > Thanks & Regards, > Bhaskaran > > > > > -Original Message- > From: Rathinasamy, Bhaskaran (ext) > Sent: Wednesday, February 20, 2008 5:19 PM > To: 'John Drescher' > Cc: bacula-users@lists.sourceforge.net > Subject: RE: [Bacula-users] Increase DeSpooling Speed. > > Hello John, > Many Thanks . > > I have tested the raw drive speed with vdump and bs=1024, The drive > speed is around 35MB/s during vdump. > > The tape drive is SCSI LVD.I had tried replacing the cable twice.but no > improvement in speed through bacula. > > Does a spool file on raid0 increases the despooling speed significantly. > Because my spooling directory is /var/backup and it is not raided and no > spool size limit is set. > > We use a single processor machine with 4 GB memory and the spool folder > is ext3. > > Is your despooling speed with Compression or not? Either software or > hardware compression? > > > > Thanks & Regards, > Bhaskaran > > -Original Message- > From: John Drescher [mailto:[EMAIL PROTECTED] > Sent: Wednesday, February 20, 2008 4:55 PM > To: Rathinasamy, Bhaskaran (ext) > Cc: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Increase DeSpooling Speed. > > On Feb 20, 2008 10:38 AM, Rathinasamy, Bhaskaran (ext) > <[EMAIL PROTECTED]> wrote: > > Hello, > > > > We have a LTO-3 Ultrium Tape Library.Bacula 2.2.7 is implemented in > SLES > > 9. > > > > The despooling speed to the tape is 7.8 MB/sec always, even with > > hardware compression and no hardware compression. > > > Is your tape drive scsi? Possiblty the problem is a scsi problem. > > You can test raw drive speeds with dd something like: > > time dd if=/dev/sda of=/dev/nst0 bs=1G count=10 > > If you have hardware compression on do not use /dev/zero because you > will get a very unrealistic result. > > > > > Is it possible to increase the despooling speed. > > > I get d
Re: [Bacula-users] Bat under windows ?
Bruno Friedmann wrote: > Hi all, > > I'm asking myself if there's a simple way ( I've no win32 compiler tools ) to > get bat running under windows ? > It's not really for production use, but for demo purpose. > > I've read that pgm made under qt4 could run under win32 ( even without the > trolltech licence cost ). > > Is there any work in progress along there ? > > Thanks. > I second this, after seeing Kern demo bat at the ukuug seminar a few days ago, it looks really good. Btw, the mysql plugin looks great too! Keep up the good work! Cheers Simon - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] file daemon support for Windows 2003 64-bit
Can anyone confirm whether the file daemon (client) runs on Windows 64-bit, and if so, what if any restrictions there are? - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] file daemon support for Windows 2003 64-bit
> Can anyone confirm whether the file daemon (client) runs on Windows 64-bit, > and if so, what if any restrictions there are? > I can confirm that it works for XP 64 bit. John - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How does bacula determine files to backup in incremental mode
> On Fri, 22 Feb 2008 09:37:31 -0500, John Drescher said: > > On Thu, Feb 21, 2008 at 6:45 AM, Rainer Koenig > <[EMAIL PROTECTED]> wrote: > > Hi Arno, > > > > Am Donnerstag, 21. Februar 2008 12:03 schrieb Arno Lehmann: > > > > > > > What it comes down to is that you need to know why the access times > > > of the directories are modified. Virus scanners are known to do this > > > sometimes, but actually, any process could. > > > > Yes, I had also clamav installed due to a dependency. I removed it now > > and tomorrow I'll see if this solves the problem. > > > You may need to add the > > noatime or nodiratime to your mount for your filesystem if you do not > already have this. Why? Bacula doesn't use the atime when deciding about backups. __Martin - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Volume recycled
> > > > > If you don't prefer to use bscan etc, and have just recycled the tape in > the > catalog, maybe you still have a backup of your catalog that was stored > before the recycle (and that still exists in the catalog)? > > In that case, you could: > - restore the database dump file from your later backups > - rebuild another catalog database from your catalog backup > - temporarily start using the new catalog (console use command) > - make the restores you need > - resume to original catalog (eg. 'use bacula') > - drop the temporary database > > Of course, you may also feel more free to experiment with bscan, if you > first run the make_catalog_backup script. It generates an ascii dump file > that you can always use to restore your catalog, if you screw things with > eg. bscan. I understood the explanations, but this is my situation: +-+---+---+-+--+--+--+-+--+---+---+-+ | MediaId | VolumeName| VolStatus | Enabled | VolBytes | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-+---+---+-+--+--+--+-+--+---+---+-+ | 34 | Full_App_0001 | Recycle | 1 | 1| 0| 7776000 | 1 | 0| 0 | File | 2007-12-02 01:19:11 | | 51 | Full_App_0002 | Error | 1 | 0| 0| 7776000 | 1 | 0| 0 | File | 2007-10-07 00:55:23 | | 52 | Full_App_0003 | Error | 1 | 0| 0| 7776000 | 1 | 0| 0 | File | 2007-10-07 00:55:23 | +-+---+---+-+--+--+--+-+--+---+---+-+ +-+---+---+-+--+--+--+-+--+---+---+-+ | MediaId | VolumeName| VolStatus | Enabled | VolBytes | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-+---+---+-+--+--+--+-+--+---+---+-+ | 14 | Diff_App_0001 | Used | 1 | 1819260 | 0| 2332800 | 1 | 0| 0 | File | 2007-12-09 00:11:44 | | 38 | Diff_App_0002 | Recycle | 1 | 1| 0| 2332800 | 1 | 0| 0 | File | 2007-10-21 00:07:09 | | 42 | Diff_App_0003 | Used | 1 | 82219626 | 0| 2332800 | 1 | 0| 0 | File | 2007-11-18 00:09:31 | | 46 | Diff_App_0004 | Used | 1 | 82326133 | 0| 2332800 | 1 | 0| 0 | File | 2007-11-25 00:13:39 | +-+---+---+-+--+--+--+-+--+---+---+-+ +-+--+---+-+--+--+--+-+--+---+---+-+ | MediaId | VolumeName | VolStatus | Enabled | VolBytes | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-+--+---+-+--+--+--+-+--+---+---+-+ | 3 | Inc_App_0001 | Used | 1 | 156419 | 0| 604800 | 1 | 0| 0 | File | 2007-12-08 00:15:32 | | 9 | Inc_App_0002 | Recycle | 1 | 1| 0| 604800 | 1 | 0| 0 | File | 2007-12-05 00:10:27 | | 18 | Inc_App_0003 | Used | 1 | 158094 | 0| 604800 | 1 | 0| 0 | File | 2007-12-07 00:09:55 | | 22 | Inc_App_0004 | Used | 1 | 270926 | 0| 604800 | 1 | 0| 0 | File | 2007-12-11 00:11:34 | | 26 | Inc_App_0005 | Used | 1 | 1731305 | 0| 604800 | 1 | 0| 0 | File | 2007-12-06 00:09:34 | | 30 | Inc_App_0006 | Used | 1 | 335207 | 0| 604800 | 1 | 0| 0 | File | 2007-12-10 00:14:26 | +-+--+---+-+--+--+--+-+--+---+---+-+ Where in the documentation I can understand the meaning of each field that table? Do I have a chance of recovering something? -- Eduardo Júnior GNU/Linux user #423272 :wq - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/__
Re: [Bacula-users] How does bacula determine files to backup in incremental mode
> Why? Bacula doesn't use the atime when deciding about backups. > I stand corrected. John - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Volume Data Error
I need some help. I have never had a problem like this before and have done lots of restores in the past. I'm trying to do a restore, and I get errors like: 22-Feb 10:53 bork-sd: restore.2008-02-22_10.53.24 Error: block.c:317 Volume data error at 3:2491723325! Block checksum mismatch in block=238354 len=64512: calc=de16ed0e blk=bce75d60 22-Feb 10:53 bork-dir: restore.2008-02-22_10.53.24 Error: Bacula 2.0.3 (06Mar07): 22-Feb-2008 10:53:29 I really really really need to restore this data. Is there any way around this? Is there any way to repair that volume? What in the world happened, and how do I know that my other volumes have not been compromised as well?? Desperately Needing Help, Misty - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Volume Data Error
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Misty Stanley-Jones Sent: Friday, February 22, 2008 10:58 AM To: Bacula-users@lists.sourceforge.net Subject: [Bacula-users] Volume Data Error I need some help. I have never had a problem like this before and have done lots of restores in the past. I'm trying to do a restore, and I get errors like: 22-Feb 10:53 bork-sd: restore.2008-02-22_10.53.24 Error: block.c:317 Volume data error at 3:2491723325! Block checksum mismatch in block=238354 len=64512: calc=de16ed0e blk=bce75d60 22-Feb 10:53 bork-dir: restore.2008-02-22_10.53.24 Error: Bacula 2.0.3 (06Mar07): 22-Feb-2008 10:53:29 I really really really need to restore this data. Is there any way around this? Is there any way to repair that volume? What in the world happened, and how do I know that my other volumes have not been compromised as well?? Desperately Needing Help, Misty ADDITIONAL INFO: 1. These are disk-based volumes, not tapes. 2. I was able to get the data I needed by avoiding the full backup volume and extracting all the differential and incremental ones on top of each other. Not ideal but I'm not in emergency mode. So somehow my FULL backup volume got corrupted, and that's about the worst thing that can happen. Any ideas why this happened? - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula-fd on HP-UX/AIX
Download and bootstrap pkgsrc[.org] for AIX and HFUX. Send me (and [EMAIL PROTECTED]) and build errors, your config.log, and uname -a output and I will ensure that they are processed acccordingly ~BAS On Wed, 2008-01-23 at 12:23 -0800, Hemant Shah wrote: > I am also trying to compile it on AIX 5.3 and I am not having any > luck. > I would like to replace our veritas netbackup system with bacula. I > was -- Brian A. Seklecki <[EMAIL PROTECTED]> Collaborative Fusion, Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Knocking out cancelled job from the director?
On Fri, 2008-01-25 at 10:19 +0100, Jesper Krogh wrote: > > I have a job sitting in the director as "cancelled", it is occupying one > > of the two concurrent jobs the director is allowed to run but is not > > present in either the storage daemon or the corresponding file-daemon. > > > > Has been haning for about 1.5 hours now. Long outstanding issue. ~BAS > And then it came through by itself: > 25-Jan 07:57 bacula-sd JobId 12892: Job Genome_Daily.2008-01-25_01.06.21 > marked to be canceled. > 25-Jan 08:10 bacula-sd JobId 12892: Job Genome_Daily.2008-01-25_01.06.21 > marked to be canceled. > 25-Jan 08:10 bacula-sd JobId 12892: Fatal error: append.c:159 Error > reading data header from FD. ERR=Connection reset by peer > 25-Jan 08:10 bacula-sd JobId 12892: Job write elapsed time = 00:21:35, > Transfer rate = 1.044 M bytes/second > 25-Jan 09:49 bacula-dir JobId 12892: Fatal error: Network error with FD > during Backup: ERR=Connection reset by peer > 25-Jan 09:49 bacula-dir JobId 12892: Fatal error: No Job status returned > from FD > > Jesper -- Brian A. Seklecki <[EMAIL PROTECTED]> Collaborative Fusion, Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] delete job records from pool/volume
Dan Langille schrieb: > On Feb 19, 2008, at 8:46 AM, Gunnar Thielebein wrote: > >> Hi group :-) >> >> This is a short one: >> >> Is there the possibility to delete job records only for a specific >> pool/volume? Apart of sql magic ;-) > > > > Why would you want to do that? What is the objective? To delete the > Volume? > > To delete the pool? > > I get the feeling you have an objective in mind and you think the > answer to the above > question will get you there. What is that objective? > just saw that I send my first answer directly to dave... meanwhile I found a solution with "purge jobs volume" - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Redundant Run in Schedule
On Friday 22 February 2008 15.44:56 Bastian Friedrich wrote: > Hi, > > On Wednesday 20 February 2008, Bastian Friedrich wrote: > > when configuring multiple "Run"s in a schedule that occur at the same > > time, they are run sequentially: > > Run = Full Pool = PoolSpecial w01 mon at 8:00 > > Run = Full Pool = Pool mon at 8:00 > > results in two executions of the job that refers to the schedule. > > [...] > > > I'd have a need for exclusiveness in this case: run the job only in case > > it is not referenced before. Would you also regard this as a sensible > > modification? > > despite the fact that we found workarounds during our discussion (on > bacula-users), I wrote a small patch: > > With the attached patch, Schedule statements can include an > optional "QueueOnce" flag (i.e. QueueOnce=true, with QueueOnce=false being > default). If this flag is set, a job will only be queued once per minute. > With this enabled, one can e.g. schedule a job "first of month" as well > as "mondays"; the job will only be queued once if a monday is a first of a > month as well, and both events were queued for the same minute. > > This modifies the current behavior, where jobs were queued twice. > > Because of Bacula's internal logic, the _last_ "Run" statement in a > schedule wins; we can understand this as "overwriting" statements. > > What do you think about this idea/code? Well, I like the idea (the feature), but I'm a little concerned about the solution you propose from several stand points: 1. I found it hard to understand what you wanted to do until I read the code, so I am concerned that this concept could be hard to document correctly. 2. The current list of Run directives are essentially ANDed. That is Bacula will walk down the list and schedule all that apply. I think what you really are trying to do is to setup two Run directives to be ORed, and perhaps that could be handled by a slightly different syntax such as: Or { Run = Level=Full Pool = PoolSpecial w01 mon at 8:00 Run = Level=Full Pool = Pool mon at 8:00 } or perhaps some other keyword such as OneOf ... wouldn't that be much clearer? There would be a bit more work to implement this (not really hard) but it seems to me it would be much clearer to the user. 3. It is interesting that this comes just at this moment, because just tonight I was starting to work on the new "Duplicate Jobs" directive group for Jobs. That is a fairly comprehensive set of directives that tell Bacula how to deal with duplicate jobs. This too could also be a solution to the problem somewhat like #1, but much less elegant or explict than my proposal #2. That is if you do not allow duplicate jobs of the same level, using the current syntax, you could put both Run statements, and if a second one started, it would simply be killed -- a bit kludgie, but it would work nicely. Comments? Best regards, Kern > > Oh, btw: the attached patch is against SVN trunk. > > Thx, Best regards, >Bastian - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users