Re: [Bacula-users] Bacula 5.2.13 ClientRunAfterJob
Thank you for the replies. Seems that external script is the way to go. Thank you, On Sat, Mar 21, 2015 at 10:22 PM, Davide Franco bacula-...@dflc.ch wrote: Hi Peter, As Kern mentioned, I'd mention a shell script instead of the whole command in the Job definition. I do this on my jobs and it work like a charm. Hope it will help you to solve your problem. Best regards Davide On Sat, Mar 21, 2015 at 8:28 PM, Kern Sibbald k...@sibbald.com wrote: Hello, Bacula does not run a shell when it executes a command as you are doing. Consequently shell characters such as * will be treated as themselves rather than shell characters (in this case a wild card). This is because such interpretation is done by the shell. So the solution is to either prefix the command with a call to a shell (lots of escaping to do) or *much* simpler, run the command in a shell script that you execute. If I am not mistaken, there are nice examples in the manual. Obviously (I hope), if you can be more specific and avoid wild cards you will not need a shell command prefix or a script. I am not patient enough to work out all the escaping to pass shell characters correctly to the shell, so I always use a script when doing complicated stuff. Best regards, Kern On 15-03-19 06:54 AM, Peter Wood wrote: Thank you for the reply. Yes, I've done all the debug steps I can think of. Permissions is a good point though. The file permissions are 600 root:root. Only root can remove the file. Bacula-fd process runs as root so I'm expecting it will be able to remove the file, right? SELinux is disabled. On Thu, Mar 19, 2015 at 4:39 AM, Ana Emília M. Arruda emiliaarr...@gmail.com wrote: Hi Peter, Have you checked if the job falis or not? The Client Run After Jog do not runs if the job falis. You can use the bellow if you want that the script runs despite of the job falis or not: Run Script { RunsWhen = After RunsOnFailure = yes Commnad = /bin/rm -f /backup/daily/mysql-Slave* } Have you tried to run the command at the commnad line? Have you checked permissions? Have you checked any messages at Bacula's log file? Is the mysqldump generating the file at /backup/daily/? Best regards, Ana On Wed, Mar 18, 2015 at 11:54 PM, Peter Wood peterwood...@gmail.com wrote: In Bacula 5.2.13 I use Client Run Before Job = mysqldump to dump the database before the backup starts. After it is complete I use Client Run After Job = /bin/rm -f /backup/daily/mysql-Slave* with the intend to remove the backup file due to lack of free space to keep more then one backup. The backup report shows that it ran the command: 18-Mar 09:39 db1-fd JobId 12980: shell command: run ClientAfterJob /bin/rm -f /backup/daily/mysql-Slave* Unfortunately the file is not removed. Any idea? Is it the use of wildcard? Thanks, -- Peter -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bacula-users mailing listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership
Re: [Bacula-users] Bacula 5.2.13 ClientRunAfterJob
Thank you for the reply. Yes, I've done all the debug steps I can think of. Permissions is a good point though. The file permissions are 600 root:root. Only root can remove the file. Bacula-fd process runs as root so I'm expecting it will be able to remove the file, right? SELinux is disabled. On Thu, Mar 19, 2015 at 4:39 AM, Ana Emília M. Arruda emiliaarr...@gmail.com wrote: Hi Peter, Have you checked if the job falis or not? The Client Run After Jog do not runs if the job falis. You can use the bellow if you want that the script runs despite of the job falis or not: Run Script { RunsWhen = After RunsOnFailure = yes Commnad = /bin/rm -f /backup/daily/mysql-Slave* } Have you tried to run the command at the commnad line? Have you checked permissions? Have you checked any messages at Bacula's log file? Is the mysqldump generating the file at /backup/daily/? Best regards, Ana On Wed, Mar 18, 2015 at 11:54 PM, Peter Wood peterwood...@gmail.com wrote: In Bacula 5.2.13 I use Client Run Before Job = mysqldump to dump the database before the backup starts. After it is complete I use Client Run After Job = /bin/rm -f /backup/daily/mysql-Slave* with the intend to remove the backup file due to lack of free space to keep more then one backup. The backup report shows that it ran the command: 18-Mar 09:39 db1-fd JobId 12980: shell command: run ClientAfterJob /bin/rm -f /backup/daily/mysql-Slave* Unfortunately the file is not removed. Any idea? Is it the use of wildcard? Thanks, -- Peter -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula 5.2.13 ClientRunAfterJob
In Bacula 5.2.13 I use Client Run Before Job = mysqldump to dump the database before the backup starts. After it is complete I use Client Run After Job = /bin/rm -f /backup/daily/mysql-Slave* with the intend to remove the backup file due to lack of free space to keep more then one backup. The backup report shows that it ran the command: 18-Mar 09:39 db1-fd JobId 12980: shell command: run ClientAfterJob /bin/rm -f /backup/daily/mysql-Slave* Unfortunately the file is not removed. Any idea? Is it the use of wildcard? Thanks, -- Peter -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.13 bacula-dir fails to start
Right on. Thanks Martin. After recompiling with python enabled things started working again. On Tue, Oct 21, 2014 at 9:52 AM, Martin Simmons mar...@lispworks.com wrote: On Mon, 20 Oct 2014 15:40:47 -0700, Peter Wood said: I have bacula-5.2.13 running on CentOS6 x86_64. It's been running for an year but a few days ago I had to restart the server and now the bacula-dir errors out: [root@backup bacula]# bin/bacula-dir -d 99 -f bacula-dir: dird.c:223-0 Debug level = 99 bacula-dir: dir_plugins.c:160-0 Load dir plugins bacula-dir: dir_plugins.c:162-0 No dir plugin dir! bacula-dir: postgresql.c:241-0 pg_real_connect done bacula-dir: postgresql.c:243-0 db_user=bacula db_name=bacula db_password=* bin/bacula-dir: symbol lookup error: bin/bacula-dir: undefined symbol: _Z23init_python_interpreterP28init_python_interpreter_args [root@backup bacula]# The other daemons bacula-fd and bacula-sd are starting successfully from what I can tell without having access to the director. Last restart was in Aug and things were working. Since then I upgraded OS packages and installed custom bacula-client package. I suspect that is what broke things but I can't find why. All config files look good and haven't been touched for a long time. Any idea what could be the problem? The Director was built with python, but it looks like your bacula shared libraries (libbac*.so) were built without python. Maybe the shared libraries were reinstalled with the custom bacula-client package? Is it the same version? __Martin -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula 5.2.13 bacula-dir fails to start
I have bacula-5.2.13 running on CentOS6 x86_64. It's been running for an year but a few days ago I had to restart the server and now the bacula-dir errors out: [root@backup bacula]# bin/bacula-dir -d 99 -f bacula-dir: dird.c:223-0 Debug level = 99 bacula-dir: dir_plugins.c:160-0 Load dir plugins bacula-dir: dir_plugins.c:162-0 No dir plugin dir! bacula-dir: postgresql.c:241-0 pg_real_connect done bacula-dir: postgresql.c:243-0 db_user=bacula db_name=bacula db_password=* bin/bacula-dir: symbol lookup error: bin/bacula-dir: undefined symbol: _Z23init_python_interpreterP28init_python_interpreter_args [root@backup bacula]# The other daemons bacula-fd and bacula-sd are starting successfully from what I can tell without having access to the director. Last restart was in Aug and things were working. Since then I upgraded OS packages and installed custom bacula-client package. I suspect that is what broke things but I can't find why. All config files look good and haven't been touched for a long time. Any idea what could be the problem? Thanks, -- Peter -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File volumes and scratch pool
Hi Ben, The settings you recommended yesterday worked and all expired volumes from all pools were moved to Scratch. Thanks. Now all my pools have volumes with data on them in compliance with the backup policy I setup. As soon as a volume expires it gets moved to Scratch. That means the Scratch pool is the only place where Bacula can find usable volumes when needed. Looking at the number of volumes in Scratch I can tell if I have enough for the nightly backup but this is rough estimate. To make sure there will be enough I usually will add 10-15 brand new ones in Scratch. These new volumes in Scratch I consider like safety buffer. Use them if needed but do not break the backup policy. Just because there are empty volumes with no data doesn't mean I want to keep data for longer then the policy defines. If I wanted to, then I would have changed the policy. My volumes are of type files so using new volumes vs recycling expired ones just fills up the file system with old data. It makes it hard to manage and forecast filesystem space needs. I have never understood Bacula's desire to override my policy and insist on preserving data that I already defined as useless. I guess I'm a control freak. Anyway. Thanks for the help. -- Peter On Wed, Oct 1, 2014 at 2:16 PM, Roberts, Ben ben.robe...@gsacapital.com wrote: Hi Peter, Bacula will always prefer to take a new volume (in Append state) rather than reuse an existing one (in the Purged, Recycled state) so as to preserve your backed up data for as long as possible. It will also use volumes in the backup pool and only resort to taking volumes from the Scratch pool when there are none suitable for writing in the pool. I’m not entirely certain which of those two criteria will win out; I think purged volumes in the backup pool will be used before Bacula tries to use an unused volume from the Scratch pool but you may wish to check this. I can understand why you’d want Bacula to reuse old volumes within one of your backup pools first (so as to maximise the number of volumes left in the scratch pool), but can’t see a reason why you’d prefer to reuse recycled volumes already moved back into the scratch pool before unused ones. If you did mean the latter, perhaps you can reply with your reasoning. Ben Roberts *From:* Peter Wood [mailto:peterwood...@gmail.com] *Sent:* 01 October 2014 22:01 *To:* Roberts, Ben *Cc:* bacula-users@lists.sourceforge.net *Subject:* Re: [Bacula-users] File volumes and scratch pool Thank you Ben. I applied the changes and I'll monitor the behavior to confirm I did it right. If I have mix of recycled volumes and brand new, never used volumes in Scratch, which ones Bacula will grab first? I'd really prefer to setup Bacula so it will reuse the old ones instead of grabbing brand new volumes. On Wed, Oct 1, 2014 at 1:07 PM, Roberts, Ben ben.robe...@gsacapital.com wrote: Hi Peter, You need to set “Recycle Pool = Scratch” in your Scratch pool (and make sure you haven’t overridden it in any other poo)l. Note that this setting is applied to the volume when it’s created, so after changing the Scratch pool definition you will need to update all your volumes to re-apply the Recycle pool. It might be easiest to do this by directly modifying the catalog, else you can use a quick bash script to generate a sequence of “update volume…” commands to echo into bconsole. Regards, Ben Roberts *From:* Peter Wood [mailto:peterwood...@gmail.com] *Sent:* 01 October 2014 20:56 *To:* bacula-users@lists.sourceforge.net *Subject:* [Bacula-users] File volumes and scratch pool In Bacula-5.2.13 how do I setup expired volumes to go in the Scratch pool? I create new file type volumes like this: label storage=File volume=vol001 pool=Scratch When needed volumes are taken out of Scratch and assigned to the appropriate pool and used. Once in the pull they are never released back to Scratch after they expire. After retentionperiod ends volumes are reused but only within the pool they have been originally assigned. I see volume properties that may be related but I can't find documentation about them: scratchpoolid: 0 recyclepoolid: 0 Any help in making volumes go in Scratch after retention period expire so they can be reused by any job. Thank you, -- Peter -- This email and any files transmitted with it contain confidential and proprietary information and is solely for the use of the intended recipient. If you are not the intended recipient please return the email to the sender and delete it from your computer and you must not use, disclose, distribute, copy, print or rely on this email or its contents. This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction
[Bacula-users] File volumes and scratch pool
In Bacula-5.2.13 how do I setup expired volumes to go in the Scratch pool? I create new file type volumes like this: label storage=File volume=vol001 pool=Scratch When needed volumes are taken out of Scratch and assigned to the appropriate pool and used. Once in the pull they are never released back to Scratch after they expire. After retentionperiod ends volumes are reused but only within the pool they have been originally assigned. I see volume properties that may be related but I can't find documentation about them: scratchpoolid: 0 recyclepoolid: 0 Any help in making volumes go in Scratch after retention period expire so they can be reused by any job. Thank you, -- Peter -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File volumes and scratch pool
Thank you Ben. I applied the changes and I'll monitor the behavior to confirm I did it right. If I have mix of recycled volumes and brand new, never used volumes in Scratch, which ones Bacula will grab first? I'd really prefer to setup Bacula so it will reuse the old ones instead of grabbing brand new volumes. On Wed, Oct 1, 2014 at 1:07 PM, Roberts, Ben ben.robe...@gsacapital.com wrote: Hi Peter, You need to set “Recycle Pool = Scratch” in your Scratch pool (and make sure you haven’t overridden it in any other poo)l. Note that this setting is applied to the volume when it’s created, so after changing the Scratch pool definition you will need to update all your volumes to re-apply the Recycle pool. It might be easiest to do this by directly modifying the catalog, else you can use a quick bash script to generate a sequence of “update volume…” commands to echo into bconsole. Regards, Ben Roberts *From:* Peter Wood [mailto:peterwood...@gmail.com] *Sent:* 01 October 2014 20:56 *To:* bacula-users@lists.sourceforge.net *Subject:* [Bacula-users] File volumes and scratch pool In Bacula-5.2.13 how do I setup expired volumes to go in the Scratch pool? I create new file type volumes like this: label storage=File volume=vol001 pool=Scratch When needed volumes are taken out of Scratch and assigned to the appropriate pool and used. Once in the pull they are never released back to Scratch after they expire. After retentionperiod ends volumes are reused but only within the pool they have been originally assigned. I see volume properties that may be related but I can't find documentation about them: scratchpoolid: 0 recyclepoolid: 0 Any help in making volumes go in Scratch after retention period expire so they can be reused by any job. Thank you, -- Peter -- This email and any files transmitted with it contain confidential and proprietary information and is solely for the use of the intended recipient. If you are not the intended recipient please return the email to the sender and delete it from your computer and you must not use, disclose, distribute, copy, print or rely on this email or its contents. This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect those of GSA Capital. GSA Capital Partners LLP is authorised and regulated by the Financial Conduct Authority and is registered in England and Wales at Stratton House, 5 Stratton Street, London W1J 8LA, number OC309261. GSA Capital Services Limited is registered in England and Wales at the same address, number 5320529. -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Console - shows conflicting info
Brad, What I'd do is run the bconsole's estimate command to verify the connection. If it is successful you should see a short summary report how many files the job will backup, sizes, and so on. It will not backup anything. If it fails login to the client and run: ps -ef | grep bacula You should see bacula-fd process running. I suspect bacula-fd is not running based on what you've posted so far. If all good so far check your firewall settings. Make sure the client allows connection on tcp/udp port 9102. -- Peter On Mon, Dec 2, 2013 at 10:53 AM, Thomas Lohman thom...@mtl.mit.edu wrote: 25-Nov 13:38 home-server-dir JobId 144: Fatal error: Network error with FD during Backup: ERR=Connection reset by peer 25-Nov 13:38 home-server-dir JobId 144: Fatal error: No Job status returned from FD. 25-Nov 13:38 home-server-dir JobId 144: Error: Bacula home-server-dir 5.2.5 (26Jan12): I am not sure what your exact configuration is but my guess/hunch is that your jobs are being spooled to the server, but while they are then being de-spooled to your volumes, the connection back to the client is cut off for whatever reason (Connection reset by peer error). This I think would explain why the client may in fact think it finished ok but the server doesn't. That is probably technically a bug and not a feature. :) Look at the Bacula Heartbeat Interval option if you are not using this already and see if that helps to keep the connection alive. hope this helps, --tom -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup strategy with Bacula
Thanks all for the help. To use the space more efficiently what I decided to do is use vtapes, limit the file size to 10GB and after a full backup just scp the related files to a server in the office. I still have to do the proof of concept setup but from what I'm reading this should be doable. Bill, great article. Thanks. -- Peter On Thu, Mar 28, 2013 at 12:25 AM, Geert Stappers geert.stapp...@vanadgroup.com wrote: Op 2013-03-27 om 13:12 schreef Paul Mather: On Mar 26, 2013, at 11:44 PM, Bill Arlofski wrote: On 03/26/13 22:33, Wood Peter wrote: Hi, I'm planning to use Bacula to do file level backup of about 15 Linux systems. Total backup size is about 2TB. For Bacula server I'm thinking to buy Dell PE R520 with 24TB internal storage (8x 3TB disks) and use virtual tapes. Hi Peter. First, you're going to want some RAID level on that server and RAID0 is not (IMHO) RAID at all. :) At a minimum you're going to want to set up (for example) RAID5 in the server which will give you a maximum of 7 x 3 = 21 TB since the equivalent of 1 full drive is used for parity, spread across all of the drives in a RAID5 array. Having said that, RAID5 does not have the best write speeds, but other RAID levels that will give more redundancy and better write speeds will use significantly more drives and give you less total storage. You may need to spend some time to consider your redundancy and read/write throughput requirements. Also, you will generally want to configure at least one drive as a hot spare so the RAID controller can automatically and immediately fail over to it in the case of a drive failure in the array. So that takes away at least 3 more TB, so now your down to 18TB total storage with a minimally configured RAID5 array with 1 hot spare. Just some things to consider. :) Another thing to consider is that with large capacity drives (3 TB) combined into large RAID-5 arrays there is an increased likelihood of a catastrophic array failure during an array rebuild due to an initial drive failure. For this reason, RAID-6 would be preferred for such large arrays. Reliability of large RAID arrays is one of the motivations behind raidz3 (triple-parity redundancy) in ZFS. See, e.g., http://queue.acm.org/detail.cfm?id=1670144 for details. Thanks for the advice on RAID configuration. Now we can go back to 24TB internal storage (8x 3TB disks) and use virtual tapes. so (cheap) disks as alternative to backup tapes. Groeten Stappers -- Own the Future-Intelreg; Level Up Game Demo Contest 2013 Rise to greatness in Intel's independent game demo contest. Compete for recognition, cash, and the chance to get your game on Steam. $5K grand prize plus 10 genre and skill prizes. Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Own the Future-Intel(R) Level Up Game Demo Contest 2013 Rise to greatness in Intel's independent game demo contest. Compete for recognition, cash, and the chance to get your game on Steam. $5K grand prize plus 10 genre and skill prizes. Submit your demo by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users