Re: [Bacula-users] Bacula BLOCKED waiting for mount
On 2015-08-03 08:45, Michael Schwager wrote: Under v. 5.x, I discovered that Bacula's tape usage algorithm went something like this: Use an eligible tape from the library. I discovered (I think) that I could use any tape that met Bacula's rewritable criteria. Now, I don't know what the algorithm is. I had 5.x go from 001_010 to 002_010, then 003_010, etc. even though there were 5 more blank tapes in magazine 001 and 15 blank tapes in each 002, 003, etc. So I'm pretty sure Bacula's tape selection algorithm is unpredictable -- at least to an end-user who isn't Kern. You just don't get hit by it very often. Dima -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Out of virtual tapes
Yes, I think so, for daily full backups. I have 5 years retentions for my full backups, but they are full monthly backups. For daily and weekly backups, we use incremental and differential types. Regards, Ana On Mon, Aug 3, 2015 at 6:19 PM, Dimitri Maziuk dmaz...@bmrb.wisc.edu wrote: On 08/03/2015 04:07 PM, Ana Emília M. Arruda wrote: Hello Robert, Yes, you should rethink your pool/volume configurations, specially volumes and pool sizes regarding the amount of space you have to accommodate them. You have a very high retention period. 365 days is high? There are places where 3650 is a legally-mandated minimum. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
On Sun, Aug 2, 2015 at 5:43 PM, Michael Schwager mschwa...@mochotrading.com wrote: Starngely, we have another tape that SHOULD be available. Tapes in this pool are set to recycle after 90 days, and 90 days ago from today is May 4th. We have a tape that was last written to on March 28. So why would Bacula block on the unavailable tape? To answer my own question (with help from the other denizens of the Bacula-cave): A couple of months ago I needed to do a restore, but I had set some of the retention times too short. So the files for my Job were not in the database any longer. I scanned the tape and got the data back into the database. I'm not exactly sure what I'd done or why it broke, but I can tell you that the JobMedia database rows included references to JobId 687, and that was the ID of a job that contained my file. In any case, I'm sure these old references were what caused Bacula to not be able to mark the tape as recyclable, and it thought the tape was in use, so it requested the next tape in line (EPW681L3). I'm beginning to learn about the reasons why behind the warnings not to purge, and to let Bacula manage everything, and so on. At this point I think my Bacula is where I want it and I can move forward. Thank you all for your help, suggestions, and patience. *- Mike Schwager* * Linux Network Engineer, Mocho Trading LLC* * 312-646-4783 Phone312-637-0011 Cell312-957-9804 Fax* -- This message is for the named person(s) use only. It may contain confidential proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Mocho Trading LLC reserves the right to monitor all e-mail communications through its networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] bacula_t selinux problem
Hello, It realize later versions of selinux causes problems with restoring from Bacula, and that there are various methods to get around the problem. However, I notice there are also stat messages like those shown below: 03-Aug 14:06 host JobId 123952: Could not stat /var/spool/postfix/public/showq: ERR=Permission denied 03-Aug 14:06 host JobId 123952: Could not stat /var/spool/postfix/public/cleanup: ERR=Permission denied 0 03-Aug 14:06 host JobId 123952: Could not stat /var/run/rpcbind.sock: ERR=Permission denied 03-Aug 14:06 host JobId 123952: Could not stat /var/run/acpid.socket: ERR=Permission denied 03-Aug 14:06 host JobId 123952: Could not stat /var/run/dbus/system_bus_socket: ERR=Permission denied Many (if not all) have the permissions srw-rw-rw- on the object. With bacula_t set to permissive, the errors don't seem to occur. Besides the restore problem which I can get around and the one above, are there any other problems with leaving bacula_t enforced? Is the problem above something to be worried about? I rather leave bacula_t enforced if possible (unless it is not recommended). Thanks in advance, -craig -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup restore
On 4/08/2015 2:41 PM, More, Ankush wrote: Hi Team, I want to restore backup of one Linux and window client to different Linux and Window server for testing. Is it possible? If yes then what command I should use or need any configuration changes? Can I initiate restore from client? Yes. Read the manual, it is all explained. Cheers, GaryB-) -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Backup restore
Hi Team, I want to restore backup of one Linux and window client to different Linux and Window server for testing. Is it possible? If yes then what command I should use or need any configuration changes? Can I initiate restore from client? Thank you, Ankush More This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
On 08/03/2015 03:35 PM, Michael Schwager wrote: In my case, there are Jobs associated with the MediaId in the JobMedia database table, but this JobId does not exist in the Job table. This means there should be a foreign key on jobmedia: jobid references job (jobid) on delete cascade. Looking at my 7.0.5/postgres 9.2, there should also be mediaid references media (mediaid) on delete cascade. It does explain it: in my case I'm pretty sure it happened after something went pear-shaped and a running backup crashed or otherwise ended abnormally. Presumably leaving some of the tables updated but not the others. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Out of virtual tapes
On 08/03/2015 04:07 PM, Ana Emília M. Arruda wrote: Hello Robert, Yes, you should rethink your pool/volume configurations, specially volumes and pool sizes regarding the amount of space you have to accommodate them. You have a very high retention period. 365 days is high? There are places where 3650 is a legally-mandated minimum. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
Thanks. I will look at the max time settings. Regarding setting the enabled field, I think rather than that, I will use list nextvol, and make sure the changer is populated with the proper tape. This should keep Bacula satisfied. I can even query the database and look for some of the other tapes that should be coming available, if I find that Bacula will likely write to more than one volume. *- Mike Schwager* * Linux Network Engineer, Mocho Trading LLC* * 312-646-4783 Phone312-637-0011 Cell312-957-9804 Fax* On Mon, Aug 3, 2015 at 2:15 PM, Bill Arlofski waa-bac...@revpol.com wrote: On 08/03/2015 09:45 AM, Michael Schwager wrote: Kern, Thanks for the reply from The Man himself :-) . I ran update slots, and it's true that EPW681L3 is not in the library. But that's as I planned it it: Why would it go to EPW681L3 when EPW680L3 is right there in the library? It is in the same pool, it is as full as EPW681L3 once was, it's older than the file/job retention period, but somehow Bacula chose 681 instead of 680 which is what I was expecting. As a matter of fact, I marked all the eligible tapes as full: First I marked 681 full then Bacula chose 682. So I marked that one as full. Now 683 and beyond are younger than the file/job retention periods, so Bacula thinks it has no tapes. Meanwhile I'm pulling my hair out because I'm saying, Bacula- 680 is *right there*! Use it! But no joy. Can I somehow mark a tape as usable? Under v. 5.x, I discovered that Bacula's tape usage algorithm went something like this: Use an eligible tape from the library. I discovered (I think) that I could use any tape that met Bacula's rewritable criteria. Now, I don't know what the algorithm is. This means it will be difficult for me to choose the proper tape to put in the library. I need to know before the weekend starts so it doesn't hang up my backups. Hi Michael, You might try setting the enabled field to '0' for all volumes which are - or should be - inaccessible (not in library), and therefore should not be considered by Bacula as viable. I had to implement this on my system a couple years back, and it has been working fine ever since: http://revpol.com/node/146 I wrote that script to work out this same problem I was having when vchanger drives were not in the drive caddy, and hence the file volumes on them were not available. You will surely have to adapt it, or probably just pick a few things from it for your specific system because it is highly vchanger v0.8.6 centric. :) Having said that, it is typically best to let Bacula manage what volumes it wants, when it wants them, but in my situation, each hard drive I use with vchanger has about 70 10GB volumes so I am not severely limiting Bacula, just guiding it a little and giving it a nudge. e.g.: I do not touch retention times, nor do I manually force purges etc. :) Also, regarding my other question: How do I make Bacula fail hard if it doesn't find the media it needs? I have used the various job max ... time settings for things like this in the past. Bill -- Bill Arlofski http://www.revpol.com/bacula -- Not responsible for anything below this line -- https://lists.sourceforge.net/lists/listinfo/bacula-users -- This message is for the named person(s) use only. It may contain confidential proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Mocho Trading LLC reserves the right to monitor all e-mail communications through its networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Backup finished, but Fatal error: Network error with FD/Connection reset by peer
Hello, we use bacula now for about 6 years and it works great. Some 6 month ago we switched to another Bacula Server. The switch included changing the OS from Linux to FreeBSD (to get better flexibility with ZFS, etc.). Since the move we experience some problems. I keep logs for about 3 month so right now I can say that for about 3100 Backup Jobs 11 Jobs fail with: 02-Aug 09:13 backupserver-dir JobId 110334: Fatal error: Network error with FD during Backup: ERR=Connection reset by peer 02-Aug 09:13 backupserver-dir JobId 110334: Error: Bacula backupserver-dir 5.2.12 (12Sep12): But it seems the backup has finished just fine. After changing the the volume status from Error to Used we can restore files. It seems that after the backup is finished, a communication attempt between the director and the client fails somehow. All those clients are in the same LAN Network. The backup time is comparable, also the amount of files etc. It *seems* to only affect Windows, but I can not verify this fact as I do not have logs beyond 3 month. I read some-where that there could be problems with some sort of timeout in the FreeBSD network stack, but before twiddling with some knobs I really would appreciate if someone else had similar problems in the past and knows what the root cause is. Here an example of one failed (02. Aug) and two success backups from the same job over the last 3 weeks: ERROR: 02-Aug 09:13 backupserver-dir JobId 110334: Fatal error: Network error with FD during Backup: ERR=Connection reset by peer 02-Aug 09:13 backupserver-dir JobId 110334: Error: Bacula backupserver-dir 5.2.12 (12Sep12): Scheduled time: 01-Aug-2015 15:03:01 Start time: 01-Aug-2015 22:08:09 End time: 02-Aug-2015 09:13:31 Elapsed time: 11 hours 5 mins 22 secs Priority: 12 FD Files Written: 547,868 SD Files Written: 547,868 FD Bytes Written: 1,202,478,052,857 (1.202 TB) SD Bytes Written: 1,202,608,809,023 (1.202 TB) Rate: 30120.7 KB/s OK: Scheduled time: 25-Jul-2015 15:03:00 Start time: 25-Jul-2015 21:52:17 End time: 26-Jul-2015 08:24:29 Elapsed time: 10 hours 32 mins 12 secs Priority: 12 FD Files Written: 545,740 SD Files Written: 545,740 FD Bytes Written: 1,193,323,311,780 (1.193 TB) SD Bytes Written: 1,193,453,558,522 (1.193 TB) Rate: 31459.5 KB/s OK: Scheduled time: 18-Jul-2015 15:03:00 Start time: 18-Jul-2015 20:26:21 End time: 19-Jul-2015 05:52:25 Elapsed time: 9 hours 26 mins 4 secs Priority: 12 FD Files Written: 543,122 SD Files Written: 543,122 FD Bytes Written: 1,176,812,345,702 (1.176 TB) SD Bytes Written: 1,176,941,989,504 (1.176 TB) Rate: 34648.8 KB/s Thank you, Best regards Ray -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
On 08/03/2015 09:45 AM, Michael Schwager wrote: Kern, Thanks for the reply from The Man himself :-) . I ran update slots, and it's true that EPW681L3 is not in the library. But that's as I planned it it: Why would it go to EPW681L3 when EPW680L3 is right there in the library? It is in the same pool, it is as full as EPW681L3 once was, it's older than the file/job retention period, but somehow Bacula chose 681 instead of 680 which is what I was expecting. As a matter of fact, I marked all the eligible tapes as full: First I marked 681 full then Bacula chose 682. So I marked that one as full. Now 683 and beyond are younger than the file/job retention periods, so Bacula thinks it has no tapes. Meanwhile I'm pulling my hair out because I'm saying, Bacula- 680 is *right there*! Use it! But no joy. Can I somehow mark a tape as usable? Under v. 5.x, I discovered that Bacula's tape usage algorithm went something like this: Use an eligible tape from the library. I discovered (I think) that I could use any tape that met Bacula's rewritable criteria. Now, I don't know what the algorithm is. This means it will be difficult for me to choose the proper tape to put in the library. I need to know before the weekend starts so it doesn't hang up my backups. Hi Michael, You might try setting the enabled field to '0' for all volumes which are - or should be - inaccessible (not in library), and therefore should not be considered by Bacula as viable. I had to implement this on my system a couple years back, and it has been working fine ever since: http://revpol.com/node/146 I wrote that script to work out this same problem I was having when vchanger drives were not in the drive caddy, and hence the file volumes on them were not available. You will surely have to adapt it, or probably just pick a few things from it for your specific system because it is highly vchanger v0.8.6 centric. :) Having said that, it is typically best to let Bacula manage what volumes it wants, when it wants them, but in my situation, each hard drive I use with vchanger has about 70 10GB volumes so I am not severely limiting Bacula, just guiding it a little and giving it a nudge. e.g.: I do not touch retention times, nor do I manually force purges etc. :) Also, regarding my other question: How do I make Bacula fail hard if it doesn't find the media it needs? I have used the various job max ... time settings for things like this in the past. Bill -- Bill Arlofski http://www.revpol.com/bacula -- Not responsible for anything below this line -- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] MaxWaitTime and auto cancel a job waiting for a tape?
I read in this link http://marc.info/?l=bacula-bugsm=127548196026667 that The current definition of MaxWaitTime take the following status in account: JS_WaitFD JS_WaitSD JS_WaitMedia JS_WaitMount JS_WaitStoreRes JS_WaitJobRes JS_WaitClientRes JS_WaitMaxJobs JS_WaitPriority JS_WaitMedia JS_WaitMount JS_WaitMaxJobs But the documentation describes it as the maximum allowed time that a job may block waiting for a resource (such as waiting for a tape to be mounted, or waiting for the storage or file daemons to perform their duties). I try to use [Max Wait Time = 30 seconds] to auto cancel a job when it starts and waits for a tape, but I have problems with it. it count this delay from schedule or start time instead of blocking time! Any alternative to this buggy MaxWaitTime to achieve my goal? +-- |This was sent by jalil1...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] File Set for backing up whole harddisk (with all partitions) Windows
What setting should I use to configure a FileSet for backing up a Windows disk with all it's partitions? Best regartd, Luc -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
Kern,
Thanks for the reply from The Man himself :-) .
I ran update slots, and it's true that EPW681L3 is not in the library. But
that's as I planned it it: Why would it go to EPW681L3 when EPW680L3 is
right there in the library? It is in the same pool, it is as full as
EPW681L3 once was, it's older than the file/job retention period, but
somehow Bacula chose 681 instead of 680 which is what I was expecting.
As a matter of fact, I marked all the eligible tapes as full: First I
marked 681 full then Bacula chose 682. So I marked that one as full. Now
683 and beyond are younger than the file/job retention periods, so Bacula
thinks it has no tapes. Meanwhile I'm pulling my hair out because I'm
saying, Bacula- 680 is *right there*! Use it! But no joy. Can I somehow
mark a tape as usable?
Under v. 5.x, I discovered that Bacula's tape usage algorithm went
something like this: Use an eligible tape from the library. I discovered
(I think) that I could use any tape that met Bacula's rewritable criteria.
Now, I don't know what the algorithm is. This means it will be difficult
for me to choose the proper tape to put in the library. I need to know
before the weekend starts so it doesn't hang up my backups.
Also, regarding my other question: How do I make Bacula fail hard if it
doesn't find the media it needs?
*- Mike Schwager*
* Linux Network Engineer, Mocho Trading LLC*
* 312-646-4783 Phone312-637-0011 Cell312-957-9804 Fax*
On Sun, Aug 2, 2015 at 11:40 PM, Kern Sibbald k...@sibbald.com wrote:
Hello,
Sorry you are having problems. You did a perfect job of supplying the
right information :-)
It looks like for some reason you did not have tape EPW681L3 in the
autochanger the last time you did an update slots barcodes so the slot
for that Volume is zero, which means that Bacula will not try to use it --
even though it is recycled.
Every time you remove or add a tape to the library you must subsequently
manually run update slots barcodes so that Bacula knows what Volumes are
in the library.
Best regards,
Kern
On 03.08.2015 00:43, Michael Schwager wrote:
Hello,
We are running Bacula 7.0.5 on CentOS 7.0. This weekend, one of our full
stores blocked waiting on a tape. However I was trying to enjoy the nice
Chicago weather and trying not to think about work, so needless to say our
backup failed.
Starngely, we have another tape that SHOULD be available. Tapes in this
pool are set to recycle after 90 days, and 90 days ago from today is May
4th.
We have a tape that was last written to on March 28. So why would Bacula
block on the unavailable tape? Can I set it to fail and move on if a tape
is not available? I will never be able to satisfy Bacula over the weekend-
I would rather it fail than hold everything up.
Thanks. Here's some Bacula queries; first my client entry from
bacula-dir.conf:
Client {
Name = fscluster2-backup
...
File Retention = 3 months
Job Retention = 3 months
AutoPrune = yes # Prune expired Jobs/Files
}
Here's what status jobs says:
Device HP_LTO-3 (/dev/nst0) is not open.
Device is BLOCKED waiting for mount of volume EPW681L3,
Pool:Tape
Media type: LTO-3
Drive 0 is not loaded.
==
...
Used Volume status:
Reserved volume: EPW681L3 on tape device HP_LTO-3 (/dev/nst0)
Reader=0 writers=1 reserves=0 volinuse=0
But list media shows a volume in the changer which was last written to
on March 28; much older than 90 days. Why didn't Bacula use it?
*list media
Automatically selected Catalog: MyCatalog
Using Catalog MyCatalog
Pool: Tape
+-++---+-+-+--+--+-+--+---+---+-+
| MediaId | VolumeName | VolStatus | Enabled | VolBytes| VolFiles
| VolRetention | Recycle | Slot | InChanger | MediaType |
LastWritten |
+-++---+-+-+--+--+-+--+---+---+-+
| 16 | EPW680L3 | Full | 1 | 557,867,971,584 | 145
|7,776,000 | 1 |1 | 1 | LTO-3 | 2015-03-28
07:11:04 |
| 17 | EPW681L3 | Recycle | 1 | 1 |0
|7,776,000 | 1 |0 | 0 | LTO-3 | 2015-04-25
08:08:59 |
...
| 46 | EPW692L3 | Full | 1 | 532,896,215,040 | 163
|7,776,000 | 1 |2 | 1 | LTO-3 | 2015-08-01
09:13:24 |
| 47 | EPW693L3 | Full | 1 | 420,250,650,624 | 105
|7,776,000 | 1 |8 | 1 | LTO-3 | 2015-08-01
11:55:27 |
...
| 68 | ERX840L3 | Full | 1 | 508,789,241,856 | 130
|7,776,000 | 1 |5 | 1 | LTO-3 | 2015-07-25
10:08:25 |
Re: [Bacula-users] File Set for backing up whole harddisk (with all partitions) Windows
What setting should I use to configure a FileSet for backing up a Windows disk with all it's partitions? Hello Luc. RTNM (Read the Nice Manual =P): http://www.bacula.org/5.2.x-manuals/en/main/main/Configuring_Director.html#SECTION0014110010 Kudos, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II Do you need Bacula training? https://www.udemy.com/bacula-backup-software/?couponCode=bacula-list +55 61 8268-4220 Site: http://bacula.us FB: heitor.faria === Best regartd, Luc -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users From: Luc A ellass...@gmail.com To: bacula-users@lists.sourceforge.net Sent: Monday, August 3, 2015 8:12:19 AM Subject: [Bacula-users] File Set for backing up whole harddisk (with all partitions) Windows Best regartd, Luc -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users-- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
Hello Michael, I have been using Bacula tape volumes for 7 years. Since 3.X version up to now with 7.0.5 version. I had no problem with bacula recycling algorithm. About your problem, the EPW680L3 volume should be used instead EPW681L3 volume if these two volumes had the same settings (volume retention, recycle flag, enabled, etc.) and both volumes had status full or used. I mean the same settings because if you change pool settings in your configuration files and do not apply them for the existent volumes, even if they are in the same pool, they will have different settings. If EPW681L3 volume, for some reason, was marked purged (manually or automatic), then Bacula would use this volume instead EPW680L3 volume. Lots of others situations could occurs here that caused Bacula to use EPW681L3 volume instead the EPW680L3 volume. You can get the next volume that bacula would use for your backups with the list nextvol command from bconsole. This command returns only the first volume and if your backups spans over multiple tapes, this command will not be so helpful. I´m attaching a script that I have been using for a few months and I think that it could help you to know previously the tape(s) that Bacula should use for your backups in a specific date/time. IMHO, Bacula has a very stable recycling algorithm, and if things are well configured, it should work as expected. Best regards, Ana On Mon, Aug 3, 2015 at 4:15 PM, Bill Arlofski waa-bac...@revpol.com wrote: On 08/03/2015 09:45 AM, Michael Schwager wrote: Kern, Thanks for the reply from The Man himself :-) . I ran update slots, and it's true that EPW681L3 is not in the library. But that's as I planned it it: Why would it go to EPW681L3 when EPW680L3 is right there in the library? It is in the same pool, it is as full as EPW681L3 once was, it's older than the file/job retention period, but somehow Bacula chose 681 instead of 680 which is what I was expecting. As a matter of fact, I marked all the eligible tapes as full: First I marked 681 full then Bacula chose 682. So I marked that one as full. Now 683 and beyond are younger than the file/job retention periods, so Bacula thinks it has no tapes. Meanwhile I'm pulling my hair out because I'm saying, Bacula- 680 is *right there*! Use it! But no joy. Can I somehow mark a tape as usable? Under v. 5.x, I discovered that Bacula's tape usage algorithm went something like this: Use an eligible tape from the library. I discovered (I think) that I could use any tape that met Bacula's rewritable criteria. Now, I don't know what the algorithm is. This means it will be difficult for me to choose the proper tape to put in the library. I need to know before the weekend starts so it doesn't hang up my backups. Hi Michael, You might try setting the enabled field to '0' for all volumes which are - or should be - inaccessible (not in library), and therefore should not be considered by Bacula as viable. I had to implement this on my system a couple years back, and it has been working fine ever since: http://revpol.com/node/146 I wrote that script to work out this same problem I was having when vchanger drives were not in the drive caddy, and hence the file volumes on them were not available. You will surely have to adapt it, or probably just pick a few things from it for your specific system because it is highly vchanger v0.8.6 centric. :) Having said that, it is typically best to let Bacula manage what volumes it wants, when it wants them, but in my situation, each hard drive I use with vchanger has about 70 10GB volumes so I am not severely limiting Bacula, just guiding it a little and giving it a nudge. e.g.: I do not touch retention times, nor do I manually force purges etc. :) Also, regarding my other question: How do I make Bacula fail hard if it doesn't find the media it needs? I have used the various job max ... time settings for things like this in the past. Bill -- Bill Arlofski http://www.revpol.com/bacula -- Not responsible for anything below this line -- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users list_available_volumes.sh Description: Bourne shell script -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Out of virtual tapes
Hello Robert,
Yes, you should rethink your pool/volume configurations, specially volumes
and pool sizes regarding the amount of space you have to accommodate them.
You have a very high retention period. It is importante to assure you will
have enough space for bacula creating new volumes in the pool and not
having problems with your partition running out of space.
Best regards,
Ana
On Mon, Aug 3, 2015 at 5:35 PM, RAT rober...@netzero.net wrote:
This is very similar to the recent BLOCKED waiting on ... thread.
Q: I am thinking of making the Max Volume Bytes and Max Volumes larger to
fix this. Ok?
Running Jobs:
Console connected at 03-Aug-15 15:28
Console connected at 03-Aug-15 15:30
JobId Type Level Files Bytes Name Status
==
2632 Back Full 143,148 340.4 G BackupClient1 is waiting for an appendable
Volume
2633 Back Full 0 0 BackupClient1-to-Tape is waiting on max Client jobs
2657 Back Full 0 0 BackupCatalog is waiting for higher priority jobs to
finish
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
}
*list media Pool=File
+-++---+-++--+--+-+--+---+---+-+
| mediaid | volumename | volstatus | enabled | volbytes | volfiles |
volretention | recycle | slot | inchanger | mediatype | lastwritten |
+-++---+-++--+--+-+--+---+---+-+
| 1 | Vol-0001 | Full | 1 | 53,687,069,319 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 16:11:13 |
| 3 | Vol-0003 | Full | 1 | 53,687,067,597 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 23:07:16 |
| 4 | Vol-0004 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 23:11:19 |
| 5 | Vol-0005 | Full | 1 | 53,687,062,015 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:07:19 |
| 6 | Vol-0006 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:34:44 |
| 7 | Vol-0007 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:40:01 |
| 8 | Vol-0008 | Full | 1 | 53,687,046,615 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-20 23:07:48 |
| 13 | Vol-0013 | Full | 1 | 53,687,080,132 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-20 23:12:15 |
| 14 | Vol-0014 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-20 23:17:18 |
| 15 | Vol-0015 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-20 23:22:09 |
| 16 | Vol-0016 | Full | 1 | 53,687,035,854 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-21 23:08:15 |
| 17 | Vol-0017 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-21 23:13:24 |
| 18 | Vol-0018 | Full | 1 | 53,687,080,137 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-21 23:18:31 |
| 19 | Vol-0019 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-21 23:23:34 |
| 20 | Vol-0020 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-21 23:28:47 |
| 21 | Vol-0021 | Full | 1 | 53,687,067,953 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:08:05 |
| 22 | Vol-0022 | Full | 1 | 53,687,080,136 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:13:14 |
| 23 | Vol-0023 | Full | 1 | 53,687,080,134 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:18:20 |
| 24 | Vol-0024 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:23:24 |
| 25 | Vol-0025 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:28:22 |
| 26 | Vol-0026 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:33:33 |
| 27 | Vol-0027 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:38:50 |
| 28 | Vol-0028 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:43:55 |
| 29 | Vol-0029 | Full | 1 | 53,687,080,133 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:48:55 |
| 30 | Vol-0030 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:54:05 |
| 31 | Vol-0031 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-22 23:59:05 |
| 32 | Vol-0032 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-23 00:04:23 |
| 33 | Vol-0033 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-23 00:09:36 |
| 34 | Vol-0034 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0
| File1 | 2015-03-23
Re: [Bacula-users] MaxWaitTime and auto cancel a job waiting for a tape?
Hello, Max Wait Time works as expected. I´m affraid 30 seconds is not enough time to wait for a tape to be mounted if you´re using a tape library. The default time that bacula waits for a device that is busy (initializing or something else) is 5 minutes. If you´re working with a tape library I would recommend you to set this value to 5 minutes. Best regards, Ana On Mon, Aug 3, 2015 at 1:34 PM, jalil1408 bacula-fo...@backupcentral.com wrote: I read in this link http://marc.info/?l=bacula-bugsm=127548196026667 that The current definition of MaxWaitTime take the following status in account: JS_WaitFD JS_WaitSD JS_WaitMedia JS_WaitMount JS_WaitStoreRes JS_WaitJobRes JS_WaitClientRes JS_WaitMaxJobs JS_WaitPriority JS_WaitMedia JS_WaitMount JS_WaitMaxJobs But the documentation describes it as the maximum allowed time that a job may block waiting for a resource (such as waiting for a tape to be mounted, or waiting for the storage or file daemons to perform their duties). I try to use [Max Wait Time = 30 seconds] to auto cancel a job when it starts and waits for a tape, but I have problems with it. it count this delay from schedule or start time instead of blocking time! Any alternative to this buggy MaxWaitTime to achieve my goal? +-- |This was sent by jalil1...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula BLOCKED waiting for mount
Well, to answer my own question, here's the algorithm (use The Source,
Luke!):
CLUE1: During the backup, after the previous tape filled, my log file says,
There are no more Jobs associated with Volume EPW681L3. Marking it
purged.
That text comes from ua_purge.c, which runs the following SQL command in
the is_volume_purged() function:
SELECT 1 FROM JobMedia WHERE MediaId=%s LIMIT 1
The MediaId comes from an argument to the command. So somehow Bacula is
looking over the volumes. How? And somehow it skipped over the volume I
thought it would choose, EPW680L3. Why?
According to autoprune.c, the aforementioned message string comes from
prune_volumes() in auto_prune.c. But before I dig into that, I need to back
up. prune_volumes() was called by find_next_volume_for_append(), which does
this:
* 1. Look for volume with Append status.
...The query is:
SELECT MediaId,VolumeName,VolJobs,VolFiles,VolBlocks,
VolBytes,VolMounts,VolErrors,VolWrites,MaxVolBytes,VolCapacityBytes,
MediaType,VolStatus,PoolId,VolRetention,VolUseDuration,MaxVolJobs,
MaxVolFiles,Recycle,Slot,FirstWritten,LastWritten,InChanger,
EndFile,EndBlock,VolParts,LabelType,LabelDate,StorageId,
Enabled,LocationId,RecycleCount,InitialWrite,
ScratchPoolId,RecyclePoolId,VolReadTime,VolWriteTime,ActionOnPurge
FROM Media WHERE PoolId=%s AND MediaType='%s' AND Enabled=1
AND VolStatus='%s' AND InChanger=1 AND StorageId=%s
with
if (strcmp(mr-VolStatus, Recycle) == 0 ||
strcmp(mr-VolStatus, Purged) == 0) {
order = AND Recycle=1 ORDER BY LastWritten ASC,MediaId; /* take
oldest that can be recycled */
} else {
order =
sql_media_order_most_recently_written[db_get_type_index(mdb)];/* take
most recently written */
}
Now here's what happens if it does not find a volume with Append status:
if (!ok) {
* No volume found, apply algorithm
Here is the argorithm:
* 2. Try finding a recycled volume
if (!ok) {
* 3. Try recycling any purged volume
if (!ok) {
* 4. Try pruning Volumes
if (!ok create) {
* 5. Try pulling a volume from the Scratch pool
* If we are using an Autochanger and have not found
* a volume, retry looking for any volume.
if (!ok InChanger) {
InChanger = false;
continue; /* retry again accepting any volume */
}
So- there is an algorithm! Quite elaborate, at that.
Now back to prune_volumes(). If it cannot find a volume, it will prune one.
prune_volumes() says,
* Prune at least one Volume in current Pool. This is called from
* catreq.c = next_vol.c when the Storage daemon is asking for another
* volume and no appendable volumes are available.
prune_volumes first looks in the Scratch pool somehow. Mine is empty so
I'll skip right over that part.
Next, it performs a query on the database:
/*
* Get the List of all media ids in the current Pool or whose
* RecyclePoolId is the current pool or the scratch pool
*/
const char *select = SELECT DISTINCT MediaId,LastWritten FROM Media
WHERE
(PoolId=%s OR RecyclePoolId IN (%s)) AND MediaType='%s' %s
ORDER BY LastWritten ASC,MediaId;
the PoolId comes from mr-PoolId, where mr is of type MEDIA_DBR which is
a class that is a reflection of a row in the Media table of the databes.
ua-jcr which I presume is a Job Control Record. The PoolId's can be
found in the Pool table. In my case, my Pool name is Tape so my PoolId is
5. I don't have a RecyclePoolId but that seems pretty straightforward. In
any event, in my case it's probably the same as my Pool id: 5. MediaType
comes from mr-MediaType which again comes straight from the
bacula-dir.conf file; my MediaType is 'LTO-3'. The last string (the fourth
%s) is I believe either empty or AND InChanger=1 AND StorageId IN (%s),
where StorageId comes from the Storage table. For me, I have a Storage
record that specifies a name of LTO-3 so StorageId found in the database is
simply 3.
So my backup is trying to write to a tape in my 'Tape' Pool, on my 'LTO-3'
Storage device. Thus the full query should be:
*select = SELECT DISTINCT MediaId,LastWritten FROM Media WHERE
(PoolId=5 OR RecyclePoolId IN (5)) AND MediaType='LTO-3'
ORDER BY LastWritten ASC,MediaId;
At this point, it goes through each volume and prunes them. When it finds a
volume that's then purged, it will use it. My volume EPW680L3 is not
purged, so it doesn't use it.
How is purged defined? This is found in is_volume_purged() in ua_purge.c:
- If FirstWritten or LastWritten are 0, the volume is not purged (I
believe 0 is the same as 1970-01-01 00:00:00)
- If the Volume Status is Purged then the volume is purged.
- If there are no Jobs associated with the MediaId in the JobMedia
database table, then the volume is purged.
In my case, there are Jobs associated with the MediaId in the JobMedia
database table, but this JobId
[Bacula-users] Out of virtual tapes
This is very similar to the recent BLOCKED waiting on ... thread. Q: I am
thinking of making the Max Volume Bytes and Max Volumes larger to fix this.
Ok? Running Jobs:
Console connected at 03-Aug-15 15:28
Console connected at 03-Aug-15 15:30
JobId Type Level Files Bytes Name Status
==
2632 Back Full 143,148 340.4 G BackupClient1 is waiting for an appendable
Volume
2633 Back Full 0 0 BackupClient1-to-Tape is waiting on max Client jobs
2657 Back Full 0 0 BackupCatalog is waiting for higher priority jobs to finish
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
} *list media Pool=File
+-++---+-++--+--+-+--+---+---+-+
| mediaid | volumename | volstatus | enabled | volbytes | volfiles |
volretention | recycle | slot | inchanger | mediatype | lastwritten |
+-++---+-++--+--+-+--+---+---+-+
| 1 | Vol-0001 | Full | 1 | 53,687,069,319 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 16:11:13 |
| 3 | Vol-0003 | Full | 1 | 53,687,067,597 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 23:07:16 |
| 4 | Vol-0004 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-18 23:11:19 |
| 5 | Vol-0005 | Full | 1 | 53,687,062,015 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:07:19 |
| 6 | Vol-0006 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:34:44 |
| 7 | Vol-0007 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-19 23:40:01 |
| 8 | Vol-0008 | Full | 1 | 53,687,046,615 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-20 23:07:48 |
| 13 | Vol-0013 | Full | 1 | 53,687,080,132 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-20 23:12:15 |
| 14 | Vol-0014 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-20 23:17:18 |
| 15 | Vol-0015 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-20 23:22:09 |
| 16 | Vol-0016 | Full | 1 | 53,687,035,854 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-21 23:08:15 |
| 17 | Vol-0017 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-21 23:13:24 |
| 18 | Vol-0018 | Full | 1 | 53,687,080,137 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-21 23:18:31 |
| 19 | Vol-0019 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-21 23:23:34 |
| 20 | Vol-0020 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-21 23:28:47 |
| 21 | Vol-0021 | Full | 1 | 53,687,067,953 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:08:05 |
| 22 | Vol-0022 | Full | 1 | 53,687,080,136 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:13:14 |
| 23 | Vol-0023 | Full | 1 | 53,687,080,134 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:18:20 |
| 24 | Vol-0024 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:23:24 |
| 25 | Vol-0025 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:28:22 |
| 26 | Vol-0026 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:33:33 |
| 27 | Vol-0027 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:38:50 |
| 28 | Vol-0028 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:43:55 |
| 29 | Vol-0029 | Full | 1 | 53,687,080,133 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:48:55 |
| 30 | Vol-0030 | Full | 1 | 53,687,080,140 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:54:05 |
| 31 | Vol-0031 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-22 23:59:05 |
| 32 | Vol-0032 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 00:04:23 |
| 33 | Vol-0033 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 00:09:36 |
| 34 | Vol-0034 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 00:14:40 |
| 35 | Vol-0035 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 00:19:50 |
| 36 | Vol-0036 | Full | 1 | 53,687,080,139 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 00:24:54 |
| 37 | Vol-0037 | Full | 1 | 53,687,047,602 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 08:44:33 |
| 38 | Vol-0038 | Full | 1 | 53,687,080,138 | 12 | 31,536,000 | 1 | 0 | 0 |
File1 | 2015-03-23 08:49:35 |
| 39 | Vol-0039 | Full | 1 | 53,687,080,141 | 12 | 31,536,000 | 1 |
