[Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Rich Otero]

Hello,

We make a backup product that uses Bacula as the backend. Our systems run
Ubuntu v12.04 (kernel v3.12.17) with Bacula v5.2.12. Our product sets up
the job resource definitions based on the user's choices, it validates the
configuration with `bacula-dir -t`, and then it issues the bconsole
`reload` command to load the new job resource. On a particular customer
system, we find that the new job resource has not been loaded after the
`reload` command has completed. If we give the `run` command, the new job
is not in the list of jobs that can be executed. If we restart bacula-dir,
then the job resource is loaded and it can be run. We haven't been able to
figure out why `reload` isn't doing what we expect, so I hope that you can
offer some advice. I have attached the Director configuration file from the
affected system (bacula-dir.conf) and the file where we put the job
resource definitions (jobs.conf), which is imported by the main
configuration file.

I've found a few relevant points in researching this problem online. The
bconsole documentation seems to discourage you from using `reload`:

"While it is possible to reload the Director's configuration on the fly,
even while jobs are executing, this is a complex operation and not without
side effects. Accordingly, if you have to reload the Director's
configuration while Bacula is running, it is advisable to restart the
Director at the next convenient opportunity."

http://www.bacula.org/5.2.x-manuals/en/console/console/Bacula_Console.html

Indeed, I have found that restarting the Director causes the new job
resource to be loaded, but this is not a good solution for us because we
often have many Bacula jobs running simultaneously; the "convenient" times
when the Director could be restarted will be too infrequent.

I also found a closed bug ticket from a user who describes practically the
same problem:

http://bugs.bacula.org/view.php?id=1573

Thank you in advance for your help.

Regards,
Rich Otero
Director, Technical Support and Professional Services
EditShare
rot...@editshare.com
617-782-0479


jobs.conf
Description: Binary data


bacula-dir.conf
Description: Binary data
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Heitor Faria]

> Hello,

> We make a backup product that uses Bacula as the backend. Our systems run 
> Ubuntu
> v12.04 (kernel v3.12.17) with Bacula v5.2.12. Our product sets up the job
> resource definitions based on the user's choices, it validates the
> configuration with `bacula-dir -t`, and then it issues the bconsole `reload`
> command to load the new job resource. On a particular customer system, we find
> that the new job resource has not been loaded after the `reload` command has
> completed. If we give the `run` command, the new job is not in the list of 
> jobs
> that can be executed. If we restart bacula-dir, then the job resource is 
> loaded
> and it can be run. We haven't been able to figure out why `reload` isn't doing
> what we expect, so I hope that you can offer some advice. I have attached the
> Director configuration file from the affected system (bacula-dir.conf) and the
> file where we put the job resource definitions (jobs.conf), which is imported
> by the main configuration file.

> I've found a few relevant points in researching this problem online. The
> bconsole documentation seems to discourage you from using `reload`:

> "While it is possible to reload the Director's configuration on the fly, even
> while jobs are executing, this is a complex operation and not without side
> effects. Accordingly, if you have to reload the Director's configuration while
> Bacula is running, it is advisable to restart the Director at the next
> convenient opportunity."

> http://www.bacula.org/5.2.x-manuals/en/console/console/Bacula_Console.html

> Indeed, I have found that restarting the Director causes the new job resource 
> to
> be loaded, but this is not a good solution for us because we often have many
> Bacula jobs running simultaneously; the "convenient" times when the Director
> could be restarted will be too infrequent.

> I also found a closed bug ticket from a user who describes practically the 
> same
> problem:

> http://bugs.bacula.org/view.php?id=1573

Hello Rich: is there any message after using the reload command? (message 
command) 
Can you post the the output of a show jobs after the reload command? 
Ps.: Bacula is two major releases ahead and I think this hypothetical bug would 
only be fixed if reproduced in 7.2.0 version, but I'm not a developer. 

> Thank you in advance for your help.

> Regards,
> Rich Otero
> Director, Technical Support and Professional Services
> EditShare
> rot...@editshare.com
> 617-782-0479

Regards, 
=== 
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified 
Administrator II 
Do you need Bacula training? http://bacula.us/video-classes/ 
I do Bacula training and deploy in any city of the world. More information: 
http://bacula.us/ 
+55 61 8268-4220 
Site: http://bacula.us FB: heitor.faria 
=== 
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Roberts, Ben]

> On a particular customer system, we find that the new job resource has not 
> been loaded after the `reload` command has completed.

My workflow relies on the reload command heavily (puppet drops in updated 
config files and executes the reload command afterwards; this normally works 
very well. Occasionally I have problems and this is always evidenced by “Error: 
Too many open reload requests. Request ignored” in the messages output. The 
normal cause for this in my experience is too many connected bconsoles (which 
appear to count against the director’s concurrent job limit?) Killing off the 
old bconsole sessions that people have spawned under screen weeks before and 
forgotten usually fixes this. I rarely have to restart the director service 
more than once or twice a year and that’s normally for other issues.

Regards,
Ben Roberts

This email and any files transmitted with it contain confidential and 
proprietary information and is solely for the use of the intended recipient.
If you are not the intended recipient please return the email to the sender and 
delete it from your computer and you must not use, disclose, distribute, copy, 
print or rely on this email or its contents.
This communication is for informational purposes only.
It is not intended as an offer or solicitation for the purchase or sale of any 
financial instrument or as an official confirmation of any transaction.
Any comments or statements made herein do not necessarily reflect those of GSA 
Capital.
GSA Capital Partners LLP is authorised and regulated by the Financial Conduct 
Authority and is registered in England and Wales at Stratton House, 5 Stratton 
Street, London W1J 8LA, number OC309261.
GSA Capital Services Limited is registered in England and Wales at the same 
address, number 5320529.
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Rich Otero]

Ben, thanks for the input. We used to have that problem too: We had a bug
in our code that would result in a lot of old bconsoles being left open and
we would hit the concurrent jobs limit. We fixed that some time ago, but
your message and someone else's yesterday are directing me to the message.
I haven't looked at those yet.

Regards,
Rich Otero
Director, Technical Support and Professional Services
EditShare
rot...@editshare.com
617-782-0479

On Tue, Oct 20, 2015 at 2:47 AM, Roberts, Ben 
wrote:

> > On a particular customer system, we find that the new job resource has
> not been loaded after the `reload` command has completed.
>
>
>
> My workflow relies on the reload command heavily (puppet drops in updated
> config files and executes the reload command afterwards; this normally
> works very well. Occasionally I have problems and this is always evidenced
> by “Error: Too many open reload requests. Request ignored” in the messages
> output. The normal cause for this in my experience is too many connected
> bconsoles (which appear to count against the director’s concurrent job
> limit?) Killing off the old bconsole sessions that people have spawned
> under screen weeks before and forgotten usually fixes this. I rarely have
> to restart the director service more than once or twice a year and that’s
> normally for other issues.
>
>
>
> Regards,
>
> Ben Roberts
>
>
> --
> This email and any files transmitted with it contain confidential and
> proprietary information and is solely for the use of the intended
> recipient. If you are not the intended recipient please return the email to
> the sender and delete it from your computer and you must not use, disclose,
> distribute, copy, print or rely on this email or its contents. This
> communication is for informational purposes only. It is not intended as an
> offer or solicitation for the purchase or sale of any financial instrument
> or as an official confirmation of any transaction. Any comments or
> statements made herein do not necessarily reflect those of GSA Capital. GSA
> Capital Partners LLP is authorised and regulated by the Financial Conduct
> Authority and is registered in England and Wales at Stratton House, 5
> Stratton Street, London W1J 8LA, number OC309261. GSA Capital Services
> Limited is registered in England and Wales at the same address, number
> 5320529.
>
>
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Heitor Faria]

Hello Rich, 

You can always retrieve the messages from the Bacula log. It is saved in the 
destination defined in the Messages directive (bacula-dir.conf): append = 
"/var/log/bacula.log" = all, !skipped 
It is possible to cat it and grep in the search for reload messages output. 
E.g.: 

cat /var/log/bacula.log | grep ERROR 

or 

cat /var/log/bacula.log | grep Error: 

Regards, 
=== 
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified 
Administrator II 
Do you need Bacula training? http://bacula.us/video-classes/ 
I do Bacula training and deploy in any city of the world. More information: 
http://bacula.us/ 
+55 61 8268-4220 
Site: http://bacula.us FB: heitor.faria 
=== 

> From: "Rich Otero" 
> To: "Roberts, Ben" 
> Cc: bacula-users@lists.sourceforge.net
> Sent: Tuesday, October 20, 2015 11:24:54 AM
> Subject: Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job
> Resource Definitions

> Ben, thanks for the input. We used to have that problem too: We had a bug in 
> our
> code that would result in a lot of old bconsoles being left open and we would
> hit the concurrent jobs limit. We fixed that some time ago, but your message
> and someone else's yesterday are directing me to the message. I haven't looked
> at those yet.

> Regards,
> Rich Otero
> Director, Technical Support and Professional Services
> EditShare
> rot...@editshare.com
> 617-782-0479

> On Tue, Oct 20, 2015 at 2:47 AM, Roberts, Ben < ben.robe...@gsacapital.com >
> wrote:

>>> On a particular customer system, we find that the new job resource has not 
>>> been
>> > loaded after the `reload` command has completed.

>> My workflow relies on the reload command heavily (puppet drops in updated 
>> config
>> files and executes the reload command afterwards; this normally works very
>> well. Occasionally I have problems and this is always evidenced by “Error: 
>> Too
>> many open reload requests. Request ignored” in the messages output. The 
>> normal
>> cause for this in my experience is too many connected bconsoles (which appear
>> to count against the director’s concurrent job limit?) Killing off the old
>> bconsole sessions that people have spawned under screen weeks before and
>> forgotten usually fixes this. I rarely have to restart the director service
>> more than once or twice a year and that’s normally for other issues.

>> Regards,

>> Ben Roberts

>> This email and any files transmitted with it contain confidential and
>> proprietary information and is solely for the use of the intended recipient. 
>> If
>> you are not the intended recipient please return the email to the sender and
>> delete it from your computer and you must not use, disclose, distribute, 
>> copy,
>> print or rely on this email or its contents. This communication is for
>> informational purposes only. It is not intended as an offer or solicitation 
>> for
>> the purchase or sale of any financial instrument or as an official 
>> confirmation
>> of any transaction. Any comments or statements made herein do not necessarily
>> reflect those of GSA Capital. GSA Capital Partners LLP is authorised and
>> regulated by the Financial Conduct Authority and is registered in England and
>> Wales at Stratton House, 5 Stratton Street, London W1J 8LA, number OC309261.
>> GSA Capital Services Limited is registered in England and Wales at the same
>> address, number 5320529.

> --

> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Kern Sibbald]

Hello,

I recommend that you move up to Ubuntu 14.04 (maybe not so easy from a
QA stand point), but more importantly move up to Bacula 7.2.0 (or 7.2.1
which will be out within the next month).  Sinc version 5.2.x, we have
done some major rewrites of the reload command including a new directive
so, Bacula 7.2.x will very likely resolve all the problems you mention
about reload. 

Best regards,
Kern

On 10/19/2015 03:29 PM, Rich Otero wrote:
> Hello,
>
> We make a backup product that uses Bacula as the backend. Our systems
> run Ubuntu v12.04 (kernel v3.12.17) with Bacula v5.2.12. Our product
> sets up the job resource definitions based on the user's choices, it
> validates the configuration with `bacula-dir -t`, and then it issues
> the bconsole `reload` command to load the new job resource. On a
> particular customer system, we find that the new job resource has not
> been loaded after the `reload` command has completed. If we give the
> `run` command, the new job is not in the list of jobs that can be
> executed. If we restart bacula-dir, then the job resource is loaded
> and it can be run. We haven't been able to figure out why `reload`
> isn't doing what we expect, so I hope that you can offer some advice.
> I have attached the Director configuration file from the affected
> system (bacula-dir.conf) and the file where we put the job resource
> definitions (jobs.conf), which is imported by the main configuration file.
>
> I've found a few relevant points in researching this problem online.
> The bconsole documentation seems to discourage you from using `reload`:
>
> "While it is possible to reload the Director's configuration on the
> fly, even while jobs are executing, this is a complex operation and
> not without side effects. Accordingly, if you have to reload the
> Director's configuration while Bacula is running, it is advisable to
> restart the Director at the next convenient opportunity."
>
> http://www.bacula.org/5.2.x-manuals/en/console/console/Bacula_Console.html
>
> Indeed, I have found that restarting the Director causes the new job
> resource to be loaded, but this is not a good solution for us because
> we often have many Bacula jobs running simultaneously; the
> "convenient" times when the Director could be restarted will be too
> infrequent.
>
> I also found a closed bug ticket from a user who describes practically
> the same problem:
>
> http://bugs.bacula.org/view.php?id=1573
>
> Thank you in advance for your help.
>
> Regards,
> Rich Otero
> Director, Technical Support and Professional Services
> EditShare
> rot...@editshare.com 
> 617-782-0479
>
>
> --
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Kern Sibbald]

Oh, I forgot to mention that when you start the console, if I am not
mistaken (to be verified), it "locks onto" the most recent
bacula-dir.conf file as if it were a regular job, and consequently, if
you want bconsole to show a more recent bacula-dir.conf I reccommend
restarting bconsole.  In fact, keeping old bconsoles connected to the
Director can block a given bacula-dir.conf in memory, and at some point,
the Director will not be able to do a "reload" command.  So if you are
doing a lot of reloads, be sure to kill off old bconsole sessions.

Best regards,
Kern

On 10/19/2015 03:29 PM, Rich Otero wrote:
> Hello,
>
> We make a backup product that uses Bacula as the backend. Our systems
> run Ubuntu v12.04 (kernel v3.12.17) with Bacula v5.2.12. Our product
> sets up the job resource definitions based on the user's choices, it
> validates the configuration with `bacula-dir -t`, and then it issues
> the bconsole `reload` command to load the new job resource. On a
> particular customer system, we find that the new job resource has not
> been loaded after the `reload` command has completed. If we give the
> `run` command, the new job is not in the list of jobs that can be
> executed. If we restart bacula-dir, then the job resource is loaded
> and it can be run. We haven't been able to figure out why `reload`
> isn't doing what we expect, so I hope that you can offer some advice.
> I have attached the Director configuration file from the affected
> system (bacula-dir.conf) and the file where we put the job resource
> definitions (jobs.conf), which is imported by the main configuration file.
>
> I've found a few relevant points in researching this problem online.
> The bconsole documentation seems to discourage you from using `reload`:
>
> "While it is possible to reload the Director's configuration on the
> fly, even while jobs are executing, this is a complex operation and
> not without side effects. Accordingly, if you have to reload the
> Director's configuration while Bacula is running, it is advisable to
> restart the Director at the next convenient opportunity."
>
> http://www.bacula.org/5.2.x-manuals/en/console/console/Bacula_Console.html
>
> Indeed, I have found that restarting the Director causes the new job
> resource to be loaded, but this is not a good solution for us because
> we often have many Bacula jobs running simultaneously; the
> "convenient" times when the Director could be restarted will be too
> infrequent.
>
> I also found a closed bug ticket from a user who describes practically
> the same problem:
>
> http://bugs.bacula.org/view.php?id=1573
>
> Thank you in advance for your help.
>
> Regards,
> Rich Otero
> Director, Technical Support and Professional Services
> EditShare
> rot...@editshare.com 
> 617-782-0479
>
>
> --
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bconsole Reload Sometimes Doesn't Load New Job Resource Definitions

[Kern Sibbald]

Hello Ben,

Thanks, this confirms what I wrote in my previous email.

Best regards,
Kern

On 10/19/2015 11:47 PM, Roberts, Ben wrote:
>
> > On a particular customer system, we find that the new job resource
> has not been loaded after the `reload` command has completed.
>
>  
>
> My workflow relies on the reload command heavily (puppet drops in
> updated config files and executes the reload command afterwards; this
> normally works very well. Occasionally I have problems and this is
> always evidenced by “Error: Too many open reload requests. Request
> ignored” in the messages output. The normal cause for this in my
> experience is too many connected bconsoles (which appear to count
> against the director’s concurrent job limit?) Killing off the old
> bconsole sessions that people have spawned under screen weeks before
> and forgotten usually fixes this. I rarely have to restart the
> director service more than once or twice a year and that’s normally
> for other issues.
>
>  
>
> Regards,
>
> Ben Roberts
>
>  
>
> 
> This email and any files transmitted with it contain confidential and
> proprietary information and is solely for the use of the intended
> recipient. If you are not the intended recipient please return the
> email to the sender and delete it from your computer and you must not
> use, disclose, distribute, copy, print or rely on this email or its
> contents. This communication is for informational purposes only. It is
> not intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. Any comments or statements made herein do not necessarily
> reflect those of GSA Capital. GSA Capital Partners LLP is authorised
> and regulated by the Financial Conduct Authority and is registered in
> England and Wales at Stratton House, 5 Stratton Street, London W1J
> 8LA, number OC309261. GSA Capital Services Limited is registered in
> England and Wales at the same address, number 5320529.
>
>
>
> --
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users