Re: dnsblog lifetime

[Doug Hardie]

> On 24 April 2018, at 13:48, Wietse Venema  wrote:
> 
> Doug Hardie:
>>> On 22 April 2018, at 05:50, Wietse Venema  wrote:
>>> 
>>> Doug Hardie:
 I understood from the dnsblog man page that each dnsblog process
 only lives for a "limited amount of time".  I noticed this because
 I have over 50 dnsblog processes running on a fairly light duty
 postfix server.  Some of them are over a week old.  At first I
 thought they must have been orphaned, but looking through maillog,
 I find entries in the last few minutes from the oldest and the
 newest.  I didn't check all of them, but it appears they are all
 in use.  Looking at the source for postfix-3.3-20180114 (on web),
 it appears dnsblog checks one IP address and then exits.  I believe
 I can limit the number of dnsblog processes in master.cf (currently
 set to 0), but I am not sure that is a good idea.  How long are
 these processes supposed to live?
>>> 
>>> According to source, dnsblog processes exclude themselves from the
>>> max_use limit (max_idle remains in effect). I suppose I turned off
>>> max_use because these processes are postscreen helpers. Postscreen
>>> was designed to handle a much larger client load than to the rest
>>> of Postfix. Under extreme loads like 1+ connections/second,
>>> one does not want to be creating 100+ processes/second, as that
>>> would limit scalability.
>>> 
>>> The dnsblog processes still terminate after 100s idle time. On my
>>> lightly-loaded server, there currently is no dnsblog process running.
> 
> I think that we can avoid the need for warnings in documentation,
> by making the dnsblog service act according to the spirit of the
> max_idle and max_use settings, even if it cannot act by the letter.
> 
> With a given max_idle and max_use setting, a process is expected
> to terminate within approximately (max_idle * max_use) seconds.
> That is, on a low-volume (but not too low) server, a process may
> hang around for a few hours (100*100 = 1 seconds).
> 
> Even if the dnsblog process cannot enforce max_use literally (because
> dbsnlog may have to handle a huge number of requests during peak
> load), the process could still retire voluntarily after (max_idle
> * max_use) seconds, without any negative performance impact.
> 
> I'll look into implementing that.

Either way works for me.  I just got confused when I saw the durations of the 
processes and then read the man page.  I thought I had configured something 
incorrectly as they didn't match.  If I had your first response in the man page 
I would have said ahhh, now I understand.  

-- Doug

Re: dnsblog lifetime

[Wietse Venema]

Doug Hardie:
> > On 22 April 2018, at 05:50, Wietse Venema  wrote:
> > 
> > Doug Hardie:
> >> I understood from the dnsblog man page that each dnsblog process
> >> only lives for a "limited amount of time".  I noticed this because
> >> I have over 50 dnsblog processes running on a fairly light duty
> >> postfix server.  Some of them are over a week old.  At first I
> >> thought they must have been orphaned, but looking through maillog,
> >> I find entries in the last few minutes from the oldest and the
> >> newest.  I didn't check all of them, but it appears they are all
> >> in use.  Looking at the source for postfix-3.3-20180114 (on web),
> >> it appears dnsblog checks one IP address and then exits.  I believe
> >> I can limit the number of dnsblog processes in master.cf (currently
> >> set to 0), but I am not sure that is a good idea.  How long are
> >> these processes supposed to live?
> > 
> > According to source, dnsblog processes exclude themselves from the
> > max_use limit (max_idle remains in effect). I suppose I turned off
> > max_use because these processes are postscreen helpers. Postscreen
> > was designed to handle a much larger client load than to the rest
> > of Postfix. Under extreme loads like 1+ connections/second,
> > one does not want to be creating 100+ processes/second, as that
> > would limit scalability.
> > 
> > The dnsblog processes still terminate after 100s idle time. On my
> > lightly-loaded server, there currently is no dnsblog process running.

I think that we can avoid the need for warnings in documentation,
by making the dnsblog service act according to the spirit of the
max_idle and max_use settings, even if it cannot act by the letter.

With a given max_idle and max_use setting, a process is expected
to terminate within approximately (max_idle * max_use) seconds.
That is, on a low-volume (but not too low) server, a process may
hang around for a few hours (100*100 = 1 seconds).

Even if the dnsblog process cannot enforce max_use literally (because
dbsnlog may have to handle a huge number of requests during peak
load), the process could still retire voluntarily after (max_idle
* max_use) seconds, without any negative performance impact.

I'll look into implementing that.

Wietse

Re: dnsblog lifetime

[Doug Hardie]

> On 22 April 2018, at 05:50, Wietse Venema  wrote:
> 
> Doug Hardie:
>> I understood from the dnsblog man page that each dnsblog process
>> only lives for a "limited amount of time".  I noticed this because
>> I have over 50 dnsblog processes running on a fairly light duty
>> postfix server.  Some of them are over a week old.  At first I
>> thought they must have been orphaned, but looking through maillog,
>> I find entries in the last few minutes from the oldest and the
>> newest.  I didn't check all of them, but it appears they are all
>> in use.  Looking at the source for postfix-3.3-20180114 (on web),
>> it appears dnsblog checks one IP address and then exits.  I believe
>> I can limit the number of dnsblog processes in master.cf (currently
>> set to 0), but I am not sure that is a good idea.  How long are
>> these processes supposed to live?
> 
> According to source, dnsblog processes exclude themselves from the
> max_use limit (max_idle remains in effect). I suppose I turned off
> max_use because these processes are postscreen helpers. Postscreen
> was designed to handle a much larger client load than to the rest
> of Postfix. Under extreme loads like 1+ connections/second,
> one does not want to be creating 100+ processes/second, as that
> would limit scalability.
> 
> The dnsblog processes still terminate after 100s idle time. On my
> lightly-loaded server, there currently is no dnsblog process running.
> 
> Apparently your server has enough traffic to keep postscreen alive,
> and as a consequence, a collection of dnsblog processes.
> 
> I suppose you could reduce max_idle, but don't go overboard and
> set it to something small like 1s. That would be counterproductive.

Thanks for the info.  I never would have expected my server to be that busy.  I 
would suggest that the man page for dnsblog be updated in the first paragraph 
of configuration parameters to something like:

Changes to main.cf are picked up automatically as the dnsblog processes are 
started.  The dnsblog processes terminate after 100 seconds of idle time.  That 
is the default value and can be changed with the max idle configuration 
parameter.  However, reducing it to a small value is likely to be 
counterproductive.  Use the command "postfix reload" to speed up a change.

-- Doug

Re: dnsblog lifetime

[Wietse Venema]

Doug Hardie:
> I understood from the dnsblog man page that each dnsblog process
> only lives for a "limited amount of time".  I noticed this because
> I have over 50 dnsblog processes running on a fairly light duty
> postfix server.  Some of them are over a week old.  At first I
> thought they must have been orphaned, but looking through maillog,
> I find entries in the last few minutes from the oldest and the
> newest.  I didn't check all of them, but it appears they are all
> in use.  Looking at the source for postfix-3.3-20180114 (on web),
> it appears dnsblog checks one IP address and then exits.  I believe
> I can limit the number of dnsblog processes in master.cf (currently
> set to 0), but I am not sure that is a good idea.  How long are
> these processes supposed to live?

According to source, dnsblog processes exclude themselves from the
max_use limit (max_idle remains in effect). I suppose I turned off
max_use because these processes are postscreen helpers. Postscreen
was designed to handle a much larger client load than to the rest
of Postfix. Under extreme loads like 1+ connections/second,
one does not want to be creating 100+ processes/second, as that
would limit scalability.

The dnsblog processes still terminate after 100s idle time. On my
lightly-loaded server, there currently is no dnsblog process running.

Apparently your server has enough traffic to keep postscreen alive,
and as a consequence, a collection of dnsblog processes.

I suppose you could reduce max_idle, but don't go overboard and
set it to something small like 1s. That would be counterproductive.

Wiemaketse