Re: [HACKERS] Incorrect formula for SysV IPC parameters
Thank you for pushing this. At Tue, 16 Feb 2016 15:07:32 +0900, Fujii Masao wrote in > > Attached patch uses the above formula for 9.3. I'm thinking to push your > > patches to 9.2, 9.4, 9.5, master, also push the attached one to 9.3. > > Comments? > > Pushed. Thanks for the report and patches! regards, -- Kyotaro Horiguchi NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect formula for SysV IPC parameters
On Fri, Feb 12, 2016 at 11:19 PM, Fujii Masao wrote:
> On Fri, Feb 5, 2016 at 2:17 PM, Kyotaro HORIGUCHI
> wrote:
>> At Thu, 4 Feb 2016 21:43:04 +0900, Fujii Masao wrote
>> in
>>> On Wed, Feb 3, 2016 at 12:51 PM, Kyotaro HORIGUCHI
>>> wrote:
>>> > Hello, I found that the formulas to calculate SEMMNI and SEMMNS
>>> > are incorrect in 9.2 and later.
>>> >
>>> > http://www.postgresql.org/docs/9.5/static/kernel-resources.html
>>> >
>>> > But actually the number of semaphores PostgreSQL needs is
>>> > calculated as following in 9.4 and later.
>> ...
>>> > So, the formula for SEMMNI should be
>>> >
>>> > ceil((max_connections + autovacuum_max_workers + max_worker_processes +
>>> > 5) / 16)
>>> >
>>> > and SEMMNS should have the same fix.
>>> >
>>> >
>>> > In 9.3 and 9.2, the documentation says the same thing but
>> ...
>>> > ceil((max_connections + autovacuum_max_workers + 5) / 16)
>>> >
>>> > In 9.1, NUM_AUXILIARY_PROCS is 3 so the documentations is correct.
>>>
>>> Good catch!
>>
>> Thanks.
>>
>>> ISTM that you also need to change the descriptions about SEMMNI and SEMMNS
>>> under the Table 17-1.
>>
>> Oops! Thank you for pointing it out.
>>
>> The original description doesn't mention the magic-number ('5' in
>> the above formulas, which previously was '4') so I haven't added
>> anything about it.
>>
>> Process of which the number is limited by max_worker_processes is
>> called 'background process' (not 'backend worker') in the
>> documentation so I used the name to mention it in the additional
>> description.
>>
>> The difference in the body text for 9.2, 9.3 is only a literal
>> '4' to '5' in the formula.
>
> Thanks for updating the patches!
>
> They look good to me except that the formulas don't include the number of
> background processes requesting shared memory access, i.e.,
> GetNumShmemAttachedBgworkers(), in 9.3. Isn't it better to use the following
> formula in 9.3?
>
> ceil((max_connections + autovacuum_max_workers + number of
> background proceses + 5) / 16)
>
> Attached patch uses the above formula for 9.3. I'm thinking to push your
> patches to 9.2, 9.4, 9.5, master, also push the attached one to 9.3.
> Comments?
Pushed. Thanks for the report and patches!
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect formula for SysV IPC parameters
On Mon, Feb 15, 2016 at 2:31 PM, Kyotaro HORIGUCHI
wrote:
> Thanks for looking at this.
>
> At Fri, 12 Feb 2016 23:19:45 +0900, Fujii Masao wrote
> in
>> >> ISTM that you also need to change the descriptions about SEMMNI and SEMMNS
>> >> under the Table 17-1.
>> >
>> > Oops! Thank you for pointing it out.
>> >
>> > The original description doesn't mention the magic-number ('5' in
>> > the above formulas, which previously was '4') so I haven't added
>> > anything about it.
>> >
>> > Process of which the number is limited by max_worker_processes is
>> > called 'background process' (not 'backend worker') in the
>> > documentation so I used the name to mention it in the additional
>> > description.
>> >
>> > The difference in the body text for 9.2, 9.3 is only a literal
>> > '4' to '5' in the formula.
>>
>> Thanks for updating the patches!
>>
>> They look good to me except that the formulas don't include the number of
>> background processes requesting shared memory access, i.e.,
>> GetNumShmemAttachedBgworkers(), in 9.3. Isn't it better to use the following
>> formula in 9.3?
>>
>> ceil((max_connections + autovacuum_max_workers + number of
>> background proceses + 5) / 16)
>>
>> Attached patch uses the above formula for 9.3. I'm thinking to push your
>> patches to 9.2, 9.4, 9.5, master, also push the attached one to 9.3.
>> Comments?
>
> One concern is that users don't have any simple way to know how
> many bg-workers a server runs in their current configuration.
Users need to read the document of the extensions they want to load,
to see the number of background worker processes which will be running.
> The formula donsn't make sense without it.
IMO, documenting "incorrect" formula can cause more troubles.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect formula for SysV IPC parameters
Thanks for looking at this.
At Fri, 12 Feb 2016 23:19:45 +0900, Fujii Masao wrote
in
> >> ISTM that you also need to change the descriptions about SEMMNI and SEMMNS
> >> under the Table 17-1.
> >
> > Oops! Thank you for pointing it out.
> >
> > The original description doesn't mention the magic-number ('5' in
> > the above formulas, which previously was '4') so I haven't added
> > anything about it.
> >
> > Process of which the number is limited by max_worker_processes is
> > called 'background process' (not 'backend worker') in the
> > documentation so I used the name to mention it in the additional
> > description.
> >
> > The difference in the body text for 9.2, 9.3 is only a literal
> > '4' to '5' in the formula.
>
> Thanks for updating the patches!
>
> They look good to me except that the formulas don't include the number of
> background processes requesting shared memory access, i.e.,
> GetNumShmemAttachedBgworkers(), in 9.3. Isn't it better to use the following
> formula in 9.3?
>
> ceil((max_connections + autovacuum_max_workers + number of
> background proceses + 5) / 16)
>
> Attached patch uses the above formula for 9.3. I'm thinking to push your
> patches to 9.2, 9.4, 9.5, master, also push the attached one to 9.3.
> Comments?
One concern is that users don't have any simple way to know how
many bg-workers a server runs in their current configuration.
The formula donsn't make sense without it.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect formula for SysV IPC parameters
On Fri, Feb 5, 2016 at 2:17 PM, Kyotaro HORIGUCHI
wrote:
> At Thu, 4 Feb 2016 21:43:04 +0900, Fujii Masao wrote
> in
>> On Wed, Feb 3, 2016 at 12:51 PM, Kyotaro HORIGUCHI
>> wrote:
>> > Hello, I found that the formulas to calculate SEMMNI and SEMMNS
>> > are incorrect in 9.2 and later.
>> >
>> > http://www.postgresql.org/docs/9.5/static/kernel-resources.html
>> >
>> > But actually the number of semaphores PostgreSQL needs is
>> > calculated as following in 9.4 and later.
> ...
>> > So, the formula for SEMMNI should be
>> >
>> > ceil((max_connections + autovacuum_max_workers + max_worker_processes + 5)
>> > / 16)
>> >
>> > and SEMMNS should have the same fix.
>> >
>> >
>> > In 9.3 and 9.2, the documentation says the same thing but
> ...
>> > ceil((max_connections + autovacuum_max_workers + 5) / 16)
>> >
>> > In 9.1, NUM_AUXILIARY_PROCS is 3 so the documentations is correct.
>>
>> Good catch!
>
> Thanks.
>
>> ISTM that you also need to change the descriptions about SEMMNI and SEMMNS
>> under the Table 17-1.
>
> Oops! Thank you for pointing it out.
>
> The original description doesn't mention the magic-number ('5' in
> the above formulas, which previously was '4') so I haven't added
> anything about it.
>
> Process of which the number is limited by max_worker_processes is
> called 'background process' (not 'backend worker') in the
> documentation so I used the name to mention it in the additional
> description.
>
> The difference in the body text for 9.2, 9.3 is only a literal
> '4' to '5' in the formula.
Thanks for updating the patches!
They look good to me except that the formulas don't include the number of
background processes requesting shared memory access, i.e.,
GetNumShmemAttachedBgworkers(), in 9.3. Isn't it better to use the following
formula in 9.3?
ceil((max_connections + autovacuum_max_workers + number of
background proceses + 5) / 16)
Attached patch uses the above formula for 9.3. I'm thinking to push your
patches to 9.2, 9.4, 9.5, master, also push the attached one to 9.3.
Comments?
Regards,
--
Fujii Masao
fix_doc_sysvipc_93.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect formula for SysV IPC parameters
At Thu, 4 Feb 2016 21:43:04 +0900, Fujii Masao wrote in
> On Wed, Feb 3, 2016 at 12:51 PM, Kyotaro HORIGUCHI
> wrote:
> > Hello, I found that the formulas to calculate SEMMNI and SEMMNS
> > are incorrect in 9.2 and later.
> >
> > http://www.postgresql.org/docs/9.5/static/kernel-resources.html
> >
> > But actually the number of semaphores PostgreSQL needs is
> > calculated as following in 9.4 and later.
...
> > So, the formula for SEMMNI should be
> >
> > ceil((max_connections + autovacuum_max_workers + max_worker_processes + 5)
> > / 16)
> >
> > and SEMMNS should have the same fix.
> >
> >
> > In 9.3 and 9.2, the documentation says the same thing but
...
> > ceil((max_connections + autovacuum_max_workers + 5) / 16)
> >
> > In 9.1, NUM_AUXILIARY_PROCS is 3 so the documentations is correct.
>
> Good catch!
Thanks.
> ISTM that you also need to change the descriptions about SEMMNI and SEMMNS
> under the Table 17-1.
Oops! Thank you for pointing it out.
The original description doesn't mention the magic-number ('5' in
the above formulas, which previously was '4') so I haven't added
anything about it.
Process of which the number is limited by max_worker_processes is
called 'background process' (not 'backend worker') in the
documentation so I used the name to mention it in the additional
description.
The difference in the body text for 9.2, 9.3 is only a literal
'4' to '5' in the formula.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
>From 6ed2f296cc5899f75a2e817f470e5da07bcb0d2c Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi
Date: Wed, 3 Feb 2016 11:35:43 +0900
Subject: [PATCH] Fix the formula to calculate SEMMNI and SEMMNS in
documentation
---
doc/src/sgml/runtime.sgml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 0db3807..2bc0a91 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -645,13 +645,13 @@ psql: could not connect to server: No such file or directory
SEMMNI
Maximum number of semaphore identifiers (i.e., sets)
-at least ceil((max_connections + autovacuum_max_workers + 4) / 16)
+at least ceil((max_connections + autovacuum_max_workers + 5) / 16)
SEMMNS
Maximum number of semaphores system-wide
-ceil((max_connections + autovacuum_max_workers + 4) / 16) * 17 plus room for other applications
+ceil((max_connections + autovacuum_max_workers + 5) / 16) * 17 plus room for other applications
@@ -712,7 +712,7 @@ psql: could not connect to server: No such file or directory
linkend="sysvipc-parameters">). The parameter SEMMNI
determines the limit on the number of semaphore sets that can
exist on the system at one time. Hence this parameter must be at
-least ceil((max_connections + autovacuum_max_workers + 4) / 16).
+least ceil((max_connections + autovacuum_max_workers + 5) / 16).
Lowering the number
of allowed connections is a temporary workaround for failures,
which are usually confusingly worded No space
--
1.8.3.1
>From c196c82665e4081a6f90938ac86bb63e4e60e45c Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi
Date: Wed, 3 Feb 2016 11:07:12 +0900
Subject: [PATCH] Fix the formula to calculate SEMMNI and SEMMNS in
documentation
---
doc/src/sgml/runtime.sgml | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index cda05f5..dd42dd0 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -645,13 +645,13 @@ psql: could not connect to server: No such file or directory
SEMMNI
Maximum number of semaphore identifiers (i.e., sets)
-at least ceil((max_connections + autovacuum_max_workers + 4) / 16)
+at least ceil((max_connections + autovacuum_max_workers + max_worker_processes + 5) / 16)
SEMMNS
Maximum number of semaphores system-wide
-ceil((max_connections + autovacuum_max_workers + 4) / 16) * 17 plus room for other applications
+ceil((max_connections + autovacuum_max_workers + max_worker_processes + 5) / 16) * 17 plus room for other applications
@@ -699,20 +699,22 @@ psql: could not connect to server: No such file or directory
PostgreSQL uses one semaphore per allowed connection
-() and allowed autovacuum worker
-process (), in sets of 16.
+(), allowed autovacuum worker process
+() and allowed background
+process (), in sets of 16.
Each such set will
also contain a 17th semaphore which contains a magic
number, to detect collision with semaphore sets used by
other applications. The maximum number of semaphores in the system
is set by SEMMNS, which consequently must be at least
as high as max_connections plus
-autovacuum_max_workers, plus one
Re: [HACKERS] Incorrect formula for SysV IPC parameters
On Wed, Feb 3, 2016 at 12:51 PM, Kyotaro HORIGUCHI wrote: > Hello, I found that the formulas to calculate SEMMNI and SEMMNS > are incorrect in 9.2 and later. > > http://www.postgresql.org/docs/9.5/static/kernel-resources.html > > All of them say that the same thing as following, > > | SEMMNI Maximum number of semaphore identifiers (i.e., sets) > | > | at least ceil((max_connections + autovacuum_max_workers + 4) / 16) > | > | SEMMNSMaximum number of semaphores system-wide > | > | ceil((max_connections + autovacuum_max_workers + 4) / 16) * 17 > |plus room for other applications > > But actually the number of semaphores PostgreSQL needs is > calculated as following in 9.4 and later. > > numSemas = MaxConnections + NUM_AUXILIARY_PROCS(=4) > MaxConnections = max_connections + autovacuum_max_workers + 1 + >max_worker_processes > > So, the formula for SEMMNI should be > > ceil((max_connections + autovacuum_max_workers + max_worker_processes + 5) / > 16) > > and SEMMNS should have the same fix. > > > In 9.3 and 9.2, the documentation says the same thing but > actually it is calculated as following, > > numSemas = MaxConnections + NUM_AUXILIARY_PROCS(=4) > MaxConnections = max_connections + autovacuum_max_workers + 1 + >GetNumShmemAttachedBgworkers() > > Omitting GetNumShmemAttachedBgworkers, the actual formula is > > ceil((max_connections + autovacuum_max_workers + 5) / 16) > > > In 9.1, NUM_AUXILIARY_PROCS is 3 so the documentations is correct. > > > I attached two patches for 9.2-9.3 and 9.4-9.6dev > respectively. patch command complains a bit on applying it on > 9.2. Good catch! ISTM that you also need to change the descriptions about SEMMNI and SEMMNS under the Table 17-1. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
