RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-10-03 Thread REIX, Tony
VIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Wed, Sep 4, 2019 at 10:30 AM Alvaro Herrera wrote: > On 2019-Feb-03, Thomas Munro wrote: > > On Sat, Feb 2, 2019 at 12:49 AM Thomas Munro > > wrote: > > > I am planning to commit the 0001 patch shortly

RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-10-01 Thread REIX, Tony
2019 14:39 À : REIX, Tony Cc : Thomas Munro ; Andres Freund ; Robert Haas ; Pg Hackers ; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? Hi Tony, On 2019-Sep-27, REIX, Tony wrote: > Hello Thomas, Alvaro, > > Sorry for the late answer, I missed your m

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-27 Thread Alvaro Herrera
Hi Tony, On 2019-Sep-27, REIX, Tony wrote: > Hello Thomas, Alvaro, > > Sorry for the late answer, I missed your message of September 10. (I'm > working on several different projects in parallel.) > Let me talk with Sylvie ASAP and see when I will be able to test it, probably > next week, Tuesd

RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-27 Thread REIX, Tony
Tony De : Alvaro Herrera Envoyé : jeudi 26 septembre 2019 21:22 À : Thomas Munro Cc : REIX, Tony ; Andres Freund ; Robert Haas ; Pg Hackers ; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On 2019-Sep-10, Thomas Munro wrote: > Here&#

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-10, Thomas Munro wrote: > Here's a quick rebase. It needs testing, review and (probably) > adjustment from AIX users. I'm not going to be able to do anything > with it on my own due to lack of access, though I'm happy to help get > this committed eventually. If we don't get any trac

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-09 Thread Thomas Munro
On Wed, Sep 4, 2019 at 10:30 AM Alvaro Herrera wrote: > On 2019-Feb-03, Thomas Munro wrote: > > On Sat, Feb 2, 2019 at 12:49 AM Thomas Munro > > wrote: > > > I am planning to commit the 0001 patch shortly, unless there are > > > objections. I attach a new version, which improves the documentatio

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-03 Thread Alvaro Herrera
On 2019-Feb-03, Thomas Munro wrote: > On Sat, Feb 2, 2019 at 12:49 AM Thomas Munro > wrote: > > I am planning to commit the 0001 patch shortly, unless there are > > objections. I attach a new version, which improves the documentation > > a bit (cross-referencing the new GUC and the section on sy

RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-07 Thread REIX, Tony
d3ttkJL8JIc.&URL=http%3a%2f%2fwww.atos.net%2f> De : Thomas Munro Envoyé : jeudi 7 février 2019 03:30 À : REIX, Tony Cc : EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Thu, Feb 7, 2019 at 4:08 AM REIX, Tony wrote: > I&#

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-06 Thread Thomas Munro
On Sun, Feb 3, 2019 at 10:56 PM Thomas Munro wrote: > Committed 0001. > I've moved the CF entry to the next Commitfest, since we still have to > fix up and commit the 0002 patch for AIX. For the record, one problem with the shared_memory_type=sysv patch as committed is that if you set huge_pages

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-03 Thread Thomas Munro
On Sat, Feb 2, 2019 at 12:49 AM Thomas Munro wrote: > I am planning to commit the 0001 patch shortly, unless there are > objections. I attach a new version, which improves the documentation > a bit (cross-referencing the new GUC and the section on sysctl > settings). That will give us shared_mem

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-01 Thread Thomas Munro
On Sat, Feb 2, 2019 at 2:23 AM REIX, Tony wrote: > Hu I have an issue here: > > # cd /home2/freeware/src/packages/BUILD/postgresql-11.1/64bit/doc/src/sgml > # gmake > /opt/freeware/bin/xmllint --path . --noout --valid postgres.sgml > /opt/freeware/bin/xsltproc --path . --stringparam pg.version

RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-01 Thread REIX, Tony
ovd3ttkJL8JIc.&URL=http%3a%2f%2fwww.atos.net%2f> De : Thomas Munro Envoyé : vendredi 1 février 2019 14:49:01 À : REIX, Tony Cc : Andres Freund; Robert Haas; Pg Hackers; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? Bonjour Tony, On Sat, Jan 5,

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-02-01 Thread Thomas Munro
Bonjour Tony, On Sat, Jan 5, 2019 at 3:35 AM REIX, Tony wrote: > Thanks for the 2 patches! > > I've seen also the discussion around this subject. Very interesting. Should I > wait for a decision been taken? or should I study and experiment your patches > before? I am planning to commit the 000

RE: Shared Memory: How to use SYSV rather than MMAP ?

2019-01-04 Thread REIX, Tony
eund; Robert Haas; Pg Hackers; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Wed, Dec 19, 2018 at 4:17 AM REIX, Tony wrote: > Here is the patch we are using now on AIX for enabling SysV shm for AIX, > which improves greatly the performance on AIX. > &g

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-01-04 Thread Peter Eisentraut
On 27/12/2018 00:53, Thomas Munro wrote: > mmap and sysv were the same, but there > did seem to be a measurable speed-up available at high client counts > with kern.ipc.shm_use_phys=1 and thus for sysv only, for people > prepared to set 3 sysctls and this proposed new GUC. Maybe the effect > would

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-30 Thread Thomas Munro
On Thu, Dec 27, 2018 at 7:25 AM Andres Freund wrote: > On December 26, 2018 6:48:31 PM GMT+01:00, Robert Haas > wrote: > >I disagree. I think there is a growing body of evidence that > >b0fc0df9364d2d2d17c0162cf3b8b59f6cb09f67 killed performance on many > >types of non-Linux systems. This is t

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Thomas Munro
On Thu, Dec 27, 2018 at 8:59 AM Thomas Munro wrote: > ... So far I can't reproduce the effect here, probably due to > lack of cores and probably also various changes that have been made > (but not the main ones described in that report, apparently). I decided to blow today's coffee money on a cou

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Robert Haas
On Wed, Dec 26, 2018 at 1:57 PM Tom Lane wrote: > You misread what I said. I don't say that we shouldn't fix this; > what I'm saying is we should not do so via a user-configurable knob. > We should be able to auto-configure this and just handle it internally. > I have zero faith in the idea that

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Thomas Munro
On Thu, Dec 27, 2018 at 6:48 AM Robert Haas wrote: > On Wed, Dec 26, 2018 at 11:43 AM Tom Lane wrote: > > Thomas Munro writes: > > > Since it's not fixing a bug, we wouldn't back-patch that into existing > > > releases. But I agree that we should do something like this for > > > PostgreSQL 12,

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Tom Lane
Robert Haas writes: > On Wed, Dec 26, 2018 at 11:43 AM Tom Lane wrote: >> I'm -1 on making this user configurable via a GUC; that adds documentation >> and compatibility burdens that we don't need, for something of no value >> to 99.99% of users. > ... > You may be right that this is of no value

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Andres Freund
On December 26, 2018 6:48:31 PM GMT+01:00, Robert Haas wrote: >I disagree. I think there is a growing body of evidence that >b0fc0df9364d2d2d17c0162cf3b8b59f6cb09f67 killed performance on many >types of non-Linux systems. This is the first report I recall about >AIX, but there have been prev

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Robert Haas
On Wed, Dec 26, 2018 at 11:43 AM Tom Lane wrote: > Thomas Munro writes: > > Since it's not fixing a bug, we wouldn't back-patch that into existing > > releases. But I agree that we should do something like this for > > PostgreSQL 12, and I think we should make it user configurable. > > I'm -1 on

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-26 Thread Tom Lane
Thomas Munro writes: > Since it's not fixing a bug, we wouldn't back-patch that into existing > releases. But I agree that we should do something like this for > PostgreSQL 12, and I think we should make it user configurable. I'm -1 on making this user configurable via a GUC; that adds documenta

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-25 Thread Thomas Munro
On Wed, Dec 19, 2018 at 4:17 AM REIX, Tony wrote: > Here is the patch we are using now on AIX for enabling SysV shm for AIX, > which improves greatly the performance on AIX. > > It is compile time. > > It seems to me that you'd like this to become a shared_memory_type GUC. > Correct? However, I

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-12-18 Thread REIX, Tony
8JIc.&URL=http%3a%2f%2fwww.atos.net%2f> De : REIX, Tony Envoyé : lundi 26 novembre 2018 18:00:15 À : Thomas Munro Cc : Andres Freund; Robert Haas; Pg Hackers; EMPEREUR-MOT, SYLVIE; BERGAMINI, DAMIEN Objet : RE: Shared Memory: How to use SYSV rather than MMAP ?

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-26 Thread REIX, Tony
Hi Thomas, You said: I think we should respect the huge_pages GUC, as we do on Linux and Windows (since there are downsides to using large pages, maybe not everyone would want that). It could even be useful to allow different page sizes to be requested by GUC (I see that DB2 has an option to u

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-26 Thread REIX, Tony
as; Pg Hackers; EMPEREUR-MOT, SYLVIE; BERGAMINI, DAMIEN Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Sat, Nov 24, 2018 at 4:54 AM REIX, Tony wrote: > Here is a patch for enabling SystemV Shared Memory on AIX, for 64K or bigger > page size, rather than using MMAP shared memory

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-23 Thread Thomas Munro
On Sat, Nov 24, 2018 at 4:54 AM REIX, Tony wrote: > Here is a patch for enabling SystemV Shared Memory on AIX, for 64K or bigger > page size, rather than using MMAP shared memory, which is slower on AIX. > We have tested this code with 64K pages and pgbench, on AIX 7.2 TL2 Power 9, > and it pro

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-23 Thread REIX, Tony
fwww.atos.net%2f> De : REIX, Tony Envoyé : mercredi 21 novembre 2018 09:45:12 À : Thomas Munro; Andres Freund Cc : Robert Haas; Pg Hackers; EMPEREUR-MOT, SYLVIE; BERGAMINI, DAMIEN Objet : RE: Shared Memory: How to use SYSV rather than MMAP ? Hi Thomas, Andres, I still h

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-21 Thread REIX, Tony
Hi Thomas, Andres, I still have to reread/study in depth the discussion in this thread in order to understand all these information. However, we've already got a very good performance improvement of pgbench on AIX 7.2 / Power9 with that change: + ~38% in best case. See below for the details.

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 9:07 AM Andres Freund wrote: > On 2018-11-21 09:00:58 +1300, Thomas Munro wrote: > > On Wed, Nov 21, 2018 at 4:37 AM REIX, Tony wrote: > > > YES ! Reading this file, your suggestion should work ! Thx ! > > > > > > I've rebuilt and run the basic tests. We'll relaunch our te

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Andres Freund
Hi, On 2018-11-21 09:00:58 +1300, Thomas Munro wrote: > On Wed, Nov 21, 2018 at 4:37 AM REIX, Tony wrote: > > YES ! Reading this file, your suggestion should work ! Thx ! > > > > I've rebuilt and run the basic tests. We'll relaunch our tests asap. > > I would be surprised if that makes a differe

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 4:37 AM REIX, Tony wrote: > YES ! Reading this file, your suggestion should work ! Thx ! > > I've rebuilt and run the basic tests. We'll relaunch our tests asap. I would be surprised if that makes a difference: anonymous-mmap-then-fork and SysV shm are just two different w

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread REIX, Tony
novembre 2018 15:53:11 À : REIX, Tony Cc : pgsql-hack...@postgresql.org; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Tue, Nov 20, 2018 at 8:36 AM REIX, Tony wrote: > We are trying to understand why pgbench on AIX is slower compared to > Linux/Power on

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Robert Haas
On Tue, Nov 20, 2018 at 8:36 AM REIX, Tony wrote: > We are trying to understand why pgbench on AIX is slower compared to > Linux/Power on the same HW/Disks. > > So, we have yet no idea about what may be the root cause and what should be > changed. > > So, changing: dynamic_shared_memory_type

RE: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread REIX, Tony
postgresql.org; EMPEREUR-MOT, SYLVIE Objet : Re: Shared Memory: How to use SYSV rather than MMAP ? On Tue, Nov 20, 2018 at 5:11 AM REIX, Tony wrote: > On AIX, since with MMAP we have only 4K pages though we can have 64K pages > with SYSV, we'd like to experiment with SYSV rather

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Robert Haas
On Tue, Nov 20, 2018 at 5:11 AM REIX, Tony wrote: > On AIX, since with MMAP we have only 4K pages though we can have 64K pages > with SYSV, we'd like to experiment with SYSV rather than MMAP and measure the > impact to the performance. Are you trying to move the main shared memory segment or th

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
On Tue, Nov 20, 2018 at 11:11 PM REIX, Tony wrote: > On AIX, since with MMAP we have only 4K pages though we can have 64K pages > with SYSV, we'd like to experiment with SYSV rather than MMAP and measure the > impact to the performance. > > Looking at file: src/include/storage/dsm_impl.h , it se

Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread REIX, Tony
Hi, On AIX, since with MMAP we have only 4K pages though we can have 64K pages with SYSV, we'd like to experiment with SYSV rather than MMAP and measure the impact to the performance. Looking at file: src/include/storage/dsm_impl.h , it seemed to me that replacing the line: #define DEFAULT_