Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission denied”

2015-10-16 Thread Dmitry Vasilyev

On Пт, 2015-10-16 at 09:02 +0530, Amit Kapila wrote:
> On Thu, Oct 15, 2015 at 8:35 PM, Dmitry Vasilyev <d.vasilyev@postgres
> pro.ru> wrote:
> > I think that function dsm_impl_windows() with EACCES error should
> > not
> > do ereport() with FATAL level. It works, but it is likely to make
> > an
> > infinite loop if the user will receive EACCES error.
> > 
> > 
> Currently we are using error level as ERROR for creating dsm during
> postmaster startup which is not right and rather we should use error
> level as LOG.  Can you please try with the attached patch and see
> if the issue is fixed for you.
> 
> Another some what related point is currently we are using random()
> function to ensure a unique name for dsm and it seems to me that
> it is always going to generate same number on first invocation (at
> least
> thats what happening on windows) due to which you are seeing the
> error.  Another options could be to append current pid or data
> directory
> path as we are doing in win32_shmem.c.  I think this could be an
> optimization which can be done in addition to the fix attached (we
> can
> do this as a separate patch as well, if we agreed to do anything).
> 
> 
> With Regards,
> Amit Kapila.
> EnterpriseDB: http://www.enterprisedb.com


In that case your patch is working. There was LOG level message with Permission 
denied (from the operation point it’s not clear how to respond to message like 
this).
But as I understand if operating system can’t create shared memory we will try 
to distinguish that chunk up to infinity.
As I see it’s better to stop instance in this case.
So I guess it’s a good idea to remain all as it is now (FATAL level) but add 
PostmasterPid to name (as you suggest). There’s patch in attachments.


Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 921f029..55335bf 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -68,6 +68,7 @@
 #include "utils/guc.h"
 #include "utils/memutils.h"
 #include "postmaster/postmaster.h"
+#include "miscadmin.h"
 
 #ifdef USE_DSM_POSIX
 static bool dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
@@ -631,7 +632,7 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
 	 * similar to main shared memory. We can change here once issue mentioned
 	 * in GetSharedMemName is resolved.
 	 */
-	snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle);
+	snprintf(name, 64, "%s.%d.%u", SEGMENT_NAME_PREFIX, PostmasterPid, handle);
 
 	/*
 	 * Handle teardown cases.  Since Windows automatically destroys the object

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission denied”

2015-10-15 Thread Dmitry Vasilyev
 I’ve created 2 unprivileged users: user1 and user2; and registered 2
services thorough pg_ctl register respectively under these 2 users. If
first service is running second could not start and vice versa.



Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission denied”

2015-10-15 Thread Dmitry Vasilyev
I think that function dsm_impl_windows() with EACCES error should not
do ereport() with FATAL level. It works, but it is likely to make an
infinite loop if the user will receive EACCES error.

On Чт, 2015-10-15 at 17:46 +0300, Dmitry Vasilyev wrote:
>  I’ve created 2 unprivileged users: user1 and user2; and registered 2
> services thorough pg_ctl register respectively under these 2 users.
> If
> first service is running second could not start and vice versa.
> 
> 
> ----
> Dmitry Vasilyev
> Postgres Professional: http://www.postgrespro.com
> Russian Postgres Company
> 
> 
> 
-- 



Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
new file mode 100644
index 921f029..417048e
*** a/src/backend/storage/ipc/dsm_impl.c
--- b/src/backend/storage/ipc/dsm_impl.c
*** dsm_impl_windows(dsm_op op, dsm_handle h
*** 689,694 
--- 689,696 
  		if (!hmap)
  		{
  			_dosmaperr(GetLastError());
+ 			if (errno == EACCES)
+ return false;
  			ereport(elevel,
  	(errcode_for_dynamic_shared_memory(),
    errmsg("could not create shared memory segment \"%s\": %m",

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-12 Thread Dmitry Vasilyev
Hello, Amit!
On Пн, 2015-10-12 at 11:25 +0530, Amit Kapila wrote:
> On Sun, Oct 11, 2015 at 9:12 PM, Tom Lane <t...@sss.pgh.pa.us> wrote:
> >
> > Magnus Hagander <mag...@hagander.net> writes:
> > > On Sun, Oct 11, 2015 at 5:22 PM, Tom Lane <t...@sss.pgh.pa.us>
> wrote:
> > >> I'm a bit suspicious that we may have leaked a handle to the
> shared
> > >> memory block someplace, for example.  That would explain why
> this
> > >> symptom is visible now when it was not in 2009.  Or maybe it's
> dependent
> > >> on some feature that we didn't test back then --- for instance,
> if
> > >> the logging collector is in use, could it have inherited a
> handle and
> > >> not closed it?
> >
> > > Even if we leaked it, it should go away when the other processes
> died.
> >
> > I'm fairly certain that we do not kill/restart the logging
> collector
> > during a database restart (because it's impossible to reproduce the
> > original stderr destination if we do).  
> 
> True and it seems this is the reason for issue we are discussing
> here.
> The reason why this happens is that during creation of shared memory
> (PGSharedMemoryCreate()), we duplicate the handle such that it
> become inheritable by all child processes.  Then during fork
> (syslogger_forkexec()->postmaster_forkexec()->internal_forkexec) we
> always inherit the handles which causes syslogger to get a copy of
> shared memory handle which it neither uses and nor closes it.
> 
> I could easily reproduce the issue if logging collector is on and
> even if
> we try to increase the loop count or sleep time
> in PGSharedMemoryCreate(),
> it doesn't change the situation as the syslogger has a valid handle
> to
> shared memory.  One way to fix is to just close the shared memory
> handle
> in sys logger as we are not going to need it and attached patch which
> does
> this fixes the issue for me.  Another invasive fix in case we want to
> retain shared memory handle for some purpose (future requirement)
> could
> be to send some signal to syslogger in restart path so that it can
> release
> the shared memory handle.
> 
> 
> 
> With Regards,
> Amit Kapila.
> EnterpriseDB: http://www.enterprisedb.com
Specified patch with "ifdef WIN32" is working for me. Maybe it’s
necessary to check open handlers from replication for example?
--
Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: [HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-12 Thread Dmitry Vasilyev
Hello Tom!
On Пн, 2015-10-12 at 16:35 -0400, Tom Lane wrote:
> I wrote:
> > This is kind of a mess :-(.  But it does look like what we want is
> > for SubPostmasterMain to do more than nothing when it chooses not
> > to
> > reattach.  Probably that should include resetting UsedShmemSegAddr
> > to
> > NULL, as well as closing the handle.
> 
> After poking around a bit more, I propose the attached patch.  I've
> checked that this is happy with an EXEC_BACKEND Unix build, but I'm
> not
> able to test it on Windows ... would somebody do that?
> 
> BTW, it appears from this that Cygwin builds have been broken right
> along
> in a different way: according to the code in sysv_shmem's
> PGSharedMemoryReAttach, Cygwin does cause a re-attach to occur, which
> we
> were not undoing for putatively-not-connected-to-shmem child
> processes.
> That's a robustness problem because it breaks the postmaster's
> expectation
> that it's safe to not reinitialize shmem after a crash of one of
> those
> processes.  I believe this patch fixes that problem as well, though
> if
> anyone can test it on Cygwin that wouldn't be a bad thing either.
> 
>   regards, tom lane
> 

This patch is working for me,
binaries: https://goo.gl/32j7QE (MSVC 2010, build script here: 
https://github.com/postgrespro/pgwininstall).


--
Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-10 Thread Dmitry Vasilyev
I have written, what service stopped. This action is repeatable.
You can run command 'psql -c "do $$ unpack p,1x8 $$ language plperlu;"'
and after this windows service will stop. 

On Сб, 2015-10-10 at 10:23 -0500, Tom Lane wrote:
> Robert Haas <robertmh...@gmail.com> writes:
> > On Fri, Oct 9, 2015 at 5:52 AM, Dmitry Vasilyev
> > > postgres=# select 1;
> > > server closed the connection unexpectedly
> > > This probably means the server terminated abnormally
> > > before or while processing the request.
> > > The connection to the server was lost. Attempting reset: Failed.
> 
> > Hmm.  I'd expect that to cause a crash-and-restart cycle, just like
> > a
> > SIGQUIT would cause a crash-and-restart cycle on Linux.  But I
> > would
> > expect the server to end up running again at the end, not stopped.
> 
> It *is* a crash and restart cycle, or at least no evidence to the
> contrary has been provided.
> 
> Whether psql's attempt to do an immediate reconnect succeeds or not
> is
> very strongly timing-dependent, on both Linux and Windows.  It's easy
> for it to attempt the reconnection before crash recovery is complete,
> and then you get the above symptom.  Personally I get a "Failed"
> result
> more often than not, regardless of platform.
> 
>   regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-10 Thread Dmitry Vasilyev
Hello Tom!

On Сб, 2015-10-10 at 10:55 -0500, Tom Lane wrote:
> Dmitry Vasilyev <d.vasil...@postgrespro.ru> writes:
> > I have written, what service stopped. This action is repeatable.
> > You can run command 'psql -c "do $$ unpack p,1x8 $$ language
> > plperlu;"'
> > and after this windows service will stop.
> 
> Well, (a) that probably means that your plperl installation is
> broken,
> and (b) you still haven't convinced me that you had an actual service
> stop, and not just that the recovery time was longer than psql would
> wait before retrying the connection.  Can you start a fresh psql
> session after waiting a few seconds?
> 
>   regards, tom lane

This is knowned bug of perl:

perl -e ' unpack p,1x8'
Segmentation fault (core dumped)

backend of postgres is crashed, and windows service is stopped:

C:\Users\vadv>sc query postgresql-X64-9.4 | findstr /i "STATE"
S
TATE  : 1  STOPPED


The log you can see bellow:

2015-10-10 19:00:13 AST LOG:  database system was interrupted; last
known up at 2015-10-10 18:54:47 AST
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 5 to 13
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 2 to 2
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 2 to 2
2015-10-10 19:00:13 AST DEBUG:  checkpoint record is at 0/16A01C8
2015-10-10 19:00:13 AST DEBUG:  redo record is at 0/16A01C8; shutdown
TRUE
2015-10-10 19:00:13 AST DEBUG:  next transaction ID: 0/678; next OID:
16393
2015-10-10 19:00:13 AST DEBUG:  next MultiXactId: 1; next
MultiXactOffset: 0
2015-10-10 19:00:13 AST DEBUG:  oldest unfrozen transaction ID: 667, in
database 1
2015-10-10 19:00:13 AST DEBUG:  oldest MultiXactId: 1, in database 1
2015-10-10 19:00:13 AST DEBUG:  transaction ID wrap limit is
2147484314, limited by database with OID 1
2015-10-10 19:00:13 AST DEBUG:  MultiXactId wrap limit is 2147483648,
limited by database with OID 1
2015-10-10 19:00:13 AST DEBUG:  starting up replication slots
2015-10-10 19:00:13 AST LOG:  database system was not properly shut
down; automatic recovery in progress
2015-10-10 19:00:13 AST DEBUG:  resetting unlogged relations: cleanup 1
init 0
2015-10-10 19:00:13 AST LOG:  redo starts at 0/16A0230
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/12057; tid 0/3
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/12059; tid 1/3
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/12060; tid 1/2
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/11979; tid 31/63
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/11984; tid 16/34
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:13 AST CONTEXT:  xlog redo insert: rel
1663/12135/11889; tid 67/5
2015-10-10 19:00:13 AST DEBUG:  mapped win32 error code 80 to 17
2015-10-10 19:00:

[HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-09 Thread Dmitry Vasilyev
I’ve started PostgreSQL server on Windows and then I kill client
backend’s process by taskkill the service was stopped: 

postgres=# select pg_backend_pid();
 pg_backend_pid

   1976

postgres=# \! taskkill /pid 1976 /f
SUCCESS: The process with PID 1976 has been terminated.
postgres=# select 1;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>


If I kill backend’s process on Linux then service not failing. So
what’s the problem? Why PostgreSQL is so strange on Windows?


--
Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres service stops when I kill client backend on Windows

2015-10-09 Thread Dmitry Vasilyev
This code stoped server too:

postgres=# do $$ unpack p,1x8 $$ language plperlu;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>




--> > 
> Hello Dmitry
> 
> > -Original Message-
> > From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-owne
> > r...@postgresql.org] On Behalf Of Dmitry Vasilyev
> > Sent: Freitag, 9. Oktober 2015 11:52
> > To: pgsql-hackers@postgresql.org
> > Subject: [HACKERS] Postgres service stops when I kill client
> > backend on Windows
> > 
> > I’ve started PostgreSQL server on Windows and then I kill client
> > backend’s process by taskkill the service was stopped:
> > 
> > postgres=# select pg_backend_pid();
> >  pg_backend_pid
> > 
> >    1976
> > 
> > postgres=# \! taskkill /pid 1976 /f
> > SUCCESS: The process with PID 1976 has been terminated.
> > postgres=# select 1;
> > server closed the connection unexpectedly
> > This probably means the server terminated abnormally
> > before or while processing the request.
> > The connection to the server was lost. Attempting reset: Failed.
> > !>
> > 
> > 
> > If I kill backend’s process on Linux then service not failing. So
> > what’s the problem? Why PostgreSQL is so strange on Windows?
> 
> I can't say what happens on windows, but I don't undestand either why
> you want to kill the session you are in.
> Besides that why don't you use pg_terminate_backend?
> 
> db=> select pg_backend_pid();
>  pg_backend_pid
> 
>    8808
> (1 row)
> 
> db=> select pg_terminate_backend(8808);
> FATAL:  terminating connection due to administrator command
> server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
> The connection to the server was lost. Attempting reset: Succeeded.
> db=> select pg_backend_pid();
>  pg_backend_pid
> 
>    8500
> (1 row)
> 
> Regards
> Charles
> 
> > 
> > 
> > --
> > Dmitry Vasilyev
> > Postgres Professional: http://www.postgrespro.com
> > The Russian Postgres Company
> > 
> > 
> > --
> > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-hackers
> 
> 
> 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Streaming Replication clusters and load balancing

2015-09-21 Thread Dmitry Vasilyev
Hi!
By default, HAproxy configuration can not be changed without breaking a
connection with the client :)

--
Dmitry Vasilyev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Fri, 2015-09-18 at 12:27 +1000, James Sewell wrote:
> Hello all,
> 
> I have recently been working with PostgreSQL and HAProxy to provide
> seamless load balancing to a group of database servers. This on it's
> own isn't a hard thing: I have an implementation finished and am now
> thinking about the best way to bring it to a production ready state
> which could be used by others, and used in load-balancers other than
> HAProxy with minimal config changes.
> 
> My initial requirements were:
> 
> Given a group of PostgreSQL servers check each x seconds and:
> Allow read/write access only to the master server (via IPA / portA)
> Disallow access if there are multiple master servers
> Allow read access to all servers (via IPB / portB) as long as the
> following holds:
> They are attached to the current master server via streaming
> replication (or they are the current master server)
> They can currently contact the master server (safest option, disallow
> all access when master-less)
> They are in the same timeline as the master server (do I need this
> check?)
> The master server reports that they have less than x bytes lag
> HAProxy can talk to PostgreSQL for a health check via TCP or PSQL
> (connection check only). Neither of these allow the logic above -
> therefore this logic has to be hosted outside of HAProxy. This might
> change in the future if HAProxy gets the ability to send SQL
> statements (like an F5 can).
> 
> Today the best way to provide this information to  HAProxy (and many
> other load balancers, application frameworks, proxies, monitoring
> solutions) is via HTTP, where HTTP 200 is pass the check and HTTP 503
> is fail the check (and don't load balance to this node). In my case I
> have a script which accepts HTTP requests to /read to check if this
> node is available for read only and /write which checks if this node
> is available for read/writes.
> 
> The options as I see them are:
> Implement a script / small app which connects to PostgreSQL and
> executes these checks
> Implemented and proven today at many independent sites
> Should it run on HAProxy server or PSQL server? 
> Integrated HTTP server  or x.inetd script?
> Platform independence?
> What if it dies?
> Implement a custom PostgreSQL BGworker which provides this
> information over HTTP
> No outside of PostgreSQL config needed
> No reliance on another daemon / interface being up
> libmicrohttpd or similar should help with  platform independence
> Security / acceptance by community?
> Only newer versions of PostgreSQL
> Spend the time working on getting SQL checks into HAProxy
> What about other platforms which only support HTTP?
> I think all of the options would benefit from a PSQL extension which
> does the following:
> Encapsulates the check logic (easier to upgrade, manipulate)
> Stores historic check data for a number of hours / days / months
> Stores defaults (override via HTTP could be possible for things like
> lag)
> Does anyone else have any thoughts on this topic? 
> 
> Eventually many cool features could flow out of this kind of work:
> Integration with High Availability products - I have this working
> with EnterpriseDB EFM now.
> Locate the current master using the HA product 
> more than one master doesn't cause loss of service as long as HA
> state is sane
> Locate all clustered standby servers using the HA product
> if a standby is removed from the HA cluster, it is removed from load
> balancing
> if a standby is not part of the cluster, it is removed from load
> balancing (even if it is part of streaming replication)
> HTTP replication status requests which facilitate dynamically
> managing HAProxy (or other) PostgreSQL server pools
> Add a node to streaming replication, it automatically shows up in the
> pool and starts being checked to see if it can service reads
> Great for cloud scale out
> Allocation of additional load balancer groups based on some criteria
> (?), for example
> read/write (as above)
> read only (as above)
> data warehouse (reporting reads only)
> DR (replica with no reads or writes - until it becomes a master)
> Keen to hear comments.
> 
> Cheers,
> 
> James Sewell,
> Solutions Architect 
> __
>  
> 
> Level 2, 50 Queen St, Melbourne VIC 3000
> 
> P (+61) 3 8370 8000  W www.lisasoft.com ; F (+61) 3 8370 8099
>  
> 
> The contents of this email are confidential and may be subject to
> legal or professional privilege and copyright. No representation is
> made that this email is free of viruses or other defects. If you have
> received this communication in error, you may not copy or distribute
> any part of it or otherwise disclose its contents to anyone. Please
> advise the sender of your incorrect receipt of this correspondence.

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Dmitry Vasilyev
We did not got any affect on core 64 with smt = 8, and we not have a 64
-cpu x86 machine with disable HT feature. 
You can set scale > 1000 and with shared_buffers >> size of index
pgbench_accounts_pkey.
You can also increase the concurrency: not only access top of b-tree
index, but also to a specific buffer: select * from pgbench_accounts
where aid = 1;


On Mon, 2015-08-31 at 17:43 +0200, Tomas Vondra wrote:
> 
> On 08/31/2015 12:54 PM, YUriy Zhuravlev wrote:
> > Hello hackers
> > 
> > Recently, we were given access to the test server is IBM, 9119-MHE
> > with 8 CPUs
> > * 8 cores * 8 threads. We decided to take advantage of this and to
> > find
> > bottlenecks for read scalability (pgbench -S).
> > 
> > All detail you can read here:
> > http://www.postgrespro.ru/blog/pgsql/2015/08/30/p8scaling
> > 
> > Performance 9.4 stopped growing after 100 clients, and 9.5 / 9.6
> > stopped after
> > 150 (at 4 NUMA nodes). After research using pref we saw that
> > inhibits
> > ProcArrayLock in GetSnaphotData. But inserting the stub instead of
> > GetSnapshotData not significantly increased scalability. Trying to
> > find the
> > bottleneck with gdb, we found another place. We have noticed s_lock
> > in
> > PinBuffer and UnpinBuffer. For the test we rewrited PinBuffer and
> > UnpinBuffer
> > by atomic operations and we liked the result. Degradation of
> > performance
> > almost completely disappeared, and went scaling up to 400 clients
> > (4 NUMA
> > nodes with 256 "CPUs").
> > 
> > To scale Postgres for large NUMA machine must be ported to the
> > atomic
> > operations bufmgr. During our tests, we no found errors in our
> > patch, but most
> > likely it is not true and this patch only for test.
> > 
> > Who has any thoughts?
> 
> Well, I could test the patch on a x86 machine with 4 sockets (64
> cores), 
> but I wonder whether it makes sense at this point, as the patch
> really 
> is not correct (judging by what Andres says).
> 
> Also, what pgbench scale was used for the testing?
> 
> regards
> 
> --
> Tomas Vondra  http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
> 
> 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers