[PERFORM] semaphore waits and performance stall

2014-03-26 Thread Matthew Spilich
Hi everyone! I've been working on a puzzling issue for a few days am am hoping that someone has seen something similar or can help. There have been some odd behaviors on one of my production facing postgres servers. version info from postgres: PostgreSQL 9.1.9 on x86_64-unknown-linux-gnu, c

Re: [PERFORM] Stalls on PGSemaphoreLock

2014-03-26 Thread Pavy Philippe
Here, we were the transparent hugepage always actif: cat /sys/kernel/mm/redhat_transparent_hugepage/enabled [always] never We changed to: cat /sys/kernel/mm/redhat_transparent_hugepage/enabled always [never] For the semaphore, our initial configuration was: cat /

RE : [PERFORM] Stalls on PGSemaphoreLock

2014-03-26 Thread Pavy Philippe
Hello Recently I have a similar problem. The first symptom was a freeze of the connection and 100% of CPU SYS during 2 et 10 minutes, 1 or 2 times per day. Connection impossible, slow query. The strace on one backend show a very long system call on semop(). We have a node with 48 cores dans 128

Re: [PERFORM] Connection pooling - Number of connections

2014-03-26 Thread Brett Wooldridge
Hi, Brett Wooldridge here, one of the principals of HikariCP. I thought I'd wade into the conversation pool a little myself if you guys don't mind. Speaking to David's point... >> Reaching the maxPoolSize from the minPoolSize means creating the >> connections at the crucial moment where the clie

Re: [PERFORM] Connection pooling - Number of connections

2014-03-26 Thread Josh Berkus
On 03/24/2014 06:27 AM, Brett Wooldridge wrote: > This was one of the reasons I was proposing the fixed pool design. In my > experience, even in pools that maintain a minimum number of idle > connections, responding to spike demands is problematic. If you have a > pool with say 30 max. connection

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
On Wed, Mar 26, 2014 at 5:14 PM, Shaun Thomas wrote: > * Checkpoints must commit dirty shared buffers to disk. The larger this is, > the more risk you have when checkpoints come, up to and including an > unresponsive database. Writing to disks isn't free, and sadly this is still > on the slower

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Shaun Thomas
> On most machines the limit is higher than you'd ever want to set it. I > have a set of servers with 1TB RAM and shared buffers on them is set > to 10G and even that is probably higher than it needs to be. The old > 1/4 of memory advice comes from the days when db server memory > was in the 1 to

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread desmodemone
Yes, I rember was 1024*G*b , sorry, 2014-03-26 14:23 GMT+01:00 Albe Laurenz : > desmodemone wrote: > > > max is 1024mb. > > That must be a typo. > It can surely be much higher. > > Yours, > Laurenz Albe >

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Scott Marlowe
On Wed, Mar 26, 2014 at 6:21 AM, Alexey Vasiliev wrote: > I read from several sources, what maximum shared_buffers is 8GB. > > Does this true? If yes, why exactly this number is maximum number of > shared_buffers for good performance (on Linux 64-bits)? On most machines the limit is higher than y

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Martin French
Markella Skempri wrote on 26/03/2014 13:47:47: > > Thanks Martin, > However this is a database dedicated server and nothing much else is > running on it. Also, I never saw this happening with 9.2 – but I > can’t vouch for the size of files that I was uploading. > > From: Martin French > S

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Markella Skempri
Thanks Martin, However this is a database dedicated server and nothing much else is running on it. Also, I never saw this happening with 9.2 – but I can’t vouch for the size of files that I was uploading. From: Martin French Sent: Wednesday, March 26, 2014 1:32 PM To: Markella Skempri Cc:

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Martin French
> max is 1024mb. > you have to test your workload if it's too low you will get too much > i/o ( the filesystem cache could help.. not always /*nfs*/), if too > high your cpu will be eated by lru/ latch/ and so on. > Mat Dba The max is most certainly NOT 1024MB. http://www.postgresql.org/docs/9.

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Martin French
> I wanted to follow up from this question. I’m running on 9.3.4 > My DB server has 32GB ram so I have assigned 8GB > shared_buffer_memory. It is quite a big db but with not much > traffic. When there is traffic, it’s usually big. > > Lately, the kernel has been killing the postmaster for havin

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Markella Skempri
I wanted to follow up from this question. I’m running on 9.3.4 My DB server has 32GB ram so I have assigned 8GB shared_buffer_memory. It is quite a big db but with not much traffic. When there is traffic, it’s usually big. Lately, the kernel has been killing the postmaster for having assigned t

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Albe Laurenz
desmodemone wrote: > max is 1024mb. That must be a typo. It can surely be much higher. Yours, Laurenz Albe -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread desmodemone
Il 26/mar/2014 13:36 "Ilya Kosmodemiansky" < ilya.kosmodemian...@postgresql-consulting.com> ha scritto: > > Hi Alexey, > > On Wed, Mar 26, 2014 at 1:21 PM, Alexey Vasiliev wrote: > > I read from several sources, what maximum shared_buffers is 8GB. > > I believe that was an issue on some older vers

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Ilya Kosmodemiansky
Hi Alexey, On Wed, Mar 26, 2014 at 1:21 PM, Alexey Vasiliev wrote: > I read from several sources, what maximum shared_buffers is 8GB. I believe that was an issue on some older versions, and thats why was mentioned in several talks. Today it is a sort of apocrypha. > Does this true? If yes, why

[PERFORM] Why shared_buffers max is 8GB?

2014-03-26 Thread Alexey Vasiliev
I read from several sources, what maximum shared_buffers is 8GB. Does this true? If yes, why exactly this number is maximum number of  shared_buffers for good performance (on Linux 64-bits)? Thanks!