Re: Anyone interested in improving postgresql scaling?

2007-04-17 Thread Kevin Way
Kris Kennaway wrote: On Tue, Apr 17, 2007 at 02:51:55PM -0400, Kevin Way wrote: I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE and 4BSD. The results are available at: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 Unfortunately

Re: Anyone interested in improving postgresql scaling?

2007-04-17 Thread Massimo Lusetti
On Tue, 17 Apr 2007 14:51:55 -0400 Kevin Way <[EMAIL PROTECTED]> wrote: > I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE > and 4BSD. The results are available at: > > http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 > > Unfortunately,

Re: Anyone interested in improving postgresql scaling?

2007-04-17 Thread Kris Kennaway
On Tue, Apr 17, 2007 at 02:51:55PM -0400, Kevin Way wrote: > I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE > and 4BSD. The results are available at: > > http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 > > Unfortunately, I can't run t

Re: Anyone interested in improving postgresql scaling?

2007-04-17 Thread Kevin Way
Robert Watson wrote: On Tue, 10 Apr 2007, Kevin Way wrote: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only

Re: Anyone interested in improving postgresql scaling?

2007-04-17 Thread Robert Watson
On Tue, 10 Apr 2007, Kevin Way wrote: Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough r

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: > On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: >> Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is >> that Postgres never has more than one process waiting on any particular >> SysV semaphore, and so the problem doesn't re

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: > On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: >> Anyway I'd be interested to know what the test case is, and which PG >> version you were testing. > I used 8.2 (and some older version when I first noticed it a year ago) > and either sysbench

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: > I have not studied the exact code path, but there are indeed multiple > wakeups happening from the semaphore code (as many as the number of > active postgresql processes). It is easy to instrument > sleepq_broadcast() and log them when they happen. Ther

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 06:26:37PM -0400, Tom Lane wrote: > Kris Kennaway <[EMAIL PROTECTED]> writes: > > On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: > >> Anyway I'd be interested to know what the test case is, and which PG > >> version you were testing. > > > I used 8.2 (and some ol

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 05:36:17PM -0400, Tom Lane wrote: > Kris Kennaway <[EMAIL PROTECTED]> writes: > > I have not studied the exact code path, but there are indeed multiple > > wakeups happening from the semaphore code (as many as the number of > > active postgresql processes). It is easy to in

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 03:52:00PM -0400, Tom Lane wrote: > Kris Kennaway <[EMAIL PROTECTED]> writes: > > On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: > >> Oh, I'm sure the BSD kernel acts as you describe. But Mark's point is > >> that Postgres never has more than one process waiting

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 02:46:56PM -0400, Tom Lane wrote: > Kris Kennaway <[EMAIL PROTECTED]> writes: > >>> Make SYSV semaphores less dumb about process wakeups. Currently > >>> whenever the semaphore state changes, all processes sleeping on the > >>> semaphore are woken, even if we only have rele

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Kris Kennaway <[EMAIL PROTECTED]> writes: >>> Make SYSV semaphores less dumb about process wakeups. Currently >>> whenever the semaphore state changes, all processes sleeping on the >>> semaphore are woken, even if we only have released enough resources >>> for one waiting process to claim. >> Co

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 10:23:42AM -0400, Tom Lane wrote: > Mark Kirkwood <[EMAIL PROTECTED]> writes: > > Kris Kennaway wrote: > >> If so, then your task is the following: > >> > >> Make SYSV semaphores less dumb about process wakeups. Currently > >> whenever the semaphore state changes, all proc

Re: Anyone interested in improving postgresql scaling?

2007-04-10 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 10:41:04PM +1200, Mark Kirkwood wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, eve

Re: [HACKERS] Anyone interested in improving postgresql scaling?

2007-04-10 Thread Tom Lane
Mark Kirkwood <[EMAIL PROTECTED]> writes: > Kris Kennaway wrote: >> If so, then your task is the following: >> >> Make SYSV semaphores less dumb about process wakeups. Currently >> whenever the semaphore state changes, all processes sleeping on the >> semaphore are woken, even if we only have rel

Re: Anyone interested in improving postgresql scaling?

2007-04-10 Thread Maxime Henrion
Mark Kirkwood wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if we only have released enough resource

Re: Anyone interested in improving postgresql scaling?

2007-04-10 Thread Mark Kirkwood
Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim.

Re: Anyone interested in improving postgresql scaling?

2007-04-09 Thread Kris Kennaway
On Tue, Apr 10, 2007 at 12:04:32AM -0400, Kevin Way wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if

Re: Anyone interested in improving postgresql scaling?

2007-04-09 Thread Kevin Way
Kris Kennaway wrote: If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim.

Anyone interested in improving postgresql scaling?

2007-02-25 Thread Kris Kennaway
If so, then your task is the following: Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim. i.e. there is a thunde