Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-11-04 Thread Robert Creager
On Thu, 20 Oct 2005 17:35:31 -0400 Tom Lane <[EMAIL PROTECTED]> wrote: > Robert Creager <[EMAIL PROTECTED]> writes: > > Interesting. 7.4.1 is worse for this test, as two jump up to 130k. But, my > > app runs fine against 7.4.1... > > > Would it still be helpful to try and pull together a test c

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-11-04 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > I have a Perl script, a Perl module and a 1Mb database (from pg_dump > -F c). Are you interested at this time in receiving this? Sure. Please send it off-list, of course. > An upgrade to RC2 might occur when RC2 comes out, unless there would > be gre

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-24 Thread Qingqing Zhou
"Simon Riggs" <[EMAIL PROTECTED]> wrote >> >> How do other databases deal with this? I can't imagine we are the only >> ones. Are we doing something different than them? > > I'm not sure the people qualified to answer that are able to do so. What > do other OSS projects do about this is more lik

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-24 Thread Simon Riggs
On Fri, 2005-10-21 at 17:17 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Certainly there is a lack of ideas as to how to fix it, as you mention > > in (3). This shows to me that the solution lies in one of two areas: a) > > the solution has not yet been considered or b) the

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-24 Thread Simon Riggs
On Mon, 2005-10-24 at 13:08 -0500, Kevin Grittner wrote: > Try getting lucky in google with "spinlock contention is greater" Kevin's reference is actually to a Sybase manual page; very interesting. This refers to the idea of subdividing the lock into may partitions. That's already been discussed,

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot

2005-10-24 Thread Kevin Grittner
Try getting lucky in google with "spinlock contention is greater" >>> Bruce Momjian >>> How do other databases deal with this? I can't imagine we are the only ones. Are we doing something different than them? ---(end of broadcast)--- TIP 1: if

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-24 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Certainly there is a lack of ideas as to how to fix it, as you mention > > in (3). This shows to me that the solution lies in one of two areas: a) > > the solution has not yet been considered or b) the solution has already > > been thoug

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Certainly there is a lack of ideas as to how to fix it, as you mention > in (3). This shows to me that the solution lies in one of two areas: a) > the solution has not yet been considered or b) the solution has already > been thought of and for whatever rea

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-21 Thread Simon Riggs
On Fri, 2005-10-21 at 09:52 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > It would be good right now to have a multi-process test harness that > > would allow us to test out different spin lock code without the rest of > > PostgreSQL getting in the way of testing. If we can i

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot

2005-10-21 Thread Kevin Grittner
Remember the suggestion I made that PostgreSQL add the capability to define named caches and bind specific objects to those caches? One of the reasons Sybase recommends using such named caches (per their performance tuning documentation) is to reduce spinlock contention. I don't know whether Postg

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > It would be good right now to have a multi-process test harness that > would allow us to test out different spin lock code without the rest of > PostgreSQL getting in the way of testing. If we can isolate the issue > outside of PostgreSQL it will be much ea

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-21 Thread Simon Riggs
On Thu, 2005-10-20 at 16:54 -0600, Robert Creager wrote: > On Thu, 20 Oct 2005 23:28:21 +0100 > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > If the CS is the same, then it will tell us that the issue is not data > > dependent. If the CS drops, it tells us that it is an activity performed > > on t

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-20 Thread Robert Creager
On Thu, 20 Oct 2005 23:28:21 +0100 Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2005-10-20 at 14:59 -0600, Robert Creager wrote: > > On Thu, 20 Oct 2005 21:19:18 +0100 > > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > > Try this to recreate the problem: > > > http://archives.postgresql.org/p

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-20 Thread Simon Riggs
On Thu, 2005-10-20 at 14:59 -0600, Robert Creager wrote: > On Thu, 20 Oct 2005 21:19:18 +0100 > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > Try this to recreate the problem: > > http://archives.postgresql.org/pgsql-performance/2004-04/msg00280.php > > > > Yup, that does it. Three hits the leve

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-20 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > Interesting. 7.4.1 is worse for this test, as two jump up to 130k. But, my > app > runs fine against 7.4.1... > Would it still be helpful to try and pull together a test case from my app > against 8.1beta3? Yes, if you can show a case where 8.1 is m

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-20 Thread Robert Creager
On Thu, 20 Oct 2005 21:19:18 +0100 Simon Riggs <[EMAIL PROTECTED]> wrote: > Try this to recreate the problem: > http://archives.postgresql.org/pgsql-performance/2004-04/msg00280.php > Yup, that does it. Three hits the level I see with my application ~100k. Two hits about 50k, one does nothing

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-20 Thread Simon Riggs
On Tue, 2005-10-18 at 12:24 -0600, Robert Creager wrote: > Robert Creager <[EMAIL PROTECTED]> wrote: > > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > Please try this patch and see if it reduces the CS storm: > Sorry, didn't work. Took about an hour, and now it's now at the CS storm > (averaging 9

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-18 Thread Josh Berkus
Robert, > Anyone know of a script that can replay a PostgreSQL log file? Then I > could log all queries, wait till the problem hits, and then replay to > see if that reproduces it... log_statement=true in your postgresql.conf file. The trick is weeding out all the other non-query stuff. -- -

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-18 Thread Robert Creager
On Tue, 18 Oct 2005 12:24:03 -0600 Robert Creager <[EMAIL PROTECTED]> wrote: > On Tue, 18 Oct 2005 10:29:43 -0600 > Robert Creager <[EMAIL PROTECTED]> wrote: > > > On Tue, 18 Oct 2005 00:25:25 +0100 > > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > > > Please try this patch and see if it reduce

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-18 Thread Robert Creager
On Tue, 18 Oct 2005 10:29:43 -0600 Robert Creager <[EMAIL PROTECTED]> wrote: > On Tue, 18 Oct 2005 00:25:25 +0100 > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > Please try this patch and see if it reduces the CS storm: > > Sorry, didn't work. Took about an hour, and now it's now at the CS st

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-18 Thread Robert Creager
On Tue, 18 Oct 2005 00:25:25 +0100 Simon Riggs <[EMAIL PROTECTED]> wrote: > > Please try this patch and see if it reduces the CS storm: > > http://archives.postgresql.org/pgsql-patches/2005-10/msg00091.php Yes, I will. I'd been trying to figure out what triggered it, as I was unable to reproduc

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-17 Thread Simon Riggs
On Thu, 2005-10-13 at 21:20 -0600, Robert Creager wrote: > When grilled further on (Thu, 13 Oct 2005 22:44:54 -0400), > Tom Lane <[EMAIL PROTECTED]> confessed: > > > Robert Creager <[EMAIL PROTECTED]> writes: > > > I've been having this problem since trying to upgrade from 7.4.1 to 8.03, > and > >

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of 8.1beta3

2005-10-13 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > Is there anything I might be able to do (without the test case) that > would help figure out what's happening? oprofile stats would be enlightening, perhaps. I'm particularly interested in why 7.4 is behaving better than newer versions --- that does no

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of

2005-10-13 Thread Robert Creager
When grilled further on (Thu, 13 Oct 2005 22:44:54 -0400), Tom Lane <[EMAIL PROTECTED]> confessed: > Robert Creager <[EMAIL PROTECTED]> writes: > > I've been having this problem since trying to upgrade from 7.4.1 to 8.03, and > > now 8.1. > > Can you put together a test case that other people cou

Re: [HACKERS] Seeing context switch storm with 10/13 snapshot of 8.1beta3

2005-10-13 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > I've been having this problem since trying to upgrade from 7.4.1 to 8.03, and > now 8.1. Can you put together a test case that other people could use to reproduce it? regards, tom lane ---(end of broadca

[HACKERS] Seeing context switch storm with 10/13 snapshot of 8.1beta3

2005-10-13 Thread Robert Creager
I've been having this problem since trying to upgrade from 7.4.1 to 8.03, and now 8.1. It's a dual Xenon machine: Linux annette.stortek.com 2.4.22-26mdkenterprise #1 SMP Wed Jan 7 07:10:39 MST 2004 i686 unknown unknown GNU/Linux PostgreSQL version is: