Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Ryan Johnson
On 14/04/2014 4:30 PM, Ryan Johnson wrote: FYI, here's a plot of performance over time. Each point in the graph is throughput (in tps) over a 10-second measurement (~20 minutes total), against a 12 WH TPC-C dataset with 24 clients banging on

Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Ryan Johnson
On 14/04/2014 10:14 AM, Kevin Grittner wrote: Ryan Johnson wrote: every time I shut down a database and bring it back up, SSI seems to go slower. There's one thing to rule out up front -- that would be a long-lived prepared transacti

Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Kevin Grittner
Ryan Johnson wrote: > what factors might cause a prepared transaction to exist in the > first place? As part of a "distributed transaction" using "two phase commit" a PREPARE TRANSACTION statement would have had to run against PostgreSQL: http://www.postgresql.org/docs/current/interactive/sql-p

Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Ryan Johnson
On 14/04/2014 10:14 AM, Kevin Grittner wrote: Ryan Johnson wrote: every time I shut down a database and bring it back up, SSI seems to go slower. There's one thing to rule out up front -- that would be a long-lived prepared transaction. Please post the output of these queries: select versio

Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Kevin Grittner
Ryan Johnson wrote: > every time I shut down a database and bring it back up, SSI seems > to go slower. There's one thing to rule out up front -- that would be a long-lived prepared transaction. Please post the output of these queries: select version(); show max_prepared_transactions; select *

Re: [PERFORM] SSI slows down over time

2014-04-14 Thread Ryan Johnson
On 09/04/2014 5:21 PM, Bruce Momjian wrote: On Mon, Apr 7, 2014 at 10:38:52AM -0400, Ryan Johnson wrote: The two * entries were produced by runs under SI, and confirm that the rest of the system has not been slowing down nearly as much as SSI. SI throughput dropped by 5% as the database quadrup

[PERFORM] SSI slows down over time

2014-04-14 Thread Ryan Johnson
Hi all, (Referred here from pgsql-performance) tl;dr: every time I shut down a database and bring it back up, SSI seems to go slower. In order to avoid thousands of SSI aborts due to running out of shared memory, I've had to set max_predicate_locks to several thousand (2000 is tolerable, 8000

Re: [PERFORM] SSI slows down over time

2014-04-12 Thread Ryan Johnson
On 06/04/2014 10:55 AM, Tom Lane wrote: Ryan Johnson writes: I get a strange behavior across repeated runs: each 100-second run is a bit slower than the one preceding it, when run with SSI (SERIALIZABLE). ... So the question: what should I look for to diagnose/triage this problem? In the past

Re: [PERFORM] SSI slows down over time

2014-04-09 Thread Bruce Momjian
On Mon, Apr 7, 2014 at 10:38:52AM -0400, Ryan Johnson wrote: > The two * entries were produced by runs under SI, and confirm that > the rest of the system has not been slowing down nearly as much as > SSI. SI throughput dropped by 5% as the database quadrupled in size. > SSI throughput dropped by

Re: [PERFORM] SSI slows down over time

2014-04-07 Thread Ryan Johnson
On 05/04/2014 10:25 PM, Ryan Johnson wrote: Hi all, Disclaimer: this question probably belongs on the hackers list, but the instructions say you have to try somewhere else first... toss-up between this list and a bug report; list seemed more appropriate as a starting point. Happy to file a bu

Re: [PERFORM] SSI slows down over time

2014-04-07 Thread Ryan Johnson
On 06/04/2014 10:55 AM, Tom Lane wrote: Ryan Johnson writes: I get a strange behavior across repeated runs: each 100-second run is a bit slower than the one preceding it, when run with SSI (SERIALIZABLE). ... So the question: what should I look for to diagnose/triage this problem? In the past

Re: [PERFORM] SSI slows down over time

2014-04-07 Thread Ryan Johnson
On 06/04/2014 4:30 AM, Heikki Linnakangas wrote: On 04/06/2014 05:25 AM, Ryan Johnson wrote: I've tried linux perf, but all it says is that lots of time is going to LWLock (but callgraph tracing doesn't work in my not-bleeding-edge kernel). Make sure you compile with the "-fno-omit-frame-point

Re: [PERFORM] SSI slows down over time

2014-04-06 Thread Tom Lane
Ryan Johnson writes: > I get a strange behavior across repeated runs: each 100-second run is a > bit slower than the one preceding it, when run with SSI (SERIALIZABLE). > ... So the question: what should I look for to diagnose/triage this problem? In the past I've seen behaviors like this that

Re: [PERFORM] SSI slows down over time

2014-04-06 Thread Heikki Linnakangas
On 04/06/2014 05:25 AM, Ryan Johnson wrote: I've tried linux perf, but all it says is that lots of time is going to LWLock (but callgraph tracing doesn't work in my not-bleeding-edge kernel). Make sure you compile with the "-fno-omit-frame-pointer" flag. - Heikki -- Sent via pgsql-performance

[PERFORM] SSI slows down over time

2014-04-05 Thread Ryan Johnson
Hi all, Disclaimer: this question probably belongs on the hackers list, but the instructions say you have to try somewhere else first... toss-up between this list and a bug report; list seemed more appropriate as a starting point. Happy to file a bug if that's more appropriate, though. This