Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 05:08:17PM +0200, Andres Freund wrote: > On 2014-10-04 11:07:13 -0400, Bruce Momjian wrote: > > On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: > > > On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: > > > > On 2014-10-03 19:54:36 -0400, Tom Lane

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Andres Freund
On 2014-10-04 11:07:13 -0400, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: > > On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: > > > On 2014-10-03 19:54:36 -0400, Tom Lane wrote: > > > > Bruce Momjian writes: > > > > > Agreeed. Also, reality

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: > On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: > > On 2014-10-03 19:54:36 -0400, Tom Lane wrote: > > > Bruce Momjian writes: > > > > Agreeed. Also, reality check --- we can't change postgresql.conf easily > > > > wit

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Mark Kirkwood
On 04/10/14 12:10, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:00:36PM +1300, Mark Kirkwood wrote: I don't think we can offer absolutely accurate tuning advice, but I'm sure we can give some guidance. Let me try. +1 I think it is ok to document our reason for providing the new GUC - alon

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 5:23 PM, Andres Freund wrote: How are we ever going to be able to tune it further without feedback from actual production usage? With improved targeted synthetic test cases that isolate the bottleneck to where it's really obvious, way more obvious than it will ever be in productio

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Andres Freund writes: > On 2014-10-03 19:54:36 -0400, Tom Lane wrote: >> Good point: independently of all else, it's a bit late to be adding new >> features to 9.4. > This is getting absurd. The feature was there three days ago. Well, an undocumented feature isn't a feature.

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Josh Berkus
On 10/03/2014 05:04 PM, Bruce Momjian wrote: > On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: >> On 2014-10-03 19:54:36 -0400, Tom Lane wrote: >>> Bruce Momjian writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily without an initdb, and I think

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: > On 2014-10-03 19:54:36 -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > Agreeed. Also, reality check --- we can't change postgresql.conf easily > > > without an initdb, and I think our last 9.4 initdb is going to be > > > 9.4b

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 19:54:36 -0400, Tom Lane wrote: > Bruce Momjian writes: > > Agreeed. Also, reality check --- we can't change postgresql.conf easily > > without an initdb, and I think our last 9.4 initdb is going to be > > 9.4beta3, which is going to be packaged on Monday. > > Good point: independe

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Bruce Momjian writes: > Agreeed. Also, reality check --- we can't change postgresql.conf easily > without an initdb, and I think our last 9.4 initdb is going to be > 9.4beta3, which is going to be packaged on Monday. Good point: independently of all else, it's a bit late to be adding new feature

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 07:39:25PM -0400, Greg Smith wrote: > I do not disagree with you fundamentally here: this *is* worth > refining further, for sure, and the gains might be even greater if > that keeps going. My guess is just that the Postgres community > would not get a net benefit chasing t

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 10:11 AM, Andres Freund wrote: So 25% performance on a relatively small machine improvements aren't worth a GUC? That are likely to be larger on a bigger machine? I utterly fail to see why that's a service to our users. I didn't say that. I said I don't think they're worth a GUC t

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 12:00:36PM +1300, Mark Kirkwood wrote: > >I don't think we can offer absolutely accurate tuning advice, but I'm > >sure we can give some guidance. Let me try. > > > > +1 > > I think it is ok to document our reason for providing the new GUC - > along with that fact that it

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Mark Kirkwood
On 04/10/14 11:21, Andres Freund wrote: On 2014-10-03 18:16:28 -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers aren't

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:16:28 -0400, Bruce Momjian wrote: > On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: > > > Do we really want to expose a setting a few of us _might_ ask customers > > > to change? > > > > They also will try that themselves. Our customers aren't a horde of dumb > > pe

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: > > Do we really want to expose a setting a few of us _might_ ask customers > > to change? > > They also will try that themselves. Our customers aren't a horde of dumb > people. Some of them are willing to try things if they hit scalab

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:08:56 -0400, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: > > On 2014-10-03 17:55:19 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: > > > >> Well, I think the issue is that havi

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: > On 2014-10-03 17:55:19 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: > > >> Well, I think the issue is that having a GUC that can't reasonably be > > >> tuned by 95% of our

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Andres Freund writes: > On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: >> Well, I think the issue is that having a GUC that can't reasonably be >> tuned by 95% of our users is nearly useless. Few users are going to run >> benchmarks to see what the optimal value is. > It's possible to convin

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 17:55:19 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: > >> Well, I think the issue is that having a GUC that can't reasonably be > >> tuned by 95% of our users is nearly useless. Few users are going to run > >> benchmarks to s

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: > > On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: > > > On 10/3/14, 8:26 AM, Andres Freund wrote: > > > >#define NUM_XLOGINSERT_LOCKS 1 > > > >tps = 52.711939 (including connect

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Merlin Moncure
On Fri, Oct 3, 2014 at 2:20 PM, Robert Haas wrote: > On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas > wrote: >> I stand by my decision to make it a #define, at least until someone voices >> their objection in the form of a documentation patch. > > I think that's exactly right. If we knew use

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas wrote: > I stand by my decision to make it a #define, at least until someone voices > their objection in the form of a documentation patch. I think that's exactly right. If we knew users should tune this, then we might be able to make it self-tu

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 09:42 PM, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: > Every GUC add complexity to the system because people have to understand > it to know if they should tune it. No GUC is zero-cost. Please see my blog post about the cost

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: > > Every GUC add complexity to the system because people have to understand > > it to know if they should tune it. No GUC is zero-cost. > > Please see my blog post about the cost of adding GUCs: > > http://mom

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Bruce Momjian writes: > On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: >> Not all GUC need to be straight forward to tune. >> If the gains are worthy I don't see any reason not to have it. > Every GUC add complexity to the system because people have to understand > it to know if th

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 3:10 PM, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: > > On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: > > > I remember Informix had a setting that had no description except > "try > > > different values to s

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: > > I remember Informix had a setting that had no description except "try > > different values to see if it helps performance" --- we don't want to do > > that.

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: > I remember Informix had a setting that had no description except "try > different values to see if it helps performance" --- we don't want to do > that. > > What if we emit a server message if the setting is too low?

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 1:40 PM, Bruce Momjian wrote: > On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: > > On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: > > > On 10/3/14, 8:26 AM, Andres Freund wrote: > > > >#define NUM_XLOGINSERT_LOCKS 1 > > > >tps = 52.711939 (including con

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: > On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: > > On 10/3/14, 8:26 AM, Andres Freund wrote: > > >#define NUM_XLOGINSERT_LOCKS 1 > > >tps = 52.711939 (including connections establishing) > > >#define NUM_XLOGINSERT_LOCKS 8 > > >

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: > On 10/3/14, 8:26 AM, Andres Freund wrote: > >#define NUM_XLOGINSERT_LOCKS 1 > >tps = 52.711939 (including connections establishing) > >#define NUM_XLOGINSERT_LOCKS 8 > >tps = 286.496054 (including connections establishing) > >#define NUM_XLOGIN

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps = 286.496054 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 16 tps = 346.113313 (including connections establishin

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-02 20:08:33 -0400, Greg Smith wrote: > I did a fair dive into double-checking the decision to just leave > xloginsert_locks fixed at 8 for 9.4. My conclusion: good call, move along. > Further improvements beyond what the 8-way split gives sure are possible. > But my guess from chasing

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 5:08 PM, Greg Smith wrote: > When 9.4 is already giving a more than 100% gain on this targeted test case, > I can't see that chasing after maybe an extra 10% is worth having yet > another GUC around. Especially when it will probably take multiple tuning > steps before you'r

[HACKERS] Fixed xloginsert_locks for 9.4

2014-10-02 Thread Greg Smith
I did a fair dive into double-checking the decision to just leave xloginsert_locks fixed at 8 for 9.4. My conclusion: good call, move along. Further improvements beyond what the 8-way split gives sure are possible. But my guess from chasing them a little is that additional places will pop u