Re: [HACKERS] WITHIN GROUP patch

2013-10-10 Thread Pavel Stehule
2013/10/11 Andrew Gierth > > "Pavel" == Pavel Stehule writes: > > >> I found so following error message is not too friendly (mainly because > >> this functionality will be new) > >> > >> postgres=# select dense_rank(3,3,2) within group (order by num desc, > odd) > >> from test4; > >> E

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
Robert, >> The counter-proposal to "auto-tuning" is just to raise the default for >> work_mem to 4MB or 8MB. Given that Bruce's current formula sets it at >> 6MB for a server with 8GB RAM, I don't really see the benefit of going >> to a whole lot of code and formulas in order to end up at a figur

[HACKERS] Re: space reserved for WAL record does not match what was written: panic on windows

2013-10-10 Thread Noah Misch
On Thu, Oct 10, 2013 at 03:23:30PM +0200, Andres Freund wrote: > On 2013-10-10 08:59:47 -0400, Robert Haas wrote: > > On Tue, Oct 8, 2013 at 6:24 PM, Andres Freund > > wrote: > > > Do you have a better alternative? Making the computation unconditionally > > > 64bit will have a runtime overhead an

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Mark Kirkwood
On 11/10/13 17:33, Jaime Casanova wrote: On Thu, Oct 10, 2013 at 5:32 PM, Mark Kirkwood wrote: Quietly replying to myself - looking at the code the sampler does 3000 random page reads... FWIW, something that bothers me is that there is 3000 random page reads... i mean, why 3000? how do you get

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Mark Kirkwood
On 11/10/13 17:08, Satoshi Nagayasu wrote: > (2013/10/11 7:32), Mark Kirkwood wrote: >> On 11/10/13 11:09, Mark Kirkwood wrote: >>> On 16/09/13 16:20, Satoshi Nagayasu wrote: (2013/09/15 11:07), Peter Eisentraut wrote: > On Sat, 2013-09-14 at 16:18 +0900, Satoshi Nagayasu wrote: >> I'm

Re: [HACKERS] Patch: FORCE_NULL option for copy COPY in CSV mode

2013-10-10 Thread Amit Kapila
On Thu, Oct 10, 2013 at 6:45 PM, Andrew Dunstan wrote: > > On 10/09/2013 11:47 PM, Amit Kapila wrote: >> >> >> One of the advantage, I could see using "NULL For .." syntax is >> that already we have one syntax with which user can specify what >> strings can be replaced with NULL, now just to h

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Jaime Casanova
On Thu, Oct 10, 2013 at 5:32 PM, Mark Kirkwood wrote: > > Quietly replying to myself - looking at the code the sampler does 3000 > random page reads... FWIW, something that bothers me is that there is 3000 random page reads... i mean, why 3000? how do you get that number as absolute for good accu

Re: [HACKERS] Patch for reserved connections for replication users

2013-10-10 Thread Amit Kapila
On Thu, Oct 10, 2013 at 10:06 PM, Gibheer wrote: > On Thu, 10 Oct 2013 09:55:24 +0530 > Amit Kapila wrote: > >> On Thu, Oct 10, 2013 at 3:17 AM, Gibheer >> wrote: >> > On Mon, 7 Oct 2013 11:39:55 +0530 >> > Amit Kapila wrote: >> >> Robert Haas wrote: >> >> On Mon, Aug 5, 2013 at 2:04 AM, Andres

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Satoshi Nagayasu
(2013/10/11 7:32), Mark Kirkwood wrote: > On 11/10/13 11:09, Mark Kirkwood wrote: >> On 16/09/13 16:20, Satoshi Nagayasu wrote: >>> (2013/09/15 11:07), Peter Eisentraut wrote: On Sat, 2013-09-14 at 16:18 +0900, Satoshi Nagayasu wrote: > I'm looking forward to seeing more feedback on this a

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Amit Kapila
On Fri, Oct 11, 2013 at 5:05 AM, Andres Freund wrote: > Hi, > On 2013-10-11 03:44:01 +0900, Fujii Masao wrote: >> I'm afraid that the patch has only limited effects in WAL reduction and >> performance improvement unless the database contains highly-compressible >> data like large blank characters

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Fujii Masao
On Fri, Oct 11, 2013 at 8:35 AM, Andres Freund wrote: > Hi, > On 2013-10-11 03:44:01 +0900, Fujii Masao wrote: >> I'm afraid that the patch has only limited effects in WAL reduction and >> performance improvement unless the database contains highly-compressible >> data like large blank characters

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Fujii Masao
On Fri, Oct 11, 2013 at 3:44 AM, Fujii Masao wrote: > On Fri, Oct 11, 2013 at 1:20 AM, Dimitri Fontaine > wrote: >> Hi, >> >> I did a partial review of this patch, wherein I focused on the patch and >> the code itself, as I saw other contributors already did some testing on >> it, so that we know

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 09:35 PM, Peter Eisentraut wrote: On Tue, 2013-10-08 at 10:04 -0400, Andrew Dunstan wrote: On 10/07/2013 08:47 PM, Peter Eisentraut wrote: I suspect this line submake-libpq: $(libdir)/libpq.so ; will cause problems on platforms with a different extension (e.g. OS X). suggeste

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 09:37 PM, Peter Eisentraut wrote: On Mon, 2013-10-07 at 22:00 -0400, Andrew Dunstan wrote: The code has been sitting in HEAD for several months, and I committed on the back branches because it was wanted. New features normally go through a full development cycle including extensi

Re: [HACKERS] WITHIN GROUP patch

2013-10-10 Thread Andrew Gierth
> "Pavel" == Pavel Stehule writes: >> I found so following error message is not too friendly (mainly because >> this functionality will be new) >> >> postgres=# select dense_rank(3,3,2) within group (order by num desc, odd) >> from test4; >> ERROR: Incorrect number of arguments for hy

Re: [HACKERS] [COMMITTERS] pgsql: Replace duplicate_oids with Perl implementation

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 8:23 PM, Peter Eisentraut wrote: > Replace duplicate_oids with Perl implementation > > It is more portable, more robust, and more readable. smilodon seems unhappy with this: cd ../../../src/include/catalog && ./duplicate_oids sh: ./duplicate_oids: not found -- Robert Ha

Re: [HACKERS] ENABLE/DISABLE CONSTRAINT NAME

2013-10-10 Thread wangshuo
On 2013-10-10 02:10, Robert Haas wrote: I agree with these concerns, as well as those raised by Tom Lane and Fabien COELHO, and I think they indicate that we shouldn't accept this patch. So I'm marking this as Rejected. On 2013-10-11 06:48, Jim Nasby wrote: I see a use case for disabling

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-10 Thread Peter Eisentraut
On Mon, 2013-10-07 at 22:00 -0400, Andrew Dunstan wrote: > The code has been sitting in HEAD for several months, and I > committed on the back branches because it was wanted. New features normally go through a full development cycle including extensive beta testing, especially when they contain

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-10 Thread Peter Eisentraut
On Tue, 2013-10-08 at 10:04 -0400, Andrew Dunstan wrote: > On 10/07/2013 08:47 PM, Peter Eisentraut wrote: > > > > I suspect this line > > > > submake-libpq: $(libdir)/libpq.so ; > > > > will cause problems on platforms with a different extension (e.g. OS X). > > > suggested fix is below. Hmm, t

Re: [HACKERS] GIN improvements part 1: additional information

2013-10-10 Thread Peter Eisentraut
You linked to this email from the commitfest entry, but there is no patch here. You probably meant a different email. Check please. On Tue, 2013-10-08 at 21:48 +0300, Heikki Linnakangas wrote: > On 04.10.2013 14:13, Alexander Korotkov wrote: > > On Fri, Oct 4, 2013 at 12:28 PM, Heikki Linnakang

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 6:27 PM, Josh Berkus wrote: >> More generally, Josh has made repeated comments that various proposed >> value/formulas for work_mem are too low, but obviously the people who >> suggested them didn't think so. So I'm a bit concerned that we don't >> all agree on what the en

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 6:36 PM, Bruce Momjian wrote: > Patch attached. ISTM that we have broad consensus that doing this at initdb time is more desirable than doing it in the server on the fly. Not everyone agrees with that (you don't, for instance) but there were many, many votes in favor of t

Re: [HACKERS] CommitFest progress

2013-10-10 Thread Robert Haas
On Wed, Oct 9, 2013 at 2:03 PM, Robert Haas wrote: > I therefore propose that we start by marking all of the patches that > are currently Waiting on Author as Returned with Feedback. Most of > them have been that way for a long time. Hearing no objections, I went through and did this, but skippe

Re: [HACKERS] [COMMITTERS] pgsql: Revive line type

2013-10-10 Thread Peter Eisentraut
On Thu, 2013-10-10 at 10:51 -0400, Robert Haas wrote: > On Wed, Oct 9, 2013 at 10:43 PM, Peter Eisentraut wrote: > > Revive line type > > Kevin just pointed out to me that there are a bunch of buildfarm > failures. I'm looking at the ones that are attributable to shared > memory, but there seem

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 7:59 PM, Josh Berkus wrote: >>> (5) Default to POSIX, and allow for SysV as a compile-time option for >>> platforms with poor POSIX memory support. >> >> OK, I did #5. Let's see how that works. > > Andrew pointed out upthread that, since platforms are unlikely to change >

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Josh Berkus
>> (5) Default to POSIX, and allow for SysV as a compile-time option for >> platforms with poor POSIX memory support. > > OK, I did #5. Let's see how that works. Andrew pointed out upthread that, since platforms are unlikely to change what they support dynamically, we could do this at initdb ti

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 5:14 PM, Josh Berkus wrote: >>> Doesn't #2 negate all advantages of this effort? Bringing sysv >>> management back on the table seems like a giant step backwards -- or >>> am I missing something? >> >> Not unless there's no difference between "the default" and "the only op

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Andres Freund
Hi, On 2013-10-11 03:44:01 +0900, Fujii Masao wrote: > I'm afraid that the patch has only limited effects in WAL reduction and > performance improvement unless the database contains highly-compressible > data like large blank characters column. It really depends on the contents > of the database. S

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Jim Nasby
On 10/10/13 9:44 AM, MauMau wrote: From: "Robert Haas" On Thu, Oct 10, 2013 at 1:23 AM, Magnus Hagander wrote: I think it would be even simpler, and more reliable, to start with the parameter to initdb - I like that. But instead of having it set a new variable based on that and then autotune

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Andres Freund
On 2013-10-10 12:13:20 -0400, Robert Haas wrote: > and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. It would be nice to get confirmed what the reason fo

Re: [HACKERS] logical changeset generation v6.2

2013-10-10 Thread Andres Freund
Hi Robert, On 2013-10-09 14:49:46 -0400, Robert Haas wrote: > I spent some time looking at the sample plugin (patch 9/12). Here are > some review comments: > > - I think that the decoding plugin interface should work more like the > foreign data wrapper interface. Instead of using pg_dlsym to l

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 03:27:17PM -0700, Josh Berkus wrote: > > > More generally, Josh has made repeated comments that various proposed > > value/formulas for work_mem are too low, but obviously the people who > > suggested them didn't think so. So I'm a bit concerned that we don't > > all agree

Re: [HACKERS] ENABLE/DISABLE CONSTRAINT NAME

2013-10-10 Thread Jim Nasby
On 10/9/13 1:10 PM, Robert Haas wrote: On Tue, Sep 24, 2013 at 10:40 PM, Peter Eisentraut wrote: On Tue, 2013-09-24 at 11:58 +0200, Bernd Helmle wrote: Hmm not sure i understand this argument either: this patch doesn't allow disabling a primary key. It only supports FKs and CHECK constraints e

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 03:40:17PM -0700, Josh Berkus wrote: > > >> I don't follow that. Why would using a connection pooler change the > >> multiples > >> of work_mem that a connection would use? > > > > I assume that a connection pooler would keep processes running longer, > > so even if they

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 02:44:12PM -0400, Peter Eisentraut wrote: > On 10/10/13 11:31 AM, Bruce Momjian wrote: > > Let me walk through the idea of adding an available_mem setting, that > > Josh suggested, and which I think addresses Robert's concern about > > larger shared_buffers and Windows serve

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
>> I don't follow that. Why would using a connection pooler change the >> multiples >> of work_mem that a connection would use? > > I assume that a connection pooler would keep processes running longer, > so even if they were not all using work_mem, they would have that memory > mapped into the

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 11:18:28AM -0700, Josh Berkus wrote: > Bruce, > > >> That's way low, and frankly it's not worth bothering with this if all > >> we're going to get is an incremental increase. In that case, let's just > >> set the default to 4MB like Robert suggested. > > > > Uh, well, 100

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Mark Kirkwood
On 11/10/13 11:09, Mark Kirkwood wrote: > On 16/09/13 16:20, Satoshi Nagayasu wrote: >> (2013/09/15 11:07), Peter Eisentraut wrote: >>> On Sat, 2013-09-14 at 16:18 +0900, Satoshi Nagayasu wrote: I'm looking forward to seeing more feedback on this approach, in terms of design and performan

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 11:14:27AM -0700, Jeff Janes wrote: > The assumption that each connection won't use lots of work_mem is also > false, I think, especially in these days of connection poolers. > > > I don't follow that. Why would using a connection pooler change the multiples > of

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
> More generally, Josh has made repeated comments that various proposed > value/formulas for work_mem are too low, but obviously the people who > suggested them didn't think so. So I'm a bit concerned that we don't > all agree on what the end goal of this activity looks like. The counter-proposa

Re: [HACKERS] ECPG FETCH readahead

2013-10-10 Thread Alvaro Herrera
Boszormenyi Zoltan escribió: > 2013-09-10 03:04 keltezéssel, Peter Eisentraut írta: > >You need to update the dblink regression tests. > > Done. Dude, this is an humongous patch. I was shocked by it initially, but on further reading, I observed that it's only a huge patch which also does some me

Re: [HACKERS] [PoC] pgstattuple2: block sampling to reduce physical read

2013-10-10 Thread Mark Kirkwood
On 16/09/13 16:20, Satoshi Nagayasu wrote: > (2013/09/15 11:07), Peter Eisentraut wrote: >> On Sat, 2013-09-14 at 16:18 +0900, Satoshi Nagayasu wrote: >>> I'm looking forward to seeing more feedback on this approach, >>> in terms of design and performance improvement. >>> So, I have submitted this

Re: [HACKERS] Re: Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2013-10-10 Thread Alvaro Herrera
We have this block: + { + /* +* This is the window we want - but we have to tweak the +* definition slightly (e.g. to support the IGNORE NULLS frame +* option) as we're not using the default (i.e. parent) frame +* options. +

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Josh Berkus
Robert, >> Doesn't #2 negate all advantages of this effort? Bringing sysv >> management back on the table seems like a giant step backwards -- or >> am I missing something? > > Not unless there's no difference between "the default" and "the only option". Well, per our earlier discussion about "

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 1:30 PM, Alvaro Herrera wrote: >> > Just noticed that you changed the timer to struct Instrumentation. Not >> > really sure about that change. Since you seem to be using only the >> > start time and counter, wouldn't it be better to store only those? >> > Particularly uns

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 4:00 PM, Merlin Moncure wrote: >> (2) Default to using System V shared memory. If people want POSIX >> shared memory, let them change the default. > > Doesn't #2 negate all advantages of this effort? Bringing sysv > management back on the table seems like a giant step bac

[HACKERS] Pattern matching operators a index

2013-10-10 Thread soroosh sardari
Hi I'm developing a new type for character string, like varchar. I wrote operators for btree and so forth. I wonder how pattern matching operators using btree index, because btree operator class ony knows about >, >=, <=, and = operators, but operators for pattern matching, such as LIKE, are not k

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread pilum . 70
The V7-Patch applied cleanly and I got no issues in my first tests. The change from column session_start to a function seems very reasonable for me. Concernig the usability, I would like to suggest a minor change, that massively increases the usefulness of the patch for beginners, who often us

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread David Fetter
On Thu, Oct 10, 2013 at 12:13:20PM -0400, Robert Haas wrote: > Since, as has been previously discussed in this forum on multiple > occasions [citation needed], the default System V shared memory limits > are absurdly low on many systems, the dynamic shared memory patch > defaults to POSIX shared me

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 02:45 PM, Robert Haas wrote: On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan wrote: On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas wrote: (2) Default to using System V shared memory. If people want POSIX shared memory, let them change the default. After some consideration, I th

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Alvaro Herrera
Daniel Farina escribió: > Given that, perhaps a way to fix this is something like this patch-fragment: > > """ > { > PGSS_TUP_V1_0 = 1, > PGSS_TUP_V1_1, > - PGSS_TUP_LATEST > + PGSS_TUP_V1_2 > } pgssTupVersion; > > +#define PGSS_TUP_LATEST PGSS_TUP_V1_2 > """ This sounds good. I have see

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 10:49 AM, Alvaro Herrera wrote: > Daniel Farina escribió: >> On Thu, Oct 10, 2013 at 7:40 AM, Fujii Masao wrote: > >> > In my test, I found that pg_stat_statements.total_time always indicates a >> > zero. >> > I guess that the patch might handle pg_stat_statements.total_t

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Merlin Moncure
On Thu, Oct 10, 2013 at 11:13 AM, Robert Haas wrote: > Since, as has been previously discussed in this forum on multiple > occasions [citation needed], the default System V shared memory limits > are absurdly low on many systems, the dynamic shared memory patch > defaults to POSIX shared memory, w

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 3:41 PM, Christopher Browne wrote: > On Thu, Oct 10, 2013 at 12:28 PM, Bruce Momjian wrote: >> How do we handle the Python dependency, or is this all to be done in >> some other language? I certainly am not ready to take on that job. > > I should think it possible to reim

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 10:12 AM, Fujii Masao wrote: > On Fri, Oct 11, 2013 at 12:19 AM, Daniel Farina wrote: >> Probably. >> >> The idea is that without those fields it's, to wit, impossible to >> explain non-monotonic movement in metrics of those queries for precise >> use in tools that insist

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Christopher Browne
On Thu, Oct 10, 2013 at 12:28 PM, Bruce Momjian wrote: > How do we handle the Python dependency, or is this all to be done in > some other language? I certainly am not ready to take on that job. I should think it possible to reimplement it in C. It was considerably useful to start by implementi

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan wrote: > > On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas wrote: > >> (2) Default to using System V shared memory. If people want POSIX > >> shared memory, let them change the default. > > > >> After so

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 02:35 PM, Robert Haas wrote: On Thu, Oct 10, 2013 at 2:21 PM, Andrew Dunstan wrote: Other votes? Other ideas? 5) test and set it in initdb. Are you advocating for that option, or just calling out that it's possible? I'd say that's closely related to option #3, except at initd

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Magnus Hagander
On Thu, Oct 10, 2013 at 8:46 PM, Robert Haas wrote: > On Thu, Oct 10, 2013 at 2:45 PM, Josh Berkus wrote: >> On 10/10/2013 11:41 AM, Robert Haas wrote: >>> tunedb --available-memory=32GB >>> >>> ...and it will print out a set of proposed configuration settings. If >>> we want a mode that rewrite

Re: [HACKERS] [v9.4] row level security

2013-10-10 Thread Marc Munro
On Wed, 2013-09-04 at 14:35 +, Robert Haas wrote: > > On Fri, Aug 30, 2013 at 3:43 PM, Tom Lane wrote: > > I think it's entirely sensible to question whether we should reject > (not > > "hold up") RLS if it has major covert-channel problems. > > We've already had this argument before, about

Re: [HACKERS] strange behavior of pg_trgm's similarity function

2013-10-10 Thread Fujii Masao
On Thu, Oct 10, 2013 at 11:00 PM, Merlin Moncure wrote: > On Thu, Oct 10, 2013 at 7:12 AM, Heikki Linnakangas > wrote: >> On 10.10.2013 15:03, Fujii Masao wrote: >>> >>> Hi, >>> >>> The behavior of pg_trgm's similarity function seems strange. Is this >>> intentional? >>> >>> I was thinking that t

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Peter Eisentraut
On 10/10/13 11:45 AM, Bruce Momjian wrote: > I think the big win for a tool would be to query the user about how they > are going to be using Postgres, and that can then spit out values the > user can add to postgresql.conf, or to a config file that is included at > the end of postgresql.conf. I t

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Peter Geoghegan
On Wed, Oct 9, 2013 at 10:21 PM, Magnus Hagander wrote: >> Well, the Postgres defaults won't really change, because the default >> vacuum_work_mem will be -1, which will have vacuum defer to >> maintenance_work_mem. Under this scheme, vacuum only *prefers* to get >> bound working memory size from

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan wrote: > On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas wrote: >> (2) Default to using System V shared memory. If people want POSIX >> shared memory, let them change the default. > >> After some consideration, I think my vote is for option #2. > > W

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
On 10/10/2013 11:41 AM, Robert Haas wrote: > tunedb --available-memory=32GB > > ...and it will print out a set of proposed configuration settings. If > we want a mode that rewrites the configuration file, we could have: > > tunedb --available-memory=32GB --rewrite-config-file=$PATH > > ...but t

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Magnus Hagander
On Thu, Oct 10, 2013 at 8:41 PM, Robert Haas wrote: > On Thu, Oct 10, 2013 at 12:28 PM, Bruce Momjian wrote: >> On Thu, Oct 10, 2013 at 12:00:54PM -0400, Stephen Frost wrote: >>> * Bruce Momjian (br...@momjian.us) wrote: >>> > Well, I like the idea of initdb calling the tool, though the tool then

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Fujii Masao
On Fri, Oct 11, 2013 at 2:49 AM, Alvaro Herrera wrote: > Daniel Farina escribió: >> On Thu, Oct 10, 2013 at 7:40 AM, Fujii Masao wrote: > >> > In my test, I found that pg_stat_statements.total_time always indicates a >> > zero. >> > I guess that the patch might handle pg_stat_statements.total_ti

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 2:45 PM, Josh Berkus wrote: > On 10/10/2013 11:41 AM, Robert Haas wrote: >> tunedb --available-memory=32GB >> >> ...and it will print out a set of proposed configuration settings. If >> we want a mode that rewrites the configuration file, we could have: >> >> tunedb --avai

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Peter Eisentraut
On 10/10/13 11:31 AM, Bruce Momjian wrote: > Let me walk through the idea of adding an available_mem setting, that > Josh suggested, and which I think addresses Robert's concern about > larger shared_buffers and Windows servers. I think this is a promising idea. available_mem could even be set au

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Peter Geoghegan
On Thu, Oct 10, 2013 at 11:43 AM, Robert Haas wrote: > On Thu, Oct 10, 2013 at 1:37 PM, Josh Berkus wrote: >> So, the question is: can we reasonably determine, at initdb time, how >> much RAM the system has? > > As long as you are willing to write platform-dependent code, yes. That's why trying

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Peter Geoghegan
On Thu, Oct 10, 2013 at 11:41 AM, Robert Haas wrote: > I don't see why it can't be done in C. The server is written in C, > and so is initdb. So no matter where we do this, it's gonna be in C. > Where does Python enter into it? I mentioned that pgtune was written in Python, but as you say that'

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Fujii Masao
On Fri, Oct 11, 2013 at 1:20 AM, Dimitri Fontaine wrote: > Hi, > > I did a partial review of this patch, wherein I focused on the patch and > the code itself, as I saw other contributors already did some testing on > it, so that we know it applies cleanly and work to some good extend. Thanks a lo

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 1:37 PM, Josh Berkus wrote: > So, the question is: can we reasonably determine, at initdb time, how > much RAM the system has? As long as you are willing to write platform-dependent code, yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgr

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 12:28 PM, Bruce Momjian wrote: > On Thu, Oct 10, 2013 at 12:00:54PM -0400, Stephen Frost wrote: >> * Bruce Momjian (br...@momjian.us) wrote: >> > Well, I like the idea of initdb calling the tool, though the tool then >> > would need to be in C probably as we can't require p

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Peter Geoghegan
On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas wrote: > (2) Default to using System V shared memory. If people want POSIX > shared memory, let them change the default. > After some consideration, I think my vote is for option #2. Wouldn't that become the call of packagers? Wasn't there already so

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Robert Haas
On Thu, Oct 10, 2013 at 2:21 PM, Andrew Dunstan wrote: >> Other votes? Other ideas? > > 5) test and set it in initdb. Are you advocating for that option, or just calling out that it's possible? I'd say that's closely related to option #3, except at initdb time rather than run-time - and it migh

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
> It also doesn't address my point that, if we are worst-case-scenario > default-setting, we're going to end up with defaults which aren't > materially different from the current defaults. In which case, why even > bother with this whole exercise? Oh, and let me reiterate: the way to optimize wo

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 12:13 PM, Robert Haas wrote: Since, as has been previously discussed in this forum on multiple occasions [citation needed], the default System V shared memory limits are absurdly low on many systems, the dynamic shared memory patch defaults to POSIX shared memory, which has often b

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
Bruce, >> That's way low, and frankly it's not worth bothering with this if all >> we're going to get is an incremental increase. In that case, let's just >> set the default to 4MB like Robert suggested. > > Uh, well, 100 backends at 6MB gives us 600MB, and if each backend uses > 3x work_mem, th

Re: [HACKERS] dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread Claudio Freire
On Thu, Oct 10, 2013 at 1:13 PM, Robert Haas wrote: > (1) Define the issue as "not our problem". IOW, as of now, if you > want to use PostgreSQL, you've got to either make POSIX shared memory > work on your machine, or change the GUC that selects the type of > dynamic shared memory used. > > (2)

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Jeff Janes
On Wed, Oct 9, 2013 at 8:06 AM, Andrew Dunstan wrote: > > On 10/09/2013 10:45 AM, Bruce Momjian wrote: > >> On Wed, Oct 9, 2013 at 04:40:38PM +0200, Pavel Stehule wrote: >> >>> Effectively, if every session uses one full work_mem, you end up >>> with >>> total work_mem usage equal to s

[HACKERS] Re: dynamic shared memory: wherein I am punished for good intentions

2013-10-10 Thread David Johnston
Robert Haas wrote > Unfortunately, the buildfarm > isn't entirely happy with this decision. On buildfarm member anole > (HP-UX B.11.31), allocation of dynamic shared memory fails with a > "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", whi

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Oct 10, 2013 at 12:00:54PM -0400, Stephen Frost wrote: > > I'm really not impressed with this argument. Either the user is going > > to go and modify the config file, in which case I would hope that they'd > > at least glance around at wh

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Alvaro Herrera
Daniel Farina escribió: > On Thu, Oct 10, 2013 at 7:40 AM, Fujii Masao wrote: > > In my test, I found that pg_stat_statements.total_time always indicates a > > zero. > > I guess that the patch might handle pg_stat_statements.total_time wrongly. > > > > +values[i++] = DatumGetTimestamp( >

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
All, We can't reasonably require user input at initdb time, because most users don't run initdb by hand -- their installer does it for them. So any "tuning" which initdb does needs to be fully automated. So, the question is: can we reasonably determine, at initdb time, how much RAM the system ha

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Fujii Masao
On Wed, Oct 9, 2013 at 1:35 PM, Haribabu kommi wrote: > On 08 October 2013 18:42 KONDO Mitsumasa wrote: >>(2013/10/08 20:13), Haribabu kommi wrote: >>> I will test with sync_commit=on mode and provide the test results. >>OK. Thanks! > > Pgbench test results with synchronous_commit mode as on. Tha

Re: [HACKERS] Compression of full-page-writes

2013-10-10 Thread Fujii Masao
On Tue, Oct 8, 2013 at 10:07 PM, KONDO Mitsumasa wrote: > Hi, > > I tested dbt-2 benchmark in single instance and synchronous replication. Thanks! > Unfortunately, my benchmark results were not seen many differences... > > > * Test server >Server: HP Proliant DL360 G7 >CPU:Xeon E5640

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 10:20:02AM -0700, Josh Berkus wrote: > On 10/09/2013 02:15 PM, Bruce Momjian wrote: > > and for shared_buffers of 2GB: > > > > test=> show shared_buffers; > > shared_buffers > > > > 2GB > > (1 row) > > > > test=> SHOW work_mem

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
On 10/09/2013 02:15 PM, Bruce Momjian wrote: > and for shared_buffers of 2GB: > > test=> show shared_buffers; >shared_buffers > >2GB > (1 row) > > test=> SHOW work_mem; >work_mem > -- >6010kB > (1 r

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Josh Berkus
> Because 'maintenance' operations were rarer, so we figured we could use > more memory in those cases. Once we brought Autovacuum into core, though, we should have changed that. However, I agree with Magnus that the simple course is to have an autovacuum_worker_memory setting which overrides ma

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 12:59:39PM -0400, Andrew Dunstan wrote: > > On 10/10/2013 12:45 PM, Bruce Momjian wrote: > >On Thu, Oct 10, 2013 at 12:39:04PM -0400, Andrew Dunstan wrote: > >>On 10/10/2013 12:28 PM, Bruce Momjian wrote: > >>>How do we handle the Python dependency, or is this all to be don

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Fujii Masao
On Fri, Oct 11, 2013 at 12:19 AM, Daniel Farina wrote: > Probably. > > The idea is that without those fields it's, to wit, impossible to > explain non-monotonic movement in metrics of those queries for precise > use in tools that insist on monotonicity of the fields, which are all > cumulative to

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 12:45 PM, Bruce Momjian wrote: On Thu, Oct 10, 2013 at 12:39:04PM -0400, Andrew Dunstan wrote: On 10/10/2013 12:28 PM, Bruce Momjian wrote: How do we handle the Python dependency, or is this all to be done in some other language? I certainly am not ready to take on that job. W

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-10-10 Thread Andrew Dunstan
On 09/19/2013 06:12 PM, Pavel Stehule wrote: 2013/9/16 Satoshi Nagayasu mailto:sn...@uptime.jp>> I'm looking at this patch, and I have a question here. Should "DROP TRIGGER IF EXISTS" ignore error for non-existing trigger and non-existing table? Or just only for non-existin

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 12:39:04PM -0400, Andrew Dunstan wrote: > > On 10/10/2013 12:28 PM, Bruce Momjian wrote: > > > >How do we handle the Python dependency, or is this all to be done in > >some other language? I certainly am not ready to take on that job. > > > Without considering any wider

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Andrew Dunstan
On 10/10/2013 12:28 PM, Bruce Momjian wrote: How do we handle the Python dependency, or is this all to be done in some other language? I certainly am not ready to take on that job. Without considering any wider question here, let me just note this: Anything that can be done in this area in

Re: [HACKERS] [COMMITTERS] pgsql: Add record_image_ops opclass for matview concurrent refresh.

2013-10-10 Thread Kevin Grittner
Kevin Grittner wrote: > Add record_image_ops opclass for matview concurrent refresh. The buildfarm pointed out that I had not handled pass-by-value data types correctly.  Fixed based on advice from Robert.  We'll see whether that clears up the part of the buildfarm breakage attributed to this pa

Re: [HACKERS] Patch for reserved connections for replication users

2013-10-10 Thread Gibheer
On Thu, 10 Oct 2013 09:55:24 +0530 Amit Kapila wrote: > On Thu, Oct 10, 2013 at 3:17 AM, Gibheer > wrote: > > On Mon, 7 Oct 2013 11:39:55 +0530 > > Amit Kapila wrote: > >> Robert Haas wrote: > >> On Mon, Aug 5, 2013 at 2:04 AM, Andres Freund > >> wrote: > >> >>> Hmm. It seems like this match

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

2013-10-10 Thread Bruce Momjian
On Thu, Oct 10, 2013 at 12:00:54PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Well, I like the idea of initdb calling the tool, though the tool then > > would need to be in C probably as we can't require python for initdb. > > The tool would not address Robert's is

Re: [HACKERS] Add json_typeof() and json_is_*() functions.

2013-10-10 Thread Andrew Dunstan
On 08/06/2013 08:42 AM, Robert Haas wrote: On Fri, Aug 2, 2013 at 8:22 AM, Andrew Tipton wrote: But without json_is_scalar(), the choice is one of these two forms: json_typeof() NOT IN ('object', 'array') json_typeof() IN ('string', 'number', 'boolean', 'null') The first of those is wha

  1   2   >