Re: [HACKERS] Create collation incorrect error code

2012-06-01 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, May 31, 2012 at 8:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: What platform? Ubuntu 12.04. FWIW, I get the same on Fedora 16. It works if I say locale='nb_NO.utf8' to prevent case-folding, so apparently case of the locale name has something

[HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 08:55, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.06.2012 10:28, Simon Riggs wrote: Checkpointer starts before bgwriter to avoid missing fsync requests. Noted while testing Hot Standby startup. The processes are just forked and it will take some time

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-06-01 Thread Sergey Koposov
On Fri, 1 Jun 2012, Simon Riggs wrote: Why do you have 10,000 tables and why is it important to drop them so quickly? 1 tables are there, because that's the number of partitions. And I'm dropping them at the moment, because I'm doing testing. So it won't be really crucial for

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Sergey Koposov
On Thu, 31 May 2012, Jeff Janes wrote: No, idt_match is getting filled by multi-threaded copy() and then joined with 4 other big tables like idt_phot. The result is then split into partitions. That does make things more complicated. But you could you partition it at that level and then do

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-06-01 Thread Simon Riggs
On 1 June 2012 12:34, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Fri, 1 Jun 2012, Simon Riggs wrote: Why do you have 10,000 tables and why is it important to drop them so quickly? 1 tables are there, because that's the number of partitions. And I'm dropping them at the moment,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On May31, 2012, at 20:50 , Robert Haas wrote: Suppose we introduce two new buffer flags, BUF_NAILED and BUF_NAIL_REMOVAL. When we detect excessive contention on the buffer header spinlock, we set BUF_NAILED. Once we do that, the buffer can't be evicted until that flag is removed, and

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 1 June 2012 08:55, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.06.2012 10:28, Simon Riggs wrote: Checkpointer starts before bgwriter to avoid missing fsync requests. Noted while testing Hot Standby startup. The processes

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 7:47 AM, Florian Pflug f...@phlo.org wrote: On May31, 2012, at 20:50 , Robert Haas wrote: Suppose we introduce two new buffer flags, BUF_NAILED and BUF_NAIL_REMOVAL.  When we detect excessive contention on the buffer header spinlock, we set BUF_NAILED.  Once we do that,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: A potential issue with this line of thinking is that your pin delay queue could get highly pressured by outer portions of the query (as in the OP's case) that will get little or no benefit from the delayed pin. But choosing a sufficiently sized drain

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 14:29, Tom Lane t...@sss.pgh.pa.us wrote: Surely that commit is useless.  Fsync requests go into a queue in shared memory, which had better have been set up by the postmaster.  There is no requirement that the receiving process exist before somebody can put a request into the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 1 June 2012 14:29, Tom Lane t...@sss.pgh.pa.us wrote: Surely that commit is useless.  Fsync requests go into a queue in shared memory, which had better have been set up by the postmaster.  There is no requirement that the receiving process exist

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 14:59, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 1 June 2012 14:29, Tom Lane t...@sss.pgh.pa.us wrote: Surely that commit is useless.  Fsync requests go into a queue in shared memory, which had better have been set up by the postmaster.  

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 1 June 2012 14:59, Tom Lane t...@sss.pgh.pa.us wrote: Ah.  Well, as long as the overflowed fsyncs do get handled on the requesting side, I see no bug here.  No objection to changing the order in which we launch the processes, but as Heikki says,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 15:45 , Tom Lane wrote: Merlin Moncure mmonc...@gmail.com writes: A potential issue with this line of thinking is that your pin delay queue could get highly pressured by outer portions of the query (as in the OP's case) that will get little or no benefit from the delayed

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 8:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: A potential issue with this line of thinking is that your pin delay queue could get highly pressured by outer portions of the query (as in the OP's case)  that will get little or no

Re: [HACKERS] Create collation incorrect error code

2012-06-01 Thread Tom Lane
I wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 31, 2012 at 8:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: What platform? Ubuntu 12.04. FWIW, I get the same on Fedora 16. This seems to be a glibc bug: it won't set errno correctly if you've already asked about the same wrong

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
On Fri, Jun 1, 2012 at 5:57 PM, Florian Pflug f...@phlo.org wrote: My proposed algorithm could be made to use exactly that criterion by tracking a little bit more state. We'd have to tag queue entries with a flag indicating whether they are  Unpinned (COLD)  Pinned, and unpinning should be

Re: [HACKERS] Schema version management

2012-06-01 Thread Kevin Grittner
Joel Jacobson wrote: hm, is it even possible to have function names with slashes? test=# create function is/it\even possible?() returns void language plpgsql as $$begin end;$$; CREATE FUNCTION test=# select is/it\even possible?(); is/it\even+ possible? -- (1 row) -Kevin

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs by queing UnpinBuffer() requests for a while before actually updating shared state. So, what happens when somebody wants a cleanup lock on the buffer you've

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-06-01 Thread Robert Haas
On Thu, May 31, 2012 at 3:52 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It may be an option to separate the case into two; a situation to execute the given query immediately just after optimization and never reused, and others. Yes. Simon suggested exactly this a while back, and I agree with

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Jeff Janes
On Fri, Jun 1, 2012 at 10:51 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: We'd drain the unpin queue whenever we don't expect a PinBuffer() request to happen for a while. Returning to the main loop is an obvious such place,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 19:51 , Robert Haas wrote: On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs by queing UnpinBuffer() requests for a while before actually updating shared state. So, what happens when

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 2:06 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Fri, Jun 1, 2012 at 10:51 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: We'd drain the unpin queue whenever we don't expect a PinBuffer() request to

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 2:54 PM, Florian Pflug f...@phlo.org wrote: On Jun1, 2012, at 19:51 , Robert Haas wrote: On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs by queing UnpinBuffer() requests for a while

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Another thought is that if the problem is limited to the root index block, then we could consider less general solutions, like allowing backends to cache the root index block and sending some kind of invalidation when it gets split. Possibly worth

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-01 Thread Peter Geoghegan
This is a benchmark I performed on the same hardware, for tpc-b.sql, with a commit_delay of 0 and 4000 for the patch: http://tpcbdelay.staticloud.com/ There is a rather large improvement in throughput here. Robert previously complained that our group commit implementation didn't do very well on

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 21:07 , Robert Haas wrote: On Fri, Jun 1, 2012 at 2:54 PM, Florian Pflug f...@phlo.org wrote: On Jun1, 2012, at 19:51 , Robert Haas wrote: On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug f...@phlo.org wrote: We'd drain the unpin queue whenever we don't expect a PinBuffer()

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 3:16 PM, Florian Pflug f...@phlo.org wrote: Ok, now you've lost me. If the read() blocks, how on earth can a few additional pins/unpins ever account for any meaningful execution time? It seems to me that once read() blocks we're talking about a delay in the order of the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-06-01 Thread Robert Haas
On Thu, May 31, 2012 at 11:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm ... first question is do we actually care whether the clocks are synced to the millisecond level, ie what is it you'd do differently if you know that the master and slave clocks are synced more closely than you can

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 3:40 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 1, 2012 at 3:16 PM, Florian Pflug f...@phlo.org wrote: Ok, now you've lost me. If the read() blocks, how on earth can a few additional pins/unpins ever account for any meaningful execution time? It seems to

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
On Sat, Jun 2, 2012 at 1:48 AM, Merlin Moncure mmonc...@gmail.com wrote: Buffer pins aren't a cache: with a cache you are trying to mask a slow operation (like a disk i/o) with a faster such that the amount of slow operations are minimized.  Buffer pins however are very different in that we

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-06-01 Thread Kohei KaiGai
2012/6/1 Robert Haas robertmh...@gmail.com: On Thu, May 31, 2012 at 3:52 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It may be an option to separate the case into two; a situation to execute the given query immediately just after optimization and never reused, and others. Yes.  Simon

[HACKERS] relation complex types

2012-06-01 Thread Jaime Casanova
Hi, I knew that we create an entry in pg_type for every table we create, what i didn't know is that we actually create 2 entries. for example CREATE TABLE foo (i int); will create types foo and _foo. so, any reason to create 2 entries? anyway, what really kept my attention is that CREATE