Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Fabien COELHO
Hello Robert, I think you're making a mountain out of a molehill. Probably. I tend to try the minimum effort first. I implemented this today in about three hours. The patch is attached. Great! Your patch is 544 lines, my size evaluation was quite good:-) Note that I probably spent 10

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Fabien COELHO
I think you're making a mountain out of a molehill. I implemented this today in about three hours. I think you're greatly understating your own efficiency at shift/reducing parser mountains down to molehills. Fabien even guessed the LOC size of the resulting patch with less than a 9%

Re: [HACKERS] Selectivity estimation for inet operators

2014-09-25 Thread Heikki Linnakangas
On 09/07/2014 07:09 PM, Emre Hasegeli wrote: I updated the patch to cover semi and anti joins with eqjoinsel_semi(). I think it is better than returning a constant. What you did there is utterly unacceptable from a modularity standpoint; and considering that the values will be nowhere near

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a visibility routine. OK. Which visibility routine should I use, and should I try to create a variant that doesn't set hint bits?

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 11:41:29 +0200, and...@2ndquadrant.com wrote: I've not yet followed your premise that you actually need one that doesn't set hint bits... Oh. All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks,

Re: [HACKERS] make pg_controldata accept -D dirname

2014-09-25 Thread Heikki Linnakangas
On 09/24/2014 05:48 PM, Abhijit Menon-Sen wrote: Updated patches attached. Thanks, applied some version of these. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Gregory Smith
On 9/25/14, 1:41 AM, David Johnston wrote: If the error message is written correctly most people upon seeing the error will simply fix their configuration and move on - regardless of whether they were proactive in doing so having read the release notes. The important part to realize here is

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-25 Thread Heikki Linnakangas
On 09/16/2014 01:20 PM, David Rowley wrote: + /* +* We mustn't allow any joins to be removed if there are any pending +* foreign key triggers in the queue. This could happen if we are planning +* a query that has been executed from within a volatile function and

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 1:27 AM, Gregory Smith gregsmithpg...@gmail.com wrote: On 9/24/14, 10:10 PM, Robert Haas wrote: I think you're making a mountain out of a molehill. I implemented this today in about three hours. I think you're greatly understating your own efficiency at shift/reducing

Re: [HACKERS] Sloppy thinking about leakproof properties of opclass co-members

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 5:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: It strikes me that there's a significant gap in the whole leakproof function business, namely that no consideration has been given to planner-driven transformations of queries. As an example, if we have a = b and b = c, the

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Gregory Smith
On 9/25/14, 8:38 AM, Robert Haas wrote: That's my whole reason for not wanting to adopt Fabien's approach in the first place: I was cool with exposing C's modulo operator, but any other modulo semantics seem like they should be built up from general-purpose primitives. Maybe; I don't quite

Re: [HACKERS] Spinlocks and compiler/memory barriers

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 2:45 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-09 17:54:03 -0400, Robert Haas wrote: So, that's committed, then. I think we should pick something that uses spinlocks and is likely to fail spectacularly if we haven't got this totally right yet, and

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread David Johnston
On Thursday, September 25, 2014, Gregory Smith gregsmithpg...@gmail.com wrote: On 9/25/14, 1:41 AM, David Johnston wrote: If the error message is written correctly most people upon seeing the error will simply fix their configuration and move on - regardless of whether they were proactive in

[HACKERS] Inefficient barriers on solaris with sun cc

2014-09-25 Thread Andres Freund
Hi, Binaries compiled on solaris using sun studio cc currently don't have compiler and memory barriers implemented. That means we fall back to relatively slow generic implementations for those. Especially compiler, read, write barriers will be much slower than necessary (since they all just need

Re: [HACKERS] RLS Design

2014-09-25 Thread Thom Brown
On 19 September 2014 17:54, Stephen Frost sfr...@snowman.net wrote: Thom, * Thom Brown (t...@linux.com) wrote: On 19 September 2014 17:32, Stephen Frost sfr...@snowman.net wrote: * Thom Brown (t...@linux.com) wrote: On 14 September 2014 16:38, Stephen Frost sfr...@snowman.net wrote:

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Simon Riggs
On 25 September 2014 10:41, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a visibility routine. OK. Which visibility routine should I

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Andres Freund
On 2014-09-24 17:39:19 -0300, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 9/24/14 9:21 AM, Tom Lane wrote: Agreed, but what about non-GCC compilers? Stick AC_PROG_CC_C99 into configure.in. I think that's a bad idea, unless you mean to do it

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Heikki Linnakangas
On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: On 9/10/14 1:38 PM, Heikki Linnakangas wrote: On 09/10/2014 02:26 PM, Marko Tiikkaja wrote: So I wonder if I shouldn't try and instead keep the code closer to what it is in HEAD right now; I could call enlargeStringInfo() first, then hand out a

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Tue, Sep 23, 2014 at 5:50 PM, Robert Haas robertmh...@gmail.com wrote: The patch I attached the first time was just the last commit in the git repository where I wrote the patch, rather than the changes that I made on top of that commit. So, yes, the results from the previous message are

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Marko Tiikkaja
On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: OK, I've attemped to do that in the attached. I'm pretty sure I didn't get all of the overflows right, so someone should probably take a really good look at it. (I'm not too confident the original code

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Oskari Saarenmaa
24.09.2014, 23:26, Tom Lane kirjoitti: Peter Eisentraut pete...@gmx.net writes: On 9/24/14 9:21 AM, Tom Lane wrote: Agreed, but what about non-GCC compilers? Stick AC_PROG_CC_C99 into configure.in. I think that's a bad idea, unless you mean to do it only on Solaris. If we do that

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Alvaro Herrera
Andres Freund wrote: On 2014-09-24 17:39:19 -0300, Alvaro Herrera wrote: AFAIK we cannot move all the way to C99, because MSVC doesn't support it. FWIW, msvc has supported a good part of C99 for long while. There's bits and pieces it doesn't, but it's not things I think we're likely to

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 14:43:14 +0100, Simon Riggs wrote: On 25 September 2014 10:41, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Andres Freund
On 2014-09-25 10:56:56 -0300, Alvaro Herrera wrote: Andres Freund wrote: From VS 2013 onwards they're trying hard to be C99 and C11 compatible. Sounds great. Is VS2013 released already? Yes. If so, maybe we can think about moving to C99 in 2016 or so; at least assuming you can build for

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas robertmh...@gmail.com wrote: 1. To see the effect of reduce-replacement-locking.patch, compare the first TPS number in each line to the third, or the second to the fourth. At scale factor 1000, the patch wins in all of the cases with 32 or more

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Heikki Linnakangas
On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and avoid the extra palloc and memcpy, but this isn't

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:02 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas robertmh...@gmail.com wrote: 1. To see the effect of reduce-replacement-locking.patch, compare the first TPS number in each line to the third, or the second to the fourth.

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Marko Tiikkaja
On 9/25/14 4:08 PM, Heikki Linnakangas wrote: On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan p...@heroku.com wrote: The patch currently lacks a way of referencing datums rejected for insertion when updating. The way MySQL handles the issue seems questionable. They allow you to do something like this: INSERT INTO upsert (key, val) VALUES (1

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 9:34 AM, Andres Freund and...@2ndquadrant.com wrote: Binaries compiled on solaris using sun studio cc currently don't have compiler and memory barriers implemented. That means we fall back to relatively slow generic implementations for those. Especially compiler, read,

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:51:17 -0400, Robert Haas wrote: On Tue, Sep 23, 2014 at 5:50 PM, Robert Haas robertmh...@gmail.com wrote: The patch I attached the first time was just the last commit in the git repository where I wrote the patch, rather than the changes that I made on top of that commit.

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:02:25 -0500, Merlin Moncure wrote: On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas robertmh...@gmail.com wrote: 1. To see the effect of reduce-replacement-locking.patch, compare the first TPS number in each line to the third, or the second to the fourth. At scale factor 1000,

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund and...@2ndquadrant.com wrote: That leads me to wonder: Have you measured different, lower, number of buffer mapping locks? 128 locks is, if we'd as we should align them properly, 8KB of memory. Common L1 cache sizes are around 32k... Amit has

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:22:47 -0400, Robert Haas wrote: On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund and...@2ndquadrant.com wrote: That leads me to wonder: Have you measured different, lower, number of buffer mapping locks? 128 locks is, if we'd as we should align them properly, 8KB of

Re: [HACKERS] RLS Design

2014-09-25 Thread Stephen Frost
Thom, * Thom Brown (t...@linux.com) wrote: I find it a bit of a limitation that I can't specify both INSERT and UPDATE for a policy. I'd want to be able to specify something like this: CREATE POLICY no_greys_allowed ON colours FOR INSERT, UPDATE WITH CHECK (name NOT IN

Re: [HACKERS] Index scan optimization

2014-09-25 Thread Simon Riggs
On 22 September 2014 14:46, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 09/22/2014 04:45 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 09/22/2014 07:47 AM, Rajeev rastogi wrote: So my proposal is to skip the condition check on the first scan key

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs si...@2ndquadrant.com wrote: IMHO it is impossible to know if any of the other code is correct until we have a clear and stable vision of what the command is supposed to perform. +1. The inner workings are less important than what the feature

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:09:30 -0400, Robert Haas wrote: I think the long-term solution here is that we need a lock-free hash table implementation for our buffer mapping tables, because I'm pretty sure that just cranking the number of locks up and up is going to start to have unpleasant side effects

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund and...@2ndquadrant.com wrote: Why stop at 128 mapping locks? Theoretical downsides to having more mapping locks have been mentioned a few times but has this ever been measured? I'm starting to wonder if the # mapping locks should be dependent

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:24 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:22:47 -0400, Robert Haas wrote: On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund and...@2ndquadrant.com wrote: That leads me to wonder: Have you measured different, lower, number of buffer mapping

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:34:57 -0500, Merlin Moncure wrote: On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund and...@2ndquadrant.com wrote: Why stop at 128 mapping locks? Theoretical downsides to having more mapping locks have been mentioned a few times but has this ever been measured? I'm starting

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: TBH I've also been wondering whether any of these proposed cures are better than the disease. I couldn't agree more. There's something to be said for just leaving this alone. The changes that can be argued to make the

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs si...@2ndquadrant.com wrote: To me, being able to say pg_ctl promote_right_now -m yes_i_mean_it seems like a friendlier interface than making somebody shut down the server, run pg_resetxlog, and start it up again. It makes sense to go from paused

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 15:35, Robert Haas robertmh...@gmail.com wrote: The only problem I see is if the newly inserted row matches one row on one unique value and a different row on a different unique index. Turning the INSERT into an UPDATE will still fail on one or other, no matter which

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: TBH I've also been wondering whether any of these proposed cures are better than the disease. I couldn't agree more. There's something to be said for just leaving this alone.

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-24 21:36:50 +0100, Simon Riggs wrote: On 18 September 2014 01:22, Robert Haas robertmh...@gmail.com wrote: fast promotion was actually a supported option in r8 of Postgres but this option was removed when we implemented streaming replication in r9.0 The *rough* requirement

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 11:13:50 -0400, Robert Haas wrote: On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs si...@2ndquadrant.com wrote: To me, being able to say pg_ctl promote_right_now -m yes_i_mean_it seems like a friendlier interface than making somebody shut down the server, run pg_resetxlog, and

Re: [HACKERS] delta relations in AFTER triggers

2014-09-25 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: You cast the TuplestoreRelation to Plan, and pass it to CopyPlanFields. That will crash, because TuplestoreRelation is nothing like a Plan: Oops. That's a copy/paste error I should have noticed. Fixed, even though the node type might be going

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-19 15:40:14 +0300, Heikki Linnakangas wrote: On 09/18/2014 09:27 PM, Heikki Linnakangas wrote: I'll try to write a more polished patch tomorrow. We'll then see what it looks like, and can decide if we want it. Ok, here are two patches. One is a refined version of my earlier patch,

Re: [HACKERS] RLS Design

2014-09-25 Thread Thom Brown
On 25 September 2014 15:26, Stephen Frost sfr...@snowman.net wrote: I expected this to still trigger an error due to the first policy. Am I to infer from this that the policy model is permissive rather than restrictive? That's correct and I believe pretty clear in the documentation- policies

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:34 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 11:13:50 -0400, Robert Haas wrote: On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs si...@2ndquadrant.com wrote: To me, being able to say pg_ctl promote_right_now -m yes_i_mean_it seems like a friendlier

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:21 AM, Simon Riggs si...@2ndquadrant.com wrote: On 25 September 2014 15:35, Robert Haas robertmh...@gmail.com wrote: The only problem I see is if the newly inserted row matches one row on one unique value and a different row on a different unique index. Turning the

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 7:04 PM, Peter Geoghegan p...@heroku.com wrote: On Fri, Sep 19, 2014 at 2:54 PM, Peter Geoghegan p...@heroku.com wrote: Probably not - it appears to make very little difference to unoptimized pass-by-reference types whether or not datum1 can be used (see my simulation

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 09:01 AM, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decision soon. This issue has held up the next beta (like jsonb has blocked previous beta) for *weeks*. Yes, please! -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Emre Hasegeli
Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: be rechecked from heap tuple before tuple is returned. If literalrecheck/ flag isn't set then it's true by default for compatibility reasons. The literalrecheck/ flag can be

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar dilip.ku...@huawei.com wrote: On 24 August 2014 11:33, Amit Kapila Wrote Thanks for you comments, i have worked on both the review comment lists, sent on 19 August, and 24 August. Latest patch is attached with the mail.. Hi Dilip, I think

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decision soon. This issue has held up the next beta (like jsonb has blocked previous beta) for *weeks*. Personally it doesn't make me very happy that Heikki

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 16:29, Andres Freund and...@2ndquadrant.com wrote: To me, being able to say pg_ctl promote_right_now -m yes_i_mean_it seems like a friendlier interface than making somebody shut down the server, run pg_resetxlog, and start it up again. It makes sense to go from paused

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:14 AM, Bruce Momjian wrote: On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decision soon. This issue has held up the next beta (like jsonb has blocked previous beta) for *weeks*.

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: On 09/25/2014 10:14 AM, Bruce Momjian wrote: On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decision soon. This issue has held up the next beta (like

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Thu, Sep 25, 2014 at 9:00 PM, Emre Hasegeli e...@hasegeli.com wrote: Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: be rechecked from heap tuple before tuple is returned. If literalrecheck/ flag isn't set then it's true by

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:20 AM, Andres Freund wrote: On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: On 09/25/2014 10:14 AM, Bruce Momjian wrote: On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decision soon.

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: On 25 September 2014 16:29, Andres Freund and...@2ndquadrant.com wrote: I think that's not really related. Such a promotion doesn't cause data loss in the sense of loosing data a *clueful* operator wanted to keep. Yes, it can be used wrongly,

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:26 AM, Andres Freund wrote: On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: If Heikki says it's ready, I'll test. So far he's said that it wasn't done yet. http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Yeah, and that didn't include some of Tom's bug

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:29:51 -0700, Josh Berkus wrote: On 09/25/2014 10:26 AM, Andres Freund wrote: On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: If Heikki says it's ready, I'll test. So far he's said that it wasn't done yet.

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: If Heikki says it's ready, I'll test. So far he's said that it wasn't done yet. http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 1:30 PM, Andres Freund and...@2ndquadrant.com wrote: Yes it does cause data loss. The clueful operator has no idea where they are so there is no used rightly in that case. What? There definitely are cases where you don't need to know that to the T. Just think of the -

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Peter Eisentraut
On 9/25/14 11:03 AM, Robert Haas wrote: I couldn't agree more. There's something to be said for just leaving this alone. I agree. potentitally draw complaints. But I also agree with his last one - of those three possible complaints, I certainly prefer I had to fix my configuration file

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas robertmh...@gmail.com wrote: The top issue on my agenda is figuring out a way to get rid of the extra SortSupport object. Really? I'm surprised. Clearly the need to restart heap tuple copying from scratch, in order to make the datum1 representation

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Peter Eisentraut
On 9/24/14 4:58 PM, Stephen Frost wrote: Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I think the case for pgstat_get_backend_current_activity() and pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make and seems acceptable to me; but I would leave

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan p...@heroku.com wrote: Value locking === To date, on-list discussion around UPSERT has almost exclusively concerned what I've called value locking; the idea of locking values in unique indexes in the abstract (to establish the right to insert

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 09/25/2014 10:26 AM, Andres Freund wrote: On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: If Heikki says it's ready, I'll test. So far he's said that it wasn't done yet. http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Yeah, and

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 11:22 AM, Tom Lane wrote: In the interests of pushing this forward, I will work today on trying to finish and review Heikki's offsets-and-lengths patch so that we have something we can do performance testing on. I doubt that the performance testing will tell us anything we don't

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:17 AM, Simon Riggs si...@2ndquadrant.com wrote: Basically, I have absolutely no idea whether I object to or agree with 1) and don't know where to look to find out. We need a clear exposition of design and the alternatives. My approach would be to insert an index

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion. This means we *will* be breaking on-disk compatibility with 9.4beta2, which means pg_upgrade will need to be taught to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:35 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs si...@2ndquadrant.com wrote: IMHO it is impossible to know if any of the other code is correct until we have a clear and stable vision of what the command is supposed to

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion. This means we *will* be breaking on-disk compatibility

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:05 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas robertmh...@gmail.com wrote: The top issue on my agenda is figuring out a way to get rid of the extra SortSupport object. Really? I'm surprised. Clearly the need to restart

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas robertmh...@gmail.com wrote: I've never been a fan of putting the index name in there. Me neither. Although I do understand Kevin's concern about the user's intent surrounding which unique index to merge on. I agree that's stuff that a DML

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion.

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:17 PM, Simon Riggs si...@2ndquadrant.com wrote: 1. You don't accept that value locks must be easily released in the event of a conflict. Is anyone in this camp? It's far from obvious to me what side of this question Andres is on at this stage, for example. Robert

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:53 AM, Robert Haas robertmh...@gmail.com wrote: I haven't looked at that part of the patch in detail yet, so... not really. But I don't see why you'd ever need to restart heap tuple copying. At most you'd need to re-extract datum1 from the tuples you have already

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 09:00:07PM +0200, Andres Freund wrote: On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Alvaro Herrera
Bruce Momjian wrote: 3. 9.3 multi-xact bugs spooked us into being more careful Uh. Multixact changes in 9.3 were infinitely more invasive than the jsonb changes will ever be. a) they touched basic visibility design and routines, which are complex, understood by very few people, and

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 12:11 PM, Robert Haas robertmh...@gmail.com wrote: I think that something like this might work, but the devil is in the details. Suppose two people try to upsert into the same table at the same time. There's one index. If the transactions search that index for

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs si...@2ndquadrant.com wrote: The way forwards, in my view, is to define precisely the behaviour we wish to have. That definition will include the best current mechanism for running an UPSERT using INSERT/UPDATE/loops and comparing that against what

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Thu, Sep 25, 2014 at 10:00 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar dilip.ku...@huawei.com wrote: On 24 August 2014 11:33, Amit Kapila Wrote Thanks for you comments, i have worked on both the review comment lists, sent on 19 August, and

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Gregory Smith
On 9/25/14, 2:02 PM, Peter Eisentraut wrote: But having the same parameter setting mean different things in different versions is the path to complete madness. Could we go so far as to remove support for unitless time settings eventually? The fact that people are setting raw numbers in the

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:11, Robert Haas robertmh...@gmail.com wrote: My approach would be to insert an index tuple for that value into the index, but with the leaf ituple marked with an xid rather than a ctid. If someone tries to insert into the index they would see this and wait for the

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 19:59, Peter Geoghegan p...@heroku.com wrote: On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas robertmh...@gmail.com wrote: I've never been a fan of putting the index name in there. Me neither. Although I do understand Kevin's concern about the user's intent surrounding which

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 9/24/14 4:58 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I think the case for pgstat_get_backend_current_activity() and pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make and seems

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 3:17 PM, Peter Geoghegan p...@heroku.com wrote: To find out how much that optimization buys, you should use tuples with many variable-length columns (say, 50) preceding the text column you're sorting on. I won't be surprised if that turns out to be expensive enough to

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
Hi, On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: Hi Andres, Robert. I've attached four patches here. Cool. Will review. 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to earlier in StartupXLOG. 2. Inside that function, issue fsync()s for the main forks

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:38, Peter Geoghegan p...@heroku.com wrote: On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs si...@2ndquadrant.com wrote: The way forwards, in my view, is to define precisely the behaviour we wish to have. That definition will include the best current mechanism for running

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Gregory Smith (gregsmithpg...@gmail.com) wrote: On 9/25/14, 2:02 PM, Peter Eisentraut wrote: But having the same parameter setting mean different things in different versions is the path to complete madness. Could we go so far as to remove support for unitless time settings eventually?

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 1:21 PM, Simon Riggs si...@2ndquadrant.com wrote: The test index is unique, so our to-be-inserted value exists on only one page, hence page locking applies while we insert it. The next person to insert waits for the page lock and then sees the test tuple. The page lock

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 18:30, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: On 25 September 2014 16:29, Andres Freund and...@2ndquadrant.com wrote: I think that's not really related. Such a promotion doesn't cause data loss in the sense of loosing

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: * Also recovery shouldn't be regarded successful if the reset fails - * e.g. because of ENOSPC. OK. * Doing this in a separate pass is advantageous for performance reasons * because it allows the kernel to perform all the flushes

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
On 2014-09-26 02:34:06 +0530, Abhijit Menon-Sen wrote: At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: Unless I miss something this isn't sufficient. We need to fsync the files in the data directory, not just the toplevel directory? No, of course you're right. So a separate

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 1:48 PM, Simon Riggs si...@2ndquadrant.com wrote: A. UPDATE/INSERT privilege infrastructure. Add tests to it, make it separately committable, so we can get that done. Submit to Oct CF; get that done early. Makes sense. As long as we assume that we want a unified syntax

  1   2   >