Re: [HACKERS] SQL MERGE is quite distinct from UPSERT

2014-07-20 Thread Craig Ringer
On 07/20/2014 12:55 PM, Peter Geoghegan wrote: > There is a *really* big > demand for UPSERT from users, not MERGE, although MERGE is certainly > useful too. The inability to efficiently say "Add this unique-keyed row, or if a row of the same key already exists replace it atomically" is a fundame

Re: [HACKERS] SQL MERGE is quite distinct from UPSERT

2014-07-20 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/21/2014 01:40 AM, Martijn van Oosterhout wrote: > FWIW, I agree. MERGE is hard enough as it is, but trying to > guarentee some kind of atomicity makes it nigh on impossible. > Indeed, after reading what you wrote I think it may well be > impossib

Re: [HACKERS] 9.5: UPDATE/DELETE .. ORDER BY .. LIMIT ..

2014-07-20 Thread Marko Tiikkaja
On 2014-06-24 11:08, Heikki Linnakangas wrote: IMHO this needs to work with inheritance if we are to accept it. It would be a rather strange limitation for no apparent reason, other than that we didn't bother to implement it. It doesn't seem very difficult in theory to add the table OID to the pl

[HACKERS] Draft release notes are up for 9.3.5

2014-07-20 Thread Tom Lane
I've committed first-draft release notes into HEAD (only, so far). As in the past couple of release cycles, I've only created a full new documentation section for 9.3.5; tomorrow I will copy final text into the back-branch release-note files, trimming away whatever does not apply to a given branch.

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Peter Geoghegan
On Sun, Jul 20, 2014 at 10:56 AM, Tom Lane wrote: > However, this is certainly a behavioral change. Perhaps squeeze it > into 9.4, but not the back braches? +1 -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
If you do not like my normalization hack (I do not like it much either:-), I have suggested to add "&& !IsA(parsetree, DeallocateStmt)" to the condition above, which would ignore DEALLOCATE as PREPARE and EXECUTE are currently and rightfully ignored. Well, EXECUTE isn't actually ignored, but

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Tom Lane
Andres Freund writes: > On 2014-07-20 17:01:50 +0200, Fabien COELHO wrote: >> If you do not like my normalization hack (I do not like it much either:-), I >> have suggested to add "&& !IsA(parsetree, DeallocateStmt)" to the condition >> above, which would ignore DEALLOCATE as PREPARE and EXECUTE a

Re: [HACKERS] SQL MERGE is quite distinct from UPSERT

2014-07-20 Thread Martijn van Oosterhout
On Sat, Jul 19, 2014 at 09:55:19PM -0700, Peter Geoghegan wrote: > At a high level SQL MERGE is quite distinct from UPSERT, in that it is > a utility command that performs inserts, updates and deletes while > avoiding race conditions (e.g. unique constraint violations) on a more > or less best effo

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-07-20 Thread Tomas Vondra
Attached v9 of the patch. Aside from a few minor fixes, the main change is that this is assumed to be combined with the "dense allocation" patch. It also rewrites the ExecHashIncreaseNumBuckets to follow the same pattern as ExecHashIncreaseNumBatches (i.e. scanning chunks directly, instead of buc

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-20 Thread Tomas Vondra
On 19.7.2014 20:24, Tomas Vondra wrote: > On 13.7.2014 21:32, Tomas Vondra wrote: >> The current patch only implemnents this for tuples in the main >> hash table, not for skew buckets. I plan to do that, but it will >> require separate chunks for each skew bucket (so we can remove it >> without mes

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
[...]. If we do something we should go for the && !IsA(parsetree, DeallocateStmt), not the normalization. Ok. The latter is pretty darn bogus. Yep:-) I'm fine with ignoring DEALLOCATE altogether. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Andres Freund
On 2014-07-20 17:01:50 +0200, Fabien COELHO wrote: > > >That's because PREPARE isn't executed as it's own statement, but done on > >the protocol level (which will need noticeably fewer messages). There's > >no builtin logic to ignore actual PREPARE statements. > > ISTM that there is indeed a spec

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
That's because PREPARE isn't executed as it's own statement, but done on the protocol level (which will need noticeably fewer messages). There's no builtin logic to ignore actual PREPARE statements. ISTM that there is indeed a special handling in function pgss_ProcessUtility for PREPARE and E

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
That's because PREPARE isn't executed as it's own statement, but done on the protocol level (which will need noticeably fewer messages). There's no builtin logic to ignore actual PREPARE statements. ISTM that there is indeed a special handling in function pgss_ProcessUtility for PREPARE and E

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-20 Thread Tomas Vondra
On 20.7.2014 00:12, Tomas Vondra wrote: > On 19.7.2014 23:07, Tomas Vondra wrote: >> On 19.7.2014 20:28, Tomas Vondra wrote: >> For the first case, a WARNING at the end of estimate_hash_bucketsize >> says this: >> >> WARNING: nbuckets=8388608.00 estfract=0.01 >> WARNING: nbuckets=65536.00 est

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Andres Freund
On 2014-07-20 14:43:27 +0200, Fabien COELHO wrote: > >a) Consider using the extended query protocol. > >b) consider using unnamed prepared statements to reduce the number of > > roundtrips > >c) wonder why PREPARE/DEALLOCATE are so much more frequent than the > > actualy query execution. > > (1)

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
Hello Andres, Why isn't the driver using the extended query protocol? Sending PREPARE/EXECUTE/DEALLOCATE wastes roundtrips... It seems to me that it would be more helful if these similar entries where aggregated together, that is if the query "normalization" could ignore the name of the descr

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Marko Tiikkaja
On 2014-07-20 14:06, Andres Freund wrote: On 2014-07-20 13:54:01 +0200, Andres Freund wrote: On 2014-04-01 16:45:29 +0200, Fabien COELHO wrote: I noticed that my pg_stat_statements is cluttered with hundreds of entries like "DEALLOCATE dbdpg_p123456_7", occuring each only once. Why isn't the

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Andres Freund
On 2014-07-20 13:54:01 +0200, Andres Freund wrote: > Hi, > > On 2014-04-01 16:45:29 +0200, Fabien COELHO wrote: > > I noticed that my pg_stat_statements is cluttered with hundreds of entries > > like "DEALLOCATE dbdpg_p123456_7", occuring each only once. > > Why isn't the driver using the extende

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Andres Freund
Hi, On 2014-04-01 16:45:29 +0200, Fabien COELHO wrote: > I noticed that my pg_stat_statements is cluttered with hundreds of entries > like "DEALLOCATE dbdpg_p123456_7", occuring each only once. Why isn't the driver using the extended query protocol? Sending PREPARE/EXECUTE/DEALLOCATE wastes round

Re: [HACKERS] pg_stat_statements cluttered with "DEALLOCATE dbdpg_p*"

2014-07-20 Thread Fabien COELHO
Hello devs, I noticed that my pg_stat_statements is cluttered with hundreds of entries like "DEALLOCATE dbdpg_p123456_7", occuring each only once. Here is a patch and sql test file to: * normalize DEALLOCATE utility statements in pg_stat_statements Some drivers such as DBD:Pg generate proce

Re: [HACKERS] Built-in binning functions

2014-07-20 Thread Simon Riggs
On 16 July 2014 20:35, Pavel Stehule wrote: > > > > 2014-07-16 10:04 GMT+02:00 Petr Jelinek : > >> On 08/07/14 02:14, Tom Lane wrote: >>> >>> Petr Jelinek writes: here is a patch implementing varwidth_bucket (naming is up for discussion) function which does binning with variable bu

[HACKERS] Production block comparison facility

2014-07-20 Thread Simon Riggs
The block comparison facility presented earlier by Heikki would not be able to be used in production systems. ISTM that it would be desirable to have something that could be used in that way. ISTM easy to make these changes * optionally generate a FPW for every WAL record, not just first change a