Re: [HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Jeff Davis
On Sun, 2015-02-22 at 00:07 -0500, Tom Lane wrote: > If you want to have just *one* variable but change its name and type, > I'd be ok with that. Thank you for taking a quick look. Committed as a simple rename from "context" to "set". Regards, Jeff Davis -- Sent via pgsql-hackers mai

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Jeff Davis
On Sun, 2015-02-22 at 03:14 +0100, Tomas Vondra wrote: > >> SELECT COUNT(x) FROM ( > >> SELECT a, array_agg(i) AS x FRM test GROUP BY 1 > >> ) foo; > > > > That's actually a bogus test -- array_agg is never executed. > > Really? How could that happen when the result of array_agg() is pas

Re: [HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Tom Lane
Jeff Davis writes: > This is just to make the variable names in this function consistent with > the rest of the file (and less confusing). No functional impact, so I'll > commit soon unless someone objects. If I'm reading this right, this creates two different variables pointing at the same objec

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-21 Thread Petr Jelinek
On 13/02/15 18:43, Heikki Linnakangas wrote: Ok, I don't hear any loud objections to min_wal_size and max_wal_size, so let's go with that then. Attached is a new version of this. It now comes in four patches. The first three are just GUC-related preliminary work, the first of which I posted on

Re: [HACKERS] Replication identifiers, take 4

2015-02-21 Thread Petr Jelinek
Now that the issue with padding seems to no longer exists since the patch works both with and without padding, I went through the code and here are some comments I have (in no particular order). In CheckPointReplicationIdentifier: + * FIXME: Add a CRC32 to the end. The function already does

[HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Jeff Davis
This is just to make the variable names in this function consistent with the rest of the file (and less confusing). No functional impact, so I'll commit soon unless someone objects. Previously, this was part of the memory accounting patch, but that hasn't made it in yet. Might as well commit the c

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Tomas Vondra
On 22.2.2015 02:38, Jeff Davis wrote: > On Wed, 2015-01-28 at 23:25 +0100, Tomas Vondra wrote: >> 3) moves the assert into the 'if (release)' branch > > You missed one, but I got it. Oh, thanks! > >> 4) includes the comments proposed by Ali Akbar in his reviews >> >>Warnings at makeArrayRes

Re: [HACKERS] proposal: searching in array function - array_position

2015-02-21 Thread Petr Jelinek
On 28/01/15 08:15, Pavel Stehule wrote: 2015-01-28 0:01 GMT+01:00 Jim Nasby mailto:jim.na...@bluetreble.com>>: On 1/27/15 4:36 AM, Pavel Stehule wrote: It is only partially identical - I would to use cache for array_offset, but it is not necessary for array_offsets ..

[HACKERS] Abbreviated keys for text cost model fix

2015-02-21 Thread Peter Geoghegan
Over in the abbreviated keys for numeric thread, Tomas Vondra reports a case where the ad-hoc cost model of abbreviated keys/sortsupport for text is far too conservative, and aborts a case where abbreviated keys can still greatly help. Previously, I proposed additions to the cost model that dealt

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Jeff Davis
On Wed, 2015-01-28 at 23:25 +0100, Tomas Vondra wrote: > 3) moves the assert into the 'if (release)' branch You missed one, but I got it. > 4) includes the comments proposed by Ali Akbar in his reviews > >Warnings at makeArrayResult/makeMdArrayResult about freeing memory >with private su

Re: [HACKERS] Parallel Seq Scan

2015-02-21 Thread Robert Haas
On Tue, Feb 17, 2015 at 11:22 AM, Andres Freund wrote: >> I still think this belongs in heapam.c somehow or other. If the logic >> is all in the executor, then it becomes impossible for any code that >> doensn't use the executor to do a parallel heap scan, and that's >> probably bad. It's not ha

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-21 Thread Petr Jelinek
On 21/02/15 22:09, Andrew Dunstan wrote: On 02/16/2015 09:05 PM, Petr Jelinek wrote: I found one more issue with the 1.2--1.3 upgrade script, the DROP FUNCTION pg_stat_statements(); should be DROP FUNCTION pg_stat_statements(bool); since in 1.2 the function identity has changed. I think a

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 23:09, Peter Geoghegan wrote: > On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra > wrote: >>> Although I cannot easily explain the disparity in performance between >>> 1M and 5M sized sets for this query: >>> >>> select count(distinct randtxt) from stuff_text >>> >>> You did make sure t

[HACKERS] SSL renegotiation

2015-02-21 Thread Emil Lenngren
Hi. I noticed your latest mail thread about ssl renegotiations and this commit by hlinnaka: https://github.com/postgres/postgres/commit/272923a0a6956187471df4f032eee06559520390 and how renegotiation is done since that. Before this commit, after sending a Hello Request, the backend waits until the

Re: [HACKERS] Precedence of standard comparison operators

2015-02-21 Thread Tom Lane
Attached is a draft patch to bring the precedence of comparison operators and IS tests into line with the SQL standard. I have not yet looked into producing warnings for changes in parsing decisions; but I was gratified to discover that this patch results in none, nada, zero changes in any of our

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-21 17:43:09 +0100, Andres Freund wrote: > One thing I was considering was to do the regtype and regproc lookups > directly in the tool. That'd have two advantages: 1) it'd make it > possible to refer to typenames in pg_proc, 2) It'd be much faster. Right > now most of initdb's time is do

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 2:09 PM, Peter Geoghegan wrote: > I can recreate the problem you see with text sort regressions. > Abbreviation is aborted for the case in question, unsurprisingly, and > fairly far in. With that many tuples, the idea of taking abbreviated > cardinality as a proxy for full

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra wrote: >> Although I cannot easily explain the disparity in performance between >> 1M and 5M sized sets for this query: >> >> select count(distinct randtxt) from stuff_text >> >> You did make sure that the queries didn't spill to disk, right? Or >> th

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Tom Lane
After some more hacking, the only remaining uses of foo[1] in struct declarations are: 1. A couple of places where the array is actually the only struct member; for some unexplainable reason gcc won't let you use flexible array syntax in that case. 2. struct sqlda_struct in ecpg's sqlda-native.h.

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-21 14:51:32 -0500, Stephen Frost wrote: > It'd be *really* nice to be able to pass an object identifier to some > function and get back the CREATE (in particular, though perhaps DROP, or > whatever) command for it. This gets us *awful* close to that without > actually giving it to us an

Re: [HACKERS] NOT NULL markings for BKI columns

2015-02-21 Thread Tom Lane
Andres Freund writes: > On 2015-02-20 19:06:50 -0300, Alvaro Herrera wrote: >> I think we should just save this part of the patch until some use turns up. > I pondered this for a while and I don't agree. If the flag had been > available a couple column that now use 0 instead of NULLs and such wou

Re: [HACKERS] NOT NULL markings for BKI columns

2015-02-21 Thread Andres Freund
On 2015-02-20 19:06:50 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2015-02-15 12:54:45 -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > BTW, the precedent of PG_USED_FOR_ASSERTS_ONLY would suggest calling > > > this one PG_FORCE_NOT_NULL, or at least using underscores for

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-21 Thread Andrew Dunstan
On 02/16/2015 09:05 PM, Petr Jelinek wrote: On 17/02/15 02:57, Andrew Dunstan wrote: On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek wrote: We definitely want this feature, I wished to have this info many t

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Andres Freund
On 2015-02-21 15:16:55 -0500, Tom Lane wrote: > Andres, would you double-check the changes in reorderbuffer.c? > There were some weird calculations with > offsetof(ReorderBufferTupleBuf, data) - offsetof(HeapTupleHeaderData, t_bits) > which Michael simplified in a way that's not 100% equivalent.

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 11:15 AM, Heikki Linnakangas wrote: > Ah, ok, I can see the confusion now. Cool. >> Do we not wait on anything, and just declare that we're done? Then I >> think that breaks exclusion constraint enforcement, because we need to >> rescan the index to do that (i.e., "goto r

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Tom Lane
Michael Paquier writes: > And after all those commits attached is a patch changing > HeapTupleHeaderData, using the following macro to track the size of > the structure: > #define SizeofHeapTupleHeader offsetof(HeapTupleHeaderData, t_bits) I've pushed this with a few minor fixes (mostly, using MA

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 20:33, Peter Geoghegan wrote: > On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan > >> That's odd. I have a hard time thinking of why the datum sort >> patch could be at fault, though. > > Oh, wait. For queries like this, which I now see in your > spreadsheet: > > select * from (s

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 19:57, Peter Geoghegan wrote: > On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra > wrote: >> The gains for text are also very nice, although in this case that only >> happens for the smallest scale (1M rows), and for larger scales it's >> actually slower than current master :-( > > That

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On February 21, 2015 7:20:04 PM CET, Andrew Dunstan wrote: > >On 02/21/2015 11:43 AM, Tom Lane wrote: > >> { >> oid => 2249, >> oiddefine => 'CSTRINGOID', >> typname => 'cstring', >> typlen => -2, >> typbyval => 1, >> ... >> } > > >which

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Stephen Frost
Alvaro, all, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > This is a repost of the patch to add CREATE command deparsing support to > event triggers. It now supports not only CREATE but also ALTER and > other commands such as GRANT/REVOKE, COMMENT ON and SECURITY LABEL. > This patch series

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan wrote: > On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra > wrote: >> The gains for text are also very nice, although in this case that only >> happens for the smallest scale (1M rows), and for larger scales it's >> actually slower than current maste

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-21 Thread Heikki Linnakangas
On 02/21/2015 12:15 AM, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 1:07 PM, Heikki Linnakangas wrote: Then I refuse to believe that the livelock hazard exists, without the pre-check. If you have a livelock scenario in mind, it really shouldn't be that difficult to write down the list of ste

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra wrote: > The gains for text are also very nice, although in this case that only > happens for the smallest scale (1M rows), and for larger scales it's > actually slower than current master :-( That's odd. I have a hard time thinking of why the datum s

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Kevin Grittner
Tom Lane wrote: > Robert Haas writes: >> I'm not really sure what Josh was talking about in that answer. In >> terms of doing this automatically, I doubt that's likely to happen >> until we have a way to automatically update a materialized view when >> the underlying data changes --- and Kevin Gr

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 11:43 AM, Tom Lane wrote: Andrew Dunstan writes: On 02/21/2015 09:39 AM, Andrew Dunstan wrote: Personally, I think I would prefer that we use JSON (and yes, there's a JSON::Tiny module, which definitely lives up to its name). For one thing, we've made a feature of supporting JSO

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Robert Haas
On Sat, Feb 21, 2015 at 1:04 PM, Tom Lane wrote: > Robert Haas writes: >> I'm not really sure what Josh was talking about in that answer. In >> terms of doing this automatically, I doubt that's likely to happen >> until we have a way to automatically update a materialized view when >> the underl

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Tom Lane
Robert Haas writes: > I'm not really sure what Josh was talking about in that answer. In > terms of doing this automatically, I doubt that's likely to happen > until we have a way to automatically update a materialized view when > the underlying data changes --- and Kevin Grittner has done a bunc

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Robert Haas
On Sun, Feb 15, 2015 at 10:05 PM, Eric Grinstein wrote: > Are there any plans for implementing query rewriting (i.e. allowing the > optimizer to substitute materialized views for queries) > in upcoming versions? I have recently seen this answer, saying that query > rewriting could be achieved usin

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-21 11:34:09 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: > >> On 2/20/15 8:46 PM, Josh Berkus wrote: > >>> Or what about just doing CSV? > > >> I don't think that would actually address the problems. It would just > >> be the

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Tom Lane
Andrew Dunstan writes: > On 02/21/2015 09:39 AM, Andrew Dunstan wrote: >> Personally, I think I would prefer that we use JSON (and yes, there's >> a JSON::Tiny module, which definitely lives up to its name). >> For one thing, we've made a feature of supporting JSON, so arguably we >> should eat

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Tom Lane
Andres Freund writes: > On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: >> On 2/20/15 8:46 PM, Josh Berkus wrote: >>> Or what about just doing CSV? >> I don't think that would actually address the problems. It would just >> be the same format as now with different delimiters. > Yea, we ne

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
Hi, On 2015-02-15 01:48:15 -0300, Alvaro Herrera wrote: > One line of defense which I just tought about is that instead of > sprinkling EventTriggerStashCommand() calls all over ProcessUtilitySlow, > we should add only one at the bottom. Doesn't sound like a bad idea, but I'm not sure whether it'

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
Hi Gavin, On 21.2.2015 06:35, Gavin Flower wrote: > On 21/02/15 18:18, Tomas Vondra wrote: >> >> OK, so I've repeated the benchmarks with both patches applied, and I >> think the results are interesting. I extended the benchmark a bit - see >> the SQL script attached. >> >>1) multiple queries

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 09:39 AM, Andrew Dunstan wrote: On 02/21/2015 05:04 AM, Andres Freund wrote: Yes, that's a good point. I have zero desire to open-code a format though, I think that's a bad idea. We could say we just include Yaml::Tiny, that's what it's made for. Personally, I think I would

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 05:04 AM, Andres Freund wrote: Yes, that's a good point. I have zero desire to open-code a format though, I think that's a bad idea. We could say we just include Yaml::Tiny, that's what it's made for. Personally, I think I would prefer that we use JSON (and yes, there's a JSO

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Michael Paquier
On Fri, Feb 20, 2015 at 8:57 PM, Michael Paquier wrote: > Attached is a new series. 0001 and 0002 are the same, 0003 and 0004 > the backend structures listed previously. I noticed as well that > indexed_tlist in setrefs.c meritates some attention. And after all those commits attached is a patch c

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-19 14:39:27 -0300, Alvaro Herrera wrote: > diff --git a/src/backend/catalog/objectaddress.c > b/src/backend/catalog/objectaddress.c > index d899dd7..2bbc15d 100644 > --- a/src/backend/catalog/objectaddress.c > +++ b/src/backend/catalog/objectaddress.c > @@ -531,6 +531,12 @@ ObjectTypeMa

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-19 17:14:57 -0300, Alvaro Herrera wrote: > The ddl_command_start event occurs just before the execution of > a CREATE, ALTER, DROP, SECURITY LABEL, COMMENT, GRANT or REVOKE > command. No check whether the affected object exists or doesn't > exist is performed befo

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: > On 2/20/15 8:46 PM, Josh Berkus wrote: > > What about YAML? That might have been added somewhat earlier. > > YAML isn't included in Perl, but there is > > Module::Build::YAML - Provides just enough YAML support so that > Module::Build works e