Re: ON CONFLICT DO NOTHING on pg_dump

2018-06-16 Thread Dilip Kumar
On Thu, Jun 14, 2018 at 4:09 PM, Surafel Temesgen wrote: > > > thank you for pointing me that i add basic test and it seems to me the rest > of the test is covered by column_inserts test @@ -172,6 +172,7 @@ typedef struct _dumpOptions char*outputSuperuser; int sequence_data; /* dump sequ

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik wrote: pg_wal_prefetch function will infinitely traverse WAL and prefetch block references i

Re: WAL prefetch

2018-06-16 Thread Thomas Munro
On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra wrote: > On 06/15/2018 08:01 PM, Andres Freund wrote: >> On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: >>> On 14.06.2018 09:52, Thomas Munro wrote: Why stop at the page cache... what about shared buffers? >>> >>> It is good question. I

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/16/2018 12:06 PM, Thomas Munro wrote: On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: Why stop at the page cache... what about shared

Re: [HACKERS] Statement-level rollback

2018-06-16 Thread Simon Riggs
On 15 June 2018 at 21:23, Alvaro Herrera wrote: > I think the main objectionable point is that of making servers behave in > a way that could lose data, if applications assume that transactions > behave in the way they do today. I propose that we solve this by > allowing this feature to be enabl

Re: Postgres 11 release notes

2018-06-16 Thread Komяpa
> > > I'm not sure it is usefull in release notes since it is more about API, > and not > > user-facing change. Just in case. > > GiST opclasses now can omit compress and decompress functions. If > compress > > function is omited, IndexOnlyScan is enabled for opclass without any > extra > > change.

Re: WAL prefetch

2018-06-16 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 06/16/2018 12:06 PM, Thomas Munro wrote: > >On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra > > wrote: > >>On 06/15/2018 08:01 PM, Andres Freund wrote: > >>>On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > On 14.06.20

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-16 Thread Amit Kapila
On Sat, Jun 16, 2018 at 10:44 AM, Amit Kapila wrote: > On Thu, Jun 14, 2018 at 9:54 PM, Tom Lane wrote: >> Rajkumar Raghuwanshi writes: >>> I am getting a server crash for below test case. >> >>> postgres=# select (select max((select t1.c2 from test t1 where t1.c1 = >>> t2.c1))) from test t2; >>

Query Rewrite for Materialized Views (Postgres Extension)

2018-06-16 Thread John Dent
Hi folks, I thought I’d share an update to my pet project, which dynamically rewrites queries to target materialized views when they are available and can satisfy a query (or part of it) with a lower cost plan. The extension is now a regular EXTENSION and no longer tied in to the FDW mechanism

Re: row_to_json(), NULL values, and AS

2018-06-16 Thread Tom Lane
I wrote: > Neil Conway writes: >> On Fri, Jun 15, 2018 at 7:53 AM Tom Lane wrote: >>> I'm a bit hesitant to muck with this behavior, given that it's stood >>> for ~20 years. However, if we did want to touch it, maybe the right >>> thing would be to give up the absolutist position that f(x) and x

Re: POC: GROUP BY optimization

2018-06-16 Thread Tomas Vondra
On 06/13/2018 06:56 PM, Teodor Sigaev wrote: >> I.e. we already do reorder the group clauses to match ORDER BY, to only >> require a single sort. This happens in preprocess_groupclause(), which >> also explains the reasoning behind that. > Huh. I missed that. That means group_keys_reorder_by_pat

Re: GCC 8 warnings

2018-06-16 Thread Tom Lane
Andres Freund writes: > On 2018-04-28 12:16:39 -0400, Tom Lane wrote: >> In the meantime, I think our response to GCC 8 should just be to >> enable -Wno-format-truncation. Certainly so in the back branches. >> There isn't one of these changes that is actually fixing a bug, >> which to me says tha

Re: GCC 8 warnings

2018-06-16 Thread Andres Freund
Hi, On 2018-06-16 13:29:55 -0400, Tom Lane wrote: > I propose the attached patch to disable these warnings if the compiler > knows the switch for them. I did not turn them off for CXX though; > anyone think there's a need to? No, not for now. I don't think it's likely that the amount of C++ wil

Re: GCC 8 warnings

2018-06-16 Thread Tom Lane
Andres Freund writes: > On 2018-06-16 13:29:55 -0400, Tom Lane wrote: >> + # Similarly disable useless truncation warnings from gcc 8+ >> + PGAC_PROG_CC_VAR_OPT(NOT_THE_CFLAGS, [-Wformat-truncation]) >> + if test -n "$NOT_THE_CFLAGS"; then >> +CFLAGS="$CFLAGS -Wno-format-truncation" >> + f

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: > > > On 06/15/2018 08:01 PM, Andres Freund wrote: > > On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > > > > > > > > > On 14.06.2018 09:52, Thomas Munro wrote: > > > > On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik > > > > wrote: >

Re: WAL prefetch

2018-06-16 Thread Andres Freund
Hi, On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: > Usage: > 1. At master: create extension wal_prefetch > 2. At replica: Call pg_wal_prefetch() function: it will not return until you > interrupt it. FWIW, I think the proper design would rather be a background worker that does this wor

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/16/2018 09:02 PM, Andres Freund wrote: > On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: >> >> >> On 06/15/2018 08:01 PM, Andres Freund wrote: >>> On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: > On Thu, Jun 14, 2018 at 1

Re: WAL prefetch

2018-06-16 Thread Andres Freund
Hi, On 2018-06-16 21:34:30 +0200, Tomas Vondra wrote: > > - it leads to guaranteed double buffering, in a way that's just about > > guaranteed to *never* be useful. Because we'd only prefetch whenever > > there's an upcoming write, there's simply no benefit in the page > > staying in the pag

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-06-16 Thread Komяpa
Hi! It is cool to see this in Postgres 11. However: > 4) vacuum_cleanup_index_scale_factor can be set either by GUC or > reloption. > Default value is 0.1. So, by default cleanup scan is triggered after > increasing of > table size by 10%. > vacuum_cleanup_index_scale_factor can be set to the

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:02, Andres Freund wrote: On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:23, Andres Freund wrote: Hi, On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: Usage: 1. At master: create extension wal_prefetch 2. At replica: Call pg_wal_prefetch() function: it will not return until you interrupt it. FWIW, I think the proper design would rather b

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 23:25:34 +0300, Konstantin Knizhnik wrote: > > > On 16.06.2018 22:02, Andres Freund wrote: > > On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: > > > > > > On 06/15/2018 08:01 PM, Andres Freund wrote: > > > > On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > > > > > > >

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 23:31:49 +0300, Konstantin Knizhnik wrote: > > > On 16.06.2018 22:23, Andres Freund wrote: > > Hi, > > > > On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: > > > Usage: > > > 1. At master: create extension wal_prefetch > > > 2. At replica: Call pg_wal_prefetch() function: i

Re: Removing "Included attributes in B-tree indexes" section from docs

2018-06-16 Thread Alvaro Herrera
On 2018-Jun-15, Peter Geoghegan wrote: > I propose removing the "Included attributes in B-tree indexes" > top-level section of chapter 63 from the user facing documentation. Hi Peter, I don't necessarily object to the proposed change, but I think you should generally wait a bit longer for others

Re: Slow planning time for simple query

2018-06-16 Thread Amit Kapila
On Sun, Jun 10, 2018 at 1:19 AM, Tom Lane wrote: > Maksim Milyutin writes: >> On hot standby I faced with the similar problem. >> ... >> is planned 4.940 ms on master and *254.741* ms on standby. > > Presumably the problem is that the standby isn't authorized to change > the btree index's "entry

Re: Slow planning time for simple query

2018-06-16 Thread Amit Kapila
On Thu, Jun 14, 2018 at 4:34 AM, Maksim Milyutin wrote: > 13.06.2018 12:40, Maksim Milyutin wrote: > > On 09.06.2018 22:49, Tom Lane wrote: > > Maksim Milyutin writes: > > On hot standby I faced with the similar problem. > ... > is planned 4.940 ms on master and *254.741* ms on standby. > > (I wo

Re: ON CONFLICT DO NOTHING on pg_dump

2018-06-16 Thread Dilip Kumar
On Thu, Jun 14, 2018 at 4:09 PM, Surafel Temesgen wrote: > > > thank you for pointing me that i add basic test and it seems to me the rest > of the test is covered by column_inserts test @@ -172,6 +172,7 @@ typedef struct _dumpOptions char*outputSuperuser; int sequence_data; /* dump sequ

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik wrote: pg_wal_prefetch function will infinitely traverse WAL and prefetch block references i

Re: WAL prefetch

2018-06-16 Thread Thomas Munro
On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra wrote: > On 06/15/2018 08:01 PM, Andres Freund wrote: >> On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: >>> On 14.06.2018 09:52, Thomas Munro wrote: Why stop at the page cache... what about shared buffers? >>> >>> It is good question. I

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/16/2018 12:06 PM, Thomas Munro wrote: On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: Why stop at the page cache... what about shared

Re: [HACKERS] Statement-level rollback

2018-06-16 Thread Simon Riggs
On 15 June 2018 at 21:23, Alvaro Herrera wrote: > I think the main objectionable point is that of making servers behave in > a way that could lose data, if applications assume that transactions > behave in the way they do today. I propose that we solve this by > allowing this feature to be enabl

Re: Postgres 11 release notes

2018-06-16 Thread Komяpa
> > > I'm not sure it is usefull in release notes since it is more about API, > and not > > user-facing change. Just in case. > > GiST opclasses now can omit compress and decompress functions. If > compress > > function is omited, IndexOnlyScan is enabled for opclass without any > extra > > change.

Re: WAL prefetch

2018-06-16 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 06/16/2018 12:06 PM, Thomas Munro wrote: > >On Sat, Jun 16, 2018 at 9:38 PM, Tomas Vondra > > wrote: > >>On 06/15/2018 08:01 PM, Andres Freund wrote: > >>>On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > On 14.06.20

Re: server crashed with TRAP: FailedAssertion("!(!parallel_aware || pathnode->path.parallel_safe)"

2018-06-16 Thread Amit Kapila
On Sat, Jun 16, 2018 at 10:44 AM, Amit Kapila wrote: > On Thu, Jun 14, 2018 at 9:54 PM, Tom Lane wrote: >> Rajkumar Raghuwanshi writes: >>> I am getting a server crash for below test case. >> >>> postgres=# select (select max((select t1.c2 from test t1 where t1.c1 = >>> t2.c1))) from test t2; >>

Query Rewrite for Materialized Views (Postgres Extension)

2018-06-16 Thread John Dent
Hi folks, I thought I’d share an update to my pet project, which dynamically rewrites queries to target materialized views when they are available and can satisfy a query (or part of it) with a lower cost plan. The extension is now a regular EXTENSION and no longer tied in to the FDW mechanism

Re: row_to_json(), NULL values, and AS

2018-06-16 Thread Tom Lane
I wrote: > Neil Conway writes: >> On Fri, Jun 15, 2018 at 7:53 AM Tom Lane wrote: >>> I'm a bit hesitant to muck with this behavior, given that it's stood >>> for ~20 years. However, if we did want to touch it, maybe the right >>> thing would be to give up the absolutist position that f(x) and x

Re: POC: GROUP BY optimization

2018-06-16 Thread Tomas Vondra
On 06/13/2018 06:56 PM, Teodor Sigaev wrote: >> I.e. we already do reorder the group clauses to match ORDER BY, to only >> require a single sort. This happens in preprocess_groupclause(), which >> also explains the reasoning behind that. > Huh. I missed that. That means group_keys_reorder_by_pat

Re: GCC 8 warnings

2018-06-16 Thread Tom Lane
Andres Freund writes: > On 2018-04-28 12:16:39 -0400, Tom Lane wrote: >> In the meantime, I think our response to GCC 8 should just be to >> enable -Wno-format-truncation. Certainly so in the back branches. >> There isn't one of these changes that is actually fixing a bug, >> which to me says tha

Re: GCC 8 warnings

2018-06-16 Thread Andres Freund
Hi, On 2018-06-16 13:29:55 -0400, Tom Lane wrote: > I propose the attached patch to disable these warnings if the compiler > knows the switch for them. I did not turn them off for CXX though; > anyone think there's a need to? No, not for now. I don't think it's likely that the amount of C++ wil

Re: GCC 8 warnings

2018-06-16 Thread Tom Lane
Andres Freund writes: > On 2018-06-16 13:29:55 -0400, Tom Lane wrote: >> + # Similarly disable useless truncation warnings from gcc 8+ >> + PGAC_PROG_CC_VAR_OPT(NOT_THE_CFLAGS, [-Wformat-truncation]) >> + if test -n "$NOT_THE_CFLAGS"; then >> +CFLAGS="$CFLAGS -Wno-format-truncation" >> + f

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: > > > On 06/15/2018 08:01 PM, Andres Freund wrote: > > On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > > > > > > > > > On 14.06.2018 09:52, Thomas Munro wrote: > > > > On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik > > > > wrote: >

Re: WAL prefetch

2018-06-16 Thread Andres Freund
Hi, On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: > Usage: > 1. At master: create extension wal_prefetch > 2. At replica: Call pg_wal_prefetch() function: it will not return until you > interrupt it. FWIW, I think the proper design would rather be a background worker that does this wor

Re: WAL prefetch

2018-06-16 Thread Tomas Vondra
On 06/16/2018 09:02 PM, Andres Freund wrote: > On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: >> >> >> On 06/15/2018 08:01 PM, Andres Freund wrote: >>> On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: > On Thu, Jun 14, 2018 at 1

Re: WAL prefetch

2018-06-16 Thread Andres Freund
Hi, On 2018-06-16 21:34:30 +0200, Tomas Vondra wrote: > > - it leads to guaranteed double buffering, in a way that's just about > > guaranteed to *never* be useful. Because we'd only prefetch whenever > > there's an upcoming write, there's simply no benefit in the page > > staying in the pag

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-06-16 Thread Komяpa
Hi! It is cool to see this in Postgres 11. However: > 4) vacuum_cleanup_index_scale_factor can be set either by GUC or > reloption. > Default value is 0.1. So, by default cleanup scan is triggered after > increasing of > table size by 10%. > vacuum_cleanup_index_scale_factor can be set to the

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:02, Andres Freund wrote: On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:23, Andres Freund wrote: Hi, On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: Usage: 1. At master: create extension wal_prefetch 2. At replica: Call pg_wal_prefetch() function: it will not return until you interrupt it. FWIW, I think the proper design would rather b

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 23:25:34 +0300, Konstantin Knizhnik wrote: > > > On 16.06.2018 22:02, Andres Freund wrote: > > On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: > > > > > > On 06/15/2018 08:01 PM, Andres Freund wrote: > > > > On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: > > > > > > >

Re: WAL prefetch

2018-06-16 Thread Andres Freund
On 2018-06-16 23:31:49 +0300, Konstantin Knizhnik wrote: > > > On 16.06.2018 22:23, Andres Freund wrote: > > Hi, > > > > On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: > > > Usage: > > > 1. At master: create extension wal_prefetch > > > 2. At replica: Call pg_wal_prefetch() function: i

Re: Removing "Included attributes in B-tree indexes" section from docs

2018-06-16 Thread Alvaro Herrera
On 2018-Jun-15, Peter Geoghegan wrote: > I propose removing the "Included attributes in B-tree indexes" > top-level section of chapter 63 from the user facing documentation. Hi Peter, I don't necessarily object to the proposed change, but I think you should generally wait a bit longer for others

Re: Slow planning time for simple query

2018-06-16 Thread Amit Kapila
On Sun, Jun 10, 2018 at 1:19 AM, Tom Lane wrote: > Maksim Milyutin writes: >> On hot standby I faced with the similar problem. >> ... >> is planned 4.940 ms on master and *254.741* ms on standby. > > Presumably the problem is that the standby isn't authorized to change > the btree index's "entry

Re: Slow planning time for simple query

2018-06-16 Thread Amit Kapila
On Thu, Jun 14, 2018 at 4:34 AM, Maksim Milyutin wrote: > 13.06.2018 12:40, Maksim Milyutin wrote: > > On 09.06.2018 22:49, Tom Lane wrote: > > Maksim Milyutin writes: > > On hot standby I faced with the similar problem. > ... > is planned 4.940 ms on master and *254.741* ms on standby. > > (I wo