Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Chapman Flack
On 03/26/19 21:39, Ryan Lambert wrote: > I can't verify technical accuracy for many of the details (nuances between > XPath 1.0, et. al), but overall my experience with the XML functionality > lines up with what has been documented here. By the way, in case it's buried too far back in the email

Re: speeding up planning with partitions

2019-03-26 Thread David Rowley
On Wed, 27 Mar 2019 at 18:39, Amit Langote wrote: > > On 2019/03/27 14:26, David Rowley wrote: > > Perhaps the way to make this work, at least in the long term is to do > > in the planner what we did in the executor in d73f4c74dd34. > > Maybe you meant 9ddef36278a9? Probably. > What would be

Re: COPY FROM WHEN condition

2019-03-26 Thread Andres Freund
Hi, On 2019-01-29 07:22:16 -0800, Andres Freund wrote: > On January 29, 2019 6:25:59 AM PST, Tomas Vondra > wrote: > >On 1/29/19 8:18 AM, David Rowley wrote: > >> ... > >> Here are my performance tests of with and without your change to the > >> memory contexts (I missed where you posted your

Re: [HACKERS] Block level parallel vacuum

2019-03-26 Thread Haribabu Kommi
On Wed, Mar 27, 2019 at 1:31 AM Masahiko Sawada wrote: > On Tue, Mar 26, 2019 at 10:19 AM Haribabu Kommi > wrote: > > > > > > + for (i = 0; i < nindexes; i++) > > + { > > + LVIndStats *s = &(copied_indstats[i]); > > + > > + if (s->updated) > > + lazy_update_index_statistics(Irel[i],

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 14:26, David Rowley wrote: > On Wed, 27 Mar 2019 at 18:13, Amit Langote > wrote: >> >> On 2019/03/27 13:50, Amit Langote wrote: >>> On 2019/03/27 12:06, Amit Langote wrote: I wonder if I should rework inherit.c so that its internal interfaces don't pass around parent

Re: speeding up planning with partitions

2019-03-26 Thread David Rowley
On Wed, 27 Mar 2019 at 18:13, Amit Langote wrote: > > On 2019/03/27 13:50, Amit Langote wrote: > > On 2019/03/27 12:06, Amit Langote wrote: > >> I wonder if I should rework inherit.c so that its internal interfaces > >> don't pass around parent Relation, but make do with the RelOptInfo? I'll >

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 13:50, Amit Langote wrote: > On 2019/03/27 12:06, Amit Langote wrote: >> I wonder if I should rework inherit.c so that its internal interfaces >> don't pass around parent Relation, but make do with the RelOptInfo? I'll >> need to add tupdesc and reltype fields to RelOptInfo to go

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Chapman Flack
On 03/26/19 21:39, Ryan Lambert wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: not tested > Documentation:tested, passed Thanks for the

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 12:06, Amit Langote wrote: > I wonder if I should rework inherit.c so that its internal interfaces > don't pass around parent Relation, but make do with the RelOptInfo? I'll > need to add tupdesc and reltype fields to RelOptInfo to go ahead with that > though. To give more context

RE: Libpq support to connect to standby server as priority

2019-03-26 Thread Tsunakawa, Takayuki
From: Haribabu Kommi [mailto:kommi.harib...@gmail.com] > while going through the old patch where the GUC_REPORT is implemented, Tom > has commented the logic of sending the signal to all backends to process > the hot standby exit with SIGHUP, if we add the logic of updating the GUC > variable

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-03-26 Thread Masahiko Sawada
Thank you for sharing the updated patch! On Tue, Mar 26, 2019 at 6:26 PM Pavan Deolasee wrote: > > > On Fri, Mar 22, 2019 at 12:19 PM Masahiko Sawada > wrote: >> >> I've looked at the patch and have comments and questions. >> >> +/* >> + * While we are holding the lock on the page,

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Tom Lane
Ryan Lambert writes: > I have two recommendations for features.sgml. You state: >> relies on the libxml library > Should this be clarified as the libxml2 library? That's what I installed to > build postgres from source (Ubuntu 16/18). If it is the libxml library and > the "2" is

Re: Misleading errors with column references in default expressions and partition bounds

2019-03-26 Thread Michael Paquier
On Tue, Mar 26, 2019 at 10:03:35AM -0400, Tom Lane wrote: > +1 for the general idea, but I find the switch a bit overly verbose. > Do we really need to force every new EXPR_KIND to visit this spot, > when so few of them have a need to do anything? I'd be a bit inclined > to simplify it to > >

Re: pg_malloc0() instead of pg_malloc()+memset()

2019-03-26 Thread Michael Paquier
On Tue, Mar 26, 2019 at 09:14:46AM +, Daniel Gustafsson wrote: > Nice, I had missed them as I my eyes set on pg_malloc(). I've done another > pass over > the codebase and I can't spot any other on top of the additional ones you > found where > MemSet() in palloc0 is preferrable over

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-03-26 Thread Michael Paquier
On Wed, Mar 27, 2019 at 11:40:12AM +0900, Amit Langote wrote: > I'm thinking of adding this to open items under Older Bugs. Attached the > patch that I had posted on -bugs, but it's only a rough sketch as > described above, not a full fix. Adding it to the section for older bugs sounds fine to

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-03-26 Thread Amit Langote
On 2019/03/08 19:22, Amit Langote wrote: > On 2019/03/07 20:36, Amit Langote wrote: >> On Thu, Mar 7, 2019 at 11:17 AM Amit Langote >> wrote: >>> The problem start when ALTER TABLE users ALTER COLUMN is executed. >>> create table users(user_id int, name varchar(64), unique (user_id, name)) >>>

Re: Shouldn't current_schema() be at least PARALLEL RESTRICTED?

2019-03-26 Thread Michael Paquier
On Tue, Mar 26, 2019 at 02:16:00PM +, Daniel Gustafsson wrote: > The new status of this patch is: Ready for Committer Thanks Daniel for the review, committed. -- Michael signature.asc Description: PGP signature

Re: performance issue in remove_from_unowned_list()

2019-03-26 Thread Tomas Vondra
On Thu, Mar 21, 2019 at 02:22:40AM +0100, Tomas Vondra wrote: ... OK, so here is a bit more polished version of the dlist-based patch. It's almost identical to what I posted before, except that it: 1) undoes the non-working optimization in DropRelationFiles() 2) removes

RE: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Tsunakawa, Takayuki
From: Masahiko Sawada [mailto:sawada.m...@gmail.com] > On Wed, Mar 27, 2019 at 2:30 AM Robert Haas wrote: > > > > On Tue, Mar 26, 2019 at 11:23 AM Masahiko Sawada > wrote: > > > > I don't see a patch with the naming updated, here or there, and I'm > > > > going to be really unhappy if we end up

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Ryan Lambert
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Overall I think this patch [1] improves the docs and help

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-03-26 Thread Haribabu Kommi
On Tue, Mar 26, 2019 at 9:08 PM Amit Kapila wrote: > On Mon, Mar 25, 2019 at 6:55 PM Haribabu Kommi > wrote: > > > > > > Thanks to everyone for their opinions and suggestions to improve. > > > > Without parallel workers, there aren't many internal implementation > > logic code that causes the

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Masahiko Sawada
On Wed, Mar 27, 2019 at 2:30 AM Robert Haas wrote: > > On Tue, Mar 26, 2019 at 11:23 AM Masahiko Sawada > wrote: > > > I don't see a patch with the naming updated, here or there, and I'm > > > going to be really unhappy if we end up with inconsistent naming > > > between two patches that do

Re: Improvement of installation document

2019-03-26 Thread Yugo Nagata
On Tue, 26 Mar 2019 22:18:53 +0900 (JST) Tatsuo Ishii wrote: > > One of our clients suggested that the installation document[1] lacks > > description > > about requriements of installing *-devel packages. For example, > > postgresqlxx-devel > > is required for using --with-pgsql, and

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-03-26 21:01:27 -0400, Stephen Frost wrote: > > I'm also not convinced that these changes to pg_basebackup will be free > > of issues that may impact users in a negative way, making me concerned > > that we're going to end up doing

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 21:01:27 -0400, Stephen Frost wrote: > I'm also not convinced that these changes to pg_basebackup will be free > of issues that may impact users in a negative way, making me concerned > that we're going to end up doing more harm than good with such a change > being back-patched.

Re: setLastTid() and currtid()

2019-03-26 Thread Inoue, Hiroshi
Hi Andres, Sorry for the late reply. On 2019/03/26 9:44, Andres Freund wrote: Hi, For the tableam work I'd like to remove heapam.h from nodeModifyTable.c. The only remaining impediment to that is a call to setLastTid(), which is defined in tid.c but declared in heapam.h. That doesn't seem

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-03-26 20:18:31 -0400, Stephen Frost wrote: > > > >>I thought Robert's response was generally good, pointing out that > > > >>we're talking about this being an issue if the corruption happens in a > > > >>certain set of bytes. That

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Mar 26, 2019 at 08:18:31PM -0400, Stephen Frost wrote: > >* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > >>On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: > >>>On 2019-03-26 19:22:03 -0400, Stephen

Re: MSVC Build support with visual studio 2019

2019-03-26 Thread Haribabu Kommi
On Wed, Mar 27, 2019 at 3:03 AM Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > > On 3/20/19 8:36 PM, Haribabu Kommi wrote: > > Hi Hackers, > > > > Here I attached a patch that supports building of PostgreSQL with VS > 2019. > > VS 2019 is going to release on Apr 2nd 2019, it will be

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 08:18:31PM -0400, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: >On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: >>* Andres Freund (and...@anarazel.de) wrote: >>> As

minimizing pg_stat_statements performance overhead

2019-03-26 Thread Raymond Martin
Hello hackers, This email is regarding the Postgres pg_stat_statements extension. I noticed that enabling pg_stat_statements can effect performance. I thought that changing the pg_stat_statements.track parameter to 'none' could reduce this overhead without requiring a restart to remove it from

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 20:18:31 -0400, Stephen Frost wrote: > > >>I thought Robert's response was generally good, pointing out that > > >>we're talking about this being an issue if the corruption happens in a > > >>certain set of bytes. That said, I'm happy to see improvements in > > >>this area but

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > Here a benchmark doing that using pgbench's script weight feature. Wow, I didn't know that pgbench has evolved to have such a convenient feature. Thanks for telling me how to utilize it in testing. PostgreSQL is cool! Regards

Re: basebackup checksum verification

2019-03-26 Thread Peter Geoghegan
On Tue, Mar 26, 2019 at 5:10 PM Tomas Vondra wrote: > Bogus might be a bit too harsh, but yeah - failure to reliably detect > obviously > invalid checksums when the LSN just happens to be high due to randomness is > not > a good thing. We'll still detect pages corrupted in other places, but

Re: speeding up planning with partitions

2019-03-26 Thread Amit Langote
On 2019/03/27 1:06, Tom Lane wrote: > Amit Langote writes: >> 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's >> created for partitioned parent table, as it's pointless. I was planning >> to propose it later, but decided to post it now if we're modifying the >> nearby

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-03-26 Thread Tsunakawa, Takayuki
From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] > My understanding of what David wrote is that the slowness of bloated hash > table is hard to notice, because planning itself is pretty slow. With the > "speeding up planning with partitions" patch, planning becomes quite fast, > so the

Re: Ordered Partitioned Table Scans

2019-03-26 Thread David Rowley
Thanks for having another look. On Wed, 27 Mar 2019 at 00:22, Julien Rouhaud wrote: > A few, mostly nitpicking, comments: > > + if (rel->part_scheme != NULL && IS_SIMPLE_REL(rel) && > + partitions_are_ordered(root, rel)) > > shouldn't the test be IS_PARTITIONED_REL(rel) instead of

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: > >On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: > >>* Andres Freund (and...@anarazel.de) wrote: > >>> As detailed in > >>>

Re: Pluggable Storage - Andres's take

2019-03-26 Thread Andres Freund
Hi, On 2019-02-22 14:52:08 -0500, Robert Haas wrote: > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > Thanks for the review. Attached v2. > > Thanks. I took this, combined it with Andres's > v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did > some polishing of

Re: basebackup checksum verification

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 04:49:21PM -0700, Andres Freund wrote: Hi, On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: Greetings, * Andres Freund (and...@anarazel.de) wrote: > As detailed in > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > the way the backend's

Re: basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 19:22:03 -0400, Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > As detailed in > > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > > the way the backend's basebackup checksum verification works makes its > > error

Re: partitioned tables referenced by FKs

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Alvaro Herrera wrote: > Thanks for the thorough testing and bug analysis! It was spot-on. I've > applied your two proposed fixes, as well as added a new test setup that > covers both these bugs. The attached set is rebased on 7c366ac969ce. Attached is rebased on 126d63122232.

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-26 Thread legrand legrand
here is a new version: - "track_planning" GUC added to permit to keep previous behavior unchanged - columns names have been changed / added: total_plan_time, total_exec_time, total_time - trailing whitespaces and comments wider than 80 characters not fixed

Re: basebackup checksum verification

2019-03-26 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > As detailed in > https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de > the way the backend's basebackup checksum verification works makes its > error detection capabilities very dubious. I disagree that it's 'very

Fwd: Gsoc proposal perffarn

2019-03-26 Thread Victor Kukshiev
Hello, my name is Victor Kuvshiev. Currently I'm third-year student of Petrozavodsk State University, studying information systems and technologies. I have relatively good knowledge of HTML, CSS and Python also have some skills in javascript language. example of my works: ruletka, console game

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Tom Lane
Ryan Lambert writes: > Is the xml-functions-type-docfix-4.patch [1] the one needing review? I'll > test applying it and review the changes in better detail. Is there a > section in the docs that shows how to verify if the updated pages render > properly? I would assume the pages are build when

Re: speeding up planning with partitions

2019-03-26 Thread Tom Lane
Amit Langote writes: > On 2019/03/23 6:07, Tom Lane wrote: >> I also feel like you used a dartboard while deciding where to insert the >> call in query_planner(); dropping it into the middle of a sequence of >> equivalence-class-related operations seems quite random. Maybe we could >> delay that

Re: Fix XML handling with DOCTYPE

2019-03-26 Thread Ryan Lambert
Ok, I'll give it a go. > If you happened to feel moved to look over a documentation patch, that > would be what this CF entry most needs in the waning days of the > commitfest. Is the xml-functions-type-docfix-4.patch [1] the one needing review? I'll test applying it and review the changes in

Re: partitioned tables referenced by FKs

2019-03-26 Thread Alvaro Herrera
Hello Amit On 2019-Mar-26, Amit Langote wrote: > + Oid objectClass = getObjectClass(thisobj); > > I guess you meant to use ObjectClass, not Oid here. Absolutely. > Tested 0002 a bit more and found some problems. Thanks for the thorough testing and bug

Re: Usage of epoch in txid_current

2019-03-26 Thread Thomas Munro
On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > ... I think you could probably reclaim that space by > using a more compact representation of vacuumFlags, overflowed, > delayChkpt, nxids (it's funny, the comment says "as tightly as > possible", which clearly isn't the case). Woops, I

Re: explain plans with information about (modified) gucs

2019-03-26 Thread Tomas Vondra
On Mon, Mar 18, 2019 at 11:31:48AM +0100, Rafia Sabih wrote: On Sun, 24 Feb 2019 at 00:06, Tomas Vondra wrote: Hi, attached is an updated patch, fixing and slightly tweaking the docs. Barring objections, I'll get this committed later next week. I was having a look at this patch, and this

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
Hi út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2019-03-20 03:51, Michael Paquier wrote: > > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: > >> postgres=# update foo set name = 'bbbxx' where id = 1; -- error > >> ERROR: no

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
út 26. 3. 2019 v 19:52 odesílatel Pavel Stehule napsal: > Hi > > út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> napsal: > >> On 2019-03-20 03:51, Michael Paquier wrote: >> > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: >> >> postgres=#

Re: patch to allow disable of WAL recycling

2019-03-26 Thread Jerry Jelinek
On Thu, Mar 7, 2019 at 6:26 PM Thomas Munro wrote: > On Fri, Mar 8, 2019 at 12:35 PM Jerry Jelinek > wrote: > > On Thu, Mar 7, 2019 at 3:09 PM Thomas Munro > wrote: > >> My understanding is that it's not really the COW-ness that makes it > >> not necessary, it's the fact that fdatasync()

Re: Enable data checksums by default

2019-03-26 Thread Peter Geoghegan
On Fri, Mar 22, 2019 at 9:07 AM Tom Lane wrote: > IMO, the main value of checksums is that they allow the Postgres > project to deflect blame. That's nice for us but I'm not sure > that it's a benefit for users. I've seen little if any data to > suggest that checksums actually catch enough

Re: [HACKERS] generated columns

2019-03-26 Thread Pavel Stehule
Hi út 26. 3. 2019 v 14:33 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2019-03-20 03:51, Michael Paquier wrote: > > On Mon, Mar 18, 2019 at 03:14:09PM +0100, Pavel Stehule wrote: > >> postgres=# update foo set name = 'bbbxx' where id = 1; -- error > >> ERROR: no

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Mar-26, Tom Lane wrote: >> Alvaro Herrera writes: >>> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 >> I've never cared for the ntstatus.h reference, but how stable is >> the URL you suggest going to

Re: [Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Michael Banck
Hi, Am Dienstag, den 26.03.2019, 10:30 -0700 schrieb Andres Freund: > On 2019-03-26 18:22:55 +0100, Michael Banck wrote: > > Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > > > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM > > > generate_series(1, 100) g(i); > > >

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Tom Lane wrote: > Alvaro Herrera writes: > >> 0xC028 is STATUS_BAD_STACK, per > >> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 > >> Not sure how credible/useful a stack trace is going to be. > > > BTW I think we

Re: [PATCH][HOTFIX] vacuum_cost_delay type change from int to real have not been done everywhere

2019-03-26 Thread Tom Lane
Nikolay Shaplov writes: > In caf626b2 type of vacuum_cost_delay have been switched from int to real, > everywhere, but not in *RelOpts[] arrays. Ugh. > For some reason it continued to build and work. I'm not quite sure why it worked either; apparently, the type of that array entry doesn't

Re: [Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Andres Freund
On 2019-03-26 18:22:55 +0100, Michael Banck wrote: > Hi, > > Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM generate_series(1, > > 100) g(i); > > SELECT pg_relation_size('corruptme'); > > postgres[22890][1]=# SELECT

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 11:23 AM Masahiko Sawada wrote: > > I don't see a patch with the naming updated, here or there, and I'm > > going to be really unhappy if we end up with inconsistent naming > > between two patches that do such fundamentally similar things. -1 > > from me to committing

[Patch] Base backups and random or zero pageheaders (was: Online verification of checksums)

2019-03-26 Thread Michael Banck
Hi, Am Dienstag, den 19.03.2019, 13:00 -0700 schrieb Andres Freund: > CREATE TABLE corruptme AS SELECT g.i::text AS data FROM generate_series(1, > 100) g(i); > SELECT pg_relation_size('corruptme'); > postgres[22890][1]=# SELECT current_setting('data_directory') || '/' || >

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Tom Lane
Alvaro Herrera writes: >> 0xC028 is STATUS_BAD_STACK, per >> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55 >> Not sure how credible/useful a stack trace is going to be. > BTW I think we should update our message to use this URL

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Andrew Dunstan
On 3/26/19 12:53 PM, Alvaro Herrera wrote: > On 2019-Mar-26, Alvaro Herrera wrote: > >>> 2019-03-26 00:49:02.208 EDT [5c99ae9e.20cc:6] LOG: server process (PID >>> 8368) was terminated by exception 0xC028 >> 0xC028 is STATUS_BAD_STACK, per >>

basebackup checksum verification

2019-03-26 Thread Andres Freund
Hi, As detailed in https://postgr.es/m/20190319200050.ncuxejradurjakdc%40alap3.anarazel.de the way the backend's basebackup checksum verification works makes its error detection capabilities very dubious. I think we need to fix this before the next set of backbranch releases, or at the very

Re: pgsql: Get rid of backtracking in jsonpath_scan.l

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Alvaro Herrera wrote: > > 2019-03-26 00:49:02.208 EDT [5c99ae9e.20cc:6] LOG: server process (PID > > 8368) was terminated by exception 0xC028 > > 0xC028 is STATUS_BAD_STACK, per >

Re: PostgreSQL Participates in GSoC 2019!

2019-03-26 Thread Ritom Sonowal
Hi, I have applied and submitted an initial draft of my proposal for GSoC 2019 through the Summer of Code site. The project is titled 'pgAdmin4 Graphing Query Tool'. I would like to get some feedback for the same so that I can improve on making the final proposal better. The link to the draft is

Re: Tid scan improvements

2019-03-26 Thread Andres Freund
Hi, On 2019-03-26 19:11:13 +1300, Edmund Horner wrote: > The changes in heapam.c were required for backward scan support, as > used by ORDER BY ctid DESC and MAX(ctid); and also for FETCH LAST and > FETCH PRIOR. I have removed the backward scans functionality from the > current set of patches,

[PATCH][HOTFIX] vacuum_cost_delay type change from int to real have not been done everywhere

2019-03-26 Thread Nikolay Shaplov
Hi! In caf626b2 type of vacuum_cost_delay have been switched from int to real, everywhere, but not in *RelOpts[] arrays. For some reason it continued to build and work. But I think it is better to move vacuum_cost_delay from int to real there too... Patch attached. PS. As you can see current

Re: speeding up planning with partitions

2019-03-26 Thread Tom Lane
Amit Langote writes: > 0002 is a new patch to get rid of the duplicate RTE and PlanRowMark that's > created for partitioned parent table, as it's pointless. I was planning > to propose it later, but decided to post it now if we're modifying the > nearby code anyway. Good idea, but it needs a

Re: MSVC Build support with visual studio 2019

2019-03-26 Thread Andrew Dunstan
On 3/20/19 8:36 PM, Haribabu Kommi wrote: > Hi Hackers, > > Here I attached a patch that supports building of PostgreSQL with VS 2019. > VS 2019 is going to release on Apr 2nd 2019, it will be good if version 12 > supports compiling. The attached for is for review, it may needs some > updates >

Re: Should the docs have a warning about pg_stat_reset()?

2019-03-26 Thread Euler Taveira
Em ter, 26 de mar de 2019 às 09:54, David Rowley escreveu: > > As I mentioned in [1], I've had a few cases recently about auto-vacuum > not working. On the other thread, it was all about auto-vacuum being > configured to run too slowly. The other culprit for auto-vacuum not > working is when

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Masahiko Sawada
On Tue, Mar 26, 2019 at 10:30 PM Robert Haas wrote: > > On Tue, Mar 26, 2019 at 3:57 AM Tsunakawa, Takayuki > wrote: > > From: David Steele [mailto:da...@pgmasters.net] > > > This patch appears to have been stalled for a while. > > > > > > Takayuki -- the ball appears to be in your court.

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 01:37:33PM +, Dean Rasheed wrote: On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: > > Attached is an updated patch... I just looked through the latest set of changes and I have a couple of additional review

Re: Enable data checksums by default

2019-03-26 Thread Christoph Berg
Re: Tom Lane 2019-03-22 <4368.1553270...@sss.pgh.pa.us> > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some proof, not just > an unfounded

Re: New vacuum option to do only freezing

2019-03-26 Thread Masahiko Sawada
On Sat, Mar 23, 2019 at 3:25 AM Robert Haas wrote: > > On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada wrote: > > IIUC we've discussed the field-and-value style vacuum option. I > > suggested that since we have already the disable_page_skipping option > > the disable_page_skipping option would

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 11:59:56AM +, Dean Rasheed wrote: On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: Attached is an updated patch, fixing all the issues pointed out so far. Unless there are some objections, I plan to commit the 0001 part by the end of this CF. Part 0002 is a matter

Re: jsonpath

2019-03-26 Thread Alexander Korotkov
On Tue, Mar 26, 2019 at 5:32 PM Tom Lane wrote: > Alexander Korotkov writes: > > Got access to that buildfarm animal thanks to Tom Turelinckx. Now > > running check-world in a loop on the same commit hash with same build > > options. Error wasn't triggered yet. > > I notice that snapper is

Re: libpq compression

2019-03-26 Thread Konstantin Knizhnik
Version of the patch correctly working when no compression algorithm are avaiable. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/configure b/configure index 8068108..8ebd961 100755 --- a/configure +++ b/configure @@ -701,6

Re: FETCH FIRST clause PERCENT option

2019-03-26 Thread Tomas Vondra
On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote: On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra wrote: To give you a (admittedly, somewhat contrived and artificial example): SELECT * FROM t1 WHERE id IN ( SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS

Re: psql display of foreign keys

2019-03-26 Thread Alvaro Herrera
On 2019-Mar-26, Peter Eisentraut wrote: > On 2019-03-26 03:42, Alvaro Herrera wrote: > > Patch tester didn't like that one bit. Here's v10 with the fixup > > applied. > > Looks good to me. Thanks! I ran "make installcheck-parallel" using this psql version on all supported branches plus 9.2.

Re: [HACKERS] Block level parallel vacuum

2019-03-26 Thread Masahiko Sawada
On Tue, Mar 26, 2019 at 10:19 AM Haribabu Kommi wrote: > > > On Fri, Mar 22, 2019 at 4:06 PM Masahiko Sawada wrote: >> >> >> Attached the updated version patch. 0001 patch allows all existing >> vacuum options an boolean argument. 0002 patch introduces parallel >> lazy vacuum. 0003 patch adds -P

Re: jsonpath

2019-03-26 Thread Tom Lane
Alexander Korotkov writes: > Got access to that buildfarm animal thanks to Tom Turelinckx. Now > running check-world in a loop on the same commit hash with same build > options. Error wasn't triggered yet. I notice that snapper is using force_parallel_mode = regress ... have you got that

Re: Shouldn't current_schema() be at least PARALLEL RESTRICTED?

2019-03-26 Thread Daniel Gustafsson
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Reviewing the codepath in question (as well as the commit that

Re: Misleading errors with column references in default expressions and partition bounds

2019-03-26 Thread Tom Lane
Michael Paquier writes: > One idea which came from Amit, and it seems to me that it is a good > idea, would be to have more context-related error messages directly in > transformColumnRef(), so as we can discard at an early stage column > references which are part of contexts where there is no

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: > > On Mon, 25 Mar 2019 at 23:36, Tomas Vondra > wrote: > > > > Attached is an updated patch... > > I just looked through the latest set of changes and I have a couple of > additional review comments: > I just spotted another issue while

Re: Special role for subscriptions

2019-03-26 Thread Robert Haas
On Thu, Mar 21, 2019 at 9:28 PM Michael Paquier wrote: > By the way, as the commit fest is coming to its end in a couple of > days, and that we are still discussing how the thing should be shaped, > I would recommend to mark the patch as returned with feedback. Any > objections with that? +1.

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 3:57 AM Tsunakawa, Takayuki wrote: > From: David Steele [mailto:da...@pgmasters.net] > > This patch appears to have been stalled for a while. > > > > Takayuki -- the ball appears to be in your court. Perhaps it would be > > helpful to summarize what you think are next

Re: Offline enabling/disabling of data checksums

2019-03-26 Thread Fabien COELHO
Bonjour Michaël, Here is an attempt at improving the Notes. [...] So, the ordering of the notes for each paragraph is as follows: 1) Replication issues when mixing different checksum setups across nodes. 2) Consistency of the operations if killed. 3) Don't start Postgres while the

Re: Refactoring the checkpointer's fsync request queue

2019-03-26 Thread Robert Haas
On Tue, Mar 26, 2019 at 12:20 AM Thomas Munro wrote: > I've been trying to decide if that is a problem. Maybe there is a > performance angle, and I'm also wondering if it might increase the > risk of missing a write-back error. Of course we'll find a proper > solution to that problem (perhaps

Re: Improvement of installation document

2019-03-26 Thread Tatsuo Ishii
> One of our clients suggested that the installation document[1] lacks > description > about requriements of installing *-devel packages. For example, > postgresqlxx-devel > is required for using --with-pgsql, and openssl-devel for --with-openssl, and > so on, > but these are not documented. >

Re: partitioned tables referenced by FKs

2019-03-26 Thread Jesper Pedersen
Hi Amit, On 3/26/19 2:06 AM, Amit Langote wrote: Wouldn't you get the same numbers on HEAD too? IOW, I'm not sure how the patch here, which seems mostly about getting DDL in order to support foreign keys on partitioned tables, would have affected the result of this benchmark. Can you clarify

Re: pg_upgrade: Pass -j down to vacuumdb

2019-03-26 Thread Tom Lane
Peter Eisentraut writes: > On 2019-03-25 22:57, Tom Lane wrote: >> +fprintf(script, "echo %sYou may wish to add --jobs=N for parallel >> analyzing.%s\n", >> +ECHO_QUOTE, ECHO_QUOTE); > But then you get that information after you have already started the script. Yes, but

Should the docs have a warning about pg_stat_reset()?

2019-03-26 Thread David Rowley
As I mentioned in [1], I've had a few cases recently about auto-vacuum not working. On the other thread, it was all about auto-vacuum being configured to run too slowly. The other culprit for auto-vacuum not working is when people periodically use pg_stat_reset(). The problem with

Re: Problems with plan estimates in postgres_fdw

2019-03-26 Thread Etsuro Fujita
(2019/03/20 20:47), Etsuro Fujita wrote: Attached is an updated version of the patch set. One thing I noticed while self-reviewing the patch for UPPERREL_FINAL is: the previous versions of the patch don't show EPQ plans in EXPLAIN, as shown in the below example, so we can't check if those

Re: warning to publication created and wal_level is not set to logical

2019-03-26 Thread Lucas Viecelli
>> One idea that might be useful is to have walsenders refuse to transmit > >> any logical-replication data if they see wal_level is too low. That > >> would get users' attention pretty quickly. > > > They do: > I checked this before creating the patch > > Oh, OK, then this seems like it's

Re: FETCH FIRST clause PERCENT option

2019-03-26 Thread Surafel Temesgen
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra wrote: > > To give you a (admittedly, somewhat contrived and artificial example): > > SELECT * FROM t1 WHERE id IN ( > SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY > ); > > Maybe this example is bogus and/or does not

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Mon, 25 Mar 2019 at 23:36, Tomas Vondra wrote: > > Attached is an updated patch, fixing all the issues pointed out so far. > Unless there are some objections, I plan to commit the 0001 part by the > end of this CF. Part 0002 is a matter for PG13, as previously agreed. > Yes, I think that's

Re: Improvement of installation document

2019-03-26 Thread Yugo Nagata
Hi, I apologize that I accidentally sent the following email to this list. Please disregard this. I am sorry for making a lot of noise. Regard, On Tue, 26 Mar 2019 20:38:31 +0900 Yugo Nagata wrote: > Hi, > > One of our clients suggested that the installation document[1] lacks > description

  1   2   >