Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread David Rowley
On 16 July 2018 at 16:56, Phil Florent wrote: > I should post that in the general section but I am confused by the > sentence "A parent partition is always going to have a lower relid than > its children" > It's a little confusing since RelOptInfo has a relid field and so does RangeTblEntry.

RE: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread Phil Florent
Hi, I should post that in the general section but I am confused by the sentence "A parent partition is always going to have a lower relid than its children" In the code, relid has many meanings and not only "oid of a class" as in

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread David Rowley
On 16 July 2018 at 12:55, David Rowley wrote: > Thinking about this some more, I don't quite see any reason that the > partitioned_rels for a single hierarchy couldn't just be a Bitmapset > instead of an IntList. Of course, this is not possible since we can't pass a List of Bitmapsets to the

RE: How to make partitioning scale better for larger numbers of partitions

2018-07-15 Thread Tsunakawa, Takayuki
From: Kato, Sho [mailto:kato-...@jp.fujitsu.com] > I did pgbench -M prepared and perf record. > > UPDATE latency in prepared mode is 95% shorter than in simple mode. > SELECT latency in prepared mode is 54% shorter than in simple mode. > INSERT latency in prepared mode is 8% shorter than in

Re: patch to allow disable of WAL recycling

2018-07-15 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > I think that the right basic idea is to have a GUC that chooses between > the two implementations, but whether it can be set automatically is not > clear to me. Can initdb perhaps investigate what kind of filesystem the > WAL directory is

Refactor documentation for wait events (Was: pgsql: Add wait event for fsync of WAL segments)

2018-07-15 Thread Michael Paquier
On Fri, Jul 13, 2018 at 04:57:59PM -0500, Robert Haas wrote: > On Mon, Jul 9, 2018 at 4:41 PM, Michael Paquier wrote: >> Another idea that I have here, is to rework the page for monitoring >> stats so as we create one sub-section for each system view, and also one >> for the table of wait events.

Make foo=null a warning by default.

2018-07-15 Thread David Fetter
Folks, Per a discussion with Andrew Gierth and Vik Fearing, both of whom helped make this happen, please find attached a patch which makes it possible to get SQL standard behavior for "= NULL", which is an error. It's been upgraded to a warning, and can still be downgraded to silence (off) and

Re: why partition pruning doesn't work?

2018-07-15 Thread Robert Haas
On Sun, Jul 15, 2018 at 8:24 PM, David Rowley wrote: >> +1. In fact, maybe we ought to go a little further and have a >> relation_reopen(oid, mode) that verifies that a lock in the specified >> mode is held. > > Wouldn't it be better to just store the Relation indexed by its relid > somewhere

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread David Rowley
On 16 July 2018 at 06:55, Tom Lane wrote: > Also, in the Plan > representation, I'd suggest avoiding situations where a data structure > is sometimes a List of Lists of PartitionedRelPruneInfo and sometimes > just a single-level List. Saving one ListCell isn't worth the code > complexity and

Re: patch to allow disable of WAL recycling

2018-07-15 Thread Tom Lane
Robert Haas writes: > On Thu, Jul 5, 2018 at 4:39 PM, Andres Freund wrote: >> This is formulated *WAY* too positive. It'll have dramatic *NEGATIVE* >> performance impact of non COW filesystems, and very likely even negative >> impacts in a number of COWed scenarios (when there's enough memory to

Re: why partition pruning doesn't work?

2018-07-15 Thread Tom Lane
David Rowley writes: > On 16 July 2018 at 12:12, Robert Haas wrote: >> On Sun, Jul 15, 2018 at 1:02 PM, Tom Lane wrote: >>> What we'd be better off doing, if we go this route, is to install an >>> assertion-build-only test that verifies during relation_open(NoLock) >>> that some kind of lock is

Re: patch to allow disable of WAL recycling

2018-07-15 Thread Robert Haas
On Thu, Jul 5, 2018 at 4:39 PM, Andres Freund wrote: > This is formulated *WAY* too positive. It'll have dramatic *NEGATIVE* > performance impact of non COW filesystems, and very likely even negative > impacts in a number of COWed scenarios (when there's enough memory to > keep all WAL files in

Re: why partition pruning doesn't work?

2018-07-15 Thread David Rowley
On 16 July 2018 at 12:12, Robert Haas wrote: > On Sun, Jul 15, 2018 at 1:02 PM, Tom Lane wrote: >> What we'd be better off doing, if we go this route, is to install an >> assertion-build-only test that verifies during relation_open(NoLock) >> that some kind of lock is already held on the rel.

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread David Rowley
On 16 July 2018 at 06:55, Tom Lane wrote: > First off, given that we're now going to have a Plan data structure > that accurately reflects the partition hierarchy, I wonder whether we > couldn't get rid of the fiddling around with lists of ints and lists of > lists of ints; aren't those basically

Re: why partition pruning doesn't work?

2018-07-15 Thread Robert Haas
On Sun, Jul 15, 2018 at 1:02 PM, Tom Lane wrote: > What we'd be better off doing, if we go this route, is to install an > assertion-build-only test that verifies during relation_open(NoLock) > that some kind of lock is already held on the rel. That would protect > not only the executor, but a

Re: ENOSPC FailedAssertion("!(RefCountErrors == 0)"

2018-07-15 Thread Tom Lane
Michael Paquier writes: > On Wed, Jun 27, 2018 at 06:39:39PM -0500, Justin Pryzby wrote: >> < 2018-06-17 11:38:45.468 CDT >FATAL: could not extend file >> "base/17379/38665798": No space left on device >> < 2018-06-17 11:38:45.468 CDT >HINT: Check free disk space. >> < 2018-06-17

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-07-15 Thread Tomas Vondra
On 07/15/2018 04:43 PM, Dean Rasheed wrote: > On 15 July 2018 at 14:29, Tomas Vondra wrote: >> It's quite unclear to me how this algorithm could reliably end up with >> hist_sel=0 (in cases where we already don't end up with that). I mean, >> if a bucket matches the conditions, then the only

Re: Usage of epoch in txid_current

2018-07-15 Thread Tom Lane
Andres Freund writes: > On 2018-07-09 19:56:25 -0400, Tom Lane wrote: >> Or, perhaps, use a struct in assert builds and int64 otherwise? >> You could hide the ensuing notational differences in macros. > That should be doable. But I'd like to check if it's necessary > first. Optimizing passing an

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-07-15 Thread Tom Lane
David Rowley writes: > Anyway... Patch attached. This is method 3 of the 3 methods I thought > to fix this, so if this is not suitable then I'm out of ideas. I started to look at this patch. I think this is basically the right direction to go in, but I'm not terribly happy with the details of

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-07-15 Thread Dmitry Dolgov
> On Thu, 28 Jun 2018 at 07:54, Amit Langote > wrote: > > On 2018/06/27 22:21, Ashutosh Bapat wrote: > > On Wed, Jun 27, 2018 at 12:28 PM, Amit Langote > >> Ah, okay. I thought of reporting this because I felt the errors may have > >> to do with changes to the related code in HEAD between May

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-07-15 Thread Dean Rasheed
On 15 July 2018 at 14:29, Tomas Vondra wrote: > It's quite unclear to me how this algorithm could reliably end up with > hist_sel=0 (in cases where we already don't end up with that). I mean, > if a bucket matches the conditions, then the only way to eliminate is by > deducing that MCV already

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-07-15 Thread David Rowley
On 4 March 2018 at 21:43, David Rowley wrote: > Please find attached two patches. The first of which is intended to > resolve the issue mentioned above with consideration that it may need > to be back-patched to where LEFT JOIN removals where introduced. > > Patch two is intended to implement

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-07-15 Thread Tomas Vondra
On 07/15/2018 11:36 AM, Dean Rasheed wrote: > ... > > What I'm considering is an algorithm where we simultaneously compute 3 things: > > simple_sel - The result we would get without multivariate stats (*) > mcv_sel - The multivariate MCV result > hist_sel - The multivariate histogram result >

Re: Allow to specify a index name as ANALYZE parameter

2018-07-15 Thread Vik Fearing
On 11/07/18 11:04, Yugo Nagata wrote: > A usecase I suppose is that when a new expression index is created and that > we need only the statistics for the new index. I wonder if this shouldn't just be done automatically. -- Vik Fearing +33 6 46 75 15 36

Re: [HACKERS] [PATCH] kNN for SP-GiST

2018-07-15 Thread Andrey Borodin
Hi! > 14 июля 2018 г., в 1:31, Nikita Glukhov написал(а): > > Attached 6th version of the patches. > ... >> 2. The patch leaves contribs intact. Do extensions with sp-gist opclasses >> need to update it's behavior somehow to be used as-is? Or to support new >> functionality? > > There are no

Re: YA race condition in 001_stream_rep.pl (was Re: pgsql: Allow using the updated tuple while moving it to a different par)

2018-07-15 Thread Amit Kapila
On Sat, Jul 14, 2018 at 11:46 PM, Tom Lane wrote: > I wrote: > > I can reproduce the failure pretty reliably with a hack like the one > attached, which makes it unlikely that the walreceiver will send a > feedback message instantly when it gets the SIGHUP. > > So the issue boils down to this: the

Re: New GUC to sample log queries

2018-07-15 Thread Vik Fearing
On 10/07/18 20:34, Adrien Nayrat wrote: > On 06/27/2018 11:13 PM, Adrien Nayrat wrote: >>> 3) Is it intentional to only sample with log_min_duration_statement and >>> not also with log_duration? It seems like it should affect both. In >>> both cases, the name is too generic. Something called

Re: make installcheck-world in a clean environment

2018-07-15 Thread Alexander Lakhin
Hello Peter, 14.07.2018 13:57, Peter Eisentraut wrote: > On 06.07.18 09:45, Alexander Lakhin wrote: >> ./configure --enable-tap-tests >> make install >> make install -C contrib >> chown -R postgres:postgres /usr/local/pgsql/ >> /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data >>

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-07-15 Thread Dean Rasheed
On 13 July 2018 at 18:27, Tomas Vondra wrote: > I'm not so sure. The issue is that a lot of the MCV deductions depends > on whether we can answer questions like "Is there a single match?" or > "If we got a match in MCV, do we need to look at the non-MCV part?" This > is not very different from

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-07-15 Thread David Rowley
On 13 July 2018 at 20:20, Amit Langote wrote: > Why don't we abandon the notion altogether that > ExecSetupPartitionTupleRouting *has to* process the whole partition tree? [...] > I implemented that idea in the attached patch, which applies on top of > your 0001 patch, but I'd say it's too big