Re: [HACKERS] dropping partitioned tables without CASCADE

2017-11-05 Thread Ashutosh Bapat
Somehow the earlier patches missed qualifying pg_get_expr() by pg_catalog. Fixed it along with annotating the partitioned partition as ", PARTITIONED". On Fri, Nov 3, 2017 at 6:09 PM, Alvaro Herrera wrote: >> >> Right now, we could do that if we order the list by bound expression; >> lexically DE

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-11-05 Thread Masahiko Sawada
On Sat, Nov 4, 2017 at 7:04 AM, Alexander Korotkov wrote: > On Wed, Nov 1, 2017 at 5:55 AM, Masahiko Sawada > wrote: >> >> On Tue, Oct 31, 2017 at 6:17 PM, Alexander Korotkov >> wrote: >> > On Tue, Oct 31, 2017 at 5:16 AM, Masahiko Sawada >> > wrote: >> >> >> >> On Mon, Oct 30, 2017 at 10:16 PM

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
On 6 November 2017 at 17:30, Amit Langote wrote: > On 2017/11/03 13:32, David Rowley wrote: >> On 31 October 2017 at 21:43, Amit Langote >> wrote: >> 1. This comment seem wrong. >> >> /* >> * Since the clauses in rel->baserestrictinfo should all contain Const >> * operands, it should be possible

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-05 Thread Amit Kapila
On Sun, Nov 5, 2017 at 6:54 AM, Andres Freund wrote > On 2017-11-05 01:05:59 +0100, Robert Haas wrote: >> skip-gather-project-v1.patch does what it says on the tin. I still >> don't have a test case for this, and I didn't find that it helped very >> much, I am also wondering in which case it can

Re: [HACKERS] Flexible configuration for full-text search

2017-11-05 Thread Thomas Munro
On Sat, Oct 21, 2017 at 1:39 AM, Aleksandr Parfenov wrote: > In attachment updated patch with fixes of empty XML tags in > documentation. Hi Aleksandr, I'm not sure if this is expected at this stage, but just in case you aren't aware, with this version of the patch the binary upgrade test in src

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-05 Thread Thomas Munro
On Fri, Oct 20, 2017 at 9:05 AM, Nico Williams wrote: > Rebased (there were conflicts in the SGML files). Hi Nico FYI that version has some stray absolute paths in constraints.source: -COPY COPY_TBL FROM '@abs_srcdir@/data/constro.data'; +COPY COPY_TBL FROM '/home/nico/ws/postgres/src/test/regr

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread Amit Langote
On 2017/11/06 12:53, David Rowley wrote: > On 3 November 2017 at 17:32, David Rowley > wrote: >> 2. This code is way more complex than it needs to be. >> >> if (num_parts > 0) >> { >> int j; >> >> all_indexes = (int *) palloc(num_parts * sizeof(int)); >> j = 0; >> if (min_part_idx >= 0 && max_par

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2017-11-05 Thread Thomas Munro
On Mon, Sep 11, 2017 at 7:43 PM, Yuto Hayamizu wrote: > Suppose that there are three qual clauses in a scan node, current > postgres estimates per-tuple cost of the filter to be: >cost(A) + cost(B) + cost(C) > > And basic idea of the attached patch is: >cost(A) + clauselist_selectivity({A}

[HACKERS] PATCH: psql tab completion for SELECT

2017-11-05 Thread Edmund Horner
Hi pgsql-hackers, Here's a little draft patch to add *some* tab completion ability for SELECT in psql. I have often missed the ability, especially with invocations of utility functions. It would be nice to be able to suggest column names from the relevant tables in the query, but, as the SQL lan

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-05 Thread Amit Kapila
On Sat, Nov 4, 2017 at 2:03 AM, Tom Lane wrote: > Michael Paquier writes: >> On Fri, Nov 3, 2017 at 1:10 AM, Amit Kapila wrote: >>> On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: I've marked the CF entry closed. However, I'm not sure if we're quite done with this thread. Weren't we

Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
On 31 October 2017 at 21:43, Amit Langote wrote: > Attached updated version of the patches match_clauses_to_partkey() needs to allow for the way quals on Bool columns are represented. create table pt (a bool not null) partition by list (a); create table pt_true partition of pt for values in('t')

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-11-05 Thread Connor Wolf
Never mind, it turns out the issue boiled down to me declaring the wrong prefixType in my config function. TL;DR - PEBKAC On Sun, Nov 5, 2017 at 1:09 AM, Connor Wolf wrote: > Ok, I've got everything compiling and it installs properly, but I'm > running into problems that I think are either a si

Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread David Rowley
On 3 November 2017 at 17:32, David Rowley wrote: > 2. This code is way more complex than it needs to be. > > if (num_parts > 0) > { > int j; > > all_indexes = (int *) palloc(num_parts * sizeof(int)); > j = 0; > if (min_part_idx >= 0 && max_part_idx >= 0) > { > for (i = min_part_idx; i <= max_part_

Re: [HACKERS] Early locking option to parallel backup

2017-11-05 Thread Tom Lane
Andres Freund writes: > Well, the current approach afaics requires #relations * 2 locks, whereas > acquiring them in every worker would scale that with the number of > workers. Yeah, that's gonna be a problem with this proposal. > IIUC the problem here is that even though a lock is already > hel

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-05 Thread Tom Lane
Robert Haas writes: > This looks like it's on the right track to me. I hope Tom will look > into it, but if he doesn't I may try to get it committed myself. I do plan to take a look at it during this CF. > +/* Set or update cheapest_total_path and related fields */ > +set_cheape

Re: [HACKERS] Restricting maximum keep segments by repslots

2017-11-05 Thread Craig Ringer
On 31 October 2017 at 17:43, Kyotaro HORIGUCHI wrote: > Hello, this is a rebased version. > > It gets a change of the meaning of monitoring value along with > rebasing. > > In previous version, the "live" column mysteriously predicts the > necessary segments will be kept or lost by the next checkp

Re: [HACKERS] Display number of heap accesses for index scans

2017-11-05 Thread Tom Lane
Peter Geoghegan writes: > Andres Freund wrote: >> The number of index lookups that failed to return anything can be a >> critical performance factor in OLTP workloads. Therefore it seems like >> it'd be a good idea to extend the explain analyze output to include that >> information. > I certain

Re: [HACKERS] Custom compression methods

2017-11-05 Thread Tom Lane
Robert Haas writes: > A basic problem here is that, as proposed, DROP COMPRESSION METHOD may > break your database irretrievably. If there's no data compressed > using the compression method you dropped, everything is cool - > otherwise everything is broken and there's no way to recover. The > o

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-11-05 Thread Amit Langote
On 2017/11/03 21:39, Alvaro Herrera wrote: > Ashutosh Bapat wrote: >> On Fri, Nov 3, 2017 at 1:42 PM, Alvaro Herrera >> wrote: > >>> I think adding "is partitioned" at end of line isn't good; looks like a >>> phrase but isn't translatable. Maybe add keyword PARTITIONED instead? >> >> In that ca

Re: [HACKERS] Secondary index access optimizations

2017-11-05 Thread Thomas Munro
On Fri, Sep 8, 2017 at 3:58 AM, Konstantin Knizhnik wrote: > Updated version of the patch is attached to this mail. > Also I added support of date type to operator_predicate_proof to be able to > imply (logdate <= '2017-03-31') from (logdate < '2017-04-01') . Hi Konstantin, Is there any reason w

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-05 Thread Amit Langote
On 2017/11/03 6:24, Tom Lane wrote: > Amit Langote writes: >> On 2017/09/26 16:30, Michael Paquier wrote: >>> Cool, let's switch it back to a ready for committer status then. > >> Sure, thanks. > > Pushed with some cosmetic adjustments --- mostly, making the comments more > explicit about why we

Re: [HACKERS] Restricting maximum keep segments by repslots

2017-11-05 Thread Thomas Munro
On Tue, Oct 31, 2017 at 10:43 PM, Kyotaro HORIGUCHI wrote: > Hello, this is a rebased version. Hello Horiguchi-san, I think the "ddl" test under contrib/test_decoding also needs to be updated because it looks at pg_replication_slots and doesn't expect your new columns. -- Thomas Munro http://w

Re: [HACKERS] Custom compression methods

2017-11-05 Thread Adam Brusselback
> If there's no data compressed > using the compression method you dropped, everything is cool - > otherwise everything is broken and there's no way to recover. > The only obvious alternative is to disallow DROP altogether (or make it > not really DROP). Wouldn't whatever was using the compression

Re: [HACKERS] Statement-level rollback

2017-11-05 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Simon Riggs > A backend-based solution is required for PL procedures and functions. > > We could put this as an option into PL/pgSQL, but it seems like it is > a function of the transaction manager

Re: [HACKERS] Early locking option to parallel backup

2017-11-05 Thread Andres Freund
On 2017-11-05 17:38:39 -0500, Robert Haas wrote: > On Sun, Nov 5, 2017 at 5:17 AM, Lucas wrote: > > The patch creates a "--lock-early" option which will make pg_dump to issue > > shared locks on all tables on the backup TOC on each parallel worker start. > > That way, the backup has a very small c

Re: [HACKERS] Early locking option to parallel backup

2017-11-05 Thread Robert Haas
On Sun, Nov 5, 2017 at 5:17 AM, Lucas wrote: > The patch creates a "--lock-early" option which will make pg_dump to issue > shared locks on all tables on the backup TOC on each parallel worker start. > That way, the backup has a very small chance of failing. When it does, > happen in the first few

Re: [HACKERS] Custom compression methods

2017-11-05 Thread Robert Haas
On Sun, Nov 5, 2017 at 2:22 PM, Oleg Bartunov wrote: >> IIRC there were some concerns about what happened with pg_upgrade, >> with consuming precious toast bits, and a few other things. > > yes, pg_upgrade may be a problem. A basic problem here is that, as proposed, DROP COMPRESSION METHOD may br

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-05 Thread Robert Haas
On Sun, Nov 5, 2017 at 12:57 AM, Amit Kapila wrote: > Thanks for the confirmation. Find rebased patch attached. This looks like it's on the right track to me. I hope Tom will look into it, but if he doesn't I may try to get it committed myself. -if (rel->reloptkind == RELOPT_BASEREL) -

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-05 Thread Andres Freund
Hi, On November 5, 2017 1:33:24 PM PST, Jim Van Fleet wrote: >Ran this change with hammerdb on a power 8 firestone > >with 2 socket, 20 core >9.6 base-- 451991 NOPM >0926_master -- 464385 NOPM >11_04master -- 449177 NOPM >11_04_patch -- 431423 NOPM >-- two socket patch is a little down

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-05 Thread Jim Van Fleet
Ran this change with hammerdb on a power 8 firestone with 2 socket, 20 core 9.6 base-- 451991 NOPM 0926_master -- 464385 NOPM 11_04master -- 449177 NOPM 11_04_patch -- 431423 NOPM -- two socket patch is a little down from previous base runs With one socket 9.6 base -- 393727 NO

Re: [HACKERS] Display number of heap accesses for index scans

2017-11-05 Thread Peter Geoghegan
Andres Freund wrote: The number of index lookups that failed to return anything can be a critical performance factor in OLTP workloads. Therefore it seems like it'd be a good idea to extend the explain analyze output to include that information. I certainly agree. I've sometimes wondered if

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-11-05 Thread Robert Haas
On Fri, Nov 3, 2017 at 5:57 PM, Alexander Korotkov wrote: > On Thu, Nov 2, 2017 at 5:56 AM, Robert Haas wrote: >> > I can propose following alternative approach: teach read-only queries on >> > hot >> > standby to tolerate concurrent relation truncation. Therefore, when >> > non-existent heap pa

Re: [HACKERS] Custom compression methods

2017-11-05 Thread Oleg Bartunov
On Thu, Nov 2, 2017 at 6:02 PM, Craig Ringer wrote: > On 2 November 2017 at 17:41, Ildus Kurbangaliev > wrote: > >> In this patch compression methods is suitable for MAIN and EXTENDED >> storages like in current implementation in postgres. Just instead only >> of LZ4 you can specify any other com

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-05 Thread Claudio Freire
On Sat, Nov 4, 2017 at 8:07 PM, Юрий Соколов wrote: > 2017-11-03 5:46 GMT+03:00 Tom Lane : >> >> Sokolov Yura writes: >> > [ 0001-Improve-compactify_tuples.patch, v5 or thereabouts ] >> >> I went to check the shellsort algorithm against Wikipedia's entry, >> and found that this appears to be an i

[HACKERS] Re: [COMMITTERS] pgsql: Account for catalog snapshot in PGXACT->xmin updates.

2017-11-05 Thread Noah Misch
On Sat, Nov 04, 2017 at 12:23:36PM -0400, Tom Lane wrote: > Noah Misch writes: > > I plan to use the attached patch after the minor release tags land. If > > there's significant support, I could instead push before the wrap. > > This looks fine to me --- I think you should push now. Done. --

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-11-05 Thread Pavel Stehule
Hi 2017-11-05 4:07 GMT+01:00 Noah Misch : > On Tue, Oct 17, 2017 at 06:06:40AM +0200, Pavel Stehule wrote: > > Please, if you can, try it write. I am little bit lost :) > > I'm attaching the patch I desired. Please review. This will probably miss > this week's minor releases. If there's signif

Re: [HACKERS] Exclude pg_internal.init from base backup

2017-11-05 Thread Magnus Hagander
On Sat, Nov 4, 2017 at 4:04 AM, Michael Paquier wrote: > On Fri, Nov 3, 2017 at 4:04 PM, Petr Jelinek > wrote: > > Not specific problem to this patch, but I wonder if it should be made > > more clear that those files (there are couple above of what you added) > > are skipped no matter which dire

Re: [HACKERS] Minor comment issue in receivelog.c

2017-11-05 Thread Magnus Hagander
On Thu, Nov 2, 2017 at 5:18 PM, Bernd Helmle wrote: > Please find a minor comment fix for receivelog.c, HandleCopyStream(). > > The comments talks about a START_STREAMING command, but i think > START_REPLICATION is what's meant here. > Yeah, it is. Confusingly enough, START_STREAMING is what's r

Re: [HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-05 Thread Paul Ramsey
On Sat, Nov 4, 2017 at 10:02 PM, Amit Kapila wrote: > On Sat, Nov 4, 2017 at 4:43 AM, Tom Lane wrote: > > Paul Ramsey writes: > >>> Whether I get a parallel aggregate seems entirely determined by the > number > >>> of rows, not the cost of preparing those rows. > > > >> This is true, as far as

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-05 Thread Robert Haas
On Sun, Nov 5, 2017 at 2:24 AM, Andres Freund wrote: >> shm-mq-reduce-receiver-latch-set-v1.patch causes the receiver to only >> consume input from the shared queue when the amount of unconsumed >> input exceeds 1/4 of the queue size. This caused a large performance >> improvement in my testing b

Re: [HACKERS] Early locking option to parallel backup

2017-11-05 Thread Michael Paquier
On Sun, Nov 5, 2017 at 7:17 PM, Lucas wrote: > The patch creates a "--lock-early" option which will make pg_dump to issue > shared locks on all tables on the backup TOC on each parallel worker start. > That way, the backup has a very small chance of failing. When it does, > happen in the first few

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-11-05 Thread Connor Wolf
Ok, I've got everything compiling and it installs properly, but I'm running into problems that I think are either a side-effect of implementing picksplit incorrectly (likely), or a bug in SP-GiST(?). Program received signal SIGSEGV, Segmentation fault. __memcpy_sse2_unaligned () at ../sysdeps/x86_