[HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-18 Thread Michael Paquier
Hi all, Per an offline report from Moshe Jacobson, it is possible to trigger easily cache lookup errors using pg_describe_object with invalid object IDs and pg_describe_object(). I had a closer look at things in this area, to notice that there are other user-facing failures as many things use the

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Masahiko Sawada
On Wed, Jul 19, 2017 at 2:41 PM, Fabien COELHO wrote: > >>> While reviewing Corey's \if patch, I complained that there was some >>> amount >>> of copy-paste in "psql/command.c". >>> >>> Here is an attempt at merging some functions which removes 160 lines of >>> code. >> >> >> Thank you for the pat

Re: [HACKERS] Pluggable storage

2017-07-18 Thread Haribabu Kommi
On Sat, Jul 15, 2017 at 12:30 PM, Robert Haas wrote: > On Fri, Jul 14, 2017 at 8:35 AM, Haribabu Kommi > wrote: > > To replace tuple with slot, I took trigger and SPI calls as the first > step > > in modifying > > from tuple to slot, Here I attached a WIP patch. The notable changes are, > > > >

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-07-18 Thread Mark Dilger
> On Jul 18, 2017, at 9:13 PM, Mark Dilger wrote: > >> >> On Jul 17, 2017, at 2:34 PM, Mark Dilger wrote: >> >>> >>> On Jul 17, 2017, at 2:14 PM, Tom Lane wrote: >>> >>> Mark Dilger writes: > On Jul 17, 2017, at 12:54 PM, Mark Dilger wrote: On Jul 15, 2017, at 3:00 PM, Tom Lane

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Fabien COELHO
While reviewing Corey's \if patch, I complained that there was some amount of copy-paste in "psql/command.c". Here is an attempt at merging some functions which removes 160 lines of code. Thank you for the patch. Is this an item for PG11? Yes, as it is submitted to CF 2017-09. -- Fabien.

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Yugo Nagata
On Tue, 18 Jul 2017 10:10:49 -0400 Tom Lane wrote: Thank you for your comments. I understand the problem of my proposal patch. > Andres Freund writes: > > On 2017-07-18 09:42:31 -0400, Tom Lane wrote: > >> I wonder if a better answer wouldn't be to reduce PGSTAT_STAT_INTERVAL. > > > Not sure i

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-07-18 Thread Mark Dilger
> On Jul 17, 2017, at 2:34 PM, Mark Dilger wrote: > >> >> On Jul 17, 2017, at 2:14 PM, Tom Lane wrote: >> >> Mark Dilger writes: On Jul 17, 2017, at 12:54 PM, Mark Dilger wrote: >>> On Jul 15, 2017, at 3:00 PM, Tom Lane wrote: > The types abstime, reltime, and tinterval need to go

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-18 Thread Craig Ringer
On 19 July 2017 at 07:26, Tom Lane wrote: > Justin Pryzby writes: > > I've seen this before while doing SET STATISTICS on a larger number of > columns > > using xargs, but just came up while doing ADD of a large number of > columns. > > Seems to be roughly linear in number of children but superl

[HACKERS] Re: Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-18 Thread Amit Langote
On 2017/07/18 16:20, Etsuro Fujita wrote: > On 2017/07/18 11:03, Robert Haas wrote: >> On Mon, Jul 10, 2017 at 5:44 PM, Robert Haas wrote: >>> The posted patches look OK to me. Barring developments, I will commit >>> them on 2017-07-17, or send another update by then. >> >> Committed them. > > T

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Masahiko Sawada
On Sat, Apr 1, 2017 at 3:04 AM, Fabien COELHO wrote: > > Hello, > > While reviewing Corey's \if patch, I complained that there was some amount > of copy-paste in "psql/command.c". > > Here is an attempt at merging some functions which removes 160 lines of > code. > Thank you for the patch. Is thi

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-18 Thread Tom Lane
Justin Pryzby writes: > I've seen this before while doing SET STATISTICS on a larger number of columns > using xargs, but just came up while doing ADD of a large number of columns. > Seems to be roughly linear in number of children but superlinear WRT columns. > I think having to do with catalog u

Re: [HACKERS] More flexible LDAP auth search filters?

2017-07-18 Thread Mark Cave-Ayland
On 17/07/17 18:08, Magnus Hagander wrote: > On Mon, Jul 17, 2017 at 6:47 PM, Mark Cave-Ayland > mailto:mark.cave-ayl...@ilande.co.uk>> > wrote: > Great to hear from you! It has definitely been a while... > > Indeed. You should spend more time on these lists :P Well I do get the emails, unfo

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alvaro Herrera
Alexander Korotkov wrote: > The problem is that you need to have not only opclass entries for the > operators, but also operators themselves. I.e. separate operators for > int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, int4[] @>> numeric. You > tried to add multiple pg_amop rows for single o

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alvaro Herrera
Mark Rofail wrote: > On Tue, 18 Jul 2017 at 7:43 pm, Alexander Korotkov > wrote: > > > separate operators for int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, > > int4[] @>> numeric. > > > > My only comment on the separate operators is its high maintenance. Any new > datatype introduced a co

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Tom Lane
Andres Freund writes: > ... If we were to go this route we'd have to probably move > the callback assignment into the ExecInit* routines, and possibly > replace the switch in ExecInitNode() with another callback, assigned in > make_*, and implement callbacks for ExecShutdown, MultiExecProcNode et

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Tom Lane
Andres Freund writes: > On 2017-07-18 15:45:53 -0400, Tom Lane wrote: >> While I'm uncomfortable with making such a change post-beta2, I'm one >> heck of a lot *more* uncomfortable with shipping v10 with no stack depth >> checks in the executor mainline. Fleshing this out and putting it in >> v10

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Andres Freund
On 2017-07-18 15:45:53 -0400, Tom Lane wrote: > Andres Freund writes: > > Attached is a trivial patch implementing 1) and a proof-of-concept hack > > for 2). > > The comment you made previously, as well as the proposed commit message > for 0001, suggest that you've forgotten that pre-v10 execQual

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Tom Lane
Andres Freund writes: > Attached is a trivial patch implementing 1) and a proof-of-concept hack > for 2). The comment you made previously, as well as the proposed commit message for 0001, suggest that you've forgotten that pre-v10 execQual.c had several check_stack_depth() calls. Per its header

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Julien Rouhaud
On 18/07/2017 14:04, Andres Freund wrote: > On 2017-07-17 23:04:43 +0200, Julien Rouhaud wrote: >> Is it v11 material or is there any chance to make it in v10? > > I think it'd make sense to apply the first to v10 (and earlier), and the > second to v11. I think this isn't a terribly risky patch,

Re: [HACKERS] More race conditions in logical replication

2017-07-18 Thread Alvaro Herrera
Alvaro Herrera wrote: > After going over this a bunch more times, I found other problems. For > example, I noticed that if I create a temporary slot in one session, > then another session would rightly go to sleep if it tries to drop that > slot. But the slot goes away when the first session disc

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Mark Rofail
On Tue, 18 Jul 2017 at 7:43 pm, Alexander Korotkov wrote: > separate operators for int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, > int4[] @>> numeric. > My only comment on the separate operators is its high maintenance. Any new datatype introduced a corresponding operator should be create

Re: [HACKERS] JSONB - JSONB operator feature request

2017-07-18 Thread David Fetter
On Tue, Jul 18, 2017 at 01:36:32PM +0200, david.tu...@linuxbox.cz wrote: > Hi, > > some users and me used hstore - hstore for example storing only changed > rows in trigger like: > > hsore(NEW) - hstore(OLD) > > There isn't same operator/function in JSON/JSONB. We can only remove keys > from J

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Mark Rofail
On Tue, 18 Jul 2017 at 7:43 pm, Alexander Korotkov wrote: > On T upue, Jul 18, 2017 at 2:24 AM, Mark Rofail > wrote: > >> On Wed, Jul 12, 2017 at 12:53 AM, Alvaro Herrera < >> alvhe...@2ndquadrant.com> wrote: >>> >>> We have one opclass for each type combination -- int4 to int2, int4 to >>> int4

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alexander Korotkov
On Tue, Jul 18, 2017 at 2:24 AM, Mark Rofail wrote: > On Wed, Jul 12, 2017 at 12:53 AM, Alvaro Herrera > wrote: >> >> We have one opclass for each type combination -- int4 to int2, int4 to >> int4, int4 to int8, etc. You just need to add the new strategy to all >> the opclasses. > > > I tried

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-07-18 Thread Sokolov Yura
On 2017-06-05 16:22, Sokolov Yura wrote: Good day, everyone. This patch improves performance of contended LWLock. Patch makes lock acquiring in single CAS loop: 1. LWLock->state is read, and ability for lock acquiring is detected. If there is possibility to take a lock, CAS tried. If CAS wer

Re: [HACKERS] JSONB - JSONB operator feature request

2017-07-18 Thread David Fetter
On Tue, Jul 18, 2017 at 01:36:32PM +0200, david.tu...@linuxbox.cz wrote: > Hi, > > some users and me used hstore - hstore for example storing only changed > rows in trigger like: > > hstore(NEW) - hstore(OLD) > > There isn't same operator/function in JSON/JSONB. We can only remove keys > from

[HACKERS] GSoC 2017: weekly progress reports (week 7)

2017-07-18 Thread Shubham Barai
Project: Explicitly support predicate locks in index AMs besides b-tree Hi, During this week, I read documentation and source code of BRIN index to understand its implementation. But later I found out that it is unlikely to implement page level or tuple level predicate locking in BRIN index. In t

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Tom Lane
Andres Freund writes: > On 2017-07-18 09:42:31 -0400, Tom Lane wrote: >> I wonder if a better answer wouldn't be to reduce PGSTAT_STAT_INTERVAL. > Not sure if that really does that much to solve the concern. Well, it reduces the amount of data churn that a statement shorter than PGSTAT_STAT_INTE

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2017-07-18 Thread David Fetter
On Tue, Jul 18, 2017 at 08:38:25AM +0200, Michael Paquier wrote: > On Tue, Jul 18, 2017 at 3:45 AM, David Fetter wrote: > > The one I run into frequently is in a proprietary fork, RDS Postgres. > > It'll happily dump out COMMENT ON EXTENSION plpgsq IS ... > > which is great as far as it goes, but

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Andres Freund
Hi, On 2017-07-18 09:42:31 -0400, Tom Lane wrote: > I wonder if a better answer wouldn't be to reduce PGSTAT_STAT_INTERVAL. > I don't think that value has been reconsidered since the code was written, > circa turn of the century. Maybe even make it configurable, though that > could be overkill.

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Tom Lane
Andres Freund writes: > That seems like it'd add a good number of new wakeups, or at least > scheduling of wakeups. Yes, as it stands this will result in a huge increase in alarm-scheduling kernel call traffic. I understand the issue but I do not think this is an acceptable path to a fix. > Or

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Andres Freund
Hi, On 2017-07-18 21:49:27 +0900, Yugo Nagata wrote: > After a DML is perfomed, the statistics is sent to pgstat by > pgsent_report_sent(). > However, the mininum time between stas file update is PGSTAT_STAT_INTERVAL, > so if > a few DMLs are performed with short interval, some statistics could

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

2017-07-18 Thread K S, Sandhya (Nokia - IN/Bangalore)
Hi Craig, While testing for another scenario of continuous postgres server restart, we got many cores of sh-QUIT and along with that we got cores for rm-QUIT. It is pointing to rm of the archive file but we were not able to get the bt as the stack is corrupted. We got below info from gdb: Core

[HACKERS] JSONB - JSONB operator feature request

2017-07-18 Thread david . turon
Hi, some users and me used hstore - hstore for example storing only changed rows in trigger like: hsore(NEW) - hstore(OLD) There isn't same operator/function in JSON/JSONB. We can only remove keys from JSONB, but not equal key-value pairs. Is there any chance to have same feature with JSON/JS

[HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-18 Thread Yugo Nagata
Hi, After a DML is perfomed, the statistics is sent to pgstat by pgsent_report_sent(). However, the mininum time between stas file update is PGSTAT_STAT_INTERVAL, so if a few DMLs are performed with short interval, some statistics could not be sent until the backend is shutdown. This is not a p

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Fabien COELHO
Hello Victor, While reviewing Corey's \if patch, I complained that there was some amount of copy-paste in "psql/command.c". Here is an attempt at merging some functions which removes 160 lines of code. I was looking through your patch. It seems good, the of the functions was very similar.

[HACKERS] [PATCH] Add citext_pattern_ops to citext contrib module

2017-07-18 Thread Alexey Chernyshov
Hi all, The attached patch introduces citext_pattern_ops for citext extension type like text_pattern_ops for text type. Here are operators ~<~, ~<=~, ~>~, ~>=~ combined into citext_pattern_ops operator class. These operators simply compare underlying citext values as C strings with memcmp() f

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Victor Drobny
On 2017-03-31 21:04, Fabien COELHO wrote: Hello, While reviewing Corey's \if patch, I complained that there was some amount of copy-paste in "psql/command.c". Here is an attempt at merging some functions which removes 160 lines of code. Hello, I was looking through your patch. It seems good

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-18 Thread Andres Freund
On 2017-07-17 23:04:43 +0200, Julien Rouhaud wrote: > On 17/07/2017 16:57, Andres Freund wrote: > > The latter obviously isn't ready, but might make clearer what I'm > > thinking about. > > It did for me, and FWIW I like this approach. Cool. > > If we were to go this route we'd have to probably

Re: [HACKERS] bug in locking an update tuple chain

2017-07-18 Thread Amit Kapila
On Sat, Jul 15, 2017 at 2:30 AM, Alvaro Herrera wrote: > A customer of ours reported a problem in 9.3.14 while inserting tuples > in a table with a foreign key, with many concurrent transactions doing > the same: after a few insertions worked sucessfully, a later one would > return failure indicat

Re: [HACKERS] Proposal for CSN based snapshots

2017-07-18 Thread Michael Paquier
On Wed, Jun 21, 2017 at 1:48 PM, Alexander Kuzmenkov wrote: > Glad to see you working on this! I've been studying this topic too. Attached > you can find a recently rebased version of Heikki's v4 patch. > I also fixed a bug that appeared on report-receipts isolation test: > XidIsConcurrent should

[HACKERS] Increase Vacuum ring buffer.

2017-07-18 Thread Sokolov Yura
Good day, every one. I investigated autovacuum performance, and found that it suffers a lot from small ring buffer. It suffers in a same way bulk writer suffered before Tom Lane's commit 6382448cf96: Tom Lane 2009-06-23 00:04:28 For bulk write operations (eg COPY IN), use a ring buffer of 16M

Re: [HACKERS] PgFDW connection invalidation by ALTER SERVER/ALTER USER MAPPING

2017-07-18 Thread Kyotaro HORIGUCHI
Thank you for the comments. At Mon, 17 Jul 2017 16:09:04 -0400, Tom Lane wrote in <9897.1500322...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > This is the revased and revised version of the previous patch. > > A few more comments: > > * Per the spec for CacheRegisterSyscacheCallback, a zer

[HACKERS] Re: Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-18 Thread Etsuro Fujita
On 2017/07/18 11:03, Robert Haas wrote: On Mon, Jul 10, 2017 at 5:44 PM, Robert Haas wrote: The posted patches look OK to me. Barring developments, I will commit them on 2017-07-17, or send another update by then. Committed them. Thank you! Best regards, Etsuro Fujita -- Sent via pgsql

Re: [HACKERS] asynchronous execution

2017-07-18 Thread Kyotaro HORIGUCHI
Hello, At Tue, 11 Jul 2017 10:28:51 +0200, Antonin Houska wrote in <6448.1499761731@localhost> > Kyotaro HORIGUCHI wrote: > > Effectively it is a waiting-queue followed by a > > completed-list. The point of the compaction is keeping the order > > of waiting or not-yet-completed requests, which