Re: Enhance pg_stat_wal_receiver view to display connected host

2018-03-27 Thread Haribabu Kommi
On Wed, Mar 28, 2018 at 12:54 PM, Michael Paquier wrote: > On Wed, Mar 28, 2018 at 11:28:32AM +1100, Haribabu Kommi wrote: > > I updated the pg_stat_wal_receiver patch with the new PQhost() function > > behavior and updated the view with two columns, (remote_server and > >

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Pavan Deolasee
On Wed, Mar 28, 2018 at 8:28 AM, Peter Geoghegan wrote: > On Tue, Mar 27, 2018 at 2:28 AM, Pavan Deolasee > wrote: > > (Version 26) > > I have some feedback on this version: > > * ExecMergeMatched() needs to determine tuple lock mode for > EvalPlanQual()

Re: PostgreSQL crashes with SIGSEGV

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 2:23 AM, Kyotaro HORIGUCHI wrote: >> > If no one objects, I'll mark this as Ready for Commit in a couple >> > of days. >> >> Thank you for the review, Horiguchi-san. It's hard to decide how >> important each goal is when coming up with a

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Pavan Deolasee
On Wed, Mar 28, 2018 at 7:36 AM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 02:02:10PM -0400, Tom Lane wrote: > > > > > Well, the point of checkpoints is that WAL data before the last one > > should no longer matter anymore, isn't it? > > I have to agree with Tom here.

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-03-27 Thread Tom Lane
Craig Ringer writes: > TL;DR: Pg should PANIC on fsync() EIO return. Surely you jest. > Retrying fsync() is not OK at > least on Linux. When fsync() returns success it means "all writes since the > last fsync have hit disk" but we assume it means "all writes since the

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Andrew Dunstan
On Wed, Mar 28, 2018 at 12:57 PM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 10:09:59PM -0400, Robert Haas wrote: >>> I have to agree with Tom here. If you force pg_rewind to replay more >>> WAL records from a checkpoint older than the checkpoint prior to where >>> WAL

Re: Parallel Aggregates for string_agg and array_agg

2018-03-27 Thread Tom Lane
Tomas Vondra writes: > On 03/27/2018 04:51 AM, David Rowley wrote: >> Seems I didn't mean "trans types". I should have said aggregate >> function argument types. > I'm not sure that's better than the check proposed by Tom. An argument > type without send/receive

Re: [HACKERS] [PATCH] Lockable views

2018-03-27 Thread Yugo Nagata
On Tue, 27 Mar 2018 23:28:04 +0900 Yugo Nagata wrote: I found the previous patch was broken and this can't handle views that has subqueries as bellow;  CREATE VIEW lock_view6 AS SELECT * from (select * from lock_tbl1) sub; I fixed this and attached the updated version

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 2:28 AM, Pavan Deolasee wrote: > (Version 26) I have some feedback on this version: * ExecMergeMatched() needs to determine tuple lock mode for EvalPlanQual() in a way that's based on how everything else works; it's not okay to just use

Re: Cast jsonb to numeric, int, float, bool

2018-03-27 Thread Michael Paquier
On Mon, Mar 12, 2018 at 12:43:20PM -0400, Tom Lane wrote: > Another fundamental problem is that implicit casts mask mistakes. > If there's an implicit cast to numeric, that applies everywhere not > only where it was what you meant. For some context on this you might > go back to the archives

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Michael Paquier
On Tue, Mar 27, 2018 at 10:09:59PM -0400, Robert Haas wrote: >> I have to agree with Tom here. If you force pg_rewind to replay more >> WAL records from a checkpoint older than the checkpoint prior to where >> WAL has forked at promotion then you have a risk of losing data. > > Oh! I see now.

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-27 Thread Michael Paquier
On Wed, Mar 28, 2018 at 04:45:49AM +0900, Fujii Masao wrote: > The patch looks good to me! Barring objection, I will commit it > and back-patch to 9.5 where pg_rewind was added. Thanks in advance! I just eyeballed the patch again and I don't see issues with what's used here. The thing should

PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-03-27 Thread Craig Ringer
Hi all Some time ago I ran into an issue where a user encountered data corruption after a storage error. PostgreSQL played a part in that corruption by allowing checkpoint what should've been a fatal error. TL;DR: Pg should PANIC on fsync() EIO return. Retrying fsync() is not OK at least on

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Michael Paquier
On Tue, Mar 27, 2018 at 06:32:08PM -0400, Chapman Flack wrote: > Is 2dd9322 a commit? I'm having difficulty finding it: > > https://git.postgresql.org/gitweb/?p=postgresql.git=search=HEAD=commit=2dd9322 > > Am I searching wrong? > > I probably won't have more time to look at this tonight, but

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Robert Haas
On Tue, Mar 27, 2018 at 10:06 PM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 02:02:10PM -0400, Tom Lane wrote: >> Robert Haas writes: >>> On Tue, Mar 27, 2018 at 11:41 AM, Tom Lane wrote: This is ignoring the possibility

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Michael Paquier
On Tue, Mar 27, 2018 at 02:02:10PM -0400, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 27, 2018 at 11:41 AM, Tom Lane wrote: >>> This is ignoring the possibility of damaged data in between, ie >>> A ... B ... CHKPT ... C ... a few zeroed pages

Re: Enhance pg_stat_wal_receiver view to display connected host

2018-03-27 Thread Michael Paquier
On Wed, Mar 28, 2018 at 11:28:32AM +1100, Haribabu Kommi wrote: > I updated the pg_stat_wal_receiver patch with the new PQhost() function > behavior and updated the view with two columns, (remote_server and > remote_port) instead of three as earlier. > > Updated patch attached. Thanks Hari for

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-27 Thread Michael Paquier
On Wed, Mar 28, 2018 at 11:06:23AM +1100, Haribabu Kommi wrote: > On Wed, Mar 28, 2018 at 3:35 AM, Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: >> >> Committed after fixing up the documentation a bit as suggested by others. > > Thanks. +1. Thanks for working on this Hari, Peter

Re: Oddity in COPY FROM handling of check constraints on partition tables

2018-03-27 Thread Amit Langote
Fujita-san, On 2018/03/27 22:00, Etsuro Fujita wrote: > Hi, > > While updating the tuple-routing-for-foreign-partitions patch, I noticed > oddity in the COPY FROM handling of check constraints on partition > tables. Here is an example: > > postgres=# create table pt (a int, b int) partition by

Re: [HACKERS] logical decoding of two-phase transactions

2018-03-27 Thread Andres Freund
On 2018-03-27 10:19:37 +0100, Simon Riggs wrote: > On 23 March 2018 at 15:26, Simon Riggs wrote: > > > Reviewing 0003-Add-support-for-logging-GID-in-commit-abort-WAL-reco > > > > Looks fine, reworked patch attached > > * added changes to xact.h from patch 4 so that this is

Re: PL/pgSQL nested CALL with transactions

2018-03-27 Thread Tomas Vondra
On 03/24/2018 03:14 PM, Peter Eisentraut wrote: > On 3/22/18 11:50, Tomas Vondra wrote: > >> 2) spi.c >> >> I generally find it confusing when there are negative flags, which are >> then negated whenever used. That is pretty the "no_snapshots" case, >> because it means "don't manage snapshots"

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Tomas Vondra
On 03/27/2018 07:34 PM, Tomas Vondra wrote: > On 03/27/2018 07:03 PM, Dean Rasheed wrote: >> On 27 March 2018 at 14:58, Dean Rasheed wrote: >>> On 27 March 2018 at 01:36, Tomas Vondra >>> wrote: 4) handling of NOT clauses in MCV

Re: Enhance pg_stat_wal_receiver view to display connected host

2018-03-27 Thread Haribabu Kommi
On Tue, Jan 30, 2018 at 4:02 PM, Michael Paquier wrote: > On Tue, Jan 30, 2018 at 03:10:12PM +1100, Haribabu Kommi wrote: > > Ok, understood. As the libpq gives preference to hostaddr connection > > parameter than host while connecting. How about going with one column

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Andrew Dunstan
On Wed, Mar 28, 2018 at 1:21 AM, Pavan Deolasee wrote: > > > TBH I still don't see why this does not provide the same guarantee that the > current code provides, but given the concerns expressed by others, I am not > gonna pursue beyond a point. But one last time :-) > >

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Tomas Vondra
On 03/28/2018 12:32 AM, Chapman Flack wrote: > On 03/26/18 01:43, Michael Paquier wrote: > >> Have a look at BKP_REMOVABLE then. This was moved to page headers in >> 2dd9322, still it seems to me that the small benefits outlined on this >> thread don't justify breaking tools relying on this

Re: Parallel Aggregates for string_agg and array_agg

2018-03-27 Thread Tomas Vondra
On 03/27/2018 04:51 AM, David Rowley wrote: > On 27 March 2018 at 13:45, David Rowley wrote: >> On 27 March 2018 at 12:49, Tom Lane wrote: >>> Oh, I thought of another thing that would need to get done, if we decide >>> to commit this.

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-27 Thread Haribabu Kommi
On Wed, Mar 28, 2018 at 3:35 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/27/18 02:20, Michael Paquier wrote: > > On Tue, Mar 27, 2018 at 04:47:41PM +1100, Haribabu Kommi wrote: > >> updated patch attached with additional doc updates as per the suggestion > >> from the

Re: Backend memory dump analysis

2018-03-27 Thread Tom Lane
Peter Eisentraut writes: > How about this one as well: > portal->portalContext = AllocSetContextCreate(TopPortalContext, > "PortalContext", >

Re: Backend memory dump analysis

2018-03-27 Thread Peter Eisentraut
On 3/27/18 12:47, Tom Lane wrote: > Here's an updated patch that adjusts the output format per discussion: > > - context identifier at the end of the line, so it's easier to see the > numbers > > - identifiers truncated at 100 bytes, control characters replaced by > spaces > > Also, I

Re: Undesirable entries in typedefs list

2018-03-27 Thread Andrew Dunstan
On Wed, Mar 28, 2018 at 8:32 AM, Tom Lane wrote: > Andrew Dunstan writes: >>> pgindent already has a list of blacklisted typedefs (see lines 121 to 123) > > Oh, so it does. > >> Here's a patch to pgindent which I think will do what you want

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-27 Thread Michael Paquier
On Wed, Mar 28, 2018 at 04:13:25AM +0900, Fujii Masao wrote: > This code is almost ok in practice, but using the check of > "strstr(path, localpath) == path" is more robust here? No problems with that either. > Using the following code instead is more robust? > This original code is almost ok in

Re: WIP: Covering + unique indexes.

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 10:14 AM, Teodor Sigaev wrote: >> b) I don't like an idea to limiting usage of that field if we can do not >> that. Future usage could use it, for example, for different compression >> technics or something else.Or even removing t_tid from inner tuples to

Re: new function for tsquery creartion

2018-03-27 Thread Dmitry Ivanov
select websearch_to_tsquery('simple', 'abc or!def'); websearch_to_tsquery -- 'abc' | 'def' (1 row) This is wrong ofc, I've attached the fixed version. -- Dmitry Ivanov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Chapman Flack
On 03/26/18 01:43, Michael Paquier wrote: > Have a look at BKP_REMOVABLE then. This was moved to page headers in > 2dd9322, still it seems to me that the small benefits outlined on this > thread don't justify breaking tools relying on this flag set, especially > if there is no replacement for

Re: new function for tsquery creartion

2018-03-27 Thread Dmitry Ivanov
Hi everyone, I'd like to share some intermediate results. Here's what has changed: 1. OR operator is now case-insensitive. Moreover, trailing whitespace is no longer used to identify it: select websearch_to_tsquery('simple', 'abc or'); websearch_to_tsquery -- 'abc' &

Re: Undesirable entries in typedefs list

2018-03-27 Thread Tom Lane
Andrew Dunstan writes: >> pgindent already has a list of blacklisted typedefs (see lines 121 to 123) Oh, so it does. > Here's a patch to pgindent which I think will do what you want fairly > simply, assuming you have identified all the offending tokens. Hm, this

Re: [HACKERS] A design for amcheck heapam verification

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 6:48 AM, Pavan Deolasee wrote: > + * When index-to-heap verification is requested, a Bloom filter is used to > + * fingerprint all tuples in the target index, as the index is traversed to > + * verify its structure. A heap scan later verifies the

Re: PATCH: Configurable file mode mask

2018-03-27 Thread David Steele
On 3/20/18 11:14 PM, Michael Paquier wrote: > On Tue, Mar 20, 2018 at 05:44:22PM -0400, Stephen Frost wrote: >> * David Steele (da...@pgmasters.net) wrote: >>> On 3/16/18 11:12 AM, Stephen Frost wrote: >>> It seems to me that pg_basebackup and pg_receivexlog should have a -g >>> option to control

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-27 Thread Fujii Masao
On Sun, Mar 11, 2018 at 11:04 PM, Michael Paquier wrote: > On Fri, Mar 09, 2018 at 08:22:49AM +, Tsunakawa, Takayuki wrote: >> Thanks for reviewing. All done. > > Thanks. Please be careful with the indentation of the patch. Attached > is a slightly-updated version with

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-27 Thread Fujii Masao
On Tue, Mar 27, 2018 at 10:55 AM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 01:32:41AM +0900, Fujii Masao wrote: >> +1. It's better for us to focus on the code change of the fillter on >> pg_rewind >> rather than such "refactoring". > > (filter takes one 'l', not two)

Re: [HACKERS] Surjective functional indexes

2018-03-27 Thread Simon Riggs
On 23 March 2018 at 15:54, Simon Riggs wrote: > So please could you make the change? Committed, but I still think that change would be good. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] path toward faster partition pruning

2018-03-27 Thread Jesper Pedersen
Hi, On 03/27/2018 01:46 PM, Jesper Pedersen wrote: Running v41 with "partition_prune" under valgrind gives the attached report. The reports mostly involve interaction with catcache.c and dynahash.c, so something for a separate thread. Best regards,  Jesper

Re: All Taxi Services need Index Clustered Heap Append

2018-03-27 Thread fedor
Colleagues incomprehensible situation: it seems that the statistics for autovacuum analyze is not passed to the slave. After a manual run, vacuum analyze all comes to life. the essence of this is a frequently changing table when I request a replica I get on a big Heap Fetches: 18623 the request

Explain buffers wrong counter with parallel plans

2018-03-27 Thread Adrien Nayrat
Hello, I notice Explain report wrong counters with parallel plans : create unlogged table t1 (c1 int); insert into t1 select generate_series(1,1); vacuum t1; (stop PG, drop caches,start) set track_io_timing to on; explain (analyze,buffers) select count(*) from t1;

Re: Re: csv format for psql

2018-03-27 Thread Pavel Stehule
2018-03-27 11:35 GMT+02:00 Fabien COELHO : > > Hello Pavel, > > I'd like to convince you to compromise, because otherwise I'm afraid we > will not get this feature. > > 1. use special default string for formats that doesn't field sep - like >>> "not used" or some 2.

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 1:15 AM, Simon Riggs wrote: > Accepted, the only question is whether it affects UPDATE as well cos > it looks like it should. If you mean an UPDATE FROM self-join, then I suppose that it does, in a very limited way. The difference is that there are

Re: [HACKERS] path toward faster partition pruning

2018-03-27 Thread Tom Lane
Alvaro Herrera writes: > Amit Langote wrote: >> [Jesper] also pointed out a case with a >> list-partitioned table where pruning doesn't a produce a result as one >> would expect and what constraint exclusion would produce. >> >> create table lp (a char) partition by list

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 27, 2018 at 11:41 AM, Tom Lane wrote: >> This is ignoring the possibility of damaged data in between, ie >> A ... B ... CHKPT ... C ... a few zeroed pages ... D ... CHKPT ... E ... F > It's hard for me to believe that

Re: [HACKERS] path toward faster partition pruning

2018-03-27 Thread Alvaro Herrera
Amit Langote wrote: > [Jesper] also pointed out a case with a > list-partitioned table where pruning doesn't a produce a result as one > would expect and what constraint exclusion would produce. > > create table lp (a char) partition by list (a); > create table lp_ad partition of lp for values

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Robert Haas
On Tue, Mar 27, 2018 at 11:41 AM, Tom Lane wrote: > Pavan Deolasee writes: >> On Tue, Mar 27, 2018 at 7:28 PM, Tom Lane wrote: >>> If you have to search backwards, this breaks it. Full stop. > >> We don't really need to fetch

Re: [HACKERS] path toward faster partition pruning

2018-03-27 Thread Jesper Pedersen
Hi Amit, On 03/27/2018 06:42 AM, Amit Langote wrote: I have managed to make the recursion go away in the attached updated version. I guess that's the result of employing the idea of a "output register" for individual pruning steps as mentioned in Robert's email upthread where he detailed the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Tomas Vondra
On 03/27/2018 04:58 PM, Dean Rasheed wrote: > On 27 March 2018 at 01:36, Tomas Vondra wrote: >> BTW I think there's a bug in handling the fullmatch flag - it should not >> be passed to AND/OR subclauses the way it is, because then >> >> WHERE a=1 OR (a=2 AND

Re: JIT compiling with LLVM v12

2018-03-27 Thread Andres Freund
On 2018-03-27 10:05:47 -0400, Peter Eisentraut wrote: > On 3/13/18 19:40, Andres Freund wrote: > > I've pushed a revised and rebased version of my JIT patchset. > > What is the status of this item as far as the commitfest is concerned? 7/10 committed. Inlining, Explain, Docs remain. Greetings,

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Tomas Vondra
On 03/27/2018 07:03 PM, Dean Rasheed wrote: > On 27 March 2018 at 14:58, Dean Rasheed wrote: >> On 27 March 2018 at 01:36, Tomas Vondra wrote: >>> 4) handling of NOT clauses in MCV lists (and in histograms) >>> >>> The query you posted does

Re: Parallel Aggregates for string_agg and array_agg

2018-03-27 Thread Tomas Vondra
On 03/27/2018 04:58 PM, Tom Lane wrote: > David Rowley writes: >> On 27 March 2018 at 13:26, Alvaro Herrera wrote: >>> synchronized_seqscans is another piece of precedent in the area, FWIW. > >> This is true. I guess the order of

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-27 Thread Robert Haas
On Tue, Mar 27, 2018 at 7:42 AM, Robert Haas wrote: > On Tue, Mar 27, 2018 at 1:45 AM, Amit Kapila wrote: >> If we don't want to go with the upperrel logic, then maybe we should >> consider just merging some of the other changes from my previous

Re: Backend memory dump analysis

2018-03-27 Thread Tom Lane
Alvaro Herrera writes: > My only gripe is the pattern where the identifier needs to be > re-installed when resetting the context. I don't think we need to hold > push for that reason alone, but I bet we'll be revisiting that. Yeah, that's slightly annoying; if I'd found

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-27 Thread Robert Haas
On Tue, Mar 27, 2018 at 11:17 AM, Tom Lane wrote: > So, your proposal is to do nothing and just hope we don't make the same > mistake again in future? That wasn't really my proposal, but if it were, it would be at least as good as your proposal of making changing things in an

Re: WIP: Covering + unique indexes.

2018-03-27 Thread Peter Geoghegan
On Tue, Mar 27, 2018 at 10:07 AM, Teodor Sigaev wrote: > Storing number of attributes in now unused t_tid seems to me not so good > idea. a) it could (and suppose, should) separate patch, at least it's not > directly connected to covering patch, it could be added even before

Re: WIP: Covering + unique indexes.

2018-03-27 Thread Teodor Sigaev
b) I don't like an idea to limiting usage of that field if we can do not that. Future usage could use it, for example, for different compression technics or something else.Or even removing t_tid from inner tuples to save 2 bytes in IndexTupleData. Of course, I remember about aligment, but it

Re: Backend memory dump analysis

2018-03-27 Thread Alvaro Herrera
Great stuff. My only gripe is the pattern where the identifier needs to be re-installed when resetting the context. I don't think we need to hold push for that reason alone, but I bet we'll be revisiting that. I suppose this infrastructure can be used to implement the idea in

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
On 27 March 2018 at 14:58, Dean Rasheed wrote: > On 27 March 2018 at 01:36, Tomas Vondra wrote: >> 4) handling of NOT clauses in MCV lists (and in histograms) >> >> The query you posted does not fail anymore... >> > Ah, it turns out the

Re: Proposal: http2 wire format

2018-03-27 Thread Damir Simunic
> > > I'm rapidly losing interest. Unless this goes back toward the concrete and > practical I think it's going nowhere. Your message is exactly what I was hoping for. Thanks for your guidance and support, really appreciate you. Let me now get busy and earn your continued interest and

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-27 Thread Peter Eisentraut
On 3/27/18 02:20, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 04:47:41PM +1100, Haribabu Kommi wrote: >> updated patch attached with additional doc updates as per the suggestion >> from the upthreads. > > Thanks Hari for the quick update. It looks to me that this is shaped as > suggested.

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Tom Lane
Pavan Deolasee writes: > On Tue, Mar 27, 2018 at 7:28 PM, Tom Lane wrote: >> If you have to search backwards, this breaks it. Full stop. > We don't really need to fetch the previous record. We really need to find > the last checkpoint prior to a

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Tom Lane
Simon Riggs writes: > On 27 March 2018 at 14:58, Tom Lane wrote: >> The point of the xl_prev links is, essentially, to allow verification >> that we are reading a coherent series of WAL records, ie that record B >> which appears to follow record A

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-27 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 26, 2018 at 7:23 PM, Tom Lane wrote: >> While the minimal patch would be fine for now, what I'm worried about >> is preventing future mistakes. It seems highly likely that the reason >>

Re: jsonpath

2018-03-27 Thread Nikita Glukhov
Attached 14th version of the patches: * refactored predicates, introduced 3-valued JsonPathBool type instead of JsonPathExecResult * refactored JsonPathExecResult: now it is typedefed to ErrorData * * fixed recursive wildcard accessor (.**) in strict mode: structural errors after .** are

Re: Cast jsonb to numeric, int, float, bool

2018-03-27 Thread Teodor Sigaev
Hi! I took a look on patch and it seems to me it looks both incomplete and oveflowing. It suggests cast from numeric and bool, but for numeric it suggests only numeric, int4 and int8. This choice looks irrational. I think, it should support from/to numeric/bool/text only. If we want to have

Re: Parallel Aggregates for string_agg and array_agg

2018-03-27 Thread Tom Lane
David Rowley writes: > On 27 March 2018 at 13:26, Alvaro Herrera wrote: >> synchronized_seqscans is another piece of precedent in the area, FWIW. > This is true. I guess the order of aggregation could be made more > certain if we remove the

Project Proposal for GSOC 2018

2018-03-27 Thread Ravindu Perera
Hello! I have attached the project proposal to develop the PostgreSQL Performance Farm Website for GSOC 2018 with this email. Please review my proposal. Thanks! Regards, Ravindu Perera GSOC 2018 Project Proposal - Performance Farm Web Application.pdf Description: Adobe PDF document

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
On 27 March 2018 at 01:36, Tomas Vondra wrote: > BTW I think there's a bug in handling the fullmatch flag - it should not > be passed to AND/OR subclauses the way it is, because then > > WHERE a=1 OR (a=2 AND b=2) > > will probably set it to 'true' because of

Re: Foreign keys and partitioned tables

2018-03-27 Thread Alvaro Herrera
Peter Eisentraut wrote: > Since the row triggers patch has been committed, do you plan to send an > update on this patch? Yes, I'll do that shortly. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Re: Cast jsonb to numeric, int, float, bool

2018-03-27 Thread Aleksander Alekseev
Hello everyone, > Since this patch was updated after being set a Ready for Committer and > there appear to be some open questions, I have set it to Needs Review. I decided to take a look. The patch didn't apply after changes made in fd1a421f, but I fixed that. Also there were no tests. I fixed

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Pavan Deolasee
On Tue, Mar 27, 2018 at 7:28 PM, Tom Lane wrote: > > > I had not noticed that in the kerfuffle over the more extreme version, > but I think this is a different route to breaking the same guarantees. > The point of the xl_prev links is, essentially, to allow verification >

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Simon Riggs
On 27 March 2018 at 14:58, Tom Lane wrote: > The point of the xl_prev links is, essentially, to allow verification > that we are reading a coherent series of WAL records, ie that record B > which appears to follow record A actually is supposed to follow it. > This throws away

Re: Foreign keys and partitioned tables

2018-03-27 Thread Peter Eisentraut
On 3/11/18 22:40, Alvaro Herrera wrote: > [ Resending an email from yesterday. Something is going very wrong with > my outgoing mail provider :-( ] > > Rebase of the prior code, on top of the improved row triggers posted > elsewhere. I added some more tests too, and fixed a couple of small >

Re: WIP: a way forward on bootstrap data

2018-03-27 Thread John Naylor
Tom Lane wrote: >> -Maybe document examples of how to do bulk-editing of data files? > > +1. In the end, that's the reason we're doing all this work, so showing > people how to benefit seems like a good thing. I'll hold off on posting a new patchset until I add this to the documentation, but I

Re: [HACKERS] [PATCH] Lockable views

2018-03-27 Thread Yugo Nagata
On Tue, 6 Feb 2018 11:12:37 -0500 Robert Haas wrote: > On Tue, Feb 6, 2018 at 1:28 AM, Tatsuo Ishii wrote: > >> But what does that have to do with locking? > > > > Well, if the view is not updatable, I think there will be less point > > to allow to

Re: pg_class.reltuples of brin indexes

2018-03-27 Thread Alvaro Herrera
Tom Lane wrote: > Tomas Vondra writes: > > I think number of index tuples makes sense, as long as that's what the > > costing needs. That is, it's up to the index AM to define it. But it > > clearly should not flap like this ... > > > And it's not just BRIN. This is

Re: XML/XPath issues: text/CDATA in XMLTABLE, XPath evaluated with wrong context

2018-03-27 Thread Alvaro Herrera
Markus Winand wrote: Hi Markus, > I’ve found two issues with XML/XPath integration in PostgreSQL. Two > patches are attached. I’ve just separated them because the second one > is an incompatible change. Thanks for these. I'll have a look at them after the commitfest is over. In the meantime,

Re: pg_class.reltuples of brin indexes

2018-03-27 Thread Tom Lane
Tomas Vondra writes: > I think number of index tuples makes sense, as long as that's what the > costing needs. That is, it's up to the index AM to define it. But it > clearly should not flap like this ... > And it's not just BRIN. This is what I get with a GIN

Re: Ensure that maxlen is an integer value in dict_int configuration

2018-03-27 Thread Daniel Gustafsson
> On 27 Mar 2018, at 15:04, Arthur Zakirov wrote: > But the patch breaks options parsing in another place. After the patch: > So the right fix could be as it done in postgres_fdw_validator() for > 'fetch_size' option. Doh. I had a strtol() first but found the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
On 27 March 2018 at 01:36, Tomas Vondra wrote: > 4) handling of NOT clauses in MCV lists (and in histograms) > > The query you posted does not fail anymore... > Ah, it turns out the previous query wasn't actually failing for the reason I thought it was -- it was

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Tom Lane
Robert Haas writes: > Taking a look at this version, I think the key thing we have to decide > is whether we're comfortable with this: > --- a/src/include/access/xlogrecord.h > +++ b/src/include/access/xlogrecord.h > @@ -42,7 +42,7 @@ typedef struct XLogRecord > { >

PostgreSQL 11 Release Management Team & Feature Freeze

2018-03-27 Thread Jonathan S. Katz
Hi, The Release Management Team (RMT) for the PostgreSQL 11 release has been assembled and has determined that the feature freeze date for the PostgreSQL 11 release will be April 7, 2018. This means that any feature that will be going into the PostgreSQL 11 release must be committed before

Re: [HACKERS] A design for amcheck heapam verification

2018-03-27 Thread Pavan Deolasee
On Tue, Mar 27, 2018 at 8:50 AM, Peter Geoghegan wrote: > On Fri, Mar 23, 2018 at 7:13 AM, Andrey Borodin > wrote: > > I've just flipped patch to WoA. But if above issues will be fixed I > think that patch is ready for committer. > > Attached is v7, which has

Re: Vacuum: allow usage of more than 1GB of work mem

2018-03-27 Thread Aleksander Alekseev
Hello everyone, I would like to let you know that unfortunately these patches don't apply anymore. Also personally I'm a bit confused by the last message that has 0001- and 0003- patches attached but not the 0002- one.

pgbench doc typos

2018-03-27 Thread Fabien COELHO
Attached patch fixes two very minor typos in pgbench recently added documentations: - one about the mod() function - two about the hash() function -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 41d9030..e4b37dd 100644 ---

Re: Ensure that maxlen is an integer value in dict_int configuration

2018-03-27 Thread Arthur Zakirov
On Tue, Mar 27, 2018 at 01:17:20PM +0200, Daniel Gustafsson wrote: > I recently had a usecase for dict_int, typoed my script and spent longer than > I’d like to admit finding said typo. The attached patch scratches my itch by > ensuring that the maxlen parameter to dict_int is integer, instead of

Re: Problem while setting the fpw with SIGHUP

2018-03-27 Thread Michael Paquier
On Tue, Mar 27, 2018 at 09:01:20PM +0900, Kyotaro HORIGUCHI wrote: > The current UpdateFullPageWrites is safe on standby and promotion > so what we should consider is only the non-standby case. I think > what we should do is just calling RecoveryInProgress() at the > beginning of CheckPointerMain,

Oddity in COPY FROM handling of check constraints on partition tables

2018-03-27 Thread Etsuro Fujita
Hi, While updating the tuple-routing-for-foreign-partitions patch, I noticed oddity in the COPY FROM handling of check constraints on partition tables. Here is an example: postgres=# create table pt (a int, b int) partition by list (a); CREATE TABLE postgres=# create table p1 partition of pt

Re: pg_class.reltuples of brin indexes

2018-03-27 Thread Tomas Vondra
On 03/27/2018 01:58 PM, Masahiko Sawada wrote: > Hi, > > I found that pg_class.reltuples of brin indexes can be either the > number of index tuples or the number of heap tuples. > > =# create table test as select generate_series(1,10) as c; > =# create index test_brin on test using brin (c);

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2018-03-27 Thread Teodor Sigaev
Thanks to everyone, pushed Andrey Borodin wrote: 27 марта 2018 г., в 13:45, Alexander Korotkov > написал(а): On Tue, Mar 27, 2018 at 11:16 AM, Andrey Borodin > wrote: > 27

Re: Online enabling of checksums

2018-03-27 Thread Tomas Vondra
On 03/27/2018 08:56 AM, Magnus Hagander wrote: > On Mon, Mar 26, 2018 at 10:09 PM, Tomas Vondra > > wrote: > > Hi, > > I see enable_data_checksums() does this: > >     if (cost_limit <= 0) >        

Re: [PROPOSAL] Shared Ispell dictionaries

2018-03-27 Thread Arthur Zakirov
On Mon, Mar 26, 2018 at 11:27:48AM -0400, Tom Lane wrote: > Arthur Zakirov writes: > > I'm not sure that I understood the second case correclty. Can cache > > invalidation help in this case? I don't have confident knowledge of cache > > invalidation. It seems to me that

Re: pgbench - test whether a variable exists

2018-03-27 Thread Fabien COELHO
Patch v2 is a rebase. Patch v3 is also a rebase. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 41d9030..7b3badf 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -971,6 +971,8 @@ pgbench options d integer

Re: Problem while setting the fpw with SIGHUP

2018-03-27 Thread Kyotaro HORIGUCHI
At Tue, 27 Mar 2018 16:46:30 +0900, Michael Paquier wrote in <20180327074630.gd9...@paquier.xyz> > I have finally been able to spend more time on this issue, and checked > for a couple of hours all the calls to RecoveryInProgress() that could > be triggered within a critical

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Robert Haas
On Tue, Mar 27, 2018 at 6:35 AM, Andrew Dunstan wrote: > Leaving aside the arguments about process, the patch is pretty small > and fairly straightforward. Given the claimed performance gains that's > a nice bang for the buck. > > I haven't seen any obvious holes,

pg_class.reltuples of brin indexes

2018-03-27 Thread Masahiko Sawada
Hi, I found that pg_class.reltuples of brin indexes can be either the number of index tuples or the number of heap tuples. =# create table test as select generate_series(1,10) as c; =# create index test_brin on test using brin (c); =# analyze test; =# select relname, reltuples, relpages from

  1   2   >