Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 04:00:42PM +0900, Amit Langote wrote: > Thanks for looking at this. Your patch seems better, because it allows us > to keep the error message consistent with the message one would get with > list-partitioned syntax. Thanks for confirming. I think that it would be nice as

Re: Prevent extension creation in temporary schemas

2019-03-06 Thread Chris Travers
On Wed, Mar 6, 2019 at 3:19 AM Michael Paquier wrote: > On Tue, Mar 05, 2019 at 12:47:54PM +, Chris Travers wrote: > > I tried installing a test extension into a temp schema. I found > > this was remarkably difficult to do because pg_temp did not work (I > > had to create a temporary table a

Re: Prevent extension creation in temporary schemas

2019-03-06 Thread Chris Travers
On Wed, Mar 6, 2019 at 9:33 AM Chris Travers wrote: > > >> Thoughts? >> > To re-iterate, my experience with PostgreSQL is that people doing particularly exotic work in PostgreSQL can expect to hit equally exotic bugs. I have a list that I will not bore people with here. I think there is a gener

pg_basebackup against older server versions

2019-03-06 Thread Devrim Gündüz
Hi, Apologies if this has been discussed before: When I run pg_basebackup in git head against v11 server, it treats v11 as v12: Does not create recovery.conf, adds recovery parameters to postgresql.auto.conf, and also creates standby.signal file. Is this expected, or a bug? Regards, -- Devrim

Re: Make drop database safer

2019-03-06 Thread Ashwin Agrawal
On Mon, Feb 11, 2019 at 3:55 PM Ashwin Agrawal wrote: > > Thanks for the response and inputs. > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund wrote: > >> Hi, >> >> On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: >> > Current sequence of operations for drop database (dropdb()) >> > 1. Start

Re: pg_basebackup against older server versions

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 11:55:12AM +0300, Devrim Gündüz wrote: > Apologies if this has been discussed before: When I run pg_basebackup in git > head against v11 server, it treats v11 as v12: Does not create recovery.conf, > adds recovery parameters to postgresql.auto.conf, and also creates > stand

bug in update tuple routing with foreign partitions

2019-03-06 Thread Amit Langote
Hi, (added Fujita-san) I noticed a bug with how UPDATE tuple routing initializes ResultRelInfos to use for partition routing targets. Specifically, the bug occurs when UPDATE targets include a foreign partition that is locally modified (as opposed to being modified directly on the remove server)

Re: Proposal for Signal Detection Refactoring

2019-03-06 Thread Chris Travers
Here's a new patch. No rush on it. I am moving it to next commitfest anyway because as code documentation I think this is a low priority late in the release cycle. The changes mostly address Andres's feedback above. -- Best Regards, Chris Travers Head of Database Tel: +49 162 9037 210 | Skyp

patch tester symbols

2019-03-06 Thread Erik Rijkers
Hi, Context: I'm trying to compile the jsonpath v36 patches (these apply OK), and on top of those the jsonfunctions and jsontable patch series. That fails for me (on 0001-Implementation-of-SQL-JSON-path-language-v36.patch), and now I'm wondering why that does not agree with what the patch-te

Re: pg_basebackup against older server versions

2019-03-06 Thread Sergei Kornilov
Hello My fault. I thought pg_basebackup works only with same major version, sorry. How about attached patch? regards, Sergeidiff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c index 3d2d4cd0b9..e1aa2c1cfc 100644 --- a/src/bin/pg_basebackup/pg_basebackup.c ++

RE: proposal: pg_restore --convert-to-text

2019-03-06 Thread José Arthur Benetasso Villanova
On Thu, 28 Feb 2019, Imai, Yoshikazu wrote: Is there no need to rewrite the Description in the Doc to state we should specify either -d or -f option? (and also it might be better to write if -l option is given, neither -d nor -f option isn't necessarily needed.) Since the default part of

openLogOff is not needed anymore

2019-03-06 Thread Antonin Houska
... obviously due to commit c24dcd0. The following patch removes it. diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index ecd12fc53a..0fdd82a287 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -771,13 +771,11 @@ static

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Wed, 6 Mar 2019 at 20:19, Michael Paquier wrote: > > On Wed, Mar 06, 2019 at 07:45:06PM +1300, David Rowley wrote: > > Can anyone see any fundamental reason that we should not create a > > partitioned table by doing CREATE TABLE followed by ATTACH PARTITION? > > If not, I'll write a patch that

Re: Inheriting table AMs for partitioned tables

2019-03-06 Thread David Rowley
On Wed, 6 Mar 2019 at 07:19, Robert Haas wrote: > > On Tue, Mar 5, 2019 at 12:59 PM Andres Freund wrote: > > Based on this mail I'm currently planning to simply forbid specifying > > USING for partitioned tables. Then we can argue about this later. > > +1. I actually think that might be the righ

Re: WIP: Avoid creation of the free space map for small tables

2019-03-06 Thread John Naylor
On Fri, Jan 25, 2019 at 9:50 AM Amit Kapila wrote: > Once we agree on the code, we need to test below scenarios: > (a) upgrade from all supported versions to the latest version > (b) upgrade standby with and without using rsync. Although the code hasn't been reviewed yet, I went ahead and tested

Re: patch tester symbols

2019-03-06 Thread Nikita Glukhov
On 06.03.2019 13:11, Erik Rijkers wrote: Hi, Context: I'm trying to compile the jsonpath v36 patches (these apply OK), and on top of those the jsonfunctions and jsontable patch series. That fails for me (on 0001-Implementation-of-SQL-JSON-path-language-v36.patch), and now I'm wondering why

Re: patch tester symbols

2019-03-06 Thread Erik Rijkers
On 2019-03-06 13:00, Nikita Glukhov wrote: On 06.03.2019 13:11, Erik Rijkers wrote: Context: I'm trying to compile the jsonpath v36 patches (these apply OK), and on top of those the jsonfunctions and jsontable patch series. Patch 0001-Implementation-of-SQL-JSON-path-language-v36.patch from

Re: Problems with plan estimates in postgres_fdw

2019-03-06 Thread Etsuro Fujita
(2019/02/22 17:17), Etsuro Fujita wrote: (2019/02/21 6:19), Jeff Janes wrote: With your tweaks, I'm still not seeing the ORDER-less LIMIT get pushed down when using use_remote_estimate in a simple test case, either with this set of patches, nor in the V4 set. However, without use_remote_estima

Re: bug in update tuple routing with foreign partitions

2019-03-06 Thread Etsuro Fujita
Hi Amit, (2019/03/06 18:33), Amit Langote wrote: I noticed a bug with how UPDATE tuple routing initializes ResultRelInfos to use for partition routing targets. Specifically, the bug occurs when UPDATE targets include a foreign partition that is locally modified (as opposed to being modified dir

Re: Patch to document base64 encoding

2019-03-06 Thread Karl O. Pinc
On Tue, 5 Mar 2019 23:23:20 -0600 "Karl O. Pinc" wrote: > Added documentation for hex and escape encodings, > including output formats and what are acceptable > inputs. Attached: doc_base64_v6.patch Added index entries for encode and decode functions above the encoding documentation. As index

Batch insert in CTAS/MatView code

2019-03-06 Thread Paul Guo
Hello, Postgres hackers, The copy code has used batch insert with function heap_multi_insert() to speed up. It seems that Create Table As or Materialized View could leverage that code also to boost the performance also. Attached is a patch to implement that. That was done by Taylor (cc-ed) and me.

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Tom Lane
David Rowley writes: > As far as I can see, the biggest fundamental difference with doing > things this way will be that the column order of partitions will be > preserved, where before it would inherit the order of the partitioned > table. I'm a little unsure if doing this column reordering was

Server Crash in logical decoding if used inside --single mode

2019-03-06 Thread tushar
Hi, Steps to reproduce on Master Sources - .) Perform initdb ( ./initdb -D data) .) set wal_level=logical in postgresql.conf file .)Connect to psql in single-user mode  ( ./postgres --single  -D data  postgres) .)Create logical replication slot followed by select * from pg_logical_slot_get

Re: openLogOff is not needed anymore

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 6:10 AM Antonin Houska wrote: > ... obviously due to commit c24dcd0. The following patch removes it. Committed, after a short struggle to get the patch out of the email body in a usable form. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreS

Re: [HACKERS] CLUSTER command progress monitor

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 8:03 PM Alvaro Herrera wrote: > One, err, small issue with that idea is that we need the param numbers > not to conflict for any "progress update providers" that are to be used > simultaneously by any command. Is that really an issue? I think progress reporting -- at least

Re: Patch to document base64 encoding

2019-03-06 Thread Karl O. Pinc
On Wed, 6 Mar 2019 07:09:48 -0600 "Karl O. Pinc" wrote: > On Tue, 5 Mar 2019 23:23:20 -0600 > "Karl O. Pinc" wrote: > > > Added documentation for hex and escape encodings, > > including output formats and what are acceptable > > inputs. Attached: doc_base64_v7.patch Improved escape decoding

Re: openLogOff is not needed anymore

2019-03-06 Thread Antonin Houska
Robert Haas wrote: > On Wed, Mar 6, 2019 at 6:10 AM Antonin Houska wrote: > > ... obviously due to commit c24dcd0. The following patch removes it. > > Committed, after a short struggle to get the patch out of the email > body in a usable form. It was just convenient for me to use vc-diff emacs

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Andrew Dunstan
On 3/4/19 9:39 AM, David Steele wrote: > On 3/1/19 3:59 AM, Chapman Flack wrote: >> The following review has been posted through the commitfest application: >> make installcheck-world:  tested, passed >> Implements feature:   tested, passed >> Spec compliant:   not tested >> Documenta

Re: [HACKERS] CLUSTER command progress monitor

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Tue, Mar 5, 2019 at 8:03 PM Alvaro Herrera > wrote: > > One, err, small issue with that idea is that we need the param numbers > > not to conflict for any "progress update providers" that are to be used > > simultaneously by any command. > > Is that reall

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Chapman Flack
On 3/6/19 10:12 AM, Andrew Dunstan wrote: > Having reviewed the thread, I'm with Andres and Tom. Maybe though we > should have a note somewhere to the effect that you can't use VARIADIC > with these. Perhaps such a note belongs hoisted into the functions-conditional section of the manual, making

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-06 Thread Andrey Borodin
Hi! > 20 февр. 2019 г., в 17:06, Alexey Kondratov > написал(а): > >> >> I will work out this one with postgres -C and come back till the next >> commitfest. I found that something similar is already used in pg_ctl and >> there is a mechanism for finding valid executables in exec.c. So it do

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera wrote: > I think the idea of it being a generic tunable for assorted behavior > changes, rather than specific to WAL recycling, is a good one. I'm > unsure about your proposed name -- maybe "wal_cow_filesystem" is better? I *really* dislike this. F

Re: A separate table level option to control compression

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 5:30 PM Andrew Dunstan wrote: > This is a nice idea, and I'm a bit surprised it hasn't got more > attention. The patch itself seems very simple and straightforward, > although it could probably do with having several sets of eyeballs on it. I haven't needed this for anythin

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Andrew Dunstan
On 3/6/19 10:38 AM, Robert Haas wrote: > On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera > wrote: >> I think the idea of it being a generic tunable for assorted behavior >> changes, rather than specific to WAL recycling, is a good one. I'm >> unsure about your proposed name -- maybe "wal_cow_fi

Re: Make drop database safer

2019-03-06 Thread Tomas Vondra
On 2/12/19 12:55 AM, Ashwin Agrawal wrote: Thanks for the response and inputs. On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > wrote: Hi, On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: > Current sequence of operations for drop database (dropdb()

Re: PostgreSQL vs SQL/XML Standards

2019-03-06 Thread Chapman Flack
This CF entry shows Pavel and me as reviewers, but the included patches were also produced by one or the other of us, so additional review by someone who isn't us seems appropriate. :) Would it make sense to remove one or both of us from the 'reviewers' field in the app, to make it more obviously

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 19:45:06 +1300, David Rowley wrote: > Over on [1] Andres pointed out that the pg_dump support for the new to > PG12 tablespace inheritance feature is broken. This is the feature > added in ca4103025dfe26 to allow a partitioned table to have a > tablespace that acts as the defaul

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 10:55 AM Andrew Dunstan wrote: > > I *really* dislike this. For one thing, it means that users don't > > have control over the behaviors individually. For another, the > > documentation is now quite imprecise about what the option actually > > does, while expecting users t

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Andrew Dunstan
On 3/6/19 11:30 AM, Robert Haas wrote: > On Wed, Mar 6, 2019 at 10:55 AM Andrew Dunstan > wrote: >>> I *really* dislike this. For one thing, it means that users don't >>> have control over the behaviors individually. For another, the >>> documentation is now quite imprecise about what the opti

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera > wrote: > > I think the idea of it being a generic tunable for assorted behavior > > changes, rather than specific to WAL recycling, is a good one. I'm > > unsure about your proposed name -- maybe "wal_cow_file

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 11:37 AM Andrew Dunstan wrote: > Well, let's put the question another way. Is there any reason to allow > skipping zero filling if we are recycling? That seems possibly > dangerous. I can imagine turning off recycling but leaving on > zero-filling, although I don't have a co

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 11:41 AM Alvaro Herrera wrote: > I can understand this argument. Is there really a reason to change > those two behaviors separately? See my previous rely to Andrew, but also, I think you're putting the burden of proof in the wrong place. You could equally well ask "Is th

Re: [PATCH] kNN for btree

2019-03-06 Thread Nikita Glukhov
Attached 9th version of the patches. On 03.03.2019 12:46, Anastasia Lubennikova wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentatio

Re: [HACKERS] Incomplete startup packet errors

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 5:35 PM Andrew Dunstan wrote: > OK, I think we have agreement on Tom's patch. Do we want to backpatch > it? It's a change in behaviour, but I find it hard to believe anyone > relies on the existence of these annoying messages, so my vote would be > to backpatch it. I don't

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
I want your dictating software. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Online verification of checksums

2019-03-06 Thread Robert Haas
On Sat, Mar 2, 2019 at 4:38 PM Tomas Vondra wrote: > FWIW I don't think this qualifies as torn page - i.e. it's not a full > read with a mix of old and new data. This is partial write, most likely > because we read the blocks one by one, and when we hit the last page > while the table is being ext

Re: Online verification of checksums

2019-03-06 Thread Robert Haas
On Sat, Mar 2, 2019 at 5:45 AM Michael Banck wrote: > Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: > > On Tue, Sep 18, 2018 at 10:37 AM Michael Banck > > wrote: > > > I have added a retry for this as well now, without a pg_sleep() as well. > > > This catches around 80% of the half

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera wrote: > I want your dictating software. I'm afraid this is just me and a keyboard, but sadly for me you're not the first person to accuse me of producing giant walls of text. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Pavel Stehule
st 6. 3. 2019 v 16:24 odesílatel Chapman Flack napsal: > On 3/6/19 10:12 AM, Andrew Dunstan wrote: > > > Having reviewed the thread, I'm with Andres and Tom. Maybe though we > > should have a note somewhere to the effect that you can't use VARIADIC > > with these. > > Perhaps such a note belongs

Re: Online verification of checksums

2019-03-06 Thread Andres Freund
On 2019-03-06 12:33:49 -0500, Robert Haas wrote: > On Sat, Mar 2, 2019 at 5:45 AM Michael Banck > wrote: > > Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: > > > On Tue, Sep 18, 2018 at 10:37 AM Michael Banck > > > wrote: > > > > I have added a retry for this as well now, without a

Re: Server Crash in logical decoding if used inside --single mode

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, tushar wrote: > backend> select * from pg_logical_slot_get_changes('m7',null,null); [...] > TRAP: FailedAssertion("!(slot != ((void *)0) && slot->active_pid != 0)", > File: "slot.c", Line: 428) > Aborted (core dumped) See argumentation in https://www.postgresql.org/message-id/fla

Re: [HACKERS] Block level parallel vacuum

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:26 AM Masahiko Sawada wrote: > Okay, attached the latest version of patch set. I've incorporated all > comments I got and separated the patch for making vacuum options a > Node (0001 patch). And the patch doesn't use parallel_workers. It > might be proposed in the another

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera > wrote: > > I want your dictating software. > > I'm afraid this is just me and a keyboard, but sadly for me you're not > the first person to accuse me of producing giant walls of text. Well, I don't have a pro

Re: pgsql: tableam: introduce table AM infrastructure.

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Andres Freund wrote: > tableam: introduce table AM infrastructure. Thanks for doing this!! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Robert Haas
On Fri, Mar 1, 2019 at 3:33 AM Kyotaro HORIGUCHI wrote: > > > It is artificial (or acutually wont't be repeatedly executed in a > > > session) but anyway what can get benefit from > > > catalog_cache_memory_target would be a kind of extreme. > > > > I agree. So then let's not have it. > > Ah... Y

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:02 PM Alvaro Herrera wrote: > Well, I don't have a problem reading long texts; my problem is that I'm > unable to argue as quickly. That's my secret weapon... except that it's not much of a secret. > I do buy your argument, though (if reluctantly); in particular I was >

Re: Patch to document base64 encoding

2019-03-06 Thread Fabien COELHO
Attached: doc_base64_v7.patch Patch applies cleanly, doc compiles, navigation tested and ok. "... section 6.8" -> "... Section 6.8" (capital S). "The string and the binary encode and decode functions..." sentence looks strange to me, especially with the English article that I do not really

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Alvaro Herrera
On 2019-Feb-08, Tomas Vondra wrote: > I'm wondering if we should just get rid of all such optimizations, and > make the unowned list doubly-linked (WIP patch attached, needs fixing > the comments etc.). +1 for that approach. Did you consider using a dlist? -- Álvaro Herrerahttp

Re: New vacuum option to do only freezing

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 11:29 PM Masahiko Sawada wrote: > Attached updated patch incorporated all of comments. Also I've added > new reloption vacuum_index_cleanup as per discussion on the "reloption > to prevent VACUUM from truncating empty pages at the end of relation" > thread. Autovacuums also

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Georgios Kokolatos
Overall the patch looks good and according to the previous discussion fulfils its purpose. It might be worthwhile to also check for errors on close in SaveSlotToPath(). pgstat_report_wait_end(); CloseTransientFile(fd); /* rename to permanent file, fsync file and directo

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:53 PM Alvaro Herrera wrote: > On 2019-Feb-08, Tomas Vondra wrote: > > I'm wondering if we should just get rid of all such optimizations, and > > make the unowned list doubly-linked (WIP patch attached, needs fixing > > the comments etc.). > > +1 for that approach. +1 for

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Tomas Vondra
On 3/6/19 7:52 PM, Alvaro Herrera wrote: > On 2019-Feb-08, Tomas Vondra wrote: > >> I'm wondering if we should just get rid of all such optimizations, and >> make the unowned list doubly-linked (WIP patch attached, needs fixing >> the comments etc.). > > +1 for that approach. > > Did you cons

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 6:26 PM, Robert Haas wrote: > On Sat, Mar 2, 2019 at 4:38 PM Tomas Vondra > wrote: >> FWIW I don't think this qualifies as torn page - i.e. it's not a full >> read with a mix of old and new data. This is partial write, most likely >> because we read the blocks one by one, and when we hit

Re: Pluggable Storage - Andres's take

2019-03-06 Thread Andres Freund
Hi, On 2019-03-05 23:07:21 -0800, Andres Freund wrote: > My next steps are: > - final polish & push the basic DDL and pg_dump patches Done and pushed. Some collation dependent fallout, I'm hoping I've just fixed that. > - cleanup & polish the ON CONFLICT refactoring Here's a cleaned up version

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 6:42 PM, Andres Freund wrote: > On 2019-03-06 12:33:49 -0500, Robert Haas wrote: >> On Sat, Mar 2, 2019 at 5:45 AM Michael Banck >> wrote: >>> Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: On Tue, Sep 18, 2018 at 10:37 AM Michael Banck wrote: > I have ad

Re: Online verification of checksums

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 20:37:39 +0100, Tomas Vondra wrote: > Not sure how to integrate it into the CLI tool, though. Perhaps we it > could require connection info so that it can execute a function, when > executed in online mode? To me the right fix would be to simply have this run as part of the clus

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Tomas Vondra
On 3/6/19 8:04 PM, Robert Haas wrote: > On Wed, Mar 6, 2019 at 1:53 PM Alvaro Herrera > wrote: >> On 2019-Feb-08, Tomas Vondra wrote: >>> I'm wondering if we should just get rid of all such optimizations, and >>> make the unowned list doubly-linked (WIP patch attached, needs fixing >>> the commen

Re: Make drop database safer

2019-03-06 Thread Ashwin Agrawal
On Wed, Mar 6, 2019 at 7:56 AM Tomas Vondra wrote: > > > On 2/12/19 12:55 AM, Ashwin Agrawal wrote: > > > > Thanks for the response and inputs. > > > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > > wrote: > > > > Hi, > > > > On 2019-02-08 16:36:13 -0800, Ale

Re: Optimization of some jsonb functions

2019-03-06 Thread Andrew Dunstan
On 3/5/19 5:24 AM, David Steele wrote: > On 2/22/19 2:05 AM, Nikita Glukhov wrote: >> Attached set of patches with some jsonb optimizations that were made >> during >> comparison of performance of ordinal jsonb operators and jsonpath >> operators. > > This patch was submitted just before the last

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread Andrew Dunstan
On 3/6/19 1:38 PM, Jeremy Schneider wrote: > On 3/5/19 14:14, Andrew Dunstan wrote: >> This patch is tiny, seems perfectly reasonable, and has plenty of >> support. I'm going to commit it shortly unless there are last minute >> objections. > +1 > done. cheers andrew -- Andrew Dunstan

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 8:41 PM, Andres Freund wrote: > Hi, > > On 2019-03-06 20:37:39 +0100, Tomas Vondra wrote: >> Not sure how to integrate it into the CLI tool, though. Perhaps we it >> could require connection info so that it can execute a function, when >> executed in online mode? > > To me the right fix

Re: [HACKERS] Incomplete startup packet errors

2019-03-06 Thread Andrew Dunstan
On 3/6/19 12:12 PM, Robert Haas wrote: > On Tue, Mar 5, 2019 at 5:35 PM Andrew Dunstan > wrote: >> OK, I think we have agreement on Tom's patch. Do we want to backpatch >> it? It's a change in behaviour, but I find it hard to believe anyone >> relies on the existence of these annoying messages,

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Tom Lane
Robert Haas writes: > OK, so this is getting simpler, but I'm wondering why we need > dlist_move_tail() at all. It is a well-known fact that maintaining > LRU ordering is expensive and it seems to be unnecessary for our > purposes here. Yeah ... LRU maintenance was another thing that used to be

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread Tomas Vondra
On 3/6/19 12:10 AM, David Rowley wrote: > Thanks for chipping in on this. > > On Wed, 6 Mar 2019 at 01:53, Tomas Vondra > wrote: >> But on the other hand it feels a bit weird that we increase this one >> value and leave all the other (also very conservative) defaults alone. > > Which others

Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andres Freund
Hi, After my tableam patch Andrew's buildfarm animal started failing in the cross-version upgrades: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2019-03-06%2019%3A32%3A24 But I actually don't think that't really fault of the tableam patch. The reason for the assertion is that

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Robert Haas
On Fri, Mar 1, 2019 at 5:06 PM Joe Conway wrote: > Seems like it would be better to modify the arguments to > CloseTransientFile() to include the filename being closed, errorlevel, > and fail_on_error or something similar. Then all the repeated ereport > stanzas could be eliminated. Hmm. I'm not

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 3:03 PM Peter Geoghegan wrote: > I agree that the parts covered by the first patch in the series are > very unlikely to need changes, but I hesitate to commit it weeks ahead > of the other patches. I know I'm stating the obvious here, but we don't have many weeks left at th

Re: Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andrew Dunstan
On 3/6/19 3:41 PM, Andres Freund wrote: > Hi, > > After my tableam patch Andrew's buildfarm animal started failing in the > cross-version upgrades: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2019-03-06%2019%3A32%3A24 Incidentally, I just fixed a bug that was preventing t

Re: Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 16:46:07 -0500, Andrew Dunstan wrote: > > On 3/6/19 3:41 PM, Andres Freund wrote: > > Hi, > > > > After my tableam patch Andrew's buildfarm animal started failing in the > > cross-version upgrades: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2019-03-06%2

Re: shared-memory based stats collector

2019-03-06 Thread Robert Haas
On Sun, Feb 24, 2019 at 11:53 PM Kyotaro HORIGUCHI wrote: > The two aboves are fixed in the attached v17. Andres just drew my attention to patch 0004 in this series, which is definitely not OK. That patch allows the postmaster to use dynamic shared memory, claiming: "Shared memory baesd stats co

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 08:54, Andrew Dunstan wrote: > > > On 3/6/19 1:38 PM, Jeremy Schneider wrote: > > On 3/5/19 14:14, Andrew Dunstan wrote: > >> This patch is tiny, seems perfectly reasonable, and has plenty of > >> support. I'm going to commit it shortly unless there are last minute > >> objec

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 03:36, Tom Lane wrote: > > David Rowley writes: > > As far as I can see, the biggest fundamental difference with doing > > things this way will be that the column order of partitions will be > > preserved, where before it would inherit the order of the partitioned > > table.

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 05:17, Andres Freund wrote: > I'm also concerned that the the current catalog representation isn't > right. As I said: > > > I also find it far from clear that: > > > > > > The tablespace_name is > > the name > > of the tablespace in which the new tabl

Re: Patch to document base64 encoding

2019-03-06 Thread Karl O. Pinc
On Wed, 6 Mar 2019 19:30:16 +0100 (CET) Fabien COELHO wrote: > "... section 6.8" -> "... Section 6.8" (capital S). Fixed. > "The string and the binary encode and decode functions..." sentence > looks strange to me, especially with the English article that I do > not really master, so maybe it i

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Andres Freund
Hi, On 2019-03-07 11:31:15 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 05:17, Andres Freund wrote: > > I'm also concerned that the the current catalog representation isn't > > right. As I said: > > > > > I also find it far from clear that: > > > > > > > > > The tablespace_

Re: shared-memory based stats collector

2019-03-06 Thread Andres Freund
Hi, > From 88740269660d00d548910c2f3aa631878c7cf0d4 Mon Sep 17 00:00:00 2001 > From: Kyotaro Horiguchi > Date: Thu, 21 Feb 2019 12:42:07 +0900 > Subject: [PATCH 4/6] Allow dsm to use on postmaster. > > DSM is inhibited to be used on postmaster. Shared memory baesd stats > collector needs it to w

Re: Pluggable Storage - Andres's take

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > Here's a cleaned up version of that patch. David, Alvaro, you also > played in that area, any objections? I think this makes that part of the > code easier to read actually. Robert, thanks for looking at that patch > already. I only had a quick

Re: Pluggable Storage - Andres's take

2019-03-06 Thread Andres Freund
Hi, On 2019-03-07 11:56:57 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > > Here's a cleaned up version of that patch. David, Alvaro, you also > > played in that area, any objections? I think this makes that part of the > > code easier to read actually. Robert,

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Jerry Jelinek
On Wed, Mar 6, 2019 at 11:02 AM Alvaro Herrera wrote: > On 2019-Mar-06, Robert Haas wrote: > > > On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera > wrote: > > > I want your dictating software. > > > > I'm afraid this is just me and a keyboard, but sadly for me you're not > > the first person to ac

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Tomas Vondra
On 3/6/19 9:17 PM, Tom Lane wrote: > Robert Haas writes: >> OK, so this is getting simpler, but I'm wondering why we need >> dlist_move_tail() at all. It is a well-known fact that maintaining >> LRU ordering is expensive and it seems to be unnecessary for our >> purposes here. > > Yeah ... LRU m

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-03-06 Thread Bruce Momjian
On Wed, Mar 6, 2019 at 10:49:17AM -0800, Jeremy Schneider wrote: > Might it make sense to generalize a little bit to secret management? It > would be *great* if PostgreSQL could have a standard "secrets" API which > could then use plugins or extensions to provide an internal > implementation (soft

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Peter Geoghegan
On Wed, Mar 6, 2019 at 1:37 PM Robert Haas wrote: > I know I'm stating the obvious here, but we don't have many weeks left > at this point. I have not reviewed any code, but I have been > following this thread and I'd really like to see this work go into > PostgreSQL 12, assuming it's in good eno

Re: pgsql: Removed unused variable, openLogOff.

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 02:47:16PM +, Robert Haas wrote: > Removed unused variable, openLogOff. Is that right for the report if data is written in chunks? The same patch has been proposed a couple of weeks ago, and I commented about it as follows: https://www.postgresql.org/message-id/2019012

RE: speeding up planning with partitions

2019-03-06 Thread Imai, Yoshikazu
Amit-san, On Wed, Mar 6, 2019 at 5:38 AM, Amit Langote wrote: > On 2019/03/06 11:09, Imai, Yoshikazu wrote: > > [0004 or 0005] > > There are redundant process in add_appendrel_other_rels (or > expand_xxx_rtentry()?). > > I modified add_appendrel_other_rels like below and it actually worked. > > >

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 02:54:52PM +, Georgios Kokolatos wrote: > Overall the patch looks good and according to the previous > discussion fulfils its purpose. > > It might be worthwhile to also check for errors on close in > SaveSlotToPath(). Thanks for the feedback, added. I have spent som

Re: [bug fix] Produce a crash dump before main() on Windows

2019-03-06 Thread Kyotaro HORIGUCHI
Hello. At Tue, 5 Mar 2019 16:45:53 +1100, Haribabu Kommi wrote in > > I don't have an idea about which is better behavior, but does > > this work for you? > > > > > > https://docs.microsoft.com/en-us/windows/desktop/wer/collecting-user-mode-dumps > > > > No, this option is not generating local

Re: Online verification of checksums

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 08:53:57PM +0100, Tomas Vondra wrote: > Not sure. AFAICS that would to require a single transaction, and if we > happen to add some sort of throttling (which is a feature request I'd > expect pretty soon to make it usable on live clusters) that might be > quite long-running.

Re: few more wait events to add to docs

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 11:08:12AM -0800, Jeremy Schneider wrote: > LWLock order in documentation: > 1) CamelCase LWLocks: individually named - see lwlocknames.txt > 2) lowercase LWLocks: tranches > 2a) SLRUs - see SimpleLruInit() callers on doxygen > 2b) Shared Buffer (buffer_content, buffer_io) >

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 11:37, Andres Freund wrote: > > On 2019-03-07 11:31:15 +1300, David Rowley wrote: > > Do you think it's fine to reword the docs to make this point more > > clear, or do you see this as a fundamental problem with the patch? > > Hm, both? I mean I wouldn't necessarily character

Re: pgsql: tableam: introduce table AM infrastructure.

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 03:03:44PM -0300, Alvaro Herrera wrote: > On 2019-Mar-06, Andres Freund wrote: >> tableam: introduce table AM infrastructure. > > Thanks for doing this!! +1. -- Michael signature.asc Description: PGP signature

  1   2   3   >