Re: adding partitioned tables to publications

2019-11-10 Thread Amit Langote
On Fri, Nov 8, 2019 at 1:27 PM Amit Langote wrote: > Anyway, I've attached two patches -- 0001 is a refactoring patch. 0002 > implements the feature. 0002 didn't contain necessary pg_dump changes, which fixed in the attached new version. Thanks, Amit

Re: 'Invalid lp' during heap_xlog_delete

2019-11-10 Thread Michael Paquier
On Fri, Nov 08, 2019 at 06:44:08PM -0800, Daniel Wood wrote: > I repro'ed on PG11 and PG10 STABLE but several months old. > I looked at 6d05086 but it doesn't address the core issue. > > DropRelFileNodeBuffers prevents the checkpoint from writing all > needed dirty pages for any REDO's that exist

Re: [HACKERS] Block level parallel vacuum

2019-11-10 Thread Dilip Kumar
On Mon, Nov 11, 2019 at 12:26 PM Masahiko Sawada wrote: > > On Mon, 11 Nov 2019 at 15:06, Dilip Kumar wrote: > > > > On Mon, Nov 11, 2019 at 9:57 AM Masahiko Sawada > > wrote: > > > > > > On Fri, 8 Nov 2019 at 18:48, Amit Kapila wrote: > > > > > > > > On Tue, Oct 29, 2019 at 12:37 PM Masahiko

Re: tableam vs. TOAST

2019-11-10 Thread Ashutosh Sharma
Hi Craig, Please find my response inline below. On Sun, Nov 10, 2019 at 2:39 PM Craig Ringer wrote: > > On Thu, 7 Nov 2019 at 22:45, Ashutosh Sharma wrote: >> > > In fact, I suspect this is PostgreSQL successfully protecting itself from an > unsafe situation. > > Does the host have

Re: cost based vacuum (parallel)

2019-11-10 Thread Dilip Kumar
On Mon, Nov 11, 2019 at 9:43 AM Dilip Kumar wrote: > > On Fri, Nov 8, 2019 at 11:49 AM Amit Kapila wrote: > > > > On Fri, Nov 8, 2019 at 9:39 AM Dilip Kumar wrote: > > > > > > I have done some experiments on this line. I have first produced a > > > case where we can show the problem with the

Re: pg_waldump and PREPARE

2019-11-10 Thread Michael Paquier
On Mon, Nov 11, 2019 at 01:21:28PM +0900, Fujii Masao wrote: > Thanks for the review! You are right. > I fixed this issue in the attached patch. The proposed format looks fine to me. I have just one comment. All three callers of standby_desc_invalidations() don't actually need to print any data

Re: [HACKERS] Block level parallel vacuum

2019-11-10 Thread Masahiko Sawada
On Mon, 11 Nov 2019 at 15:06, Dilip Kumar wrote: > > On Mon, Nov 11, 2019 at 9:57 AM Masahiko Sawada > wrote: > > > > On Fri, 8 Nov 2019 at 18:48, Amit Kapila wrote: > > > > > > On Tue, Oct 29, 2019 at 12:37 PM Masahiko Sawada > > > wrote: > > > > > > > > I realized that v31-0006 patch

Re: [HACKERS] Block level parallel vacuum

2019-11-10 Thread Dilip Kumar
On Mon, Nov 11, 2019 at 9:57 AM Masahiko Sawada wrote: > > On Fri, 8 Nov 2019 at 18:48, Amit Kapila wrote: > > > > On Tue, Oct 29, 2019 at 12:37 PM Masahiko Sawada > > wrote: > > > > > > I realized that v31-0006 patch doesn't work fine so I've attached the > > > updated version patch that also

Re: Ordering of header file inclusion

2019-11-10 Thread Amit Kapila
On Sun, Nov 10, 2019 at 5:30 PM vignesh C wrote: > [review_latest_patch]: Do we want to consider the ordering of map file inclusions as well (see the changes pointed out below)? If so, what all we should validate, is compilation of these modules sufficient? Tom, anyone, do you have any opinion

Re: MarkBufferDirtyHint() and LSN update

2019-11-10 Thread Michael Paquier
On Thu, Oct 31, 2019 at 09:43:47AM +0100, Antonin Houska wrote: > Tomas Vondra wrote: >> Isn't this prevented by locking of the buffer header? Both FlushBuffer >> and MarkBufferDirtyHint do obtain that lock. I see MarkBufferDirtyHint >> does a bit of work before, but that's related to

Re: [HACKERS] Block level parallel vacuum

2019-11-10 Thread Masahiko Sawada
On Fri, 8 Nov 2019 at 18:48, Amit Kapila wrote: > > On Tue, Oct 29, 2019 at 12:37 PM Masahiko Sawada > wrote: > > > > I realized that v31-0006 patch doesn't work fine so I've attached the > > updated version patch that also incorporated some comments I got so > > far. Sorry for the

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-11-10 Thread Thomas Munro
On Sat, Nov 9, 2019 at 8:41 AM Ashwin Agrawal wrote: > On Thu, Nov 7, 2019 at 8:44 PM Thomas Munro wrote: >> I'm planning to commit these three patches on Monday. I've attached >> versions with whitespace-only changes from pgindent, and commit >> messages lightly massaged and updated to point

Re: pg_waldump and PREPARE

2019-11-10 Thread Fujii Masao
On Fri, Nov 8, 2019 at 1:33 PM Andrey Lepikhov wrote: > > > > On 08/11/2019 09:26, Kyotaro Horiguchi wrote: > > Hello. > > > > At Fri, 8 Nov 2019 08:23:41 +0500, Andrey Lepikhov > > wrote in > >>> Can I switch the status back to "Needs review"? > >>> Regards, > >>> > >> > >> One issue is that

Re: cost based vacuum (parallel)

2019-11-10 Thread Dilip Kumar
On Fri, Nov 8, 2019 at 11:49 AM Amit Kapila wrote: > > On Fri, Nov 8, 2019 at 9:39 AM Dilip Kumar wrote: > > > > I have done some experiments on this line. I have first produced a > > case where we can show the problem with the existing shared costing > > patch (worker which is doing less I/O

Re: Coding in WalSndWaitForWal

2019-11-10 Thread Amit Kapila
On Mon, Nov 11, 2019 at 7:53 AM Michael Paquier wrote: > > On Sun, Nov 10, 2019 at 10:43:33AM +0530, Amit Kapila wrote: > > On Sun, Nov 10, 2019 at 5:51 AM Jeff Janes wrote: > >> in src/backend/replication/walsender.c, there is the section > >> quoted below. It looks like nothing interesting

Re: CountDBSubscriptions check in dropdb

2019-11-10 Thread Amit Kapila
On Mon, Nov 11, 2019 at 6:43 AM Michael Paquier wrote: > > On Sun, Nov 10, 2019 at 08:48:27AM +0530, Amit Kapila wrote: > > I am planning to go with Peter's suggestion and will push in > > HEAD-only. So, I think that should be fine. > > I was just looking at this thread, and my take would be to

Re: dropdb --force

2019-11-10 Thread Amit Kapila
On Fri, Nov 8, 2019 at 4:57 PM Pavel Stehule wrote: > > pá 8. 11. 2019 v 11:50 odesílatel Amit Kapila > napsal: >> >> Thanks, but are you planning to look at the other thread mentioned in >> my previous email? We need to finish that before this. > > > I check it now - it has sense. the

Re: Collation versions on Windows (help wanted, apply within)

2019-11-10 Thread Thomas Munro
On Sat, Nov 9, 2019 at 10:20 AM Juan José Santamaría Flecha wrote: >> Do any Windows hackers want to help get it into shape? Some things to >> do: test it, verify that the _WIN32_WINNT >= 0x0600 stuff makes sense >> (why do we target such ancient Windows releases anyway?) > > You have to keep in

Re: Ordering of header file inclusion

2019-11-10 Thread Amit Kapila
On Sun, Nov 10, 2019 at 5:30 PM vignesh C wrote: > > On Fri, Nov 8, 2019 at 2:22 PM Kuntal Ghosh > wrote: > > > > On Sat, Nov 2, 2019 at 7:42 AM vignesh C wrote: > > > > > > > > > > Thanks Amit for committing the changes. > > > I found couple of more inconsistencies, the attached patch

Re: Coding in WalSndWaitForWal

2019-11-10 Thread Michael Paquier
On Sun, Nov 10, 2019 at 10:43:33AM +0530, Amit Kapila wrote: > On Sun, Nov 10, 2019 at 5:51 AM Jeff Janes wrote: >> in src/backend/replication/walsender.c, there is the section >> quoted below. It looks like nothing interesting happens between >> the GetFlushRecPtr just before the loop starts,

Re: CountDBSubscriptions check in dropdb

2019-11-10 Thread Michael Paquier
On Sun, Nov 10, 2019 at 08:48:27AM +0530, Amit Kapila wrote: > I am planning to go with Peter's suggestion and will push in > HEAD-only. So, I think that should be fine. I was just looking at this thread, and my take would be to just apply that on HEAD. Good catch by the way. -- Michael

Re: psql \d for wide tables / pattern for individual columns

2019-11-10 Thread Euler Taveira
Em dom., 10 de nov. de 2019 às 18:29, Justin Pryzby escreveu: > > We have some very wide tables (historically, up to 1600 columns ; this is > improved now, but sometimes still several hundred, with numerous pages output > to psql pager). Is is reasonable to suggest adding a psql command to show

Re: [bug fix] Produce a crash dump before main() on Windows

2019-11-10 Thread Michael Paquier
On Sun, Nov 10, 2019 at 11:24:34AM -0500, Tom Lane wrote: > I feel the pain on this --- indirect dependencies are not well reported > on any platform that I deal with :-(. Still, I'm having a hard time > convincing myself that this patch is a good idea. It seems very likely > that it will break

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-11-10 Thread Alexander Korotkov
Hi! I'm sorry for late reply. I was busy with various things. Also digging into these details took some time. Please find my explanation below. On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote: > In general, it seems very important to be clear about exactly how the > key space works.

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Tom Lane
Daniel Gustafsson writes: > Applies, builds clean and passes light testing. Thanks for checking! > I can see the appeal of > including it before the wrap, even though I personally would've held off. Nah, I'm not gonna risk it at this stage. I concur with your point that this is an ancient

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Daniel Gustafsson
> On 10 Nov 2019, at 22:12, Tom Lane wrote: > > Daniel Gustafsson writes: >> On 10 Nov 2019, at 20:07, Tom Lane wrote: >>> (Note: this patch is shown with --ignore-space-change >>> to make it more reviewable, but I did re-pgindent the code.) Then >>> 0002 actually adds the array and range

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Daniel Gustafsson
> On 10 Nov 2019, at 22:05, Tom Lane wrote: > Daniel Gustafsson writes: >>> + /* arrays over any type selected so far */ >>> + " SELECT >>> t.oid FROM pg_catalog.pg_type t, x WHERE typelem = x.oid AND typtype = 'b' " >

psql \d for wide tables / pattern for individual columns

2019-11-10 Thread Justin Pryzby
We have some very wide tables (historically, up to 1600 columns ; this is improved now, but sometimes still several hundred, with numerous pages output to psql pager). Is is reasonable to suggest adding a psql command to show a table's definition, without all the columns listed? Or limit display

Re: Index Skip Scan

2019-11-10 Thread Tomas Vondra
Hi, I've looked at the patch again - in general it seems in pretty good shape, all the issues I found are mostly minor. Firstly, I'd like to point out that not all of the things I complained about in my 2019/06/23 review got addressed. Those were mostly related to formatting and code style, and

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Tom Lane
Daniel Gustafsson writes: > On 10 Nov 2019, at 20:07, Tom Lane wrote: >> (Note: this patch is shown with --ignore-space-change >> to make it more reviewable, but I did re-pgindent the code.) Then >> 0002 actually adds the array and range cases. > Was the source pgindented, but not committed,

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Tom Lane
Daniel Gustafsson writes: > On 10 Nov 2019, at 20:07, Tom Lane wrote: >> Although this is a really straightforward patch and I've tested it >> against appropriate old versions (9.1 and 9.2), I'm very hesitant >> to shove it in so soon before a release wrap. Should I do that, or >> let it wait

Re: pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Daniel Gustafsson
> On 10 Nov 2019, at 20:07, Tom Lane wrote: > 0001 refactors the code in question > so that we have only one copy not three-and-growing. The only > difference between the three copies was that one case didn't bother > to search indexes, but I judged that that wasn't an optimization we > need to

pg_upgrade fails to detect unsupported arrays and ranges

2019-11-10 Thread Tom Lane
While composing the release note entry for commits 8d48e6a72 et al (handle recursive type dependencies while checking for unsupported types in pg_upgrade), I realized that there's a huge hole in pg_upgrade's test for such cases. It looks for domains containing the unsupported type, and for

Re: ssl passphrase callback

2019-11-10 Thread Magnus Hagander
On Wed, Nov 6, 2019 at 7:24 PM Bruce Momjian wrote: > On Fri, Nov 1, 2019 at 01:57:29PM -0400, Andrew Dunstan wrote: > > > > On 11/1/19 11:01 AM, Robert Haas wrote: > > > On Thu, Oct 31, 2019 at 11:37 AM Andrew Dunstan > > > wrote: > > >> This patch provides a hook for a function that can

Re: Using multiple extended statistics for estimates

2019-11-10 Thread Tomas Vondra
On Sat, Nov 09, 2019 at 02:32:27PM -0800, Mark Dilger wrote: On 11/9/19 12:33 PM, Mark Dilger wrote: On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10

Re: Using multiple extended statistics for estimates

2019-11-10 Thread Tomas Vondra
On Sat, Nov 09, 2019 at 12:33:05PM -0800, Mark Dilger wrote: On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to

Re: [bug fix] Produce a crash dump before main() on Windows

2019-11-10 Thread Tom Lane
Michael Paquier writes: > On Sun, Nov 10, 2019 at 06:03:08PM +0800, Craig Ringer wrote: >> I had to go hunt around with Dependency Walker to figure out the actual >> missing DLL the last time I had to deal with this. > Ahah. Yes, That's exactly what I used a couple of years back on that. >

AW: Missing constant propagation in planner on hash quals causes join slowdown

2019-11-10 Thread Hans Buschmann
Thanks for looking at it. I think these two queries are equivalent, as shown by the explain. In both cases the index scan only selects tuples with xx_season=3 as shown in both explains: Index Cond: (tmaster.t1_season = 3) Index Cond: (tfact.t2_season = 3) So no tuple

Re: [HACKERS] [PATCH] Generic type subscripting

2019-11-10 Thread Dmitry Dolgov
> On Thu, Oct 31, 2019 at 05:35:28AM +0300, Nikita Glukhov wrote: > Hi. I added new 5th patch to this patch set. Thank you! > Performance was compared on 4 tables with 10M rows: > > -- [ i ] > CREATE TABLE arr_1 AS > SELECT jsonb_build_array(i)::jsonb js > FROM generate_series(1, 1000) i; >

Re: Reorderbuffer crash during recovery

2019-11-10 Thread vignesh C
On Sat, Nov 9, 2019 at 5:07 PM Amit Kapila wrote: > > On Fri, Nov 8, 2019 at 10:05 AM vignesh C wrote: > > > > On Thu, Nov 7, 2019 at 10:01 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2019-11-07 17:03:44 +0530, Amit Kapila wrote: > > > > On Thu, Nov 7, 2019 at 4:48 PM Tomas Vondra >

Re: [bug fix] Produce a crash dump before main() on Windows

2019-11-10 Thread Michael Paquier
On Sun, Nov 10, 2019 at 06:03:08PM +0800, Craig Ringer wrote: > I had to go hunt around with Dependency Walker to figure out the actual > missing DLL the last time I had to deal with this. Ahah. Yes, That's exactly what I used a couple of years back on that. These errors are hard to track and

Re: [bug fix] Produce a crash dump before main() on Windows

2019-11-10 Thread Craig Ringer
On Mon, 23 Jul 2018 at 16:45, Michael Paquier wrote: > On Mon, Jul 23, 2018 at 08:16:52AM +, Tsunakawa, Takayuki wrote: > > I guess that is due to some missing files related to the libraries > > listed in shared_preload_library. If so, no, because this patch > > relates to failure before

Re: [bug fix] Produce a crash dump before main() on Windows

2019-11-10 Thread Craig Ringer
On Wed, 18 Jul 2018 at 12:10, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > At Wed, 18 Jul 2018 11:12:06 +0800, Craig Ringer > wrote in yqtpidgg...@mail.gmail.com> > > On 26 February 2018 at 12:06, Tsunakawa, Takayuki < > > tsunakawa.ta...@jp.fujitsu.com> wrote: > > > > > From:

Re: Allow cluster_name in log_line_prefix

2019-11-10 Thread Craig Ringer
On Sun, 3 Nov 2019 at 07:22, Vik Fearing wrote: > On 31/10/2019 08:47, Fujii Masao wrote: > > On Mon, Oct 28, 2019 at 1:33 PM Craig Ringer > wrote: > >> Hi folks > >> > >> I was recently surprised to notice that log_line_prefix doesn't support > a cluster_name placeholder. I suggest adding one.

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2019-11-10 Thread Craig Ringer
On Thu, 19 Sep 2019 at 10:04, Michael Paquier wrote: > On Wed, Sep 18, 2019 at 10:37:27AM +0900, Michael Paquier wrote: > > I am attaching an updated patch for now that I would like to commit. > > Are there more comments about the shape of the patch, the name of the > > columns for the function,

Re: libpq sslpassword parameter and callback function

2019-11-10 Thread Craig Ringer
On Fri, 1 Nov 2019 at 07:27, Andrew Dunstan wrote: > > On 10/31/19 6:34 PM, Andrew Dunstan wrote: > > This time with attachment. > > > > > > On 10/31/19 6:33 PM, Andrew Dunstan wrote: > >> This patch provides for an sslpassword parameter for libpq, and a hook > >> that a client can fill in for a

Re: Handy describe_pg_lock function

2019-11-10 Thread Craig Ringer
On Sun, 10 Nov 2019 at 13:42, Tom Lane wrote: > Andres Freund writes: > > On 2019-11-08 14:49:25 +0800, Craig Ringer wrote: > >> I recently found the need to pretty-print the contents of pg_locks. So > >> here's a little helper to do it, for anyone else who happens to have > that > >> need.

Re: Allow superuser to grant passwordless connection rights on postgres_fdw

2019-11-10 Thread Craig Ringer
On Mon, 4 Nov 2019 at 12:20, Stephen Frost wrote: > Greetings, > > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > > On 11/1/19 12:58 PM, Robert Haas wrote: > > > On Thu, Oct 31, 2019 at 4:58 PM Andrew Dunstan > > > wrote: > > >> This patch allows the superuser to grant passwordless

Re: Add a GUC variable that control logical replication

2019-11-10 Thread Craig Ringer
On Wed, 18 Sep 2019 at 16:39, Quan Zongliang < zongliang.q...@postgresdata.com> wrote: > > Sybase has a feature to turn off replication at the session level: set > replication = off, which can be temporarily turned off when there is a > maintenance action on the table. Our users also want this

Re: Collation versioning

2019-11-10 Thread Thomas Munro
Some more thoughts: 1. If you create an index on an expression that includes a COLLATE or a partial index that has one in the WHERE clause, you get bogus warnings: postgres=# create table t (v text); CREATE TABLE postgres=# create index on t(v) where v > 'hello' collate "en_NZ"; WARNING: index

Re: tableam vs. TOAST

2019-11-10 Thread Craig Ringer
On Thu, 7 Nov 2019 at 22:45, Ashutosh Sharma wrote: > On Thu, Nov 7, 2019 at 7:35 PM Robert Haas wrote: > > > > On Thu, Nov 7, 2019 at 1:15 AM Ashutosh Sharma > wrote: > > > @Robert, Myself and Prabhat have tried running the test-cases that > > > caused the checkpointer process to crash

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-11-10 Thread shawn wang
Hi hackers, By arrange, I will complete the modification of the front-end tool to support TDE. Now I have completed the modification of the pg_waldump, pg_resetwal, and pg_rewind tools. My design: 1. Add two options, -D and -c, to the front-end tools. You can use -c to get a password of the user

Re: segfault in geqo on experimental gcc animal

2019-11-10 Thread Fabien COELHO
Hello Andres, I don't think there's been any relevant code changes since the last success. last success: 2019-11-09 09:20:28.346 CET [28785:1] LOG: starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc (GCC) 10.0.0 20191102 (experimental), 64-bit first failure: 2019-11-09