Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Amit Kapila
On Wed, Sep 16, 2020 at 9:41 AM Tom Lane wrote: > > Amit Kapila writes: > > So, can we assume that the current code can only cause the problem in > > CCA builds bot not in any practical scenario because after having a > > lock on relation probably there shouldn't be any invalidation which > >

RE: Transactions involving multiple postgres foreign servers, take 2

2020-09-15 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > The resolver process has two functionalities: resolving foreign > transactions automatically when the user issues COMMIT (the case you > described in the second paragraph), and resolving foreign transaction > when the corresponding backend no longer exist or when the

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Ashutosh Sharma
On Wed, Sep 16, 2020 at 10:40 AM Ashutosh Sharma wrote: > > On Wed, Sep 16, 2020 at 9:14 AM Tom Lane wrote: > > > > Ashutosh Sharma writes: > > > On Wed, Sep 16, 2020 at 1:25 AM Tom Lane wrote: > > >> * Should any of the other tables in the test be converted to temp? > > > > > Are you trying

Re: Parallelize stream replication process

2020-09-15 Thread Paul Guo
> On Sep 16, 2020, at 11:15 AM, Li Japin wrote: > > > >> On Sep 15, 2020, at 3:41 PM, Fujii Masao wrote: >> >> >> >> On 2020/09/15 13:41, Bharath Rupireddy wrote: >>> On Tue, Sep 15, 2020 at 9:27 AM Li Japin wrote: For now, postgres use single process to send, receive and

Re: Force update_process_title=on in crash recovery?

2020-09-15 Thread Tom Lane
Thomas Munro writes: > On Wed, Sep 16, 2020 at 2:30 PM Michael Paquier wrote: >> Another thing to be careful here is WIN32, see 0921554. And slowing >> down recovery is never a good idea. > Right, that commit makes a lot of sense because it suppresses many > system calls that happen for each

Re: Force update_process_title=on in crash recovery?

2020-09-15 Thread Thomas Munro
On Wed, Sep 16, 2020 at 2:30 PM Michael Paquier wrote: > On Tue, Sep 15, 2020 at 10:01:18AM -0400, Tom Lane wrote: > > Seems like a good argument, but you'd have to be careful about the > > final state when you stop overriding update_process_title --- it can't > > be left looking like it's

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Ashutosh Sharma
On Wed, Sep 16, 2020 at 9:14 AM Tom Lane wrote: > > Ashutosh Sharma writes: > > On Wed, Sep 16, 2020 at 1:25 AM Tom Lane wrote: > >> * Should any of the other tables in the test be converted to temp? > > > Are you trying to say that we can achieve the things being done in > > test-case 1 and 2

Re: PostgreSQL 13 RC 1 release announcement draft

2020-09-15 Thread Peter Eisentraut
On 2020-09-15 18:10, Jonathan S. Katz wrote: To upgrade to PostgreSQL 13 RC 1 from Beta 3 or an earlier version of PostgreSQL 13, you will need to use a strategy similar to upgrading between major versions of PostgreSQL (e.g. `pg_upgrade` or `pg_dump` / `pg_restore`). Is this correct? I don't

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Amit Kapila
On Wed, Sep 16, 2020 at 9:02 AM Kyotaro Horiguchi wrote: > > At Wed, 16 Sep 2020 08:33:06 +0530, Amit Kapila > wrote in > > On Wed, Sep 16, 2020 at 7:46 AM Kyotaro Horiguchi > > wrote: > > > Is this means lseek(SEEK_END) doesn't count blocks that are > > > write(2)'ed (by smgrextend) but not

RE: Transactions involving multiple postgres foreign servers, take 2

2020-09-15 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > > If so, can't we stipulate that the FDW implementor should ensure that the > commit function always returns control to the caller? > > How can the FDW implementor ensure that? Since even palloc could call > ereport(ERROR) I guess it's hard to require that to all FDW >

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Tom Lane
Amit Kapila writes: > So, can we assume that the current code can only cause the problem in > CCA builds bot not in any practical scenario because after having a > lock on relation probably there shouldn't be any invalidation which > leads to this problem? No. The reason we expend so much time

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Amit Kapila
On Wed, Sep 16, 2020 at 1:16 AM Tom Lane wrote: > > I wrote: > It's not really clear to me why setting localreloid to zero is a sane > way to represent "this entry needs to be revalidated". I think a > separate flag would be more appropriate. Once we have lock on the > target relation, it seems

Re: Online checksums verification in the backend

2020-09-15 Thread Michael Paquier
On Fri, Sep 11, 2020 at 09:49:16AM +0200, Julien Rouhaud wrote: > Thanks! I got some numbers out of my pocket, using the following base configuration: wal_level = minimal fsync = off shared_buffers = '300MB' # also tested with 30MB and 60MB checksum_cost_delay = 0 # default in patch And for the

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Tom Lane
Ashutosh Sharma writes: > On Wed, Sep 16, 2020 at 1:25 AM Tom Lane wrote: >> * Should any of the other tables in the test be converted to temp? > Are you trying to say that we can achieve the things being done in > test-case 1 and 2 by having a single temp table and we should aim for > it

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Kyotaro Horiguchi
At Wed, 16 Sep 2020 08:33:06 +0530, Amit Kapila wrote in > On Wed, Sep 16, 2020 at 7:46 AM Kyotaro Horiguchi > wrote: > > Is this means lseek(SEEK_END) doesn't count blocks that are > > write(2)'ed (by smgrextend) but not yet flushed? (I don't think so, > > for clarity.) The nblocks cache is

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Ashutosh Sharma
On Wed, Sep 16, 2020 at 1:25 AM Tom Lane wrote: > > Robert Haas writes: > > On Mon, Sep 14, 2020 at 6:26 AM Ashutosh Sharma > > wrote: > >> Thanks for reporting. I'm able to reproduce the issue by creating some > >> delay just before "-- now create an unused line pointer" and use the > >>

Re: Parallelize stream replication process

2020-09-15 Thread Li Japin
> On Sep 15, 2020, at 3:41 PM, Fujii Masao wrote: > > > > On 2020/09/15 13:41, Bharath Rupireddy wrote: >> On Tue, Sep 15, 2020 at 9:27 AM Li Japin wrote: >>> >>> For now, postgres use single process to send, receive and replay the WAL >>> when we use stream replication, >>> is there any

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Amit Kapila
On Wed, Sep 16, 2020 at 7:46 AM Kyotaro Horiguchi wrote: > > At Wed, 2 Sep 2020 08:18:06 +0530, Amit Kapila > wrote in > > On Wed, Sep 2, 2020 at 7:01 AM Kyotaro Horiguchi > > wrote: > > > Isn't a relation always locked asscess-exclusively, at truncation > > > time? If so, isn't even the

Re: Parallelize stream replication process

2020-09-15 Thread Li Japin
Thanks for clarifying the questions! > On Sep 15, 2020, at 12:41 PM, Bharath Rupireddy > wrote: > > On Tue, Sep 15, 2020 at 9:27 AM Li Japin wrote: >> >> For now, postgres use single process to send, receive and replay the WAL >> when we use stream replication, >> is there any point to

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Kyotaro Horiguchi
At Wed, 16 Sep 2020 11:56:29 +0900 (JST), Kyotaro Horiguchi wrote in (Oops! Some of my comments duplicate with Tsunakawa-san, sorry.) regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Kyotaro Horiguchi
Thanks for the new version. Jamison. At Tue, 15 Sep 2020 11:11:26 +, "k.jami...@fujitsu.com" wrote in > Hi, > > > BTW, I think I see one problem in the code: > > > > > > if (RelFileNodeEquals(bufHdr->tag.rnode, rnode.node) && > > > + bufHdr->tag.forkNum == forkNum[j] && tag.blockNum >= >

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread tsunakawa.ta...@fujitsu.com
The code doesn't seem to be working correctly. (1) + for (block_num = 0; block_num <= nblocks; block_num++) should be + for (block_num = firstDelBlock[fork_num]; block_num < nblocks; block_num++) because: * You only want to

Re: Force update_process_title=on in crash recovery?

2020-09-15 Thread Michael Paquier
On Tue, Sep 15, 2020 at 10:01:18AM -0400, Tom Lane wrote: > Seems like a good argument, but you'd have to be careful about the > final state when you stop overriding update_process_title --- it can't > be left looking like it's still-in-progress on some random WAL file. > (Compare my nearby gripes

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread Kyotaro Horiguchi
At Wed, 2 Sep 2020 08:18:06 +0530, Amit Kapila wrote in > On Wed, Sep 2, 2020 at 7:01 AM Kyotaro Horiguchi > wrote: > > Isn't a relation always locked asscess-exclusively, at truncation > > time? If so, isn't even the result of lseek reliable enough? > > > > Even if the relation is locked,

Re: Feedback on table expansion hook (including patch)

2020-09-15 Thread Euler Taveira
On Thu, 7 May 2020 at 05:11, Erik Nordström wrote: > > I am looking for feedback on the possibility of adding a table expansion > hook to PostgreSQL (see attached patch). The motivation for this is to > allow extensions to optimize table expansion. In particular, TimescaleDB > does its own table

Re: Optimising compactify_tuples()

2020-09-15 Thread Peter Geoghegan
On Tue, Sep 15, 2020 at 7:01 PM Thomas Munro wrote: > On Wed, Sep 16, 2020 at 1:30 PM David Rowley wrote: > > I also did some further performance tests of something other than > > recovery. I can also report a good performance improvement in VACUUM. > > Something around the ~25% reduction mark >

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Amit Langote
On Wed, Sep 16, 2020 at 2:41 AM Tom Lane wrote: > Amit Langote writes: > > On Tue, Sep 15, 2020 at 7:28 AM Tom Lane wrote: > >> AFAICS, it is utterly silly for InitResultRelInfo to be forcing > >> a partition qual to be computed when we might not need it. > >> We could flush

Re: Optimising compactify_tuples()

2020-09-15 Thread Thomas Munro
On Wed, Sep 16, 2020 at 1:30 PM David Rowley wrote: > Thanks a lot for the detailed benchmark results and profiles. That was > useful. I've pushed both patches now. I did a bit of a sweep of the > comments on the 0001 patch before pushing it. > > I also did some further performance tests of

Re: Optimising compactify_tuples()

2020-09-15 Thread David Rowley
On Wed, 16 Sep 2020 at 02:10, Jakub Wartak wrote: > BTW: this message "redo done at 0/9749FF70 system usage: CPU: user: 13.46 s, > system: 0.78 s, elapsed: 14.25 s" is priceless addition :) Thanks a lot for the detailed benchmark results and profiles. That was useful. I've pushed both patches

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Tom Lane
Alvaro Herrera writes: > They get moderated because the nore...@postgresql.org address which > appears as sender is not subscribed to any list. Ah-hah. > I also added that > address to the whitelist now, but whether that's a great fix in the long > run is debatable. Can/should we change the

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Tom Lane
I wrote: > With this, we get through 013_partition.pl under CCA. I plan to > try to run all of subscription/ and recovery/ before concluding > there's nothing else to fix, though. Looks like the rest passes. FTR, it was possible to get through subscription/ in about 2 hours on my workstation,

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Alvaro Herrera
On 2020-Sep-15, Tom Lane wrote: > Peter Geoghegan writes: > > On Tue, Sep 15, 2020 at 3:02 PM Tom Lane wrote: > >> The tag is applied, though for some reason the pgsql-committers auto > >> e-mail about new tags hasn't been working lately. > > > Thanks. FWIW I did get the automated email

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Sep 15, 2020 at 3:02 PM Tom Lane wrote: >> The tag is applied, though for some reason the pgsql-committers auto >> e-mail about new tags hasn't been working lately. > Thanks. FWIW I did get the automated email shortly after you sent this email. Yeah, it did

Re: DROP relation IF EXISTS Docs and Tests - Bug Fix

2020-09-15 Thread Alexander Korotkov
Hi! I've skimmed through the thread and checked the patchset. Everything looks good, except one paragraph, which doesn't look completely clear. + + This emulates the functionality provided by +but sets the created object's + type definition + to domain. + As I get it states that

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Peter Geoghegan
On Tue, Sep 15, 2020 at 3:02 PM Tom Lane wrote: > The tag is applied, though for some reason the pgsql-committers auto > e-mail about new tags hasn't been working lately. Thanks. FWIW I did get the automated email shortly after you sent this email. -- Peter Geoghegan

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Tom Lane
I wrote: > I plan to tag rc1 in around six hours, ~2200UTC today, barring > trouble reports from packagers (none so far). Feel free to push your > patch once the tag appears. The tag is applied, though for some reason the pgsql-committers auto e-mail about new tags hasn't been working lately.

Re: "Unified logging system" breaks access to pg_dump debug outputs

2020-09-15 Thread Alvaro Herrera
On 2020-Sep-15, Tom Lane wrote: > I wrote: > > Alternatively, we might consider inventing an additional logging.c > > function pg_logging_increase_level() with the obvious semantics, and > > make the various programs just call that when they see a -v switch. > > That would be a slightly bigger

Re: Gripes about walsender command processing

2020-09-15 Thread Alvaro Herrera
On 2020-Sep-15, Tom Lane wrote: > [ blink ... ] So why is it that DropReplicationSlot does > > SetQueryCompletion(, CMDTAG_DROP_REPLICATION_SLOT, 0); > EndCommand(, DestRemote, false); > > when the caller will immediately after that do > > SetQueryCompletion(, CMDTAG_SELECT,

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Tom Lane
I wrote: > It's not really clear to me why setting localreloid to zero is a sane > way to represent "this entry needs to be revalidated". I think a > separate flag would be more appropriate. Once we have lock on the > target relation, it seems to me that no interesting changes should > be

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-09-15 Thread Robert Haas
On Tue, Sep 15, 2020 at 2:04 PM Andres Freund wrote: > > How is it possible? Because tuple which has a committed xmax and the > > xmax is older than the oldestXmin, should not come for freezing unless > > it is lock_only xid (because those tuples are already gone). > > There've been several

Re: Force update_process_title=on in crash recovery?

2020-09-15 Thread Justin Pryzby
On Tue, Sep 15, 2020 at 10:01:18AM -0400, Tom Lane wrote: > Thomas Munro writes: > > Based on a couple of independent reports from users with no idea how > > to judge the progress of a system recovering from a crash, Christoph > > and I wondered if we should override update_process_title for the

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Tom Lane
Robert Haas writes: > On Mon, Sep 14, 2020 at 6:26 AM Ashutosh Sharma wrote: >> Thanks for reporting. I'm able to reproduce the issue by creating some >> delay just before "-- now create an unused line pointer" and use the >> delay to start a new session either with repeatable read or >>

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Tom Lane
I wrote: > [ $subject ] I found some time to trace this down, and what it turns out to be is that apply_handle_truncate() is making use of a LogicalRepRelMapEntry's localreloid field without any consideration for the possibility that that's been set to zero as a result of a cache flush. The

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-15 Thread Robert Haas
On Mon, Sep 14, 2020 at 6:26 AM Ashutosh Sharma wrote: > Thanks for reporting. I'm able to reproduce the issue by creating some > delay just before "-- now create an unused line pointer" and use the > delay to start a new session either with repeatable read or > serializable transaction isolation

Re: Simplified version of read_binary_file (src/backend/utils/adt/genfile.c)

2020-09-15 Thread Ranier Vilela
Em ter., 15 de set. de 2020 às 14:54, Alvaro Herrera < alvhe...@2ndquadrant.com> escreveu: > I think you meant _IONBF instead of _IOFBF -- otherwise it's at odds > with the comment you add. But what is the justification for that > addition? I don't see us doing that anywhere else. > No. _IOFBF

Re: Gripes about walsender command processing

2020-09-15 Thread Tom Lane
Michael Paquier writes: > On Mon, Sep 14, 2020 at 11:34:44PM -0400, Tom Lane wrote: >> (I don't quite follow your comment about repslot drop, btw.) > There is already a command tag equivalent to DROP_REPLICATION_SLOT: > $ git grep REPLICATION -- */cmdtaglist.h >

Re: [HACKERS] [PATCH] Generic type subscripting

2020-09-15 Thread Pavel Stehule
st 9. 9. 2020 v 23:04 odesílatel Justin Pryzby napsal: > On Wed, Aug 05, 2020 at 04:04:22PM +0200, Dmitry Dolgov wrote: > > > On Sun, Aug 02, 2020 at 12:50:12PM +0200, Pavel Stehule wrote: > > > > > > > > > Maybe this could be salvaged by flushing 0005 in its current form > and > > > > > having

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 2:30 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 2:11 PM, Tom Lane wrote: >>> The other incompatibilities are only listed once, if they're minor. >>> I was just about to commit the attached. > >> Even better. +1 > > Pushed, thanks for looking. Thanks for

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Peter Geoghegan
On Mon, Sep 14, 2020 at 11:44 PM Jeff Davis wrote: > Attached. HashAgg seems to accurately reflect the filesize, as does > Sort. For the avoidance of doubt: I think that this is the right way to go, and that it should be committed shortly, before we stamp 13.0. The same goes for

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/15/20 2:11 PM, Tom Lane wrote: >> The other incompatibilities are only listed once, if they're minor. >> I was just about to commit the attached. > Even better. +1 Pushed, thanks for looking. regards, tom lane

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 2:11 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 1:05 PM, Tom Lane wrote: >>> After thinking a bit, I'm inclined to agree that we should move it >>> to "Incompatibilities". It is a core server change (third-party >>> extensions don't have a choice to opt out, as per

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/15/20 1:05 PM, Tom Lane wrote: >> After thinking a bit, I'm inclined to agree that we should move it >> to "Incompatibilities". It is a core server change (third-party >> extensions don't have a choice to opt out, as per postgis' issue), >> and even if it's

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 1:05 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 12:05 PM, Tom Lane wrote: >>> Ah. OK, we can certainly extend it to mention that. How about >>> (not bothering with markup yet) >>> >>> Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) >>> +

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-09-15 Thread Andres Freund
Hi, On 2020-09-15 12:52:25 +0530, Dilip Kumar wrote: > On Tue, Sep 15, 2020 at 11:14 AM Andres Freund wrote: > > > > On 2020-09-15 10:54:29 +0530, Dilip Kumar wrote: > > > What problem do you see if we set xmax to the InvalidTransactionId and > > > HEAP_XMAX_INVALID flag in the infomask ? > > >

Re: Simplified version of read_binary_file (src/backend/utils/adt/genfile.c)

2020-09-15 Thread Alvaro Herrera
I think you meant _IONBF instead of _IOFBF -- otherwise it's at odds with the comment you add. But what is the justification for that addition? I don't see us doing that anywhere else. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: Simplified version of read_binary_file (src/backend/utils/adt/genfile.c)

2020-09-15 Thread Ranier Vilela
Em sex., 11 de set. de 2020 às 18:43, Ranier Vilela escreveu: > Em sex., 11 de set. de 2020 às 17:44, Tom Lane > escreveu: > >> Ranier Vilela writes: >> > New version, with support to read Virtual File (pipe, FIFO and socket). >> > With assert, in case, erroneous, of trying to read a pipe,

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Tom Lane
Amit Langote writes: > On Tue, Sep 15, 2020 at 7:28 AM Tom Lane wrote: >> AFAICS, it is utterly silly for InitResultRelInfo to be forcing >> a partition qual to be computed when we might not need it. >> We could flush ResultRelInfo.ri_PartitionCheck altogether and >> have anything that was

Re: Yet another fast GiST build

2020-09-15 Thread Heikki Linnakangas
On 15/09/2020 19:46, Andrey M. Borodin wrote: 15 сент. 2020 г., в 16:36, Heikki Linnakangas написал(а): Another patch version, fixed a few small bugs pointed out by assertion failures in the regression tests. - Heikki These changes in create_index.out do not seem correct to me

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/15/20 12:05 PM, Tom Lane wrote: >> Ah. OK, we can certainly extend it to mention that. How about >> (not bothering with markup yet) >> >> Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) >> + >> +The FROM option of CREATE EXTENSION is

Re: "Unified logging system" breaks access to pg_dump debug outputs

2020-09-15 Thread Tom Lane
I wrote: > Alternatively, we might consider inventing an additional logging.c > function pg_logging_increase_level() with the obvious semantics, and > make the various programs just call that when they see a -v switch. > That would be a slightly bigger patch, but it would more easily support >

Re: Yet another fast GiST build

2020-09-15 Thread Andrey M. Borodin
> 15 сент. 2020 г., в 16:36, Heikki Linnakangas написал(а): > > Another patch version, fixed a few small bugs pointed out by assertion > failures in the regression tests. > > - Heikki > These changes in create_index.out do not seem correct to me SELECT * FROM point_tbl ORDER BY f1 <->

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 12:05 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 11:45 AM, Tom Lane wrote: >>> It is listed already in the "Additional Modules" section (about line 2940 >>> in release-13.sgml as of right now). > >> ...sort of. It talks about the feature, but does not talk about

PostgreSQL 13 RC 1 release announcement draft

2020-09-15 Thread Jonathan S. Katz
Hi, Attached is a draft of the PostgreSQL 13 RC 1 release announcement. If you have feedback, please be sure you've left it no later than 2020-09-16 AoE. Thanks! Jonathan PostgreSQL 13 RC 1 Released === The PostgreSQL Global Development Group announces that the first

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Sep 14, 2020 at 8:48 PM Peter Geoghegan wrote: >> On Mon, Sep 14, 2020 at 8:07 PM Jeff Davis wrote: >>> Sure. Will backporting either patch into REL_13_STABLE now interfere >>> with RC1 release in any way? > It is okay to skip RC1 and commit the patch/patches

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/15/20 11:45 AM, Tom Lane wrote: >> It is listed already in the "Additional Modules" section (about line 2940 >> in release-13.sgml as of right now). > ...sort of. It talks about the feature, but does not talk about the > syntax removal, which is what I was

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-15 Thread Peter Geoghegan
On Mon, Sep 14, 2020 at 8:48 PM Peter Geoghegan wrote: > On Mon, Sep 14, 2020 at 8:07 PM Jeff Davis wrote: > > Sure. Will backporting either patch into REL_13_STABLE now interfere > > with RC1 release in any way? > > The RMT will discuss this. It is okay to skip RC1 and commit the patch/patches

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 11:45 AM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On a different note, I became aware of this[1] and noticed that dropping >> "CREATE EXTENSION ... FROM" was not listed in the incompatibilities >> section, so proposing the attached. I have no strong opinions on the >> final

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On a different note, I became aware of this[1] and noticed that dropping > "CREATE EXTENSION ... FROM" was not listed in the incompatibilities > section, so proposing the attached. I have no strong opinions on the > final wording, mainly wanted to get it listed. It

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 9:49 AM, Jonathan S. Katz wrote: > On 9/15/20 5:22 AM, Masahiko Sawada wrote: >> On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut >> wrote: >>> >>> On 2020-09-09 22:57, Jonathan S. Katz wrote: + + + Parallelized vacuuming of B-tree indexes + +

Re: PG 13 release notes, first draft

2020-09-15 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/15/20 5:22 AM, Masahiko Sawada wrote: >> On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut >> wrote: >>> I don't think B-tree indexes are relevant here. AFAICT, this feature >>> applies to all indexes. > I'm not sure where I got B-tree from. I've attached a

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Tom Lane
Amit Langote writes: > On Tue, Sep 15, 2020 at 10:47 PM Tom Lane wrote: >> Ah, right. That seems like a bug but we have not attempted to fix it. > IIRC, when this behavior was brought up as a bug in past discussions, > it was decided that it will be fixed when NOT NULL constraints are >

Re: Optimising compactify_tuples()

2020-09-15 Thread Jakub Wartak
David Rowley wrote: > I've attached patches in git format-patch format. I'm proposing to commit > these in about 48 hours time unless there's some sort of objection before > then. Hi David, no objections at all, I've just got reaffirming results here, as per [1] (SLRU thread but combined

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Amit Langote
On Tue, Sep 15, 2020 at 10:47 PM Tom Lane wrote: > Amit Langote writes: > > On Tue, Sep 15, 2020 at 9:09 AM Tom Lane wrote: > >> I wrote a quick patch for this part. It seems pretty safe and probably > >> could be back-patched without fear. > > > The patch's theory that if the parent column

Re: Force update_process_title=on in crash recovery?

2020-09-15 Thread Tom Lane
Thomas Munro writes: > Based on a couple of independent reports from users with no idea how > to judge the progress of a system recovering from a crash, Christoph > and I wondered if we should override update_process_title for the > "recovering ..." message, at least until connections are

Re: Parallel copy

2020-09-15 Thread Bharath Rupireddy
On Fri, Sep 11, 2020 at 3:49 AM Greg Nancarrow wrote: > > I couldn't use the original machine from which I obtained the previous > results, but ended up using a 4-core CentOS7 VM, which showed a > similar pattern in the performance results for this test case. > I obtained the following results

Re: PG 13 release notes, first draft

2020-09-15 Thread Jonathan S. Katz
On 9/15/20 5:22 AM, Masahiko Sawada wrote: > On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut > wrote: >> >> On 2020-09-09 22:57, Jonathan S. Katz wrote: >>> + >>> + >>> + Parallelized vacuuming of B-tree indexes >>> + >>> + >> >> I don't think B-tree indexes are relevant

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Tom Lane
Amit Langote writes: > On Tue, Sep 15, 2020 at 9:09 AM Tom Lane wrote: >> I wrote a quick patch for this part. It seems pretty safe and probably >> could be back-patched without fear. > The patch's theory that if the parent column has NOT NULL set then it > must be set in child tables too does

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-15 Thread Ashutosh Bapat
On Fri, Sep 11, 2020 at 4:37 PM Masahiko Sawada wrote: > > Considering the design without the resolver process, I think we can > easily replace the latter with the manual resolution. OTOH, it's not > easy for the former. I have no idea about better design for now, > although, as you described, if

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-15 Thread Amit Kapila
On Tue, Sep 15, 2020 at 5:27 PM Ajin Cherian wrote: > > On Sat, Sep 12, 2020 at 9:40 PM Amit Kapila wrote: >> >> >> Another thing, I noticed is that originally we have subscriber-side >> support as well, see [1] (see *pgoutput* patch) but later dropped it >> due to some reasons [2]. I think we

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-15 Thread Amit Kapila
On Wed, Sep 9, 2020 at 3:33 PM Ajin Cherian wrote: > > On Mon, Sep 7, 2020 at 11:17 PM Amit Kapila wrote: >> >> >> Nikhil has a test for the same >> (0004-Teach-test_decoding-plugin-to-work-with-2PC.Jan4) in his last >> email [1]. You might want to use it to test this behavior. I think you >>

Re: Range checks of pg_test_fsync --secs-per-test and pg_test_timing --duration

2020-09-15 Thread Peter Eisentraut
On 2020-09-11 09:08, Michael Paquier wrote: On Thu, Sep 10, 2020 at 03:59:20PM +0200, Peter Eisentraut wrote: The first patch you proposed checks for errno == ERANGE, but pgbench code doesn't do that. So one of them is not correct. Sorry for the confusion, I misunderstood what you were

Re: Collation versioning

2020-09-15 Thread Peter Eisentraut
On 2020-09-08 16:45, Julien Rouhaud wrote: I usually agree with that approach, I'm just afraid that getting a consensus on the best way to do that will induce a lot of discussions, while this is probably a corner case due to general usage of hash and bloom indexes. Anyway, in order to make

Force update_process_title=on in crash recovery?

2020-09-15 Thread Thomas Munro
Hi, Based on a couple of independent reports from users with no idea how to judge the progress of a system recovering from a crash, Christoph and I wondered if we should override update_process_title for the "recovering ..." message, at least until connections are allowed. We already do that to

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-15 Thread Ajin Cherian
On Sat, Sep 12, 2020 at 9:40 PM Amit Kapila wrote: > > Another thing, I noticed is that originally we have subscriber-side > support as well, see [1] (see *pgoutput* patch) but later dropped it > due to some reasons [2]. I think we should have pgoutput support as > well, so see what is required

Re: Use incremental sort paths for window functions

2020-09-15 Thread David Rowley
On Tue, 15 Sep 2020 at 23:21, David Rowley wrote: > > On Tue, 15 Sep 2020 at 20:12, Daniel Gustafsson wrote: > > > > No comments on this version, LGTM. > > Cool. Many thanks for having a look. Pushed. 62e221e1c David

Standardize the printf format for st_size

2020-09-15 Thread Peter Eisentraut
Existing code uses various inconsistent ways to printf struct stat's st_size member. The type of that is off_t, which is in most cases a signed 64-bit integer, so use the long long int format for it. I don't think anything is affected by this in practice, but it seems reasonable to set a

Re: Yet another fast GiST build

2020-09-15 Thread Heikki Linnakangas
On 11/09/2020 09:02, Andrey M. Borodin wrote: 10 сент. 2020 г., в 17:43, Heikki Linnakangas написал(а): One more patch version attached. I fixed some memory leaks, and fixed the abbreviation on 32-bit systems, and a bunch of small comment changes and such. The patch looks fine to me. On my

Re: Use incremental sort paths for window functions

2020-09-15 Thread David Rowley
On Tue, 15 Sep 2020 at 20:12, Daniel Gustafsson wrote: > > On that note, assume we have the below scenario: > > wfunc .. (order by a), .. (order by a,b), .. (order by a,b,c) > > Currently the windows will be ordered such that a,b,c is sorted first, with > a,b > and a not having to sort. I

Re: pg_restore causing deadlocks on partitioned tables

2020-09-15 Thread Amit Langote
On Tue, Sep 15, 2020 at 7:28 AM Tom Lane wrote: > I wrote: > > However, the deadlock report suggests, and manual experimentation > > confirms, that > > > (1) TRUNCATE on a partition tries to get AccessShareLock on the parent; > > The reason for this is that > > (a) ExecuteTruncateGuts calls

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-15 Thread k.jami...@fujitsu.com
Hi, > BTW, I think I see one problem in the code: > > > > if (RelFileNodeEquals(bufHdr->tag.rnode, rnode.node) && > > + bufHdr->tag.forkNum == forkNum[j] && tag.blockNum >= > > + bufHdr->firstDelBlock[j]) > > > > Here, I think you need to use 'i' not 'j' for forkNum and > > firstDelBlock as

Re: [PATCH] Remove useless distinct clauses

2020-09-15 Thread David Rowley
On Fri, 31 Jul 2020 at 20:41, Pierre Ducroquet wrote: > > In a recent audit, I noticed that application developers have a tendency to > abuse the distinct clause. For instance they use an ORM and add a distinct at > the top level just because they don't know the cost it has, or they don't know >

Re: ECPG: proposal for new DECLARE STATEMENT

2020-09-15 Thread Michael Meskes
> This patch has now been silent for quite a while, unless someone is > interested > enough to bring it forward it seems about time to close it. I am interested but still short on time. I will definitely look into it as soon as I find some spare minutes. Michael -- Michael Meskes Michael at

Re: ECPG: proposal for new DECLARE STATEMENT

2020-09-15 Thread Daniel Gustafsson
> On 30 Mar 2020, at 18:53, David Steele wrote: > > On 1/11/20 10:41 PM, Tomas Vondra wrote: >> On Sun, Jan 12, 2020 at 03:52:48AM +0100, Tomas Vondra wrote: >>> ... >>> >>> I'm not an ecpg expert (in fact I've never even used it), so my review >>> is pretty superficial, but I only found a

Re: Autovacuum on partitioned table (autoanalyze)

2020-09-15 Thread Kyotaro Horiguchi
At Tue, 25 Aug 2020 14:28:20 +0200, Daniel Gustafsson wrote in > > I attach the latest patch that solves the above Werror. > > Could you please check it again? > > This version now pass the tests in the Travis pipeline as can be seen in the > link below, and is ready to be reviewed in the

Re: Unnecessary delay in streaming replication due to replay lag

2020-09-15 Thread lchch1...@sina.cn
Hello I read the code and test the patch, it run well on my side, and I have several issues on the patch. 1. When call RequestXLogStreaming() during replay, you pick timeline straightly from control file, do you think it should pick timeline from timeline history file? 2. In archive recovery

Re: PG 13 release notes, first draft

2020-09-15 Thread Masahiko Sawada
On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut wrote: > > On 2020-09-09 22:57, Jonathan S. Katz wrote: > > + > > + > > + Parallelized vacuuming of B-tree indexes > > + > > + > > I don't think B-tree indexes are relevant here. AFAICT, this feature > applies to all indexes.

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-15 Thread Fujii Masao
On 2020/07/17 21:02, Bharath Rupireddy wrote: On Tue, Jul 14, 2020 at 6:13 PM Ashutosh Bapat wrote: Has this been added to CF, possibly next CF? I have not added yet. Request to get it done in this CF, as the final patch for review(v3 patch) is ready and shared. We can target it to the

Re: [patch] _bt_binsrch* improvements - equal-prefix-skip binary search

2020-09-15 Thread Matthias van de Meent
On Fri, 11 Sep 2020 at 19:41, Peter Geoghegan wrote: > > Are you familiar with this thread? > > https://www.postgresql.org/message-id/cah2-wzn_nayk4pr0hrwo0stwhmxjp5qyu+x8vppt030xpqr...@mail.gmail.com > > I wrote a patch that implemented the same idea, which is sometimes > called dynamic prefix

Re: Avoid useless retrieval of defaults and check constraints in pg_dump -a

2020-09-15 Thread Julien Rouhaud
On Tue, Sep 15, 2020 at 4:48 AM Michael Paquier wrote: > > On Thu, Sep 10, 2020 at 02:31:32PM +0200, Daniel Gustafsson wrote: > > Given how unintrusive this optimization is, +1 from me to go ahead with this > > patch. pg_dump tests pass. Personally I would've updated the nearby > > comments >

Re: Subscription test 013_partition.pl fails under CLOBBER_CACHE_ALWAYS

2020-09-15 Thread Neha Sharma
Hi Tom, I have tested the subscription test 013_partition.pl with CCA enabled on HEAD and PG13, and I am able to reproduce the issue on both the versions. *Logs:* [centos@clobber-cache subscription]$ git branch * REL_13_STABLE master [centos@clobber-cache-db93 subscription]$ tail -f

  1   2   >