Re: Fix BUG #17335: Duplicate result rows in Gather node

2022-01-16 Thread Yura Sokolov
В Сб, 01/01/2022 в 15:19 +1300, David Rowley пишет: > On Fri, 31 Dec 2021 at 00:14, Yura Sokolov wrote: > > Problem: > > - Append path is created with explicitely parallel_aware = true > > - It has two child, one is trivial, other is parallel_aware = false . > > Trivial child is dropped. > > -

Re: Partial aggregates pushdown

2022-01-16 Thread Alexander Pyhalov
Julien Rouhaud писал 2022-01-14 15:16: Hi, On Mon, Nov 15, 2021 at 04:01:51PM +0300, Alexander Pyhalov wrote: I've updated patch - removed catversion dump. This version of the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_3369.log === Applying patches on top of

Re: Patch: Range Merge Join

2022-01-16 Thread Julien Rouhaud
Hi, On Wed, Nov 17, 2021 at 11:28:43PM +0100, Tomas Vondra wrote: > On 11/17/21 15:45, Thomas wrote: > > Thank you for the feedback and sorry for the oversight. I fixed the bug > > and attached a new version of the patch. > > > > Kind Regards, Thomas > > > > Am Mi., 17. Nov. 2021 um 15:03 Uhr

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-16 Thread Justin Pryzby
On Thu, Jan 06, 2022 at 10:43:37PM -0600, Justin Pryzby wrote: > On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote: > > > But in this case it really doesn't work :( > > > > > > running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL: > > > file not on PMEM: path

Re: Can there ever be out of sequence WAL files?

2022-01-16 Thread Michael Paquier
On Wed, Jan 12, 2022 at 12:23:00PM +0800, Julien Rouhaud wrote: > On Wed, Jan 12, 2022 at 01:10:25PM +0900, Michael Paquier wrote: >> xlog.c can be a good read to check the assumptions WAL replay relies >> on, with things like CheckRecoveryConsistency() or >> reachedConsistency. > > That should

Re: pg_dump/restore --no-tableam

2022-01-16 Thread Michael Paquier
On Mon, Jan 17, 2022 at 12:20:07AM -0600, Justin Pryzby wrote: > I saw that you added it to pg_dumpall. But there's a typo in --help: Thanks, fixed. -- Michael signature.asc Description: PGP signature

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-16 Thread Ilya Anfimov
On Wed, Jan 05, 2022 at 06:51:37PM -0500, Bruce Momjian wrote: > On Tue, Jan 4, 2022 at 10:22:50PM +, Finnerty, Jim wrote: [skipped] > > with the "double-xmax" representation. This would eliminate a whole > > class of coding errors and would make the code dealing with 64-bit > > XIDs

Re: pg_dump/restore --no-tableam

2022-01-16 Thread Justin Pryzby
On Mon, Jan 17, 2022 at 02:55:58PM +0900, Michael Paquier wrote: > On Tue, Jan 11, 2022 at 10:09:07PM -0600, Justin Pryzby wrote: > > I suppose you're right - I had previously renamed it from no-tableam. > > Thanks for the new version. I have noticed that support for the > option with pg_dumpall

Re: Skipping logical replication transactions on subscriber side

2022-01-16 Thread Masahiko Sawada
On Mon, Jan 17, 2022 at 2:48 PM Amit Kapila wrote: > > On Mon, Jan 17, 2022 at 9:49 AM Masahiko Sawada wrote: > > > > On Sat, Jan 15, 2022 at 7:24 PM Amit Kapila wrote: > > > > > > > > 6. > > > +static void > > > +maybe_start_skipping_changes(TransactionId xid) > > > +{ > > > +

Re: pg_dump/restore --no-tableam

2022-01-16 Thread Michael Paquier
On Tue, Jan 11, 2022 at 10:09:07PM -0600, Justin Pryzby wrote: > I suppose you're right - I had previously renamed it from no-tableam. Thanks for the new version. I have noticed that support for the option with pg_dumpall was missing, but that looks useful to me like the other switches. >

Re: A test for replay of regression tests

2022-01-16 Thread Noah Misch
On Mon, Jan 17, 2022 at 05:25:19PM +1300, Thomas Munro wrote: > Here's how it failed on fairywren, in case someone knowledgeable of > MSYS path translation etc can spot the problem: > > psql::1: ERROR: directory >

Re: Skipping logical replication transactions on subscriber side

2022-01-16 Thread Amit Kapila
On Mon, Jan 17, 2022 at 9:49 AM Masahiko Sawada wrote: > > On Sat, Jan 15, 2022 at 7:24 PM Amit Kapila wrote: > > > > > 6. > > +static void > > +maybe_start_skipping_changes(TransactionId xid) > > +{ > > + Assert(!is_skipping_changes()); > > + Assert(!in_remote_transaction); > > +

Re: row filtering for logical replication

2022-01-16 Thread Greg Nancarrow
On Sat, Jan 15, 2022 at 5:30 PM houzj.f...@fujitsu.com wrote: > > Attach the V65 patch set which addressed the above comments and Peter's > comments[1]. > I also fixed some typos and removed some unused code. > I have several minor comments for the v65-0001 patch:

Re: row filtering for logical replication

2022-01-16 Thread Peter Smith
Here are some review comments for v65-0001 (review of updates since v64-0001) ~~~ 1. src/include/commands/publicationcmds.h - rename func +extern bool contain_invalid_rfcolumn(Oid pubid, Relation relation, + List *ancestors, + AttrNumber *invalid_rfcolumn); I thought that function should be

Re: A test for replay of regression tests

2022-01-16 Thread Thomas Munro
On Sat, Jan 15, 2022 at 12:49 PM Andres Freund wrote: > On 2022-01-15 02:32:35 +1300, Thomas Munro wrote: > > 1. The way I invoke pg_regress doesn't seem to work correctly under > > the build farm client (though it works fine under make), not sure why > > yet, but reproduced here and will figure

Re: Skipping logical replication transactions on subscriber side

2022-01-16 Thread Masahiko Sawada
On Fri, Jan 14, 2022 at 9:05 PM vignesh C wrote: > > On Fri, Jan 14, 2022 at 7:49 AM Masahiko Sawada wrote: > > > > On Wed, Jan 12, 2022 at 11:10 PM vignesh C wrote: > > > > > > On Wed, Jan 12, 2022 at 11:32 AM Masahiko Sawada > > > wrote: > > > > > > > > On Wed, Jan 12, 2022 at 12:21 PM Amit

Re: Skipping logical replication transactions on subscriber side

2022-01-16 Thread Masahiko Sawada
On Sat, Jan 15, 2022 at 7:24 PM Amit Kapila wrote: > > On Fri, Jan 14, 2022 at 7:49 AM Masahiko Sawada wrote: > > > > I agree with all the comments above. I've attached an updated patch. > > > > Review comments > Thank you for the comments! > 1. > + > + > + In this case,

Re: Null commitTS bug

2022-01-16 Thread Tom Lane
Michael Paquier writes: > On Mon, Jan 17, 2022 at 11:17:24AM +0900, Kyotaro Horiguchi wrote: >> I found some confusing lines around but they need not a fix >> considering back-patching conflict? >>> i += j - i + 1; > I am not sure. Do you have anything specific in mind? Perhaps > something

Re: Add sub-transaction overflow status in pg_stat_activity

2022-01-16 Thread Dilip Kumar
On Fri, Jan 14, 2022 at 9:55 PM Tom Lane wrote: > Julien Rouhaud writes: > > Like many I previously had to investigate a slowdown due to > sub-transaction > > overflow, and even with the information available in a monitoring view > (I had > > to rely on a quick hackish extension as I couldn't

Re: Error "initial slot snapshot too large" in create replication slot

2022-01-16 Thread Dilip Kumar
On Wed, Jan 12, 2022 at 4:09 PM Julien Rouhaud wrote: > Hi, > > On Tue, Nov 02, 2021 at 04:40:39PM +0530, Dilip Kumar wrote: > > > > I have fixed this, the updated patch is attached. > > The cfbot reports that this patch doesn't compile: > https://cirrus-ci.com/task/564273490432?logs=build >

Re: Refactoring of compression options in pg_basebackup

2022-01-16 Thread Michael Paquier
On Sat, Jan 15, 2022 at 04:15:26PM +0100, Magnus Hagander wrote: > I think having --client-compress and --server-compress separately but > having --compression-level *not* being separate would be confusing and > I *think* that's what the current patch proposes? Yep, your understanding is right.

Re: Null commitTS bug

2022-01-16 Thread Michael Paquier
On Mon, Jan 17, 2022 at 11:17:24AM +0900, Kyotaro Horiguchi wrote: > At Fri, 14 Jan 2022 22:49:59 +, "Kingsborough, Alex" > wrote in >> The fix for this is very simple >> >> >> /* if we wrote out all subxids, we're done. / >> - if (j + 1 >= nsubxids) >> + if (j >= nsubxids) >>

Re: Consistently use the function name CreateCheckPoint instead of CreateCheckpoint in code comments

2022-01-16 Thread Amit Kapila
On Fri, Jan 14, 2022 at 8:05 PM Amit Kapila wrote: > > On Fri, Jan 14, 2022 at 8:55 AM Bharath Rupireddy > wrote: > > > > The function CreateCheckPoint is specified as CreateCheckpoint in some > > of the code comments whereas in other places it is correctly > > mentioned. Attaching a tiny patch

Re: Null commitTS bug

2022-01-16 Thread Kyotaro Horiguchi
At Fri, 14 Jan 2022 22:49:59 +, "Kingsborough, Alex" wrote in > The fix for this is very simple > > > /* if we wrote out all subxids, we're done. / > - if (j + 1 >= nsubxids) > + if (j >= nsubxids) > break; It looks like a thinko and the fix is correct. (It's a matter of taste

Re: Add jsonlog log_destination for JSON server logs

2022-01-16 Thread Michael Paquier
On Mon, Jan 17, 2022 at 10:48:06AM +0900, Michael Paquier wrote: > Okay, this last piece has been applied this morning, after more review > and a couple of adjustments, mainly cosmetic (pg_current_logfile > missed a refresh, incorrect copyright in jsonlog.c, etc.). Let's see > what the buildfarm

Re: Add jsonlog log_destination for JSON server logs

2022-01-16 Thread Michael Paquier
On Wed, Jan 12, 2022 at 03:27:19PM +0900, Michael Paquier wrote: > This part will have to wait a bit more, but yes, this piece should be > straight-forward. Okay, this last piece has been applied this morning, after more review and a couple of adjustments, mainly cosmetic (pg_current_logfile

New developer papercut - Makefile references INSTALL

2022-01-16 Thread Tim McNamara
Hello, I encountered a minor road bump when checking out the pg source today. The Makefile's all target includes the following help message if GNUmakefile isn't available: echo "You need to run the 'configure' program first. See the file"; \ echo "'INSTALL' for installation instructions."

Re: row filtering for logical replication

2022-01-16 Thread Peter Smith
On Thu, Dec 2, 2021 at 7:40 PM vignesh C wrote: > ... > > 2) testpub5 and testpub_syntax2 are similar, one of them can be removed: > +SET client_min_messages = 'ERROR'; > +CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, > testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5); > +RESET

Re: drop tablespace failed when location contains .. on win32

2022-01-16 Thread Julien Rouhaud
Hi, This patch is a wanted bugfix and has been waiting for an update for 2 months. Do you plan to send a new version soon?

Re: Hook for extensible parsing.

2022-01-16 Thread Julien Rouhaud
Hi, On Fri, Sep 24, 2021 at 02:33:59PM +0800, Julien Rouhaud wrote: > On Thu, Sep 23, 2021 at 10:21:20AM -0400, Tom Lane wrote: > > > > I do have sympathy for the idea that extensions would like to define > > their own statement types. I just don't see a practical way to do it > > in our

Re: [PATCH] rename column if exists

2022-01-16 Thread Julien Rouhaud
Hi, This patch has been around for about 10 months. There seems to be some support for the feature but 3 committers raised concerns about the patch, and the OP never answered, or clarified the intended use case until now. At that point I don't see this patch getting committed at all so I'm

Time to increase hash_mem_multiplier default?

2022-01-16 Thread Peter Geoghegan
The current hash_mem_multiplier default is 1.0, which is a fairly conservative default: it preserves the historic behavior, which is that hash-based executor nodes receive the same work_mem budget as sort-based nodes. I propose that the default be increased to 2.0 for Postgres 15. Arguments in

Re: default to to ON_ERROR_STOP=on (Re: psql: exit status with multiple -c and -f)

2022-01-16 Thread Julien Rouhaud
Hi, On Mon, Dec 27, 2021 at 12:31:07PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > I think the current behavior of the regression test SQL scripts is exactly > > the > > opposite of what's desirable for almost all other scripts. The attached > > makes > > ON_ERROR_STOP the default, and

Re: extended stats on partitioned tables

2022-01-16 Thread Tomas Vondra
I've pushed the part adding stxdinherit flag to the catalog, so that on master we build statistics both with and without data from the child relations. I'm not going to push the ACL refactoring. We have similar code on various other places (not addressed by the proposed patch), and it'd make

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Andres Freund
Hi, On 2022-01-16 15:28:00 -0800, Andres Freund wrote: > I hacked that up last night. And a fix or two later, it seems to be > working. What I'd missed at first is that the event needs to be reset in > reached_end_position(), otherwise we'll busy loop. > > I wonder if using a short-lived event

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Andres Freund
Hi, On 2022-01-16 17:39:11 +0100, Magnus Hagander wrote: > On Sun, Jan 16, 2022 at 5:36 PM Magnus Hagander wrote: > > > > On Sun, Jan 16, 2022 at 5:34 PM Tom Lane wrote: > > > > > > Andres Freund writes: > > > > I don't immediately see a solution for this, other than to add > > > >

Re: fix crash with Python 3.11

2022-01-16 Thread Tom Lane
Peter Eisentraut writes: > The way plpy.commit() and plpy.rollback() handle errors is not ideal. > They end up just longjmping back to the main loop, without telling the > Python interpreter about it. This hasn't been a problem so far, > apparently, but with Python 3.11-to-be, this appears to

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2022-01-16 Thread Noah Misch
Cancel that kernel upgrade idea. I no longer expect it to help... On Sun, Jan 16, 2022 at 10:19:30PM +1300, Thomas Munro wrote: > On Sun, Jan 16, 2022 at 8:12 PM Noah Misch wrote: > > For specifics of the kernel bug, see the attached test program. In brief, > > the > > bug arises if one

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-01-16 Thread Justin Pryzby
On Thu, Jan 13, 2022 at 09:52:55AM -0600, Justin Pryzby wrote: > This is failing on windows CI when I use initdb --data-checksums, as attached. > > https://cirrus-ci.com/task/5612464120266752 >

Re: Isn't wait_for_catchup slightly broken?

2022-01-16 Thread Tom Lane
Julien Rouhaud writes: > On Sat, Jan 15, 2022 at 05:58:02PM -0500, Tom Lane wrote: >> Here's a version that makes sure that callers specify a write position not >> an insert position. I also simplified the callers wherever it turned >> out that they could just use the default parameters. >

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Magnus Hagander
On Sun, Jan 16, 2022 at 5:36 PM Magnus Hagander wrote: > > On Sun, Jan 16, 2022 at 5:34 PM Tom Lane wrote: > > > > Andres Freund writes: > > > I don't immediately see a solution for this, other than to add > > > StreamCtl->stop_event (mirroring ->stop_socket) and then convert > > >

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Magnus Hagander
On Sun, Jan 16, 2022 at 5:34 PM Tom Lane wrote: > > Andres Freund writes: > > I don't immediately see a solution for this, other than to add > > StreamCtl->stop_event (mirroring ->stop_socket) and then convert > > CopyStreamPoll() to use WaitForMultipleObjects(). Microsoft's select() > >

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Tom Lane
Andres Freund writes: > I don't immediately see a solution for this, other than to add > StreamCtl->stop_event (mirroring ->stop_socket) and then convert > CopyStreamPoll() to use WaitForMultipleObjects(). Microsoft's select() > doesn't support pipes and there's no socketpair(). > Any more

Re: pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Magnus Hagander
On Sun, Jan 16, 2022 at 10:22 AM Andres Freund wrote: > > Hi, > > I was wondering in [1] what we could do about the slowest tests on > windows. > > On 2021-12-31 11:25:28 -0800, Andres Freund wrote: > > Picking a random successful cfbot run [1] I see the following tap tests > > taking > > more

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-16 Thread Alexander Lakhin
Hello Andres, 15.01.2022 01:44, Andres Freund wrote: > What about instead giving WalReceiverConn an internal WaitEventSet, and using > that consistently? I've attached a draft for that. > > Alexander, could you test with that patch applied? Unfortunately, this patch doesn't fix the issue. The test

Re: Isn't wait_for_catchup slightly broken?

2022-01-16 Thread Julien Rouhaud
Hi, On Sat, Jan 15, 2022 at 05:58:02PM -0500, Tom Lane wrote: > > Here's a version that makes sure that callers specify a write position not > an insert position. I also simplified the callers wherever it turned > out that they could just use the default parameters. LGTM, and passes make

pg_basebackup WAL streamer shutdown is bogus - leading to slow tests

2022-01-16 Thread Andres Freund
Hi, I was wondering in [1] what we could do about the slowest tests on windows. On 2021-12-31 11:25:28 -0800, Andres Freund wrote: > Picking a random successful cfbot run [1] I see the following tap tests taking > more than 20 seconds: > > 67188 ms pg_basebackup t/010_pg_basebackup.pl > 59710 ms

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2022-01-16 Thread Thomas Munro
On Sun, Jan 16, 2022 at 8:12 PM Noah Misch wrote: > For specifics of the kernel bug, see the attached test program. In brief, the > bug arises if one process is write()ing or pwrite()ing a file at about the > same time that another process is read()ing or pread()ing the same. POSIX > says the