Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-19 Thread Noah Misch
On Mon, Jan 10, 2022 at 04:25:27PM -0500, Tom Lane wrote: > Apropos of that, it's worth noting that wait_for_catchup *is* > dependent on up-to-date stats, and here's a recent run where > it sure looks like the timeout cause is AWOL stats collector: > > https://buildfarm.postgresql.org/cgi-bin/show

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-03-19 Thread Pavel Borisov
> On 2022-03-18 18:14:52 +0300, Maxim Orlov wrote: > > Subject: [PATCH v22 3/6] Use 64-bit pages in SLRU > > > > This is one step toward 64-bit XIDs. > > I think this should explain in more detail why this move is done. Neither > the > commit message nor the rest of the thread does so afaics. It's

Re: Tab completion for ALTER MATERIALIZED VIEW ... SET ACCESS METHOD

2022-03-19 Thread Michael Paquier
On Fri, Mar 18, 2022 at 03:13:05PM +0900, Michael Paquier wrote: > Thanks. This looks rather sane to me. I'll split things into 3 > commits in total, as of the psql completion, SET TABLESPACE and SET > ACCESS METHOD. The first and third patches are only for HEAD, while > the documentation hole w

Re: support for MERGE

2022-03-19 Thread Alvaro Herrera
On 2022-Mar-10, Simon Riggs wrote: > Duplicate rows should produce a uniqueness violation error in one of > the transactions, assuming there is a constraint to define the > conflict. Without such a constraint there is no conflict. > > Concurrent inserts are checked by merge-insert-update.spec, wh

Which hook to use when overriding utility commands (COPY ...)

2022-03-19 Thread Hannu Krosing
Hi Pgsql-Hackers Which hook should I use when overriding the COPY command in an extension? I am working on adding new functionalities to COPY (compression, index management, various other transports in addition to stdin and file, other data formats, etc...) and while the aim is to contribute this

Re: Which hook to use when overriding utility commands (COPY ...)

2022-03-19 Thread Michael Paquier
On Sat, Mar 19, 2022 at 12:28:46PM +0100, Hannu Krosing wrote: > Which hook should I use when overriding the COPY command in an extension? CopyStmt goes through ProcessUtility(), so you can use the hook called ProcessUtility_hook to override what you want. -- Michael signature.asc Description: P

Re: Add pg_freespacemap extension sql test

2022-03-19 Thread Dong Wook Lee
> Well, my guess is that you basically just care about being able to > detect if there is free space in the map or not, which goes down to > detecting if pg_freespace() returns 0 or a number strictly higher than > 0, so wouldn't it be enough to stick some > 0 in your test queries? I edited the pre

Re: [PATCH] pgbench: add multiconnect option

2022-03-19 Thread Fabien COELHO
Hi Sami, Pgbench is a simple benchmark tool by design, and I wonder if adding a multiconnect feature will cause pgbench to be used incorrectly. Maybe, but I do not see how it would be worse that what pgbench already allows. A real world use-case will be helpful for this thread. Basicall

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-19 Thread Justin Pryzby
On Fri, Mar 11, 2022 at 08:26:26PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > Here is a new version of the patchset. SLRU refactoring was moved to a > > separate patch. Both v14-0003 (XID_FMT macro) and v14-0004 (SLRU > > refactoring) can be delivered in PG15. > > Here is a new version

Re: pgsql: Add option to use ICU as global locale provider

2022-03-19 Thread Christoph Berg
Re: Peter Eisentraut > Since some (legacy) code still uses the libc locale facilities > directly, we still need to set the libc global locale settings even if > ICU is otherwise selected. So pg_database now has three > locale-related fields: the existing datcollate and datctype, which are > always

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-19 Thread Pavel Borisov
> > > Do you know that you can test a branch on cirrus without using CF bot or > mailing the patch to the list ? See src/tools/ci/README > Yes, sure! The main reason to post updates of this patchset is for hackers that are interested in the progress have relevant version with updates. This patch

Re: Add pg_freespacemap extension sql test

2022-03-19 Thread Fabrízio de Royes Mello
On Sat, Mar 19, 2022 at 1:18 PM Dong Wook Lee wrote: > > > Well, my guess is that you basically just care about being able to > > detect if there is free space in the map or not, which goes down to > > detecting if pg_freespace() returns 0 or a number strictly higher than > > 0, so wouldn't it be

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-19 Thread Andres Freund
Hi, On 2022-03-04 22:01:22 -0800, Andres Freund wrote: > On 2022-03-04 20:06:43 -0800, Andres Freund wrote: > > On 2022-03-05 16:39:21 +1300, Thomas Munro wrote: > > > I vote for committing that workaround into the tree temporarily, > > > because it's not just cfbot, it's also everyone's dev branc

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-19 Thread Yugo NAGATA
Hello Fabien, On Sat, 12 Mar 2022 15:54:54 +0100 (CET) Fabien COELHO wrote: > Hello Yugo-san, > > About Pgbench error handling v16: Thank you for your review! I attached the updated patches. > This patch set needs a minor rebase because of 506035b0. Otherwise, patch > compiles, global and l

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-19 Thread Robert Haas
On Fri, Mar 18, 2022 at 12:39 AM Dilip Kumar wrote: > > One question that occurred to me when looking this over is whether, or > > why, it's safe against concurrent smgr invalidations. > > We are only accessing the smgr of the source database and the > destination database. And there is no one el

Re: Problem with moderation of messages with patched attached.

2022-03-19 Thread Andres Freund
Hi, On 2022-03-03 13:37:35 +, Dave Page wrote: > On Thu, 3 Mar 2022 at 13:28, Pavel Borisov wrote: > > > The mail system doesn't have the capability to apply different moderation > >> rules for people in that way I'm afraid. > >> > > Maybe then 2MB for everyone? Otherwise it's not so conveni

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-19 Thread Andres Freund
On 2022-03-19 11:16:48 -0700, Andres Freund wrote: > Cirrus now provides a way to get the old behaviour back without pinning an old > agent version. See attached I've pushed that now. If we can come up with a better recipe for starting postgres in the background, cool. But until then...

Re: a misbehavior of partition row movement (?)

2022-03-19 Thread Alvaro Herrera
On 2022-Mar-18, Zhihong Yu wrote: > +#define AFTER_TRIGGER_OFFSET 0x07FF /* must be low-order > bits */ > +#define AFTER_TRIGGER_DONE 0x8000 > +#define AFTER_TRIGGER_IN_PROGRESS 0x4000 > > Is it better if the order of AFTER_TRIGGER_DONE > and AFTER_TRIGGER_

Re: Commitfest manager for 2022-03

2022-03-19 Thread Justin Pryzby
On Sun, Feb 27, 2022 at 04:51:16PM +0800, Julien Rouhaud wrote: > On Sat, Feb 26, 2022 at 06:37:21PM -0600, Justin Pryzby wrote: > > Can I suggest to update the CF APP to allow: > > | Target version: 16 > > > > I also suggest to update patches to indicate which are (not) being > > considered > >

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-19 Thread Tatsuo Ishii
Hi Yugo, I have looked into the patch and I noticed that is used in pgbench.sgml. e.g. AFAIK this is the only place where "endterm" is used. In other places "link" tag is used instead: Failures and Serialization/Deadlock Retries Note that the rendered result is identical. Do we want to use

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-19 Thread Tatsuo Ishii
Hi Yugo, I tested with serialization error scenario by setting: default_transaction_isolation = 'repeatable read' The result was: $ pgbench -t 10 -c 10 --max-tries=10 test transaction type: scaling factor: 10 query mode: simple number of clients: 10 number of threads: 1 maximum number of tries:

Re: a misbehavior of partition row movement (?)

2022-03-19 Thread Amit Langote
On Sun, Mar 20, 2022 at 5:13 AM Alvaro Herrera wrote: > On 2022-Mar-18, Zhihong Yu wrote: > > > +#define AFTER_TRIGGER_OFFSET 0x07FF /* must be low-order > > bits */ > > +#define AFTER_TRIGGER_DONE 0x8000 > > +#define AFTER_TRIGGER_IN_PROGRESS 0x4000 > > > >

Re: Column Filtering in Logical Replication

2022-03-19 Thread Amit Kapila
On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra wrote: > > On 3/18/22 15:43, Tomas Vondra wrote: > >> > > > > Hmmm. So the theory is that in most runs we manage to sync the tables > > faster than starting the workers, so we don't hit the limit. But on some > > machines the sync worker takes a bit lo

Hardening heap pruning code (was: BUG #17255: Server crashes in index_delete_sort_cmp() due to race condition with vacuum)

2022-03-19 Thread Peter Geoghegan
On Wed, Mar 9, 2022 at 4:46 PM Andres Freund wrote: > On 2022-03-03 19:31:32 -0800, Peter Geoghegan wrote: > > Attached is a new revision of my fix. This is more or less a > > combination of my v4 fix from November 12 [1] and Andres' > > already-committed fix (commit 18b87b20), rebased on top of H

Re: Schema variables - new implementation for Postgres 15

2022-03-19 Thread Julien Rouhaud
On Sat, Mar 19, 2022 at 04:46:13PM -0500, Justin Pryzby wrote: > On Thu, Mar 03, 2022 at 03:06:52PM +0800, Julien Rouhaud wrote: > > Hi, > > > > On Wed, Mar 02, 2022 at 06:03:06AM +0100, Pavel Stehule wrote: > > > > > > I lost commit with this change. I am sending updated patch. > > > > Thanks a

Re: A test for replay of regression tests

2022-03-19 Thread Thomas Munro
Another failure under 027_stream_regress.pl: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2022-03-16%2005%3A58%3A05 vacuum ... FAILED 3463 ms I'll try to come up with the perl needed to see the regression.diffs next time...

Re: WIP: WAL prefetch (another approach)

2022-03-19 Thread Thomas Munro
On Fri, Mar 18, 2022 at 9:59 AM Thomas Munro wrote: > I'll push 0001 today to let the build farm chew on it for a few days > before moving to 0002. Clearly 018_wal_optimize.pl is flapping and causing recoveryCheck to fail occasionally, but that predates the above commit. I didn't follow the exis

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-19 Thread Dilip Kumar
On Sun, Mar 20, 2022 at 12:03 AM Robert Haas wrote: > > On Fri, Mar 18, 2022 at 12:39 AM Dilip Kumar wrote: > > > One question that occurred to me when looking this over is whether, or > > > why, it's safe against concurrent smgr invalidations. > > > > We are only accessing the smgr of the source

Re: Column Filtering in Logical Replication

2022-03-19 Thread Amit Kapila
On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: > > On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra > wrote: > > > So the question is why those two sync workers never complete - I guess > > there's some sort of lock wait (deadlock?) or infinite loop. > > > > It would be a bit tricky to reproduce