Re: Addition of --no-sync to pg_upgrade for test speedup

2021-12-21 Thread Michael Paquier
On Mon, Dec 20, 2021 at 10:46:13AM -0300, Alvaro Herrera wrote: > On 2021-Dec-16, Michael Paquier wrote: >> In pg_upgrade, we let the flush happen with initdb --sync-only, based >> on the binary path of the new cluster, so I think that we are not >> going to miss any test coverage by skipping that.

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-21 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 2:39 AM Peter Geoghegan wrote: > > On Mon, Nov 29, 2021 at 6:51 PM Peter Geoghegan wrote: > > Attached is a WIP patch doing this. > > This has bitrot, so I attach v2, mostly just to keep the CFTester > status green. The only real change is one minor simplification to how >

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-21 Thread Shinya Kato
On 2021-12-22 02:23, Tom Lane wrote: Kyotaro Horiguchi writes: At Mon, 20 Dec 2021 21:05:23 +0900, Shinya Kato wrote in We should use EmitWarningsOnPlaceholders when we use DefineCustomXXXVariable. I don't think there is any room for debate. Unfortunately, pltcl.c defines variables both in

Re: pg_upgrade should truncate/remove its logs before running

2021-12-21 Thread Michael Paquier
On Mon, Dec 20, 2021 at 09:39:26PM -0600, Justin Pryzby wrote: > On Mon, Dec 20, 2021 at 08:21:51PM +0900, Michael Paquier wrote: >> we could choose something simpler for the default, like >> "pg_upgrade_log". I don't have a good history in naming new things, >> though :) > > I specifically calle

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
On Tue, Dec 21, 2021 at 9:03 PM vignesh C wrote: > ... > Few comments: > 1) list_free(schemarelids) is called inside if and and outside if in > break case. Can we move it above continue so that it does not gets > called in the break case: > + schemarelids = > GetAllSchemaPubl

Re: Confused comment about drop replica identity index

2021-12-21 Thread Michael Paquier
On Mon, Dec 20, 2021 at 11:57:32AM -0300, Euler Taveira wrote: > On Mon, Dec 20, 2021, at 8:11 AM, Michael Paquier wrote: >> That's mostly fine. I have made some adjustments as per the >> attached. > > Your patch looks good to me. Thanks. I have done this part for now. -- Michael signature.asc

Re: In-placre persistance change of a relation

2021-12-21 Thread Kyotaro Horiguchi
Hello, Jakub. At Tue, 21 Dec 2021 13:07:28 +, Jakub Wartak wrote in > So what's suspicious is that 122880 -> 0 file size truncation. I've > investigated WAL and it seems to contain TRUNCATE records > after logged FPI images, so when the crash recovery would kick in it probably > clears th

Re: row filtering for logical replication

2021-12-21 Thread Amit Kapila
On Wed, Dec 22, 2021 at 9:23 AM vignesh C wrote: > > On Tue, Dec 21, 2021 at 2:29 PM Peter Smith wrote: > > > > Here is the v51* patch set: > > > > I tweaked the query slightly based on Euler's changes, the explain > analyze of the updated query based on Euler's suggestions, existing > query and

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-21 Thread Greg Stark
I haven't read the patch yet. But some thoughts based on the posted output: 1) At first I was quite skeptical about losing the progress reporting. I've often found it quite useful. But looking at the examples I'm convinced. Or rather I think a better way to look at it is that the progress output

Re: postgres_fdw: incomplete subabort cleanup of connections used in async execution

2021-12-21 Thread Etsuro Fujita
Hi Alexander, On Wed, Dec 22, 2021 at 1:08 AM Alexander Pyhalov wrote: > Etsuro Fujita писал 2021-12-19 13:25: > > While working on [1], I noticed $SUBJECT: postgres_fdw resets the > > per-connection states of connections, which store async requests sent > > to remote servers in async_capable mod

Re: sequences vs. synchronous replication

2021-12-21 Thread Fujii Masao
On 2021/12/22 10:57, Tomas Vondra wrote: On 12/19/21 04:03, Amit Kapila wrote: On Sat, Dec 18, 2021 at 7:24 AM Tomas Vondra wrote: while working on logical decoding of sequences, I ran into an issue with nextval() in a transaction that rolls back, described in [1]. But after thinking abo

Re: row filtering for logical replication

2021-12-21 Thread vignesh C
On Tue, Dec 21, 2021 at 2:29 PM Peter Smith wrote: > > Here is the v51* patch set: > I tweaked the query slightly based on Euler's changes, the explain analyze of the updated query based on Euler's suggestions, existing query and Euler's query is given below: 1) updated query based on Euler's sug

Re: parallel vacuum comments

2021-12-21 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 10:24 PM Amit Kapila wrote: > > On Tue, Dec 21, 2021 at 11:24 AM Masahiko Sawada > wrote: > > > > On Tue, Dec 21, 2021 at 2:04 PM Amit Kapila wrote: > > > > > > > Thank you for the comment. Agreed. > > > > I've attached updated version patches. Please review them. > > >

Re: do only critical work during single-user vacuum?

2021-12-21 Thread Masahiko Sawada
On Wed, Dec 22, 2021 at 6:56 AM Peter Geoghegan wrote: > > On Tue, Dec 21, 2021 at 1:31 PM John Naylor > wrote: > > On second thought, we don't really need another number here. We could > > simply go by the existing failsafe parameter, and if the admin wants a > > different value, it's already po

Re: sequences vs. synchronous replication

2021-12-21 Thread Tomas Vondra
On 12/19/21 04:03, Amit Kapila wrote: On Sat, Dec 18, 2021 at 7:24 AM Tomas Vondra wrote: while working on logical decoding of sequences, I ran into an issue with nextval() in a transaction that rolls back, described in [1]. But after thinking about it a bit more (and chatting with Petr Jel

Re: Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

2021-12-21 Thread Dilip Kumar
On Wed, 22 Dec 2021 at 12:28 AM, Ashutosh Sharma wrote: > > Is it okay to share the same tablespace (infact relfile) between the > primary and standby server? Perhaps NO. > > Oops, yeah absolutely they can never share the tablespace path. So we can ignore this issue. — Dilip -- Regards, Dilip

Re: CREATEROLE and role ownership hierarchies

2021-12-21 Thread Mark Dilger
> On Dec 21, 2021, at 5:11 PM, Shinya Kato > wrote: > > I fixed the patches because they cannot be applied to HEAD. Thank you. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-12-21 Thread Anton A. Melnikov
On 03.12.2021 19:55, Andrei Zubkov wrote: On Fri, 2021-12-03 at 17:03 +0300, Andrei Zubkov wrote: ... What if we'll create a new view for such resettable fields? It will make description of views and reset functions in the docs much more clear. Hi, Andrey! I completely agree that creatin

RE: parallel vacuum comments

2021-12-21 Thread houzj.f...@fujitsu.com
On Wed, Dec 22, 2021 8:38 AM Masahiko Sawada wrote: > On Tue, Dec 21, 2021 at 10:24 PM Amit Kapila > wrote: > > > > On Tue, Dec 21, 2021 at 11:24 AM Masahiko Sawada > wrote: > > > > > > On Tue, Dec 21, 2021 at 2:04 PM Amit Kapila > wrote: > > > > > > > > > > Thank you for the comment. Agreed. >

Re: Use extended statistics to estimate (Var op Var) clauses

2021-12-21 Thread Mark Dilger
> On Dec 21, 2021, at 4:28 PM, Mark Dilger wrote: > > Maybe there is some reason this is ok. ... and there is. Sorry for the noise. The planner appears to be smart enough to know that column "salary" is not being changed, and therefore NEW.salary and OLD.salary are equal. If I test a dif

Re: parallel vacuum comments

2021-12-21 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 10:24 PM Amit Kapila wrote: > > On Tue, Dec 21, 2021 at 11:24 AM Masahiko Sawada > wrote: > > > > On Tue, Dec 21, 2021 at 2:04 PM Amit Kapila wrote: > > > > > > > Thank you for the comment. Agreed. > > > > I've attached updated version patches. Please review them. > > >

Re: Use extended statistics to estimate (Var op Var) clauses

2021-12-21 Thread Mark Dilger
> On Dec 12, 2021, at 6:21 PM, Tomas Vondra > wrote: > > <0001-Improve-estimates-for-Var-op-Var-with-the-same-Var.patch> +* It it's (variable = variable) with the same variable on both sides, it's s/It it's/If it's/ 0001 lacks regression coverage. > <0002-simplification.patch> Changi

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
On Tue, Dec 21, 2021 at 5:58 AM Euler Taveira wrote: > > On Mon, Dec 20, 2021, at 12:10 AM, houzj.f...@fujitsu.com wrote: > > Attach the V49 patch set, which addressed all the above comments on the 0002 > patch. > > I've been testing the latest versions of this patch set. I'm attaching a new > pat

Re: row filtering for logical replication

2021-12-21 Thread Peter Smith
On Mon, Dec 20, 2021 at 9:30 PM Amit Kapila wrote: > > On Mon, Dec 20, 2021 at 8:41 AM houzj.f...@fujitsu.com > wrote: > > > > Thanks for the comments, I agree with all the comments. > > Attach the V49 patch set, which addressed all the above comments on the 0002 > > patch. > > > > Few comments/s

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
On Tue, Dec 21, 2021 at 5:56 PM Peter Geoghegan wrote: > > On Tue, Dec 21, 2021 at 1:31 PM John Naylor > wrote: > > On second thought, we don't really need another number here. We could > > simply go by the existing failsafe parameter, and if the admin wants a > > different value, it's already po

Re: A test for replay of regression tests

2021-12-21 Thread Thomas Munro
Hi, Rebased and updated based on feedback. Responses to multiple emails below: On Thu, Dec 16, 2021 at 1:22 PM Michael Paquier wrote: > On Wed, Dec 15, 2021 at 05:50:45PM +0900, Michael Paquier wrote: > > Hmm. FWIW, I am working on doing similar for pg_upgrade to switch to > > TAP there, and w

Re: do only critical work during single-user vacuum?

2021-12-21 Thread Peter Geoghegan
On Tue, Dec 21, 2021 at 1:31 PM John Naylor wrote: > On second thought, we don't really need another number here. We could > simply go by the existing failsafe parameter, and if the admin wants a > different value, it's already possible to specify > vacuum_failsafe_age/vacuum_multixact_failsafe_ag

Re: daitch_mokotoff module

2021-12-21 Thread Dag Lem
Hello again, It turns out that there actually exists an(other) implementation of the Daitch-Mokotoff Soundex System which gets it right; the JOS Soundex Calculator at https://www.jewishgen.org/jos/jossound.htm Other implementations I have been able to find, like the one in Apache Commons Coded use

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
I wrote: > Instead of a boolean, it seems like the new option should specify some > age below which VACUUM will skip the table entirely, and above which > will enter fail-safe mode. As mentioned earlier, the shutdown hint > could spell out the exact command. With this design, it would specify > th

Re: Buildfarm support for older versions

2021-12-21 Thread Larry Rosenman
On 12/16/2021 3:23 pm, Andrew Dunstan wrote: On 12/16/21 15:53, Larry Rosenman wrote: I get: ERROR for site owner: Invalid domain for site key on https://pgbuildfarm.org/cgi-bin/register-form.pl try https://buildfarm.postgresql.org/cgi-bin/register-form.pl cheers andrew I filled out th

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-12-21 Thread Mark Dilger
> On Dec 1, 2021, at 10:59 AM, Bossart, Nathan wrote: > > here is a v3 It took a while for me to get to this @@ -1304,33 +1370,46 @@ HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer, TransactionId *de if (HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask)) { + if (u

Re: minor gripe about lax reloptions parsing for views

2021-12-21 Thread Mark Dilger
Rebased patch attached: v3-0001-Reject-storage-options-in-toast-namespace-in-view.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

2021-12-21 Thread Ashutosh Sharma
This is happening because in the case of the primary server, we let the checkpointer process to unlink the first segment of the rel file but that's not the case with the standby server. In case of standby, the startup/recovery process unlinks the first segment of the rel file immediately during WAL

Re: pg14 psql broke \d datname.nspname.relname

2021-12-21 Thread Mark Dilger
Rebased patch attached: v3-0001-Reject-patterns-with-too-many-parts-or-wrong-db.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-21 Thread Tom Lane
I wrote: > 0003 looks to me like it was superseded by 75d22069e. I do not > particularly like that patch though; it seems both inefficient > and ill-designed. 0003 is adding a check in an equally bizarre > place. Why isn't add_placeholder_variable the place to deal > with this? Also, once we kn

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
On Tue, Dec 21, 2021 at 3:56 AM Masahiko Sawada wrote: > > On Tue, Dec 21, 2021 at 1:53 PM Peter Geoghegan wrote: > > > > On Mon, Dec 20, 2021 at 8:40 PM Masahiko Sawada > > wrote: > > > BTW a vacuum automatically enters failsafe mode under the situation > > > where the user has to run a vacuum

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-21 Thread Tom Lane
Kyotaro Horiguchi writes: > At Mon, 20 Dec 2021 21:05:23 +0900, Shinya Kato > wrote in >> We should use EmitWarningsOnPlaceholders when we use >> DefineCustomXXXVariable. >> I don't think there is any room for debate. > Unfortunately, pltcl.c defines variables both in pltcl and pltclu but > th

Re: postgres_fdw: incomplete subabort cleanup of connections used in async execution

2021-12-21 Thread Alexander Pyhalov
Etsuro Fujita писал 2021-12-19 13:25: Hi, While working on [1], I noticed $SUBJECT: postgres_fdw resets the per-connection states of connections, which store async requests sent to remote servers in async_capable mode, during post-abort (pgfdw_xact_callback()), but it fails to do so during post-

RE: Optionally automatically disable logical replication subscriptions on error

2021-12-21 Thread osumi.takami...@fujitsu.com
On Thursday, December 16, 2021 9:51 PM I wrote: > Attached the updated patch v14. FYI, I've conducted a test of disable_on_error flag using pg_upgrade. I prepared PG14 and HEAD applied with disable_on_error patch. Then, I setup a logical replication pair of the publisher and the subscriber by 14

Re: parallel vacuum comments

2021-12-21 Thread Amit Kapila
On Tue, Dec 21, 2021 at 11:24 AM Masahiko Sawada wrote: > > On Tue, Dec 21, 2021 at 2:04 PM Amit Kapila wrote: > > > > Thank you for the comment. Agreed. > > I've attached updated version patches. Please review them. > These look mostly good to me. Please find attached the slightly edited versio

RE: In-placre persistance change of a relation

2021-12-21 Thread Jakub Wartak
Hi Kyotaro, > I took a bit too long detour but the patch gets to pass make-world for me. Good news, v10 passes all the tests for me (including TAP recover ones). There's major problem I think: drop table t6; create unlogged table t6 (id bigint, t text); create sequence s1; insert into t6 select

Re: Multi-Column List Partitioning

2021-12-21 Thread Amit Langote
On Tue, Dec 21, 2021 at 2:47 PM Ashutosh Sharma wrote: > On Mon, Dec 20, 2021 at 7:04 PM Amit Langote wrote: >> On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma >> wrote: >> > >> > Hi, >> > >> > Is this okay? >> > >> > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); >>

RE: [PATCH] pg_stat_toast v0.3

2021-12-21 Thread kuroda.hay...@fujitsu.com
Dear Gunnar, > The attached v0.3 includes > * columns "storagemethod" and "compressmethod" added as per Hayato > Kuroda's suggestion I prefer your implementation that referring another system view. > * gathering timing information Here is a minor comment: I'm not sure when we should start to

Re: Schema variables - new implementation for Postgres 15

2021-12-21 Thread Pavel Stehule
út 21. 12. 2021 v 13:36 odesílatel Justin Pryzby napsal: > On Tue, Dec 21, 2021 at 01:29:00PM +0100, Pavel Stehule wrote: > > Hi > > > > út 21. 12. 2021 v 0:09 odesílatel Justin Pryzby > > napsal: > > > > > I don't understand what 0002 patch does relative to the 0001 patch. > > > Is 0002 to chan

Re: Schema variables - new implementation for Postgres 15

2021-12-21 Thread Justin Pryzby
On Tue, Dec 21, 2021 at 01:29:00PM +0100, Pavel Stehule wrote: > Hi > > út 21. 12. 2021 v 0:09 odesílatel Justin Pryzby > napsal: > > > I don't understand what 0002 patch does relative to the 0001 patch. > > Is 0002 to change the error messages from "schema variables" to "session > > variables"

Re: Schema variables - new implementation for Postgres 15

2021-12-21 Thread Pavel Stehule
Hi út 21. 12. 2021 v 0:09 odesílatel Justin Pryzby napsal: > I don't understand what 0002 patch does relative to the 0001 patch. > Is 0002 to change the error messages from "schema variables" to "session > variables" , in a separate commit to show that the main patch doesn't > change > regressio

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-21 Thread Dilip Kumar
On Thu, Dec 16, 2021 at 9:26 PM Neha Sharma wrote: > > Hi, > > While testing the v8 patches in a hot-standby setup, it was observed the > master is crashing with the below error; > > 2021-12-16 19:32:47.757 +04 [101483] PANIC: could not fsync file > "pg_tblspc/16385/PG_15_202112111/16386/16391"

Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

2021-12-21 Thread Dilip Kumar
While testing the below case with the hot standby setup (with the latest code), I have noticed that the checkpointer process crashed with the $subject error. As per my observation, we have registered the SYNC_REQUEST when inserting some tuple into the table, and later on ALTER SET TABLESPACE we hav

Re: In-placre persistance change of a relation

2021-12-21 Thread Kyotaro Horiguchi
At Tue, 21 Dec 2021 17:13:21 +0900 (JST), Kyotaro Horiguchi wrote in > Ugh! I completely forgot about TAP tests.. Thanks for the testing and > sorry for the bugs. > > This is a bit big change so I need a bit of time before posting the > next version. I took a bit too long detour but the patch

Re: row filtering for logical replication

2021-12-21 Thread vignesh C
On Tue, Dec 21, 2021 at 2:29 PM Peter Smith wrote: > > Here is the v51* patch set: > > Main changes from Euler's v50* are > 1. Most of Euler's "fixes" patches are now merged back in > 2. Patches are then merged per Amit's suggestion [Amit 20/12] > 3. Some other review comments are addressed > > ~~

Re: Failed transaction statistics to measure the logical replication progress

2021-12-21 Thread Greg Nancarrow
On Mon, Dec 20, 2021 at 8:40 PM osumi.takami...@fujitsu.com wrote: > > Updated the patch to address your concern. > Some review comments on the v18 patches: v18-0002 doc/src/sgml/monitoring.sgml (1) tablesync worker stats? Shouldn't the comment below only mention the apply worker? (since we're

Re: simplifying foreign key/RI checks

2021-12-21 Thread Zhihong Yu
On Mon, Dec 20, 2021 at 5:17 AM Amit Langote wrote: > On Mon, Dec 20, 2021 at 6:19 PM Zhihong Yu wrote: > > On Sun, Dec 19, 2021 at 10:20 PM Amit Langote > wrote: > >> > >> On Mon, Dec 20, 2021 at 2:00 PM Corey Huinker > wrote: > >> > Sorry for the delay. This patch no longer applies, it has s

Re: In-placre persistance change of a relation

2021-12-21 Thread Kyotaro Horiguchi
Ugh! I completely forgot about TAP tests.. Thanks for the testing and sorry for the bugs. This is a bit big change so I need a bit of time before posting the next version. At Mon, 20 Dec 2021 13:38:35 +, Jakub Wartak wrote in > 1) check-worlds seems OK but make -C src/test/recovery check