Re: Support logical replication of DDLs

2023-04-25 Thread Masahiko Sawada
On Tue, Mar 28, 2023 at 3:22 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, March 28, 2023 1:41 PM Amit Kapila > wrote: > > > > On Mon, Mar 27, 2023 at 5:37 PM Amit Kapila > > wrote: > > > > > > On Mon, Mar 27, 2023 at 12:07 PM Amit Kapila > > wrote: > > > > > > > > On Mon, Mar 27, 2023 at 2

Re: Support logical replication of DDLs

2023-04-25 Thread Masahiko Sawada
On Wed, Apr 26, 2023 at 2:56 PM Amit Kapila wrote: > > On Wed, Apr 26, 2023 at 10:01 AM Masahiko Sawada > wrote: > > > > On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Aport from above comments, I splitted the code related to verbose > > > mode to a separate patch.

Re: Support logical replication of DDLs

2023-04-25 Thread Amit Kapila
On Wed, Apr 26, 2023 at 10:01 AM Masahiko Sawada wrote: > > On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) > wrote: > > > > Aport from above comments, I splitted the code related to verbose > > mode to a separate patch. And here is the new version patch set. > > > > As for DDL replication,

Re: Support logical replication of DDLs

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 12:58 PM Zhijie Hou (Fujitsu) wrote: > > Aport from above comments, I splitted the code related to verbose > mode to a separate patch. And here is the new version patch set. > As for DDL replication, we create event triggers to write deparsed DDL commands to WAL when creat

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-04-25 Thread vignesh C
On Tue, 25 Apr 2023 at 12:51, Drouvot, Bertrand wrote: > > Hi, > > On 4/25/23 6:23 AM, Amit Kapila wrote: > > On Mon, Apr 24, 2023 at 3:36 PM Drouvot, Bertrand > > wrote: > >> > >> Without the second "pg_log_standby_snapshot()" then > >> wait_for_subscription_sync() would be waiting > >> some ti

[pg_rewind] use the passing callback instead of global function

2023-04-25 Thread Junwang Zhao
`local_traverse_files` and `libpq_traverse_files` both have a callback parameter but instead use the global process_source_file which is no good for function encapsulation. -- Regards Junwang Zhao 0001-pg_rewind-use-the-passing-callback-instead-of-global.patch Description: Binary data

Re: Order changes in PG16 since ICU introduction

2023-04-25 Thread Jeff Davis
On Fri, 2023-04-21 at 22:35 +0100, Andrew Gierth wrote: > > > > > > Can lc_collate_is_c() be taught to check whether an ICU locale is > using > POSIX collation? Attached are a few small patches: 0001: don't convert C to en-US-u-va-posix 0002: handle locale C the same regardless of the provid

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

2023-04-25 Thread Thomas Munro
On Tue, Apr 25, 2023 at 12:16 PM Andres Freund wrote: > On 2023-04-24 15:32:25 -0700, Andres Freund wrote: > > We obviously can add a retry loop to FileFallocate(), similar to what's > > already present e.g. in FileRead(). But I wonder if we shouldn't go a bit > > further, and do it for all the fd

Re: pg_stat_io for the startup process

2023-04-25 Thread Andres Freund
Hi, On 2023-04-25 16:00:24 -0400, Robert Haas wrote: > On Tue, Apr 25, 2023 at 2:39 PM Andres Freund wrote: > > I'm mildly inclined to not consider it a bug, given that this looks to have > > been true for other stats for quite a while? But it does still seem worth > > improving upon - I'd make t

Re: pg_stat_io for the startup process

2023-04-25 Thread Michael Paquier
On Tue, Apr 25, 2023 at 04:00:24PM -0400, Robert Haas wrote: > ISTM that it's pretty desirable to do something about this. If the > process isn't going to report statistics properly, at least remove it > from the view. If it can be made to report properly, that would be > even better. But shipping

Re: Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread Michael Paquier
On Tue, Apr 25, 2023 at 05:29:48PM +0900, Kyotaro Horiguchi wrote: > I thought that we have decided not to do that, but I coundn't find any > discussion about it in the ML archive. Anyway, I think it is great > that we have that option. No objections from here to make that optional. It's been ar

Re: Orphaned wait event

2023-04-25 Thread Michael Paquier
On Tue, Apr 25, 2023 at 09:24:58PM +0530, Bharath Rupireddy wrote: > It looks like this patch attached upthread at [1] isn't in yet, > meaning WAIT_EVENT_SLRU_FLUSH_SYNC stays unused. IMO, it's worth > pushing it to the PG16 branch. It will help add a wait event for SLRU > page flushes. Thoughts? >

Re: pg_stat_io for the startup process

2023-04-25 Thread Robert Haas
On Tue, Apr 25, 2023 at 2:39 PM Andres Freund wrote: > I'm mildly inclined to not consider it a bug, given that this looks to have > been true for other stats for quite a while? But it does still seem worth > improving upon - I'd make the consideration when to apply the relevant patches > depend o

Re: pg_stat_io for the startup process

2023-04-25 Thread Andres Freund
Hi, On 2023-04-25 13:54:43 -0400, Melanie Plageman wrote: > On Tue, Apr 25, 2023 at 10:51:14PM +0900, Fujii Masao wrote: > > Regarding pg_stat_io for the startup process, I noticed that the counters > > are only incremented after the startup process exits, not during WAL replay > > in standby mode

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

2023-04-25 Thread Christoph Berg
Re: Andres Freund > A prototype of that approach is attached. I pushed the retry handling into the > pg_* routines where applicable. I guess we could add pg_* routines for > FileFallocate(), FilePrewarm() etc as well, but I didn't do that here. > > Christoph, could you verify this fixes your issu

Re: pg_stat_io for the startup process

2023-04-25 Thread Melanie Plageman
On Tue, Apr 25, 2023 at 10:51:14PM +0900, Fujii Masao wrote: > Hi, > > Regarding pg_stat_io for the startup process, I noticed that the counters > are only incremented after the startup process exits, not during WAL replay > in standby mode. This is because pgstat_flush_io() is only called when >

Re: base backup vs. concurrent truncation

2023-04-25 Thread Andres Freund
Hi, On 2023-04-25 11:42:43 -0400, Robert Haas wrote: > On Mon, Apr 24, 2023 at 8:03 PM Andres Freund wrote: > > What we've discussed somewhere in the past is to always truncate N+1 when > > creating the first page in N. I.e. if we extend into 23456.1, we truncate > > 23456.2 to 0 blocks. As far

Re: Issue in postgres_fdw causing unnecessary wait for cancel request reply

2023-04-25 Thread Fujii Masao
On 2023/04/21 16:39, Jelte Fennema wrote: In my opinion, PQconnectPoll and PQgetCancel should use the same parsing function or PQconnectPoll should set parsed values, making unnecessary for PQgetCancel to parse the same parameter again. Yes, I totally agree. So I think patch 0002 looks fine.

Re: Switching XLog source from archive to streaming when primary available

2023-04-25 Thread Bharath Rupireddy
On Fri, Feb 24, 2023 at 10:26 AM Bharath Rupireddy wrote: > > On Wed, Nov 16, 2022 at 11:39 AM Bharath Rupireddy > wrote: > > > > I'm attaching the v10 patch for further review. > > Needed a rebase. I'm attaching the v11 patch for further review. Needed a rebase, so attaching the v12 patch. I wo

Re: Orphaned wait event

2023-04-25 Thread Bharath Rupireddy
On Fri, Mar 24, 2023 at 12:00 PM Bharath Rupireddy wrote: > > On Fri, Mar 24, 2023 at 3:31 AM Thomas Munro wrote: > > > > On Thu, Mar 23, 2023 at 8:10 PM Bharath Rupireddy > > wrote: > > > Yeah, commit [1] removed the last trace of it. I wonder if we can add > > > a WAIT_EVENT_SLRU_FLUSH_SYNC wa

Re: base backup vs. concurrent truncation

2023-04-25 Thread Robert Haas
On Mon, Apr 24, 2023 at 8:03 PM Andres Freund wrote: > What we've discussed somewhere in the past is to always truncate N+1 when > creating the first page in N. I.e. if we extend into 23456.1, we truncate > 23456.2 to 0 blocks. As far as I can tell, that'd solve this issue? Yeah, although leavin

Re: enhancing plpgsql debug api - hooks on statements errors and function errors

2023-04-25 Thread Pavel Stehule
Hi út 25. 4. 2023 v 10:27 odesílatel Pavel Stehule napsal: > Hi > > When I implemented profiler and coverage check to plpgsql_check I had to > write a lot of hard maintaining code related to corect finishing some > operations (counter incrementing) usually executed by stmt_end and func_end > ho

Re: Order changes in PG16 since ICU introduction

2023-04-25 Thread Tom Lane
"Daniel Verite" writes: > FTR the full text search parser still uses the libc functions > is[w]space/alpha/digit... that depend on lc_ctype, whether the db > collation provider is ICU or not. Yeah, those aren't even connected up to the collation-selection mechanisms; lots of work to do there. I

Re: Should vacuum process config file reload more often

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 10:35 PM Daniel Gustafsson wrote: > > > On 25 Apr 2023, at 15:31, Masahiko Sawada wrote: > > > > On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: > >> > >>> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > >> > >>> I've attached an updated patch for fixing at_d

Re: Order changes in PG16 since ICU introduction

2023-04-25 Thread Daniel Verite
Jeff Davis wrote: > > (I'm not sure whether those operations can get redirected to ICU > > today > > or whether they still always go to libc, but we'll surely want to fix > > it eventually if the latter is still true.) > > Those operations do get redirected to ICU today. FTR the full te

Re: Request for comment on setting binary format output per session

2023-04-25 Thread Dave Cramer
On Tue, 25 Apr 2023 at 07:26, Dave Cramer wrote: > > > > On Mon, 24 Apr 2023 at 19:18, Merlin Moncure wrote: > >> >> >> On Thu, Apr 20, 2023 at 2:52 PM Dave Cramer wrote: >> >>> >>> As promised here is a patch with defines for all of the protocol >>> messages. >>> >> I created a protocol.h file

Re: Add PQsendSyncMessage() to libpq

2023-04-25 Thread Denis Laxalde
Anton Kirilov wrote: I would appeciate your thoughts on my proposal. This sounds like a useful addition to me. I've played a bit with it in Psycopg and it works fine. diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c index a16bbf32ef..e2b32c1379 100644 --- a/src/i

pg_stat_io for the startup process

2023-04-25 Thread Fujii Masao
Hi, Regarding pg_stat_io for the startup process, I noticed that the counters are only incremented after the startup process exits, not during WAL replay in standby mode. This is because pgstat_flush_io() is only called when the startup process exits. Shouldn't it be called during WAL replay as w

Re: Should vacuum process config file reload more often

2023-04-25 Thread Daniel Gustafsson
> On 25 Apr 2023, at 15:31, Masahiko Sawada wrote: > > On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: >> >>> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: >> >>> I've attached an updated patch for fixing at_dobalance condition. >> >> I revisited this and pushed it to all support

Re: Should vacuum process config file reload more often

2023-04-25 Thread Masahiko Sawada
On Tue, Apr 25, 2023 at 9:39 PM Daniel Gustafsson wrote: > > > On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > > > I've attached an updated patch for fixing at_dobalance condition. > > I revisited this and pushed it to all supported branches after another round > of > testing and reading. Th

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-25 Thread Hayato Kuroda (Fujitsu)
Dear Hackers, > Thank you for giving comments! PSA new version. Note that due to the current version could not work well on FreeBSD, maybe because of the timing issue[1]. I'm now analyzing the reason and will post the fixed version. [1]: https://cirrus-ci.com/build/4676441267240960 Best Regards

Re: Should vacuum process config file reload more often

2023-04-25 Thread Daniel Gustafsson
> On 17 Apr 2023, at 04:04, Masahiko Sawada wrote: > I've attached an updated patch for fixing at_dobalance condition. I revisited this and pushed it to all supported branches after another round of testing and reading. -- Daniel Gustafsson

Re: Request for comment on setting binary format output per session

2023-04-25 Thread Dave Cramer
On Mon, 24 Apr 2023 at 19:18, Merlin Moncure wrote: > > > On Thu, Apr 20, 2023 at 2:52 PM Dave Cramer wrote: > >> >> As promised here is a patch with defines for all of the protocol messages. >> > I created a protocol.h file and put it in src/includes >> I'm fairly sure that some of the names I

Re: ssl tests aren't concurrency safe due to get_free_port()

2023-04-25 Thread Peter Eisentraut
On 20.11.22 16:10, Andrew Dunstan wrote: On 2022-11-19 Sa 15:16, Andres Freund wrote: Hi, On 2022-11-19 10:56:33 -0500, Andrew Dunstan wrote: Perhaps we should just export a directory in configure instead of this guessing game? I think the obvious candidate would be to export top_builddir fr

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-04-25 Thread Richard Guo
On Fri, Apr 14, 2023 at 8:51 PM Etsuro Fujita wrote: > I think that the root cause for this issue would be in the > create_scan_plan handling of pseudoconstant quals when creating a > foreign-join (or custom-join) plan. Yes exactly. In create_scan_plan, we are supposed to extract all the pseud

Re: Missing update of all_hasnulls in BRIN opclasses

2023-04-25 Thread Alvaro Herrera
On 2023-Apr-24, Tomas Vondra wrote: > On 4/24/23 17:36, Alvaro Herrera wrote: > > (As for your FIXME comment in brin_memtuple_initialize, I think you're > > correct: we definitely need to reset bt_placeholder. Otherwise, we risk > > places that call it in a loop using a BrinMemTuple with one ran

Re: Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread Kyotaro Horiguchi
At Tue, 25 Apr 2023 16:38:16 +0900, torikoshia wrote in > Hi, > > Currently pg_archivecleanup doesn't remove backup history files even > when they're older than oldestkeptwalfile. > > Of course the size of backup history files are smaller than WAL files > and they wouldn't consume much disk sp

enhancing plpgsql debug api - hooks on statements errors and function errors

2023-04-25 Thread Pavel Stehule
Hi When I implemented profiler and coverage check to plpgsql_check I had to write a lot of hard maintaining code related to corect finishing some operations (counter incrementing) usually executed by stmt_end and func_end hooks. It is based on the fmgr hook and its own statement call stack. Can be

Re: In-placre persistance change of a relation

2023-04-25 Thread Kyotaro Horiguchi
At Fri, 17 Mar 2023 15:16:34 +0900 (JST), Kyotaro Horiguchi wrote in > Mmm. It took longer than I said, but this is the patch set that > includes all three parts. > > 1. "Mark files" to prevent orphan storage files for in-transaction > created relations after a crash. > > 2. In-place persist

Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread torikoshia
Hi, Currently pg_archivecleanup doesn't remove backup history files even when they're older than oldestkeptwalfile. Of course the size of backup history files are smaller than WAL files and they wouldn't consume much disk space, but a lot of backup history files(e.g. daily backup for years)

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-04-25 Thread Drouvot, Bertrand
Hi, On 4/25/23 6:43 AM, Amit Kapila wrote: On Mon, Apr 24, 2023 at 5:38 PM Drouvot, Bertrand wrote: We are waiting here at a maximum for 10 * $default_timeout (means 3 minutes) before we time out. Would you prefer to wait more than 3 minutes at a maximum? No, because I don't know what wou

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-04-25 Thread Drouvot, Bertrand
Hi, On 4/25/23 6:23 AM, Amit Kapila wrote: On Mon, Apr 24, 2023 at 3:36 PM Drouvot, Bertrand wrote: Without the second "pg_log_standby_snapshot()" then wait_for_subscription_sync() would be waiting some time on the poll for "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-04-25 Thread Pavel Luzanov
On 24.04.2023 23:53, Melanie Plageman wrote: I copied the committer who most recently touched pg_stat_io (Michael Paquier) to see if we could get someone interested in committing this docs update. I can explain my motivation by suggesting this update. pg_stat_io is a very impressive feature. S