Re: Logical replication timeout problem

2022-10-19 Thread Fabrice Chapuis
Yes the refresh of MV is on the Publisher Side. Thanks for your draft patch, I'll try it I'll back to you as soonas possible One question: why the refresh of the MV is a DDL not a DML? Regards Fabrice On Wed, 19 Oct 2022, 10:15 wangw.f...@fujitsu.com wrote: > On Tue, Oct 18, 2022 at 22:35 PM

Re: Logical WAL sender unresponsive during decoding commit

2022-10-19 Thread Amit Kapila
On Thu, Oct 20, 2022 at 5:17 AM Robert Haas wrote: > > > > Pushed. > > I think this was a good change, but there's at least one other problem > here: within ReorderBufferRestoreChanges, the while (restored < > max_changes_in_memory && *segno <= last_segno) doesn't seem to contain > a CFI. Note

Re: Understanding, testing and improving our Windows filesystem code

2022-10-19 Thread Thomas Munro
On Tue, Oct 18, 2022 at 10:00 PM Thomas Munro wrote: > * has anyone got a relevant filesystem where this fails? which way > do ReFS and SMB go? do the new calls in 0010 just fail, and if so > with which code (ie could we add our own fallback path)? Andres kindly ran these tests on some Win 10

Re: GUC values - recommended way to declare the C variables?

2022-10-19 Thread Justin Pryzby
On Thu, Oct 20, 2022 at 11:56:58AM +1100, Peter Smith wrote: > Patch 0002 adds a sanity-check function called by > InitializeGUCOptions, as suggested by Tom [2]. This is to ensure that > the GUC C variable initial values are sensible and/or have not gone > stale compared with the compiled-in

Re: Perform streaming logical transactions by background workers and parallel apply

2022-10-19 Thread Dilip Kumar
On Tue, Oct 18, 2022 at 6:25 PM Amit Kapila wrote: > > Interesting case. So I think the root of the problem is the same as > > what we have for a column is marked unique to the subscriber but not > > to the publisher. In short, two transactions which are independent of > > each other on the

Documentation refinement for Parallel Scans

2022-10-19 Thread Zhang Mingli
Hi, Found documents about parallel scan may be not so accurate. As said in parallel.smgl: ``` In a parallel sequential scan, the table's blocks will be divided among the cooperating processes. Blocks are handed out one at a time, so that access to the table remains sequential. ``` To my

Re: Fix GetWALAvailability function code comments for WALAVAIL_REMOVED return value

2022-10-19 Thread Kyotaro Horiguchi
At Wed, 19 Oct 2022 13:06:08 +0530, Bharath Rupireddy wrote in > On Wed, Oct 19, 2022 at 12:39 PM sirisha chamarthi > wrote: > > > > The current code comment says that the replication stream on a slot with > > the given targetLSN can't continue after a restart but even without a > > restart

Re: Documentation for building with meson

2022-10-19 Thread Justin Pryzby
On Wed, Oct 19, 2022 at 11:35:10AM -0700, samay sharma wrote: > Creating a new thread focussed on adding docs for building Postgres with > meson. This is a spinoff from the original thread [1] and I've attempted to > address all the feedback provided there in the attached patch. > > Please let me

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-10-19 Thread shiy.f...@fujitsu.com
On Wed, Oct 19, 2022 12:05 AM Önder Kalacı wrote: > > Attached v19. > Thanks for updating the patch. Here are some comments on v19. 1. In execReplication.c: + TypeCacheEntry **eq = NULL; /* only used when the index is not unique */ Maybe the comment here should be changed. Now it is

Re: thinko in basic_archive.c

2022-10-19 Thread Kyotaro Horiguchi
At Wed, 19 Oct 2022 10:21:12 +0530, Bharath Rupireddy wrote in > On Wed, Oct 19, 2022 at 8:58 AM Kyotaro Horiguchi > wrote: > > An archive module could clean up them at startup or at the first call > > but that might be dangerous (since archive directory is I think > > thought as external

Re: GUC values - recommended way to declare the C variables?

2022-10-19 Thread Peter Smith
PSA v2* patches. Patch 0001 is just a minor tidy of the GUC C variables of logical replication. The C variable initial values are present again, how Michael preferred them [1]. Patch 0002 adds a sanity-check function called by InitializeGUCOptions, as suggested by Tom [2]. This is to ensure that

Re: thinko in basic_archive.c

2022-10-19 Thread Kyotaro Horiguchi
At Wed, 19 Oct 2022 10:48:03 -0400, Robert Haas wrote in > On Tue, Oct 18, 2022 at 11:28 PM Kyotaro Horiguchi > wrote: > > > > Yeah, leaving a potentially unbounded number of files around after > > > > system crashes seems pretty unfriendly. I'm not sure how to fix that, > > > > exactly. > > >

Re: Logical WAL sender unresponsive during decoding commit

2022-10-19 Thread Robert Haas
On Tue, Aug 23, 2022 at 4:40 AM Amit Kapila wrote: > On Mon, Aug 22, 2022 at 4:48 PM Amit Kapila wrote: > > On Tue, Aug 16, 2022 at 2:37 PM Masahiko Sawada > > wrote: > > > I've attached patches for all supported branches. > > > > LGTM. I'll push this tomorrow unless there are

Re: refactor ownercheck and aclcheck functions

2022-10-19 Thread Corey Huinker
On Fri, Oct 14, 2022 at 3:39 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > These patches take the dozens of mostly-duplicate pg_foo_ownercheck() > and pg_foo_aclcheck() functions and replace (most of) them by common > functions that are driven by the ObjectProperty table. All

Re: ts_locale.c: why no t_isalnum() test?

2022-10-19 Thread Tom Lane
Corey Huinker writes: > I see this is already committed, but I'm curious, why do t_isalpha and > t_isdigit have the pair of /* TODO */ comments? This unfinished business > isn't explained anywhere in the file. We really ought to be consulting the locale/collation passed to the SQL-level operator

Re: ts_locale.c: why no t_isalnum() test?

2022-10-19 Thread Corey Huinker
On Wed, Oct 5, 2022 at 3:53 PM Tom Lane wrote: > I happened to wonder why various places are testing things like > > #define ISWORDCHR(c)(t_isalpha(c) || t_isdigit(c)) > > rather than using an isalnum-equivalent test. The direct answer > is that ts_locale.c/.h provides no such test

Decoupling antiwraparound autovacuum from special rules around auto cancellation

2022-10-19 Thread Peter Geoghegan
I think that we should decouple the PROC_VACUUM_FOR_WRAPAROUND autocancellation behavior in ProcSleep() from antiwraparound autovacuum itself. In other words I think that it should be possible to cancel an autovacuum that happens to be an antiwraparound autovacuum, just as if were any other

Re: START_REPLICATION SLOT causing a crash in an assert build

2022-10-19 Thread Andres Freund
Hi, On 2022-10-13 15:57:28 +0900, Masahiko Sawada wrote: > I've attached an updated patch. I've added the common function to > start pg_recvlogical and wait for it to become active. Please review > it. > +# Start pg_recvlogical process and wait for it to become active. > +sub

pg_recvlogical prints bogus error when interrupted

2022-10-19 Thread Andres Freund
Hi, While reviewing https://postgr.es/m/CAD21AoBe2o2D%3Dxyycsxw2bQOD%3DzPj7ETuJ5VYGN%3DdpoTiCMRJQ%40mail.gmail.com I noticed that pg_recvlogical prints "pg_recvlogical: error: unexpected termination of replication stream: " when signalled with SIGINT/TERM. Oddly enough, that looks to have

Re: shared memory stats ideas

2022-10-19 Thread Peter Geoghegan
On Wed, Oct 19, 2022 at 11:19 AM Andres Freund wrote: > We e.g. currently can't track the number of blocks written out in a relation, > because we don't have a Relation at that point. Nor can't we really get hold > of one, as the writeback can happen in a different database without access to >

Re: problems with making relfilenodes 56-bits

2022-10-19 Thread Andres Freund
Hi, On 2022-10-17 17:14:21 -0400, Robert Haas wrote: > I have to admit that I worried about the same thing that Matthias > raises, more or less. But I don't know whether I'm right to be > worried. A variable-length representation of any kind is essentially a > gamble that values requiring fewer

Re: Reducing the chunk header sizes on all memory context types

2022-10-19 Thread Andres Freund
Hi, On 2022-10-11 10:21:17 +0700, John Naylor wrote: > On Tue, Oct 11, 2022 at 5:31 AM David Rowley wrote: > > > > The proposed patches in [1] do aim to make additional usages of the > > slab allocator, and I have a feeling that we'll want to fix the > > performance of slab.c before those.

Documentation for building with meson

2022-10-19 Thread samay sharma
Hi, Creating a new thread focussed on adding docs for building Postgres with meson. This is a spinoff from the original thread [1] and I've attempted to address all the feedback provided there in the attached patch. Please let me know your thoughts. [1]

shared memory stats ideas

2022-10-19 Thread Andres Freund
Hi, shortly after shared memory stats went in I had a conversation with Lukas about what it'd enable us going forward. I also chatted with Peter about autovacuum related stats. I started to write an email, but then somehow lost the draft and couldn't bring myself to start from scratch. Here's a

Re: interrupted tap tests leave postgres instances around

2022-10-19 Thread Andres Freund
On 2022-10-19 17:38:50 +0200, Alvaro Herrera wrote: > Pushed this. It should improve things significantly. Thanks for working on this!

Re: Mingw task for Cirrus CI

2022-10-19 Thread Andres Freund
Hi, On 2022-10-19 17:59:22 +0300, Melih Mutlu wrote: > All the things you mentioned above are done. > Then I should increase CCACHE_MAXSIZE, right? Made it 500MB for MinGW. Yes. > + > + on_failure: > +<<: *on_failure_meson > +cores_script: | > + %BASH_EXE% -lc "cd build

Re: Avoid memory leaks during base backups

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 8:10 PM Robert Haas wrote: > > Well this still touches postgres.c. And I still think it's an awful > lot of machinery for a pretty trivial problem. As a practical matter, > nobody is going to open a connection and sit there and try to start a > backup over and over again

Standby recovers records from wrong timeline

2022-10-19 Thread Ants Aasma
When standby is recovering to a timeline that doesn't have any segments archived yet it will just blindly blow past the timeline switch point and keeps on recovering on the old timeline. Typically that will eventually result in an error about incorrect prev-link, but under unhappy circumstances

Re: interrupted tap tests leave postgres instances around

2022-10-19 Thread Alvaro Herrera
Pushed this. It should improve things significantly. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 6:30 PM Alvaro Herrera wrote: > > 0001 seems mostly OK, but I don't like some of these new function names. > I see you've named them so that they are case-consistent with the name > of the struct member that they affect, but I don't think that's a good > criterion. I

Re: [Proposal] Add foreign-server health checks infrastructure

2022-10-19 Thread Önder Kalacı
Hi, > As far as I can think of, it should probably be a single background task > > checking whether the server is down. If so, sending an invalidation > message > > to all the backends such that related backends could act on the > > invalidation and throw an error. This is to cover the use-case

Re: Mingw task for Cirrus CI

2022-10-19 Thread Melih Mutlu
Hi, Andres Freund , 19 Eki 2022 Çar, 06:19 tarihinde şunu yazdı: > It's a bit odd to separate the CCACHE_* variables from each other > (e.g. BUILD_DIR is inbetween them)... > > Perhaps add a comment explaining that otherwise plpython tests fail? > With this src/tools/ci/cores_backtrace.sh

Re: thinko in basic_archive.c

2022-10-19 Thread Robert Haas
On Tue, Oct 18, 2022 at 11:28 PM Kyotaro Horiguchi wrote: > > > Yeah, leaving a potentially unbounded number of files around after > > > system crashes seems pretty unfriendly. I'm not sure how to fix that, > > > exactly. > > Unbounded number of sequential crash-restarts itself is a more serious

Re: Avoid memory leaks during base backups

2022-10-19 Thread Robert Haas
On Wed, Oct 19, 2022 at 3:04 AM Bharath Rupireddy wrote: > > Echoing with what I mentioned upthread in [1], I don't quite > > understand why this patch needs to touch basebackup.c, walsender.c > > and postgres.c. In the case of a replication command processed by a > > WAL sender, memory

Re: Checking for missing heap/index files

2022-10-19 Thread Robert Haas
On Tue, Oct 18, 2022 at 5:44 PM Tom Lane wrote: > My concern about that is that it implies touching a whole lot of > places, and if you miss even one then you've lost whatever guarantee > you thought you were getting. More, there's no easy way to find > all the relevant places (some will be in

Re: allowing for control over SET ROLE

2022-10-19 Thread Robert Haas
On Sun, Oct 16, 2022 at 12:34 PM Stephen Frost wrote: > As we work through splitting up the privileges and managing them in a > more fine-grained way, it seems clear that we'll need to have a similar > split for ADMIN rights on roles- that is, we'll need to be able to > say "role X is allowed to

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Alvaro Herrera
0001 seems mostly OK, but I don't like some of these new function names. I see you've named them so that they are case-consistent with the name of the struct member that they affect, but I don't think that's a good criterion. I propose SetrunningBackups -> XLogBackupSetRunning()

RE: Perform streaming logical transactions by background workers and parallel apply

2022-10-19 Thread kuroda.hay...@fujitsu.com
Dear Hou, Thanks for updating the patch! Followings are my comments. === 01. applyparallelworker.c - SIZE_STATS_MESSAGE ``` /* * There are three fields in each message received by the parallel apply * worker: start_lsn, end_lsn and send_time. Because we have updated these * statistics in the

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 4:26 PM Bharath Rupireddy wrote: > > On Wed, Oct 19, 2022 at 4:23 PM Alvaro Herrera > wrote: > > > Thanks for looking. Pushed now. > > Thanks. I will rebase and post the other patches soon. Please review the attached v7 patch-set. -- Bharath Rupireddy PostgreSQL

Re: allowing for control over SET ROLE

2022-10-19 Thread Robert Haas
On Wed, Oct 12, 2022 at 4:59 PM Nathan Bossart wrote: > I'm not sure about tying the ownership stuff to this new SET privilege. > While you noted some practical advantages, I'd expect users to find it kind > of surprising. Also, for predefined roles, I think you need to be careful > about

Re: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-19 Thread Amit Kapila
On Wed, Oct 19, 2022 at 1:08 PM Masahiko Sawada wrote: > > On Wed, Oct 19, 2022 at 11:58 AM Masahiko Sawada > wrote: > > > > > > I've attached two patches that need to be back-patched to all branches > > and includes Change-1, Change-2, and a test case for them. FYI this > > patch resolves the

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 4:23 PM Alvaro Herrera wrote: > > On 2022-Oct-19, Bharath Rupireddy wrote: > > > When the standby is in recovery, calls to XLogInsertRecord() or > > AdvanceXLInsertBuffer()) where forcePageWrites is being used, won't > > happen, no? > > > > * Note that

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Alvaro Herrera
On 2022-Oct-19, Bharath Rupireddy wrote: > When the standby is in recovery, calls to XLogInsertRecord() or > AdvanceXLInsertBuffer()) where forcePageWrites is being used, won't > happen, no? > > * Note that forcePageWrites has no effect during an online backup from > - * the standby. >

Improve tab completion for ALTER STATISTICS

2022-10-19 Thread vignesh C
HI, I noticed that the tab completion for ALTER STATISTICS .. SET was not handled. The attached patch displays SCHEMA and STATISTICS for tab completion of ALTER STATISTICS name SET. Regards, Vignesh From 2177167ee74ab3748e23aac15c65b8391e73528f Mon Sep 17 00:00:00 2001 From: "vignesh.c" Date:

Re: make_ctags: use -I option to ignore pg_node_attr macro

2022-10-19 Thread Yugo NAGATA
On Wed, 19 Oct 2022 18:11:13 +0900 (JST) Tatsuo Ishii wrote: > > By the way, after executing both make_etags and make_ctags, trying tag jump > > in my vim causes the following error even though there are correct tags > > files. > > > > E431: Format error in tags file

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 2:30 PM Alvaro Herrera wrote: > > Another point before we move on with your 0002 is that forcePageWrites > is no longer useful and we can remove it, as per the attached. +1. The following comment enables us to rely on runningBackups and get rid of forcePageWrites

Re: make_ctags: use -I option to ignore pg_node_attr macro

2022-10-19 Thread Tatsuo Ishii
> By the way, after executing both make_etags and make_ctags, trying tag jump > in my vim causes the following error even though there are correct tags files. > > E431: Format error in tags file "backend/access/heap/TAGS" > > Removing all TAGS files as below can resolve this error. > $ find .

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Alvaro Herrera
Another point before we move on with your 0002 is that forcePageWrites is no longer useful and we can remove it, as per the attached. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "No deja de ser humillante para una persona de ingenio saber que no hay tonto que

Re: Move backup-related code to xlogbackup.c/.h

2022-10-19 Thread Alvaro Herrera
On 2022-Oct-14, Bharath Rupireddy wrote: > On Fri, Oct 14, 2022 at 1:54 PM Alvaro Herrera > wrote: > > But alternatively, we could just remove emit_warning as a flag and have > > the warning be emitted always; then we can use the boolean for the other > > purpose. I don't think the extra

Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf

2022-10-19 Thread Drouvot, Bertrand
Hi, On 10/14/22 7:30 AM, Michael Paquier wrote: On Wed, Oct 12, 2022 at 08:17:14AM +0200, Drouvot, Bertrand wrote: Indeed, ;-) The code could be split to tackle things step-by-step: - One refactoring patch to introduce token_regcomp() and token_regexec(), with the introduction of a new

Re: make_ctags: use -I option to ignore pg_node_attr macro

2022-10-19 Thread Yugo NAGATA
On Wed, 19 Oct 2022 17:17:17 +0900 (JST) Tatsuo Ishii wrote: > >> > By the way, in passing, how about adding "tags" and "TAGS" to > >> > .gitignore file? > >> > >> Sounds like a good idea. > > > > Ok, the patch is attached. > > I have search the mail archive and found this: > >

Re: make_ctags: use -I option to ignore pg_node_attr macro

2022-10-19 Thread Tatsuo Ishii
>> > By the way, in passing, how about adding "tags" and "TAGS" to >> > .gitignore file? >> >> Sounds like a good idea. > > Ok, the patch is attached. I have search the mail archive and found this:

RE: Logical replication timeout problem

2022-10-19 Thread wangw.f...@fujitsu.com
On Tue, Oct 18, 2022 at 22:35 PM Fabrice Chapuis wrote: > Hello Amit, > > In version 14.4 the timeout problem for logical replication happens again > despite > the patch provided for this issue in this version. When bulky materialized > views > are reloaded it broke logical replication. It is

Some comments that should've covered MERGE

2022-10-19 Thread Richard Guo
Hi hackers, I happened to notice $subject. Attach a trivial patch for that. Thanks Richard v1-0001-Some-comments-that-should-ve-covered-MERGE.patch Description: Binary data

Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf

2022-10-19 Thread Drouvot, Bertrand
Hi, On 10/19/22 3:18 AM, Michael Paquier wrote: On Tue, Oct 18, 2022 at 09:14:21AM +0200, Drouvot, Bertrand wrote: BTW, what about adding a new TAP test (dedicated patch) to test the behavior in case of errors during the regexes compilation in pg_ident.conf and pg_hba.conf (not done yet)? (we

Re: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-19 Thread Masahiko Sawada
On Wed, Oct 19, 2022 at 11:58 AM Masahiko Sawada wrote: > > On Tue, Oct 18, 2022 at 9:53 PM Masahiko Sawada wrote: > > > > On Tue, Oct 18, 2022 at 7:49 PM Amit Kapila wrote: > > > > > > On Tue, Oct 18, 2022 at 1:45 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > > > I think because

Re: Fix GetWALAvailability function code comments for WALAVAIL_REMOVED return value

2022-10-19 Thread Bharath Rupireddy
On Wed, Oct 19, 2022 at 12:39 PM sirisha chamarthi wrote: > > Hi Hackers, > > The current code comment says that the replication stream on a slot with the > given targetLSN can't continue after a restart but even without a restart the > stream cannot continue. The slot is invalidated and the

Fix GetWALAvailability function code comments for WALAVAIL_REMOVED return value

2022-10-19 Thread sirisha chamarthi
Hi Hackers, The current code comment says that the replication stream on a slot with the given targetLSN can't continue after a restart but even without a restart the stream cannot continue. The slot is invalidated and the walsender process is terminated by the checkpoint process. Attaching a

Re: Avoid memory leaks during base backups

2022-10-19 Thread Bharath Rupireddy
On Mon, Oct 17, 2022 at 1:09 PM Michael Paquier wrote: > > On Fri, Oct 14, 2022 at 09:56:31PM +, Cary Huang wrote: > > I applied your v5 patch on the current master and run valgrind on it > > while doing a basebackup with simulated error. No memory leak > > related to backup is observed.

Re: Getting rid of SQLValueFunction

2022-10-19 Thread Michael Paquier
(Adding Tom in CC, in case.) On Tue, Oct 18, 2022 at 04:35:33PM -0400, Corey Huinker wrote: > I agree that we shouldn't spend too much effort on a good error message > here, but perhaps we should have the message mention that it is > date/time-related? A person git-grepping for this error message

Re: Bug: pg_regress makefile does not always copy refint.so

2022-10-19 Thread Alvaro Herrera
Hi, On 2022-Oct-18, Donghang Lin wrote: > > I have a somewhat-related-but-not-really complaint. I recently had need to > > have refint.so, autoinc.so and regress.so in the install directory; but it > > turns out that there's no provision at all to get them installed. > The current pgsql