Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf

2022-10-17 Thread Michael Paquier
On Mon, Oct 17, 2022 at 07:56:02PM +0200, Drouvot, Bertrand wrote: > On 10/14/22 7:30 AM, Michael Paquier wrote: >> This approach would not stick with >> pg_ident.conf though, as we validate the fields in each line when we >> put our hands on ident_user and after the base validation of a line >>

Re: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread Japin Li
On Tue, 18 Oct 2022 at 12:00, houzj.f...@fujitsu.com wrote: > Agreed. Here is new version patch which changed the error code and > moved the whole command out of the message according to Álvaro's comment. > My bad! The patch looks good to me. -- Regrads, Japin Li. ChengDu WenWu

RE: [Proposal] Add foreign-server health checks infrastructure

2022-10-17 Thread osumi.takami...@fujitsu.com
On Monday, October 17, 2022 9:25 PM Kuroda, Hayato/黒田 隼人 wrote: > > I mainly followed the steps there and > > replaced the command "SELECT" for the remote table at 6-9 with "INSERT" > > command. > > Then, after waiting for few seconds, the "COMMIT" succeeded like below > > output, even after the

Re: pg_upgrade test failure

2022-10-17 Thread Justin Pryzby
On Tue, Oct 18, 2022 at 01:06:15PM +0900, Michael Paquier wrote: > On Tue, Oct 18, 2022 at 09:47:37AM +1300, Thomas Munro wrote: > > * Server 2019, as used on CI, still uses the traditional NT semantics > > (unlink is asynchronous, when all handles closes) > > * the fix I proposed has the right

RE: Perform streaming logical transactions by background workers and parallel apply

2022-10-17 Thread houzj.f...@fujitsu.com
On Tuesday, October 18, 2022 10:36 AM Peter Smith wrote: > > Hi, here are my review comments for patch v38-0001. Thanks for the comments. > ~~~ > > 12. get_transaction_apply_action > > I still felt like there should be some tablesync checks/comments in > this function, just for sanity, even

Re: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-17 Thread Amit Kapila
On Tue, Oct 18, 2022 at 6:29 AM Masahiko Sawada wrote: > > On Mon, Oct 17, 2022 at 4:40 PM Amit Kapila wrote: > > > > > > IIUC, here you are speaking of three different changes. Change-1: Add > > a check in AssertTXNLsnOrder() to skip assert checking till we reach > > start_decoding_at.

Re: pg_upgrade test failure

2022-10-17 Thread Michael Paquier
On Tue, Oct 18, 2022 at 09:47:37AM +1300, Thomas Munro wrote: > * Server 2019, as used on CI, still uses the traditional NT semantics > (unlink is asynchronous, when all handles closes) > * the fix I proposed has the right effect (I will follow up with tests > to demonstrate) Wow, nice

RE: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread houzj.f...@fujitsu.com
On Monday, October 17, 2022 6:14 PM Amit Kapila wrote: > > On Mon, Oct 17, 2022 at 2:41 PM Alvaro Herrera > wrote: > > > > On 2022-Oct-17, Peter Smith wrote: > > > > > On Mon, Oct 17, 2022 at 6:43 PM Alvaro Herrera > wrote: > > > > > > I'm not sure that ERRCODE_SYNTAX_ERROR is the right thing

Re: [PATCH] Allow usage of archive .backup files as backup_label

2022-10-17 Thread Laurenz Albe
On Tue, 2022-10-18 at 10:55 +0900, Michael Paquier wrote: > On Mon, Aug 22, 2022 at 05:16:58PM +0200, Michael Banck wrote: > > The .backup files written to the archive (if archiving is on) are very > > similar to the backup_label that's written/returned by > > pg_stop_backup()/pg_backup_stop(),

Re: Perform streaming logical transactions by background workers and parallel apply

2022-10-17 Thread Peter Smith
Hi, here are my review comments for patch v38-0001. == .../replication/logical/applyparallelworker.c 1. parallel_apply_start_worker + /* Try to get a free parallel apply worker. */ + foreach(lc, ParallelApplyWorkersList) + { + ParallelApplyWorkerInfo *tmp_winfo; + + tmp_winfo =

Re: Switching XLog source from archive to streaming when primary available

2022-10-17 Thread Bharath Rupireddy
On Tue, Oct 11, 2022 at 8:40 AM Nathan Bossart wrote: > > On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote: > > On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart > > wrote: > >> I wonder if it would be better to simply remove this extra polling of > >> pg_wal as a prerequisite to

Re: New "single-call SRF" APIs are very confusingly named

2022-10-17 Thread Michael Paquier
On Mon, Oct 17, 2022 at 10:13:33AM +0900, Michael Paquier wrote: > That's just assign_record_type_typmod(), which would make sure to fill > the cache for a RECORD tupdesc. How about "fill the cache with the > information of the tuple descriptor type, for a transient RECORD > datatype"? If you

Re: [PATCH] Allow usage of archive .backup files as backup_label

2022-10-17 Thread Michael Paquier
On Mon, Aug 22, 2022 at 05:16:58PM +0200, Michael Banck wrote: > The .backup files written to the archive (if archiving is on) are very > similar to the backup_label that's written/returned by > pg_stop_backup()/pg_backup_stop(), they just have a few extra lines > about the end of backup process

RE: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-17 Thread kuroda.hay...@fujitsu.com
Dear Sawada-san, Amit, > IIUC Change-2 is required in v16 and HEAD but not mandatory in v15 and > v14. The reason why we need Change-2 is that there is a case where we > mark only subtransactions as containing catalog change while not doing > that for its top-level transaction. In v15 and v14,

Re: Unnecessary lateral dependencies implied by PHVs

2022-10-17 Thread Andy Fan
Hi Richard: On Mon, Oct 10, 2022 at 10:35 AM Richard Guo wrote: > ... I'm asking because > PHVs may imply lateral dependencies which may make us have to use > nestloop join. > I thought lateral join imply nestloop join, am I missing something? Here is my simple testing. postgres=# explain

Re: POC: Better infrastructure for automated testing of concurrency issues

2022-10-17 Thread Craig Ringer
On Tue, 23 Feb 2021 at 08:09, Peter Geoghegan wrote: > On Tue, Dec 8, 2020 at 2:42 AM Alexander Korotkov > wrote: > > Thank you for your feedback! > > It would be nice to use this patch to test things that are important > but untested inside vacuumlazy.c, such as the rare >

Re: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-17 Thread Masahiko Sawada
On Mon, Oct 17, 2022 at 4:40 PM Amit Kapila wrote: > > On Wed, Oct 12, 2022 at 11:18 AM Masahiko Sawada > wrote: > > > > Please note that to pass the new regression tests, the fix proposed in > > a related thread[1] is required. Particularly, we need: > > > > @@ -1099,6 +1099,9 @@

Re: havingQual vs hasHavingQual buglets

2022-10-17 Thread Richard Guo
On Tue, Oct 18, 2022 at 5:37 AM Tom Lane wrote: > I came across a couple of places in the planner that are checking > for nonempty havingQual; but since these bits run after > const-simplification of the HAVING clause, that produces the wrong > answer for a constant-true HAVING clause (which'll

Re: Allow WindowFuncs prosupport function to use more optimal WindowClause options

2022-10-17 Thread David Rowley
Thanks for having a look at this. On Fri, 14 Oct 2022 at 10:52, Zhihong Yu wrote: > + req->frameOptions = (FRAMEOPTION_ROWS | > +FRAMEOPTION_START_UNBOUNDED_PRECEDING | > +FRAMEOPTION_END_CURRENT_ROW); > > The bit combination appears

Re: Warning about using pg_stat_reset() and pg_stat_reset_shared()

2022-10-17 Thread David Rowley
On Tue, 18 Oct 2022 at 08:07, Bruce Momjian wrote: > Patch applied back to PG 10, thanks. Thanks. David

Re: Allow WindowFuncs prosupport function to use more optimal WindowClause options

2022-10-17 Thread David Rowley
On Tue, 18 Oct 2022 at 12:18, Tom Lane wrote: > > Erwin Brandstetter writes: > > I am thinking of building a test case to run > > - all existing window functions > > - with all basic variants of frame definitions > > - once with ROWS, once with RANGE > > - on basic table that has duplicate and

Re: Allow WindowFuncs prosupport function to use more optimal WindowClause options

2022-10-17 Thread Tom Lane
Erwin Brandstetter writes: > I am thinking of building a test case to run > - all existing window functions > - with all basic variants of frame definitions > - once with ROWS, once with RANGE > - on basic table that has duplicate and NULL values in partition and > ordering columns > - in all

Re: Allow WindowFuncs prosupport function to use more optimal WindowClause options

2022-10-17 Thread Erwin Brandstetter
On Thu, 13 Oct 2022 at 02:34, David Rowley wrote: > On Wed, 12 Oct 2022 at 16:33, Vik Fearing wrote: > > Per spec, the ROW_NUMBER() window function is not even allowed to have a > > frame specified. > > > > b) The window framing clause of WDX shall not be present. > > > > Also, the

havingQual vs hasHavingQual buglets

2022-10-17 Thread Tom Lane
I came across a couple of places in the planner that are checking for nonempty havingQual; but since these bits run after const-simplification of the HAVING clause, that produces the wrong answer for a constant-true HAVING clause (which'll be folded to empty). Correct code is to check

Re: problems with making relfilenodes 56-bits

2022-10-17 Thread Robert Haas
On Wed, Oct 12, 2022 at 5:13 PM Andres Freund wrote: > > I think a signficant part of this improvement comes from the premise > > of starting with a fresh database. tablespace OID will indeed most > > likely be low, but database OID may very well be linearly distributed > > if concurrent

Re: pg_upgrade test failure

2022-10-17 Thread Thomas Munro
On Mon, Oct 3, 2022 at 7:29 PM Michael Paquier wrote: > On Mon, Oct 03, 2022 at 04:03:12PM +1300, Thomas Munro wrote: > > So I think that setting is_lnk = false is good enough here. Do > > you see a hole in it? > > I cannot think on one, on top of my head. Thanks for the > explanation. Some

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-10-17 Thread Andres Freund
On 2022-10-17 13:34:02 -0400, Robert Haas wrote: > I don't feel quite as confident that not attempting a cleanup lock on > the new bucket's primary page is OK. I think it should be fine. The > existing comment even says it should be fine. But, that comment could > be wrong, and I'm not sure that I

Re: pub/sub - specifying optional parameters without values.

2022-10-17 Thread Justin Pryzby
On Fri, Oct 14, 2022 at 07:54:37PM +1100, Peter Smith wrote: > Hi hackers. > > This post is about parameter default values. Specifically. it's about > the CREATE PUBLICATION and CREATE SUBSCRIPTION syntax, although the > same issue might apply to other commands I am unaware of... The same thing

Re: Warning about using pg_stat_reset() and pg_stat_reset_shared()

2022-10-17 Thread Bruce Momjian
On Wed, Oct 12, 2022 at 12:04:08PM -0400, Bruce Momjian wrote: > > Maybe the random sample taken by ANALYZE for your case didn't happen > > to land on any pages with dead tuples? > > Ah, good point, I missed that in pgstat_report_analyze(). I will apply > the patch then in a few days, thanks.

Re: Eliminating SPI from RI triggers - take 2

2022-10-17 Thread Robert Haas
On Sat, Oct 15, 2022 at 1:47 AM Amit Langote wrote: > I have merged your incremental patch into 0003. Note that if someone goes to commit 0003, they would have no idea that I contributed to the effort. You should probably try to keep a running list of co-authors, reviewers, or other people that

Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf

2022-10-17 Thread Drouvot, Bertrand
Hi, On 10/14/22 7:30 AM, Michael Paquier wrote: On Wed, Oct 12, 2022 at 08:17:14AM +0200, Drouvot, Bertrand wrote: Indeed, ;-) I have also looked at make_auth_token(), and wondered if it could be possible to have this routine compile the regexes. I think that it makes sense. This

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-10-17 Thread Robert Haas
On Mon, Oct 17, 2022 at 1:02 PM Andres Freund wrote: > That's true in general, but the case of fixing a bug in one place but not in > another nearby is a different story. I agree, but I still think we shouldn't let the perfect be the enemy of the good. > > That code indeed seems stupid, because

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-10-17 Thread Andres Freund
Hi, On 2022-10-17 10:43:16 -0400, Robert Haas wrote: > On Fri, Oct 14, 2022 at 2:21 PM Andres Freund wrote: > > On 2022-10-14 10:40:11 +0530, Amit Kapila wrote: > > > On Fri, Oct 14, 2022 at 2:25 AM Andres Freund wrote: > > > Fair point. How about something like: "XXX Do we really need to check

Re: ssl tests aren't concurrency safe due to get_free_port()

2022-10-17 Thread Andrew Dunstan
On 2022-10-04 Tu 01:39, Andrew Dunstan wrote: > On 2022-10-02 Su 12:49, Andres Freund wrote: >> 2) Use a lockfile containing a pid to protect the choice of a port within a >>build directory. Before accepting a port get_free_port() would check if >> the >>a lockfile exists for the port

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-10-17 Thread Robert Haas
On Fri, Oct 14, 2022 at 2:21 PM Andres Freund wrote: > On 2022-10-14 10:40:11 +0530, Amit Kapila wrote: > > On Fri, Oct 14, 2022 at 2:25 AM Andres Freund wrote: > > Fair point. How about something like: "XXX Do we really need to check > > for cleanup lock on the new bucket? Here, we initialize

Re: PATCH: Using BRIN indexes for sorted output

2022-10-17 Thread Matthias van de Meent
On Mon, 17 Oct 2022 at 05:43, Tomas Vondra wrote: > > On 10/16/22 22:17, Matthias van de Meent wrote: > > On Sun, 16 Oct 2022 at 16:34, Tomas Vondra > > wrote: > >> Try to formulate the whole algorithm. Maybe I'm missing something. > >> > >> The current algorithm is something like this: > >> >

Re: Question about pull_up_sublinks_qual_recurse

2022-10-17 Thread Andy Fan
Hi Tom: Thanks for your reply! I have self reviewed the below message at 3 different time periods to prevent from too inaccurate replies. It may be more detailed than it really needed, but it probably can show where I am lost. On Sat, Oct 15, 2022 at 3:27 AM Tom Lane wrote: > > If the

Re: thinko in basic_archive.c

2022-10-17 Thread Robert Haas
On Fri, Oct 14, 2022 at 4:45 AM Bharath Rupireddy wrote: > What happens to the left-over temp files after a server crash? Will > they be lying around in the archive directory? I understand that we > can't remove such files because we can't distinguish left-over files > from a crash and the temp

RE: [Proposal] Add foreign-server health checks infrastructure

2022-10-17 Thread kuroda.hay...@fujitsu.com
Dear Osumi-san, > I mainly followed the steps there and > replaced the command "SELECT" for the remote table at 6-9 with "INSERT" > command. > Then, after waiting for few seconds, the "COMMIT" succeeded like below output, > even after the server stop of the worker side. > Additionally, the last

RE: [Proposal] Add foreign-server health checks infrastructure

2022-10-17 Thread kuroda.hay...@fujitsu.com
Dear Horiguchi-san, > Might be on slight different direction, but it looks to me a bit too > much to use WaitEventSet to check only if a socket is live or not. > > A quick search in the tree told me that we could use pqSocketCheck() > instead, and I think it would be the something that "could

Re: create subscription - improved warning message

2022-10-17 Thread Amit Kapila
On Mon, Oct 17, 2022 at 7:17 AM Peter Smith wrote: > > > Updated as sugggested. > + +Sometimes, either by choice (e.g. create_slot = false), +or by necessity (e.g. connect = false), the remote +replication slot is not created automatically during +CREATE SUBSCRIPTION. In these

Re: remove no longer necessary Perl compatibility hack

2022-10-17 Thread Richard Guo
On Mon, Oct 17, 2022 at 4:24 PM Alvaro Herrera wrote: > While messing about with Cluster.pm I noticed that we don't need the > hack to work around lack of parent.pm in very old Perl versions, because > we no longer support those versions (per commit 4c1532763a00). Trivial > patch attached.

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2022-10-17 Thread Damir Belyalov
Updated the patch due to conflicts when applying to master. > diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml index c25b52d0cb..22c992e6f6 100644 --- a/doc/src/sgml/ref/copy.sgml +++ b/doc/src/sgml/ref/copy.sgml @@ -34,6 +34,7 @@ COPY { table_name [ ( format_name FREEZE

Re: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread Amit Kapila
On Mon, Oct 17, 2022 at 2:41 PM Alvaro Herrera wrote: > > On 2022-Oct-17, Peter Smith wrote: > > > On Mon, Oct 17, 2022 at 6:43 PM Alvaro Herrera > > wrote: > > > > I'm not sure that ERRCODE_SYNTAX_ERROR is the right thing here; sounds > > > like ERRCODE_FEATURE_NOT_SUPPORTED might be more

Bitmapsets as Nodes

2022-10-17 Thread Amit Langote
Hi, For a couple of patches that I am working on ([1], [2]), I have needed to put Bitmapsets into a List that is in turn part of a Plan tree or a Node tree that may be written (using outNode) and read (using nodeRead). Bitmapsets not being a Node themselves causes the write/read of such

Re: generic plans and "initial" pruning

2022-10-17 Thread Amit Langote
On Wed, Oct 12, 2022 at 4:36 PM Amit Langote wrote: > On Fri, Jul 29, 2022 at 1:20 PM Amit Langote wrote: > > On Thu, Jul 28, 2022 at 1:27 AM Robert Haas wrote: > > > 0001 adds es_part_prune_result but does not use it, so maybe the > > > introduction of that field should be deferred until it's

Re: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread Alvaro Herrera
On 2022-Oct-17, Peter Smith wrote: > On Mon, Oct 17, 2022 at 6:43 PM Alvaro Herrera > wrote: > > I'm not sure that ERRCODE_SYNTAX_ERROR is the right thing here; sounds > > like ERRCODE_FEATURE_NOT_SUPPORTED might be more appropriate. > > I thought maybe

Re: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread Peter Smith
On Mon, Oct 17, 2022 at 6:43 PM Alvaro Herrera wrote: > > Hello > > On 2022-Oct-17, houzj.f...@fujitsu.com wrote: > > > alter subscription sub add publication pub2; > > > Because I was executing the ADD PUBLICATION command, I feel the hint should > > also mention it instead of SET PUBLICATION. >

RFC: multi TOAST-tables support

2022-10-17 Thread Nikita Malakhov
Hi hackers! Reference TOAST mechanics assumes that a relation has a single TOAST relation for all it's TOASTable columns. While working on Pluggable TOAST [1] we've found that single TOAST relation for a relation is a bit of a problem, because different Toasters could have different TOAST table

Re: [Proposal] Add foreign-server health checks infrastructure

2022-10-17 Thread Kyotaro Horiguchi
At Mon, 17 Oct 2022 07:27:21 +, "kuroda.hay...@fujitsu.com" wrote in > > In other words, a variation of pgfdw_connection_check_internal() > > could potentially go into interfaces/libpq/libpq-fe.h > > (backend/libpq/pqcomm.c or src/interfaces/libpq/fe-connect.c). > > Hmm, IIUC libpq related

remove no longer necessary Perl compatibility hack

2022-10-17 Thread Alvaro Herrera
Hello While messing about with Cluster.pm I noticed that we don't need the hack to work around lack of parent.pm in very old Perl versions, because we no longer support those versions (per commit 4c1532763a00). Trivial patch attached. -- Álvaro HerreraBreisgau, Deutschland —

RE: create subscription - improved warning message

2022-10-17 Thread shiy.f...@fujitsu.com
On Mon, Oct 17, 2022 9:47 AM Peter Smith wrote: > > On Sun, Oct 16, 2022 at 12:14 AM Amit Kapila > wrote: > > > > On Fri, Oct 14, 2022 at 8:22 AM Peter Smith > wrote: > > > > > > On Thu, Oct 13, 2022 at 9:07 AM Peter Smith > wrote: > > > > > ... > > > PSA a patch for adding examples of how to

Re: Improve errhint for ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2022-10-17 Thread Alvaro Herrera
Hello On 2022-Oct-17, houzj.f...@fujitsu.com wrote: > alter subscription sub add publication pub2; > Because I was executing the ADD PUBLICATION command, I feel the hint should > also mention it instead of SET PUBLICATION. Hmm, ok. But: > @@ -1236,8 +1237,9 @@ AlterSubscription(ParseState

Re: TRAP: FailedAssertion("prev_first_lsn < cur_txn->first_lsn", File: "reorderbuffer.c", Line: 927, PID: 568639)

2022-10-17 Thread Amit Kapila
On Wed, Oct 12, 2022 at 11:18 AM Masahiko Sawada wrote: > > Please note that to pass the new regression tests, the fix proposed in > a related thread[1] is required. Particularly, we need: > > @@ -1099,6 +1099,9 @@ SnapBuildCommitTxn(SnapBuild *builder, > XLogRecPtr lsn, TransactionId xid, >

Re: Avoid memory leaks during base backups

2022-10-17 Thread Michael Paquier
On Fri, Oct 14, 2022 at 09:56:31PM +, Cary Huang wrote: > I applied your v5 patch on the current master and run valgrind on it > while doing a basebackup with simulated error. No memory leak > related to backup is observed. Regression is also passing. Echoing with what I mentioned upthread in

RE: [Proposal] Add foreign-server health checks infrastructure

2022-10-17 Thread kuroda.hay...@fujitsu.com
Dear Önder, Thanks for giving suggestions! > Still, the reestablish mechanism can be further simplified with > WL_SOCKET_CLOSED event such as the following (where we should probably > rename pgfdw_connection_check_internal): Sounds reasonable. I think it may be included in this patch. I will

Re: Add regular expression testing for user name mapping in the peer authentication TAP test

2022-10-17 Thread Drouvot, Bertrand
Hi, On 10/17/22 4:07 AM, Michael Paquier wrote: On Sat, Oct 15, 2022 at 07:54:30AM +0200, Drouvot, Bertrand wrote: Right. Giving a second thought to the non matching case, I think I'd prefer to concatenate the system_user to the system_user instead. This is what v2 does. Fine by me, so

Re: Suppressing useless wakeups in walreceiver

2022-10-17 Thread Kyotaro Horiguchi
Thanks for taking this up. At Sat, 15 Oct 2022 20:59:00 -0700, Nathan Bossart wrote in > On Thu, Oct 13, 2022 at 12:09:54PM -0700, Nathan Bossart wrote: > > On Thu, Oct 13, 2022 at 12:37:39PM +0200, Alvaro Herrera wrote: > >> The main reason is that it seems odd to have startpointTLI in the