RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Peter, > 1. > +-- fail - utilizing streaming = parallel with time-delayed > replication is not supported > +CREATE SUBSCRIPTION regress_testsub CONNECTION > 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = > false, streaming = parallel, min_send_delay = 123); > >

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for reviewing! PSA new version. > 1. > If the subscription sets min_send_delay parameter, an apply worker passes the > value to the publisher as an output plugin option. And then, the walsender > will > delay the transaction sending for given milliseconds. > > ~ > > 1a.

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-02-20 Thread Jim Jones
I'm withdrawing this patch, as the same feature was already implemented in a different patch written by Jacob[1] Thanks everyone! Best, Jim 1- https://www.postgresql.org/message-id/flat/caawbhmi4v9zeavfuscdfx1por3zwrv9fuxkv_2marqvyc-m...@mail.gmail.com#199c1f49fbefa6be401db35f5cfa7742

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2023-02-20 Thread Amit Langote
On Tue, Feb 21, 2023 at 12:40 AM Tom Lane wrote: > Alvaro Herrera writes: > > On 2023-Feb-20, Amit Langote wrote: > >> One more thing we could try is come up with a postgres_fdw test case, > >> because it uses the RelOptInfo.userid value for remote-costs-based > >> path size estimation. But

Re: [Proposal] Add foreign-server health checks infrastructure

2023-02-20 Thread Peter Smith
Here are some review comments for v32-0001. == Commit message 1. PQconnCheck() function allows to check the status of the socket by polling the socket. This function is currently available only on systems that support the non-standard POLLRDHUP extension to the poll system call, including

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Michael Paquier
On Mon, Feb 20, 2023 at 04:43:22PM -0800, Nathan Bossart wrote: > On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: >> On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >>> I'm happy to move this new test to wherever folks think it should go. I'll >>> look around to

Re: WAL Insertion Lock Improvements

2023-02-20 Thread Nathan Bossart
On Thu, Feb 09, 2023 at 11:51:28AM +0530, Bharath Rupireddy wrote: > On Thu, Feb 9, 2023 at 3:36 AM Nathan Bossart > wrote: >> Overall, I think this patch is in reasonable shape. > > Thanks for reviewing. Please see the attached v5 patch. I'm marking this as ready-for-committer. I think a

Re: SQL/JSON revisited

2023-02-20 Thread Amit Langote
On Tue, Feb 21, 2023 at 12:09 PM Amit Langote wrote: > On Mon, Feb 20, 2023 at 11:41 PM Erik Rijkers wrote: > > Op 20-02-2023 om 08:35 schreef Amit Langote: > > > Rebased again over queryjumble overhaul. > > But the following statement is a problem. It does not crash but it goes > > off,

Re: Weird failure with latches in curculio on v15

2023-02-20 Thread Nathan Bossart
On Tue, Feb 21, 2023 at 09:03:27AM +0900, Michael Paquier wrote: > On Tue, Feb 14, 2023 at 09:47:55AM -0800, Nathan Bossart wrote: >> Here is a new version of the stopgap/back-branch fix for restore_command. >> This is more or less a rebased version of v4 with an added stderr message >> as Andres

Re: File descriptors in exec'd subprocesses

2023-02-20 Thread Thomas Munro
Something bothered me about the previous versions: Which layer should add O_CLOEXEC, given that we have md.c -> PathNameOpenXXX() -> BasicOpenFile() -> open()? Previously I had md.c adding it, but on reflection, it makes no sense to open a "File" (virtual file descriptor) that is *not*

Allow ordered partition scans in more cases

2023-02-20 Thread David Rowley
Over on [1], Benjamin highlighted that we don't do ordered partition scans in some cases where we could. Basically, what was added in 959d00e9d only works when at least one child path has pathkeys that suit the required query pathkeys. If the partitions or partitioned table does not have an

Re: Add WAL read stats to pg_stat_wal

2023-02-20 Thread Kyotaro Horiguchi
At Mon, 20 Feb 2023 20:15:00 +0530, Bharath Rupireddy wrote in > Having said that, what's the problem if we use shared memory to report > the shutdown checkpoint to the postmaster? In case of abnormal > shutdown where shared memory gets corrupted, we don't even write a > shutdown checkpoint,

Re: Add WAL read stats to pg_stat_wal

2023-02-20 Thread Kyotaro Horiguchi
At Mon, 20 Feb 2023 08:29:06 -0800, Andres Freund wrote in > Hi, > > On 2023-02-20 14:21:39 +0900, Kyotaro Horiguchi wrote: > > I totally agree that it will be useful, but I'm not quite sure how > > checkpointer would be able to let postmaster know about that state > > without requiring access

psql memory leaks

2023-02-20 Thread Kyotaro Horiguchi
I noticed that \bind is leaking memory for each option. =# SELECT $1, $2, $3 \ bind 1 2 3 \g The leaked memory blocks are comming from psql_scan_slash_option(). The attached small patch resolves that issue. I looked through the function's call sites, but I didn't find the same mistake.

Re: psql \watch 2nd argument: iteration count

2023-02-20 Thread Kyotaro Horiguchi
At Mon, 20 Feb 2023 10:45:53 -0800, Andrey Borodin wrote in > > > > Also, if we do not provide a timespan, 2 seconds are selected. But if > > > > we provide an incorrect argument - 1 second is selected. > > > > PFA the patch that adds iteration count argument and makes timespan > > > > argument

Re: Support logical replication of DDLs

2023-02-20 Thread Zheng Li
On Mon, Feb 20, 2023 at 3:23 AM Masahiko Sawada wrote: > > On Fri, Feb 17, 2023 at 1:13 PM Zheng Li wrote: > > > > > > I've implemented a prototype to allow replicated objects to have the > > > > same owner from the publisher in > > > >

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-20 Thread Amit Kapila
On Tue, Feb 21, 2023 at 3:31 AM Peter Smith wrote: > > > 2. > The combination of parallel streaming mode and min_send_delay is not allowed. > This is because in parallel streaming mode, we start applying the transaction > stream as soon as the first change arrives without knowing the

meson logs environment

2023-02-20 Thread Andrew Dunstan
I've noticed that `meson test` logs the complete environment in meson_logs/testlog.txt. That seems unnecessary and probably undesirable for the buildfarm client. Is there any way to suppress that, or at least only print some relevant subset? (The buildfarm client itself only reports an

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-20 Thread Peter Smith
Here are some review comments for the v3-0001 test code. == src/test/regress/sql/subscription.sql 1. +-- fail - utilizing streaming = parallel with time-delayed replication is not supported +CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: > On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >> I'm happy to move this new test to wherever folks think it should go. I'll >> look around to see if I can find a better place, too. > > I think that

Re: Rework of collation code, extensibility

2023-02-20 Thread Jeff Davis
On Mon, 2023-02-13 at 15:45 -0800, Jeff Davis wrote: > New version attached. Changes: These patches, especially 0001, have been around for a while, and they've received some review attention with no outstanding TODOs that I'm aware of. I plan to commit v10 (or something close to it) soon unless

Re: Weird failure with latches in curculio on v15

2023-02-20 Thread Thomas Munro
On Tue, Feb 21, 2023 at 1:03 PM Michael Paquier wrote: > Perhaps beginning a new thread with a patch and a summary would be > better at this stage? Another thing I am wondering is if it could be > possible to test that rather reliably. I have been playing with a few > scenarios like holding the

Re: Weird failure with latches in curculio on v15

2023-02-20 Thread Michael Paquier
On Tue, Feb 14, 2023 at 09:47:55AM -0800, Nathan Bossart wrote: > Here is a new version of the stopgap/back-branch fix for restore_command. > This is more or less a rebased version of v4 with an added stderr message > as Andres suggested upthread. So, this thread has moved around many subjects,

meson and sslfiles.mk in src/test/ssl/

2023-02-20 Thread Michael Paquier
Hi all, While playing with the SSL tests last week, I have noticed that we don't have a way to regenerate the SSL files with meson for its TAP suite. It seems to me that we had better transfer the rules currently stored in sslfiles.mk into something that meson can use? Another approach may be

Re: Support logical replication of DDLs

2023-02-20 Thread Jonathan S. Katz
On 2/19/23 11:14 PM, Amit Kapila wrote: On Sun, Feb 19, 2023 at 7:50 AM Jonathan S. Katz wrote: On 2/17/23 4:15 AM, Amit Kapila wrote: This happens because of the function used in the index expression. Now, this is not the only thing, the replication can even fail during DDL replication

Disable rdns for Kerberos tests

2023-02-20 Thread Stephen Frost
Greetings, The name canonicalization support for Kerberos is doing us more harm than good in the regression tests, so I propose we disable it. Patch attached. Thoughts? Thanks, Stephen From 992d946d17c79d240ac6587998e2f94b12a726de Mon Sep 17 00:00:00 2001 From: Stephen Frost Date: Mon, 20

Re: ICU locale validation / canonicalization

2023-02-20 Thread Jeff Davis
New patch attached. The new patch also includes a GUC that (when enabled) validates that the collator is actually found. On Mon, 2023-02-20 at 15:46 +0100, Peter Eisentraut wrote: > a) BCP47 tags are preferred, and Agreed. > b) They don't work with ICU versions before 54. I tried in versions

Re: Weird failure with latches in curculio on v15[

2023-02-20 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Sun, Feb 19, 2023 at 08:06:24PM +0530, Robert Haas wrote: > > I mean, my idea was to basically just have one big callback: > > ArchiverModuleMainLoopCB(). Which wouldn't return, or perhaps, would > > only return when archiving was

Re: [PATCH] Add pretty-printed XML output option

2023-02-20 Thread Jim Jones
On 18.02.23 19:09, Peter Eisentraut wrote: On 17.02.23 23:24, Nikolay Samokhvalov wrote: My idea was to follow the SQL standard (part 14, SQL/XML); unfortunately, there is no free version, but there are drafts at http://www.wiscorp.com/SQLStandards.html

Re: improving user.c error messages

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 11:02:10AM -0800, Nathan Bossart wrote: > On Mon, Feb 20, 2023 at 08:54:48AM +0100, Peter Eisentraut wrote: >> I'm concerned about the loose use of "privilege" here. A privilege is >> something I can grant. So if someone doesn't have the "REPLICATION >> privilege", as in

Re: pg_walinspect memory leaks

2023-02-20 Thread Michael Paquier
On Mon, Feb 20, 2023 at 11:34:03AM -0800, Jeff Davis wrote: > Committed to 16 with the changes to GetXLogSummaryStats() as well. > Committed unmodified version of your 15 backport. Thank you! Thanks for taking care of the FPI code path, Jeff! -- Michael signature.asc Description: PGP signature

Re: Weird failure with latches in curculio on v15[

2023-02-20 Thread Michael Paquier
On Sun, Feb 19, 2023 at 08:06:24PM +0530, Robert Haas wrote: > I mean, my idea was to basically just have one big callback: > ArchiverModuleMainLoopCB(). Which wouldn't return, or perhaps, would > only return when archiving was totally caught up and there was nothing > more to do right now. And

Re: Proposal: Support custom authentication methods using hooks

2023-02-20 Thread Stephen Frost
Greetings, * Andrey Chudnovsky (achudnovs...@gmail.com) wrote: > The thread link is here: > https://www.postgresql.org/message-id/flat/CABkiuWo4fJQ7dhqgYLtJh41kpCkT6iXOO8Eym3Rdh5tx2RJCJw%40mail.gmail.com#f94c36969a68a07c087fa9af0f5401e1 Thanks for pointing out the updates on that thread, I've

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-02-20 Thread Stephen Frost
Greetings, * mahendrakar s (mahendrakarfo...@gmail.com) wrote: > The "issuer" field has been removed to align with the RFC > implementation - https://www.rfc-editor.org/rfc/rfc7628. > This patch "v6" is a single patch to support the OAUTH BEARER token > through psql connection string. > Below

Re: Silent overflow of interval type

2023-02-20 Thread Tom Lane
Nick Babadzhanian writes: > Please find attached the v2 of the said patch with the tests added. Pushed with light editing (for instance, I don't think interval.sql is the place to test timestamp operators, even if the result is an interval). > I don't know how the decision on > backpatching is

Re: Missing free_var() at end of accum_sum_final()?

2023-02-20 Thread Joel Jacobson
Hi, My apologies, it seems my email didn't reach the list, probably due to the benchmark plot images being to large. Here is the email again, but with URLs to the images instead, and benchmark updated with results for the 0005-fixed-buf.patch. -- On Mon, Feb 20, 2023, at 12:32, Dean Rasheed

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-20 Thread Peter Smith
Here are some review comments for patch v3-0001. (I haven't looked at the test code yet) == Commit Message 1. If the subscription sets min_send_delay parameter, an apply worker passes the value to the publisher as an output plugin option. And then, the walsender will delay the transaction

Re: Killing off removed rels properly

2023-02-20 Thread Tom Lane
I wrote: > But while I'm looking at this, 3c569049b seems kind of broken on > its own terms. Why is it populating so little of the IndexOptInfo > for a partitioned index? I realize that we're not going to directly > plan anything using such an index, but not populating fields like > sortopfamily

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2023-02-20 Thread Alvaro Herrera
Found one last problem: if you do "-f foobar.sql -M prepared" in that order, then the prepare fails because the statement names would not be assigned when the file is parsed. This coding only supported doing "-M prepared -f foobar.sql", which funnily enough is the only one that

Allow MERGE to be executed from PL/Tcl

2023-02-20 Thread Dean Rasheed
Another one noticed in the MERGE RETURNING patch -- this allows PL/Tcl to execute MERGE (i.e., don't fail when SPI returns SPI_OK_MERGE). I'm not sure if anyone uses PL/Tcl anymore, but it's a trivial fix, probably not worth a regression test case. Regards, Dean diff --git a/src/pl/tcl/pltcl.c

Re: Missing cases from SPI_result_code_string()

2023-02-20 Thread Dean Rasheed
On Mon, 20 Feb 2023 at 19:39, Tom Lane wrote: > > Ugh. Grepping around, it looks like pltcl_process_SPI_result > is missing a case for SPI_OK_MERGE as well. > Yes, I was about to post a patch for that too. That's the last case that I found, looking around. Regards, Dean

Re: meson: Optionally disable installation of test modules

2023-02-20 Thread Andres Freund
On 2023-02-20 19:43:56 +0100, Peter Eisentraut wrote: > I don't think any callers try to copy a directory source, so the > shutil.copytree() stuff isn't necessary. I'd like to use it for installing docs outside of the normal install target. Of course we could add the ability at a later point, but

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-20 Thread Andres Freund
Hi, On 2023-02-21 08:33:22 +1300, David Rowley wrote: > On Tue, 21 Feb 2023 at 07:30, Andres Freund wrote: > > 2) We should introduce an API mcxt.c API to perform allocations that the > >caller promises not to individually free. > > It's not just pfree. Offhand, there's also repalloc, >

Re: meson: Non-feature feature options

2023-02-20 Thread Andres Freund
Hi, On 2023-02-20 19:53:53 +0100, Peter Eisentraut wrote: > On 20.02.23 13:33, Nazir Bilal Yavuz wrote: > > I added SSL and UUID patches. UUID patch has two different fixes: > > > > 1 - v1-0002-meson-Refactor-UUID-option.patch: Adding 'auto' choice to > > 'uuid' combo option. > > > > 2 -

Re: Move defaults toward ICU in 16?

2023-02-20 Thread Jeff Davis
On Mon, 2023-02-20 at 15:55 +0100, Peter Eisentraut wrote: > I'm confused.  We are not going to try to change existing databases > to a > different collation provider during pg_upgrade, are we? No, certainly not. I interpreted Pavel's comments as a comparison of ICU and libc in general and not

Re: Missing cases from SPI_result_code_string()

2023-02-20 Thread Tom Lane
Dean Rasheed writes: > Another one noticed from the MERGE RETURNING patch -- the switch > statement in SPI_result_code_string() is missing cases for > SPI_OK_TD_REGISTER and SPI_OK_MERGE. Ugh. Grepping around, it looks like pltcl_process_SPI_result is missing a case for SPI_OK_MERGE as well.

Re: meson: Non-feature feature options

2023-02-20 Thread Daniel Gustafsson
> On 20 Feb 2023, at 19:53, Peter Eisentraut > wrote: > I would rename the ssl_type variable to ssl_library, so that if we ever > expose that as an option, it would be consistent with uuid_library. +1, ssl_library is a better name. -- Daniel Gustafsson

Re: pg_walinspect memory leaks

2023-02-20 Thread Jeff Davis
On Mon, 2023-02-20 at 15:17 +0530, Bharath Rupireddy wrote: > Similarly, the loops in GetXLogSummaryStats() too > don't palloc any memory, so no memory leak. Break on palloc in gdb in that loop and you'll see a palloc in CStringGetTextDatum(name). In general, you should expect *GetDatum() to

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-20 Thread David Rowley
On Tue, 21 Feb 2023 at 07:30, Andres Freund wrote: > 2) We should introduce an API mcxt.c API to perform allocations that the >caller promises not to individually free. It's not just pfree. Offhand, there's also repalloc, GetMemoryChunkSpace and GetMemoryChunkContext too. I am interested in

Re: Killing off removed rels properly

2023-02-20 Thread Tom Lane
I wrote: >> It looks like we're somehow triggering the elide-a-left-join code >> when we shouldn't? So the reason why we see this with a partitioned target table and not with a regular target table reduces to this bit in preprocess_targetlist: /* * For non-inherited UPDATE/DELETE/MERGE,

Re: meson: Non-feature feature options

2023-02-20 Thread Nazir Bilal Yavuz
Hi, On Mon, 20 Feb 2023 at 21:53, Peter Eisentraut wrote: > > But what does uuid_library=auto do? Which one does it pick? This is > not a behavior we currently have, is it? Yes, we didn't have that behavior before. It checks uuid libs by the order of 'e2fs', 'bsd' and 'ossp'. It uses the

Re: improving user.c error messages

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 08:54:48AM +0100, Peter Eisentraut wrote: > I'm concerned about the loose use of "privilege" here. A privilege is > something I can grant. So if someone doesn't have the "REPLICATION > privilege", as in the above example, I would expect to be able to do "GRANT >

Re: meson: Non-feature feature options

2023-02-20 Thread Peter Eisentraut
On 20.02.23 13:33, Nazir Bilal Yavuz wrote: I added SSL and UUID patches. UUID patch has two different fixes: 1 - v1-0002-meson-Refactor-UUID-option.patch: Adding 'auto' choice to 'uuid' combo option. 2 - v1-0002-meson-Refactor-UUID-option-with-uuid_library.patch: Making 'uuid' feature

Missing cases from SPI_result_code_string()

2023-02-20 Thread Dean Rasheed
Another one noticed from the MERGE RETURNING patch -- the switch statement in SPI_result_code_string() is missing cases for SPI_OK_TD_REGISTER and SPI_OK_MERGE. The SPI_OK_TD_REGISTER case goes back all the way, so I suppose it should be back-patched to all supported branches, though evidently

Re: psql \watch 2nd argument: iteration count

2023-02-20 Thread Andrey Borodin
> > > Also, if we do not provide a timespan, 2 seconds are selected. But if > > > we provide an incorrect argument - 1 second is selected. > > > PFA the patch that adds iteration count argument and makes timespan > > > argument more consistent. > > > > That should probably be fixed. > > And should

Re: meson: Optionally disable installation of test modules

2023-02-20 Thread Peter Eisentraut
On 09.02.23 16:30, Nazir Bilal Yavuz wrote: On 2/8/23 13:30, Peter Eisentraut wrote: If you feel that your patch is ready, please add it to the commit fest. I look forward to reviewing it. Thanks! Commit fest entry link: https://commitfest.postgresql.org/42/4173/ I tested this a bit. It

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-20 Thread Andres Freund
Hi, On 2023-02-17 09:52:01 -0800, Andres Freund wrote: > On 2023-02-17 17:26:20 +1300, David Rowley wrote: > Random note: > > I wonder if we should having a bitmap (in an int) in front of aset's > freelist. In a lot of cases we incur plenty cache misses, just to find the > freelist bucket empty.

Re: Performance issues with parallelism and LIMIT

2023-02-20 Thread David Geier
Hi, On 2/8/23 11:42, Tomas Vondra wrote: On 2/1/23 14:41, David Geier wrote: Yeah, this is a pretty annoying regression. We already can hit poor behavior when matching rows are not distributed uniformly in the tables (which is what LIMIT costing assumes), and this makes it more likely to hit

Re: Ignoring BRIN for HOT updates (was: -udpates seems broken)

2023-02-20 Thread Matthias van de Meent
Hi, On Sun, 19 Feb 2023 at 16:04, Tomas Vondra wrote: > > Hi, > > On 2/19/23 02:03, Matthias van de Meent wrote: > > On Thu, 16 Jun 2022 at 15:05, Tomas Vondra > > wrote: > >> > >> I've pushed the revert. Let's try again for PG16. > > > > As we discussed in person at the developer meeting,

Re: Missing free_var() at end of accum_sum_final()?

2023-02-20 Thread Joel Jacobson
Hi, I found another small but significant improvement of the previous patch: else if (ndigits < var->buf_len) { -memset(var->buf, 0, var->buf_len); +var->buf[0] = 0; var->digits = var->buf + 1; var->ndigits = ndigits; } We don't need to set all buf elements to zero, only the

Re: Killing off removed rels properly

2023-02-20 Thread Tom Lane
I wrote: > Hmm, there's something else going on here. After getting rid of the > assertion failure, I see that the plan looks like > # explain MERGE INTO tt USING st ON tt.tid = st.sid WHEN NOT MATCHED THEN > INSERT > VALUES (st.sid); > QUERY PLAN

Re: SQL/JSON revisited

2023-02-20 Thread Andres Freund
Hi, On 2023-02-20 16:35:52 +0900, Amit Langote wrote: > Subject: [PATCH v4 03/10] SQL/JSON query functions > +/* > + * Evaluate a JSON error/empty behavior result. > + */ > +static Datum > +ExecEvalJsonBehavior(JsonBehavior *behavior, bool *is_null) > +{ > + *is_null = false; > + > +

Re: add PROCESS_MAIN to VACUUM

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 10:31:11AM -0600, Justin Pryzby wrote: > On Fri, Jan 13, 2023 at 03:30:15PM -0800, Nathan Bossart wrote: >> On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote: >> > For completeness, did you consider CLUSTER and REINDEX options as well? >> >> I have not, but I can

Re: Killing off removed rels properly

2023-02-20 Thread Tom Lane
Hmm, there's something else going on here. After getting rid of the assertion failure, I see that the plan looks like # explain MERGE INTO tt USING st ON tt.tid = st.sid WHEN NOT MATCHED THEN INSERT VALUES (st.sid); QUERY PLAN

Re: Add index scan progress to pg_stat_progress_vacuum

2023-02-20 Thread Imseih (AWS), Sami
Thanks! > I think PROGRESS_VACUUM_INDEXES_TOTAL and > PROGRESS_VACUUM_INDEXES_PROCESSED are better for consistency. The rest > looks good to me. Took care of that in v25. Regards -- Sami Imseih Amazon Web Services v25-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.patch

Re: Killing off removed rels properly

2023-02-20 Thread Tom Lane
Alexander Lakhin writes: > After this change the following query triggers an assert: > CREATE TABLE tt (tid integer PRIMARY KEY) PARTITION BY LIST (tid); > CREATE TABLE ttp PARTITION OF tt DEFAULT; > CREATE TABLE st (sid integer); > MERGE INTO tt USING st ON tt.tid = st.sid WHEN NOT MATCHED

Re: add PROCESS_MAIN to VACUUM

2023-02-20 Thread Justin Pryzby
On Fri, Jan 13, 2023 at 03:30:15PM -0800, Nathan Bossart wrote: > On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote: > > For completeness, did you consider CLUSTER and REINDEX options as well? > > I have not, but I can put together patches for those as well. Are you planning to do that

Re: Allow tailoring of ICU locales with custom rules

2023-02-20 Thread Daniel Verite
Peter Eisentraut wrote: [patch v5] Two quick comments: - pg_dump support need to be added for CREATE COLLATION / DATABASE - there doesn't seem to be a way to add rules to template1. If someone wants to have icu rules and initial contents to their new databases, I think they need to

Re: Add WAL read stats to pg_stat_wal

2023-02-20 Thread Andres Freund
Hi, On 2023-02-20 14:21:39 +0900, Kyotaro Horiguchi wrote: > I totally agree that it will be useful, but I'm not quite sure how > checkpointer would be able to let postmaster know about that state > without requiring access to shared memory.

Assert failure with MERGE into partitioned table with RLS

2023-02-20 Thread Dean Rasheed
As part of the MERGE RETURNING patch I noticed a suspicious Assert() in ExecInitPartitionInfo() that looked like it needed updating for MERGE. After more testing, I can confirm that this is indeed a pre-existing bug, that can be triggered using MERGE into a partitioned table that has RLS enabled

Re: pg_crc32c_armv8.c:35:9: error: implicit declaration of function '__crc32cb' is invalid in C99

2023-02-20 Thread Markur Sens
> On 20 Feb 2023, at 5:47 PM, Tom Lane wrote: > > Markur Sens mailto:markurs...@gmail.com>> writes: >> I suddenly (?, IIRC a few days ago this worked fine) started getting the >> following error while trying to following while building from source on >> Apple M1 >> This is on REL_13_9 but

Incorrect command tag row count for MERGE with a cross-partition update

2023-02-20 Thread Dean Rasheed
Playing around with MERGE some more, I noticed that the command tag row count is wrong if it does a cross-partition update: CREATE TABLE target (a int, b int) PARTITION BY LIST (b); CREATE TABLE target_p1 PARTITION OF target FOR VALUES IN (1); CREATE TABLE target_p2 PARTITION OF target FOR VALUES

Re: pg_crc32c_armv8.c:35:9: error: implicit declaration of function '__crc32cb' is invalid in C99

2023-02-20 Thread Tom Lane
Markur Sens writes: > I suddenly (?, IIRC a few days ago this worked fine) started getting the > following error while trying to following while building from source on Apple > M1 > This is on REL_13_9 but happens to all other releases too. > In file included from pg_crc32c_armv8.c:17: >

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2023-02-20 Thread Tom Lane
Alvaro Herrera writes: > On 2023-Feb-20, Amit Langote wrote: >> One more thing we could try is come up with a postgres_fdw test case, >> because it uses the RelOptInfo.userid value for remote-costs-based >> path size estimation. But adding a test case to contrib module's >> suite test a core

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-02-20 Thread Aleksander Alekseev
Hi, > OK, here is the patchset v53 where I mostly modified the commit > messages. It is explicitly said that 0001 modifies the WAL records and > why we decided to do it in this patch. Additionally any mention of > 64-bit XIDs is removed since it is not guaranteed that the rest of the > patches

Re: "out of relcache_callback_list slots" after multiple calls to pg_logical_slot_get_binary_changes

2023-02-20 Thread Tom Lane
Kyotaro Horiguchi writes: > I'm pretty sure that everytime an output plugin is initialized on a > process, it installs the same set of syscache/relcache callbacks each > time. Do you think we could simply stop duplicate registration of > those callbacks by using a static boolean? It would be

pg_crc32c_armv8.c:35:9: error: implicit declaration of function '__crc32cb' is invalid in C99

2023-02-20 Thread Markur Sens
I suddenly (?, IIRC a few days ago this worked fine) started getting the following error while trying to following while building from source on Apple M1 This is on REL_13_9 but happens to all other releases too. In file included from pg_crc32c_armv8.c:17:

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2023-02-20 Thread Alvaro Herrera
On 2023-Feb-20, Amit Langote wrote: > On Fri, Feb 17, 2023 at 9:02 PM Alvaro Herrera > wrote: > > I tried a few things for a new test case, but I was unable to find > > anything useful. Maybe an intermediate view, I thought; no dice. > > Maybe one with a security barrier would do? Anyway,

Re: Improving inferred query column names

2023-02-20 Thread David G. Johnston
On Mon, Feb 20, 2023 at 8:08 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 11.02.23 20:24, Andres Freund wrote: > > > > I think on a green field it'd be clearly better to do something like the > > above. What does give me pause is that it seems quite likely to break > >

Re: pg_init_privs corruption.

2023-02-20 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Floris Van Nee writes: > > This is as far as I can see the same case as what I reported a few years > > ago here: > > https://www.postgresql.org/message-id/flat/1574068566573.13088%40Optiver.com#488bd647ce6f5d2c92764673a7c58289 > > There was

Re: Improving inferred query column names

2023-02-20 Thread Peter Eisentraut
On 11.02.23 20:24, Andres Freund wrote: Not useful column names: SELECT SUM(reads), SUM(writes) FROM pg_stat_io; column names: sum, sum So i often end up manually writing: SELECT SUM(reads) AS sum_reads, SUM(writes) AS sum_writes, ... FROM pg_stat_io; Of course we can't infer useful column

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-02-20 Thread Alexander Lakhin
18.02.2023 23:09, Andres Freund wrote: Hi, On 2023-02-18 18:00:00 +0300, Alexander Lakhin wrote: 18.02.2023 04:06, Andres Freund wrote: Maybe it's just the backend started for the money test has got the same PID (5948) that the backend for the name test had? I somehow mashed name and money

Re: Move defaults toward ICU in 16?

2023-02-20 Thread Peter Eisentraut
On 17.02.23 21:43, Jeff Davis wrote: select row_number() over (order by nazev collate "cs-x-icu"), nazev from obce except select row_number() over (order by nazev collate "cs_CZ"), nazev from obce; returns a not empty set. So minimally for Czech collate, an index rebuild is necessary Yes,

Re: Add WAL read stats to pg_stat_wal

2023-02-20 Thread Bharath Rupireddy
On Mon, Feb 20, 2023 at 10:51 AM Kyotaro Horiguchi wrote: > > At Thu, 16 Feb 2023 11:11:38 -0800, Andres Freund wrote > in > > I wonder if we should keep the checkpointer around for longer. If we have > > checkpointer signal postmaster after it wrote the shutdown checkpoint, > > postmaster

Re: ICU locale validation / canonicalization

2023-02-20 Thread Peter Eisentraut
On 10.02.23 18:53, Jeff Davis wrote: To represent ICU locale strings in the catalog consistently, we have two choices, which as far as I can tell are equivalent: 1. ICU format Locale IDs. These are more readable, and still specified (albeit non-standard). 2. BCP47 language tags. These are

Re: SQL/JSON revisited

2023-02-20 Thread Erik Rijkers
Op 20-02-2023 om 08:35 schreef Amit Langote: Rebased again over queryjumble overhaul. Hi, But the following statement is a problem. It does not crash but it goes off, half-freezing the machine, and only comes back after fanatic Ctrl-C'ing. select json_query(jsonb '[3,4]', '$[*]'

Re: psql \watch 2nd argument: iteration count

2023-02-20 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@enterprisedb.com) wrote: > On 17.02.23 00:33, Andrey Borodin wrote: > > From time to time I want to collect some stats from locks, activity > > and other stat views into one table from different time points. In > > this case the \watch psql command

Re: Add support for unit "B" to pg_size_pretty()

2023-02-20 Thread Justin Pryzby
On Mon, Feb 20, 2023 at 07:44:15AM +0100, Peter Eisentraut wrote: > This patch adds support for the unit "B" to pg_size_pretty(). This makes it It seems like what it actually does is to support "B" in pg_size_bytes() - is that what you meant ? pg_size_pretty() already supports "bytes", so this

Re: Add WAL read stats to pg_stat_wal

2023-02-20 Thread Bharath Rupireddy
On Fri, Feb 17, 2023 at 12:41 AM Andres Freund wrote: > > On 2023-02-16 23:39:00 +0530, Bharath Rupireddy wrote: > > While working on [1], I was in need to know WAL read stats (number of > > times and amount of WAL data read from disk, time taken to read) to > > measure the benefit. I had to

Re: psql \watch 2nd argument: iteration count

2023-02-20 Thread Peter Eisentraut
On 17.02.23 00:33, Andrey Borodin wrote: From time to time I want to collect some stats from locks, activity and other stat views into one table from different time points. In this case the \watch psql command is very handy. However, it's not currently possible to specify the number of times a

Re: some namespace.c refactoring

2023-02-20 Thread Peter Eisentraut
On 15.02.23 06:04, Tom Lane wrote: I have very serious concerns first about whether it even preserves the existing semantics, and second about whether there is a performance penalty. We can work out the performance issues, but what are your concerns about semantics?

Re: some namespace.c refactoring

2023-02-20 Thread Peter Eisentraut
On 15.02.23 19:04, Alvaro Herrera wrote: That said, I think most of this code is invoked for DDL, where performance is not so critical; probably just fixing get_object_property_data to not be so naïve would suffice. Ok, I'll look into that. Queries are another matter. I can't think of a way

Re: File* argument order, argument types

2023-02-20 Thread Peter Eisentraut
On 18.02.23 01:52, Andres Freund wrote: I don't know what to actually propose. I guess the least bad I can see is to pick one type & argument order that we document to be the default, with a caveat placed above the functions not following the argument order. Order wise, I think we should choose

Re: dynamic result sets support in extended query protocol

2023-02-20 Thread Peter Eisentraut
On 31.01.23 12:07, Peter Eisentraut wrote: Applying this patch, your test queries seem to work correctly. Great! This is quite WIP, especially because there's a couple of scenarios uncovered by tests that I'd like to ensure correctness about, but if you would like to continue adding tests

Re: meson: Non-feature feature options

2023-02-20 Thread Nazir Bilal Yavuz
Hi, I added SSL and UUID patches. UUID patch has two different fixes: 1 - v1-0002-meson-Refactor-UUID-option.patch: Adding 'auto' choice to 'uuid' combo option. 2 - v1-0002-meson-Refactor-UUID-option-with-uuid_library.patch: Making 'uuid' feature option and adding new 'uuid_library' combo

I wish libpq add get_unresolv_host() and set_unresolv_host_ip() function

2023-02-20 Thread 32686647
I want to resolve the domain name asynchronously before connecting to the target; one way is parse sql connect parameters string; like if ("host=" in conn_params) and ("hostaddr=" not in conn_params): get_domain_name_and_resolv_and_add_hostaddr_to_conn_params_string() Another way is to add it

Re: File descriptors in exec'd subprocesses

2023-02-20 Thread Thomas Munro
I had missed one: the "watch" end of the postmaster pipe also needs FD_CLOEXEC. From 36f8ed2406307f8b1578ae85510c5a07718e1ea8 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Mon, 20 Feb 2023 23:26:36 +1300 Subject: [PATCH v3 1/3] Don't leak descriptors into subprograms. Open data and WAL files

Re: Allow logical replication to copy tables in binary format

2023-02-20 Thread Melih Mutlu
Amit Kapila , 16 Şub 2023 Per, 15:47 tarihinde şunu yazdı: > On Mon, Jan 30, 2023 at 4:19 PM Melih Mutlu > wrote: > >> 8. Note that the COPY binary format isn't portable across platforms > >> (Windows to Linux for instance) or major versions > >>

Re: Allow logical replication to copy tables in binary format

2023-02-20 Thread Melih Mutlu
Hi, Hayato Kuroda (Fujitsu) , 20 Şub 2023 Pzt, 10:12 tarihinde şunu yazdı: > Dear Melih, > > Thank you for updating the patch. Before reviewing, I found that > cfbot have not accepted v8 patch [1]. > Thanks for letting me know. Attached the fixed version of the patch. Best, -- Melih Mutlu

Re: File* argument order, argument types

2023-02-20 Thread Ashutosh Bapat
On Sat, Feb 18, 2023 at 6:23 AM Andres Freund wrote: > > Hi, > > While trying to add additional File* functions (FileZero, FileFallocat) I went > back and forth about the argument order between "amount" and "offset". > > We have: > > extern int FilePrefetch(File file, off_t offset, off_t amount,

  1   2   >