Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:26 AM Ted Yu wrote: > > > On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: > >> Hi, >> I was reading src/backend/replication/logical/applyparallelworker.c . >> In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I >> think the `ParallelApplyTxnHash`

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-10 Thread Nathan Bossart
On Tue, Jan 10, 2023 at 10:59:14AM +0530, Amit Kapila wrote: > I haven't looked in detail but isn't it better to explain somewhere in > the comments that it achieves to rate limit the restart of workers in > case of error and allows them to restart immediately in case of > subscription parameter

Re: Add 64-bit XIDs into PostgreSQL 15

2023-01-10 Thread Aleksander Alekseev
Hi Maxim, > Anyway. Let's discuss on-disk page format, shall we? Here are my two cents. > AFAICS, we have a following options: > [...] > 2. Put special in every page where base for XIDs are stored. This is what we > have done in the current patch set. The approach of using special space IMO

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Andres Freund
Hi, On 2023-01-11 01:25:06 +0900, 斯波隼斗 wrote: > This question is about ClockSweepTick function and the code is below. > https://github.com/postgres/postgres/blob/24d2b2680a8d0e01b30ce8a41c4eb3b47aca5031/src/backend/storage/buffer/freelist.c#L146-L165 > > The value of expected, NBuffers, wrapped

Re: Transparent column encryption

2023-01-10 Thread Mark Dilger
> On Dec 31, 2022, at 6:17 AM, Peter Eisentraut > wrote: > > Another update, with some merge conflicts resolved. Hi Peter, thanks for the patch! I wonder if logical replication could be made to work more easily with this feature. Specifically, subscribers of encrypted columns will need

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: > Hi, > I was reading src/backend/replication/logical/applyparallelworker.c . > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I > think the `ParallelApplyTxnHash` should be released. > > Please see the patch. > > Thanks > Here

releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
Hi, I was reading src/backend/replication/logical/applyparallelworker.c . In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I think the `ParallelApplyTxnHash` should be released. Please see the patch. Thanks

Re: split TOAST support out of postgres.h

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 9:46 AM Tom Lane wrote: > Now, there is a fair question whether splitting this code out of > postgres.h is worth any trouble at all. TBH my initial reaction > had been "no". But once we found that only 40-ish backend files > need to read this new header, I became a "yes"

Re: allowing for control over SET ROLE

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 2:28 AM Jeff Davis wrote: > The risks of SECURITY INVOKER are more serious. It inherently means > that one user is writing code, and another is executing it. And in the > SQL world of triggers, views, expression indexes and logical > replication, the invoker often doesn't

Re: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 8:26 AM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > Dear tom, > > > I think that it's a really bad idea to require postgres_fdw.sql > > to have two expected-files: that will be a maintenance nightmare. > > Please put whatever it is that needs a variant

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Drouvot, Bertrand
Hi, On 1/10/23 2:22 AM, Michael Paquier wrote: On Mon, Jan 09, 2023 at 08:30:00AM +0530, Bharath Rupireddy wrote: A recent commit [1] added --save-fullpage option to pg_waldump to extract full page images (FPI) from WAL records and save them into files (one file per FPI) under a specified

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Hayato Kuroda (Fujitsu)
Dear tom, > I think that it's a really bad idea to require postgres_fdw.sql > to have two expected-files: that will be a maintenance nightmare. > Please put whatever it is that needs a variant expected-file > into its own, hopefully very small and seldom-changed, test script. > Or rethink whether

can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread 斯波隼斗
This question is about ClockSweepTick function and the code is below. https://github.com/postgres/postgres/blob/24d2b2680a8d0e01b30ce8a41c4eb3b47aca5031/src/backend/storage/buffer/freelist.c#L146-L165 The value of expected, NBuffers, wrapped variable is fixed in the while loop, so that when the

Re: [PATCH] random_normal function

2023-01-10 Thread Paul Ramsey
On Tue, Jan 10, 2023 at 6:34 AM Tom Lane wrote: > > Dean Rasheed writes: > > I double-checked the one-in-a-billion claim, and it looks about right > > for each test. > > Thanks for double-checking my arithmetic. > > > The rest looks good to me, except there's a random non-ASCII character > >

Re: allowing for control over SET ROLE

2023-01-10 Thread Robert Haas
On Sat, Jan 7, 2023 at 12:00 AM Noah Misch wrote: > The docs are silent on the SET / OWNER TO connection. Hence, Reviewing the documentation again today, I realized that the documentation describes the rules for changing the ownership of an object in a whole bunch of places which this patch

Re: [PATCH] pgbench: add multiconnect option

2023-01-10 Thread Fabien COELHO
Hello Jelte, This patch seems to have quite some use case overlap with my patch which adds load balancing to libpq itself: https://www.postgresql.org/message-id/flat/pr3pr83mb04768e2ff04818eeb2179949f7...@pr3pr83mb0476.eurprd83.prod.outlook.com Thanks for the pointer. The end purpose of

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Justin Pryzby
On Tue, Jan 10, 2023 at 03:20:18PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > I propose using windows VMs instead of containers, the patch is attached. > Currently, windows containers are used on the CI, but these container images > are needs to get pulled on every CI run, also they are slow to

Re: Announcing Release 15 of the PostgreSQL Buildfarm client

2023-01-10 Thread Andrew Dunstan
On 2022-12-31 Sa 10:02, Andrew Dunstan wrote: > Changes > > > * check if a branch is up to date before trying to run it > This only applies if the |branches_to_build| setting is a keyword > rather than a list of branches. It reduces the number of useless > calls to |git pull| to

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz wrote: > > On 9/12/22 1:23 AM, vignesh C wrote: > > On Fri, 9 Sept 2022 at 11:12, Amit Kapila wrote: > > > > Thanks for pushing the patch. I have closed this entry in commitfest. > > I will wait for some more time and see the response regarding

Re: SQL/JSON revisited

2023-01-10 Thread Andrew Dunstan
On 2023-01-10 Tu 07:51, Elena Indrupskaya wrote: > Hi, > > The Postgres Pro documentation team prepared another SQL/JSON > documentation patch (attached), to apply on top of > v1-0009-Documentation-for-SQL-JSON-features.patch. > The new patch: > - Fixes minor typos > - Does some rewording agreed

Re: split TOAST support out of postgres.h

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut > wrote: >>> SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension >>> breakage en masse. I would revert this. >> Well, that was sort of my thinking, but people seemed to like this. I'm >> happy to

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-01-10 Thread Dean Rasheed
Rebased version attached. Regards, Dean diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml new file mode 100644 index b87ad5c..1482ede --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -396,8 +396,8 @@ originally matched appears later in the list of actions. On the

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Andrew Dunstan
On 2023-01-10 Tu 07:09, Jelte Fennema wrote: > Having looked closer now, I'm pretty sure you should base this patch > on top of my patch: https://commitfest.postgresql.org/41/4081/ > Mainly because you also need the token version of pg_role, which is > one of the things my patch adds. Ok, that

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 10, 2023 11:28 AM I wrote: > On Tuesday, December 27, 2022 6:29 PM Tuesday, December 27, 2022 6:29 PM > wrote: > > Thanks for reviewing our patch! PSA new version patch set. > Now, the patches fails to apply to the HEAD, because of recent commits > (c6e1f62e2c and 216a784829c)

Re: [PATCH] random_normal function

2023-01-10 Thread Tom Lane
Dean Rasheed writes: > I double-checked the one-in-a-billion claim, and it looks about right > for each test. Thanks for double-checking my arithmetic. > The rest looks good to me, except there's a random non-ASCII character > instead of a hyphen in "Kolmogorov-Smirnov" (because I copy-pasted

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 3, 2023 8:22 PM shveta malik wrote: > Please find a few minor comments. Thanks for your review ! > 1. > + diffms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), > + > >TimestampTzPlusMilliseconds(ts, MySubscription->minapplydelay)); on > unix, above code

Re: Bug in check for unreachable MERGE WHEN clauses

2023-01-10 Thread Dean Rasheed
On Mon, 2 Jan 2023 at 12:13, Dean Rasheed wrote: > > Re-reading my latest MERGE patch, I realised there is a trivial, > pre-existing bug in the check for unreachable WHEN clauses, which > means it won't spot an unreachable WHEN clause if it doesn't have an > AND condition. > > So the checks need

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 3, 2023 4:01 PM vignesh C wrote: Hi, thanks for your review ! > 1) This global variable can be removed as it is used only in send_feedback > which > is called from maybe_delay_apply so we could pass it as a function argument: > + * delay, avoid having positions of the

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Matthias van de Meent
On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > On 2023-01-09 17:50:10 +0100, Matthias van de Meent wrote: > > Wouldn't it be enough to only fix the constructions in > > FullXidRelativeTo() and widen_snapshot_xid() (as attached, $topic does > > not occur with the patch), and (optionally) bump

Re: pgbench - adding pl/pgsql versions of tests

2023-01-10 Thread Fabien COELHO
Hello, The attached patch adds pl/pgsql versions of "tpcb-like" and "simple-update" internal test scripts Why not, it makes sense because it is relevant to some usage patterns. Why not having the select version as a version as well? If we are going to follow this road, we could also

Re: split TOAST support out of postgres.h

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut wrote: > >>> Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed. > >>> I think that bears out my feeling that fmgr.h wasn't a great location: > >>> I count 117 #includes of that, many of which are in .h files themselves > >>>

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread torikoshia
On Thursday, July 22, 2021 1:05 PM, tanghy(dot)fnst(at)fujitsu(dot)com wrote I found a problem when using tab-completion as follows: CREATE SUBSCRIPTION my_subscription CONNECTION 'host=localhost port=5432 dbname=postgres' [TAB] The word 'PUBLICATION' couldn't be auto completed as expected.

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, It didn't work again. Sending numbers until I figure out how to solve this. Scheduling Step: VM + VS 2019: 00.17m Container + VS 2019: 03.51m VM + MinGW64: 00.16m Container + MinGW64: 04.28m Execution step: VM + VS 2019: 12.16m Container + VS 2019: 26.02m VM + MinGW64: 07.55m

Re: SQL/JSON revisited

2023-01-10 Thread Elena Indrupskaya
Hi, The Postgres Pro documentation team prepared another SQL/JSON documentation patch (attached), to apply on top of v1-0009-Documentation-for-SQL-JSON-features.patch. The new patch: - Fixes minor typos - Does some rewording agreed with Nikita Glukhov - Updates Docbook markup to make tags

Re: Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-10 Thread Ashutosh Bapat
On Fri, Jan 6, 2023 at 8:28 PM Tom Lane wrote: > > Ashutosh Bapat writes: > > we cast a quoted value to UNKNOWN type, but this is a special value > > null which can be casted to any SQL data type. Probably we could add a > > ANYNULLTYPE or some such generic null type which can be casted to any >

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, Tables didn't seem nice on web interface. Re-sending with correct formatting. Scheduling step: | VS 2019 | MinGW64 -- VM | 00:17m | 00:16m -- Container | 03:51m |

Re: [PATCH]Feature improvement for MERGE tab completion

2023-01-10 Thread Dean Rasheed
On Tue, 3 Jan 2023 at 12:30, vignesh C wrote: > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > This is because 0001 has been committed. Re-uploading 0002, to keep the CF-bot happy. Reviewing 0002... I'm not entirely convinced that the PartialMatches()

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-10 Thread Jakub Wartak
On Wed, Jan 4, 2023 at 6:38 PM Robert Haas wrote: > > On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote: > > As you well know, psql's FETCH_COUNT mechanism is far older than > > single-row mode. I don't think anyone's tried to transpose it > > onto that. I agree that it seems like a good idea to

Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, I propose using windows VMs instead of containers, the patch is attached. Currently, windows containers are used on the CI, but these container images are needs to get pulled on every CI run, also they are slow to run. These VM images are created in the same way how container images are

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Jelte Fennema
Having looked closer now, I'm pretty sure you should base this patch on top of my patch: https://commitfest.postgresql.org/41/4081/ Mainly because you also need the token version of pg_role, which is one of the things my patch adds. > if (regexp_pgrole[0] == '+') For these lines you'll need to

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-10 Thread Masahiko Sawada
On Mon, Jan 9, 2023 at 5:59 PM John Naylor wrote: > > > [working on templating] > > In the end, I decided to base my effort on v8, and not v12 (based on one of > my less-well-thought-out ideas). The latter was a good experiment, but it did > not lead to an increase in readability as I had

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Dilip Kumar
On Tue, Jan 10, 2023 at 10:26 AM houzj.f...@fujitsu.com wrote: > > On Monday, January 9, 2023 4:51 PM Amit Kapila > wrote: > > > > On Sun, Jan 8, 2023 at 11:32 AM houzj.f...@fujitsu.com > > wrote: > > > > > > On Sunday, January 8, 2023 11:59 AM houzj.f...@fujitsu.com > > wrote: > > > > Attach

Re: Support for dumping extended statistics

2023-01-10 Thread Hari krishna Maddileti
Thanks Team for showing interest. Please find the attached patch, which uses the same approach as mentioned in previous email to implement input functions to parse pg_distinct, pg_dependency and pg_mcv_list strings. Regards, Hari From: Bruce Momjian Date: Saturday, 7 January 2023 at 8:10 AM

Some revises in adding sorting path

2023-01-10 Thread Richard Guo
While reviewing [1], I visited other places where sorting is needed, and have some findings. In add_paths_with_pathkeys_for_rel, we do not try incremental sort atop of the epq_path, which I think we can do. I'm not sure how useful this is in real world since the epq_path is used only for EPQ

Re: [PATCH] random_normal function

2023-01-10 Thread Dean Rasheed
On Tue, 10 Jan 2023 at 08:33, Dean Rasheed wrote: > > The rest looks good to me, except there's a random non-ASCII character > instead of a hyphen in "Kolmogorov-Smirnov" (because I copy-pasted the > name from some random website). > Oh, never mind. I see you already fixed that. I should finish

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Dean Rasheed
While doing some random testing, I noticed that the following is broken in HEAD: SELECT COUNT(DISTINCT random()) FROM generate_series(1,10); ERROR: ORDER/GROUP BY expression not found in targetlist It appears to have been broken by 1349d279, though I haven't looked at the details. I'm

Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-01-10 Thread Drouvot, Bertrand
Hi hackers, While working on [1], I noticed that xl_hash_vacuum_one_page.ntuples is an int. Unless I'm missing something, It seems to me that it would make more sense to use an uint16 (like this is done for gistxlogDelete.ntodelete for example). Please find attached a patch proposal to do so.

Re: Collation version tracking for macOS

2023-01-10 Thread Peter Eisentraut
On 05.12.22 22:33, Thomas Munro wrote: On Tue, Dec 6, 2022 at 6:45 AM Joe Conway wrote: On 12/5/22 12:41, Jeff Davis wrote: On Mon, 2022-12-05 at 16:12 +1300, Thomas Munro wrote: 1. I think we should seriously consider provider = ICU63. I still think search-by-collversion is a little too

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-10 Thread Marco Slot
On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > We'd like to be able to call the lock manager's WaitForLockers() and > WaitForLockersMultiple() from SQL. Below I describe our use case, but > basically I'm wondering if this: > > 1. Seems like a reasonable thing to do > > 2. Would

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread Maxim Orlov
On Mon, 9 Jan 2023 at 21:36, Corey Huinker wrote: > > I chose a name that would avoid collisions with anything a user might > potentially throw into their environment, so if the var "OS" is fairly > standard is a reason to avoid using it. Also, going with our own env var > allows us to stay in

Re: split TOAST support out of postgres.h

2023-01-10 Thread Peter Eisentraut
On 10.01.23 08:39, Noah Misch wrote: On Tue, Jan 10, 2023 at 06:07:49AM +0100, Peter Eisentraut wrote: On 30.12.22 17:50, Tom Lane wrote: Peter Eisentraut writes: On 28.12.22 16:07, Tom Lane wrote: I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included so widely, I doubt it

Re: [PATCH] random_normal function

2023-01-10 Thread Dean Rasheed
On Mon, 9 Jan 2023 at 23:38, Tom Lane wrote: > > I tried this on some 32-bit big-endian hardware (NetBSD on macppc) > to verify my thesis that the results of random() are now machine > independent. That part works, but the random_normal() tests didn't; Ah yes, I wondered about that. > I saw

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 10/01/23 10:53, David Rowley wrote: On Tue, 10 Jan 2023 at 06:15, Ankit Kumar Pandey wrote: > Do we have any pending items for this patch now? I'm just wondering if not trying this when the query has a DISTINCT clause is a copout. What I wanted to avoid was doing additional sorting

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-10 Thread Drouvot, Bertrand
Hi, On 1/6/23 6:41 PM, Bharath Rupireddy wrote: On Fri, Jan 6, 2023 at 11:47 AM Bharath Rupireddy wrote: On Thu, Jan 5, 2023 at 6:51 PM Bharath Rupireddy wrote: I'm also wondering if it would make sense to extend the test coverage of it (and pg_waldump) to "validate" that both extracted

Re: typos

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 1:18 PM Michael Paquier wrote: > > On Tue, Jan 10, 2023 at 12:24:40PM +0530, Amit Kapila wrote: > > Thanks for noticing this. I'll take care of this and some other typo > > patches together. > > Does this include 0010? I was just looking at the whole set and this > one

<    1   2