Re: typos

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 1:18 PM Michael Paquier wrote: > > On Tue, Jan 10, 2023 at 12:24:40PM +0530, Amit Kapila wrote: > > Thanks for noticing this. I'll take care of this and some other typo > > patches together. > > Does this include 0010? I was just looking at the whole set and this > one loo

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-10 Thread Drouvot, Bertrand
Hi, On 1/6/23 6:41 PM, Bharath Rupireddy wrote: On Fri, Jan 6, 2023 at 11:47 AM Bharath Rupireddy wrote: On Thu, Jan 5, 2023 at 6:51 PM Bharath Rupireddy wrote: I'm also wondering if it would make sense to extend the test coverage of it (and pg_waldump) to "validate" that both extracted

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 10/01/23 10:53, David Rowley wrote: On Tue, 10 Jan 2023 at 06:15, Ankit Kumar Pandey wrote: > Do we have any pending items for this patch now? I'm just wondering if not trying this when the query has a DISTINCT clause is a copout. What I wanted to avoid was doing additional sorting wor

Re: [PATCH] random_normal function

2023-01-10 Thread Dean Rasheed
On Mon, 9 Jan 2023 at 23:38, Tom Lane wrote: > > I tried this on some 32-bit big-endian hardware (NetBSD on macppc) > to verify my thesis that the results of random() are now machine > independent. That part works, but the random_normal() tests didn't; Ah yes, I wondered about that. > I saw low

Re: split TOAST support out of postgres.h

2023-01-10 Thread Peter Eisentraut
On 10.01.23 08:39, Noah Misch wrote: On Tue, Jan 10, 2023 at 06:07:49AM +0100, Peter Eisentraut wrote: On 30.12.22 17:50, Tom Lane wrote: Peter Eisentraut writes: On 28.12.22 16:07, Tom Lane wrote: I dunno, #3 seems kind of unprincipled. Also, since fmgr.h is included so widely, I doubt it

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread Maxim Orlov
On Mon, 9 Jan 2023 at 21:36, Corey Huinker wrote: > > I chose a name that would avoid collisions with anything a user might > potentially throw into their environment, so if the var "OS" is fairly > standard is a reason to avoid using it. Also, going with our own env var > allows us to stay in pe

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-10 Thread Marco Slot
On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > We'd like to be able to call the lock manager's WaitForLockers() and > WaitForLockersMultiple() from SQL. Below I describe our use case, but > basically I'm wondering if this: > > 1. Seems like a reasonable thing to do > > 2. Would b

Re: Collation version tracking for macOS

2023-01-10 Thread Peter Eisentraut
On 05.12.22 22:33, Thomas Munro wrote: On Tue, Dec 6, 2022 at 6:45 AM Joe Conway wrote: On 12/5/22 12:41, Jeff Davis wrote: On Mon, 2022-12-05 at 16:12 +1300, Thomas Munro wrote: 1. I think we should seriously consider provider = ICU63. I still think search-by-collversion is a little too ma

Change xl_hash_vacuum_one_page.ntuples from int to uint16

2023-01-10 Thread Drouvot, Bertrand
Hi hackers, While working on [1], I noticed that xl_hash_vacuum_one_page.ntuples is an int. Unless I'm missing something, It seems to me that it would make more sense to use an uint16 (like this is done for gistxlogDelete.ntodelete for example). Please find attached a patch proposal to do so.

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Dean Rasheed
While doing some random testing, I noticed that the following is broken in HEAD: SELECT COUNT(DISTINCT random()) FROM generate_series(1,10); ERROR: ORDER/GROUP BY expression not found in targetlist It appears to have been broken by 1349d279, though I haven't looked at the details. I'm somewhat

Re: [PATCH] random_normal function

2023-01-10 Thread Dean Rasheed
On Tue, 10 Jan 2023 at 08:33, Dean Rasheed wrote: > > The rest looks good to me, except there's a random non-ASCII character > instead of a hyphen in "Kolmogorov-Smirnov" (because I copy-pasted the > name from some random website). > Oh, never mind. I see you already fixed that. I should finish r

Some revises in adding sorting path

2023-01-10 Thread Richard Guo
While reviewing [1], I visited other places where sorting is needed, and have some findings. In add_paths_with_pathkeys_for_rel, we do not try incremental sort atop of the epq_path, which I think we can do. I'm not sure how useful this is in real world since the epq_path is used only for EPQ chec

Re: Support for dumping extended statistics

2023-01-10 Thread Hari krishna Maddileti
Thanks Team for showing interest. Please find the attached patch, which uses the same approach as mentioned in previous email to implement input functions to parse pg_distinct, pg_dependency and pg_mcv_list strings. Regards, Hari From: Bruce Momjian Date: Saturday, 7 January 2023 at 8:10 AM T

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-10 Thread Dilip Kumar
On Tue, Jan 10, 2023 at 10:26 AM houzj.f...@fujitsu.com wrote: > > On Monday, January 9, 2023 4:51 PM Amit Kapila > wrote: > > > > On Sun, Jan 8, 2023 at 11:32 AM houzj.f...@fujitsu.com > > wrote: > > > > > > On Sunday, January 8, 2023 11:59 AM houzj.f...@fujitsu.com > > wrote: > > > > Attach

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-10 Thread Masahiko Sawada
On Mon, Jan 9, 2023 at 5:59 PM John Naylor wrote: > > > [working on templating] > > In the end, I decided to base my effort on v8, and not v12 (based on one of > my less-well-thought-out ideas). The latter was a good experiment, but it did > not lead to an increase in readability as I had hoped.

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Jelte Fennema
Having looked closer now, I'm pretty sure you should base this patch on top of my patch: https://commitfest.postgresql.org/41/4081/ Mainly because you also need the token version of pg_role, which is one of the things my patch adds. > if (regexp_pgrole[0] == '+') For these lines you'll need to ch

Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, I propose using windows VMs instead of containers, the patch is attached. Currently, windows containers are used on the CI, but these container images are needs to get pulled on every CI run, also they are slow to run. These VM images are created in the same way how container images are

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-10 Thread Jakub Wartak
On Wed, Jan 4, 2023 at 6:38 PM Robert Haas wrote: > > On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote: > > As you well know, psql's FETCH_COUNT mechanism is far older than > > single-row mode. I don't think anyone's tried to transpose it > > onto that. I agree that it seems like a good idea to t

Re: [PATCH]Feature improvement for MERGE tab completion

2023-01-10 Thread Dean Rasheed
On Tue, 3 Jan 2023 at 12:30, vignesh C wrote: > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > This is because 0001 has been committed. Re-uploading 0002, to keep the CF-bot happy. Reviewing 0002... I'm not entirely convinced that the PartialMatches() cha

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, Tables didn't seem nice on web interface. Re-sending with correct formatting. Scheduling step: | VS 2019 | MinGW64 -- VM | 00:17m | 00:16m -- Container | 03:51m | 04:28m

Re: Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-10 Thread Ashutosh Bapat
On Fri, Jan 6, 2023 at 8:28 PM Tom Lane wrote: > > Ashutosh Bapat writes: > > we cast a quoted value to UNKNOWN type, but this is a special value > > null which can be casted to any SQL data type. Probably we could add a > > ANYNULLTYPE or some such generic null type which can be casted to any >

Re: SQL/JSON revisited

2023-01-10 Thread Elena Indrupskaya
Hi, The Postgres Pro documentation team prepared another SQL/JSON documentation patch (attached), to apply on top of v1-0009-Documentation-for-SQL-JSON-features.patch. The new patch: - Fixes minor typos - Does some rewording agreed with Nikita Glukhov - Updates Docbook markup to make tags cons

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Nazir Bilal Yavuz
Hi, It didn't work again. Sending numbers until I figure out how to solve this. Scheduling Step: VM + VS 2019: 00.17m Container + VS 2019: 03.51m VM + MinGW64: 00.16m Container + MinGW64: 04.28m Execution step: VM + VS 2019: 12.16m Container + VS 2019: 26.02m VM + MinGW64: 07.55m Container

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread torikoshia
On Thursday, July 22, 2021 1:05 PM, tanghy(dot)fnst(at)fujitsu(dot)com wrote I found a problem when using tab-completion as follows: CREATE SUBSCRIPTION my_subscription CONNECTION 'host=localhost port=5432 dbname=postgres' [TAB] The word 'PUBLICATION' couldn't be auto completed as expected.

Re: split TOAST support out of postgres.h

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut wrote: > >>> Wow, 41 files requiring varatt.h is a lot fewer than I would have guessed. > >>> I think that bears out my feeling that fmgr.h wasn't a great location: > >>> I count 117 #includes of that, many of which are in .h files themselves > >>>

Re: pgbench - adding pl/pgsql versions of tests

2023-01-10 Thread Fabien COELHO
Hello, The attached patch adds pl/pgsql versions of "tpcb-like" and "simple-update" internal test scripts Why not, it makes sense because it is relevant to some usage patterns. Why not having the select version as a version as well? If we are going to follow this road, we could also consid

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Matthias van de Meent
On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > On 2023-01-09 17:50:10 +0100, Matthias van de Meent wrote: > > Wouldn't it be enough to only fix the constructions in > > FullXidRelativeTo() and widen_snapshot_xid() (as attached, $topic does > > not occur with the patch), and (optionally) bump

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 3, 2023 4:01 PM vignesh C wrote: Hi, thanks for your review ! > 1) This global variable can be removed as it is used only in send_feedback > which > is called from maybe_delay_apply so we could pass it as a function argument: > + * delay, avoid having positions of the flushe

Re: Bug in check for unreachable MERGE WHEN clauses

2023-01-10 Thread Dean Rasheed
On Mon, 2 Jan 2023 at 12:13, Dean Rasheed wrote: > > Re-reading my latest MERGE patch, I realised there is a trivial, > pre-existing bug in the check for unreachable WHEN clauses, which > means it won't spot an unreachable WHEN clause if it doesn't have an > AND condition. > > So the checks need t

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 3, 2023 8:22 PM shveta malik wrote: > Please find a few minor comments. Thanks for your review ! > 1. > + diffms = TimestampDifferenceMilliseconds(GetCurrentTimestamp(), > + > >TimestampTzPlusMilliseconds(ts, MySubscription->minapplydelay)); on > unix, above code lo

Re: [PATCH] random_normal function

2023-01-10 Thread Tom Lane
Dean Rasheed writes: > I double-checked the one-in-a-billion claim, and it looks about right > for each test. Thanks for double-checking my arithmetic. > The rest looks good to me, except there's a random non-ASCII character > instead of a hyphen in "Kolmogorov-Smirnov" (because I copy-pasted th

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-10 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 10, 2023 11:28 AM I wrote: > On Tuesday, December 27, 2022 6:29 PM Tuesday, December 27, 2022 6:29 PM > wrote: > > Thanks for reviewing our patch! PSA new version patch set. > Now, the patches fails to apply to the HEAD, because of recent commits > (c6e1f62e2c and 216a784829c) a

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Andrew Dunstan
On 2023-01-10 Tu 07:09, Jelte Fennema wrote: > Having looked closer now, I'm pretty sure you should base this patch > on top of my patch: https://commitfest.postgresql.org/41/4081/ > Mainly because you also need the token version of pg_role, which is > one of the things my patch adds. Ok, that

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-01-10 Thread Dean Rasheed
Rebased version attached. Regards, Dean diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml new file mode 100644 index b87ad5c..1482ede --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -396,8 +396,8 @@ originally matched appears later in the list of actions. On the o

Re: split TOAST support out of postgres.h

2023-01-10 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 3:48 AM Peter Eisentraut > wrote: >>> SET_VARSIZE alone appears in 74 pgxn distributions, so I predict extension >>> breakage en masse. I would revert this. >> Well, that was sort of my thinking, but people seemed to like this. I'm >> happy to cons

Re: SQL/JSON revisited

2023-01-10 Thread Andrew Dunstan
On 2023-01-10 Tu 07:51, Elena Indrupskaya wrote: > Hi, > > The Postgres Pro documentation team prepared another SQL/JSON > documentation patch (attached), to apply on top of > v1-0009-Documentation-for-SQL-JSON-features.patch. > The new patch: > - Fixes minor typos > - Does some rewording agreed

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Amit Kapila
On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz wrote: > > On 9/12/22 1:23 AM, vignesh C wrote: > > On Fri, 9 Sept 2022 at 11:12, Amit Kapila wrote: > > > > Thanks for pushing the patch. I have closed this entry in commitfest. > > I will wait for some more time and see the response regarding the

Re: Announcing Release 15 of the PostgreSQL Buildfarm client

2023-01-10 Thread Andrew Dunstan
On 2022-12-31 Sa 10:02, Andrew Dunstan wrote: > Changes > > > * check if a branch is up to date before trying to run it > This only applies if the |branches_to_build| setting is a keyword > rather than a list of branches. It reduces the number of useless > calls to |git pull| to alm

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Justin Pryzby
On Tue, Jan 10, 2023 at 03:20:18PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > I propose using windows VMs instead of containers, the patch is attached. > Currently, windows containers are used on the CI, but these container images > are needs to get pulled on every CI run, also they are slow to run

Re: [PATCH] pgbench: add multiconnect option

2023-01-10 Thread Fabien COELHO
Hello Jelte, This patch seems to have quite some use case overlap with my patch which adds load balancing to libpq itself: https://www.postgresql.org/message-id/flat/pr3pr83mb04768e2ff04818eeb2179949f7...@pr3pr83mb0476.eurprd83.prod.outlook.com Thanks for the pointer. The end purpose of the

Re: allowing for control over SET ROLE

2023-01-10 Thread Robert Haas
On Sat, Jan 7, 2023 at 12:00 AM Noah Misch wrote: > The docs are silent on the SET / OWNER TO connection. Hence, Reviewing the documentation again today, I realized that the documentation describes the rules for changing the ownership of an object in a whole bunch of places which this patch fail

Re: [PATCH] random_normal function

2023-01-10 Thread Paul Ramsey
On Tue, Jan 10, 2023 at 6:34 AM Tom Lane wrote: > > Dean Rasheed writes: > > I double-checked the one-in-a-billion claim, and it looks about right > > for each test. > > Thanks for double-checking my arithmetic. > > > The rest looks good to me, except there's a random non-ASCII character > > inst

can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread 斯波隼斗
This question is about ClockSweepTick function and the code is below. https://github.com/postgres/postgres/blob/24d2b2680a8d0e01b30ce8a41c4eb3b47aca5031/src/backend/storage/buffer/freelist.c#L146-L165 The value of expected, NBuffers, wrapped variable is fixed in the while loop, so that when the v

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Hayato Kuroda (Fujitsu)
Dear tom, > I think that it's a really bad idea to require postgres_fdw.sql > to have two expected-files: that will be a maintenance nightmare. > Please put whatever it is that needs a variant expected-file > into its own, hopefully very small and seldom-changed, test script. > Or rethink whether

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Drouvot, Bertrand
Hi, On 1/10/23 2:22 AM, Michael Paquier wrote: On Mon, Jan 09, 2023 at 08:30:00AM +0530, Bharath Rupireddy wrote: A recent commit [1] added --save-fullpage option to pg_waldump to extract full page images (FPI) from WAL records and save them into files (one file per FPI) under a specified direc

Re: [Proposal] Add foreign-server health checks infrastructure

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 8:26 AM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > Dear tom, > > > I think that it's a really bad idea to require postgres_fdw.sql > > to have two expected-files: that will be a maintenance nightmare. > > Please put whatever it is that needs a variant exp

Re: allowing for control over SET ROLE

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 2:28 AM Jeff Davis wrote: > The risks of SECURITY INVOKER are more serious. It inherently means > that one user is writing code, and another is executing it. And in the > SQL world of triggers, views, expression indexes and logical > replication, the invoker often doesn't k

Re: split TOAST support out of postgres.h

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 9:46 AM Tom Lane wrote: > Now, there is a fair question whether splitting this code out of > postgres.h is worth any trouble at all. TBH my initial reaction > had been "no". But once we found that only 40-ish backend files > need to read this new header, I became a "yes"

releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
Hi, I was reading src/backend/replication/logical/applyparallelworker.c . In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I think the `ParallelApplyTxnHash` should be released. Please see the patch. Thanks

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: > Hi, > I was reading src/backend/replication/logical/applyparallelworker.c . > In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I > think the `ParallelApplyTxnHash` should be released. > > Please see the patch. > > Thanks > Here

Re: Transparent column encryption

2023-01-10 Thread Mark Dilger
> On Dec 31, 2022, at 6:17 AM, Peter Eisentraut > wrote: > > Another update, with some merge conflicts resolved. Hi Peter, thanks for the patch! I wonder if logical replication could be made to work more easily with this feature. Specifically, subscribers of encrypted columns will need t

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Andres Freund
Hi, On 2023-01-11 01:25:06 +0900, 斯波隼斗 wrote: > This question is about ClockSweepTick function and the code is below. > https://github.com/postgres/postgres/blob/24d2b2680a8d0e01b30ce8a41c4eb3b47aca5031/src/backend/storage/buffer/freelist.c#L146-L165 > > The value of expected, NBuffers, wrapped

Re: Add 64-bit XIDs into PostgreSQL 15

2023-01-10 Thread Aleksander Alekseev
Hi Maxim, > Anyway. Let's discuss on-disk page format, shall we? Here are my two cents. > AFAICS, we have a following options: > [...] > 2. Put special in every page where base for XIDs are stored. This is what we > have done in the current patch set. The approach of using special space IMO is

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-10 Thread Nathan Bossart
On Tue, Jan 10, 2023 at 10:59:14AM +0530, Amit Kapila wrote: > I haven't looked in detail but isn't it better to explain somewhere in > the comments that it achieves to rate limit the restart of workers in > case of error and allows them to restart immediately in case of > subscription parameter ch

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:26 AM Ted Yu wrote: > > > On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: > >> Hi, >> I was reading src/backend/replication/logical/applyparallelworker.c . >> In `pa_allocate_worker`, when pa_launch_parallel_worker returns NULL, I >> think the `ParallelApplyTxnHash` shoul

Re: fixing CREATEROLE

2023-01-10 Thread Robert Haas
On Thu, Jan 5, 2023 at 2:53 PM Robert Haas wrote: > On Tue, Jan 3, 2023 at 3:11 PM Robert Haas wrote: > > Committed and back-patched 0001 with fixes for the issues that you pointed > > out. > > > > Here's a trivial rebase of the rest of the patch set. > > I committed 0001 and 0002 after improvin

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Nathan Bossart
On Tue, Jan 10, 2023 at 02:57:43AM -0500, Tom Lane wrote: > Michael Paquier writes: >> Any objections about getting 947789f applied to REL_13_STABLE and >> REL_12_STABLE and see this issue completely gone from all the versions >> supported? > > No objections to back-patching the fix... +1 -- N

Re: Handle infinite recursion in logical replication setup

2023-01-10 Thread Jonathan S. Katz
On 1/10/23 10:17 AM, Amit Kapila wrote: On Tue, Jan 10, 2023 at 8:13 AM Jonathan S. Katz wrote: This consistently created the deadlock in my testing. Discussing with Masahiko off-list, this is due to a deadlock from 4 processes: the walsenders on A and B, and the apply workers on A and B. Th

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 12:40 PM Andres Freund wrote: > > I think. `expected = originalVictim + 1;` line should be in while loop > > (before acquiring spin lock) so that, even in the case above, expected > > variable is incremented for each loop and CAS operation will be successful > > at some poi

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-10 Thread Ted Yu
On Tue, Jan 10, 2023 at 9:43 AM Ted Yu wrote: > > > On Tue, Jan 10, 2023 at 9:26 AM Ted Yu wrote: > >> >> >> On Tue, Jan 10, 2023 at 9:25 AM Ted Yu wrote: >> >>> Hi, >>> I was reading src/backend/replication/logical/applyparallelworker.c . >>> In `pa_allocate_worker`, when pa_launch_parallel_wo

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Robert Haas
On Mon, Jan 9, 2023 at 5:59 PM Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 12:51 PM Robert Haas wrote: > > I feel that you should at least have a reproducer for these problems > > posted to the thread, and ideally a regression test, before committing > > things. I think it's very hard to unde

Re: can while loop in ClockSweepTick function be kind of infinite loop in some cases?

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 13:11:35 -0500, Robert Haas wrote: > On Tue, Jan 10, 2023 at 12:40 PM Andres Freund wrote: > > > I think. `expected = originalVictim + 1;` line should be in while loop > > > (before acquiring spin lock) so that, even in the case above, expected > > > variable is incremented for

Re: Avoiding "wrong tuple length" errors at the end of VACUUM on pg_database update (Backpatch of 947789f to v12 and v13)

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 02:57:43 -0500, Tom Lane wrote: > No objections to back-patching the fix, but I wonder if we can find > some mechanical way to prevent this sort of error in future. What about a define that forces external toasting very aggressively for catalog tables, iff they have a toast tabl

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 15:03:42 +0100, Matthias van de Meent wrote: > On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > > On 2023-01-09 17:50:10 +0100, Matthias van de Meent wrote: > > > Wouldn't it be enough to only fix the constructions in > > > FullXidRelativeTo() and widen_snapshot_xid() (as att

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread Ankit Kumar Pandey
On 10/01/23 10:53, David Rowley wrote: the total cost is the same for both of these, but the execution time seems to vary quite a bit. This is really weird, I tried it different ways (to rule out any issues due to caching) and execution time varied in spite of having same cost. Maybe

Re: psql: Add role's membership options to the \du+ command

2023-01-10 Thread Pavel Luzanov
Added the patch to the open commitfest: https://commitfest.postgresql.org/42/4116/ Feel free to reject if it's not interesting. -- Pavel Luzanov

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Andres Freund
Hi, On 2023-01-10 09:22:12 -0600, Justin Pryzby wrote: > > There is more than 2x speed gain when VMs are used. > > One consideration is that if windows runs twice as fast, we'll suddenly > start using twice as many resources at cirrus/google/amazon - the > windows task has been throttling everyth

Re: Show various offset arrays for heap WAL records

2023-01-10 Thread Andres Freund
Hi, On 2023-01-09 19:59:42 -0800, Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 1:58 PM Andres Freund wrote: > > A couple times when investigating data corruption issues, the last time just > > yesterday in [1], I needed to see the offsets affected by PRUNE and VACUUM > > records. As that's pro

Re: RFC: logical publication via inheritance root?

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 12:41 AM Aleksander Alekseev wrote: > I would like to point out that we shouldn't necessarily support > multiple inheritance in all the possible cases, at least not in the > first implementation. Supporting simple cases of inheritance would be > already a valuable feature ev

Re: Transparent column encryption

2023-01-10 Thread Mark Dilger
> On Jan 10, 2023, at 9:26 AM, Mark Dilger wrote: > >-- Cryptographically connected to the encrypted record >patient_id BIGINT NOT NULL, >patient_ssn CHAR(11), > >-- The encrypted record >patient_record TEXT ENCRYPTED WITH (column_encryption_key = cek1, >

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 10:50 AM Robert Haas wrote: > Look, I don't want to spend time arguing about what seem to me to be > basic principles of good software engineering. When I don't put test > cases into my patches, people complain at me and tell me that I'm a > bad software engineer because I

Re: logical decoding and replication of sequences, take 2

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 1:32 PM Tomas Vondra wrote: > 0001 is a fix for the pre-existing issue in logicalmsg_decode, > attempting to build a snapshot before getting into a consistent state. > AFAICS this only affects assert-enabled builds and is otherwise > harmless, because we are not actually us

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 11:47 AM Peter Geoghegan wrote: > In summary, I think that there is currently no way that we can have > the VM (or the PD_ALL_VISIBLE flag) concurrently unset, while leaving > the page all_frozen. It can happen and leave the page all_visible, but > not all_frozen, due to th

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Robert Haas
On Tue, Jan 10, 2023 at 2:48 PM Peter Geoghegan wrote: > What I actually said was that there is no reason to declare up front > that the only circumstances under which a fix could be committed is > when a clean repro is available. I never said that a test case has > little or no value, and I certa

Re: heapgettup refactoring

2023-01-10 Thread Melanie Plageman
On Thu, Jan 5, 2023 at 8:52 AM Peter Eisentraut wrote: > > Ok, let's look through these patches starting from the top then. > > v4-0001-Add-no-movement-scan-helper.patch > > This makes sense overall; there is clearly some duplicate code that can > be unified. > > It appears that during your rebasi

Re: Add SHELL_EXIT_CODE to psql

2023-01-10 Thread Corey Huinker
On Tue, Jan 10, 2023 at 3:54 AM Maxim Orlov wrote: > > > On Mon, 9 Jan 2023 at 21:36, Corey Huinker > wrote: > >> >> I chose a name that would avoid collisions with anything a user might >> potentially throw into their environment, so if the var "OS" is fairly >> standard is a reason to avoid us

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-10 Thread Matthias van de Meent
On Tue, 10 Jan 2023 at 20:14, Andres Freund wrote: > > Hi, > > On 2023-01-10 15:03:42 +0100, Matthias van de Meent wrote: > > On Mon, 9 Jan 2023 at 20:34, Andres Freund wrote: > > > It's not too hard to fix in individual places, but I suspect that we'll > > > introduce the bug in future places wi

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 12:19 PM Robert Haas wrote: > I don't understand what distinction you're making. It seems like > hair-splitting to me. We should be able to reproduce problems like > this reliably, at least with the aid of a debugger and some > breakpoints, before we go changing the code.

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Sandro Santilli
On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: > Have you considered the idea of instead inventing a "\include" facility [...] > cases, you still need one script file for each supported upgrade step That's exactly the problem we're trying to solve here. The include support is nice on

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 7:40 AM Andrew Dunstan wrote: > I'm confused. A client cert might not have a hostname at all, and isn't > used to verify the connecting address, but to verify the username. It > needs to have a CN/DN equal to the user name of the connection, or that > maps to that name via p

Re: allowing for control over SET ROLE

2023-01-10 Thread Jeff Davis
On Tue, 2023-01-10 at 11:45 -0500, Robert Haas wrote: > So the risks, which in theory are all very similar, are in practice > far greater in the PostgreSQL context, basically because our default > setup is about 40 years behind the times in terms of implementing > best > practices. I agree that hu

Re: Allow DISTINCT to use Incremental Sort

2023-01-10 Thread David Rowley
On Tue, 10 Jan 2023 at 16:07, Richard Guo wrote: > Sorry I didn't make myself clear. I mean currently on HEAD in planner.c > from line 4847 to line 4857, we have the code to make sure we always use > the more rigorous clause for explicit-sort case. I think this code is > not necessary, because w

Wasted Vacuum cycles when OldestXmin is not moving

2023-01-10 Thread sirisha chamarthi
Hi Hackers, vacuum is not able to clean up dead tuples when OldestXmin is not moving (because of a long running transaction or when hot_standby_feedback is behind). Even though OldestXmin is not moved from the last time it checked, it keeps retrying every autovacuum_naptime and wastes CPU cycles a

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-10 Thread Tom Lane
I wrote: > Amit Langote writes: >> Thanks for the patch. It looks good, though I guess you said that we >> should also change the error message that CREATE TABLE ... PARTITION >> OF emits to match the other cases while we're here. I've attached a >> delta patch. > Thanks. I hadn't touched that

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-10 Thread Jacob Champion
On Mon, Jan 9, 2023 at 7:07 AM Jelte Fennema wrote: > I also took a closer look at the code, and the only comment I have is: > > > appendPQExpBuffer(&conn->errorMessage, > > These calls can all be replaced by the recently added libpq_append_conn_error Argh, thanks for the catch. Fixed. > Finally

wal_compression = method:level

2023-01-10 Thread Justin Pryzby
Is it desirable to support specifying a level ? Maybe there's a concern about using high compression levels, but I'll start by asking if the feature is wanted at all. Previous discussion at: 20210614012412.ga31...@telsasoft.com >From cb30e17cf19fffa370a887d28d6d7e683d588b71 Mon Sep 17 00:00:00 2

Re: verbose mode for pg_input_error_message?

2023-01-10 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 04:18:59PM -0500, Andrew Dunstan wrote: > On 2023-01-02 Mo 10:44, Tom Lane wrote: >> I don't think that just concatenating those strings would make for a >> pleasant API. More sensible, perhaps, to have a separate function >> that returns a record. Or we could redefine the

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-10 Thread Tom Lane
Sandro Santilli writes: > On Mon, Jan 09, 2023 at 05:51:49PM -0500, Tom Lane wrote: >> ... you still need one script file for each supported upgrade step > That's exactly the problem we're trying to solve here. > The include support is nice on itself, but won't solve our problem. The script-file

Can we let extensions change their dumped catalog schemas?

2023-01-10 Thread Jacob Champion
Hi all, I've been talking to other Timescale devs about a requested change to pg_dump, and there's been quite a bit of back-and-forth to figure out what, exactly, we want. Any mistakes here are mine, but I think we've been able to distill it down to the following request: We'd like to be allowed

Re: Use windows VMs instead of windows containers on the CI

2023-01-10 Thread Thomas Munro
On Wed, Jan 11, 2023 at 8:20 AM Andres Freund wrote: > On 2023-01-10 09:22:12 -0600, Justin Pryzby wrote: > > > There is more than 2x speed gain when VMs are used. > > > > One consideration is that if windows runs twice as fast, we'll suddenly > > start using twice as many resources at cirrus/goog

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 12:08 PM Peter Geoghegan wrote: > Actually, FreezeMultiXactId() can fully remove an xmax that has some > member XIDs >= OldestXmin, provided FRM_NOOP processing isn't > possible, at least when no individual member is still running. Doesn't > have to involve transaction abor

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread David Rowley
On Wed, 11 Jan 2023 at 08:17, Ankit Kumar Pandey wrote: > > > > On 10/01/23 10:53, David Rowley wrote: > > > the total cost is the same for both of these, but the execution time > > seems to vary quite a bit. > > This is really weird, I tried it different ways (to rule out any issues > due to > >

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-10 Thread David Rowley
On Tue, 10 Jan 2023 at 18:36, Tom Lane wrote: > > David Rowley writes: > > Ideally, our sort costing would just be better, but I think that > > raises the bar a little too high to start thinking of making > > improvements to that for this patch. > > It's trickier than it looks, cf f4c7c410e. But

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-10 Thread Tom Lane
torikoshia writes: > I updated the patch going along with the v3 direction. I think this adds about as many failure modes as it removes, if not more. * The connection string doesn't necessarily end with "'"; it could be a dollar-quoted string. * If it is a dollar-quoted string, there could be "

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 05:25:44PM +0100, Drouvot, Bertrand wrote: > I like the idea of comparing the full page (and not just the LSN) but > I'm not sure that adding the pageinspect dependency is a good thing. > > What about extracting the block directly from the relation file and > comparing it w

delay starting WAL receiver

2023-01-10 Thread Nathan Bossart
I discussed this a bit in a different thread [0], but I thought it deserved its own thread. After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed that the recovery tests consistently take much longer. Upon further inspection, it looks like a similar race condition to the one de

Re: Allow +group in pg_ident.conf

2023-01-10 Thread Michael Paquier
On Tue, Jan 10, 2023 at 09:42:19AM -0500, Andrew Dunstan wrote: > Ok, that sounds reasonable, but the cfbot doesn't like patches that > depend on other patches in a different email. Maybe you can roll this up > as an extra patch in your next version? It's pretty small. This can go two ways if both

Re: Infinite Interval

2023-01-10 Thread Joseph Koshakow
On Sun, Jan 8, 2023 at 11:17 PM jian he wrote: > > > > On Sun, Jan 8, 2023 at 4:22 AM Joseph Koshakow wrote: >> >> On Sat, Jan 7, 2023 at 3:05 PM Joseph Koshakow wrote: >> > >> > On Sat, Jan 7, 2023 at 3:04 PM Joseph Koshakow wrote: >> > > >> > > I think this patch is just about ready for revie

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-10 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 4:39 PM Peter Geoghegan wrote: > * Run VACUUM FREEZE. We need FREEZE in order to be able to hit the > relevant visibilitymap_set() call site (the one that passes > VISIBILITYMAP_ALL_FROZEN as its flags, without also passing > VISIBILITYMAP_ALL_VISIBLE). > > Now all_visible_

Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-10 Thread Masahiko Sawada
Hi, I realized that in CreateDecodingContext() function, we update both slot->data.two_phase and two_phase_at without acquiring the spinlock: /* Mark slot to allow two_phase decoding if not already marked */ if (ctx->twophase && !slot->data.two_phase) { slot->data.two_phase =

  1   2   >