Re: Emitting JSON to file using COPY TO

2024-01-15 Thread jian he
On Tue, Jan 16, 2024 at 11:46 AM jian he wrote: > > > I think the reason is maybe related to the function copy_dest_startup. I was wrong about this sentence. in the function CopyOneRowTo `if (!cstate->opts.json_mode)` else branch change to the following: else { Datum rowdata; StringInfo result;

Introduce a new API for TableAmRoutine

2024-01-15 Thread Japin Li
Hi, hackers Recently, I'm trying to implement a new TAM for PostgreSQL, I find there is no API for handling table's option. For example: CREATE TABLE t (...) USING new_am WITH (...); Is it possible add a new API to handle table's option in TableAmRoutine? -- Regrads, Japin Li.

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Masahiko Sawada
On Tue, Jan 16, 2024 at 1:07 PM Amit Kapila wrote: > > On Tue, Jan 16, 2024 at 9:03 AM shveta malik wrote: > > > > On Sat, Jan 13, 2024 at 12:54 PM Amit Kapila > > wrote: > > > > > > On Fri, Jan 12, 2024 at 5:50 PM shveta malik > > > wrote: > > > > > > > > There are multiple approaches

Re: [HACKERS] Allow INSTEAD OF DELETE triggers to modify the tuple for RETURNING

2024-01-15 Thread jian he
On Tue, Jan 9, 2024 at 6:21 PM vignesh C wrote: > > > doc seems to still have an issue. > > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F45%2F4617 > > > > In the regress test, do we need to clean up the created object after we use it. > > tested passed, looking at

RE: A failure in t/038_save_logical_slots_shutdown.pl

2024-01-15 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Bharath, > This is a more strict check because it is possible that even if the > latest confirmed_flush location is not persisted there is no > meaningful decodable WAL between whatever the last confirmed_flush > location saved on disk and the shutdown_checkpoint record. >

Re: [PATCH] Exponential backoff for auth_delay

2024-01-15 Thread Abhijit Menon-Sen
At 2024-01-04 08:30:36 +0100, mba...@gmx.net wrote: > > +typedef struct AuthConnRecord > +{ > + charremote_host[NI_MAXHOST]; > + boolused; > + double sleep_time; /* in milliseconds */ > +} AuthConnRecord; Do we really need a "used" field

Re: speed up a logical replica setup

2024-01-15 Thread Shubham Khanna
On Thu, Dec 21, 2023 at 11:47 AM Amit Kapila wrote: > > On Wed, Dec 6, 2023 at 12:53 PM Euler Taveira wrote: > > > > On Thu, Nov 9, 2023, at 8:12 PM, Michael Paquier wrote: > > > > On Thu, Nov 09, 2023 at 03:41:53PM +0100, Peter Eisentraut wrote: > > > On 08.11.23 00:12, Michael Paquier wrote: >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-15 Thread Masahiko Sawada
On Sun, Jan 14, 2024 at 10:43 PM John Naylor wrote: > > On Fri, Jan 12, 2024 at 3:49 PM Masahiko Sawada wrote: > > > > On Thu, Jan 11, 2024 at 9:28 AM Masahiko Sawada > > wrote: > > > So I agree to remove both max_bytes and num_items from the control > > > object.Also, as you mentioned, we can

Re: Add test module for Table Access Method

2024-01-15 Thread Japin Li
On Tue, 16 Jan 2024 at 13:15, Bharath Rupireddy wrote: > On Tue, Jan 16, 2024 at 10:28 AM Michael Paquier wrote: >> >> Hmm. I'd rather have it do something useful in terms of test coverage >> rather than being just an empty skull. >> >> How about adding the same kind of coverage as

Re: heavily contended lwlocks with long wait queues scale badly

2024-01-15 Thread Michael Paquier
On Thu, Jan 11, 2024 at 09:47:33AM -0500, Jonathan S. Katz wrote: > I have similar data sources to Nathan/Michael and I'm trying to avoid piling > on, but one case that's interesting occurred after a major version upgrade > from PG10 to PG14 on a database supporting a very active/highly concurrent

Re: minor replication slot docs edits

2024-01-15 Thread Bharath Rupireddy
On Mon, Jan 15, 2024 at 9:08 PM Alvaro Herrera wrote: > > Pursuant to a comment I made a few months ago[1], I propose the attached > changes to replication slots documentation. In essence, I want to > explain that replication slots are good, and the max_size GUC, before > moving on to explain

Re: Add test module for Table Access Method

2024-01-15 Thread Bharath Rupireddy
On Tue, Jan 16, 2024 at 10:28 AM Michael Paquier wrote: > > Hmm. I'd rather have it do something useful in terms of test coverage > rather than being just an empty skull. > > How about adding the same kind of coverage as dummy_index_am with a > couple of reloptions then? That can serve as a

Re: introduce dynamic shared memory registry

2024-01-15 Thread Bharath Rupireddy
On Sun, Jan 14, 2024 at 3:11 AM Nathan Bossart wrote: > > Here is a new version of the patch set with these changes. Thanks. Here are some comments on v7-0002. 1. +GetNamedDSMSegment(const char *name, size_t size, + void (*init_callback) (void *ptr), bool *found) +{ + +

Re: Add test module for Table Access Method

2024-01-15 Thread Michael Paquier
On Mon, Jan 15, 2024 at 03:40:30PM +0100, Jelte Fennema-Nio wrote: > On Mon, 15 Jan 2024 at 14:26, Aleksander Alekseev > wrote: >> To be fair, Postgres uses TAM internally, so there is at least one >> complete and up-to-date real-life example. > > Sure, but that one is quite hard to follow if

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Dilip Kumar
On Tue, Jan 16, 2024 at 9:37 AM Amit Kapila wrote: > > On Tue, Jan 16, 2024 at 9:03 AM shveta malik wrote: > > > Agreed and as said earlier I think it is better to make it a > PGC_SIGHUP. Also, not sure we can say it is a non-standard way as > already autovacuum launcher is handled in the same

Re: reorganize "Shared Memory and LWLocks" section of docs

2024-01-15 Thread Bharath Rupireddy
On Sun, Jan 14, 2024 at 2:58 AM Nathan Bossart wrote: > > Great. I've attached a v3 with a couple of fixes suggested in the other > thread [0]. I'll wait a little while longer in case anyone else wants to > take a look. The v3 patch looks good to me except for a nitpick: the input parameter

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Amit Kapila
On Tue, Jan 16, 2024 at 9:03 AM shveta malik wrote: > > On Sat, Jan 13, 2024 at 12:54 PM Amit Kapila wrote: > > > > On Fri, Jan 12, 2024 at 5:50 PM shveta malik wrote: > > > > > > There are multiple approaches discussed and tried when it comes to > > > starting a slot-sync worker. I am

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-01-15 Thread Bharath Rupireddy
On Tue, Jan 16, 2024 at 7:25 AM Jeff Davis wrote: > > On Fri, 2024-01-12 at 17:17 -0800, Jeff Davis wrote: > > I think 0004 needs a bit more work, so I'm leaving it off for now, > > but > > I'll bring it back in the next patch set. > > Here's the next patch set. 0001 - 0003 are mostly the same

Re: Emitting JSON to file using COPY TO

2024-01-15 Thread jian he
On Tue, Jan 9, 2024 at 4:40 AM Joe Conway wrote: > > On 1/8/24 14:36, Dean Rasheed wrote: > > On Thu, 7 Dec 2023 at 01:10, Joe Conway wrote: > >> > >> The attached should fix the CopyOut response to say one column. > >> > > > > Playing around with this, I found a couple of cases that generate an

Re: Synchronizing slots from primary to standby

2024-01-15 Thread shveta malik
On Sat, Jan 13, 2024 at 12:54 PM Amit Kapila wrote: > > On Fri, Jan 12, 2024 at 5:50 PM shveta malik wrote: > > > > There are multiple approaches discussed and tried when it comes to > > starting a slot-sync worker. I am summarizing all here: > > > > 1) Make slotsync worker as an Auxiliary

Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-15 Thread David Rowley
While working on [1], I noticed some strange code in DiscreteKnapsack() which seems to be aiming to copy the Bitmapset. It's not that obvious at a casual glance, but: sets[j] = bms_del_members(sets[j], sets[j]); this is aiming to zero all the words in the set by passing the same set in both

Re: Add PQsendSyncMessage() to libpq

2024-01-15 Thread Michael Paquier
On Mon, Jan 15, 2024 at 10:49:56AM +0100, Alvaro Herrera wrote: > the new function pqPipelineSyncInternal is not a wrapper for these other > two functions -- the opposite is true actually. We tend to use the term > "workhorse" or "internal workhorse" for this kind of thing. Indeed, makes sense.

Re: Revise the Asserts added to bimapset manipulation functions

2024-01-15 Thread David Rowley
On Tue, 2 Jan 2024 at 20:18, Richard Guo wrote: > I think one purpose of introducing REALLOCATE_BITMAPSETS is to help find > dangling pointers to Bitmapset's. From this point of view, I agree that > we should call bms_copy_and_free() in bms_add_members(), because the > bitmapset 'a' might be

Re: Make mesage at end-of-recovery less scary.

2024-01-15 Thread Kyotaro Horiguchi
Thank you for the comments. At Fri, 12 Jan 2024 15:03:26 +0300, Aleksander Alekseev wrote in > ``` > +p = (char *) record; > +pe = p + XLOG_BLCKSZ - (RecPtr & (XLOG_BLCKSZ - 1)); > + > +while (p < pe && *p == 0) > +p++; > + > +if (p == pe) > ``` > >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-15 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 15 Jan 2024 16:03:41 +0900, Masahiko Sawada wrote: >> Defining one more static const struct instead of providing a >> convenient (but a bit tricky) macro may be straightforward: >> >> static const

Re: POC: GROUP BY optimization

2024-01-15 Thread Richard Guo
On Mon, Jan 15, 2024 at 3:56 PM Alexander Korotkov wrote: > On Mon, Jan 15, 2024 at 8:42 AM Richard Guo > wrote: > > On Mon, Jan 15, 2024 at 8:20 AM Alexander Korotkov > wrote: > >> > >> Thank you for providing the test case relevant for this code change. > >> The revised patch incorporating

Re: make pg_ctl more friendly

2024-01-15 Thread Junwang Zhao
Hi Nathan, On Tue, Jan 16, 2024 at 5:39 AM Nathan Bossart wrote: > > + POSTMASTER_RECOVERY_SHUTDOWN, > > Perhaps this should be POSTMASTER_SHUTDOWN_IN_RECOVERY to match the state > in the control file? Agreed > > + case POSTMASTER_RECOVERY_SHUTDOWN: > +

Re: Error "initial slot snapshot too large" in create replication slot

2024-01-15 Thread Kyotaro Horiguchi
At Fri, 12 Jan 2024 11:28:09 -0500, Robert Haas wrote in > However, I wonder whether this whole area is in need of a bigger > rethink. There seem to be a number of situations in which the split > into xip and subxip arrays is not very convenient, and also some > situations where it's quite

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-01-15 Thread Peter Geoghegan
On Mon, Jan 15, 2024 at 2:32 PM Matthias van de Meent wrote: > Can you pull these planner changes into their own commit(s)? > As mentioned upthread, it's a significant change in behavior that > should have separate consideration and reference in the commit log. I > really don't think it should be

Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [Fixed Already]

2024-01-15 Thread Kirk Wolak
On Mon, Jan 15, 2024 at 9:03 AM Daniel Gustafsson wrote: > > On 15 Jan 2024, at 07:24, Kirk Wolak wrote: > > > You have a commit [1] that MIGHT fix this. > > I have a script that recreates the problem, using random data in pg_temp. > > And a nested cursor. > > Running your reproducer script

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Peter Smith
Here are some review comments for patch v61-0002 == doc/src/sgml/logical-replication.sgml 1. + + Examples: logical replication failover The current documentation structure (after the patch is applied) looks like this: 30.1. Publication 30.2. Subscription 30.2.1. Replication Slot

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-15 Thread torikoshia
On 2024-01-16 00:17, Alexander Korotkov wrote: On Mon, Jan 15, 2024 at 8:44 AM Masahiko Sawada wrote: On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov wrote: > > On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada wrote: > > Thank you for updating the patch. Here are two comments: > > > >

Re: Add PQsendSyncMessage() to libpq

2024-01-15 Thread Michael Paquier
On Mon, Jan 15, 2024 at 10:01:59AM +0100, Jelte Fennema-Nio wrote: > Error message should be "second SELECT" not "first SELECT". Same note > for the error message in the third pipeline, where it still says > "second SELECT". > > same issue: s/first/second/g (and s/second/third/g for the existing >

Re: make pg_ctl more friendly

2024-01-15 Thread Nathan Bossart
+ POSTMASTER_RECOVERY_SHUTDOWN, Perhaps this should be POSTMASTER_SHUTDOWN_IN_RECOVERY to match the state in the control file? + case POSTMASTER_RECOVERY_SHUTDOWN: + print_msg(_("PITR shutdown\n")); +

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-01-15 Thread Jeff Davis
On Mon, 2024-01-15 at 15:53 -0500, Joe Conway wrote: > I took a quick scan through the patch. The only thing that jumped out > at > me was that it seems like it might make sense to use > quote_literal_cstr() rather than defining your own > appendEscapedValue() > function? The rules are

Re: Fixing backslash dot for COPY FROM...CSV

2024-01-15 Thread Robert Haas
On Mon, Dec 18, 2023 at 3:36 PM Daniel Verite wrote: > PFA a patch that attempts to fix the bug that \. on a line > by itself is handled incorrectly by COPY FROM ... CSV. > This issue has been discussed several times previously, > for instance in [1] and [2], and mentioned in the > doc for \copy

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-15 Thread Melanie Plageman
On Mon, Jan 15, 2024 at 12:29:57PM -0500, Robert Haas wrote: > On Fri, Jan 12, 2024 at 4:05 PM Melanie Plageman > wrote: > > Yea, that works for now. I mean, I think the way we should do it is > > update the FSM in lazy_scan_noprune(), but, for the purposes of this > > patch, yes.

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-01-15 Thread Joe Conway
On 1/12/24 20:17, Jeff Davis wrote: On Fri, 2024-01-05 at 16:11 +0530, Ashutosh Bapat wrote: I don't think we need to add a test for every FDW. E.g. adding a test in file_fdw would be pointless. But postgres_fdw is special. The test could further create a foreign table ftab_foo on subscriber

Re: Custom explain options

2024-01-15 Thread Konstantin Knizhnik
On 15/01/2024 5:08 pm, Tomas Vondra wrote: My patch does not care about prefetching internal index pages. Yes, it's a limitation, but my assumption is the internal pages are maybe 0.1% of the index, and typically very hot / cached. Yes, if the index is not used very often, this may be untrue.

Re: cleanup patches for incremental backup

2024-01-15 Thread Matthias van de Meent
On Mon, 15 Jan 2024 at 17:58, Robert Haas wrote: > > On Sat, Jan 13, 2024 at 1:00 PM Alexander Lakhin wrote: > > I've found one more typo in the sgml: > > summarized_pid > > And one in a comment: > > sumamry > > > > A trivial fix is attached. > > Thanks, committed. Off-list I was notified that

Re: Forbid the use of invalidated physical slots in streaming replication.

2024-01-15 Thread Robert Haas
On Thu, Dec 7, 2023 at 8:00 AM Zhijie Hou (Fujitsu) wrote: > After looking closer, it seems this behavior started from 15f8203 which > introduced the > ReplicationSlotInvalidationCause 'invalidated', after that we check the > invalidated enum > in Xmin/Lsn computation function. Adding Andres

Re: ALTER TYPE OWNER fails to recurse to multirange

2024-01-15 Thread Robert Haas
On Mon, Jan 15, 2024 at 2:28 PM Tom Lane wrote: > I'm reasoning by analogy to array types, which are automatically > created and automatically updated to keep the same ownership > etc. properties as their base type. To the extent that multirange > types don't act exactly like that, I say it's a

Re: gai_strerror() is not thread-safe on Windows

2024-01-15 Thread Robert Haas
On Wed, Dec 6, 2023 at 8:45 PM Kyotaro Horiguchi wrote: > > So I think we should just hard-code the error messages in English and > > move on. However, English is my language so perhaps I should abstain > > and leave it to others to decide how important that is. > > I also think that would be a

Re: Built-in CTYPE provider

2024-01-15 Thread Jeff Davis
On Mon, 2024-01-15 at 15:30 +0100, Daniel Verite wrote: > Concerning the target category_test, it produces failures with > versions of ICU with Unicode < 15. The first one I see with Ubuntu > 22.04 (ICU 70.1) is: ... > I find these results interesting because they tell us what contents > can

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-01-15 Thread Matthias van de Meent
On Thu, 28 Dec 2023 at 18:28, Peter Geoghegan wrote: > > On Sat, Dec 9, 2023 at 10:38 AM Peter Geoghegan wrote: > > Attached is v8, which pretty much rips all of this stuff out. > > Attached is v9, which I'm posting just to fix bitrot. The patch > stopped cleanly applying against HEAD due to

Re: ALTER TYPE OWNER fails to recurse to multirange

2024-01-15 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 15, 2024 at 1:27 PM Tom Lane wrote: >> That's pretty broken, isn't it? joe would own the multirange if he'd >> created the range to start with. Even if you think the ownerships >> ideally should be separable, this behavior causes existing pg_dump >> files to

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-15 Thread Tom Lane
Robert Haas writes: > I know this is on Tom's to-do list which makes me a bit reluctant to > get too involved here, because certainly he knows this code better > than I do, maybe better than anyone does, but on the other hand, we > shouldn't leave server crashes unfixed for too long, so maybe I

Re: ALTER TYPE OWNER fails to recurse to multirange

2024-01-15 Thread Robert Haas
On Mon, Jan 15, 2024 at 1:27 PM Tom Lane wrote: > That's pretty broken, isn't it? joe would own the multirange if he'd > created the range to start with. Even if you think the ownerships > ideally should be separable, this behavior causes existing pg_dump > files to restore incorrectly, because

Re: [PATCH] LockAcquireExtended improvement

2024-01-15 Thread Robert Haas
Hello Jingxian Li! I agree with you that this behavior seems surprising. I don't think it's quite a bug, more of a limitation. However, I think it would be nice to fix it if we can find a good way to do that. On Wed, Nov 29, 2023 at 10:43 PM Jingxian Li wrote: > Transaction A already holds an

Re: postgres_fdw fails to see that array type belongs to extension

2024-01-15 Thread Tom Lane
David Geier writes: > On 12/27/23 18:38, Tom Lane wrote: >> Hmm. It seems odd that if an extension defines a type, the type is >> listed as a member of the extension but the array type is not. >> That makes it look like the array type is an externally-created >> thing that happens to depend on

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2024-01-15 Thread Robert Haas
On Thu, Jan 11, 2024 at 3:33 AM Kyotaro Horiguchi wrote: > Is it correct to understand that you are requesting changes as follows? > > --- a/src/bin/pg_ctl/pg_ctl.c > +++ b/src/bin/pg_ctl/pg_ctl.c > @@ -1995,11 +1995,14 @@ pgwin32_find_postmaster_pid(pid_t shell_pid) > * > *

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-15 Thread Robert Haas
On Mon, Jan 8, 2024 at 3:32 AM Richard Guo wrote: > Thanks for the suggestion. Attached is an updated patch which is added > with a commit message that tries to explain the problem and the fix. This is great. The references to "the sampling infos" are a little bit confusing to me. I'm not sure

ALTER TYPE OWNER fails to recurse to multirange

2024-01-15 Thread Tom Lane
d=# create type varbitrange as range (subtype = varbit); CREATE TYPE d=# \dT+ List of data types Schema | Name | Internal name | Size | Elements | Owner | Access privileges | Description

Re: Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-15 Thread Daniel Gustafsson
> On 15 Jan 2024, at 16:49, Kirk Wolak wrote: > On Mon, Jan 15, 2024 at 9:03 AM Daniel Gustafsson > wrote: > The script starts by creating 90 Million rows... In my world that part of > the script, plus the indexes, etc. Takes about 8-9 minutes. > And has no memory

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-15 Thread Robert Haas
On Fri, Jan 12, 2024 at 4:05 PM Melanie Plageman wrote: > Yea, that works for now. I mean, I think the way we should do it is > update the FSM in lazy_scan_noprune(), but, for the purposes of this > patch, yes. has_lpdead_items output parameter seems fine to me. Here's v2. It's not exactly

Re: cleanup patches for incremental backup

2024-01-15 Thread Robert Haas
On Sat, Jan 13, 2024 at 1:00 PM Alexander Lakhin wrote: > I've found one more typo in the sgml: > summarized_pid > And one in a comment: > sumamry > > A trivial fix is attached. Thanks, committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Rework LogicalOutputPluginWriterUpdateProgress

2024-01-15 Thread vignesh C
On Mon, 13 Mar 2023 at 08:17, wangw.f...@fujitsu.com wrote: > > On Fri, Mar 10, 2023 20:17 PM Osumi, Takamichi/大墨 昂道 > wrote: > > Hi, > > > > > > On Friday, March 10, 2023 6:32 PM Wang, Wei/王 威 > > wrote: > > > Attach the new patch set. > > Thanks for updating the patch ! One review comment on

Re: Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-15 Thread Pavel Stehule
po 15. 1. 2024 v 15:03 odesílatel Daniel Gustafsson napsal: > > On 15 Jan 2024, at 07:24, Kirk Wolak wrote: > > > You have a commit [1] that MIGHT fix this. > > I have a script that recreates the problem, using random data in pg_temp. > > And a nested cursor. > > Running your reproducer

Re: Make attstattarget nullable

2024-01-15 Thread Peter Eisentraut
On 12.01.24 12:16, Alvaro Herrera wrote: In get_attstattarget() I think we should return 0 for dropped columns without reading attstattarget, which is useless anyway, and if it did happen to return non-null, it might cause us to do stuff, which would be a waste. I ended up deciding to get rid

Re: Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-15 Thread Kirk Wolak
On Mon, Jan 15, 2024 at 9:03 AM Daniel Gustafsson wrote: > > On 15 Jan 2024, at 07:24, Kirk Wolak wrote: > > > You have a commit [1] that MIGHT fix this. > > I have a script that recreates the problem, using random data in pg_temp. > > And a nested cursor. > > Running your reproducer script

minor replication slot docs edits

2024-01-15 Thread Alvaro Herrera
Pursuant to a comment I made a few months ago[1], I propose the attached changes to replication slots documentation. In essence, I want to explain that replication slots are good, and the max_size GUC, before moving on to explain that the other methods are worse. Thanks [1]

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-15 Thread Alexander Korotkov
On Mon, Jan 15, 2024 at 8:44 AM Masahiko Sawada wrote: > > On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov > wrote: > > > > On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada > > wrote: > > > Thank you for updating the patch. Here are two comments: > > > > > > --- > > > + if

Re: Custom explain options

2024-01-15 Thread Tomas Vondra
On 1/15/24 15:22, Konstantin Knizhnik wrote: > > On 14/01/2024 11:47 pm, Tomas Vondra wrote: >> The thing that was not clear to me is who decides what to prefetch, >> which code issues the prefetch requests etc. In the github links you >> shared I see it happens in the index AM code (in

Re: archive modules loose ends

2024-01-15 Thread Nathan Bossart
On Mon, Jan 15, 2024 at 12:21:44PM +, Li, Yong wrote: > The patch looks good to me. With the context explained in the thread, > the patch is easy to understand. > The patch serves as a refactoring which pulls up common memory management > and error handling concerns into the pgarch.c. With

Re: Add test module for Table Access Method

2024-01-15 Thread Jelte Fennema-Nio
On Mon, 15 Jan 2024 at 14:26, Aleksander Alekseev wrote: > To be fair, Postgres uses TAM internally, so there is at least one > complete and up-to-date real-life example. Sure, but that one is quite hard to follow if you don't already know lots of details of the heap storage. At least for me,

Re: Built-in CTYPE provider

2024-01-15 Thread Daniel Verite
Jeff Davis wrote: > New version attached. [v16] Concerning the target category_test, it produces failures with versions of ICU with Unicode < 15. The first one I see with Ubuntu 22.04 (ICU 70.1) is: category_test: Postgres Unicode version:15.1 category_test: ICU Unicode

Re: Custom explain options

2024-01-15 Thread Konstantin Knizhnik
On 14/01/2024 11:47 pm, Tomas Vondra wrote: The thing that was not clear to me is who decides what to prefetch, which code issues the prefetch requests etc. In the github links you shared I see it happens in the index AM code (in nbtsearch.c). It is up to the particular plan node (seqscan,

Re: More new SQL/JSON item methods

2024-01-15 Thread Peter Eisentraut
Attached are two small fixup patches for your patch set. In the first one, I simplified the grammar for the .decimal() method. It seemed a bit overkill to build a whole list structure when all we need are 0, 1, or 2 arguments. Per SQL standard, the precision and scale arguments are unsigned

Re: Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-15 Thread Daniel Gustafsson
> On 15 Jan 2024, at 07:24, Kirk Wolak wrote: > You have a commit [1] that MIGHT fix this. > I have a script that recreates the problem, using random data in pg_temp. > And a nested cursor. Running your reproducer script in a tight loop for a fair bit of time on the v16 HEAD I cannot see any

Re: Add test module for Table Access Method

2024-01-15 Thread Aleksander Alekseev
Hi, > When trying to implement a table access method in the past I remember > very well that I was having a really hard time finding an example of > one. To be fair, Postgres uses TAM internally, so there is at least one complete and up-to-date real-life example. Learning curve for TAMs is

Re: postgres_fdw fails to see that array type belongs to extension

2024-01-15 Thread David Geier
Hi, I realized that ALTER EXTENSION foo ADD TYPE _bar does pretty much the same via ExecAlterExtensionContentsStmt(). So the code in the patch seems fine. On 1/8/24 12:21, David Geier wrote: The attached patch just adds a 2nd dependency between the array type and the extension, using

Re: POC: GROUP BY optimization

2024-01-15 Thread Alena Rybakina
On 15.01.2024 12:46, Andrei Lepikhov wrote: On 15/1/2024 13:42, Richard Guo wrote: On Mon, Jan 15, 2024 at 8:20 AM Alexander Korotkov mailto:aekorot...@gmail.com>> wrote:     Thank you for providing the test case relevant for this code change.     The revised patch incorporating this change

Re: Fix a possible socket leak at Windows (src/backend/port/win32/socket.c)

2024-01-15 Thread Daniel Gustafsson
> On 13 Jan 2024, at 22:38, Ranier Vilela wrote: > In the pgwin32_socket function (src/backend/port/win32/socket.c), there is a > possible socket leak if the socket cannot be made non-blocking. I don't know Windows well enough to comment on the implications of not calling closesocket here, but

Re: archive modules loose ends

2024-01-15 Thread Li, Yong
> On Nov 29, 2023, at 01:18, Nathan Bossart wrote: > > External Email > > Here is a new version of the patch with feedback addressed. > > -- > Nathan Bossart > Amazon Web Services: https://aws.amazon.com Hi Nathan, The patch looks good to me. With the context explained in the thread, the

Re: Lockless exit path for ReplicationOriginExitCleanup

2024-01-15 Thread Alvaro Herrera
Thanks, pushed. I reworded the comment again, though. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

Re: Postgres and --config-file option

2024-01-15 Thread Aleksander Alekseev
Hi, > It might be worthwhile to update the documentation if it would've helped > prevent confusion here. > Its documentation also describes this method of specifying parameters in > the 'Examples' section. I believe the documentation for 'postgres' already does a decent job in describing what

Re: Make NUM_XLOGINSERT_LOCKS configurable

2024-01-15 Thread Jakub Wartak
On Fri, Jan 12, 2024 at 7:33 AM Bharath Rupireddy wrote: > > On Wed, Jan 10, 2024 at 11:43 AM Tom Lane wrote: > > > > Bharath Rupireddy writes: > > > On Wed, Jan 10, 2024 at 10:00 AM Tom Lane wrote: > > >> Maybe. I bet just bumping up the constant by 2X or 4X or so would get > > >> most of

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Amit Kapila
On Mon, Jan 15, 2024 at 2:54 PM Bertrand Drouvot wrote: > > On Sat, Jan 13, 2024 at 10:05:52AM +0530, Amit Kapila wrote: > > On Fri, Jan 12, 2024 at 12:07 PM Bertrand Drouvot > > wrote: > > > Maybe the "best" approach would be to have a way to detect that a slot > > > has been > > > re-created

Re: automating RangeTblEntry node support

2024-01-15 Thread Peter Eisentraut
On 06.12.23 21:02, Peter Eisentraut wrote: I have been looking into what it would take to get rid of the custom_read_write and custom_query_jumble for the RangeTblEntry node type.  This is one of the larger and more complex exceptions left. (Similar considerations would also apply to the

Re: System username in pg_stat_activity

2024-01-15 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 05:16:53PM +0100, Magnus Hagander wrote: > On Thu, Jan 11, 2024 at 5:55 PM Bertrand Drouvot > wrote: > > > > I'm wondering if it would make sense to populate it for parallel workers > > too. > > I think it's doable thanks to d951052, but I'm not sure it's worth it

Re: Add PQsendSyncMessage() to libpq

2024-01-15 Thread Alvaro Herrera
On 2024-Jan-15, Michael Paquier wrote: Looks good! Just some small notes, > +/* > + * Wrapper for PQpipelineSync and PQsendPipelineSync. > * > * It's legal to start submitting more commands in the pipeline immediately, > * without waiting for the results of the current pipeline. There's

Re: POC: GROUP BY optimization

2024-01-15 Thread Andrei Lepikhov
On 15/1/2024 13:42, Richard Guo wrote: On Mon, Jan 15, 2024 at 8:20 AM Alexander Korotkov > wrote: Thank you for providing the test case relevant for this code change. The revised patch incorporating this change is attached.  Now the patchset looks

Re: Add test module for Table Access Method

2024-01-15 Thread Jelte Fennema-Nio
On Thu, 28 Sept 2023 at 03:08, Michael Paquier wrote: > dummy_index_am has included from the start additional coverage for the > various internal add_*_reloption routines, that were never covered in > the core tree. Except if I am missing something, I am not seeing some > of the extra usefulness

Re: On login trigger: take three

2024-01-15 Thread Daniel Gustafsson
> On 13 Jan 2024, at 17:00, Alexander Lakhin wrote: > I suspected that this failure was caused by autovacuum, and I've managed to > reproduce it without Valgrind or slowing down a machine. This might be due to the fact that the cleanup codepath to remove the flag when there is no login event

Re: Synchronizing slots from primary to standby

2024-01-15 Thread Bertrand Drouvot
Hi, On Sat, Jan 13, 2024 at 10:05:52AM +0530, Amit Kapila wrote: > On Fri, Jan 12, 2024 at 12:07 PM Bertrand Drouvot > wrote: > > Maybe the "best" approach would be to have a way to detect that a slot has > > been > > re-created on the primary (but that would mean rely on more than the slot >

RE: Documentation to upgrade logical replication cluster

2024-01-15 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, Thanks for updating the patch! > > 7. > > ``` > > + > > +dba@node1:/opt/PostgreSQL/postgres//bin$ pg_ctl -D > /opt/PostgreSQL/pub_data stop -l logfile > > + > > ``` > > > > Hmm. I thought you did not have to show the current directory. You were in > > the > > bin dir, but it is

Re: On login trigger: take three

2024-01-15 Thread Daniel Gustafsson
> On 13 Jan 2024, at 17:00, Alexander Lakhin wrote: > DROP EVENT TRIGGER olt; > SELECT dathasloginevt FROM pg_database WHERE datname= current_database(); > dathasloginevt > > t > (1 row) > > Although after reconnection (\c, as done in the event_trigger_login test), > this

Re: Add PQsendSyncMessage() to libpq

2024-01-15 Thread Jelte Fennema-Nio
On Mon, 15 Jan 2024 at 08:50, Michael Paquier wrote: > Yeah, I'll go with that after a second look. Attached is what I am > finishing with, and I have reproduced some numbers with the pgbench > metacommand mentioned upthread, which is reeeaaally nice. Code looks good to me. But one small notes

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-15 Thread Alexander Lakhin
Hello Michael and Bertrand, 15.01.2024 06:59, Michael Paquier wrote: The WAL records related to standby snapshots are playing a lot with the randomness of the failures we are seeing. Alexander has mentioned offlist something else: using SIGSTOP on the bgwriter to avoid these records and make

Re: Create shorthand for including all extra tests

2024-01-15 Thread Nazir Bilal Yavuz
Hi, On Wed, 10 Jan 2024 at 23:48, Peter Eisentraut wrote: > > On 05.09.23 19:26, Nazir Bilal Yavuz wrote: > > Thanks for the feedback! I updated the patch, 'needs-private-lo' > > option enables kerberos, ldap, load_balance and ssl extra tests now. > > As was discussed, I don't think "needs

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-15 Thread Bertrand Drouvot
Hi, On Mon, Jan 15, 2024 at 01:11:26PM +0900, Michael Paquier wrote: > On Sun, Jan 14, 2024 at 11:08:39PM -0500, Tom Lane wrote: > > Michael Paquier writes: > >> While thinking about that, a second idea came into my mind: a > >> superuser-settable developer GUC to disable such WAL records to be