Re: Conflict detection for update_deleted in logical replication

2025-08-25 Thread Amit Kapila
On Mon, Aug 25, 2025 at 5:05 PM Amit Kapila wrote: > > A few comments on 0001: > Some more comments: 1. + /* + * Return false if the leader apply worker has stopped retaining + * information for detecting conflicts. This implies that update_deleted + * can no longer be reliably detected. + */ + i

Re: Per backend relation statistics tracking

2025-08-25 Thread Bertrand Drouvot
Hi, On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > Hi, > > On 2025-08-12 07:48:10 +, Bertrand Drouvot wrote: > > From 9e2f8cb9a87f1d9be91f2f39ef25fbb254944968 Mon Sep 17 00:00:00 2001 > > From: Bertrand Drouvot > > Date: Mon, 4 Aug 2025 08:14:02 + > > Subject: [PATCH v1

Re: Improve error message for duplicate labels in enum types

2025-08-25 Thread Jim Jones
On 26.08.25 04:55, Yugo Nagata wrote: > Thank you for taking a look. That makes sense, so I updated the message to: > > ERROR: enum label "a" used more than once Nice. > I also added a test for duplicate enum entries to enum.sql, > since tests for existing entries are already there. +1 LGT

Re: Potential problem in commit f777d773878 and 4f7f7b03758

2025-08-25 Thread Dilip Kumar
On Tue, Aug 26, 2025 at 11:31 AM Dilip Kumar wrote: > > On Sun, Aug 24, 2025 at 5:59 PM Srinath Reddy Sadipiralla > wrote: > > > > Hi, > > Thanks Dilip and Matheus for working on this , i reviewed the latest patch > > given my Matheus and it LGTM but i have doubt that in f777d773878 commit > >

Re: Per backend relation statistics tracking

2025-08-25 Thread Bertrand Drouvot
Hi, On Mon, Aug 25, 2025 at 07:22:43PM -0500, Sami Imseih wrote: > > Adding these fields to the backend level stats spread based on the > > backend PID without the knowledge of the relation they're related with > > makes it much less interesting IMO, because we lose a lot of > > granularity value

Re: Per backend relation statistics tracking

2025-08-25 Thread Bertrand Drouvot
Hi, On Tue, Aug 26, 2025 at 08:12:45AM +0900, Michael Paquier wrote: > On Mon, Aug 25, 2025 at 05:51:38PM -0500, Sami Imseih wrote: > > I have not gone through them in detail yet, but +1 on adding backend > > activity > > stats. This provides another level of drill down to spot anomalous sessions

Re: Potential problem in commit f777d773878 and 4f7f7b03758

2025-08-25 Thread Dilip Kumar
On Sun, Aug 24, 2025 at 5:59 PM Srinath Reddy Sadipiralla wrote: > > Hi, > Thanks Dilip and Matheus for working on this , i reviewed the latest patch > given my Matheus and it LGTM but i have doubt that in f777d773878 commit the > $libdir was moved out from expand_dynamic_library_name into > lo

Re: Logical Replication of sequences

2025-08-25 Thread shveta malik
On Thu, Aug 21, 2025 at 10:08 PM vignesh C wrote: > > I have also addressed all the comments from [1] in the attached > v20250823 version patch. > [1] - > https://www.postgresql.org/message-id/CAA4eK1%2BoVQW8oP%3DLo1X8qac6dzg-fgGQ6R_F_psfokUEqe%2Ba6w%40mail.gmail.com > Thank You for the patches

Optimize JsonbContainerTypeName by reordering type checks

2025-08-25 Thread Chao Li
Hi Hacker, While reading jsonb related code, I found JsonbContainerTypeName() can be optimized. The function currently checks for the less common scalar container type before checking for objects and arrays. This patch reorders the checks to prioritize the most common cases. The macros JsonC

Re: CREATE SCHEMA ... CREATE DOMAIN support

2025-08-25 Thread jian he
On Fri, Aug 22, 2025 at 4:59 PM Kirill Reshke wrote: > > > > > the full in 11.1 is: > > 11.1 > > > > ::= > > > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > | > > > I also added CREATE SCHEMA CREATE TYPE. > With these patches applied: >

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-25 Thread Ajin Cherian
On Fri, Aug 22, 2025 at 3:44 PM shveta malik wrote: > > On Wed, Aug 20, 2025 at 10:53 AM Ajin Cherian wrote: > > > > > > I've removed them. > > Attaching patch v8 addressing the above comments. > > > > Thanks for the patch. Please find a few comments: > > 1) > When the API is in progress, and mea

Re: Memory leak of SMgrRelation object on standby

2025-08-25 Thread Jingtang Zhang
Hi~ > I purpose a patch which calls smgrdestroyall() when redo each > XLOG_CHECKPOINT_ONLINE, so that it can keep the same frequency of calling > smgrdestroyall() as background processes on primary. I don't call it for > XLOG_CHECKPOINT_SHUTDOWN because the process is about to exit so that the > m

Re: Non-reproducible AIO failure

2025-08-25 Thread Thomas Munro
On Tue, Aug 26, 2025 at 12:37 PM Andres Freund wrote: > I'm a bit confused by this focus on bitfields - both Alexander and Konstantin > stated they could reproduce the issue without the bitfields. Konstantin's message all seem to say it *did* fix it? But I do apologise for working through the sa

Re: [PATCH] no table rewrite when set column type to constrained domain

2025-08-25 Thread jian he
On Thu, Jul 10, 2025 at 2:00 AM jian he wrote: > > we can add a new boolean field, coerce_to_domain, to NewColumnValue. this > field > is set to true only when changing an existing column's type to a constrained > domain. In such cases, a table scan is enough—no table rewrite is needed. > coerce_

Re: Non-reproducible AIO failure

2025-08-25 Thread Thomas Munro
On Tue, Aug 26, 2025 at 12:45 PM Andres Freund wrote: > On 2025-08-25 10:43:21 +1200, Thomas Munro wrote: > > On Mon, Aug 25, 2025 at 6:11 AM Konstantin Knizhnik > > wrote: > > > In theory even replacing bitfield with in should not > > > avoid race condition, because they are still shared the sa

Re: Improve error message for duplicate labels in enum types

2025-08-25 Thread Yugo Nagata
Hi Jim, On Mon, 25 Aug 2025 10:08:23 +0200 Jim Jones wrote: > Hi Yugo > > On 03.07.25 17:04, Yugo Nagata wrote: > > Currently, when creating an enum type, duplicate labels are caught by a > > unique > > index on pg_enum, resulting in a generic error message. > > > > postgres=# create type t

Re: index prefetching

2025-08-25 Thread Peter Geoghegan
On Mon Aug 25, 2025 at 10:18 AM EDT, Tomas Vondra wrote: > The attached patch is a PoC implementing this. The core idea is that if > we measure "miss probability" for a chunk of requests, we can use that > to estimate the distance needed to generate e_i_c IOs. I noticed an assertion failure when t

Re: Test instability when pg_dump orders by OID

2025-08-25 Thread Tom Lane
Noah Misch writes: > On Mon, Aug 25, 2025 at 05:15:55PM -0400, Andres Freund wrote: >> I think it's also about removing painful manual testing - and imo manually >> running cross-version pg_upgrade tests is really rather painful. > I make the buildfarm client drive it. Yeah, same here. I have a

Re: Test instability when pg_dump orders by OID

2025-08-25 Thread Noah Misch
On Mon, Aug 25, 2025 at 05:15:55PM -0400, Andres Freund wrote: > On 2025-08-24 09:08:16 -0700, Noah Misch wrote: > > On Sun, Aug 24, 2025 at 11:50:01AM -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > I wonder if it's worth adding support to CI to perform the cross-version > > > > upgrad

Re: END is not a reserved word

2025-08-25 Thread David G. Johnston
On Mon, Aug 25, 2025 at 3:49 PM Vicky Vergara wrote: > On the list of reserved words, END is marked as reserved. > https://www.postgresql.org/docs/current/sql-keywords-appendix.html > The text on that page notes a couple of times that using reserved words as column labels is accepted - with the

Re: Use LW_SHARED in WakeupWalSummarizer() for WALSummarizerLock lock

2025-08-25 Thread Masahiko Sawada
On Mon, Aug 25, 2025 at 5:08 PM Nathan Bossart wrote: > > On Mon, Aug 25, 2025 at 03:38:13PM -0700, Masahiko Sawada wrote: > > While reading walsummarizer.c code, I noticed that in > > WakeupWalSummarizer() we acquire the WALSummarizerLock lock in > > LW_EXCLUSIVE mode despite only reading > > Wal

Re: Non-reproducible AIO failure

2025-08-25 Thread Andres Freund
Hi, On 2025-08-25 10:43:21 +1200, Thomas Munro wrote: > On Mon, Aug 25, 2025 at 6:11 AM Konstantin Knizhnik > wrote: > > In theory even replacing bitfield with in should not > > avoid race condition, because they are still shared the same cache line. > > I'm no expert in this stuff, but that's

Re: Parallel heap vacuum

2025-08-25 Thread Masahiko Sawada
On Fri, Jul 25, 2025 at 10:40 AM Robert Haas wrote: > > On Mon, Jul 21, 2025 at 7:49 PM Andres Freund wrote: > > That is not to say you can't have callbacks or such, it just doesn't make > > sense for those callbacks to be at the level of tableam. If you want to make > > vacuumparallel support pa

Re: Non-reproducible AIO failure

2025-08-25 Thread Andres Freund
Hi, I'm a bit confused by this focus on bitfields - both Alexander and Konstantin stated they could reproduce the issue without the bitfields. But we have observed the generated code being pretty grotty and it's caused more than enough confusion - so let's just replace them with plain uint8's and

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Sami Imseih
> On Mon, Aug 25, 2025 at 04:59:41PM -0500, Sami Imseih wrote: > > hmm, can we really avoid a shared lock when reading from shared memory? > > considering access for both reads and writes can be concurrent to shared > > memory. We are also taking an exclusive lock when writing a new tranche. > > We

Re: Per backend relation statistics tracking

2025-08-25 Thread Andres Freund
Hi, On 2025-08-12 07:48:10 +, Bertrand Drouvot wrote: > From 9e2f8cb9a87f1d9be91f2f39ef25fbb254944968 Mon Sep 17 00:00:00 2001 > From: Bertrand Drouvot > Date: Mon, 4 Aug 2025 08:14:02 + > Subject: [PATCH v1 01/10] Adding per backend relation statistics tracking > > This commit introduce

Re: Per backend relation statistics tracking

2025-08-25 Thread Sami Imseih
> Adding these fields to the backend level stats spread based on the > backend PID without the knowledge of the relation they're related with > makes it much less interesting IMO, because we lose a lot of > granularity value that we have with the pg_statio_* relations, at the > cost of more bloat,

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 04:59:41PM -0500, Sami Imseih wrote: > hmm, can we really avoid a shared lock when reading from shared memory? > considering access for both reads and writes can be concurrent to shared > memory. We are also taking an exclusive lock when writing a new tranche. We probably w

Re: Use LW_SHARED in WakeupWalSummarizer() for WALSummarizerLock lock

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 03:38:13PM -0700, Masahiko Sawada wrote: > While reading walsummarizer.c code, I noticed that in > WakeupWalSummarizer() we acquire the WALSummarizerLock lock in > LW_EXCLUSIVE mode despite only reading > WalSummarizerCtl->summarizer_pgprocno. The attached patch uses > LW_SH

Re: Explicitly enable meson features in CI

2025-08-25 Thread Jacob Champion
On Mon, Jul 28, 2025 at 11:20 AM Jacob Champion wrote: > Sounds good. I will take another look at this with a committer hat and > push Sometime Soon. Getting back to this -- for v7 I've rebased over the FASTLINK commit. I've squashed 0002 and 0003 together, too -- I think it's easier to see the n

Re: index prefetching

2025-08-25 Thread Andres Freund
Hi, On 2025-08-25 15:00:39 +0200, Tomas Vondra wrote: > Thanks. Based on the testing so far, the patch seems to be a substantial > improvement. What's needed to make this prototype committable? Mainly some testing infrastructure that can trigger this kind of stream. The logic is too finnicky for

Re: Per backend relation statistics tracking

2025-08-25 Thread Michael Paquier
On Mon, Aug 25, 2025 at 05:51:38PM -0500, Sami Imseih wrote: > I have not gone through them in detail yet, but +1 on adding backend activity > stats. This provides another level of drill down to spot anomalous sessions or > different patterns across applications. I also think we will want more than

Re: Changing the state of data checksums in a running cluster

2025-08-25 Thread Tomas Vondra
On 8/25/25 20:32, Daniel Gustafsson wrote: >> On 20 Aug 2025, at 16:37, Tomas Vondra wrote: > >> This happens quite regularly, it's not hard to hit. But I've only seen >> it to happen on a FSM, and only right after immediate shutdown. I don't >> think that's quite expected. >> >> I believe the bu

Re: Per backend relation statistics tracking

2025-08-25 Thread Sami Imseih
Thanks for the patches. I have not gone through them in detail yet, but +1 on adding backend activity stats. This provides another level of drill down to spot anomalous sessions or different patterns across applications. I also think we will want more than just relation stats. For example, columns

END is not a reserved word

2025-08-25 Thread Vicky Vergara
Hello all A simplification of the problem I found: Here is a function (taken from the postgres documentation) ``` CREATE FUNCTION add(integer, integer) RETURNS integer AS 'select $1 + $2;' LANGUAGE SQL IMMUTABLE RETURNS NULL ON NULL INPUT; ``` "Accidentally" did the following: ``

Re: Support getrandom() for pg_strong_random() source

2025-08-25 Thread Jacob Champion
On Mon, Aug 25, 2025 at 3:22 PM Masahiko Sawada wrote: > > For instance, we could > introduce a GUC parameter that lets users specify their preferred > random number source. Or the server can automatically select it based > on the kernel's FIPS mode (i.e., checking > /proc/sys/crypto/fips_enabled)

Use LW_SHARED in WakeupWalSummarizer() for WALSummarizerLock lock

2025-08-25 Thread Masahiko Sawada
Hi all, While reading walsummarizer.c code, I noticed that in WakeupWalSummarizer() we acquire the WALSummarizerLock lock in LW_EXCLUSIVE mode despite only reading WalSummarizerCtl->summarizer_pgprocno. The attached patch uses LW_SHARED mode instead. Feedback is very welcome. Regards, -- Masahi

Re: Support getrandom() for pg_strong_random() source

2025-08-25 Thread Masahiko Sawada
On Mon, Aug 25, 2025 at 1:07 PM Jacob Champion wrote: > > On Mon, Aug 25, 2025 at 11:30 AM Masahiko Sawada > wrote: > > > Gathering a couple of considerations from upthread: > > > - FIPS behavior > > > > Do you mean random numbers generated by getrandom() complaints FIPS? > > Based on my researc

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Sami Imseih
> On Mon, Aug 25, 2025 at 04:37:21PM -0500, Sami Imseih wrote: > > When we lookup from shared array only, we need to take a shared lock > > every lookup. Acquiring that lock is what I am trying to avoid. You > > are saying it's not worth optimizing that part, correct? > > Why do we need a shared lo

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 04:37:21PM -0500, Sami Imseih wrote: > When we lookup from shared array only, we need to take a shared lock > every lookup. Acquiring that lock is what I am trying to avoid. You > are saying it's not worth optimizing that part, correct? Why do we need a shared lock here? I

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 04:28:09PM -0500, Sami Imseih wrote: > Here it is. Looks reasonable to me. I'll leave this one around for a week or two to give others a chance to comment. -- nathan

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Sami Imseih
> On Fri, Aug 22, 2025 at 03:01:53PM -0500, Sami Imseih wrote: > > I kept the local array to serve consecutive reads and to avoid having to > > take a shared lock on shared memory every time GetLWTrancheName is > > called. A new LWLock to protect this array is required. > > I'm not seeing why we ne

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Sami Imseih
Here it is. I was hoping we can even get rid of NamedLWLockTrancheRequests altogether, but that's not going to be possible because RequestNamedLWLockTranche happens earlier than CreateLWLocks, so NamedLWLockTrancheRequests is essentially tracking the requested lwlocks until we get a chance to crea

Re: Test instability when pg_dump orders by OID

2025-08-25 Thread Andres Freund
Hi, On 2025-08-24 09:08:16 -0700, Noah Misch wrote: > On Sun, Aug 24, 2025 at 11:50:01AM -0400, Tom Lane wrote: > > Andres Freund writes: > > > I wonder if it's worth adding support to CI to perform the cross-version > > > upgrade test. It'd be pretty easy to install all pgdg apt postgres > > >

Re: List TAP test files in makefiles

2025-08-25 Thread Andres Freund
Hi, On 2025-08-23 12:11:51 -0400, Tom Lane wrote: > Andres Freund writes: > > FWIW, I find the autoconf/make test run experience completely unusable. It > > literally is made me embark on getting away from it. I don't understand how > > people stand it. > > Interesting perspective, because from w

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Nathan Bossart
On Fri, Aug 22, 2025 at 03:01:53PM -0500, Sami Imseih wrote: > I kept the local array to serve consecutive reads and to avoid having to > take a shared lock on shared memory every time GetLWTrancheName is > called. A new LWLock to protect this array is required. I'm not seeing why we need this cac

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Tom Lane
BTW, a couple of thoughts came to me after considering this issue awhile longer: 1. The really fundamental problem is that we don't update SpecialJoinInfo's semi_operators/semi_rhs_exprs after discovering that join-level comparisons of a particular value are unnecessary. We will then not emit the

Re: Report reorder buffer size

2025-08-25 Thread Masahiko Sawada
Hi, On Wed, Aug 13, 2025 at 5:40 AM Ashutosh Bapat wrote: > > Hi All, > The question of how to tune logical_decoding_work_mem has come up a > few times in the context of customer cases. If it is too low then the > changes are spilled to disk, slowing down replication and consuming > disk space. I

Re: Improve LWLock tranche name visibility across backends

2025-08-25 Thread Nathan Bossart
On Fri, Aug 22, 2025 at 05:33:55PM -0400, Tom Lane wrote: > Sami Imseih writes: >> One point I did not make earlier is that the tranche name lengths will >> need to be as long as we allow in dsm_registry.c. > >> #define DSMR_NAME_LEN 128 > > Huh. Why is that different from NAMEDATALEN in the fi

Re: Support getrandom() for pg_strong_random() source

2025-08-25 Thread Jacob Champion
On Mon, Aug 25, 2025 at 11:30 AM Masahiko Sawada wrote: > > Gathering a couple of considerations from upthread: > > - FIPS behavior > > Do you mean random numbers generated by getrandom() complaints FIPS? > Based on my research, there doesn't appear to be any explicit > statement indicating that L

Re: making EXPLAIN extensible

2025-08-25 Thread Robert Haas
On Tue, Mar 4, 2025 at 10:26 AM Andrei Lepikhov wrote: > I wouldn't say there is a thread in the mailing list. I mentioned this > direction of extensibility multiple times (for example, [1,2]) with no > reaction. However, letting extensions show data in explan gives this > idea additional impulse.

Re: RFC: extensible planner state

2025-08-25 Thread Robert Haas
On Wed, Aug 20, 2025 at 3:13 PM Robert Haas wrote: > Here's v2. 0001 is what you saw before with an attempt to fix the > memory context handling. 0002 removes join_search_private. All I've > tested is that the tests pass. Here's v3 with a few more patches. I'm now fairly confident I have the basi

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Sami Imseih
> > Another approach is to just change GetNamedLWLockTranche to use > > NamedLWLockTrancheArray since that is already copied in EXEC_BACKEND, and > > allow GetNamedLWLockTranche to continue to be used outside of startup. In > > this case, we will need to add num_lwlocks field to > > NamedLWLockTran

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-25 Thread Masahiko Sawada
On Mon, Aug 25, 2025 at 6:02 AM Shlok Kyal wrote: > > > Hi Sawada-san, > > I reviewed the latest patch and have following comments: > > 1. In commit message, word 'slot' is missing: > When the first logical replication is created, the system > automatically increases the effective WAL level to mai

Re: index prefetching

2025-08-25 Thread Peter Geoghegan
On Mon, Aug 25, 2025 at 2:33 PM Tomas Vondra wrote: > Right. I might have expressed it more clearly, but this is what I meant > when I said priorbatch is not causing this. Cool. > As for priorbatch, I'd still like to know where does the overhead come > from. I mean, what's the expensive part of

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Tom Lane
I wrote: > Yeah. I think this is an oversight in create_unique_paths(): it's > building an ORDER BY list without consideration for the possibility > that some of the entries are known to be constant. In fact, because > make_pathkeys_for_sortclauses will get rid of redundancies, this > example act

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Ranier Vilela
Em seg., 25 de ago. de 2025 às 16:11, Nathan Bossart < nathandboss...@gmail.com> escreveu: > On Mon, Aug 25, 2025 at 02:03:11PM -0300, Ranier Vilela wrote: > > If you don't mind I think that have one more source. > > Good catch. Committed. > Thanks. best regards, Ranier Vilela

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 02:03:11PM -0300, Ranier Vilela wrote: > If you don't mind I think that have one more source. Good catch. Committed. -- nathan

Re: Eagerly evict bulkwrite strategy ring

2025-08-25 Thread Melanie Plageman
On Mon, Aug 18, 2025 at 2:54 AM Kirill Reshke wrote: > Hi! Thanks for the review! > 1) In EvictStrategyRing we find io context for strategy: > > > + io_context = IOContextForStrategy(strategy); > > but the caller of this function (GetVictimBuffer) already has one. > Should we reuse its context,

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 01:44:22PM -0500, Sami Imseih wrote: > Another approach is to just change GetNamedLWLockTranche to use > NamedLWLockTrancheArray since that is already copied in EXEC_BACKEND, and > allow GetNamedLWLockTranche to continue to be used outside of startup. In > this case, we will

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Sami Imseih
> Attached is a repro patch. You will need to set > shared_preload_libraries = 'pg_stat_statements' > as well. I forgot to mention. Once patched, you can run a simple select statement from a new connection to force the crash. This is because GetNamedLWLockTranche will be called at pgss_ExecutorEnd

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Sami Imseih
> On Mon, Aug 25, 2025 at 12:58:08PM -0500, Sami Imseih wrote: > >> If this fails, why doesn't the call in pgss_shmem_startup() also fail? Was > >> pg_stat_statements not loaded via shared_preload_libraries? > > > > because the array is valid in postmaster, but it's not for a normal backend, > > s

Re: index prefetching

2025-08-25 Thread Tomas Vondra
On 8/25/25 19:57, Peter Geoghegan wrote: > On Mon, Aug 25, 2025 at 10:18 AM Tomas Vondra wrote: >> Almost all regressions (at least the top ones) now look like this, i.e. >> distance collapses to ~2.0, which essentially disables prefetching. > > Good to know. > >> But I no longer think it's c

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 12:58:08PM -0500, Sami Imseih wrote: >> If this fails, why doesn't the call in pgss_shmem_startup() also fail? Was >> pg_stat_statements not loaded via shared_preload_libraries? > > because the array is valid in postmaster, but it's not for a normal backend, > since it's n

Re: Support getrandom() for pg_strong_random() source

2025-08-25 Thread Masahiko Sawada
On Mon, Aug 18, 2025 at 8:38 AM Jacob Champion wrote: > > On Thu, Aug 14, 2025 at 3:16 PM Masahiko Sawada wrote: > > > > On Fri, Aug 8, 2025 at 3:37 PM Jacob Champion > > wrote: > > > > > So, my next question: is getrandom() always preferable to /dev/urandom? > > > > I believe so. While /dev/ura

Re: Adding REPACK [concurrently]

2025-08-25 Thread Mihail Nikalayeu
Antonin Houska : > I think the problem is that HeapTupleSatisfiesSelf() uses > TransactionIdIsInProgress() instead of checking the snapshot: Yes, some issues might be possible for SnapshotSelf. Possible solution is to override TransactionIdIsCurrentTransactionId to true (like you did with nParalle

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Sami Imseih
> On Fri, Aug 22, 2025 at 02:21:55PM -0500, Sami Imseih wrote: > > While working on [0], I observed that $SUBJECT. I encountered this issue > > while building test cases for [0], and in which GetNamedLWLockTranche is > > called outside of startup. > > > > [...] > > > > I repro'd this on a Windows m

Re: index prefetching

2025-08-25 Thread Peter Geoghegan
On Mon, Aug 25, 2025 at 10:18 AM Tomas Vondra wrote: > Almost all regressions (at least the top ones) now look like this, i.e. > distance collapses to ~2.0, which essentially disables prefetching. Good to know. > But I no longer think it's caused by the "priorbatch" optimization, > which delays

Re: index prefetching

2025-08-25 Thread Tomas Vondra
On 8/25/25 17:43, Thomas Munro wrote: > On Tue, Aug 26, 2025 at 2:18 AM Tomas Vondra wrote: >> Of course, this can happen even with other hit ratios, there's nothing >> special about 50%. > > Right, that's what this patch was attacking directly, basically only > giving up when misses are so spars

Re: Adding REPACK [concurrently]

2025-08-25 Thread Antonin Houska
Mihail Nikalayeu wrote: > Hi, Antonin > > > How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using > > a > > snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ? > > Yes, TransactionIdDidCommit. I think the problem is that HeapTupleSatisfiesSelf() u

Re: index prefetching

2025-08-25 Thread Tomas Vondra
On 8/25/25 16:18, Tomas Vondra wrote: > ... > > But with more hits, the hit/miss ratio simply determines the "stable" > distance. Let's say there's 80% hits, so 4 hits to 1 miss. Then the > stable distance is ~4, because we get a miss, double to 8, and then 4 > hits, so the distance drops back

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Ranier Vilela
Em seg., 25 de ago. de 2025 às 13:10, Nathan Bossart < nathandboss...@gmail.com> escreveu: > On Mon, Aug 25, 2025 at 12:13:27PM -0300, Fabrízio de Royes Mello wrote: > > On Mon, Aug 25, 2025 at 11:57 AM Nathan Bossart < > nathandboss...@gmail.com> wrote: > >> I noticed a couple more opportunities

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Sergey Soloviev
25.08.2025 16:28, Richard Guo пишет: On Sat, Aug 23, 2025 at 12:27 AM Sergey Soloviev wrote: I would like write a test in 'join.sql', but for now it requires patches to easily reproduce the bug. I appreciate it if someone could find an easier way to reproduce the bug and write a simple test.

Re: Adding REPACK [concurrently]

2025-08-25 Thread Antonin Houska
Robert Treat wrote: > On Mon, Aug 25, 2025 at 10:15 AM Mihail Nikalayeu > wrote: > > 1) we have a whole initial table snapshot with all xmin copied from > > the original table. All such xmin are committed. > > 2) appling transaction sees ALL the self-alive (no xmax) tuple in it > > because its x

Re: GetNamedLWLockTranche crashes on Windows in normal backend

2025-08-25 Thread Nathan Bossart
On Fri, Aug 22, 2025 at 02:21:55PM -0500, Sami Imseih wrote: > While working on [0], I observed that $SUBJECT. I encountered this issue > while building test cases for [0], and in which GetNamedLWLockTranche is > called outside of startup. > > [...] > > I repro'd this on a Windows machine, but on

Re: Adding REPACK [concurrently]

2025-08-25 Thread Robert Treat
On Mon, Aug 25, 2025 at 10:15 AM Mihail Nikalayeu wrote: > 1) we have a whole initial table snapshot with all xmin copied from > the original table. All such xmin are committed. > 2) appling transaction sees ALL the self-alive (no xmax) tuple in it > because its xmin\xmax is committed and Snapshot

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Tom Lane
Richard Guo writes: > The proposed patch can fix this error. However, I'm wondering if we > could address it from the unique-ification side instead. If a Var > we're trying to unique-ify is known to be equal to a constant, then we > shouldn't need to unique-ify that Var Yeah. I think this is a

Re: Adding REPACK [concurrently]

2025-08-25 Thread Mihail Nikalayeu
Hi, Antonin > How does HeapTupleSatisfiesSelf() recognize the status of any XID w/o using a > snapshot? Do you mean by checking the commit log (TransactionIdDidCommit) ? Yes, TransactionIdDidCommit. Another option is just invent a new snapshot type - SnapshotBelieveEverythingCommitted - for that

Re: vacuumdb --missing-stats-only and permission issue

2025-08-25 Thread Nathan Bossart
On Tue, Aug 26, 2025 at 12:24:27AM +0900, Yugo Nagata wrote: > Thank you for the clarification. I understand your points now, > so I'll withdraw my proposal. Okay. I'll plan on committing the documentation update in the next 24-48 hours, provided no additional feedback materializes. -- nathan

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Sergey Soloviev
25.08.2025 16:59, Andrei Lepikhov: On 25/8/2025 15:28, Richard Guo wrote: On Sat, Aug 23, 2025 at 12:27 AM Sergey Soloviev wrote: I would like write a test in 'join.sql', but for now it requires patches to easily reproduce the bug. I appreciate it if someone could find an easier way to repr

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 12:13:27PM -0300, Fabrízio de Royes Mello wrote: > On Mon, Aug 25, 2025 at 11:57 AM Nathan Bossart > wrote: >> I noticed a couple more opportunities to use the PqMsg_* macros. > > Nice. LGTM! Thanks, committed. -- nathan

Re: Adding REPACK [concurrently]

2025-08-25 Thread Robert Treat
On Sat, Aug 23, 2025 at 10:23 AM Álvaro Herrera wrote: > On 2025-08-23, Michael Banck wrote: > > On Fri, Aug 22, 2025 at 05:32:34PM -0300, Euler Taveira wrote: > > >> I don't think we need to keep vacuumdb. Packagers can keep a symlink > >> (vacuumdb) > >> to pg_repackdb. We can add a similar war

Re: Memory leak of SMgrRelation object on standby

2025-08-25 Thread Jingtang Zhang
Hi~ > Agree with that. Maybe we can call smgrdestroyall in startup process when > replaying CHECKPOINT records, just like bgwriter/checkpointer, which free > all smgr objects after any checkpoint. That seems reasonable, in that case a startup process would behave just the same as bgwriter or chec

Re: index prefetching

2025-08-25 Thread Thomas Munro
On Tue, Aug 26, 2025 at 2:18 AM Tomas Vondra wrote: > Of course, this can happen even with other hit ratios, there's nothing > special about 50%. Right, that's what this patch was attacking directly, basically only giving up when misses are so sparse we can't do anything about it for an ordered s

Re: Adding REPACK [concurrently]

2025-08-25 Thread Antonin Houska
Mihail Nikalayeu wrote: > > Not sure I understand in all details, but I don't think SnapshotSelf is the > > correct snapshot. Note that HeapTupleSatisfiesSelf() does not use its > > 'snapshot' argument at all. Instead, it considers the set of running > > transactions as it is at the time the func

Re: Problem in 'ORDER BY' of a column using a created collation?

2025-08-25 Thread jian he
On Mon, Aug 25, 2025 at 3:52 PM Nishant Sharma wrote: > > > Experiment 1:- > SQL File : PG_Exp_1.sql > > Actual Output : PG_Exp_1.out > > Created COLLATION : CREATE COLLATION test_coll ( > provider = icu, locale = 'ja-u-kr-latn-digit'); > > Experiment 2:- > SQL File : PG_Exp_2.sql > > Actual Outpu

Re: Changing gssencmode default in Psycopg

2025-08-25 Thread Jacob Champion
On Sat, Aug 23, 2025 at 9:16 AM Andres Freund wrote: > I don't know what the right solution is, but it's really not good that > something as rarely used as gss encryption causes crashes and performance > issues for everyone. This seems as good a time as any to mention that I'd like to eventually

Re: vacuumdb --missing-stats-only and permission issue

2025-08-25 Thread Yugo Nagata
On Mon, 25 Aug 2025 09:29:15 -0500 Nathan Bossart wrote: > On Mon, Aug 25, 2025 at 10:30:32AM +0900, Yugo Nagata wrote: > > The documentation fix looks good to me. However, it’s not very > > user-friendly that, > > when the user lacks the required privileges, an error from the internal > > quer

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Fabrízio de Royes Mello
On Mon, Aug 25, 2025 at 11:57 AM Nathan Bossart wrote: > > I noticed a couple more opportunities to use the PqMsg_* macros. > Nice. LGTM! -- Fabrízio de Royes Mello

Re: use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Jacob Champion
On Mon, Aug 25, 2025 at 7:57 AM Nathan Bossart wrote: > > I noticed a couple more opportunities to use the PqMsg_* macros. LGTM! --Jacob

Re: VM corruption on standby

2025-08-25 Thread Nathan Bossart
[RMT hat] On Thu, Aug 21, 2025 at 06:42:48PM -0400, Tom Lane wrote: > Alexander Korotkov writes: >> On Tue, Aug 19, 2025 at 10:50 PM Tom Lane wrote: >>> Therefore, I vote for reverting bc22dc0e0. Hopefully only >>> temporarily, but it's too late to figure out another way for v18, >>> and I don'

use PqMsg macros in fe-protocol3.c

2025-08-25 Thread Nathan Bossart
I noticed a couple more opportunities to use the PqMsg_* macros. -- nathan >From 7cdd91579734de76fcd52cabdb49671ee309ddf6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 25 Aug 2025 09:53:48 -0500 Subject: [PATCH v1 1/1] Use PqMsg_* macros in fe-protocol3.c. --- src/include/libpq/prot

Re: vacuumdb --missing-stats-only and permission issue

2025-08-25 Thread Nathan Bossart
On Mon, Aug 25, 2025 at 10:30:32AM +0900, Yugo Nagata wrote: > The documentation fix looks good to me. However, it’s not very user-friendly > that, > when the user lacks the required privileges, an error from the internal query > is > raised. Instead, how about checking whether the user has the n

Re: index prefetching

2025-08-25 Thread Tomas Vondra
On 8/20/25 00:27, Peter Geoghegan wrote: > On Tue, Aug 19, 2025 at 2:22 PM Peter Geoghegan wrote: >> That definitely seems like a problem. I think that you're saying that >> this problem happens because we have extra buffer hits earlier on, >> which is enough to completely change the ramp-up behav

Re: Adding REPACK [concurrently]

2025-08-25 Thread Mihail Nikalayeu
Hi, Antonin! > I assume you are concerned with the patch part 0005 of the v12 patch > ("Preserve visibility information of the concurrent data changes."), aren't > you? Yes, of course. I got an idea while trying to find a way to optimize it. > Not sure I understand in all details, but I don't th

Re: [BUG] Remove self joins causes 'variable not found in subplan target lists' error

2025-08-25 Thread Andrei Lepikhov
On 25/8/2025 15:28, Richard Guo wrote: On Sat, Aug 23, 2025 at 12:27 AM Sergey Soloviev wrote: I would like write a test in 'join.sql', but for now it requires patches to easily reproduce the bug. I appreciate it if someone could find an easier way to reproduce the bug and write a simple test.

Re: Making WAL archiving faster — multi-file support and async ideas

2025-08-25 Thread Greg Sabino Mullane
On Mon, Aug 25, 2025 at 4:31 AM Alyona Vinter wrote: > ... could attempt a restore using the incomplete archive. While we hope > this would cause a clear error during recovery, there is a risk that > partial application of non-sequential segments might lead to silent > corruption or other unfores

Re: [PATCH] Generate random dates/times in a specified range

2025-08-25 Thread Greg Sabino Mullane
Your v3 did not get attached to the previous email.

Re: [BUG?] check_exclusion_or_unique_constraint false negative

2025-08-25 Thread Mihail Nikalayeu
Amit Kapila : > What if the new insert happens in a page prior to the current page? I > mean that the scan won't encounter the page where Insert happens. Hmm Yes - if the TID lands to the page left of the current position, we’ll miss it as well. A lock‑based solution (version in the v10) wou

Re: Fix ALTER TABLE DROP EXPRESSION with inheritance hierarchy

2025-08-25 Thread Kirill Reshke
Looks like no CF entry for this thread. CF entry [0] created. [0] https://commitfest.postgresql.org/patch/5992/ -- Best regards, Kirill Reshke

  1   2   >