Re: VM corruption on standby

2025-08-09 Thread Andrey Borodin
> On 9 Aug 2025, at 23:54, Aleksander Alekseev wrote: > > IMHO: logging the changes first, then allowing to evict the page. VM and BufferManager code does not allow flush of a buffer until changes are logged. The problem is that our crash-exiting system destroys locks that protect buffer fr

Re: VM corruption on standby

2025-08-09 Thread Kirill Reshke
On Sun, 10 Aug 2025 at 01:55, Aleksander Alekseev wrote: > For this reason we have PageHeaderData.pd_lsn for instance - to make sure > pages are evicted only *after* the record that changed it is written > to disk (because WAL records can't be applied to pages from the > future). We don't bump th

Re: A few patches to clarify snapshot management

2025-08-09 Thread Noah Misch
On Tue, Jan 07, 2025 at 11:55:00AM +0200, Heikki Linnakangas wrote: > On 07/01/2025 00:00, Andres Freund wrote: > > On 2024-12-20 19:31:01 +0200, Heikki Linnakangas wrote: > > > While playing around some more with this, I noticed that this code in > > > GetTransactionSnapshot() is never reached, an

Re: VM corruption on standby

2025-08-09 Thread Aleksander Alekseev
Hi Andrey, > 0. checkpointer is going to flush a heap buffer but waits on content lock > 1. client is resetting PD_ALL_VISIBLE from page > 2. postmaster is killed and command client to go down > 3. client calls LWLockReleaseAll() at ProcKill() (?) > 4. checkpointer flushes buffer with reset PG_ALL

Re: VM corruption on standby

2025-08-09 Thread Kirill Reshke
On Thu, 7 Aug 2025 at 21:36, Aleksander Alekseev wrote: > Perhaps there was a good > reason to update the VM *before* creating WAL records I'm unaware of. Looks like 503c730 intentionally does it this way; however, I have not yet fully understood the reasoning behind it. -- Best regards, Kir

Re: Making type Datum be 8 bytes everywhere

2025-08-09 Thread Andrew Dunstan
On 2025-08-09 Sa 7:45 AM, Joe Conway wrote: On 8/8/25 21:14, Tom Lane wrote: I have just realized that this proposal has a rather nasty defect. Per the following comment in spgist_private.h:   * If the prefix datum is of a pass-by-value type, it is stored in its   * Datum representation, that

Re: VM corruption on standby

2025-08-09 Thread Andrey Borodin
> On 9 Aug 2025, at 18:28, Andrey Borodin wrote: > > Also I investigated that in a moment of kill -9 checkpointer flushes heap > page to disk despite content lock. I haven't found who released content lock > though. I've written this message and understood: its LWLockReleaseAll(). 0. check

plpython: Remove support for major version conflict detection

2025-08-09 Thread Peter Eisentraut
This essentially reverts commit 866566a690b, which installed safeguards against loading plpython2 and plpython3 into the same process. We don't support plpython2 anymore, so this is obsolete. The Python and PL/Python initialization now happens again in _PG_init() rather than the first time a

Re: Making type Datum be 8 bytes everywhere

2025-08-09 Thread Tom Lane
Joe Conway writes: > On 8/8/25 21:14, Tom Lane wrote: >> I have just realized that this proposal has a rather nasty defect. >> ... >> On the positive side, even if there are any SP-GiST opclasses that >> are at risk, the population of installations using them on 32-bit >> installs has got to be pr

Re: VM corruption on standby

2025-08-09 Thread Andrey Borodin
> On 7 Aug 2025, at 19:36, Aleksander Alekseev wrote: > > Maybe I misunderstood the intent of the test. You understood precisely my intent of writing the test. But it fail not due to a bug I anticipated! So far I noticed that if I move injection point before PageClearAllVisible(BufferGetPa

Re: Prolonged truncation phase during vacuum on toast table with repeated interruptions by lock waiters and a proposed POC patch

2025-08-09 Thread Shayon Mukherjee
On Thu, May 8, 2025 at 4:58 PM Shayon Mukherjee wrote: > +1 for this. I'd even be happy to even just start showing the attempts as > part of INFO or VERBOSE on conflicts & suspensions, without keeping track, > but it is useful statistical data. > > Hi hackers, Following up on the recent discuss

Re: Adding REPACK [concurrently]

2025-08-09 Thread Alvaro Herrera
On 2025-Aug-09, Mihail Nikalayeu wrote: > Hello! > > One more thing - I think build_new_indexes and > index_concurrently_create_copy are very close in semantics, so it > might be a good idea to refactor them a bit. > > I’m still concerned about MVCC-related issues. For multiple > applications, t

Re: Implement waiting for wal lsn replay: reloaded

2025-08-09 Thread Xuneng Zhou
Hi Alexander! > > In the meantime, I have > > a quick question: is WAIT FOR REPLY intended mainly for user-defined > > functions, or can internal code invoke it as well? > > Currently, WaitForLSNReplay() is assumed to only be called from > backend, as corresponding shmem is allocated only per-back

Re: Adding REPACK [concurrently]

2025-08-09 Thread Mihail Nikalayeu
Hello! One more thing - I think build_new_indexes and index_concurrently_create_copy are very close in semantics, so it might be a good idea to refactor them a bit. I’m still concerned about MVCC-related issues. For multiple applications, this is a dealbreaker, because in some cases correctness i

Re: Datum as struct

2025-08-09 Thread Tom Lane
Peter Eisentraut writes: > On 08.08.25 22:30, Andres Freund wrote: >> One thing that would be an interesting follow-up would be to make the struct >> not just carry the datum, but also the type of the field, to be filled in by >> tuple deforming and the *GetDatum() functions. Then we could assert

Re: Datum as struct

2025-08-09 Thread Peter Eisentraut
On 08.08.25 22:30, Andres Freund wrote: One thing that would be an interesting follow-up would be to make the struct not just carry the datum, but also the type of the field, to be filled in by tuple deforming and the *GetDatum() functions. Then we could assert that the correct DatumGet*() functi

Re: Making type Datum be 8 bytes everywhere

2025-08-09 Thread Joe Conway
On 8/8/25 21:14, Tom Lane wrote: I have just realized that this proposal has a rather nasty defect. Per the following comment in spgist_private.h: * If the prefix datum is of a pass-by-value type, it is stored in its * Datum representation, that is its on-disk representation is of length *

Re: Implement waiting for wal lsn replay: reloaded

2025-08-09 Thread Xuneng Zhou
Hi, > On Thu, Aug 7, 2025 at 6:01 PM Xuneng Zhou wrote: > > Thanks for working on this. > > > > I’ve just come across this thread and haven’t had a chance to dig into > > the patch yet, but I’m keen to review it soon. > > Great. Thank you for your attention to this patch. I appreciate your > in

Re: Datum as struct

2025-08-09 Thread Peter Eisentraut
On 08.08.25 22:55, Tom Lane wrote: FWIW, I don't love the "DummyDatum" name either. Maybe InvalidDatum? I specifically did not use InvalidDatum because, for example, the result of Int32GetDatum(0) is by no means "invalid". Maybe something like NullDatum or VoidDatum? A few related thought

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-08-09 Thread Michael Paquier
On Fri, Aug 08, 2025 at 09:09:20AM -0700, Nikhil Kumar Veldanda wrote: > I have a question regarding TOAST pointer handling. > > As I understand, in the current design, each attribute in a HeapTuple > can have its own TOAST pointer, and TOAST pointers are possible only > for top-level attributes.

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-08-09 Thread Michael Paquier
On Fri, Aug 08, 2025 at 03:32:16PM -0400, Tom Lane wrote: > I found some time to look at the v4 patchset, and have a bunch of > comments of different sizes. Thanks for the comments. I have replied to some of the items here: https://www.postgresql.org/message-id/aJbygEBqJgmLS0wF%40paquier.xyz Wil

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-08-09 Thread Michael Paquier
On Fri, Aug 08, 2025 at 05:02:46PM -0400, Andres Freund wrote: > On 2025-08-08 15:32:16 -0400, Tom Lane wrote: >> I found some time to look at the v4 patchset, and have a bunch of >> comments of different sizes. Thanks for the input. This patch set exists as a result of a discussion between you a