Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-13 Thread shveta malik
On Thu, Aug 14, 2025 at 7:28 AM Ajin Cherian wrote: > > > Patch v6 attached. > Thanks Ajin. Please find my comments: 1) SyncReplicationSlots: + remote_slots = fetch_or_refresh_remote_slots(wrconn, NIL); + + /* Retry until all slots are sync ready atleast */ + for (;;) + { + bool some_slot_update

memory leak in logical WAL sender with pgoutput's cachectx

2025-08-13 Thread 赵宇鹏(宇彭)
Hi all, We recently ran into a memory leak in a production logical-replication WAL-sender process. A simplified reproduction script is attached. If you run the script and then call MemoryContextStats(TopMemoryContext). you will see something like: "logical replication cache context: 5620

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-08-13 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 28 Jul 2025 22:19:36 -0700, Masahiko Sawada wrote: > The fields in 1 are mostly static fields, and the fields in 2 and 3 > are likely to be accessed in hot functions during COPY FROM. Would it > be a g

Re: VM corruption on standby

2025-08-13 Thread Kirill Reshke
On Thu, 14 Aug 2025 at 10:41, Kirill Reshke wrote: > o I am trying to reproduce is following: > > 1) Some process p1 locks some buffer (name it buf1), enters CRIT > section, calls MarkBufferDirty and hangs inside XLogInsert on CondVar > in (GetXLogBuffer -> AdvanceXLInsertBuffer). > 2) CHECKPOINT

Re: Issue with logical replication slot during switchover

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 9:29 PM Fabrice Chapuis wrote: > > Thanks for sharing this. In fact, I agree, introducing an allow_overwrite > slot property makes seems cleaner than a GUC for this specific use case. > > a) At first, an extension of pg_create_logical_replication_slot() could be > propos

Re: VM corruption on standby

2025-08-13 Thread Kirill Reshke
On Wed, 13 Aug 2025 at 16:15, I wrote: > I did not find any doc or other piece of information indicating > whether WaitEventSetWait and critical sections are allowed. But I do > thing this is bad, because we do not process interruptions during > critical sections, so it is unclear to me why we shou

Re: Add log_autovacuum_{vacuum|analyze}_min_duration

2025-08-13 Thread Shinya Kato
Hi, On Wed, Aug 13, 2025 at 5:44 PM kasaharatt wrote: > > Approach 2: > > - log_autovacuum_min_duration: Changed behavior, which controls only > > autovacuum logging. > > - log_autoanalyze_min_duration: New parameter, which controls > > autoanalyze logging. > >

Re: Excessive LOG messages from replication slot sync worker

2025-08-13 Thread Fujii Masao
On Wed, Aug 13, 2025 at 1:00 PM Amit Kapila wrote: > > On Tue, Aug 12, 2025 at 4:28 PM Fujii Masao wrote: > > > > I'm not sure these messages are useful for end users, and LOG might not be > > the right level. They seem more like debug information, so should we > > consider > > changing them to

Re: Excessive LOG messages from replication slot sync worker

2025-08-13 Thread Fujii Masao
On Wed, Aug 13, 2025 at 11:54 AM Amit Kapila wrote: > > On Tue, Aug 12, 2025 at 6:32 PM Fujii Masao wrote: > > > > Should we also update LOGICAL_REWRITE_FORMAT to use %08X? > > > > That said, the number of places to change is growing, and we're drifting > > away from the original issue So I s

Re: Make pgoutput documentation easier to find

2025-08-13 Thread Chao Li
I applied the batch and built the html docs. I tested that all hyperlinks worked for me. There are a few nit comments about wording: diff --git a/doc/src/sgml/logicaldecoding.sgml b/doc/src/sgml/logicaldecoding.sgml index 593f784b69d..4516650983d 100644 --- a/doc/src/sgml/logicaldecoding.sgml +

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 4:15 PM shveta malik wrote: > > On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu) > wrote: > > > > > > Here is the V61 patch set which addressed above comments and the comment by > > Nisha[2]. > > > > Thank You for the patch. I tested the patch, please find a few comm

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed Aug 13, 2025 at 7:50 PM EDT, Peter Geoghegan wrote: > pg@regression:5432 [2476413]=# EXPLAIN (ANALYZE ,costs off, timing off) > SELECT * FROM t WHERE a BETWEEN 16336 AND 49103 ORDER BY a desc; > ┌─┐ > │

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 5:42 PM Amit Kapila wrote: > > > > > 4) > > For the DETAIL part of resume and stop messages, how about these: > > > > The retention duration for information used in conflict detection has > > exceeded the limit of xx. > > The retention duration for information used in confl

Re: ALTER DOMAIN ADD NOT NULL NOT VALID

2025-08-13 Thread jian he
hi. rebase and minor cosmetic change. From b87caa1b2f78d3f1e94e1055219c97e038d2fe61 Mon Sep 17 00:00:00 2001 From: jian he Date: Thu, 14 Aug 2025 10:40:49 +0800 Subject: [PATCH v3 1/1] ALTER DOMAIN ADD NOT NULL NOT VALID We have NOT NULL NO VALID for table constraints, we can make domain have NOT

Re: Improve hash join's handling of tuples with null join keys

2025-08-13 Thread Chao Li
> On Aug 13, 2025, at 17:16, Chao Li wrote: > > I downloaded the patch and tested all join types: inner, left, right, full, > semi and anti. Basically my tests all passed. However, I didn't test any case > of parallel query. > > I have two nit comments: > > 1. In hashjoin.h, line 76-78, the

Re: Speed up COPY FROM text/CSV parsing using SIMD

2025-08-13 Thread KAZAR Ayoub
Following Nazir's findings about 4096 bytes being the performant line length, I did more benchmarks from my side on both TEXT and CSV formats with two different cases of normal data (no special characters) and data with many special characters. Results are con good as expected and similar to previ

Re: Update the LSN format in the comment example

2025-08-13 Thread Japin Li
On Thu, Aug 14, 2025 at 11:13:28AM +0900, Fujii Masao wrote: > On Tue, Aug 12, 2025 at 7:36 PM Fujii Masao wrote: > > > > On Tue, Aug 12, 2025 at 4:45 PM Japin Li wrote: > > > > > > Hi, all > > > > > > Commit 2633dae2e487 standardized LSN formatting with zero-padding. > > > However, > > > a mist

Re: Retail DDL

2025-08-13 Thread Ziga
Hi Andrew, On 24/07/2025 22:26, Andrew Dunstan wrote: Some years ago I gave a talk about $subject, but somehow it dropped off my radar. Now I'm looking at it again. The idea is to have a function (or set of functions) that would allow the user to get the DDL for any database object. Obviously

Re: Update the LSN format in the comment example

2025-08-13 Thread Fujii Masao
On Tue, Aug 12, 2025 at 7:36 PM Fujii Masao wrote: > > On Tue, Aug 12, 2025 at 4:45 PM Japin Li wrote: > > > > Hi, all > > > > Commit 2633dae2e487 standardized LSN formatting with zero-padding. However, > > a mistake was made (by me) in the example within a comment. > > > > This patch provides a

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Japin Li
On Thu, Aug 14, 2025 at 12:04:12PM +1000, Peter Smith wrote: > On Fri, Aug 1, 2025 at 5:43 PM Japin Li wrote: > > > > On Fri, Aug 01, 2025 at 05:18:11PM +1000, Peter Smith wrote: > > > On Wed, Jul 30, 2025 at 9:07 PM Japin Li wrote: > > > > > > > ... > > > > 3. > > > > I've also found that the VC

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Japin Li
On Wed, Aug 13, 2025 at 07:28:34PM +0300, Timur Magomedov wrote: > On Mon, 2025-08-11 at 17:39 +0800, Japin Li wrote: > > > > 1. > > +static struct > > +{ > > +    int transfn_oid;    /* Transition function's funcoid. > > Arrays are > > + * sorted in ascendi

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Peter Smith
On Fri, Aug 1, 2025 at 5:43 PM Japin Li wrote: > > On Fri, Aug 01, 2025 at 05:18:11PM +1000, Peter Smith wrote: > > On Wed, Jul 30, 2025 at 9:07 PM Japin Li wrote: > > > > > ... > > > 3. > > > I've also found that the VCI index is not working. Is this the expected > > > behavior? > > > > > > [loc

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-13 Thread Ajin Cherian
On Wed, Aug 13, 2025 at 2:47 PM shveta malik wrote: > > On Mon, Aug 11, 2025 at 1:37 PM Ajin Cherian wrote: > > > > On Fri, Aug 8, 2025 at 11:22 PM Ashutosh Bapat > > wrote: > > > > > > > > > There's also a minor merge conflict because func.sgml is not split > > > into multiple files. > > > > >

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 8:59 PM Tomas Vondra wrote: > I investigated this from a different angle, by tracing the I/O request > generated. using perf-trace. And the patterns are massively different. I tried a similar approach myself, using a variety of tools. That didn't get me very far. > So, Q1

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Peter Smith
On Thu, Aug 14, 2025 at 2:28 AM Timur Magomedov wrote: > > On Mon, 2025-08-11 at 17:39 +0800, Japin Li wrote: > > > > 1. > > +static struct > > +{ > > +int transfn_oid;/* Transition function's funcoid. > > Arrays are > > + * sorted in ascending order

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Peter Smith
On Mon, Aug 11, 2025 at 7:39 PM Japin Li wrote: > > On Fri, Aug 08, 2025 at 06:08:27PM +1000, Peter Smith wrote: > > Here are the latest v17 patches. > > > > Changes include: > > > > PATCH 0002. > > - Rebase to fix compile error, result of recent master change > > - Bugfix for an unreported EXPLAI

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Peter Smith
On Tue, Aug 12, 2025 at 1:48 PM Japin Li wrote: ... > 1. > PFA the other typos. > Fixed in v18. > 2. > I found it skip vci query context initialization in > vci_intialize_query_context() > if full page writes is disabled, Could you explain why we need full page write > enabled for VCI? > TODO

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/14/25 01:50, Peter Geoghegan wrote: > On Wed Aug 13, 2025 at 5:19 PM EDT, Tomas Vondra wrote: >> I did investigate this, and I don't think there's anything broken in >> read_stream. It happens because ReadStream has a concept of "ungetting" >> a block, which can happen after hitting some I/

Re: index prefetching

2025-08-13 Thread Thomas Munro
On Thu, Aug 14, 2025 at 9:19 AM Tomas Vondra wrote: > I did investigate this, and I don't think there's anything broken in > read_stream. It happens because ReadStream has a concept of "ungetting" > a block, which can happen after hitting some I/O limits. > > In that case we "remember" the last bl

Re: [PATCH] Silence Valgrind about SelectConfigFiles()

2025-08-13 Thread Tom Lane
Aleksander Alekseev writes: >> You didn't say what the test condition was, but from the patch >> I suppose it's a case where SelectConfigFiles is erroring out? > Steps to reproduce: > ... Huh ... don't quite see where in that recipe we'd reach a SelectConfigFiles error exit. > It's not exciting

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 7:51 PM Peter Geoghegan wrote: > Apparently random I/O is twice as fast as sequential I/O in descending order! > In > fact, this test case creates the appearance of random I/O being at least > slightly faster than sequential I/O for pages read in _ascending_ order! > > Obv

Re: `pg_ctl init` crashes when run concurrently; semget(2) suspected

2025-08-13 Thread Thomas Munro
On Wed, Aug 13, 2025 at 7:38 PM Thomas Munro wrote: > Here's a new attempt at that. It picks futexes automatically, unless > you export MACOSX_DEPLOYMENT_TARGET=14.3 or lower, and then it picks > sysv as before. I compared my hand-rolled atomics logic against FreeBSD's user space semaphores[1].

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-08-13 Thread Michael Paquier
On Wed, Aug 13, 2025 at 03:06:16PM -0500, Jim Nasby wrote: > On Fri, Aug 8, 2025 at 4:03 PM Andres Freund wrote: >> I'm not convinced that the global counter, be it a 32 or a 64 bit, approach >> has merit in general, and I'm rather sure it's the wrong thing for toast >> values. There's no straight

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed Aug 13, 2025 at 5:19 PM EDT, Tomas Vondra wrote: > I did investigate this, and I don't think there's anything broken in > read_stream. It happens because ReadStream has a concept of "ungetting" > a block, which can happen after hitting some I/O limits. > > In that case we "remember" the last

Re: [PATCH] Silence Valgrind about SelectConfigFiles()

2025-08-13 Thread Aleksander Alekseev
Hi Tom, > You didn't say what the test condition was, but from the patch > I suppose it's a case where SelectConfigFiles is erroring out? Steps to reproduce: 1. Uncomment USE_VALGRIND in src/include/pg_config_manual.h, then rebuild and setup Postgres as usual 2. In the first terminal: ``` $ va

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 23:36, Peter Geoghegan wrote: > On Wed, Aug 13, 2025 at 1:01 PM Tomas Vondra wrote: >> This seems rather bizarre, considering the two tables are exactly the >> same, except that in t2 the first column is negative, and the rows are >> fixed-length. Even heap_page_items says the tables ar

Re: index prefetching

2025-08-13 Thread Andres Freund
Hi, On 2025-08-14 01:11:07 +0200, Tomas Vondra wrote: > On 8/13/25 23:57, Peter Geoghegan wrote: > > On Wed, Aug 13, 2025 at 5:19 PM Tomas Vondra wrote: > >> It's also not very surprising this happens with backwards scans more. > >> The I/O is apparently much slower (due to missing OS prefetch),

Re: [PATCH] Silence Valgrind about SelectConfigFiles()

2025-08-13 Thread Tom Lane
Aleksander Alekseev writes: > I experimented with Valgrind after recent changes committed by Tom [1] > and catched this: > ... > I propose to correct this as attached. You didn't say what the test condition was, but from the patch I suppose it's a case where SelectConfigFiles is erroring out? Kin

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 23:57, Peter Geoghegan wrote: > On Wed, Aug 13, 2025 at 5:19 PM Tomas Vondra wrote: >> It's also not very surprising this happens with backwards scans more. >> The I/O is apparently much slower (due to missing OS prefetch), so we're >> much more likely to hit the I/O limits (max_ios

[PATCH] Silence Valgrind about SelectConfigFiles()

2025-08-13 Thread Aleksander Alekseev
Hi, I experimented with Valgrind after recent changes committed by Tom [1] and catched this: ``` 25 bytes in 1 blocks are definitely lost in loss record 18 of 49 at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) by 0x57DE35E: strdup (strdup.c:42) by 0x9D9D

Re: Enhancing Memory Context Statistics Reporting

2025-08-13 Thread Rahila Syed
Hi Torikoshia, Thank you for reviewing the patch. > > This function is added at the end of Table "9.96. Server Signaling > Functions", but since pg_get_process_memory_contexts outputs essentially > the same information as pg_log_backend_memory_contexts, it might be > better to place them next to

Re: index prefetching

2025-08-13 Thread Andres Freund
Hi, On 2025-08-14 00:23:49 +0200, Tomas Vondra wrote: > On 8/13/25 23:37, Andres Freund wrote: > > On 2025-08-13 23:07:07 +0200, Tomas Vondra wrote: > >> On 8/13/25 16:44, Andres Freund wrote: > >>> On 2025-08-13 14:15:37 +0200, Tomas Vondra wrote: > In fact, I believe this is about io_method

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 23:37, Andres Freund wrote: > Hi, > > On 2025-08-13 23:07:07 +0200, Tomas Vondra wrote: >> On 8/13/25 16:44, Andres Freund wrote: >>> On 2025-08-13 14:15:37 +0200, Tomas Vondra wrote: In fact, I believe this is about io_method. I initially didn't see the difference you desc

Re: Improve the performance of Unicode Normalization Forms.

2025-08-13 Thread Jeff Davis
On Mon, 2025-08-11 at 17:21 +0300, Alexander Borisov wrote: > As a result, I would not look into ICU at the moment, especially > since > we have a different approach. > I am currently working on optimizing unicode_normalize(). > I am trying to come up with an improved version of the algorithm in C

Re: Support tid range scan in parallel?

2025-08-13 Thread Cary Huang
Hello David Thank you so much for testing the patch more thoroughly and for sharing your patch. I see that you moved the setting of ParallelBlockTableScanDesc fields from heap_set_tidrange() to be within heap_setscanlimits() so that the missing case of (ItemPointerCompare(&highestItem, &lowestIt

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 5:19 PM Tomas Vondra wrote: > It's also not very surprising this happens with backwards scans more. > The I/O is apparently much slower (due to missing OS prefetch), so we're > much more likely to hit the I/O limits (max_ios and various other limits > in read_stream_start_p

Re: index prefetching

2025-08-13 Thread Andres Freund
Hi, On 2025-08-13 23:07:07 +0200, Tomas Vondra wrote: > On 8/13/25 16:44, Andres Freund wrote: > > On 2025-08-13 14:15:37 +0200, Tomas Vondra wrote: > >> In fact, I believe this is about io_method. I initially didn't see the > >> difference you described, and then I realized I set io_method=sync t

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 1:01 PM Tomas Vondra wrote: > This seems rather bizarre, considering the two tables are exactly the > same, except that in t2 the first column is negative, and the rows are > fixed-length. Even heap_page_items says the tables are exactly the same. > > So why would the index

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 18:01, Peter Geoghegan wrote: > On Wed, Aug 13, 2025 at 11:28 AM Andres Freund wrote: >>> With "sync" I always get this (after a restart): >>> >>>Buffers: shared hit=7435 read=52801 >>> >>> while with "worker" I get this: >>> >>>Buffers: shared hit=4879 read=52801 >>>Buff

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 16:44, Andres Freund wrote: > Hi, > > On 2025-08-13 14:15:37 +0200, Tomas Vondra wrote: >> In fact, I believe this is about io_method. I initially didn't see the >> difference you described, and then I realized I set io_method=sync to >> make it easier to track the block access. And

Re: cfbot mistakenly reports that a rebase is needed

2025-08-13 Thread Jelte Fennema-Nio
On Wed, 13 Aug 2025 at 16:39, Tom Lane wrote: > > Aleksander Alekseev writes: > > I keep an eye on my patches using cfbog [1] and I think since recently > > it started to mistakenly mark patches as "needs rebase" while in fact > > everything is OK. If we take [2] for instance, it applies fine: >

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-08-13 Thread Jim Nasby
On Fri, Aug 8, 2025 at 4:03 PM Andres Freund wrote: > I'm not convinced that the global counter, be it a 32 or a 64 bit, approach > has merit in general, and I'm rather sure it's the wrong thing for toast > values. There's no straightforward path to move away from the global > counter > for plain

Re: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-08-13 Thread Robert Haas
On Wed, Aug 13, 2025 at 3:21 AM Hayato Kuroda (Fujitsu) wrote: > To confirm; your point is that we should firstly fix to allow > acquiring/releasing > slots in the mode, then consider additional guards, is it right? Valid point. Yes. > I still cannot find enough use-cases to allow manipulating

Re: Making type Datum be 8 bytes everywhere

2025-08-13 Thread Tom Lane
Peter Eisentraut writes: > One additional comment: In network.c and varlena.c there are a couple of > comments that match /4 byte datum/ that talk about abbreviation support > in that circumstance. Those should probably be removed or reworded. Ah, didn't think to search for that. I'll take ca

Re: meson: Add _static and _shared suffixes to the library names

2025-08-13 Thread Andres Freund
Hi, On 2025-08-13 10:24:07 -0400, Andres Freund wrote: > > 2- Using '/DEBUG:FULL' instead of '/DEBUG:FASTLINK' in the Windows CI > > task but this causes more memory to be used. It seems that the error > > appears only when the '/DEBUG:FASTLINK' is set. '/DEBUG:FULL' is a > > default option, so we

Re: AIO writes vs hint bits vs checksums

2025-08-13 Thread Andres Freund
Hi, On 2024-10-30 12:45:27 -0400, Andres Freund wrote: > On 2024-10-30 13:29:01 +0200, Heikki Linnakangas wrote: > > On 30/10/2024 04:21, Andres Freund wrote: > > > Attached is a, unfortunately long, series of patches implementing what I > > > described upthread. > > > > Review of the preparatory

Re: Making type Datum be 8 bytes everywhere

2025-08-13 Thread Peter Eisentraut
On 12.08.25 21:48, Tom Lane wrote: PFA v2 with these changes. I feel like this is pretty close to committable now. This looks good to me. One additional comment: In network.c and varlena.c there are a couple of comments that match /4 byte datum/ that talk about abbreviation support in that

Re: LISTEN/NOTIFY bug: VACUUM sets frozenxid past a xid in async queue

2025-08-13 Thread Daniil Davydov
Hi, On Mon, Aug 11, 2025 at 8:41 PM Matheus Alcantara wrote: > > On Wed Aug 6, 2025 at 7:44 AM -03, Álvaro Herrera wrote: > >> My questions: > >> > >> 1. Is it acceptable to drop notifications from the async queue if > >> there are no active listeners? There might still be notifications that > >>

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 18:36, Peter Geoghegan wrote: > On Wed, Aug 13, 2025 at 8:15 AM Tomas Vondra wrote: >> 1) created a second table with an "inverse pattern" that's decreasing: >> >> create table t2 (like t) with (fillfactor = 20); >> insert into t2 select -a, b from t; >> create index idx2 on t2 (a

Re: Dropping publication breaks logical replication

2025-08-13 Thread Masahiko Sawada
On Fri, Aug 8, 2025 at 5:48 AM Amit Kapila wrote: > > On Fri, Aug 8, 2025 at 5:19 PM Ashutosh Bapat > wrote: > > > > On Tue, Aug 5, 2025 at 9:50 AM Amit Kapila wrote: > > > > > > On Fri, Aug 1, 2025 at 5:06 PM Ashutosh Bapat > > > wrote: > > > > > > > > On Fri, Aug 1, 2025 at 4:03 PM Amit Kapil

Re: Adding basic NUMA awareness

2025-08-13 Thread Tomas Vondra
On 8/13/25 17:16, Andres Freund wrote: > Hi, > > On 2025-08-07 11:24:18 +0200, Tomas Vondra wrote: >> The patch does a much simpler thing - treat the weight as a "budget", >> i.e. number of buffers to allocate before proceeding to the "next" >> partition. So it allocates 55 buffers from P1, then 4

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 8:15 AM Tomas Vondra wrote: > 1) created a second table with an "inverse pattern" that's decreasing: > > create table t2 (like t) with (fillfactor = 20); > insert into t2 select -a, b from t; > create index idx2 on t2 (a); > alter index idx2 set (deduplicate_items =

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-08-13 Thread Timur Magomedov
On Mon, 2025-08-11 at 17:39 +0800, Japin Li wrote: > > 1. > +static struct > +{ > +    int transfn_oid;    /* Transition function's funcoid. > Arrays are > + * sorted in ascending order */ > +    Oid transtype;  /* Transition data type */ > +   

Re: index prefetching

2025-08-13 Thread Peter Geoghegan
On Wed, Aug 13, 2025 at 11:28 AM Andres Freund wrote: > > With "sync" I always get this (after a restart): > > > >Buffers: shared hit=7435 read=52801 > > > > while with "worker" I get this: > > > >Buffers: shared hit=4879 read=52801 > >Buffers: shared hit=5151 read=52801 > >Buffers

Re: [PATCH] Proposal: Improvements to PDF stylesheet and table column widths

2025-08-13 Thread Peter Eisentraut
On 08.05.25 09:18, Noboru Saito wrote: 3. **Replaced `text-indent` with `margin-left` for `func_signature` and `column_definition`**: - The previous `text-indent: -3.5em` only affected the first line, causing inconsistent formatting for multi-line code blocks. Changing this to `margin-left: 0em`

Re: Issue with logical replication slot during switchover

2025-08-13 Thread Fabrice Chapuis
Thanks for sharing this. In fact, I agree, introducing an allow_overwrite slot property makes seems cleaner than a GUC for this specific use case. a) At first, an extension of pg_create_logical_replication_slot() could be proposed b) pg_alter_logical_replication_slot(): that could be is a neat app

Re: [PATCH] Proposal: Improvements to PDF stylesheet and table column widths

2025-08-13 Thread Peter Eisentraut
On 08.05.25 09:18, Noboru Saito wrote: ### Patch 2: Adjustments to table column widths Building on the first patch, the second patch adjusts the column widths of various tables in the documentation to take advantage of the increased display width. I have committed this, except the change in ta

Re: Annoying warning in SerializeClientConnectionInfo

2025-08-13 Thread Andres Freund
Hi, On 2025-08-13 12:54:27 +0900, Michael Paquier wrote: > > I think the PG_USED_FOR_ASSERTS_ONLY fix is preferable for backport, > > so I don't want to get in the way of that approach. > > The attached has been working for me. Thoughts? I think we shouldn't add the attribute to the declaration

Re: Annoying warning in SerializeClientConnectionInfo

2025-08-13 Thread Jacob Champion
On Wed, Aug 13, 2025 at 8:28 AM Andres Freund wrote: > I think we shouldn't add the attribute to the declaration, just to the > function. The caller doesn't need to know that it's unused, it's purely a > question of the specific implementation that the attribute is unused. +1 --Jacob

Re: meson: Add _static and _shared suffixes to the library names

2025-08-13 Thread Andres Freund
Hi, On 2025-08-13 09:36:23 +0300, Nazir Bilal Yavuz wrote: > Thank you all for sharing your thoughts and I am sorry for the first > proposal. It is a mistake on my part, I should have thought more. That's just iterative development, don't worry about it. > On Tue, 12 Aug 2025 at 21:47, Peter Ei

Re: Adding basic NUMA awareness

2025-08-13 Thread Andres Freund
Hi, On 2025-08-07 11:24:18 +0200, Tomas Vondra wrote: > The patch does a much simpler thing - treat the weight as a "budget", > i.e. number of buffers to allocate before proceeding to the "next" > partition. So it allocates 55 buffers from P1, then 45 buffers from P2, > and then goes back to P1 in

Re: index prefetching

2025-08-13 Thread Andres Freund
Hi, On 2025-08-13 14:15:37 +0200, Tomas Vondra wrote: > In fact, I believe this is about io_method. I initially didn't see the > difference you described, and then I realized I set io_method=sync to > make it easier to track the block access. And if I change io_method to > worker, I get different

Re: Parallel Apply

2025-08-13 Thread Bruce Momjian
On Wed, Aug 13, 2025 at 09:50:27AM +0530, Amit Kapila wrote: > On Tue, Aug 12, 2025 at 10:40 PM Bruce Momjian wrote: > > > Currently, PostgreSQL supports parallel apply only for large streaming > > > transactions (streaming=parallel). This proposal aims to extend > > > parallelism to non-streaming

Re: Possible inaccurate description of wal_compression in docs

2025-08-13 Thread Jingtang Zhang
> Do any of you have a specific wording in mind? 'Like when FPW is on …' would be okay, or '(e.g., when FPW is on …)' would be formal? -- Regards, Jingtang

Detoast iterators - take 2

2025-08-13 Thread Nikita Malakhov
Hi hackers! Some time ago there was a discussion on detoast iterators [1]. The original thread stalled, but we took it, reviewed and refactored, and used it for some of our ideas. I've refactored it onto the current master and made some tests, iterative detoast provides significant improvement ov

Re: cfbot mistakenly reports that a rebase is needed

2025-08-13 Thread Tom Lane
Aleksander Alekseev writes: > I keep an eye on my patches using cfbog [1] and I think since recently > it started to mistakenly mark patches as "needs rebase" while in fact > everything is OK. If we take [2] for instance, it applies fine: Yeah, I have observed the same with my patches (according

Re: PG 18 release notes draft committed

2025-08-13 Thread Bruce Momjian
On Wed, Aug 13, 2025 at 11:38:33AM +0530, Amit Kapila wrote: > On Fri, May 2, 2025 at 8:14 AM Bruce Momjian wrote: > > > > I have committed the first draft of the PG 18 release notes. > > > > I think it is worth mentioning about commit > 7c99dc587a010a0c40d72a0e435111ca7a371c02 as that changes th

Re: Improve tab completion for various SET/RESET forms

2025-08-13 Thread Kirill Reshke
On Wed, 13 Aug 2025 at 14:54, Shinya Kato wrote: > > While reviewing these patches, I noticed that tab completion for > parameter names is missing for SET LOCAL and SET SESSION. Would it be > possible to fix this at the same time? Indeed, SET SESSION does not suggest any parameter. That should

Re: [PATCH] Proposal: Improvements to PDF stylesheet and table column widths

2025-08-13 Thread Peter Eisentraut
On 26.06.25 08:47, Tatsuo Ishii wrote: Dear Kuroda-san, Dear Ishii-san, Actually Saito-san has already been registered [1]. And cfbot said OK. [1]: https://commitfest.postgresql.org/patch/5775/ Thank you for pointing it out. Also I see Peter has been already registered as a committer. Great

Expose custom planning data in EXPLAIN

2025-08-13 Thread Andrei Lepikhov
Hi, Background and motivation - This feature is inspired by the pg_overexplain, new EXPLAIN-related hooks and recent change of the Memoize node representation (4bc62b86849). Based on user reports from the production system, we typically receive no more than an 'explai

Re: Improve LWLock tranche name visibility across backends

2025-08-13 Thread Bertrand Drouvot
Hi, On Tue, Aug 12, 2025 at 04:16:48PM -0500, Sami Imseih wrote: > I spoke offline with Bertrand and discussed the synchronization of > the local memory from shared memory. After that discussion it is clear > we don't need to track the highest used index in shared memory. Because > the tranche_id

Re: meson vs. llvm bitcode files

2025-08-13 Thread Nazir Bilal Yavuz
Hi, On Mon, 7 Jul 2025 at 11:45, Nazir Bilal Yavuz wrote: > > Mandatory rebase, v6 is attached. Rebase is needed due to 01d6832c10, v7 is attached. -- Regards, Nazir Bilal Yavuz Microsoft From d8bba14adf6e7ade5440c01f95390e701112d07d Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Sat, 27 A

Re: index prefetching

2025-08-13 Thread Tomas Vondra
On 8/13/25 01:33, Peter Geoghegan wrote: > On Tue, Aug 12, 2025 at 7:10 PM Tomas Vondra wrote: >> Actually, this might be a consequence of how backwards scans work (at >> least in btree). I logged the block in index_scan_stream_read_next, and >> this is what I see in the forward scan (at the begin

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread Amit Kapila
On Wed, Aug 13, 2025 at 4:15 PM shveta malik wrote: > > On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu) > wrote: > > > > > > Here is the V61 patch set which addressed above comments and the comment by > > Nisha[2]. > > > > Thank You for the patch. I tested the patch, please find a few comm

cfbot mistakenly reports that a rebase is needed

2025-08-13 Thread Aleksander Alekseev
Hi, I keep an eye on my patches using cfbog [1] and I think since recently it started to mistakenly mark patches as "needs rebase" while in fact everything is OK. If we take [2] for instance, it applies fine: ``` $ git am ~/Desktop/v4-0001-pgindent-improve-formatting-of-multiline-comments.patch

Re: index prefetching

2025-08-13 Thread Nazir Bilal Yavuz
Hi, On Tue, 12 Aug 2025 at 22:30, Thomas Munro wrote: > > On Tue, Aug 12, 2025 at 11:22 PM Nazir Bilal Yavuz wrote: > > Unfortunately this doesn't work. We need to handle backwards I/O > > combining in the StartReadBuffersImpl() function too as buffer indexes > > won't have correct blocknums. Al

Re: [PATCH] Refactor SLRU to always use long file names

2025-08-13 Thread Aleksander Alekseev
Hi, > > The new status of this patch is: Ready for Committer > > Thanks for testing! > > Here is the rebased patch. Rebased. From 4d83acd61eee36784840e2d1228d0c854807bac0 Mon Sep 17 00:00:00 2001 From: Aleksander Alekseev Date: Wed, 11 Sep 2024 13:17:33 +0300 Subject: [PATCH v6] Always use long

Re: [PATCH] OAuth: fix performance bug with stuck multiplexer events

2025-08-13 Thread Andrew Dunstan
On 2025-08-12 Tu 6:25 PM, Jacob Champion wrote: On Tue, Aug 12, 2025 at 3:21 PM Andrew Dunstan wrote: I don't think that's quite going to work. The buildfarm will now get a "target not found" in the without-curl case, I suspect. I think you'll need an alternative definition of the check targ

Re: VM corruption on standby

2025-08-13 Thread Kirill Reshke
On Tue, 12 Aug 2025 at 13:00, I wrote: > While this aims to find existing VM corruption (i mean, in PG <= 17), > this reproducer does not seem to work on pg17. At least, I did not > manage to reproduce this scenario on pg17. > > This makes me think this exact corruption may be pg18-only. Is it > p

RE: Parallel Apply

2025-08-13 Thread Zhijie Hou (Fujitsu)
On Monday, August 11, 2025 12:46 PM Amit Kapila wrote: > Background and Motivation > - > In high-throughput systems, where hundreds of sessions generate data > on the publisher, the subscriber's apply process often becomes a > bottleneck due to the single apply

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu) wrote: > > > Here is the V61 patch set which addressed above comments and the comment by > Nisha[2]. > Thank You for the patch. I tested the patch, please find a few comments: 1) Now when it stops-retention and later resumes it due to the fa

Re: Proposal: Conflict log history table for Logical Replication

2025-08-13 Thread Amit Kapila
On Fri, Aug 8, 2025 at 10:01 AM Dilip Kumar wrote: > > On Fri, Aug 8, 2025 at 8:58 AM shveta malik wrote: > > > > On Thu, Aug 7, 2025 at 3:08 PM Dilip Kumar wrote: > > > > > > So logically for PostgreSQL its an > > > user table but yeah this is created and managed by the extension. > > > > > > >

Re: Improve tab completion for various SET/RESET forms

2025-08-13 Thread Shinya Kato
On Wed, Aug 13, 2025 at 5:52 PM Shinya Kato wrote: > > > I also noticed that ALTER SYSTEM RESET tab completes with all variables, > > > not just ones that have been set with ALTER SYSTEM. Getting this right > > > is a bit more complicated, since the only way to distinguish them is > > > pg_settin

Re: Fix for typo in UUIDv7 timestamp extraction

2025-08-13 Thread Andrey Borodin
Hi Erik! > On 13 Aug 2025, at 11:05, Erik Nordström wrote: > > I think I found a small typo in the function that extracts a timestamp from a > UUIDv7 (uuid_extract_timestamp). Unless I am mistaken, the constant US_PER_MS > should be used instead of NS_PER_US when converting milliseconds to >

Fix for typo in UUIDv7 timestamp extraction

2025-08-13 Thread Erik Nordström
Hi, I think I found a small typo in the function that extracts a timestamp from a UUIDv7 (uuid_extract_timestamp). Unless I am mistaken, the constant US_PER_MS should be used instead of NS_PER_US when converting milliseconds to microseconds. Fortunately, these constants are the same so the calcula

Re: Improve hash join's handling of tuples with null join keys

2025-08-13 Thread Chao Li
I downloaded the patch and tested all join types: inner, left, right, full, semi and anti. Basically my tests all passed. However, I didn't test any case of parallel query. I have two nit comments: 1. In hashjoin.h, line 76-78, the added comment says "(In the unlikely but supported case of a n

Re: Improve tab completion for various SET/RESET forms

2025-08-13 Thread Shinya Kato
On Wed, Aug 13, 2025 at 3:56 PM Shinya Kato wrote: > > On Sat, Aug 9, 2025 at 7:55 AM Dagfinn Ilmari Mannsåker > wrote: > > > > Shinya Kato writes: > > > > > On Fri, Aug 1, 2025 at 2:16 AM Dagfinn Ilmari Mannsåker > > > wrote: > > >> > > >> Dagfinn Ilmari Mannsåker writes: > > >> > > >> > I ju

Re: Add log_autovacuum_{vacuum|analyze}_min_duration

2025-08-13 Thread kasaharatt
Hi, 2025-07-02 15:37 に Shinya Kato wrote: On Mon, Jun 23, 2025 at 4:24 PM Shinya Kato wrote: On Wed, Jun 11, 2025 at 1:49 PM Shinya Kato wrote: On Thu, Jun 5, 2025 at 3:53 AM Sami Imseih wrote: > Approach 2: > - log_autovacuum_min_duration: Changed behavior, which controls only > auto

Re: GB18030-2022 Support in PostgreSQL

2025-08-13 Thread Chao Li
On 2025/8/13 15:20, Chao Li wrote: Sounds good. Let me recreate the patch. Attached is the new patch. It downloads the UCM file in make: ``` Unicode % make gb18030_to_utf8.map wget -O gb-18030-2000.ucm --no-use-server-timestamps https://raw.githubusercontent.com/unicode-org/icu-data/d9d3

Re: meson: Add _static and _shared suffixes to the library names

2025-08-13 Thread Nazir Bilal Yavuz
Hi, Thank you for looking into this! On Wed, 13 Aug 2025 at 10:50, Wolfgang Walther wrote: > > Nazir Bilal Yavuz: > > 1- Setting different names for .pdb files for shared libraries only on > > the Windows OS. I think that is the correct fix, it just adds _shared > > suffix to .pdb files of share

  1   2   >