Re: Pgoutput not capturing the generated columns

2024-08-08 Thread Peter Smith
Hi Shubham, I think the v25-0001 patch only half-fixes the problems reported in my v24-0001 review. ~ Background (from the commit message): This commit enables support for the 'include_generated_columns' option in logical replication, allowing the transmission of generated column information and

Using read stream in autoprewarm

2024-08-08 Thread Nazir Bilal Yavuz
Hi, I am working on using the read stream in autoprewarm. I observed ~10% performance gain with this change. The patch is attached. The downside of the read stream approach is that a new read stream object needs to be created for each database, relation and fork. I was wondering if this would cau

Re: Add on_error and log_verbosity options to file_fdw

2024-08-08 Thread torikoshia
On 2024-07-24 19:43, torikoshia wrote: On 2024-07-23 08:57, Michael Paquier wrote: On Mon, Jul 22, 2024 at 03:07:46PM -0700, Masahiko Sawada wrote: I'm slightly concerned that users might not want to see the NOTICE message for every scan. Unlike COPY FROM, scanning a file via file_fdw could b

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-08 Thread cca5507
Hi, Thanks for pointing it out! Here are the new version patches with a test case. -- Regards, ChangAo Chen -- Original -- From: "Bertrand

Re: Windows default locale vs initdb

2024-08-08 Thread Ertan Küçükoglu
> > I already installed Visual Studio 2022 with C++ support as suggested in > https://www.postgresql.org/docs/current/install-windows-full.html > I cloned codes in the system. > But, I cannot find any "src/tools/msvc" directory. It is missing. > Document states I need everything in there > "The too

Re: Found issues related with logical replication and 2PC

2024-08-08 Thread shveta malik
On Thu, Aug 8, 2024 at 9:53 AM Amit Kapila wrote: > > On Thu, Aug 8, 2024 at 8:54 AM shveta malik wrote: > > > > On Wed, Aug 7, 2024 at 5:43 PM Amit Kapila wrote: > > > > > So, if my > > > analysis is correct, this shouldn't be a bug and ideally, we should > > > update local_end LSN as InvalidXL

RE: Found issues related with logical replication and 2PC

2024-08-08 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Shveta, Thanks for discussing! I reported the issue because 1) I feared the risk of data loss and 2) simply because the coding looked incorrect. However, per discussion, I understood that it wouldn't lead to loss, and adding a global variable was unacceptable in this case. I modified t

Re: Logical Replication of sequences

2024-08-08 Thread Amit Kapila
On Thu, Aug 8, 2024 at 11:09 AM Peter Smith wrote: > > But, I haven't invented a new overloading for "copy_data" option > (meaning "synchronize") for sequences. The current patchset already > interprets copy_data exactly this way. > > For example, below are patch 0003 results: > > ALTER SUBSCRIPTI

Re: Incremental View Maintenance, take 2

2024-08-08 Thread Kirill Reshke
I am really sorry for splitting my review comments into multiple emails. I'll try to do a better review in a future, all-in-one. On Thu, 11 Jul 2024 at 09:24, Yugo NAGATA wrote: > > On Tue, 2 Jul 2024 17:03:11 +0900 > Yugo NAGATA wrote: > > > On Sun, 31 Mar 2024 22:59:31 +0900 > > Yugo NAGATA w

Re: [bug fix] prepared transaction might be lost when max_prepared_transactions is zero on the subscriber

2024-08-08 Thread shveta malik
On Thu, Aug 8, 2024 at 12:03 PM Amit Kapila wrote: > > On Thu, Aug 8, 2024 at 10:37 AM Hayato Kuroda (Fujitsu) > wrote: > > > ... > > > > An easiest fix is to reset session replication origin before calling the > > RecordTransactionAbort(). I think this can happen when 1) > > LogicalRepApplyLoop

Re: Incremental View Maintenance, take 2

2024-08-08 Thread Kirill Reshke
On Wed, 31 May 2023 at 20:14, Yugo NAGATA wrote: > > Hello hackers, > > Here's a rebased version of the patch-set adding Incremental View > Maintenance support for PostgreSQL. That was discussed in [1]. > > The patch-set consists of the following eleven patches. > > - 0001: Add a syntax to create

RE: [bug fix] prepared transaction might be lost when max_prepared_transactions is zero on the subscriber

2024-08-08 Thread Zhijie Hou (Fujitsu)
On Thursday, August 8, 2024 6:01 PM shveta malik wrote: > > On Thu, Aug 8, 2024 at 12:03 PM Amit Kapila > wrote: > > > > On Thu, Aug 8, 2024 at 10:37 AM Hayato Kuroda (Fujitsu) > > wrote: > > > > > ... > > > > > > An easiest fix is to reset session replication origin before calling > > > the Re

Re: Enable data checksums by default

2024-08-08 Thread Peter Eisentraut
On 07.08.24 00:46, Greg Sabino Mullane wrote: Currently, initdb only enables data checksums if passed the --data-checksums or -k argument. There was some hesitation years ago when this feature was first added, leading to the current situation where the default is off. However, many years later,

Re: Refactoring postmaster's code to cleanup after child exit

2024-08-08 Thread Thomas Munro
On Fri, Aug 2, 2024 at 11:57 AM Heikki Linnakangas wrote: > * v3-0001-Make-BackgroundWorkerList-doubly-linked.patch LGTM. > [v3-0002-Refactor-code-to-handle-death-of-a-backend-or-bgw.patch] Currently, when a child process exits, the postmaster first scans through BackgroundWorkerList, t

Re: long-standing data loss bug in initial sync of logical replication

2024-08-08 Thread Shlok Kyal
On Wed, 31 Jul 2024 at 11:17, Shlok Kyal wrote: > > On Wed, 31 Jul 2024 at 09:36, Amit Kapila wrote: > > > > On Wed, Jul 31, 2024 at 3:27 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila > > > wrote: > > > > > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh

Re: Add trim_trailing_whitespace to editorconfig file

2024-08-08 Thread Andrew Dunstan
On 2024-08-07 We 4:42 PM, Jelte Fennema-Nio wrote: On Wed, 7 Aug 2024 at 21:09, Andrew Dunstan wrote: You're not meant to use our pg_bsd_indent on its own without the appropriate flags, namely (from src/tools/pgindent/pgindent): Ah sorry, I wasn't clear in what I meant then. I meant that if y

Re: Windows default locale vs initdb

2024-08-08 Thread Andrew Dunstan
On 2024-08-08 Th 4:08 AM, Ertan Küçükoglu wrote: I already installed Visual Studio 2022 with C++ support as suggested in https://www.postgresql.org/docs/current/install-windows-full.html I cloned codes in the system. But, I cannot find any "src/tools/msvc" directory. It is m

Re: Partial aggregates pushdown

2024-08-08 Thread Jelte Fennema-Nio
SUMMARY OF THREAD The design of patch 0001 is agreed upon by everyone on the thread (so far). This adds the PARTIAL_AGGREGATE label for aggregates, which will cause the finalfunc not to run. It also starts using PARTIAL_AGGREGATE for pushdown of aggregates in postgres_fdw. In 0001 PARTIAL_AGGREGAT

Re: CREATE DATABASE with filesystem cloning

2024-08-08 Thread Nazir Bilal Yavuz
Hi, Rebased version of the patch is attached. -- Regards, Nazir Bilal Yavuz Microsoft From a419004a26410c9ad9348e2f1420695db8ca35b6 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Thu, 8 Aug 2024 15:01:48 +0300 Subject: [PATCH v9] Introduce file_copy_method GUC This GUC can be set to ei

Re: Found issues related with logical replication and 2PC

2024-08-08 Thread Amit Kapila
On Thu, Aug 8, 2024 at 2:37 PM Hayato Kuroda (Fujitsu) wrote: > > Thanks for discussing! > > I reported the issue because 1) I feared the risk of data loss and 2) simply > because the coding looked incorrect. However, per discussion, I understood > that > it wouldn't lead to loss, and adding a gl

Re: [bug fix] prepared transaction might be lost when max_prepared_transactions is zero on the subscriber

2024-08-08 Thread Amit Kapila
On Thu, Aug 8, 2024 at 3:41 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, August 8, 2024 6:01 PM shveta malik > wrote: > > > > On Thu, Aug 8, 2024 at 12:03 PM Amit Kapila > > wrote: > > > > > > On Thu, Aug 8, 2024 at 10:37 AM Hayato Kuroda (Fujitsu) > > > wrote: > > > > > > > ... > > > > > >

Re: Detailed release notes

2024-08-08 Thread Marcos Pegoraro
Em qui., 8 de ago. de 2024 às 00:11, jian he escreveu: > > adding the following line to postgres.sgml saved me. > +https://postgr.es/c/";> > Comma at end of line of these should be removed [165d581f1] , [6dbb49026] , Maybe those items which have lots of commits would be strange A space before

Re: Enable data checksums by default

2024-08-08 Thread Michael Banck
On Thu, Aug 08, 2024 at 12:11:38PM +0200, Peter Eisentraut wrote: > So I think we need to think through the upgrade experience a bit more. > Unfortunately, pg_checksums hasn't gotten to the point that we were perhaps > once hoping for that you could enable checksums on a live system. I'm > thinkin

Re: Enable data checksums by default

2024-08-08 Thread Daniel Gustafsson
> On 8 Aug 2024, at 12:11, Peter Eisentraut wrote: > My understanding was that the reason for some hesitation about adopting data > checksums was the performance impact. Not the checksumming itself, but the > overhead from hint bit logging. The last time I looked into that, you could > get p

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2024-08-08 Thread Michail Nikolaev
Hello, Matthias! > While waiting for this, here are some initial comments on the github diffs: Thanks for your review! While stress testing the POC, I found some issues unrelated to the patch that need to be fixed first. This is [1] and [2]. >> Additional index is lightweight and does not produc

Re: First draft of PG 17 release notes

2024-08-08 Thread Justin Pryzby
> Add server variable huge_page_size to report the use of huge pages by The new variable is huge_page_status; h_p_size is several years old. BTW, I was surprised that these were included: +2024-02-28 [363eb0599] Convert README to Markdown. +2024-01-25 [7014c9a4b] Doc: improve documentation for j

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-08-08 Thread Tender Wang
Alvaro Herrera 于2024年8月8日周四 06:50写道: > On 2024-Jul-26, Tender Wang wrote: > > > Junwang Zhao 于2024年7月26日周五 14:57写道: > > > > > There is a bug report[0] Tender comments might be the same issue as > > > this one, but I tried Alvaro's and mine patch, neither could solve > > > that problem, I did not

Re: [PATCH] Add crc32(text) & crc32(bytea)

2024-08-08 Thread Peter Eisentraut
On 05.08.24 17:28, Nathan Bossart wrote: This looks pretty good to me. The only point that I think deserves more discussion is the return type. Does bytea make the most sense here? Or should we consider int/bigint? The correct return type of a CRC operation in general is some kind of exact

Re: [PATCH] Add crc32(text) & crc32(bytea)

2024-08-08 Thread Nathan Bossart
On Thu, Aug 08, 2024 at 04:27:20PM +0200, Peter Eisentraut wrote: > On 05.08.24 17:28, Nathan Bossart wrote: >> This looks pretty good to me. The only point that I think deserves more >> discussion is the return type. Does bytea make the most sense here? Or >> should we consider int/bigint? > >

Re: [PATCH] Add crc32(text) & crc32(bytea)

2024-08-08 Thread Tom Lane
Nathan Bossart writes: > On Thu, Aug 08, 2024 at 04:27:20PM +0200, Peter Eisentraut wrote: >> The correct return type of a CRC operation in general is some kind of exact >> numerical type. Just pick the best one that fits the result. I don't think >> bytea is appropriate. > That would leave us

Re: Logical Replication of sequences

2024-08-08 Thread vignesh C
On Wed, 7 Aug 2024 at 10:27, shveta malik wrote: > > On Mon, Aug 5, 2024 at 10:26 AM vignesh C wrote: > > > > On Thu, 1 Aug 2024 at 04:25, Peter Smith wrote: > > > > > > Hi Vignesh, > > > > > > I noticed that when replicating sequences (using the latest patches > > > 0730_2*) the subscriber-sid

Re: Fix memory counter update in reorderbuffer

2024-08-08 Thread Masahiko Sawada
On Wed, Aug 7, 2024 at 3:17 PM Amit Kapila wrote: > > On Wed, Aug 7, 2024 at 7:42 AM Masahiko Sawada wrote: > > > > On Tue, Aug 6, 2024 at 1:12 PM Amit Kapila wrote: > > > > > > On Sat, Aug 3, 2024 at 1:21 AM Masahiko Sawada > > > wrote: > > > > > > > > I found a bug in the memory counter upda

Re: [PATCH] Add crc32(text) & crc32(bytea)

2024-08-08 Thread Nathan Bossart
On Thu, Aug 08, 2024 at 10:49:42AM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Thu, Aug 08, 2024 at 04:27:20PM +0200, Peter Eisentraut wrote: >>> The correct return type of a CRC operation in general is some kind of exact >>> numerical type. Just pick the best one that fits the result.

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-08-08 Thread Noah Misch
On Sun, Apr 07, 2024 at 01:22:51AM +0300, Alexander Korotkov wrote: > I've pushed 0001 and 0002 The partition MERGE (1adf16b8f) and SPLIT (87c21bb94) v17 patches introduced createPartitionTable() with this code: createStmt->relation = newPartName; ... wrapper->utilityStmt = (Node

Re: Enable data checksums by default

2024-08-08 Thread Greg Sabino Mullane
Thank you for the feedback. Please find attached three separate patches. One to add a new flag to initdb (--no-data-checksums), one to adjust the tests to use this flag as needed, and the final to make the actual switch of the default value (along with tests and docs). Cheers, Greg 0001-Add-new-

Re: Vectored IO in XLogWrite()

2024-08-08 Thread Robert Haas
On Tue, Aug 6, 2024 at 6:30 PM Melih Mutlu wrote: > Fixed. +iov[0].iov_base = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;; Double semicolon. Aside from that, this looks correct now, so the next question is whether we want it. To me, it seems like this isn't likely to buy ver

Re: Enable data checksums by default

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: > About the claim that it's already the de-facto standard. Maybe that is > approximately true for "serious" installations. But AFAICT, the popular > packagings don't enable checksums by default, so there is likely a > significant middle tier

fix CRC algorithm in WAL reliability docs

2024-08-08 Thread Nathan Bossart
I noticed that the "WAL reliability" documentation says that we use CRC-32 for WAL records and two-phase state files, but we've actually used CRC-32C since v9.5 (commit 5028f22). I've attached a short patch to fix this that I think should be back-patched to all supported versions. I've attached a

Re: Virtual generated columns

2024-08-08 Thread Dean Rasheed
On Thu, 8 Aug 2024 at 07:23, Peter Eisentraut wrote: > > Thank you for your extensive testing. Here is a new patch set that has > fixed all the issues you have reported (MERGE, sublinks, statistics, > ANALYZE). I had a quick look at this and found one issue, which is that it doesn't properly dea

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Tomas Vondra
On 8/7/24 21:39, Melanie Plageman wrote: > On Wed, Aug 7, 2024 at 1:06 PM Robert Haas wrote: >> >> As I mentioned to you off-list, I feel like this needs some sort of >> recency bias. Certainly vacuum, and really almost any conceivable user >> of this facility, is going to care more about accurate

Re: Enable data checksums by default

2024-08-08 Thread Tomas Vondra
On 8/8/24 19:42, Robert Haas wrote: > On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: >> About the claim that it's already the de-facto standard. Maybe that is >> approximately true for "serious" installations. But AFAICT, the popular >> packagings don't enable checksums by default, so th

Re: Which parts of src/backend/nodes/print.c are used?

2024-08-08 Thread Paul Jungwirth
On 12/4/23 05:50, Tom Lane wrote: Peter Eisentraut writes: src/backend/nodes/print.c contains a number of functions that print node types, mostly to stdout. Most of these are not actually used anywhere in the code. Are they meant to be inserted into the code ad hoc for debugging? Is anyone u

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > How could we do this? We have 1s precision, so we start with buckets for > each seconds. And we want to allow merging stuff nicely. The smallest > merges we could do is 1s -> 2s -> 4s -> 8s -> ... but let's say we do > 1s -> 10s -> 100s -> 1000s

Re: fix CRC algorithm in WAL reliability docs

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 1:51 PM Nathan Bossart wrote: > I noticed that the "WAL reliability" documentation says that we use CRC-32 > for WAL records and two-phase state files, but we've actually used CRC-32C > since v9.5 (commit 5028f22). I've attached a short patch to fix this that > I think shou

Re: Don't overwrite scan key in systable_beginscan()

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 2:46 AM Peter Eisentraut wrote: > When systable_beginscan() and systable_beginscan_ordered() choose an > index scan, they remap the attribute numbers in the passed-in scan keys > to the attribute numbers of the index, and then write those remapped > attribute numbers back in

PATCH: Add hooks for pg_total_relation_size and pg_indexes_size

2024-08-08 Thread Abdoulaye Ba
Hello PostgreSQL Hackers, I am submitting a patch to add hooks for the functions pg_total_relation_size and pg_indexes_size. These hooks allow for custom behaviour to be injected into these functions, which can be useful for extensions and other custom PostgreSQL modifications. Patch details:

Re: [patch] Imporve pqmq

2024-08-08 Thread Robert Haas
On Wed, Aug 7, 2024 at 11:24 PM Xiaoran Wang wrote: > When I use the 'pqmq' recently, I found some issues, just fix them. > > Allow the param 'dsm_segment *seg' to be NULL in function > 'pq_redirect_to_shm_mq'. As sometimes the shm_mq is created > in shared memory instead of DSM.

[18] Fix a few issues with the collation cache

2024-08-08 Thread Jeff Davis
The collation cache, which maps collation oids to pg_locale_t objects, has a few longstanding issues: 1. Using TopMemoryContext too much, with potential for leaks on error paths. 2. Creating collators (UCollator for ICU or locale_t for libc) that can leak on some error paths. For instance, the fo

Re: Don't overwrite scan key in systable_beginscan()

2024-08-08 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 8, 2024 at 2:46 AM Peter Eisentraut wrote: >> I propose to fix that by making a copy of the scan keys passed by the >> caller and make the modifications there. > This does have the disadvantage of adding more palloc overhead. It seems hard to believe that one m

Re: Proposal for Updating CRC32C with AVX-512 Algorithm.

2024-08-08 Thread Nathan Bossart
On Wed, Jun 12, 2024 at 12:37:46PM -0700, Andres Freund wrote: > I'm wonder if this isn't going in the wrong direction. We're using CRCs for > something they're not well suited for in my understanding - and are paying a > reasonably high price for it, given that even hardware accelerated CRCs aren'

Re: Restart pg_usleep when interrupted

2024-08-08 Thread Nathan Bossart
On Thu, Aug 08, 2024 at 06:49:27AM +, Bertrand Drouvot wrote: > 2.2 CLOCK_REALTIME: Its time represents seconds and nanoseconds since the > Epoch. > It means that we´re currently about 237 years away from the limit. So even, > if we were to say add 2 "recents" of them we are still about 183 y

Re: tiny step toward threading: reduce dependence on setlocale()

2024-08-08 Thread Jeff Davis
On Wed, 2024-08-07 at 22:44 +0200, Peter Eisentraut wrote: > But after this patch set, locale cannot be NULL anymore, so the third > branch is obsolete. ... > Second, there are a number of functions in like.c like the above that > take separate arguments like pg_locale_t locale, bool locale_is_c.

Re: Restart pg_usleep when interrupted

2024-08-08 Thread Nathan Bossart
On Wed, Aug 07, 2024 at 04:22:22PM -0500, Sami Imseih wrote: > Please see v7. Thanks. This one looks pretty good to me, and so I plan to commit it in the near future unless anyone voices concerns about the approach. -- nathan

Re: tiny step toward threading: reduce dependence on setlocale()

2024-08-08 Thread Peter Eisentraut
On 07.08.24 22:44, Peter Eisentraut wrote: (Now that I look at it, pg_tolower() has some short-circuiting for ASCII letters, so it would not handle Turkish-i correctly if that had been the global locale.  By removing the use of pg_tolower(), we fix that issue in passing.) It occurred to me th

Re: tiny step toward threading: reduce dependence on setlocale()

2024-08-08 Thread Peter Eisentraut
On 08.08.24 22:00, Jeff Davis wrote: On Wed, 2024-08-07 at 22:44 +0200, Peter Eisentraut wrote: But after this patch set, locale cannot be NULL anymore, so the third branch is obsolete. ... Second, there are a number of functions in like.c like the above that take separate arguments like pg_

Re: PATCH: Add hooks for pg_total_relation_size and pg_indexes_size

2024-08-08 Thread Tomas Vondra
On 8/8/24 14:18, Abdoulaye Ba wrote: > Hello PostgreSQL Hackers, > > I am submitting a patch to add hooks for the functions > pg_total_relation_size and pg_indexes_size. These hooks allow for custom > behaviour to be injected into these functions, which can be useful for > extensions and other cus

Re: Remove dependence on integer wrapping

2024-08-08 Thread Matthew Kim
I've updated patch 0004 to check the return value of pg_mul_s32_overflow(). Since tm.tm_year overflowed, the error message is hardcoded. Thanks, Matthew Kim From c142581fb5f4a26de40cdca0d8ca7d39abdb2e15 Mon Sep 17 00:00:00 2001 From: Joseph Koshakow Date: Sat, 6 Jul 2024 15:41:09 -0400 Subject: [

Re: optimizing pg_upgrade's once-in-each-database steps

2024-08-08 Thread Corey Huinker
On Tue, Aug 6, 2024 at 3:20 PM Nathan Bossart wrote: > On Sun, Aug 04, 2024 at 07:19:57PM +0100, Ilya Gladyshev wrote: > > -- End of review -- > > Thanks for the review. I've attempted to address all your feedback in v8 > of the patch set. I think the names could still use some work, but I > wa

Re: tiny step toward threading: reduce dependence on setlocale()

2024-08-08 Thread Jeff Davis
On Thu, 2024-08-08 at 23:09 +0200, Peter Eisentraut wrote: > With current master, after commit e9931bfb751, you get the first > result > in both cases. > > So this could break indexes across pg_upgrade in such configurations. Good observation. Is there any action we should take here, or just add

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-08-08 Thread Alexander Korotkov
On Thu, Aug 8, 2024 at 8:14 PM Noah Misch wrote: > On Sun, Apr 07, 2024 at 01:22:51AM +0300, Alexander Korotkov wrote: > > I've pushed 0001 and 0002 > > The partition MERGE (1adf16b8f) and SPLIT (87c21bb94) v17 patches introduced > createPartitionTable() with this code: > > createStmt->rel

Re: Logical Replication of sequences

2024-08-08 Thread Peter Smith
Hi Vignesh, I reviewed the latest v20240808-0003 patch. Attached are my minor change suggestions. == Kind Regards, Peter Smith. Fujitsu Australia diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c index 4e2f960..a77e810 100644 --- a/src/backend/catalog

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Tomas Vondra
On 8/8/24 20:59, Robert Haas wrote: > On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: >> How could we do this? We have 1s precision, so we start with buckets for >> each seconds. And we want to allow merging stuff nicely. The smallest >> merges we could do is 1s -> 2s -> 4s -> 8s -> ... but let

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 8:39 PM Tomas Vondra wrote: > Is there a reason not to make this just another SLRU, just like we do > for commit_ts? I'm not saying it's perfect, but it's an approach we > already use to solve these issues. An SLRU is essentially an infinitely large array that grows at one

Re: Detailed release notes

2024-08-08 Thread jian he
On Thu, Aug 8, 2024 at 8:51 PM Marcos Pegoraro wrote: > > Em qui., 8 de ago. de 2024 às 00:11, jian he > escreveu: >> >> >> adding the following line to postgres.sgml saved me. >> +https://postgr.es/c/";> > > > Comma at end of line of these should be removed > [165d581f1] , > [6dbb49026] , > >

Re: Remove dependence on integer wrapping

2024-08-08 Thread jian he
On Fri, Aug 9, 2024 at 6:16 AM Matthew Kim wrote: > > I've updated patch 0004 to check the return value of pg_mul_s32_overflow(). > Since tm.tm_year overflowed, the error message is hardcoded. > --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -257,7 +257,10 @@ make_date(

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Melanie Plageman
On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > > On 8/7/24 21:39, Melanie Plageman wrote: > > On Wed, Aug 7, 2024 at 1:06 PM Robert Haas wrote: > >> > >> As I mentioned to you off-list, I feel like this needs some sort of > >> recency bias. Certainly vacuum, and really almost any conceivabl

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Melanie Plageman
On Thu, Aug 8, 2024 at 3:00 PM Robert Haas wrote: > > On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > > A-D is already enough to cover 30h, with A-E it'd be ~300h. Do we need > > (or want) to keep a longer history? > > I think there is a difference of opinion about this between Melanie > and

Re: [patch] Imporve pqmq

2024-08-08 Thread Xiaoran Wang
Robert Haas 03:24 (6小时前) 发送至 我、 pgsql-hackers On Wed, Aug 7, 2024 at 11:24 PM Xiaoran Wang wrote: > When I use the 'pqmq' recently, I found some issues, just fix them. > > Allow the param 'dsm_segment *seg' to be NULL in function > 'pq_redirect_to_shm_mq'. As sometimes the shm_mq is

Re: Found issues related with logical replication and 2PC

2024-08-08 Thread shveta malik
On Thu, Aug 8, 2024 at 5:53 PM Amit Kapila wrote: > > On Thu, Aug 8, 2024 at 2:37 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Thanks for discussing! > > > > I reported the issue because 1) I feared the risk of data loss and 2) simply > > because the coding looked incorrect. However, per discussio

Re: [bug fix] prepared transaction might be lost when max_prepared_transactions is zero on the subscriber

2024-08-08 Thread shveta malik
On Thu, Aug 8, 2024 at 6:08 PM Amit Kapila wrote: > > On Thu, Aug 8, 2024 at 3:41 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Thursday, August 8, 2024 6:01 PM shveta malik > > wrote: > > > > > > On Thu, Aug 8, 2024 at 12:03 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Aug 8, 2024 at 10:

Re: Don't overwrite scan key in systable_beginscan()

2024-08-08 Thread Noah Misch
On Thu, Aug 08, 2024 at 08:46:35AM +0200, Peter Eisentraut wrote: > When systable_beginscan() and systable_beginscan_ordered() choose an index > scan, they remap the attribute numbers in the passed-in scan keys to the > attribute numbers of the index, and then write those remapped attribute > numbe

Re: Don't overwrite scan key in systable_beginscan()

2024-08-08 Thread Tom Lane
Noah Misch writes: > On Thu, Aug 08, 2024 at 08:46:35AM +0200, Peter Eisentraut wrote: >> I propose to fix that by making a copy of the scan keys passed by the caller >> and make the modifications there. > No objection, but this would obsolete at least some of these comments (the > catcache.c one

RE: Found issues related with logical replication and 2PC

2024-08-08 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > The code changes look mostly good to me. I have changed/added a few > comments in the attached modified version. > Thanks for updating the patch! It LGTM. I've tested your patch and confirmed it did not cause the data loss. I used the source which was applied v3 and additional fix

Re: Found issues related with logical replication and 2PC

2024-08-08 Thread Amit Kapila
On Fri, Aug 9, 2024 at 10:34 AM Hayato Kuroda (Fujitsu) wrote: > > > > > The code changes look mostly good to me. I have changed/added a few > > comments in the attached modified version. > > > > Thanks for updating the patch! It LGTM. I've tested your patch and confirmed > it did not cause the da

Re: Remove obsolete RECHECK keyword completely

2024-08-08 Thread Peter Eisentraut
On 05.08.24 15:44, Aleksander Alekseev wrote: I propose to remove the obsolete RECHECK keyword completely. This used to be part of CREATE OPERATOR CLASS and ALTER OPERATOR FAMILY, but it has done nothing (except issue a NOTICE) since PostgreSQL 8.4. Commit 30e7c175b81 removed support for dumpin

Re: [bug fix] prepared transaction might be lost when max_prepared_transactions is zero on the subscriber

2024-08-08 Thread Amit Kapila
On Fri, Aug 9, 2024 at 9:35 AM shveta malik wrote: > > On Thu, Aug 8, 2024 at 6:08 PM Amit Kapila wrote: > > > > > > > > I think this is a general issue that can occur not only due to 2PC. IIUC, > > > this > > > problem should arise if any ERROR arises after setting the > > > replorigin_session_

Re: Fix memory counter update in reorderbuffer

2024-08-08 Thread Amit Kapila
On Thu, Aug 8, 2024 at 9:43 PM Masahiko Sawada wrote: > > On Wed, Aug 7, 2024 at 3:17 PM Amit Kapila wrote: > > > > On Wed, Aug 7, 2024 at 7:42 AM Masahiko Sawada > > wrote: > > > > > > > > > > BTW, commit 5bec1d6bc5e also introduced > > > > ReorderBufferChangeMemoryUpdate() in ReorderBufferTru

Re: Logical Replication of sequences

2024-08-08 Thread shveta malik
On Wed, Aug 7, 2024 at 2:00 PM vignesh C wrote: > > On Wed, 7 Aug 2024 at 08:09, Amit Kapila wrote: > > > > On Tue, Aug 6, 2024 at 5:13 PM vignesh C wrote: > > > > > > On Mon, 5 Aug 2024 at 18:05, shveta malik wrote: > > > > > > > > On Mon, Aug 5, 2024 at 11:04 AM vignesh C wrote: > > > > > >