Re: proposal: schema variables

2024-07-31 Thread Laurenz Albe
On Wed, 2024-07-31 at 08:41 +0200, Pavel Stehule wrote: > Probably you didn't attach new files - the second patch is not complete. Or > you didn't make changes there? Hm. What is missing? Yours, Laurenz Albe

Re: proposal: schema variables

2024-07-31 Thread Pavel Stehule
út 30. 7. 2024 v 21:46 odesílatel Laurenz Albe napsal: > This is my review of the second patch in the series. > > Again, I mostly changed code comments and documentation. > > Noteworthy remarks: > > - A general reminder: single line comments should start with a lower case > letter and have to

Re: Incremental View Maintenance, take 2

2024-07-31 Thread Kirill Reshke
On Tue, 30 Jul 2024 at 03:32, Kirill Reshke wrote: > > On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > > > Hi! > > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > > use matview) feature, so i got interested in how it is implemented. > > > > On Thu, 11 Jul 2024 at

Re: Remove last traces of HPPA support

2024-07-30 Thread Thomas Munro
} - - S_UNLOCK(_lock.lock); - - if (test_lock.pad1 != 0x44 || test_lock.pad2 != 0x44) - { - printf("S_LOCK_TEST: failed, declared datatype is wrong size\n"); - return 1; - } - - if (!S_LOCK_FREE(_lock.lock)) - { - printf("S_LOCK_TEST: failed, lock not unlocked\n"); - return 1;

Re: long-standing data loss bug in initial sync of logical replication

2024-07-30 Thread Shlok Kyal
On Wed, 31 Jul 2024 at 09:36, Amit Kapila wrote: > > On Wed, Jul 31, 2024 at 3:27 AM Masahiko Sawada wrote: > > > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila wrote: > > > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C wrote: > > > > > > > > On Wed, 17 Jul 2024 at 11:54, Amit Kapila > > >

Re: New compiler warnings in buildfarm

2024-07-30 Thread Peter Eisentraut
On 30.07.24 18:19, Tom Lane wrote: Sometime in the last month or so, flaviventris's bleeding-edge version of gcc has started whining[1] about truncation of a string literal's implicit trailing '\0' in contexts like this: ../pgsql/src/backend/commands/copyto.c:106:41: warning: initializer-string

Re: Conflict detection and logging in logical replication

2024-07-30 Thread shveta malik
On Wed, Jul 31, 2024 at 7:40 AM Zhijie Hou (Fujitsu) wrote: > > > > > 2) > > apply_handle_delete_internal() > > > > --Do we need to check "(!edata->mtstate || edata->mtstate->operation != > > CMD_UPDATE)" in the else part as well? Can there be a scenario where during > > update flow, it is trying

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-07-30 Thread Anton A. Melnikov
Hi, Peter! On 20.01.2024 01:41, Peter Geoghegan wrote: It is quite likely that there are exactly zero affected out-of-core index AMs. I don't count pgroonga as a counterexample (I don't think that it actually fullfills the definition of a ). Basically, "amcanorder" index AMs more or less

Re: Build with LTO / -flto on macOS

2024-07-30 Thread Peter Eisentraut
On 22.07.24 16:04, Peter Eisentraut wrote: On 19.07.24 12:40, Aleksander Alekseev wrote: It seems to me that the patch is not going to become any better and it doesn't need any more attention from the reviewers. Thus I changed the status of the CF entry to "Ready for Committer". I'm happy to

Re: Add mention of execution time memory for enable_partitionwise_* GUCs

2024-07-30 Thread Ashutosh Bapat
On Tue, Jul 30, 2024 at 5:38 PM David Rowley wrote: > > On Tue, 30 Jul 2024 at 21:12, Ashutosh Bapat > wrote: > > Thanks. This looks better. Nitpick > > > > +child partitions. With this setting enabled, the number of > > executor > > +nodes whose memory usage is restricted by

Re: long-standing data loss bug in initial sync of logical replication

2024-07-30 Thread Amit Kapila
On Wed, Jul 31, 2024 at 3:27 AM Masahiko Sawada wrote: > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila wrote: > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C wrote: > > > > > > On Wed, 17 Jul 2024 at 11:54, Amit Kapila wrote: > > > > > > > > On Tue, Jul 16, 2024 at 6:54 PM vignesh C wrote:

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 10:01:50PM -0500, Nathan Bossart wrote: > On Tue, Jul 30, 2024 at 07:43:08PM -0700, Andres Freund wrote: >> My point is that _xgetbv() is made available by -mavx512vpopcntdq -mavx512bw >> alone, without needing -mxsave: > > Oh, I see. I'll work on a patch to remove that

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 07:43:08PM -0700, Andres Freund wrote: > On 2024-07-30 21:01:31 -0500, Nathan Bossart wrote: >> The main purpose of the XSAVE compiler check is to determine whether we >> need to add -mxsave in order to use _xgetbv() [0]. If that wasn't a >> factor, we could probably skip

Re: Changing default -march landscape

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 07:39:18PM -0700, Andres Freund wrote: > On 2024-06-12 20:09:45 -0500, Nathan Bossart wrote: >> The idea that's been floating around recently is to build a bunch of >> different versions of Postgres and to choose one on startup based on what >> the CPU supports. That seems

Re: Popcount optimization using AVX512

2024-07-30 Thread Andres Freund
Hi, On 2024-07-30 21:01:31 -0500, Nathan Bossart wrote: > On Tue, Jul 30, 2024 at 06:46:51PM -0700, Andres Freund wrote: > > On 2024-07-30 20:20:34 -0500, Nathan Bossart wrote: > >> On Tue, Jul 30, 2024 at 05:49:59PM -0700, Andres Freund wrote: > >> > Why are we actually checking for xsave? We're

Re: Changing default -march landscape

2024-07-30 Thread Andres Freund
Hi, Reply triggered by https://postgr.es/m/ZqmRYh3iikm1Kh3D%40nathan On 2024-06-12 20:09:45 -0500, Nathan Bossart wrote: > This is perhaps only tangentially related, but I've found it really > difficult to avoid painting ourselves into a corner with this stuff. Let's > use the SSE 4.2 CRC32C

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 06:46:51PM -0700, Andres Freund wrote: > On 2024-07-30 20:20:34 -0500, Nathan Bossart wrote: >> On Tue, Jul 30, 2024 at 05:49:59PM -0700, Andres Freund wrote: >> > Why are we actually checking for xsave? We're not using xsave itself and I >> > couldn't find a comment in

Re: Popcount optimization using AVX512

2024-07-30 Thread Andres Freund
Hi, On 2024-07-30 20:20:34 -0500, Nathan Bossart wrote: > On Tue, Jul 30, 2024 at 05:49:59PM -0700, Andres Freund wrote: > > Ah, I somehow thought we'd avoid the runtime check in case we determine at > > compile time we don't need any extra flags to enable the AVX512 stuff > > (similar > > to

Re: Do we still need parent column in pg_backend_memory_context?

2024-07-30 Thread Tom Lane
David Rowley writes: > I feel the bar is a bit lower for pg_backend_memory_contexts as it was > only added in v14, so it's not been around as long as pg_class had > been around in 2018 when we removed relhaspkey. Yeah, and also it's very much a developer-focused view with a limited audience.

Re: Can we rely on the ordering of paths in pathlist?

2024-07-30 Thread wenhui qiu
Hi Richard Guo Today is the last day of the commitfest cycle.I think this patch should be commented ,Except for the comments, I tested it good to me Thanks wenhui qiu 于2024年7月25日周四 16:18写道: > Hi Richard Guo > Is it necessary to add some comments here? > > > + if

Re: Do we still need parent column in pg_backend_memory_context?

2024-07-30 Thread David Rowley
On Wed, 31 Jul 2024 at 12:35, Tom Lane wrote: > Perhaps you can make an argument that nobody would be depending > on that column, but I fear that's wishful thinking. Or maybe you > can argue that any query using it is already broken --- but I > think that's only true if someone tries to do the

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 05:49:59PM -0700, Andres Freund wrote: > Ah, I somehow thought we'd avoid the runtime check in case we determine at > compile time we don't need any extra flags to enable the AVX512 stuff (similar > to how we deal with crc32). But it looks like that's not the case - which >

Re: Popcount optimization using AVX512

2024-07-30 Thread Thomas Munro
On Wed, Jul 31, 2024 at 12:50 PM Andres Freund wrote: > It's one thing for the avx512 path to have that overhead, but it's > particularly absurd for pg_popcount32/pg_popcount64, where > > a) The function call overhead is a larger proportion of the cost. > b) the instruction is almost universally

Re: Seq scan instead of index scan querying single row from primary key on large table

2024-07-30 Thread Andy Fan
Tom Lane writes: > I'm not sure what to do about this. I think that things might work > out better if we redefined the startup cost as "estimated cost to > retrieve the first tuple", rather than its current very-squishy > definition as "cost to initialize the scan". Actually I wanted to raise

Re: Popcount optimization using AVX512

2024-07-30 Thread Andres Freund
Hi, On 2024-07-30 16:32:07 -0500, Nathan Bossart wrote: > On Tue, Jul 30, 2024 at 02:07:01PM -0700, Andres Freund wrote: > > Now, a reasonable counter-argument would be that only some of these macros > > are > > defined for msvc ([1]). However, as it turns out, the test is broken > > today, as

Re: Do we still need parent column in pg_backend_memory_context?

2024-07-30 Thread Tom Lane
David Rowley writes: > On Wed, 31 Jul 2024 at 05:19, Melih Mutlu wrote: >> After the patch [1] that adds a path column to pg_backend_memory_context, >> the parent context can also be found in the path array. Since there are >> currently two ways to retrieve information related to the parent of

Re: Do we still need parent column in pg_backend_memory_context?

2024-07-30 Thread David Rowley
On Wed, 31 Jul 2024 at 05:19, Melih Mutlu wrote: > After the patch [1] that adds a path column to pg_backend_memory_context, the > parent context can also be found in the path array. Since there are currently > two ways to retrieve information related to the parent of a context, I wonder >

Re: Comment in portal.h

2024-07-30 Thread Andy Fan
Etsuro Fujita writes: > Hi, > > I noticed $SUBJECT while working on something else: > > /* > * Where we store tuples for a held cursor or a PORTAL_ONE_RETURNING or > * PORTAL_UTIL_SELECT query. (A cursor held past the end of its > * transaction no longer has any active

Re: race condition when writing pg_control

2024-07-30 Thread Noah Misch
On Mon, Jul 15, 2024 at 03:44:48PM +1200, Thomas Munro wrote: > On Fri, Jul 12, 2024 at 11:43 PM Noah Misch wrote: > > On Sat, May 18, 2024 at 05:29:12PM +1200, Thomas Munro wrote: > > > On Fri, May 17, 2024 at 4:46 PM Thomas Munro > > > wrote: > > > > The specific problem here is that

Re: can we mark upper/lower/textlike functions leakproof?

2024-07-30 Thread David Rowley
On Wed, 31 Jul 2024 at 09:35, Andrew Dunstan wrote: > Fast forward to now. The customer has found no observable ill effects of > marking these functions leakproof. The would like to know if there is > any reason why we can't mark them leakproof, so that they don't have to > do this in every

Re: xid_wraparound tests intermittent failure.

2024-07-30 Thread Masahiko Sawada
On Tue, Jul 30, 2024 at 3:29 AM Andrew Dunstan wrote: > > > On 2024-07-29 Mo 5:25 PM, Masahiko Sawada wrote: > > I've attached the patch. Could you please test if the patch fixes the > instability you observed? > > Since we turn off autovacuum on all three tests and we wait for > autovacuum to

Re: long-standing data loss bug in initial sync of logical replication

2024-07-30 Thread Masahiko Sawada
On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila wrote: > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C wrote: > > > > On Wed, 17 Jul 2024 at 11:54, Amit Kapila wrote: > > > > > > On Tue, Jul 16, 2024 at 6:54 PM vignesh C wrote: > > > > > > BTW, I noticed that we don't take any table-level locks for

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 04:32:07PM -0500, Nathan Bossart wrote: > On Tue, Jul 30, 2024 at 02:07:01PM -0700, Andres Freund wrote: >> Afaict we could just check for predefined preprocessor macros: >> >> echo|time gcc -c -mxsave -mavx512vpopcntdq -mavx512bw -xc -dM -E - -o >> -|grep -E

Re: Popcount optimization using AVX512

2024-07-30 Thread Nathan Bossart
On Tue, Jul 30, 2024 at 02:07:01PM -0700, Andres Freund wrote: > I've noticed that the configure probes for this are quite slow - pretty much > the slowest step in a meson setup (and autoconf is similar). While looking > into this, I also noticed that afaict the tests don't do the right thing for

Re: Popcount optimization using AVX512

2024-07-30 Thread Andres Freund
Hi, On 2024-04-23 11:02:07 -0500, Nathan Bossart wrote: > On Thu, Apr 18, 2024 at 05:13:58PM -0500, Nathan Bossart wrote: > > Makes sense, thanks. I'm planning to commit this fix sometime early next > > week. > > Committed. I've noticed that the configure probes for this are quite slow - pretty

Re: [17+] check after second call to pg_strnxfrm is too strict, relax it

2024-07-30 Thread Heikki Linnakangas
On 30/07/2024 22:31, Jeff Davis wrote: I noticed this comment in selfuncs.c regarding strxfrm(): /* * Some systems (e.g., glibc) can return a smaller value from the * second call than the first; thus the Assert must be <= not ==. */ Some callers of pg_strnxfrm() are not

Re: meson vs windows perl

2024-07-30 Thread Andrew Dunstan
On 2024-07-20 Sa 9:41 AM, Andrew Dunstan wrote: On 2024-05-28 Tu 6:13 PM, Andres Freund wrote: Hi, On 2024-04-05 16:12:12 -0400, Andrew Dunstan wrote: OK, this has been fixed and checked. The attached is what I propose. The perl command is pretty hard to read. What about using python's

Re: Support LIKE with nondeterministic collations

2024-07-30 Thread Peter Eisentraut
On 27.07.24 00:32, Paul A Jungwirth wrote: On Thu, Jun 27, 2024 at 11:31 PM Peter Eisentraut wrote: Here is an updated patch for this. I took a look at this. I added some tests and found a few that give the wrong result (I believe). The new tests are included in the attached patch, along

Re: Minor refactorings to eliminate some static buffers

2024-07-30 Thread Heikki Linnakangas
On 30/07/2024 18:44, Robert Haas wrote: On Tue, Jul 30, 2024 at 7:22 AM Heikki Linnakangas wrote: As part of the multithreading work, it'd be nice to get rid of as many global or static variables as possible. Remaining ones can be converted to thread locals as appropriate, but where possible,

Re: tiny step toward threading: reduce dependence on setlocale()

2024-07-30 Thread Jeff Davis
On Mon, 2024-07-29 at 21:45 +0200, Peter Eisentraut wrote: > I have also re-reviewed the patches and I agree they are good to go. I found a couple issues with the later patches: * There was still some confusion about the default collation vs. datcollate/datctype for callers of wchar2c

Re: New compiler warnings in buildfarm

2024-07-30 Thread Ranier Vilela
Em ter., 30 de jul. de 2024 às 13:19, Tom Lane escreveu: > Sometime in the last month or so, flaviventris's bleeding-edge > version of gcc has started whining[1] about truncation of a > string literal's implicit trailing '\0' in contexts like this: > >

Re: Suboptimal spinlock code due to volatile

2024-07-30 Thread Robert Haas
On Tue, Jul 30, 2024 at 3:46 AM Heikki Linnakangas wrote: > Aside from performance, I find "volatile" difficult to reason about. I > feel more comfortable with atomics and memory barriers. I think nearly everyone feels more comfortable with atomics and memory barriers. The semantics of volatile

Re: Minor refactorings to eliminate some static buffers

2024-07-30 Thread Robert Haas
On Tue, Jul 30, 2024 at 7:22 AM Heikki Linnakangas wrote: > As part of the multithreading work, it'd be nice to get rid of as many > global or static variables as possible. Remaining ones can be converted > to thread locals as appropriate, but where possible, it's better to just > get rid of

Re: pg_verifybackup: TAR format backup verification

2024-07-30 Thread Robert Haas
On Mon, Jul 22, 2024 at 7:53 AM Amul Sul wrote: > Fix in the attached version. First of all, in the interest of full disclosure, I suggested this project to Amul, so I'm +1 on the concept. I think making more of our backup-related tools work with tar and compressed tar formats -- and perhaps

Re: Document DateStyle effect on jsonpath string()

2024-07-30 Thread David E. Wheeler
On Jul 19, 2024, at 10:22, David E. Wheeler wrote: > > Here’s a rebase on 5784a49. I also updated the commitfest item[1] to link to > a new pull request[2], since I seem to have turned the other one into the tz > conversion bug fix. > > [1]: https://commitfest.postgresql.org/49/5101/ > [2]:

Re: jsonpath: Inconsistency of timestamp_tz() Output

2024-07-30 Thread David E. Wheeler
On Jul 30, 2024, at 07:59, Andrew Dunstan wrote: > I have pushed this. Thank you, Andrew! D

Re: speed up a logical replica setup

2024-07-30 Thread Euler Taveira
On Tue, Jul 30, 2024, at 7:35 AM, Peter Eisentraut wrote: > On 30.07.24 02:05, Euler Taveira wrote: > > I'm attaching a patch that implements option (c). While reading the code > > I noticed that I left a comment that should be removed by commit > > b9639138262. 0002 removes it. > > I have

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2024 at 4:56 PM shveta malik wrote: > > On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar wrote: > > > > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: > > > > Comment in 0002, > > > > 1) I do not see any test case that set a proper conflict type and > > conflict resolver, all

Re: WIP: parallel GiST index builds

2024-07-30 Thread Tomas Vondra
On 7/30/24 13:31, Andrey M. Borodin wrote: > > >> On 30 Jul 2024, at 14:57, Tomas Vondra wrote: >> >>> >>> How do we synchronize Shared Fake LSN with global XLogCtl->unloggedLSN? >>> Just bump XLogCtl->unloggedLSN if necessary? >>> Perhaps, consider using GetFakeLSNForUnloggedRel() instead

Re: PG17beta2: SMGR: inconsistent type for nblocks

2024-07-30 Thread Thomas Munro
On Tue, Jul 30, 2024 at 11:24 PM Matthias van de Meent wrote: > While working on rebasing the patches of Neon's fork onto the > REL_17_STABLE branch, I noticed that the nblocks arguments of various > smgr functions have inconsistent types: smgrzeroextend accepts > `nblocks` as signed integer, as

Re: Add mention of execution time memory for enable_partitionwise_* GUCs

2024-07-30 Thread David Rowley
On Tue, 30 Jul 2024 at 21:12, Ashutosh Bapat wrote: > Thanks. This looks better. Nitpick > > +child partitions. With this setting enabled, the number of executor > +nodes whose memory usage is restricted by work_mem > > This sentence appears to say that the memory usage of "all"

Re: jsonpath: Inconsistency of timestamp_tz() Output

2024-07-30 Thread Andrew Dunstan
On 2024-07-22 Mo 3:12 AM, Jeevan Chalke wrote: On Fri, Jul 19, 2024 at 7:35 PM David E. Wheeler wrote: On Jul 10, 2024, at 11:19, David E. Wheeler wrote: > Oh, and the time and date were wrong, too, because I blindly used the same conversion for dates as for timestamps.

Re: WIP: parallel GiST index builds

2024-07-30 Thread Andrey M. Borodin
> On 30 Jul 2024, at 14:57, Tomas Vondra wrote: > >> >> How do we synchronize Shared Fake LSN with global XLogCtl->unloggedLSN? Just >> bump XLogCtl->unloggedLSN if necessary? >> Perhaps, consider using GetFakeLSNForUnloggedRel() instead of shared >> counter? As long as we do not care

Re: Conflict Detection and Resolution

2024-07-30 Thread shveta malik
On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar wrote: > > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: > > Comment in 0002, > > 1) I do not see any test case that set a proper conflict type and > conflict resolver, all tests either give incorrect conflict > type/conflict resolver or the

Re: Remove last traces of HPPA support

2024-07-30 Thread Thomas Munro
On Tue, Jul 30, 2024 at 12:39 PM Thomas Munro wrote: > On Tue, Jul 30, 2024 at 11:16 AM Heikki Linnakangas wrote: > > Looks good to me. > > Thanks. I'll wait just a bit longer to see if anyone else has comments. And pushed. I am aware of a couple of build farm animals that will now fail

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-30 Thread Amit Kapila
On Tue, Jul 30, 2024 at 4:02 PM vignesh C wrote: > > On Thu, 25 Jul 2024 at 08:39, Amit Kapila wrote: > > > > On Wed, Jul 24, 2024 at 9:13 PM Tom Lane wrote: > > > > > > Amit Kapila writes: > > > > I merged these changes, made a few other cosmetic changes, and pushed > > > > the patch. > > >

Re: speed up a logical replica setup

2024-07-30 Thread Peter Eisentraut
On 30.07.24 02:05, Euler Taveira wrote: I'm attaching a patch that implements option (c). While reading the code I noticed that I left a comment that should be removed by commit b9639138262. 0002 removes it. I have committed your 0002 patch. It looks like Amit has already committed your 0001

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: Comment in 0002, 1) I do not see any test case that set a proper conflict type and conflict resolver, all tests either give incorrect conflict type/conflict resolver or the conflict resolver is ignored 0003 2) I was trying to think about this

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-07-30 Thread vignesh C
On Thu, 25 Jul 2024 at 08:39, Amit Kapila wrote: > > On Wed, Jul 24, 2024 at 9:13 PM Tom Lane wrote: > > > > Amit Kapila writes: > > > I merged these changes, made a few other cosmetic changes, and pushed the > > > patch. > > > > There is a CF entry pointing at this thread [1]. Should it be

Re: xid_wraparound tests intermittent failure.

2024-07-30 Thread Andrew Dunstan
On 2024-07-29 Mo 5:25 PM, Masahiko Sawada wrote: I've attached the patch. Could you please test if the patch fixes the instability you observed? Since we turn off autovacuum on all three tests and we wait for autovacuum to complete processing databases, these tests potentially have a similar

Re: WIP: parallel GiST index builds

2024-07-30 Thread Tomas Vondra
On 7/30/24 11:39, Andrey M. Borodin wrote: > > >> On 30 Jul 2024, at 14:05, Tomas Vondra wrote: >> >> >> >> On 7/26/24 14:13, Andrey M. Borodin wrote: >>> >>> On 26 Jul 2024, at 14:30, Andreas Karlsson wrote: I feel the tricky part about doing that is that we need to make

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-30 Thread Tomas Vondra
On 7/30/24 09:13, Sutou Kouhei wrote: > Hi, > > In <26541788-8853-4d93-86cd-5f701b13a...@enterprisedb.com> > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Mon, 29 Jul 2024 14:17:08 +0200, > Tomas Vondra wrote: > &

Re: WIP: parallel GiST index builds

2024-07-30 Thread Andrey M. Borodin
> On 30 Jul 2024, at 14:05, Tomas Vondra wrote: > > > > On 7/26/24 14:13, Andrey M. Borodin wrote: >> >> >>> On 26 Jul 2024, at 14:30, Andreas Karlsson wrote: >>> >>> I feel the tricky part about doing that is that we need to make sure the >>> fake LSNs are all less than the current

Re: Add mention of execution time memory for enable_partitionwise_* GUCs

2024-07-30 Thread Ashutosh Bapat
On Mon, Jul 29, 2024 at 10:37 AM David Rowley wrote: > > On Fri, 19 Jul 2024 at 17:24, Ashutosh Bapat > wrote: > > I am fine if we want to mention that the executor may consume a large > > amount of memory when these GUCs are turned ON. Users may decide to > > turn those OFF if they can not

Re: Is *fast* 32-bit support still important?

2024-07-30 Thread Aleksander Alekseev
Hi Joel, Here are my two cents. > 1. Who are the current users of 32-bit PostgreSQL? Pretty much any embedded system that uses just a few GB of memory may win from using a 32-bit processor (not necessarily in terms of performance, maybe in terms of price). Think of WiFi-routers, smart TVs, 3D

Re: Conflict detection and logging in logical replication

2024-07-30 Thread shveta malik
On Mon, Jul 29, 2024 at 11:44 AM Zhijie Hou (Fujitsu) wrote: > > Here is the V7 patch set that addressed all the comments so far[1][2][3]. Thanks for the patch, few comments: 1) build_index_value_desc() /* Assume the index has been locked */ indexDesc = index_open(indexoid,

Re: WIP: parallel GiST index builds

2024-07-30 Thread Tomas Vondra
On 7/26/24 14:13, Andrey M. Borodin wrote: > > >> On 26 Jul 2024, at 14:30, Andreas Karlsson wrote: >> >> I feel the tricky part about doing that is that we need to make sure the >> fake LSNs are all less than the current real LSN when the index build >> completes and while that normally

Re: Flush pgstats file during checkpoints

2024-07-30 Thread Bertrand Drouvot
Hi, On Tue, Jul 30, 2024 at 03:25:31PM +0900, Michael Paquier wrote: > On Mon, Jul 29, 2024 at 04:46:17AM +, Bertrand Drouvot wrote: > > Thanks! 0001 attached is > > v4-0001-Revert-Test-that-vacuum-removes-tuples-older-than.patch > > so I guess you did not attached the right one. > > I did

Re: Conflict detection and logging in logical replication

2024-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2024 at 1:49 PM Zhijie Hou (Fujitsu) wrote: > > > On Monday, July 29, 2024 6:59 PM Dilip Kumar > > wrote: > > > > > > On Mon, Jul 29, 2024 at 11:44 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > > > I was going through v7-0001, and I have some initial comments. > > > >

RE: speed up a logical replica setup

2024-07-30 Thread Hayato Kuroda (Fujitsu)
Dear Euler, I applied your patch and confirmed it could fix the issue [1]. METHOD === 1. shortened the snapshot interval to 3ms, and ``` -#define LOG_SNAPSHOT_INTERVAL_MS 15000 +#define LOG_SNAPSHOT_INTERVAL_MS 3 ``` 2. ran test 040_pg_createsubscriber.pl many times. Before patching, I

Re: COPY FROM crash

2024-07-30 Thread David Rowley
On Tue, 30 Jul 2024 at 19:06, Zhang Mingli wrote: > I’ve updated the comments in patch v1 with an additional Assert. Thanks. I adjusted a little and used a StaticAssert instead then pushed. StaticAssert seems better as invalid values will result in compilation failure. David

RE: Conflict detection and logging in logical replication

2024-07-30 Thread Zhijie Hou (Fujitsu)
> On Monday, July 29, 2024 6:59 PM Dilip Kumar > wrote: > > > > On Mon, Jul 29, 2024 at 11:44 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > I was going through v7-0001, and I have some initial comments. > > Thanks for the comments ! > > > > > @@ -536,11 +542,9 @@

Re: Short-circuit sort_inner_and_outer if there are no mergejoin clauses

2024-07-30 Thread Richard Guo
On Fri, Apr 26, 2024 at 2:57 PM Richard Guo wrote: > I doubt that there is measurable performance improvement. But I found > that throughout the run of the regression tests, sort_inner_and_outer is > called a total of 44,424 times. Among these calls, there are 11,064 > instances where

Re: Suboptimal spinlock code due to volatile

2024-07-30 Thread Heikki Linnakangas
On 29/07/2024 22:59, Andres Freund wrote: After being confused for a while, the explanation is fairly simple: We use volatile and dereference the address: static __inline__ int tas(volatile slock_t *lock) { slock_t _res = 1; __asm__ __volatile__( " lock

Re: Trivial revise for the check of parameterized partial paths

2024-07-30 Thread Richard Guo
On Thu, Jan 25, 2024 at 3:21 PM Richard Guo wrote: > On Sun, Jan 21, 2024 at 8:36 PM vignesh C wrote: >> I'm seeing that there has been no activity in this thread for nearly 7 >> months, I'm planning to close this in the current commitfest unless >> someone is planning to take it forward. > > >

Re: A problem about partitionwise join

2024-07-30 Thread Richard Guo
On Wed, May 8, 2024 at 5:01 PM Richard Guo wrote: > Below is what I got on my local machine. > > -- on master > > measurement | average | maximum | minimum | std_dev | > std_dev_as_perc_of_avg > ---+--+--+--+-+ > planning

Re: Is *fast* 32-bit support still important?

2024-07-30 Thread Heikki Linnakangas
On 29/07/2024 23:40, Joel Jacobson wrote: To me, it's non-obvious whether introducing `#if SIZEOF_DATUM < 8` with separate 32-bit and 64-bit code paths is worthwhile to maintain performance for both. Knowing more about $subject can hopefully help us reason about how much additional code

Re: Use pgBufferUsage for block reporting in analyze

2024-07-30 Thread Anthonin Bonnefoy
Hi, On Tue, Jul 30, 2024 at 1:13 AM Masahiko Sawada wrote: > I have one comment on 0001 patch: > The comment should also be updated or removed. Ok, I've removed the comment. > However, as the above comment says, delay_in_ms can have a duration up > to 25 days. I guess it would not be a problem

Re: COPY FROM crash

2024-07-30 Thread Zhang Mingli
Hi, Zhang Mingli www.hashdata.xyz On Jul 30, 2024 at 13:37 +0800, Kirill Reshke , wrote: > > Just to be clear, you are facing this on HEAD, on on REL_14_STABLE? Postgres HEAD.

Re: COPY FROM crash

2024-07-30 Thread Zhang Mingli
Hi, Zhang Mingli www.hashdata.xyz On Jul 30, 2024 at 13:35 +0800, David Rowley , wrote: > > If you're only able > to get it to misbehave by setting MAX_PARTITION_BUFFERS to less than > 2, then my suggested fix would be to add a comment to say that values > less than to are not supported. Right.

Re: Separate HEAP WAL replay logic into its own file

2024-07-30 Thread Li, Yong
> I think that this proposal is reasonable but we need to get > attention from a committer to move forward this proposal. > > > Thanks, > — > kou Thank you Kou for your review. I will move the CF to the next phase and see what happens. Regards, Yong

Re: Restart pg_usleep when interrupted

2024-07-30 Thread Bertrand Drouvot
Hi, On Mon, Jul 29, 2024 at 06:15:49PM -0500, Sami Imseih wrote: > > On Jul 26, 2024, at 3:27 AM, Bertrand Drouvot > > wrote: > > 3 === > > > > I gave more thoughts and I think it can be simplified a bit to reduce the > > number of operations in the while loop. > > > > What about relying on a

Re: psql: Add leakproof field to \dAo+ meta-command results

2024-07-30 Thread Yugo NAGATA
Hi, On Tue, 30 Jul 2024 01:36:55 +0200 Erik Wienhold wrote: > On 2024-07-01 15:08 +0200, Yugo NAGATA wrote: > > I would like to propose to add a new field to psql's \dAo+ meta-command > > to show whether the underlying function of an operator is leak-proof. > > +1 for making that info easily

Re: Flush pgstats file during checkpoints

2024-07-30 Thread Michael Paquier
On Mon, Jul 29, 2024 at 04:46:17AM +, Bertrand Drouvot wrote: > Thanks! 0001 attached is > v4-0001-Revert-Test-that-vacuum-removes-tuples-older-than.patch > so I guess you did not attached the right one. I did attach the right set of patches, please ignore 0001 entirely: the patch series is

Re: 040_pg_createsubscriber.pl is slow and unstable (was Re: speed up a logical replica setup)

2024-07-30 Thread Amit Kapila
On Tue, Jul 30, 2024 at 11:28 AM Ashutosh Bapat wrote: > > On Tue, Jul 30, 2024 at 9:25 AM Amit Kapila wrote: > > > > On Tue, Jul 30, 2024 at 1:48 AM Tom Lane wrote: > > > > > > Robert Haas writes: > > > > On Sun, Jun 30, 2024 at 2:40 PM Tom Lane wrote: > > > >> ... However, I added a new

Re: 040_pg_createsubscriber.pl is slow and unstable (was Re: speed up a logical replica setup)

2024-07-29 Thread Ashutosh Bapat
On Tue, Jul 30, 2024 at 9:25 AM Amit Kapila wrote: > > On Tue, Jul 30, 2024 at 1:48 AM Tom Lane wrote: > > > > Robert Haas writes: > > > On Sun, Jun 30, 2024 at 2:40 PM Tom Lane wrote: > > >> ... However, I added a new open item about how the > > >> 040_pg_createsubscriber.pl test is slow and

Re: Separate HEAP WAL replay logic into its own file

2024-07-29 Thread Sutou Kouhei
Hi, In <599e67d2-2929-4858-b8bc-f9c4ae889...@ebay.com> "Re: Separate HEAP WAL replay logic into its own file" on Fri, 26 Jul 2024 07:56:12 +, "Li, Yong" wrote: >> 1. Could you create your patch by "git format-patch -vN master" >> or so

Re: COPY FROM crash

2024-07-29 Thread Kirill Reshke
Hi! On Tue, 30 Jul 2024 at 08:52, Zhang Mingli wrote: > > Hi, all > > I got a crash when copy partition tables with mass data in Cloudberry > DB[0](based on Postgres14.4, Greenplum 7). > > I have a test on Postgres and it has the similar issue(different places but > same function). Just to be

Re: COPY FROM crash

2024-07-29 Thread David Rowley
On Tue, 30 Jul 2024 at 15:52, Zhang Mingli wrote: > I have a test on Postgres and it has the similar issue(different places but > same function). > > However it’s a little hard to reproduce because it happened when inserting > next tuple after a previous copy multi insert buffer is flushed. > >

Re: speed up a logical replica setup

2024-07-29 Thread Amit Kapila
On Tue, Jul 30, 2024 at 9:26 AM Hayato Kuroda (Fujitsu) wrote: > > > Hayato already mentioned one of the solution in a previous email [2]. > > AFAICS we can use any solution that creates a WAL record. I tested the > > following options: > > Yes, my point was that we should add an arbitrary record

Re: Incremental View Maintenance, take 2

2024-07-29 Thread Yugo NAGATA
Hi, On Tue, 30 Jul 2024 03:32:19 +0500 Kirill Reshke wrote: > On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > > > Hi! > > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > > use matview) feature, so i got interested in how it is implemented. Thank you so much for

Re: pgsql: Add more SQL/JSON constructor functions

2024-07-29 Thread Amit Langote
Hi, On Fri, Jul 26, 2024 at 11:19 PM jian he wrote: > On Fri, Jul 26, 2024 at 4:53 PM Amit Langote wrote: > > > > > > Pushed 0003-0005 ahead of 0001-0002. Will try to push them over the > > weekend. Rebased for now. Pushed them now. > { > ... > /* > * For expression nodes that

Re: 040_pg_createsubscriber.pl is slow and unstable (was Re: speed up a logical replica setup)

2024-07-29 Thread Amit Kapila
On Tue, Jul 30, 2024 at 9:56 AM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Jul 30, 2024 at 1:48 AM Tom Lane wrote: > >> If the problem can be correctly described as "pg_createsubscriber > >> takes 10 seconds or so to detect end-of-stream", > > > The problem can be defined as:

Re: 040_pg_createsubscriber.pl is slow and unstable (was Re: speed up a logical replica setup)

2024-07-29 Thread Tom Lane
Amit Kapila writes: > On Tue, Jul 30, 2024 at 1:48 AM Tom Lane wrote: >> If the problem can be correctly described as "pg_createsubscriber >> takes 10 seconds or so to detect end-of-stream", > The problem can be defined as: "pg_createsubscriber waits for an > additional (new) WAL record to be

Re: Conflict Detection and Resolution

2024-07-29 Thread shveta malik
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: >> > Please find v7 patch-set, the changes are: > Thanks Ajin for working on this. Please find few comments: 1) parse_subscription_conflict_resolvers(): Here we loop in this function to find the given conflict type in the supported list and

RE: speed up a logical replica setup

2024-07-29 Thread Hayato Kuroda (Fujitsu)
Dear Euler, > Hayato already mentioned one of the solution in a previous email [2]. > AFAICS we can use any solution that creates a WAL record. I tested the > following options: Yes, my point was that we should add an arbitrary record after the recovery_target_lsn. > (a) temporary replication

Re: 040_pg_createsubscriber.pl is slow and unstable (was Re: speed up a logical replica setup)

2024-07-29 Thread Amit Kapila
On Tue, Jul 30, 2024 at 1:48 AM Tom Lane wrote: > > Robert Haas writes: > > On Sun, Jun 30, 2024 at 2:40 PM Tom Lane wrote: > >> ... However, I added a new open item about how the > >> 040_pg_createsubscriber.pl test is slow and still unstable. > > > But that said, I see no commits in the

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-07-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Sun, 28 Jul 2024 22:49:47 +0800, Junwang Zhao wrote: > Thanks for updating the patches, I applied them and test > in my local machine, I did not use tmpfs in my test, I guess > if I r

Re: Remove last traces of HPPA support

2024-07-29 Thread Thomas Munro
On Tue, Jul 30, 2024 at 11:16 AM Heikki Linnakangas wrote: > On 30/07/2024 00:50, Thomas Munro wrote: > > On Wed, Jul 3, 2024 at 8:09 PM Tom Lane wrote: > >> Thomas Munro writes: > > OK, part on ice for now. Here's an update of the rest, > > this time also removing the barrier fallbacks as

Re: speed up a logical replica setup

2024-07-29 Thread Euler Taveira
On Mon, Jul 29, 2024, at 6:11 PM, Euler Taveira wrote: > The options are: > > (a) temporary replication slot: requires an additional replication slot. > small payload. it is extremely slow in comparison with the other > options. > (b) logical message: can be consumed by logical replication

<    1   2   3   4   5   6   7   8   9   10   >