Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread David G. Johnston
On Wed, Oct 25, 2023 at 7:13 PM David Rowley wrote: > On Thu, 26 Oct 2023 at 13:10, David G. Johnston > wrote: > > Question: Do you know whether we for certain always sort ascending here > to compute the unique values or whether if, say, there is an index on the > column in descending order (or

Re: Is this a problem in GenericXLogFinish()?

2023-10-25 Thread Jeff Davis
On Thu, 2023-10-26 at 07:00 +0300, Alexander Lakhin wrote: > It looks like the buffer is not dirty in the problematic call. Thank you for the report! I was able to reproduce and observe that the buffer is not marked dirty. The call (hashovfl.c:671): XLogRegisterBuffer(1, wbuf,

Re: Open a streamed block for transactional messages during decoding

2023-10-25 Thread Amit Kapila
On Tue, Oct 24, 2023 at 5:27 PM Zhijie Hou (Fujitsu) wrote: > > While reviewing the test_decoding code, I noticed that when skip_empty_xacts > option is specified, it doesn't open the streaming block( e.g. > pg_output_stream_start) before streaming the transactional MESSAGE even if > it's > the

Re: A performance issue with Memoize

2023-10-25 Thread Andrei Lepikhov
On 20/10/2023 17:40, Richard Guo wrote: I noticed $subject with the query below. set enable_memoize to off; explain (analyze, costs off) select * from tenk1 t1 left join lateral     (select t1.two as t1two, * from tenk1 t2 offset 0) s on t1.two = s.two;                                      

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-10-25 Thread David Rowley
On Thu, 26 Oct 2023 at 08:43, Tom Lane wrote: > I think that we can make that assumption starting with v17. > Back-patching it would be hazardous perhaps; but if there's some > function out there that depends on NUL termination, testing should > expose it before too long. Wouldn't hurt to

Re: Is this a problem in GenericXLogFinish()?

2023-10-25 Thread Alexander Lakhin
Hello hackers, 24.10.2023 03:45, Jeff Davis wrote: Committed. I've encountered a case with a hash index, when an assert added by the commit fails: CREATE TABLE t (i INT); CREATE INDEX hi ON t USING HASH (i); INSERT INTO t SELECT 1 FROM generate_series(1, 1000); BEGIN; INSERT INTO t SELECT 1

Re: Add null termination to string received in parallel apply worker

2023-10-25 Thread David Rowley
On Wed, 11 Oct 2023 at 19:54, Zhijie Hou (Fujitsu) wrote: > The parallel apply worker didn't add null termination to the string received > from the leader apply worker via the shared memory queue. This action doesn't > bring bugs as it's binary data but violates the rule established in >

Re: Remove dead code in pg_ctl.c

2023-10-25 Thread David Steele
On 10/25/23 17:30, Nathan Bossart wrote: On Wed, Oct 25, 2023 at 03:02:01PM -0500, Nathan Bossart wrote: On Wed, Oct 25, 2023 at 02:53:31PM -0400, David Steele wrote: It looks like this code was missed in 39969e2a when exclusive backup was removed. Indeed. I'll plan on committing this

Re: remaining sql/json patches

2023-10-25 Thread Amit Langote
Hi Nikita, On Thu, Oct 26, 2023 at 2:13 AM Nikita Malakhov wrote: > Amit, on previous email, patch #2 - I agree that it is not the best idea to > introduce > new type of logic into the parser, so this logic could be moved to the > executor, > or removed at all. What do you think of these

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread David Rowley
On Thu, 26 Oct 2023 at 13:10, David G. Johnston wrote: > Question: Do you know whether we for certain always sort ascending here to > compute the unique values or whether if, say, there is an index on the column > in descending order (or ascending and traversed backwards) that the data >

Re: Introduce a new view for checkpointer related stats

2023-10-25 Thread Michael Paquier
On Mon, Feb 13, 2023 at 11:31:03AM +0530, Bharath Rupireddy wrote: > Needed a rebase. Please review the attached v8 patch set. I was looking at this patch, and got a few comments. FWIW, I kind of agree with the feeling of Bertrand upthread that using "checkpoint_" in the attribute names for the

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread Bruce Momjian
On Wed, Oct 25, 2023 at 05:10:17PM -0700, David G. Johnston wrote: > The paragraph leading into the last added example needs to be tweaked: > > If DISTINCT is specified within an aggregate, the data is sorted in ascending > order while extracting unique values.  You can add an ORDER BY clause,

Re: libpq async connection and multiple hosts

2023-10-25 Thread Daniele Varrazzo
On Thu, 26 Oct 2023, 00:10 Jelte Fennema, wrote: > On Wed, 25 Oct 2023 at 18:54, Daniele Varrazzo > wrote: > > - connect_timeout > > - multiple host, hostaddr, port > > - load_balance_hosts=random > > > > Does this list sound complete? > > I think you'd also want to resolve the hostnames to IPs

Re: Should we represent temp files as unsigned long int instead of signed long int type?

2023-10-25 Thread Tom Lane
Michael Paquier writes: > In the mood of removing long because it may be 4 bytes or 8 bytes > depending on the environment, I'd suggest to change it to either int64 > or uint64. Not that it matters much for this specific case, but that > makes the code more portable. Then you're going to need a

Re: Should we represent temp files as unsigned long int instead of signed long int type?

2023-10-25 Thread Michael Paquier
On Wed, Oct 25, 2023 at 03:07:39PM -0400, Tom Lane wrote: > AFAIK, nothing particularly awful will happen if that counter wraps > around. Perhaps if you gamed the system really hard, you could cause > a collision with a still-extant temp file from the previous cycle, > but I seriously doubt that

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread David G. Johnston
On Wed, Oct 25, 2023 at 4:22 PM Bruce Momjian wrote: > On Wed, Oct 25, 2023 at 04:14:11PM -0700, David G. Johnston wrote: > > Yeah, we punt on the entire concept in the data type section: > > > > "Managing these errors and how they propagate through calculations is the > > subject of an entire

Re: pg_stat_statements and "IN" conditions

2023-10-25 Thread Michael Paquier
On Tue, Oct 17, 2023 at 10:15:41AM +0200, Dmitry Dolgov wrote: > In the current patch version I didn't add anything yet to address the > question of having more parameters to tune constants merging. The main > obstacle as I see it is that the information for that has to be > collected when

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread Bruce Momjian
On Wed, Oct 25, 2023 at 04:14:11PM -0700, David G. Johnston wrote: > Yeah, we punt on the entire concept in the data type section: > > "Managing these errors and how they propagate through calculations is the > subject of an entire branch of mathematics and computer science and will not > be >

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread David G. Johnston
On Wed, Oct 25, 2023 at 8:36 AM Bruce Momjian wrote: > On Tue, Oct 24, 2023 at 06:45:48PM -0700, David G. Johnston wrote: > > I'd prefer to keep pointing out that the ones documented are those whose > > outputs will vary due to ordering. > > Okay, I re-added it in the attached patch, and

Re: Container Types

2023-10-25 Thread Andres Freund
Hi, On 2023-10-25 15:03:04 -0700, Jeff Davis wrote: > On Tue, 2022-12-20 at 10:24 +0100, Vik Fearing wrote: > > Obviously there would have to be an actual type in order to store it > > in > > a table, but what I am most interested in here is being able to > > create > > them on the fly.  I do

Re: post-recovery amcheck expectations

2023-10-25 Thread Peter Geoghegan
On Tue, Oct 24, 2023 at 8:05 PM Noah Misch wrote: > Can't it still happen if the sequence of unfortunately timed crashes causes > deletions from left to right? Take this example, expanding the one above. > Half-kill 4, crash, half-kill 3, crash, half-kill 2 in: > > * 1 > *

Re: Improving btree performance through specializing by key shape, take 2

2023-10-25 Thread Peter Geoghegan
On Mon, Sep 25, 2023 at 9:13 AM Matthias van de Meent wrote: > I think it's fairly complete, and mostly waiting for review. > > > I don't have time to do a comprehensive (or even a fairly > > cursory) analysis of which parts of the patch are helping, and which > > are marginal or even add no

Re: libpq async connection and multiple hosts

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 18:54, Daniele Varrazzo wrote: > - connect_timeout > - multiple host, hostaddr, port > - load_balance_hosts=random > > Does this list sound complete? I think you'd also want to resolve the hostnames to IPs yourself and iterate over those one-by-one. Otherwise if the first

Re: Partial aggregates pushdown

2023-10-25 Thread Bruce Momjian
On Tue, Oct 24, 2023 at 12:12:41AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > Hi Mr.Momjian. > > > Fujii-san, to get this patch closer to finished, can I modify this version > > of the patch to improve some wording and post an > > updated version you can use for future changes? >

Re: Container Types

2023-10-25 Thread Jeff Davis
On Tue, 2022-12-20 at 10:24 +0100, Vik Fearing wrote: > Obviously there would have to be an actual type in order to store it > in > a table, but what I am most interested in here is being able to > create > them on the fly.  I do not think it is feasible to create N new types > for every type

Re: Remove dead code in pg_ctl.c

2023-10-25 Thread Nathan Bossart
On Wed, Oct 25, 2023 at 03:02:01PM -0500, Nathan Bossart wrote: > On Wed, Oct 25, 2023 at 02:53:31PM -0400, David Steele wrote: >> It looks like this code was missed in 39969e2a when exclusive backup was >> removed. > > Indeed. I'll plan on committing this shortly. Committed. -- Nathan

Re: Guiding principle for dropping LLVM versions?

2023-10-25 Thread Thomas Munro
On Wed, Oct 25, 2023 at 7:12 PM Tom Lane wrote: > Thomas Munro writes: > > 3. We exclude OSes that don't bless an LLVM release (eg macOS running > > an arbitrarily picked version), and animals running only to cover > > ancient LLVM compiled from source for coverage (Andres's sid > > menagerie).

Re: doc: a small improvement about pg_am description

2023-10-25 Thread Jeff Davis
On Wed, 2023-10-25 at 17:25 +0900, Yugo NAGATA wrote: > It seems to me that this description says pg_am contains only > index access methods but not table methods. I wonder it is missed > to fix this when tableam was supported and other documentation > was changed in

Re: walwriter interacts quite badly with synchronous_commit=off

2023-10-25 Thread Heikki Linnakangas
On 25/10/2023 21:59, Andres Freund wrote: On 2023-10-25 12:17:03 +0300, Heikki Linnakangas wrote: On 25/10/2023 02:09, Andres Freund wrote: Because of the inherent delay between the checks of XLogCtl->WalWriterSleeping and Latch->is_set, we also sometimes end up with multiple processes

Re: Remove dead code in pg_ctl.c

2023-10-25 Thread Nathan Bossart
On Wed, Oct 25, 2023 at 02:53:31PM -0400, David Steele wrote: > It looks like this code was missed in 39969e2a when exclusive backup was > removed. Indeed. I'll plan on committing this shortly. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: POC, WIP: OR-clause support for indexes

2023-10-25 Thread Robert Haas
On Sat, Oct 14, 2023 at 6:37 PM Alexander Korotkov wrote: > Regarding the GUC parameter, I don't see we need a limit. It's not > yet clear whether a small number or a large number of OR clauses are > more favorable for transformation. I propose to have just a boolean > enable_or_transformation

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-10-25 Thread Tom Lane
David Rowley writes: > I've attached a patch which builds on the previous patch and relaxes > the rule that the StringInfo must be NUL-terminated. The rule is > only relaxed for StringInfos that are initialized with > initReadOnlyStringInfo. Yeah, that's probably a reasonable way to frame it.

Re: trying again to get incremental backup

2023-10-25 Thread Robert Haas
On Wed, Oct 25, 2023 at 3:17 PM Andrew Dunstan wrote: > OK, I'll go with that. It will actually be a bit less invasive than the > patch I posted. Why's that? -- Robert Haas EDB: http://www.enterprisedb.com

Re: trying again to get incremental backup

2023-10-25 Thread Andrew Dunstan
On 2023-10-25 We 11:24, Robert Haas wrote: On Wed, Oct 25, 2023 at 10:33 AM Andrew Dunstan wrote: I'm not too worried about the maintenance burden. That said, I agree that JSON might not be the best format for backup manifests, but maybe that ship has sailed. I think it's a decision we

Re: Should we represent temp files as unsigned long int instead of signed long int type?

2023-10-25 Thread Robert Haas
On Wed, Oct 25, 2023 at 1:28 PM Ashutosh Sharma wrote: > At present, we represent temp files as a signed long int number. And > depending on the system architecture (32 bit or 64 bit), the range of > signed long int varies, for example on a 32-bit system it will range > from -2,147,483,648 to

Re: Should we represent temp files as unsigned long int instead of signed long int type?

2023-10-25 Thread Tom Lane
Ashutosh Sharma writes: > At present, we represent temp files as a signed long int number. And > depending on the system architecture (32 bit or 64 bit), the range of > signed long int varies, for example on a 32-bit system it will range > from -2,147,483,648 to 2,147,483,647. AFAIU, this will

Re: Custom tstzrange with importance factored in

2023-10-25 Thread Jeff Davis
On Fri, 2023-10-06 at 16:55 +0300, Rares Pop (Treelet) wrote: > I essentially want to be able to aggregate multiple tstzranges - each > range with its own importance. The aggregation would be like a a > join/intersect where ranges with higher importance override the ones > with lower importance.

Re: walwriter interacts quite badly with synchronous_commit=off

2023-10-25 Thread Andres Freund
Hi, On 2023-10-25 12:17:03 +0300, Heikki Linnakangas wrote: > On 25/10/2023 02:09, Andres Freund wrote: > > Because of the inherent delay between the checks of > > XLogCtl->WalWriterSleeping > > and Latch->is_set, we also sometimes end up with multiple processes > > signalling > > walwriter,

Remove dead code in pg_ctl.c

2023-10-25 Thread David Steele
Hackers, It looks like this code was missed in 39969e2a when exclusive backup was removed. Regards, -Daviddiff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c index 807d7023a99..4099d240e03 100644 --- a/src/bin/pg_ctl/pg_ctl.c +++ b/src/bin/pg_ctl/pg_ctl.c @@ -96,7 +96,6 @@ static

Does UCS_BASIC have the right CTYPE?

2023-10-25 Thread Jeff Davis
UCS_BASIC is defined in the standard as a collation based on comparing the code point values, and in UTF8 that is satisfied with memcmp(), so the collation locale for UCS_BASIC in Postgres is simply "C". But what should the result of UPPER('á' COLLATE UCS_BASIC) be? In Postgres, the answer is

Re: ResourceOwner refactoring

2023-10-25 Thread Andres Freund
Hi, On 2023-10-25 15:43:36 +0300, Heikki Linnakangas wrote: > On 10/07/2023 22:14, Andres Freund wrote: > > > /* > > > - * Initially allocated size of a ResourceArray. Must be power of two > > > since > > > - * we'll use (arraysize - 1) as mask for hashing. > > > + * Size of the fixed-size

Re: race condition in pg_class

2023-10-25 Thread Tom Lane
Smolkin Grigory writes: > We are running PG13.10 and recently we have encountered what appears to be > a bug due to some race condition between ALTER TABLE ... ADD CONSTRAINT and > some other catalog-writer, possibly ANALYZE. > The problem is that after successfully creating index on relation

Should we represent temp files as unsigned long int instead of signed long int type?

2023-10-25 Thread Ashutosh Sharma
Hi All, At present, we represent temp files as a signed long int number. And depending on the system architecture (32 bit or 64 bit), the range of signed long int varies, for example on a 32-bit system it will range from -2,147,483,648 to 2,147,483,647. AFAIU, this will not allow a session to

Re: remaining sql/json patches

2023-10-25 Thread Nikita Malakhov
Hi! Amit, on previous email, patch #2 - I agree that it is not the best idea to introduce new type of logic into the parser, so this logic could be moved to the executor, or removed at all. What do you think of these options? On Wed, Oct 18, 2023 at 5:19 AM jian he wrote: > Hi. > based on v22.

Re: libpq async connection and multiple hosts

2023-10-25 Thread Daniele Varrazzo
On Wed, 25 Oct 2023 at 17:35, Jelte Fennema wrote: > Another approach is to use tcp_user_timeout instead of connect_timeout > to skip non-responsive hosts. It's not completely equivalent though to > connection_timeout though, since it also applies when the connection > is actually being used.

Re: libpq async connection and multiple hosts

2023-10-25 Thread Daniele Varrazzo
On Wed, 25 Oct 2023 at 17:35, Jelte Fennema wrote: > You should implement load_balance_hosts=random though > by randomizing your hosts list. Good catch. So it seems that, if someone wants to build an equivalent an async version of PQconnectdb, they need to handle on their own: -

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-25 Thread Nathan Bossart
+pg_crc32c +pg_comp_crc32c_with_vmull_armv8(pg_crc32c crc, const void *data, size_t len) It looks like most of this function is duplicated from pg_comp_crc32c_armv8(). I understand that we probably need a separate function because of the runtime check, but perhaps we could create a common static

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-25 Thread Bruce Momjian
On Tue, Oct 24, 2023 at 06:45:48PM -0700, David G. Johnston wrote: > I'd prefer to keep pointing out that the ones documented are those whose > outputs will vary due to ordering. Okay, I re-added it in the attached patch, and tightened up the text. > I've been sympathetic to the user comments

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-25 Thread Andrei Zubkov
Hi Alena, On Wed, 2023-10-25 at 16:25 +0300, Alena Rybakina wrote: >  Hi! Thank you for your work on the subject. > 1. I didn't understand why we first try to find pgssEntry with a > false top_level value, and later find it with a true top_level value. In case of pg_stat_statements the top_level

Re: libpq async connection and multiple hosts

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 17:03, Daniele Varrazzo wrote: > However, ISTM that connecting to multiple hosts is not supported > either. I have a couple of issues I am looking into in psycopg 3: > > - https://github.com/psycopg/psycopg/issues/602 > - https://github.com/psycopg/psycopg/issues/674

Re: trying again to get incremental backup

2023-10-25 Thread Robert Haas
On Wed, Oct 25, 2023 at 10:33 AM Andrew Dunstan wrote: > I'm not too worried about the maintenance burden. > > That said, I agree that JSON might not be the best format for backup > manifests, but maybe that ship has sailed. I think it's a decision we could walk back if we had a good enough

Re: Synchronizing slots from primary to standby

2023-10-25 Thread Drouvot, Bertrand
Hi, On 10/9/23 12:30 PM, shveta malik wrote: PFA v22 patch-set. It has below changes: patch 001: 1) Now physical walsender wakes up logical walsender(s) by using a new CV as suggested in [1] Thanks! I think that works fine as long as the standby is up and running and catching up. The

libpq async connection and multiple hosts

2023-10-25 Thread Daniele Varrazzo
Hello, We are aware that, using async connection functions (`PQconnectStart`, `PQconnectPoll`), the `connect_timeout` parameter is not supported; this is documented at https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PQCONNECTSTARTPARAMS """ The connect_timeout connection

Re: [dynahash] do not refill the hashkey after hash_search

2023-10-25 Thread Nathan Bossart
On Wed, Oct 25, 2023 at 12:48:52PM +0700, John Naylor wrote: > On Wed, Oct 25, 2023 at 12:21 PM Tom Lane wrote: >> >> John Naylor writes: >> > I'd prefer just adding "Assert(hentry->event == oldn);" and declaring >> > hentry PG_USED_FOR_ASSERTS_ONLY. >> >> I'm not aware of any other places where

Re: [patch] pg_basebackup: mention that spread checkpoints are the default in --help

2023-10-25 Thread Peter Eisentraut
On 19.10.23 11:39, Michael Banck wrote: Hi, I believed that spread (not fast) checkpoints are the default in pg_basebackup, but noticed that --help does not specify which is which - contrary to the reference documentation. So I propose the small attached patch to clarify that. > printf(_("

Re: trying again to get incremental backup

2023-10-25 Thread Andrew Dunstan
On 2023-10-25 We 09:05, Robert Haas wrote: On Wed, Oct 25, 2023 at 7:54 AM Andrew Dunstan wrote: Robert asked me to work on this quite some time ago, and most of this work was done last year. Here's my WIP for an incremental JSON parser. It works and passes all the usual json/b tests. It

Re: Add connection active, idle time to pg_stat_activity

2023-10-25 Thread Aleksander Alekseev
Hi, > On Wed, 2023-10-25 at 16:17 +0300, Aleksander Alekseev wrote: > > On top of that not sure if I see the patch on the November commitfest > > [1]. Please make sure it's there so that cfbot will check the patch. > > Yes, this patch is listed on the November commitfest. cfbot says rebase >

Re: Add connection active, idle time to pg_stat_activity

2023-10-25 Thread Andrei Zubkov
Hi Aleksander, On Wed, 2023-10-25 at 16:17 +0300, Aleksander Alekseev wrote: > On top of that not sure if I see the patch on the November commitfest > [1]. Please make sure it's there so that cfbot will check the patch. Yes, this patch is listed on the November commitfest. cfbot says rebase

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-25 Thread Alena Rybakina
On 19.10.2023 15:40, Andrei Zubkov wrote: Hi hackers, New version 23 attached. It contains rebase to the current master. Noted that v1.11 adds new fields to the pg_stat_sstatements view, but leaves the PGSS_FILE_HEADER constant unchanged. It this correct? Hi! Thank you for your work on the

Re: Add connection active, idle time to pg_stat_activity

2023-10-25 Thread Aleksander Alekseev
Hi, > I've done a review of this patch. I found the patch idea very useful, > thank you for the patch. I've noted something observing this patch: > 1. Patch can't be applied on the current master. My review is based on >application of this patch over ac68323a878 On top of that not sure if I

Re: Add connection active, idle time to pg_stat_activity

2023-10-25 Thread Andrei Zubkov
Hi Sergey, I've done a review of this patch. I found the patch idea very useful, thank you for the patch. I've noted something observing this patch: 1. Patch can't be applied on the current master. My review is based on application of this patch over ac68323a878 2. Being applied over

Re: trying again to get incremental backup

2023-10-25 Thread Robert Haas
On Wed, Oct 25, 2023 at 7:54 AM Andrew Dunstan wrote: > Robert asked me to work on this quite some time ago, and most of this > work was done last year. > > Here's my WIP for an incremental JSON parser. It works and passes all > the usual json/b tests. It implements Algorithm 4.3 in the Dragon

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-25 Thread Andrei Zubkov
Hi Andrei, On Wed, 2023-10-25 at 13:59 +0700, Andrei Lepikhov wrote: > But minmax_stats_since and changes in the UI of the reset routine > look like syntactic sugar here. > I can't convince myself that it is really needed. Do you have some > set of cases that can enforce the changes proposed?

Re: race condition in pg_class

2023-10-25 Thread Andrey M. Borodin
> On 25 Oct 2023, at 13:39, Smolkin Grigory wrote: > > We are running PG13.10 and recently we have encountered what appears to be a > bug due to some race condition between ALTER TABLE ... ADD CONSTRAINT and > some other catalog-writer, possibly ANALYZ > I've tried to reproduce this

Re: RFC: Pluggable TOAST

2023-10-25 Thread Aleksander Alekseev
Hi Nikita, > We need community feedback on previously discussed topic [1]. > There are some long-live issues in Postgres related to the TOAST mechanics, > like [2]. > Some time ago we already proposed a set of patches with an API allowing to > plug in > different TOAST implementations into a

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-25 Thread Dilip Kumar
On Wed, Oct 25, 2023 at 5:58 PM Amit Kapila wrote: > > On Fri, Oct 20, 2023 at 9:40 AM Dilip Kumar wrote: > > > > On Sat, Oct 14, 2023 at 9:43 AM Amit Kapila wrote: > > > > > > This and other results shared by you look promising. Will there be any > > > improvement in workloads related to clog

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 13:52, Alvaro Herrera wrote: > Looking at for_each_ptr() I think it may be cleaner to follow > palloc_object()'s precedent and make it foreach_object() instead (I have > no love for the extra underscore, but I won't object to it either). And > like foreach_node, have it

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-25 Thread Amit Kapila
On Fri, Oct 20, 2023 at 9:40 AM Dilip Kumar wrote: > > On Sat, Oct 14, 2023 at 9:43 AM Amit Kapila wrote: > > > > This and other results shared by you look promising. Will there be any > > improvement in workloads related to clog buffer usage? > > I did not understand this question can you

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-10-25 Thread Alvaro Herrera
On 2023-Oct-25, tender wang wrote: > Hi >Is there any conclusion to this issue? None yet. I intend to work on this at some point, hopefully soon. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

Re: trying again to get incremental backup

2023-10-25 Thread Andrew Dunstan
On 2023-10-24 Tu 12:08, Robert Haas wrote: It looks like each file entry in the manifest takes about 150 bytes, so 1 GB would allow for 1024**3/150 = 7158278 files. That seems fine for now? I suspect a few people have more files than that. They'll just have to Maybe someone on the list can

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Alvaro Herrera
On 2023-Oct-24, Jelte Fennema wrote: > Many usages of the foreach macro in the Postgres codebase only use the > ListCell variable to then get its value. This adds macros that > simplify iteration code for that very common use case. Instead of > passing a ListCell you can pass a variable of the

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-10-25 Thread tender wang
Hi Is there any conclusion to this issue? Jehan-Guillaume de Rorthais 于2023年8月10日周四 23:03写道: > On Thu, 3 Aug 2023 11:02:43 +0200 > Alvaro Herrera wrote: > > > On 2023-Aug-03, tender wang wrote: > > > > > I think old "sub-FK" should not be dropped, that will be violates > foreign > > > key

Re: ResourceOwner refactoring

2023-10-25 Thread Heikki Linnakangas
On 10/07/2023 15:37, Peter Eisentraut wrote: A few suggestions on the API: > +static ResourceOwnerFuncs tupdesc_resowner_funcs = These aren't all "functions", so maybe another word like "info" or "description" would be more appropriate? > + .release_phase =

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-25 Thread Ashutosh Bapat
On Wed, Oct 25, 2023 at 4:23 PM José Neves wrote: > > Ok, I see. In that situation is safe indeed, as the offset is lower than the > current transaction commit. > But I think that I asked the wrong question. I guess that the right question > is: Can we receive a keep-alive message with an LSN

Re: POC, WIP: OR-clause support for indexes

2023-10-25 Thread a.rybakina
Hi! On 15.10.2023 01:34, Alexander Korotkov wrote: Hi, Alena! Thank you for your work on the subject. On Wed, Oct 4, 2023 at 10:21 PM a.rybakina wrote: I fixed the kernel dump issue and all the regression tests were successful, but I discovered another problem when I added my own

RE: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-25 Thread José Neves
Ok, I see. In that situation is safe indeed, as the offset is lower than the current transaction commit. But I think that I asked the wrong question. I guess that the right question is: Can we receive a keep-alive message with an LSN offset bigger than the commit of the open or following

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-10-25 Thread Ashutosh Bapat
On Tue, Oct 24, 2023 at 8:53 PM José Neves wrote: > > Hi there, hope to find you well. > > I have a follow-up question to this already long thread. > > Upon deploying my PostgreSQL logical replication fed application on a stale > database, I ended up running out of space, as the replication slot

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
Attached is a slightly updated version, with a bit simpler implementation of foreach_delete_current. Instead of decrementing i and then adding 1 to it when indexing the list, it now indexes the list using a postfix decrement. From 9073777a6d82e2b2db8b1ed9aef200550234d89a Mon Sep 17 00:00:00 2001

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 04:55, David Rowley wrote: > With foreach(), we commonly do "if (lc == NULL)" at the end of loops > as a way of checking if we did "break" to terminate the loop early. Afaict it's done pretty infrequently. The following crude attempt at an estimate estimates it's only done

Re: Synchronizing slots from primary to standby

2023-10-25 Thread Drouvot, Bertrand
Hi, On 10/25/23 6:57 AM, Amit Kapila wrote: On Tue, Oct 24, 2023 at 3:35 PM Drouvot, Bertrand wrote: On 10/24/23 7:44 AM, Ajin Cherian wrote: On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand wrote: 2) When we create a subscription, another slot is created during the subscription

Re: Use virtual tuple slot for Unique node

2023-10-25 Thread Ashutosh Bapat
On Tue, Oct 24, 2023 at 4:30 AM David Rowley wrote: > > On Fri, 20 Oct 2023 at 22:30, Ashutosh Bapat > wrote: > > I ran my experiments again. It seems on my machine the execution times > > do vary a bit. I ran EXPLAIN ANALYZE on the query 5 times and took > > average of execution times. I did

Re: Synchronizing slots from primary to standby

2023-10-25 Thread Drouvot, Bertrand
Hi, On 10/25/23 5:00 AM, shveta malik wrote: On Tue, Oct 24, 2023 at 11:54 AM Drouvot, Bertrand wrote: Hi, On 10/23/23 2:56 PM, shveta malik wrote: On Mon, Oct 23, 2023 at 5:52 PM Drouvot, Bertrand wrote: We are waiting for DEFAULT_NAPTIME_PER_CYCLE (3 minutes) before checking if

Re: run pgindent on a regular basis / scripted manner

2023-10-25 Thread Amit Kapila
On Tue, Oct 24, 2023 at 6:21 AM Jeff Davis wrote: > > On Wed, 2023-10-18 at 22:34 +1300, David Rowley wrote: > > It would be good to learn how many of the committers out of the ones > > you listed that --enable-indent-checks would have saved from breaking > > koel. > > I'd find that a useful

Re: walwriter interacts quite badly with synchronous_commit=off

2023-10-25 Thread Heikki Linnakangas
On 25/10/2023 02:09, Andres Freund wrote: Because of the inherent delay between the checks of XLogCtl->WalWriterSleeping and Latch->is_set, we also sometimes end up with multiple processes signalling walwriter, which can be bad, because it increases the likelihood that some of the signals may be

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-25 Thread Bharath Rupireddy
On Wed, Oct 25, 2023 at 1:50 PM Amit Kapila wrote: > > It would be better to gauge its value separately and add it once the > main patch is committed. > There should be a way to avoid this but we can decide it afterwards. I > don't want to hold the main patch for this point. What do you think?

doc: a small improvement about pg_am description

2023-10-25 Thread Yugo NAGATA
Hi, When reading the documentation about operator class, I found the following description: The pg_am table contains one row for every index access method. Support for access to regular tables is built into PostgreSQL, but all index access methods are described in pg_am. It seems to me

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-25 Thread Amit Kapila
On Wed, Oct 25, 2023 at 1:39 PM Bharath Rupireddy wrote: > > On Wed, Oct 25, 2023 at 11:39 AM Amit Kapila wrote: > > > > On Tue, Oct 24, 2023 at 1:20 PM Bharath Rupireddy > > wrote: > > > > > > > > > I spent some time on the v57 patch and it looks good to me - tests are > > > passing, no

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-25 Thread Bharath Rupireddy
On Wed, Oct 25, 2023 at 11:39 AM Amit Kapila wrote: > > On Tue, Oct 24, 2023 at 1:20 PM Bharath Rupireddy > wrote: > > > > > > I spent some time on the v57 patch and it looks good to me - tests are > > passing, no complaints from pgindent and pgperltidy. I turned the CF > > entry

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-25 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Based on your advice, I revised the patch again. > > > > I spent some time on the v57 patch and it looks good to me - tests are > > passing, no complaints from pgindent and pgperltidy. I turned the CF > > entry https://commitfest.postgresql.org/45/4273/ to RfC. > > > > Thanks, the

Re: Bug: RLS policy FOR SELECT is used to check new rows

2023-10-25 Thread Laurenz Albe
On Tue, 2023-10-24 at 15:05 -0400, Stephen Frost wrote: > On Tue, Oct 24, 2023 at 14:42 Robert Haas wrote: > > On Tue, Oct 24, 2023 at 1:46 PM Jeff Davis wrote: > > > Perhaps the idea is that if there are constraints involved, the failure > > > or success of an INSERT/UPDATE/DELETE could leak

Re: Simplify create_merge_append_path a bit for clarity

2023-10-25 Thread Richard Guo
On Tue, Oct 24, 2023 at 6:00 PM Alena Rybakina wrote: > I agree with you, and we can indeed directly set the param_info value to > NULL, and there are enough comments here to explain. > > I didn't find anything else to add in your patch. Thanks for reviewing this patch! Thanks Richard

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-25 Thread Andrei Lepikhov
On 19/10/2023 19:40, Andrei Zubkov wrote: Hi hackers, New version 23 attached. It contains rebase to the current master. I discovered the patch and parameters you've proposed. In my opinion, the stats_since parameter adds valuable information and should definitely be included in the stats

Re: A performance issue with Memoize

2023-10-25 Thread Richard Guo
On Fri, Oct 20, 2023 at 7:43 PM Pavel Stehule wrote: > +1 > > it would be great to fix this problem - I've seen this issue a few times. > Thanks for the input. I guess this is not rare in the real world. If the subquery contains lateral reference to a Var that also appears in the subquery's

Re: A performance issue with Memoize

2023-10-25 Thread Richard Guo
On Fri, Oct 20, 2023 at 6:40 PM Richard Guo wrote: > I haven't thought thoroughly about the fix yet. But one way I'm > thinking is that in create_subqueryscan_plan() we can first add the > subquery's subplan_params to root->curOuterParams, and then replace > outer-relation Vars in scan_clauses

Re: SQL:2011 application time

2023-10-25 Thread jian he
On Wed, Oct 11, 2023 at 12:47 PM Paul Jungwirth wrote: > > On 9/25/23 14:00, Peter Eisentraut wrote: > > Looking through the tests in v16-0001: > > > > +-- PK with no columns just WITHOUT OVERLAPS: > > +CREATE TABLE temporal_rng ( > > + valid_at tsrange, > > + CONSTRAINT

Re: ALTER COLUMN ... SET EXPRESSION to alter stored generated column's expression

2023-10-25 Thread Amul Sul
Here is the rebase version for the latest master head(673a17e3120). I haven't done any other changes related to the ON UPDATE trigger since that seems non-trivial; need a bit of work to add trigger support in ATRewriteTable(). Also, I am not sure yet, if we were doing these changes, and the

Re: Guiding principle for dropping LLVM versions?

2023-10-25 Thread Tom Lane
Thomas Munro writes: > Here are some systematic rules I'd like to propose to anchor this > stuff to reality and avoid future doubt and litigation: > 1. Build farm animals testing LLVM determine the set of OSes and LLVM > versions we consider. > 2. We exclude OSes that will be out of full

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-25 Thread Amit Kapila
On Tue, Oct 24, 2023 at 1:20 PM Bharath Rupireddy wrote: > > > I spent some time on the v57 patch and it looks good to me - tests are > passing, no complaints from pgindent and pgperltidy. I turned the CF > entry https://commitfest.postgresql.org/45/4273/ to RfC. > Thanks, the patch looks mostly