Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Amit Kapila
On Fri, Oct 27, 2023 at 11:24 AM Bharath Rupireddy wrote: > > On Fri, Oct 27, 2023 at 10:10 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Here is a patch for fixing to 003_logical_slots. Also, I got a comment off > > list so that it was included. > > > > ``` > > -# Setup a pg_upgrade command.

RE: pg_upgrade's object listing

2023-10-26 Thread Zhijie Hou (Fujitsu)
On Friday, October 27, 2023 1:21 PM Kyotaro Horiguchi wrote: > > Hello. > > I found the following message recently introduced in pg_upgrade: > > > pg_log(PG_VERBOSE, "slot_name: \"%s\", plugin: \"%s\", > two_phase: %s", > >slot_info->slotname, > >

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 10:10 AM Hayato Kuroda (Fujitsu) wrote: > > Here is a patch for fixing to 003_logical_slots. Also, I got a comment off > list so that it was included. > > ``` > -# Setup a pg_upgrade command. This will be used anywhere. > +# Setup a common pg_upgrade command to be used by

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 11:09 AM Amit Kapila wrote: > > On Fri, Oct 27, 2023 at 10:43 AM Michael Paquier wrote: > > > > -"invalid_logical_replication_slots.txt"); > > +"invalid_logical_slots.txt"); > > > > Or you could do something even shorter,

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Hayato Kuroda (Fujitsu)
Dear Michael, > Or you could do something even shorter, with "invalid_slots.txt". I think current one seems better, because we only support logical replication slots for now. We can extend as you said when we support physical slot as well. Also, proposed length is sufficient for fairywren [1].

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Amit Kapila
On Fri, Oct 27, 2023 at 10:43 AM Michael Paquier wrote: > > On Fri, Oct 27, 2023 at 04:40:43AM +, Hayato Kuroda (Fujitsu) wrote: > > Yeah, Bharath has already reported, I agreed that the reason was [1]. > > > > ``` > > In the Windows API (with some exceptions discussed in the following > >

Re: A recent message added to pg_upgade

2023-10-26 Thread Kyotaro Horiguchi
At Fri, 27 Oct 2023 09:51:43 +0530, Amit Kapila wrote in > On Fri, Oct 27, 2023 at 9:37 AM Peter Smith wrote: > > IIUC the only possible way to reach this error (according to the > > comment preceding it) is by the user overriding the GUC value (which > > was defaulted -1) on the command line.

Re: Introduce a new view for checkpointer related stats

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 10:32 AM Michael Paquier wrote: > > On Fri, Oct 27, 2023 at 10:23:34AM +0530, Bharath Rupireddy wrote: > > A possible way is to move existing pgstat_count_slru_flush in > > SimpleLruWriteAll closer to pg_fsync and WAIT_EVENT_SLRU_SYNC in > > SlruPhysicalWritePage, remove

Re: A recent message added to pg_upgade

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 9:52 AM Amit Kapila wrote: > > > errmsg("\"max_slot_wal_keep_size\" must be set to -1 during the upgrade"), > > errhint("Do not override \"max_slot_wal_keep_size\" using command line > > options.")); > > > > But OTOH, we don't have a value of user-passed options to ensure

pg_upgrade's object listing

2023-10-26 Thread Kyotaro Horiguchi
Hello. I found the following message recently introduced in pg_upgrade: > pg_log(PG_VERBOSE, "slot_name: \"%s\", plugin: \"%s\", > two_phase: %s", > slot_info->slotname, > slot_info->plugin, >

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Michael Paquier
On Fri, Oct 27, 2023 at 04:40:43AM +, Hayato Kuroda (Fujitsu) wrote: > Yeah, Bharath has already reported, I agreed that the reason was [1]. > > ``` > In the Windows API (with some exceptions discussed in the following > paragraphs), > the maximum length for a path is MAX_PATH, which is

Re: maybe a type_sanity. sql bug

2023-10-26 Thread Michael Paquier
On Fri, Oct 27, 2023 at 11:45:44AM +0800, jian he wrote: > The test seems to assume the following sql query should return zero row. > but it does not. I don't know much about the "relreplident" column. This is not about relreplident here, that refers to a relation's replica identity. > test1=#

Re: Introduce a new view for checkpointer related stats

2023-10-26 Thread Michael Paquier
On Fri, Oct 27, 2023 at 10:23:34AM +0530, Bharath Rupireddy wrote: > A possible way is to move existing pgstat_count_slru_flush in > SimpleLruWriteAll closer to pg_fsync and WAIT_EVENT_SLRU_SYNC in > SlruPhysicalWritePage, remove WAIT_EVENT_SLRU_FLUSH_SYNC completely, > use WAIT_EVENT_SLRU_SYNC in

Re: A recent message added to pg_upgade

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 9:36 AM Amit Kapila wrote: > > On Fri, Oct 27, 2023 at 8:52 AM Bharath Rupireddy > wrote: > > > > On Fri, Oct 27, 2023 at 8:28 AM Kyotaro Horiguchi: > > The above errhint LGTM. How about a slightly different errmsg, like > > the following? > > > > +

Re: Introduce a new view for checkpointer related stats

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 8:03 AM Michael Paquier wrote: > > Hmm. As per the existing call of pgstat_count_slru_flush() in > SimpleLruWriteAll(), routine called SimpleLruFlush() until ~13 and > dee663f78439, an incrementation of 1 for slru_stats_idx refers to all > the flushes for all the dirty

Re: race condition in pg_class

2023-10-26 Thread Noah Misch
On Wed, Oct 25, 2023 at 01:39:41PM +0300, Smolkin Grigory wrote: > We are running PG13.10 and recently we have encountered what appears to be > a bug due to some race condition between ALTER TABLE ... ADD CONSTRAINT and > some other catalog-writer, possibly ANALYZE. > The problem is that after

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Hayato Kuroda (Fujitsu)
Dear Bharath, Amit, Peter, Thank you for discussing! A patch can be available in [1]. > > > > +1 for > s/003_upgrade_logical_replication_slots.pl/003_upgrade_logical_slots.pl > > > > and s/invalid_logical_replication_slots.txt/invalid_logical_slots.txt. > > > > +1. The proposed file name sounds

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Hayato Kuroda (Fujitsu)
Dear Hou, > The BF animal fairywren[1] failed when testing > 003_upgrade_logical_replication_slots.pl. Good catch! > > The reason could be the length of this path(262) exceed the windows path > limit(260 IIRC). If so, I recall we fixed similar things before (e213de8e7) by > reducing the path

pg_dump not dumping the run_as_owner setting from version 16?

2023-10-26 Thread Philip Warner
Hi, I as far as I can tell, pg_dump does not dup the ‘run_as_owner` setting for a subscription. Should it? Should I submit a patch? It seems pretty trivial to fix if anyone else is working on it. Sent from Mail for Windows

Re: A recent message added to pg_upgade

2023-10-26 Thread Amit Kapila
On Fri, Oct 27, 2023 at 9:37 AM Peter Smith wrote: > > On Fri, Oct 27, 2023 at 1:58 PM Kyotaro Horiguchi > wrote: > > > > Hello. > > > > Some messages recently introduced by commit 29d0a77fa6 seem to deviate > > slightly from our standards. > > > > + if (*invalidated &&

Re: A recent message added to pg_upgade

2023-10-26 Thread Peter Smith
On Fri, Oct 27, 2023 at 1:58 PM Kyotaro Horiguchi wrote: > > Hello. > > Some messages recently introduced by commit 29d0a77fa6 seem to deviate > slightly from our standards. > > + if (*invalidated && SlotIsLogical(s) && IsBinaryUpgrade) > + { > +

Re: A recent message added to pg_upgade

2023-10-26 Thread Amit Kapila
On Fri, Oct 27, 2023 at 8:52 AM Bharath Rupireddy wrote: > > On Fri, Oct 27, 2023 at 8:28 AM Kyotaro Horiguchi: > The above errhint LGTM. How about a slightly different errmsg, like > the following? > > +errmsg("cannot invalidate replication slots when > in binary upgrade

maybe a type_sanity. sql bug

2023-10-26 Thread jian he
hi. The test seems to assume the following sql query should return zero row. but it does not. I don't know much about the "relreplident" column. https://git.postgresql.org/cgit/postgresql.git/tree/src/test/regress/expected/type_sanity.out#n499 demo: https://dbfiddle.uk/QFM88S2e test1=# \dt Did

Re: A recent message added to pg_upgade

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 8:28 AM Kyotaro Horiguchi wrote: > > Hello. > > Some messages recently introduced by commit 29d0a77fa6 seem to deviate > slightly from our standards. > > + if (*invalidated && SlotIsLogical(s) && IsBinaryUpgrade) > + { > +

Re: Use virtual tuple slot for Unique node

2023-10-26 Thread David Rowley
On Wed, 25 Oct 2023 at 22:48, Ashutosh Bapat wrote: > We may save the size of data in VirtualTupleTableSlot, thus avoiding > the first loop. I assume that when allocating > VirtualTupleTableSlot->data, we always know what size we are > allocating so it should be just a matter of saving it in >

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Bharath Rupireddy
On Fri, Oct 27, 2023 at 8:06 AM Amit Kapila wrote: > > > > +1 for > > > s/003_upgrade_logical_replication_slots.pl/003_upgrade_logical_slots.pl > > > and s/invalid_logical_replication_slots.txt/invalid_logical_slots.txt. > > +1. The proposed file name sounds reasonable. > > Agreed. So, how about

Re: Partial aggregates pushdown

2023-10-26 Thread Bruce Momjian
On Fri, Oct 27, 2023 at 02:44:42AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > Hi Momjian. > > Thank you for your improvement. > As a matter of detail, I think that the areas marked below are erroneous. > > -- > + Pushdown causes aggregate function cals to send partial aggregate >

Re: Document parameter count limit

2023-10-26 Thread Bruce Momjian
On Thu, Oct 26, 2023 at 04:17:19PM -0700, David G. Johnston wrote: > On Thu, Oct 26, 2023 at 4:13 PM David G. Johnston > wrote: > > On Thu, Oct 26, 2023 at 4:08 PM Tom Lane wrote: > > Bruce Momjian writes: > > Ah, I was confused.  I documented both in the attached patch. >

A recent message added to pg_upgade

2023-10-26 Thread Kyotaro Horiguchi
Hello. Some messages recently introduced by commit 29d0a77fa6 seem to deviate slightly from our standards. + if (*invalidated && SlotIsLogical(s) && IsBinaryUpgrade) + { + ereport(ERROR, +

RE: Partial aggregates pushdown

2023-10-26 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Momjian. Thank you for your improvement. As a matter of detail, I think that the areas marked below are erroneous. -- + Pushdown causes aggregate function cals to send partial aggregate ^ + function calls to the remote server. If the partial

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Amit Kapila
On Fri, Oct 27, 2023 at 3:28 AM Peter Smith wrote: > > On Fri, Oct 27, 2023 at 2:26 AM Bharath Rupireddy > wrote: > > > > On Thu, Oct 26, 2023 at 8:11 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > The BF animal fairywren[1] failed when testing > > > 003_upgrade_logical_replication_slots.pl. >

Re: Introduce a new view for checkpointer related stats

2023-10-26 Thread Michael Paquier
On Thu, Oct 26, 2023 at 10:55:00PM +0530, Bharath Rupireddy wrote: > On Thu, Oct 26, 2023 at 7:30 AM Michael Paquier wrote: >> Why is that in 0002? Isn't that something we should treat as a bug >> fix of its own, even backpatching it to make sure that the flush >> requests for individual

Re: Is this a problem in GenericXLogFinish()?

2023-10-26 Thread Michael Paquier
On Thu, Oct 26, 2023 at 09:40:09AM -0400, Robert Haas wrote: > Because of this, it is possible for bucketbuf, prevbuf, and wbuf to be > the same (your first scenario) but the second scenario you mention > (nextbuf == wbuf) should be impossible. Okay.. > It seems to me that maybe we shouldn't

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Jeff Davis
On Thu, 2023-10-26 at 17:32 -0400, Tom Lane wrote: > For starters, C locale should certainly act different from others. Agreed. ctype of "C" is 100% stable (as implemented in Postgres with special ASCII-only semantics) and simple. I'm looking for a way to offer a new middle ground between plain

Re: Document parameter count limit

2023-10-26 Thread Tom Lane
"David G. Johnston" writes: > Cleanups for consistency: > Move "identifier length" after "partition keys" (before the new "function > arguments") Yeah, the existing ordering of this table seems quite random. That would help some, by separating items having to do with database/table size from

Re: Document parameter count limit

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 4:13 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Thu, Oct 26, 2023 at 4:08 PM Tom Lane wrote: > >> Bruce Momjian writes: >> > Ah, I was confused. I documented both in the attached patch. >> >> The function one should have the same annotation as some

Re: Document parameter count limit

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 4:08 PM Tom Lane wrote: > Bruce Momjian writes: > > Ah, I was confused. I documented both in the attached patch. > > The function one should have the same annotation as some others: > > can be increased by recompiling > PostgreSQL > > I'd like to see a comment on

Re: Document parameter count limit

2023-10-26 Thread Tom Lane
Bruce Momjian writes: > Ah, I was confused. I documented both in the attached patch. The function one should have the same annotation as some others: can be increased by recompiling PostgreSQL regards, tom lane

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 4:03 PM Bruce Momjian wrote: > > Sure, done in the attached patch. > > WFM. Thank You! David J.

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread Bruce Momjian
On Thu, Oct 26, 2023 at 03:44:14PM -0700, David G. Johnston wrote: > On Thu, Oct 26, 2023 at 3:36 PM Bruce Momjian wrote: > > No sneaking.  ;-)  It would be bad to document this unevenly because it > sets expectations in other parts of the system if we don't mention it. > > > Agreed. >

Re: Document parameter count limit

2023-10-26 Thread Bruce Momjian
On Thu, Oct 26, 2023 at 06:56:40PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > Here is a patch to add this. > > "function arguments" seems like a completely wrong description > (and if we do want to document that limit, it's 100). > > "query parameters" would work, perhaps. Ah, I was

Re: Document parameter count limit

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 3:51 PM Bruce Momjian wrote: > On Wed, Nov 23, 2022 at 02:33:27PM -0600, Justin Pryzby wrote: > > On Wed, Nov 23, 2022 at 12:35:59PM -0700, David G. Johnston wrote: > > > On Wed, Nov 23, 2022 at 11:47 AM Tom Lane wrote: > > > > > > > Bruce Momjian writes: > > > > > Does

Re: Document parameter count limit

2023-10-26 Thread Tom Lane
Bruce Momjian writes: > Here is a patch to add this. "function arguments" seems like a completely wrong description (and if we do want to document that limit, it's 100). "query parameters" would work, perhaps. regards, tom lane

Re: Document parameter count limit

2023-10-26 Thread Bruce Momjian
On Wed, Nov 23, 2022 at 02:33:27PM -0600, Justin Pryzby wrote: > On Wed, Nov 23, 2022 at 12:35:59PM -0700, David G. Johnston wrote: > > On Wed, Nov 23, 2022 at 11:47 AM Tom Lane wrote: > > > > > Bruce Momjian writes: > > > > Does this come up enough to document it? I assume the error message

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Jeff Davis
On Thu, 2023-10-26 at 23:22 +0200, Daniel Verite wrote: > Neither does Unicode, which is why the ICU functions like u_isupper() > or u_toupper() don't take a locale argument. u_strToUpper() accepts a locale argument:

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 3:36 PM Bruce Momjian wrote: > No sneaking. ;-) It would be bad to document this unevenly because it > sets expectations in other parts of the system if we don't mention it. > Agreed. Last suggestion, remove the first jsonb_agg example that lacks an order by. +WITH

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread Bruce Momjian
On Thu, Oct 26, 2023 at 03:09:26PM -0700, David G. Johnston wrote: > On Thu, Oct 26, 2023 at 2:56 PM Bruce Momjian wrote: > > On Wed, Oct 25, 2023 at 10:34:10PM -0700, David G. Johnston wrote: > > I would reword the existing note to be something like: > > > > The SQL Standard

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-10-26 Thread Bharath Rupireddy
On Wed, Oct 25, 2023 at 5:45 AM Jeff Davis wrote: > > Comments: Thanks for reviewing. > * It would be good to document that this is partially an optimization > (read from memory first) and partially an API difference that allows > reading unflushed data. For instance, walsender may benefit >

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 2:56 PM Bruce Momjian wrote: > On Wed, Oct 25, 2023 at 10:34:10PM -0700, David G. Johnston wrote: > > I would reword the existing note to be something like: > > > > The SQL Standard defines specific aggregates and their properties, > including > > which of DISTINCT and/or

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Peter Smith
On Fri, Oct 27, 2023 at 2:26 AM Bharath Rupireddy wrote: > > On Thu, Oct 26, 2023 at 8:11 PM Zhijie Hou (Fujitsu) > wrote: > > > > The BF animal fairywren[1] failed when testing > > 003_upgrade_logical_replication_slots.pl. > > > > From the log, I can see pg_upgrade failed to open the > >

Re: Document aggregate functions better w.r.t. ORDER BY

2023-10-26 Thread Bruce Momjian
On Wed, Oct 25, 2023 at 10:34:10PM -0700, David G. Johnston wrote: > I would reword the existing note to be something like: > > The SQL Standard defines specific aggregates and their properties, including > which of DISTINCT and/or ORDER BY is allowed.  Due to the extensible nature of >

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-10-26 Thread David Rowley
On Thu, 26 Oct 2023 at 17:00, David Rowley wrote: > Thanks for looking at this again. I fixed up each of those and pushed > the result, mentioning the incompatibility in the commit message. > > Now that that's done, I've attached a patch which makes use of the new > initReadOnlyStringInfo

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Tom Lane
"Daniel Verite" writes: > To me the question of what we should put in pg_collation.collctype > for the "ucs_basic" collation leads to another question which is: > why do we even consider collctype in the first place? For starters, C locale should certainly act different from others. I'm not

Re: [17] Special search_path names "!pg_temp" and "!pg_catalog"

2023-10-26 Thread Nathan Bossart
On Fri, Aug 18, 2023 at 02:44:31PM -0700, Jeff Davis wrote: > +SET search_path = admin, "!pg_temp"; I think it's unfortunate that these new identifiers must be quoted. I wonder if we could call these something like "no_pg_temp". *shrug* > + * Add any implicitly-searched namespaces to

Re: Add recovery to pg_control and remove backup_label

2023-10-26 Thread David G. Johnston
On Thu, Oct 26, 2023 at 2:02 PM David Steele wrote: > Hackers, > > This was originally proposed in [1] but that thread went through a > number of different proposals so it seems better to start anew. > > The basic idea here is to simplify and harden recovery by getting rid of > backup_label and

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Daniel Verite
Peter Eisentraut wrote: > > That seems to suggest the standard answer should be 'Á' regardless of > > any COLLATE clause (though I could be misreading). I'm a bit confused > > by that... what's the standard-compatible way to specify the locale for > > UPPER()/LOWER()? If there is none,

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-26 Thread Alena Rybakina
On 25.10.2023 18:35, Andrei Zubkov wrote: Hi Alena, On Wed, 2023-10-25 at 16:25 +0300, Alena Rybakina wrote:  Hi! Thank you for your work on the subject. 1. I didn't understand why we first try to find pgssEntry with a false top_level value, and later find it with a true top_level value. In

Re: POC, WIP: OR-clause support for indexes

2023-10-26 Thread Peter Geoghegan
On Thu, Oct 26, 2023 at 12:59 PM Robert Haas wrote: > Alexander's example seems to show that it's not that simple. If I'm > reading his example correctly, with things like aid = 1, the > transformation usually wins even if the number of things in the OR > expression is large, but with things like

Add recovery to pg_control and remove backup_label

2023-10-26 Thread David Steele
Hackers, This was originally proposed in [1] but that thread went through a number of different proposals so it seems better to start anew. The basic idea here is to simplify and harden recovery by getting rid of backup_label and storing recovery information directly in pg_control. Instead

Re: RFC: Pluggable TOAST

2023-10-26 Thread Nikita Malakhov
Hi! Matthias, thank you for your patience and explanation. I'd wish I had it much earlier, it would save a lot of time. You've asked a lot of good questions, and the answers we have for some seem to be not very satisfactory, and pointed out some topics that were not mentioned before. I have to

Recovering from detoast-related catcache invalidations

2023-10-26 Thread Tom Lane
In bug #18163 [1], Alexander proved the misgivings I had in [2] about catcache detoasting being possibly unsafe: >> BTW, while nosing around I found what seems like a very nasty related >> bug. Suppose that a catalog tuple being loaded into syscache contains >> some toasted fields.

Re: POC, WIP: OR-clause support for indexes

2023-10-26 Thread Alena Rybakina
On 26.10.2023 22:58, Robert Haas wrote: On Thu, Oct 26, 2023 at 3:47 PM Alena Rybakina wrote: With small amounts of "OR" elements, the cost of orexpr is lower than with "ANY", on the contrary, higher. Alexander's example seems to show that it's not that simple. If I'm reading his example

Re: Atomic ops for unlogged LSN

2023-10-26 Thread Nathan Bossart
On Thu, Oct 26, 2023 at 03:00:58PM +, John Morris wrote: > Keeping things up to date. Here is a rebased patch with no changes from > previous one. This patch looks a little different than the last version I see posted [0]. That last version of the patch (which appears to be just about

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-26 Thread Nathan Bossart
On Thu, Oct 26, 2023 at 08:53:31AM +, Xiang Gao wrote: > On Tue, 24 Oct, 2023 20:45:39PM -0500, Nathan Bossart wrote: >>I tried this. pg_waldump on 2 million ~8kB records took around 8.1 seconds >>without the patch and around 7.4 seconds with it (an 8% improvement). >>pg_waldump on 1

Re: POC, WIP: OR-clause support for indexes

2023-10-26 Thread Robert Haas
On Thu, Oct 26, 2023 at 3:47 PM Alena Rybakina wrote: > With small amounts of "OR" elements, the cost of orexpr is lower than with > "ANY", on the contrary, higher. Alexander's example seems to show that it's not that simple. If I'm reading his example correctly, with things like aid = 1, the

Re: visibility of open cursors in pg_stat_activity

2023-10-26 Thread Robert Haas
On Thu, Oct 26, 2023 at 1:41 PM Andres Freund wrote: > Does it really matter on that level for the user whether a snapshot exists > because of repeatable read or because of a cursor? If users don't understand > backend_xmin - likely largely true - then the consequences of holding a > snapshot

Re: POC, WIP: OR-clause support for indexes

2023-10-26 Thread Alena Rybakina
Hi! Thank you for your feedback! On 25.10.2023 22:54, Robert Haas wrote: On Sat, Oct 14, 2023 at 6:37 PM Alexander Korotkov wrote: Regarding the GUC parameter, I don't see we need a limit. It's not yet clear whether a small number or a large number of OR clauses are more favorable for

Re: Partial aggregates pushdown

2023-10-26 Thread Bruce Momjian
On Thu, Oct 26, 2023 at 11:11:09AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > >and checks if the remote server version is older than the local > > server version. > >If so, > >postgres_fdw > > -->assumes that for each built-in aggregate function, the

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Jeff Davis
On Thu, 2023-10-26 at 09:21 -0700, Jeff Davis wrote: > Our initcap() is not defined in the standard, and we document that it > only differentiates between alphanumeric and non-alphanumeric > characters, so we could get that behavior pretty easily as well. If > we > wanted to do it the Unicode way

Re: visibility of open cursors in pg_stat_activity

2023-10-26 Thread Andres Freund
Hi, On 2023-10-26 11:47:32 -0400, Robert Haas wrote: > I've seen situations a few times now where somebody has sessions that > are "idle in transaction" for a long time but they feel like it should > be harmless because the transaction has no XID. However, the fact that > the transaction is idle

Re: Introduce a new view for checkpointer related stats

2023-10-26 Thread Bharath Rupireddy
On Thu, Oct 26, 2023 at 7:30 AM Michael Paquier wrote: > > I was looking at this patch, and got a few comments. Thanks. > The view for the bgwriter does not do that. I'd suggest to use > functions that are named as pg_stat_get_checkpoint_$att with shorter > $atts. It is true that "timed" is a

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-26 Thread Nathan Bossart
On Thu, Oct 26, 2023 at 07:28:35AM +, Xiang Gao wrote: > On Wed, 25 Oct, 2023 at 10:43:25 -0500, Nathan Bossart wrote: >>+# Use ARM VMULL if available and ARM CRC32C intrinsic is avaliable too. >>+if test x"$USE_ARMV8_VMULL" = x"" && (test x"$USE_ARMV8_CRC32C" = x"1" || >>test

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Jeff Davis
On Thu, 2023-10-26 at 16:49 +0200, Peter Eisentraut wrote: > On 25.10.23 20:32, Jeff Davis wrote: > > But what should the result of UPPER('á' COLLATE UCS_BASIC) be? In > > Postgres, the answer is 'á', but intuitively, one could reasonably > > expect the answer to be 'Á'. > > I think that's right. 

visibility of open cursors in pg_stat_activity

2023-10-26 Thread Robert Haas
Hi, I've seen situations a few times now where somebody has sessions that are "idle in transaction" for a long time but they feel like it should be harmless because the transaction has no XID. However, the fact that the transaction is idle doesn't mean it isn't running a query, because there

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Bharath Rupireddy
On Thu, Oct 26, 2023 at 8:11 PM Zhijie Hou (Fujitsu) wrote: > > The BF animal fairywren[1] failed when testing > 003_upgrade_logical_replication_slots.pl. > > From the log, I can see pg_upgrade failed to open the > invalid_logical_replication_slots.txt: > > # Checking for valid logical

Re: Atomic ops for unlogged LSN

2023-10-26 Thread John Morris
Keeping things up to date. Here is a rebased patch with no changes from previous one. * John Morris atomic-lsn.patch Description: atomic-lsn.patch

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Peter Eisentraut
On 25.10.23 20:32, Jeff Davis wrote: But what should the result of UPPER('á' COLLATE UCS_BASIC) be? In Postgres, the answer is 'á', but intuitively, one could reasonably expect the answer to be 'Á'. I think that's right. But what would you put into ctype to make that happen? That seems to

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-26 Thread Zhijie Hou (Fujitsu)
Hi, The BF animal fairywren[1] failed when testing 003_upgrade_logical_replication_slots.pl. From the log, I can see pg_upgrade failed to open the invalid_logical_replication_slots.txt: # Checking for valid logical replication slots # could not open file

Re: Guiding principle for dropping LLVM versions?

2023-10-26 Thread Devrim Gündüz
Hi, On Thu, 2023-10-19 at 08:13 +1300, Thomas Munro wrote: > If we used Debian as a yardstick, PostgreSQL 17 wouldn't need anything > older than LLVM 14 AFAICS.  Who else do we need to ask?  LLVM 15 is the minimum one for the platforms that I build the packages on. So LLVM >= 14 is great for

Re: RFC: Pluggable TOAST

2023-10-26 Thread Matthias van de Meent
On Thu, 26 Oct 2023 at 15:18, Aleksander Alekseev wrote: > > Hi, > > > And the goal of *THIS* topic is to gather a picture on how the community > > sees > > improvements in TOAST mechanics if it doesn't want it the way we proposed > > before, to understand which way to go with JSON advanced

Re: Is this a problem in GenericXLogFinish()?

2023-10-26 Thread Robert Haas
On Thu, Oct 26, 2023 at 3:31 AM Michael Paquier wrote: > Hmm. Looking at hash_xlog_squeeze_page(), it looks like wbuf is > expected to always be registered. So, you're right here: it should be > OK to be less aggressive with setting the page LSN on wbuf if ntups is > 0, but there's more to it?

Re: RFC: Pluggable TOAST

2023-10-26 Thread Matthias van de Meent
On Tue, 24 Oct 2023 at 22:38, Nikita Malakhov wrote: > > Hi hackers! > > We need community feedback on previously discussed topic [1]. > There are some long-live issues in Postgres related to the TOAST mechanics, > like [2]. > Some time ago we already proposed a set of patches with an API

Re: trying again to get incremental backup

2023-10-26 Thread Robert Haas
On Thu, Oct 26, 2023 at 6:59 AM Andrew Dunstan wrote: > Because we won't be removing the RD parser. Ah, OK. -- Robert Haas EDB: http://www.enterprisedb.com

Re: remaining sql/json patches

2023-10-26 Thread Nikita Malakhov
Hi, Agreed on the latter, that must not be the part of it for sure. Would think on how to make this part correct. -- Regards, Nikita Malakhov Postgres Professional The Russian Postgres Company https://postgrespro.ru/

Re: RFC: Pluggable TOAST

2023-10-26 Thread Aleksander Alekseev
Hi, > And the goal of *THIS* topic is to gather a picture on how the community sees > improvements in TOAST mechanics if it doesn't want it the way we proposed > before, to understand which way to go with JSON advanced storage and other > enhancements we already have. Previous topic was not of

Re: RFC: Pluggable TOAST

2023-10-26 Thread Nikita Malakhov
Hi, I meant discussion preceding the patch set - there was no any. And the goal of *THIS* topic is to gather a picture on how the community sees improvements in TOAST mechanics if it doesn't want it the way we proposed before, to understand which way to go with JSON advanced storage and other

Re: remaining sql/json patches

2023-10-26 Thread Amit Langote
Hi, On Thu, Oct 26, 2023 at 9:20 PM Nikita Malakhov wrote: > > Hi, > > The main goal was to correctly process invalid queries (as in examples above). > I'm not sure this could be done in type input functions. I thought that some > coercions could be checked before evaluating expressions for

Re: Synchronizing slots from primary to standby

2023-10-26 Thread Amit Kapila
On Thu, Oct 26, 2023 at 12:38 PM Hayato Kuroda (Fujitsu) wrote: > > > PFA v25 patch set. The changes are: > > Thanks for making the patch! It seems that there are lots of comments, so > I can put some high-level comments for 0001. > Sorry if there are duplicated comments. > > 1. > The patch

Re: remaining sql/json patches

2023-10-26 Thread Nikita Malakhov
Hi, The main goal was to correctly process invalid queries (as in examples above). I'm not sure this could be done in type input functions. I thought that some coercions could be checked before evaluating expressions for saving reasons. -- Regards, Nikita Malakhov Postgres Professional The

Re: Synchronizing slots from primary to standby

2023-10-26 Thread Amit Kapila
On Thu, Oct 26, 2023 at 5:38 PM Drouvot, Bertrand wrote: > > On 10/26/23 10:40 AM, Amit Kapila wrote: > > On Wed, Oct 25, 2023 at 8:49 PM Drouvot, Bertrand > > wrote: > >> > > > > Good point, I think we should enhance the WalSndWait() logic to > > address this case. > > Agree. I think it would

Re: Synchronizing slots from primary to standby

2023-10-26 Thread Drouvot, Bertrand
Hi, On 10/26/23 10:40 AM, Amit Kapila wrote: On Wed, Oct 25, 2023 at 8:49 PM Drouvot, Bertrand wrote: Good point, I think we should enhance the WalSndWait() logic to address this case. Agree. I think it would need to take care of the new CV and probably provide a way for the caller to

Re: RFC: Pluggable TOAST

2023-10-26 Thread Aleksander Alekseev
Hi, > Aleksander, previous discussion was not a discussion actually, we proposed > a set of big and complex core changes without any discussion preceding it. > That was not very good approach although the overall idea behind the patch > set is very progressive and is ready to solve some old and

Re: speed up a logical replica setup

2023-10-26 Thread Ashutosh Bapat
On Mon, Oct 23, 2023 at 9:34 AM Euler Taveira wrote: > > It is still a WIP but I would like to share it and get some feedback. > > I have started reviewing the patch. I have just read through all the code. It's well documented and clear. Next I will review the design in detail. Here are a

Re: RFC: Pluggable TOAST

2023-10-26 Thread Nikita Malakhov
Hi, Aleksander, previous discussion was not a discussion actually, we proposed a set of big and complex core changes without any discussion preceding it. That was not very good approach although the overall idea behind the patch set is very progressive and is ready to solve some old and painful

Re: Add trailing commas to enum definitions

2023-10-26 Thread Peter Eisentraut
On 23.10.23 22:34, Nathan Bossart wrote: On Mon, Oct 23, 2023 at 05:55:32PM +0800, Junwang Zhao wrote: On Mon, Oct 23, 2023 at 2:37 PM Peter Eisentraut wrote: Since C99, there can be a trailing comma after the last value in an enum C99 allows us to do this doesn't mean we must do this, this

Re: trying again to get incremental backup

2023-10-26 Thread Andrew Dunstan
On 2023-10-25 We 15:19, Robert Haas wrote: On Wed, Oct 25, 2023 at 3:17 PM Andrew Dunstan wrote: OK, I'll go with that. It will actually be a bit less invasive than the patch I posted. Why's that? Because we won't be removing the RD parser. cheers andrew -- Andrew Dunstan EDB:

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-26 Thread Andrei Zubkov
On Thu, 2023-10-26 at 15:49 +0700, Andrei Lepikhov wrote: > It is the gist of my question. If needed, You can remove the record > by > (userid, dbOid, queryId). As I understand, this extension is usually > used by an administrator. Who can reset these parameters except you > and > the DBMS?

Re: race condition in pg_class

2023-10-26 Thread Smolkin Grigory
> ALTER TABLE ADD CONSTRAINT would certainly have taken > AccessExclusiveLock on the "example" table, which should be sufficient > to prevent anything else from touching its pg_class row. The only > mechanism I can think of that might bypass that is a manual UPDATE on > pg_class, which would just

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-26 Thread Bharath Rupireddy
On Thu, Oct 26, 2023 at 2:23 PM Xiang Gao wrote: > > On Tue, 24 Oct, 2023 20:45:39PM -0500, Nathan Bossart wrote: > >I tried this. pg_waldump on 2 million ~8kB records took around 8.1 seconds > >without the patch and around 7.4 seconds with it (an 8% improvement). > >pg_waldump on 1 million

RE: CRC32C Parallel Computation Optimization on ARM

2023-10-26 Thread Xiang Gao
On Tue, 24 Oct, 2023 20:45:39PM -0500, Nathan Bossart wrote: >I tried this. pg_waldump on 2 million ~8kB records took around 8.1 seconds >without the patch and around 7.4 seconds with it (an 8% improvement). >pg_waldump on 1 million ~16kB records took around 3.2 seconds without the >patch and

  1   2   >