Re: Make pgbench exit on SIGINT more reliably

2023-06-21 Thread Yugo NAGATA
On Mon, 19 Jun 2023 16:49:05 -0700 "Tristan Partin" wrote: > On Mon Jun 19, 2023 at 6:39 AM PDT, Yugo NAGATA wrote: > > On Wed, 24 May 2023 08:58:46 -0500 > > "Tristan Partin" wrote: > > > > > On Tue May 23, 2023 at 7:31 PM CDT, Michael Paquier wrote: > > > > On Mon, May 22, 2023 at 10:02:02AM

Re: Do we want a hashset type?

2023-06-21 Thread Joel Jacobson
On Tue, Jun 20, 2023, at 18:25, Tomas Vondra wrote: > On 6/20/23 16:56, Joel Jacobson wrote: >> The reference to consistency with what we do elsewhere might not be entirely >> applicable in this context, since the set feature we're designing is a new >> beast >> in the SQL landscape. > > I don't

RE: Partial aggregates pushdown

2023-06-21 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Mr.Momjian, Mr.Pyhalov, hackers. > From: Bruce Momjian > Sent: Thursday, June 22, 2023 12:44 AM > On Tue, Jun 20, 2023 at 09:59:11AM +0300, Alexander Pyhalov wrote: > > > Therefore, it seems like it would be near-zero cost to just call > > > conn = > > > GetConnection() and then

Re: vac_truncate_clog()'s bogus check leads to bogusness

2023-06-21 Thread Noah Misch
On Wed, Jun 21, 2023 at 03:12:08PM -0700, Andres Freund wrote: > When vac_truncate_clog() returns early ... > we haven't released the lwlock that we acquired earlier > Until there's some cause for the session to call LWLockReleaseAll(), the lock > is held. Until then neither the process holding

Re: Making empty Bitmapsets always be NULL

2023-06-21 Thread David Rowley
On Thu, 22 Jun 2023 at 00:16, Ranier Vilela wrote: > 2. Only compute BITNUM when necessary. I doubt this will help. The % 64 done by BITNUM will be transformed to an AND operation by the compiler which is likely going to be single instruction latency on most CPUs which probably amounts to it

Re: Assert while autovacuum was executing

2023-06-21 Thread Amit Kapila
On Wed, Jun 21, 2023 at 11:53 AM Peter Geoghegan wrote: > > On Tue, Jun 20, 2023 at 10:27 PM Andres Freund wrote: > > As far as I can tell 72e78d831a as-is is just bogus. Unfortunately that > > likely > > also means 3ba59ccc89 is not right. > > Quite possibly. But I maintain that

Re: Support logical replication of DDLs

2023-06-21 Thread shveta malik
On Mon, Jun 12, 2023 at 7:17 AM Wei Wang (Fujitsu) wrote: > > On Thur, Jun 8, 2023 20:02 PM shveta malik wrote: > > Thank You Vignesh for handling (a), Ajin for handling (b), Shi-san and > > Hou-san for contributing in (c). > > > > The new changes are in patch 0001, 0002, 0005 and 0008. > >

Re: Preventing non-superusers from altering session authorization

2023-06-21 Thread Nathan Bossart
On Wed, Jun 21, 2023 at 04:28:43PM -0400, Joseph Koshakow wrote: > + roleTup = SearchSysCache1(AUTHOID, > ObjectIdGetDatum(AuthenticatedUserId)); > + if (!HeapTupleIsValid(roleTup)) > + ereport(FATAL, > + >

Re: Assert while autovacuum was executing

2023-06-21 Thread Amit Kapila
On Wed, Jun 21, 2023 at 10:57 AM Andres Freund wrote: > > As far as I can tell 72e78d831a as-is is just bogus. Unfortunately that likely > also means 3ba59ccc89 is not right. > Indeed. I was thinking of a fix but couldn't find one yet. One idea I am considering is to allow catalog table locks

Re: Support to define custom wait events for extensions

2023-06-21 Thread Masahiro Ikeda
On 2023-06-20 18:26, Masahiro Ikeda wrote: The followings are TODO items. * to check that meson.build works since I tested with old command `make` now I test with meson and I updated the patches to work with it. My test procedure is the following. ``` export builddir=/mnt/tmp/build export

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Nathan Bossart
On Thu, Jun 22, 2023 at 10:46:41AM +0900, Michael Paquier wrote: > On Wed, Jun 21, 2023 at 10:16:24AM -0700, Nathan Bossart wrote: >>> I think that there is a testing gap with the coverage of CLUSTER. >>> "Ownership of partitions is checked" is a test that looks for the case >>> where

Re: DROP DATABASE is interruptible

2023-06-21 Thread Andres Freund
Hi, On 2023-05-09 15:41:36 +1200, Thomas Munro wrote: > +# FIXME: It'd be good to test the actual interruption path. But it's not > +# immediately obvious how. > > I wonder if there is some way to incorporate something based on > SIGSTOP signals into the test, but I don't know how to do it on >

Re: bgwriter doesn't flush WAL stats

2023-06-21 Thread Kyotaro Horiguchi
At Wed, 21 Jun 2023 18:52:26 +0300, Nazir Bilal Yavuz wrote in > I attached a WIP patch for showing WAL stats in pg_stat_io. Yeah, your diagnosis appears accurate. I managed to trigger an assertion failure quite easily when I added "Assert(!pgstat_have_pending_wal()) just after the call to

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 10:16:24AM -0700, Nathan Bossart wrote: >> I think that there is a testing gap with the coverage of CLUSTER. >> "Ownership of partitions is checked" is a test that looks for the case >> where regress_ptnowner owns the partitioned table and one of its >> partitions,

[Patch] Use *other* indexes on the subscriber when REPLICA IDENTITY is FULL

2023-06-21 Thread Hayato Kuroda (Fujitsu)
Dear hackers, (CC: Önder because he owned the related thread) This is a follow-up thread of [1]. The commit allowed subscribers to use indexes other than PK and REPLICA IDENTITY when REPLICA IDENTITY is FULL on publisher, but the index must be a B-tree. In this proposal, I aim to extend this

Re: extended statistics n-distinct on multiple columns not used when join two tables

2023-06-21 Thread David Rowley
On Tue, 13 Jun 2023 at 23:29, Pavel Stehule wrote: >> I think it's probably worth adjusting the docs to mention this. It >> seems like it might be something that could surprise someone. >> >> Something like the attached, maybe? > > +1 Ok, I pushed that patch. Thanks. David

Re: Adding SHOW CREATE TABLE

2023-06-21 Thread Kirk Wolak
On Mon, Jun 5, 2023 at 7:43 AM Jelte Fennema wrote: > On Thu, 1 Jun 2023 at 18:57, Kirk Wolak wrote: > > Can this get turned into a Patch? Were you offering this code up for > others (me?) to pull, and work into a patch? > > [If I do the patch, I am not sure it gives you the value of reducing

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 09:26:09AM -0700, Jeff Davis wrote: > What I meant is that if you do: > > CREATE TABLE p(i INT, j INT) PARTITION BY RANGE (i); > CREATE TABLE p0 PARTITION OF p FOR VALUES FROM (00) TO (10); > CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (10) TO (20); > CREATE

Re: vac_truncate_clog()'s bogus check leads to bogusness

2023-06-21 Thread Andres Freund
Hi, On 2023-06-21 15:12:08 -0700, Andres Freund wrote: > When vac_truncate_clog() returns early, due to one of these paths: > > [...] > > Separately, I think it's quite bad that we *silently* return from > vac_truncate_clog() when finding a bogus xid. That's a quite severe condition, > we should

Re: Adding further hardening to nbtree page deletion

2023-06-21 Thread Peter Geoghegan
On Tue, Jun 20, 2023 at 11:13 PM Peter Geoghegan wrote: > FWIW, I'm almost certain that I'll completely run out of ERRORs to > demote to LOGs before too long. In fact, this might very well be the > last ERROR that I ever have to demote to a LOG to harden nbtree > VACUUM. Pushed this just now,

Re: Remove deprecation warnings when compiling PG ~13 with OpenSSL 3.0~

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 10:11:33AM +0200, Peter Eisentraut wrote: > Backpatching the OPENSSL_API_COMPAT change would set the minimum OpenSSL > version to 1.0.1, which is newer than what was so far required in those > branches. That is the reason we didn't do this. Looking at the relevant thread

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 12:55:15PM -0700, Nathan Bossart wrote: > LGTM. I think this can wait for v17 since the current behavior has been > around since 2001 and AFAIK this is the first report. While it's arguably > a bug fix, the patch also breaks some cases that work today. Agreed that

Re: Why does pg_bsd_indent need to be installed?

2023-06-21 Thread Bruce Momjian
On Tue, Jun 20, 2023 at 06:54:56PM +0200, Álvaro Herrera wrote: > On 2023-May-31, Bruce Momjian wrote: > > > I guess we could try looking for pg_bsd_indent-$MAJOR_VERSION first, > > then pg_bsd_indent. > > Do you mean with $MAJOR_VERSION being Postgres' version? That means we > need to install

Re: [PATCH] doc: add missing mention of MERGE in MVCC

2023-06-21 Thread Will Mortensen
I saw, thanks again! On Wed, Jun 21, 2023 at 4:08 PM Bruce Momjian wrote: > > On Mon, Jun 19, 2023 at 11:32:46PM -0700, Will Mortensen wrote: > > MERGE is now a data-modification command too. > > Yes, this has been applied too. > > -- > Bruce Momjian https://momjian.us > EDB

Re: [PATCH] doc: add missing mention of MERGE in MVCC

2023-06-21 Thread Bruce Momjian
On Mon, Jun 19, 2023 at 11:32:46PM -0700, Will Mortensen wrote: > MERGE is now a data-modification command too. Yes, this has been applied too. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com Only you can decide what is

vac_truncate_clog()'s bogus check leads to bogusness

2023-06-21 Thread Andres Freund
Hi, When vac_truncate_clog() returns early, due to one of these paths: /* * Do not truncate CLOG if we seem to have suffered wraparound already; * the computed minimum XID might be bogus. This case should now be * impossible due to the defenses in

Re: Preventing non-superusers from altering session authorization

2023-06-21 Thread Nathan Bossart
On Wed, Jun 21, 2023 at 04:28:43PM -0400, Joseph Koshakow wrote: > Currently, a user is allowed to execute SET SESSION AUTHORIZATION [1] > if the role they connected to PostgreSQL with was a superuser at the > time of connection. Even if the role is later altered to no longer be a > superuser, the

Preventing non-superusers from altering session authorization

2023-06-21 Thread Joseph Koshakow
Hi all, I briefly mentioned this issue in another mailing thread [0]. Currently, a user is allowed to execute SET SESSION AUTHORIZATION [1] if the role they connected to PostgreSQL with was a superuser at the time of connection. Even if the role is later altered to no longer be a superuser, the

Re: Can JoinFilter condition be pushed down into IndexScan?

2023-06-21 Thread Tomas Vondra
On 6/21/23 20:37, Bəxtiyar Neyman wrote: > Thanks Tomas for the lengthy write-up! > > Pardon the noise in the queries (LATERAL, AND true etc): they were > autogenerated by the library we wrote. > I know, but it makes them harder to read for people. If you want people to respond it's generally a

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Nathan Bossart
On Wed, Jun 21, 2023 at 09:02:49PM +0200, Drouvot, Bertrand wrote: > Please find attached a patch doing so (which is basically a revert of > d18c1d1f51). LGTM. I think this can wait for v17 since the current behavior has been around since 2001 and AFAIK this is the first report. While it's

Re: Use of additional index columns in rows filtering

2023-06-21 Thread Tomas Vondra
On 6/21/23 18:17, James Coleman wrote: > On Wed, Jun 21, 2023 at 11:28 AM Tomas Vondra > wrote: >> >> >> >> On 6/21/23 14:45, James Coleman wrote: >>> Hello, >>> >>> I've cc'd Jeff Davis on this due to a conversation we had at PGCon >>> about applying filters on index tuples during index

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Drouvot, Bertrand
On 6/21/23 4:22 PM, Drouvot, Bertrand wrote: Hi, On 6/21/23 3:43 PM, Tom Lane wrote: Kyotaro Horiguchi writes: At Wed, 21 Jun 2023 09:43:50 +0200, "Drouvot, Bertrand" wrote in Trying to connect with the 64 bytes name: $ psql -d psql: error: connection to

Re: DROP DATABASE is interruptible

2023-06-21 Thread Andres Freund
Hi, I'm hacking on this bugfix again, thanks to Evgeny's reminder on the other thread [1]. I've been adding checks for partiall-dropped databases to the following places so far: - vac_truncate_clog(), as autovacuum can't process it anymore. Otherwise a partially dropped database could easily

Re: Memory leak in incremental sort re-scan

2023-06-21 Thread James Coleman
On Thu, Jun 15, 2023 at 6:35 PM Tomas Vondra wrote: > > > > On 6/15/23 22:36, Tom Lane wrote: > > Tomas Vondra writes: > >> On 6/15/23 22:11, Tom Lane wrote: > >>> I see zero leakage in that example after applying the attached quick > >>> hack. (It might be better to make the check in the

Re: Support TZ format code in to_timestamp()

2023-06-21 Thread David Steele
On 6/21/23 20:07, Bruce Momjian wrote: On Tue, Jun 13, 2023 at 12:20:42PM -0400, Tom Lane wrote: It's annoyed me for some time that to_timestamp() doesn't implement the TZ format code that to_char() has. I finally got motivated to do something about that after the complaint at [1] that

Re: Support TZ format code in to_timestamp()

2023-06-21 Thread Bruce Momjian
On Tue, Jun 13, 2023 at 12:20:42PM -0400, Tom Lane wrote: > It's annoyed me for some time that to_timestamp() doesn't implement > the TZ format code that to_char() has. I finally got motivated to > do something about that after the complaint at [1] that jsonpath's > datetime() method can't read

Re: add non-option reordering to in-tree getopt_long

2023-06-21 Thread Nathan Bossart
On Tue, Jun 20, 2023 at 02:12:44PM +0900, Kyotaro Horiguchi wrote: > The argv elements get shuffled around many times with the > patch. However, I couldn't find a way to decrease the count without > resorting to a forward scan. So I've concluded the current approach > is them most effeicient,

Re: collation-related loose ends before beta2

2023-06-21 Thread Jonathan S. Katz
On 6/20/23 5:02 AM, Jeff Davis wrote: Status on collation loose ends: 1. There's an open item "Switch to ICU for 17". It's a little bit confusing exactly what that means, and the CF entry refers to two items, one of which is the build-time default to --with-icu. As far as I know, building with

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Nathan Bossart
On Tue, Jun 20, 2023 at 09:15:18PM -0700, Nathan Bossart wrote: > Perhaps we should add something like > > Note that while REINDEX on a partitioned index or table requires > MAINTAIN on the partitioned table, such commands skip the privilege > checks when processing the

Re: RFC: logical publication via inheritance root?

2023-06-21 Thread Jacob Champion
On Wed, Jun 21, 2023 at 3:28 AM Amit Kapila wrote: > On Tue, Jun 20, 2023 at 10:39 PM Jacob Champion > wrote: > > Making it a subscriber-side feature > > requires tight coupling between the two peers, though. (For the > > timescaledb case, how does the subscriber know which new partitions > >

Re: EBCDIC sorting as a use case for ICU rules

2023-06-21 Thread Jonathan S. Katz
On 6/21/23 12:14 PM, Jeff Davis wrote: On Wed, 2023-06-21 at 15:28 +0200, Daniel Verite wrote: At a conference this week I was asked if ICU could be able to sort like EBCDIC [2]. It turns out it has been already  asked on -general a few years ago [3] with no satisfactory answer at the time ,

Re: pg_collation.collversion for C.UTF-8

2023-06-21 Thread Daniel Verite
Thomas Munro wrote: > What could we do that would be helpful here, without affecting users > of the "true" C.UTF-8 for the rest of time? This is a Debian (+ > downstream distro) only problem as far as we know so far, and only > for Debian 11 and older. It seems to include RedHat-based

Re: Remove deprecation warnings when compiling PG ~13 with OpenSSL 3.0~

2023-06-21 Thread Andres Freund
Hi, On 2023-06-21 10:11:33 +0200, Peter Eisentraut wrote: > On 21.06.23 09:43, Michael Paquier wrote: > > On Wed, Jun 21, 2023 at 09:16:38AM +0200, Daniel Gustafsson wrote: > > > Agreed, I'd be more inclined to go with OPENSSL_API_COMPAT. If we still > > > get > > > warnings with that set then

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Jeff Davis
On Tue, 2023-06-20 at 15:52 -0700, Nathan Bossart wrote: > At the moment, I think I'm inclined to call this "existing behavior" > since > we didn't check privileges for each partition in this case even > before > MAINTAIN was introduced.  IIUC we still process the individual > partitions > in v15

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2023-06-21 Thread Jeff Davis
On Wed, 2023-06-21 at 07:53 +0900, Michael Paquier wrote: > I am not sure to understand this last sentence.  REINDEX on a > partitioned table builds a list of the indexes to work on in the > first > transaction processing the command in ReindexPartitions(), and there > is no need to process

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
On Wed, Jun 21, 2023 at 11:28 AM Tomas Vondra wrote: > > > > On 6/21/23 14:45, James Coleman wrote: > > Hello, > > > > I've cc'd Jeff Davis on this due to a conversation we had at PGCon > > about applying filters on index tuples during index scans. > > > > I've also cc'd Andres Freund because I

Re: EBCDIC sorting as a use case for ICU rules

2023-06-21 Thread Jeff Davis
On Wed, 2023-06-21 at 15:28 +0200, Daniel Verite wrote: > At a conference this week I was asked if ICU could be able to > sort like EBCDIC [2]. It turns out it has been already  asked on > -general a few years ago [3] with no satisfactory answer at the time > , > and that it can be implemented

Re: bgwriter doesn't flush WAL stats

2023-06-21 Thread Nazir Bilal Yavuz
Hi, Thanks for the explanation. On Wed, 21 Jun 2023 at 18:03, Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > > On Wed, 21 Jun 2023 at 13:04, Nazir Bilal Yavuz wrote: > > I was trying to add WAL stats to pg_stat_io. While doing that I was comparing pg_stat_wal and pg_stat_io's

Re: EBCDIC sorting as a use case for ICU rules

2023-06-21 Thread Joe Conway
On 6/21/23 09:28, Daniel Verite wrote: In the "Order changes in PG16 since ICU introduction" discussion, one sub-thread [1] was about having a credible use case for tailoring collations with custom rules, a new feature in v16. At a conference this week I was asked if ICU could be able to sort

Re: Partial aggregates pushdown

2023-06-21 Thread Bruce Momjian
On Tue, Jun 20, 2023 at 09:59:11AM +0300, Alexander Pyhalov wrote: > > Therefore, it seems like it would be near-zero cost to just call conn = > > GetConnection() and then PQserverVersion(conn), and ReleaseConnection(). > > You can then use the return value of PQserverVersion() to determine if > >

Re: Use of additional index columns in rows filtering

2023-06-21 Thread Tomas Vondra
On 6/21/23 14:45, James Coleman wrote: > Hello, > > I've cc'd Jeff Davis on this due to a conversation we had at PGCon > about applying filters on index tuples during index scans. > > I've also cc'd Andres Freund because I think this relates to his > musing in [1] that: >> One thing I have

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Nathan Bossart
On Wed, Jun 21, 2023 at 09:43:38AM -0400, Tom Lane wrote: > Kyotaro Horiguchi writes: >> IMHO, I'm not sure we should allow connections without the exact name >> being provided. In that sense, I think we might want to consider >> outright rejecting the estblishment of a connection when the given

Re: bgwriter doesn't flush WAL stats

2023-06-21 Thread Matthias van de Meent
On Wed, 21 Jun 2023 at 13:04, Nazir Bilal Yavuz wrote: > > Hi, > > I was trying to add WAL stats to pg_stat_io. While doing that I was comparing > pg_stat_wal and pg_stat_io's WAL stats and there was some inequality between > the total number of WALs. I found that the difference comes from

Re: Allow pg_archivecleanup to remove backup history files

2023-06-21 Thread torikoshia
On 2023-06-21 11:59, Kyotaro Horiguchi wrote: At Tue, 20 Jun 2023 22:27:36 +0900, torikoshia wrote in On 2023-06-19 14:37, Michael Paquier wrote: > On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote: >> Thanks, now I understand what you meant. > If I may ask, why is the refactoring of

Re: pgindent vs. pgperltidy command-line arguments

2023-06-21 Thread Peter Eisentraut
On 21.06.23 13:35, Andrew Dunstan wrote: If not, part of my patch would still be useful.  Maybe I should commit my posted patch for PG16, to keep consistency with pgindent, and then your work would presumably be considered for PG17. That sounds like a good plan. done

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Drouvot, Bertrand
Hi, On 6/21/23 3:43 PM, Tom Lane wrote: Kyotaro Horiguchi writes: At Wed, 21 Jun 2023 09:43:50 +0200, "Drouvot, Bertrand" wrote in Trying to connect with the 64 bytes name: $ psql -d psql: error: connection to server on socket "/tmp/.s.PGSQL.55448" failed:

Re: Stack overflow issue

2023-06-21 Thread Egor Chindyaskin
Hello! In continuation of the topic I would like to suggest solution. This patch adds several checks to the vulnerable functions above.diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index d85e313908..102d0e1574 100644 --- a/src/backend/access/transam/xact.c +++

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Tom Lane
Kyotaro Horiguchi writes: > At Wed, 21 Jun 2023 09:43:50 +0200, "Drouvot, Bertrand" > wrote in >> Trying to connect with the 64 bytes name: >> $ psql -d >> psql: error: connection to server on socket "/tmp/.s.PGSQL.55448" >> failed: FATAL: database

EBCDIC sorting as a use case for ICU rules

2023-06-21 Thread Daniel Verite
Hi, In the "Order changes in PG16 since ICU introduction" discussion, one sub-thread [1] was about having a credible use case for tailoring collations with custom rules, a new feature in v16. At a conference this week I was asked if ICU could be able to sort like EBCDIC [2]. It turns out it has

Re: Can JoinFilter condition be pushed down into IndexScan?

2023-06-21 Thread Tomas Vondra
On 6/21/23 05:37, Bəxtiyar Neyman wrote: > I define a table user_ranks as such: > > CREATE TABLE user_ranks ( >   id INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY, >   rank INTEGER NOT NULL, >   CONSTRAINT "by (rank, id)" UNIQUE (rank, id) > ); > > INSERT INTO user_ranks (user_id, rank)

Re: Support logical replication of DDLs

2023-06-21 Thread Jelte Fennema
(to be clear I only skimmed the end of this thread and did not look at all the previous messages) I took a quick look at the first patch (about deparsing table ddl) and it seems like this would also be very useful for a SHOW CREATE TABLE, like command. Which was suggested in this thread:

Opportunistically pruning page before update

2023-06-21 Thread James Coleman
Hello, While at PGCon I was chatting with Andres (and I think Peter G. and a few others who I can't remember at the moment, apologies) and Andres noted that while we opportunistically prune a page when inserting a tuple (before deciding we need a new page) we don't do the same for updates.

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
Hello, I've cc'd Jeff Davis on this due to a conversation we had at PGCon about applying filters on index tuples during index scans. I've also cc'd Andres Freund because I think this relates to his musing in [1] that: > One thing I have been wondering around this is whether we should not have >

Re: Making empty Bitmapsets always be NULL

2023-06-21 Thread Ranier Vilela
Hi, David Rowley wrote: >I've adjusted the attached patch to do that. I think that was room for more improvements. 1. bms_member_index Bitmapset can be const. 2. Only compute BITNUM when necessary. 3. Avoid enlargement when nwords is equal wordnum. Can save cycles when in corner cases?

Can JoinFilter condition be pushed down into IndexScan?

2023-06-21 Thread Bəxtiyar Neyman
I define a table user_ranks as such: CREATE TABLE user_ranks ( id INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY, rank INTEGER NOT NULL, CONSTRAINT "by (rank, id)" UNIQUE (rank, id) ); INSERT INTO user_ranks (user_id, rank) SELECT generate_series(1, 1), generate_series(1, 1);

Re: pgindent vs. pgperltidy command-line arguments

2023-06-21 Thread Andrew Dunstan
On 2023-06-21 We 05:09, Peter Eisentraut wrote: On 20.06.23 17:38, Andrew Dunstan wrote: +1, although I wonder if we shouldn't follow pgindent's new lead and require some argument(s). That makes sense to me.  Here is a small update with this behavior change and associated documentation

bgwriter doesn't flush WAL stats

2023-06-21 Thread Nazir Bilal Yavuz
Hi, I was trying to add WAL stats to pg_stat_io. While doing that I was comparing pg_stat_wal and pg_stat_io's WAL stats and there was some inequality between the total number of WALs. I found that the difference comes from bgwriter's WALs. bgwriter generates WAL but it doesn't flush them because

Re: RFC: logical publication via inheritance root?

2023-06-21 Thread Amit Kapila
On Tue, Jun 20, 2023 at 10:39 PM Jacob Champion wrote: > > On Fri, Jun 16, 2023 at 9:24 PM Amit Kapila wrote: > > > The other idea that came across my mind was to provide some schema > > mapping kind of feature on subscribers where we could route the tuples > > from table X to table Y provided

Re: [BUG] recovery of prepared transactions during promotion can fail

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 11:11:55AM +0200, Julian Markwort wrote: > I see you've already undone it. > Attached is a patch for 009_twophase.pl to just try this corner case at the > very end, so as not to influence other > existing tests in suite. > > When I run this on REL_14_8 I get the error

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-06-21 Thread Nishant Sharma
Looks good to me. Tested on master and it works. New patch used a bool flag to avoid calls for both FDW and custom hook's call. And a slight change in comment of "has_pseudoconstant_clauses" function. Regards, Nishant. On Wed, Jun 14, 2023 at 12:19 PM Etsuro Fujita wrote: > On Mon, Jun 5, 2023

Re: [BUG] recovery of prepared transactions during promotion can fail

2023-06-21 Thread Julian Markwort
First off, thanks for the quick reaction and reviews, I appreciate it. On Wed, 2023-06-21 at 14:14 +0900, Michael Paquier wrote: > But that won't connect work as the segment requested is now a partial > one in the primary's pg_wal, still the standby wants it. I think since 009_twophase.pl

Re: pgindent vs. pgperltidy command-line arguments

2023-06-21 Thread Peter Eisentraut
On 20.06.23 17:38, Andrew Dunstan wrote: +1, although I wonder if we shouldn't follow pgindent's new lead and require some argument(s). That makes sense to me.  Here is a small update with this behavior change and associated documentation update. I'm intending to add some of the new

Re: ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Kyotaro Horiguchi
At Wed, 21 Jun 2023 09:43:50 +0200, "Drouvot, Bertrand" wrote in > Trying to connect with the 64 bytes name: > > $ psql -d > psql: error: connection to server on socket "/tmp/.s.PGSQL.55448" > failed: FATAL: database "äää" does not >

Re: Remove deprecation warnings when compiling PG ~13 with OpenSSL 3.0~

2023-06-21 Thread Peter Eisentraut
On 21.06.23 09:43, Michael Paquier wrote: On Wed, Jun 21, 2023 at 09:16:38AM +0200, Daniel Gustafsson wrote: Agreed, I'd be more inclined to go with OPENSSL_API_COMPAT. If we still get warnings with that set then I feel those warrant special consideration rather than a blanket suppression.

Re: remap the .text segment into huge pages at run time

2023-06-21 Thread John Naylor
On Wed, Jun 21, 2023 at 10:42 AM Andres Freund wrote: > So I am wondering if you're encountering a different kind of problem. As I > mentioned, I have observed that the pages need to be clean for this to > work. For me adding a "sync path/to/postgres" makes it work on 6.3.8. Without > the sync

Re: Remove deprecation warnings when compiling PG ~13 with OpenSSL 3.0~

2023-06-21 Thread Michael Paquier
On Wed, Jun 21, 2023 at 09:16:38AM +0200, Daniel Gustafsson wrote: > Agreed, I'd be more inclined to go with OPENSSL_API_COMPAT. If we still get > warnings with that set then I feel those warrant special consideration rather > than a blanket suppression. 4d3db136 seems to be OK on REL_13_STABLE

ProcessStartupPacket(): database_name and user_name truncation

2023-06-21 Thread Drouvot, Bertrand
Hi hackers, Please find attached a patch to truncate (in ProcessStartupPacket()) the port->database_name and port->user_name in such a way to not break multibyte character boundary. Indeed, currently, one could create a database that way: postgres=# create database

Re: Consistent coding for the naming of LR workers

2023-06-21 Thread Alvaro Herrera
On 2023-Jun-21, Peter Smith wrote: > Except, please note that there are already multiple message format > strings in the HEAD code that look like "%s for subscription ...", > that are using the get_worker_name() function for the name > substitution. > > e.g. > - "%s for subscription \"%s\" will

Re: Remove deprecation warnings when compiling PG ~13 with OpenSSL 3.0~

2023-06-21 Thread Daniel Gustafsson
> On 21 Jun 2023, at 07:44, Andres Freund wrote: > On 2023-06-21 11:53:44 +0900, Michael Paquier wrote: >> I have been annoyed by these in the past when doing backpatches, as >> this creates some noise, and the only place where this counts is >> sha2_openssl.c. Thoughts about doing something

Re: [PATCH] hstore: Fix parsing on Mac OS X: isspace() is locale specific

2023-06-21 Thread Michael Paquier
On Tue, Jun 20, 2023 at 11:39:31PM -0400, Tom Lane wrote: > I'd be okay with adding \v to the set of whitespace characters in > scan.l and scanner_isspace (and other affected places) for v17. > Don't want to back-patch it though. Okay. No idea where this will lead, but for now I have sent a

Consider \v to the list of whitespace characters in the parser

2023-06-21 Thread Michael Paquier
Hi all, (Adding Evan in CC as he has reported the original issue with hstore.) $subject has showed up as a subject for discussion when looking at the set of whitespace characters that we use in the parsers:

Re: Assert while autovacuum was executing

2023-06-21 Thread Peter Geoghegan
On Tue, Jun 20, 2023 at 10:27 PM Andres Freund wrote: > As far as I can tell 72e78d831a as-is is just bogus. Unfortunately that likely > also means 3ba59ccc89 is not right. Quite possibly. But I maintain that ginInsertCleanup() is probably also bogus in a way that's directly relevant. Did you

Re: Adding further hardening to nbtree page deletion

2023-06-21 Thread Peter Geoghegan
On Tue, Jun 20, 2023 at 10:39 PM Andres Freund wrote: > But the further we go down this path, the more important it is that we provide > some way to monitor stuff like this. IME it's not particularly practical to > rely on scanning logs to find such issues at scale. I suspect we ought to add > at