Re: Raising the SCRAM iteration count

2023-03-07 Thread Michael Paquier
On Tue, Mar 07, 2023 at 02:03:05PM +0100, Daniel Gustafsson wrote: > On 7 Mar 2023, at 09:26, Daniel Gustafsson wrote: >> Right, what I meant was: can a pg_regress sql/expected test drive a psql >> interactive prompt? Your comments suggested using password.sql so I was >> curious if I was

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-07 Thread Tom Lane
Peter Eisentraut writes: > On 05.03.23 00:04, Tom Lane wrote: >> I've gone through this and have a modest suggestion: let's invent some >> wrapper functions around encode(sha256()) to reduce the cosmetic diffs >> and consequent need for closer study of patch changes. In the attached >> I called

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-07 Thread Peter Eisentraut
On 05.03.23 00:04, Tom Lane wrote: I've gone through this and have a modest suggestion: let's invent some wrapper functions around encode(sha256()) to reduce the cosmetic diffs and consequent need for closer study of patch changes. In the attached I called them "notmd5()", but I'm surely not

Re: Add pg_walinspect function with block info columns

2023-03-07 Thread Michael Paquier
On Tue, Mar 07, 2023 at 03:56:22PM +0530, Bharath Rupireddy wrote: > That would be a lot better. Not just the test, but also the > documentation can have it. Simple way to generate such a record (both > block data and FPI) is to just change the wal_level to logical in > walinspect.conf [1], see

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread houzj.f...@fujitsu.com
On Wednesday, March 8, 2023 2:51 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, March 7, 2023 9:47 PM Önder Kalacı > wrote: > > Hi, > > > > > Let me give an example to demonstrate why I thought something is fishy > here: > > > > > > > > Imagine rel has a (non-default) REPLICA IDENTITY with

Re: Add standard collation UNICODE

2023-03-07 Thread Peter Eisentraut
On 04.03.23 19:29, Jeff Davis wrote: I do like your approach though because, if someone is using a standard collation, I think "not built with ICU" (feature not supported) is a better error than "collation doesn't exist". It also effectively reserves the name "unicode". By the way, speaking of

Re: pg_upgrade and logical replication

2023-03-07 Thread Julien Rouhaud
On Sat, 4 Mar 2023, 14:13 Amit Kapila, wrote: > > > For the publisher nodes, that may be something nice to support (I'm > assuming it > > could be useful for more complex replication setups) but I'm not > interested in > > that at the moment as my goal is to reduce downtime for major upgrade of

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread houzj.f...@fujitsu.com
On Tuesday, March 7, 2023 9:47 PM Önder Kalacı wrote: Hi, > > > Let me give an example to demonstrate why I thought something is fishy > > > here: > > > > > > Imagine rel has a (non-default) REPLICA IDENTITY with Oid=. > > > Imagine the same rel has a PRIMARY KEY with Oid=. > > > > >

Re: Add standard collation UNICODE

2023-03-07 Thread Peter Eisentraut
On 04.03.23 19:29, Jeff Davis wrote: It looks like the way you've handled this is by inserting the collation with collprovider=icu even if built without ICU support. I think that's a new case, so we need to make sure it throws reasonable user-facing errors. It would look like this: => select

Re: Normalization of utility queries in pg_stat_statements

2023-03-07 Thread Michael Paquier
On Fri, Mar 03, 2023 at 09:37:27AM +0900, Michael Paquier wrote: > Thanks for double-checking, applied 0001 to finish this part of the > work. I am attaching the remaining bits as of the attached, combined > into a single patch. Doing so as a single patch was not feeling right as this actually

Re: Allow tailoring of ICU locales with custom rules

2023-03-07 Thread Jeff Davis
On Fri, 2023-03-03 at 13:45 +0100, Peter Eisentraut wrote: > You can mess with people by setting up your databases like this: > > initdb -D data --locale-provider=icu --icu-rules=' < c < b < e < d' > > ;-) Would we be the first major database to support custom collation rules? This sounds

Re: Move defaults toward ICU in 16?

2023-03-07 Thread Jeff Davis
On Fri, 2023-03-03 at 21:45 -0800, Jeff Davis wrote: > > >    0002: update template0 in new cluster (as described above) I think 0002 is about ready and I plan to commit it soon unless someone has more comments. I'm holding off on 0001 for now, because you objected. But I still think 0001 is a

Re: Date-time extraneous fields with reserved keywords

2023-03-07 Thread Keisuke Kuroda
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Thank you for the response and new patch. The scope of

RE: Allow logical replication to copy tables in binary format

2023-03-07 Thread Hayato Kuroda (Fujitsu)
Dear Melih, >> I think we should add description to doc that it is more likely happen to >> fail >> the initial copy user should enable binary format after synchronization if >> tables have original datatype. > > I tried to explain when binary copy can cause failures in the doc. What > exactly

Re: Testing autovacuum wraparound (including failsafe)

2023-03-07 Thread Peter Geoghegan
On Fri, Mar 3, 2023 at 3:34 AM Heikki Linnakangas wrote: > I took a different approach to consuming the XIDs. Instead of setting > nextXID directly, bypassing GetNewTransactionId(), this patch introduces > a helper function to call GetNewTransactionId() repeatedly. But because > that's slow, it

Re: shoud be get_extension_schema visible?

2023-03-07 Thread Pavel Stehule
st 8. 3. 2023 v 2:04 odesílatel Michael Paquier napsal: > On Mon, Mar 06, 2023 at 04:44:59PM +0900, Michael Paquier wrote: > > I can see why you'd want that, so OK from here to provide this routine > > for external consumption. Let's first wait a bit and see if others > > have any kind of

Re: optimize several list functions with SIMD intrinsics

2023-03-07 Thread Nathan Bossart
On Wed, Mar 08, 2023 at 01:54:15PM +1300, David Rowley wrote: > Interesting and quite impressive performance numbers. Thanks for taking a look. > From having a quick glance at the patch, it looks like you'll need to > take some extra time to make it work on 32-bit builds. At the moment, the

Re: Testing autovacuum wraparound (including failsafe)

2023-03-07 Thread Masahiko Sawada
On Fri, Mar 3, 2023 at 8:34 PM Heikki Linnakangas wrote: > > On 16/11/2022 06:38, Ian Lawrence Barwick wrote: > > Thanks for the patch. While reviewing the patch backlog, we have determined > > that > > the latest version of this patch was submitted before meson support was > > implemented, so

RE: [Proposal] Add foreign-server health checks infrastructure

2023-03-07 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, Thank you for reviewing! PSA new version. > > Few comments: > 1) There is no handling of forConnCheck in #else HAVE_POLL, if this is > intentional we could add some comments for the same: > static int > -pqSocketPoll(int sock, int forRead, int forWrite, time_t end_time) >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-03-07 Thread John Naylor
On Tue, Mar 7, 2023 at 8:25 AM Masahiko Sawada wrote: > > 1. Make it optional to track chunk memory space by a template parameter. It might be tiny compared to everything else that vacuum does. That would allow other users to avoid that overhead. > > 2. When context block usage exceeds the limit

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Peter Smith
On Wed, Mar 8, 2023 at 3:03 PM Amit Kapila wrote: > > On Wed, Mar 8, 2023 at 9:09 AM Peter Smith wrote: > > > > > > == > > src/backend/executor/execReplication.c > > > > 3. build_replindex_scan_key > > > > { > > Oid operator; > > Oid opfamily; > > RegProcedure regop; > > - int

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-03-07 Thread Thomas Munro
On Wed, Mar 8, 2023 at 4:43 PM Anton A. Melnikov wrote: > On 04.03.2023 00:39, Thomas Munro wrote: > > Could we make better use of the safe copy that we have in the log? > > Then the pg_backup_start() subproblem would disappear. Conceptually, > > that'd be just like the way we use FPI for data

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Amit Kapila
On Wed, Mar 8, 2023 at 9:09 AM Peter Smith wrote: > > > == > src/backend/executor/execReplication.c > > 3. build_replindex_scan_key > > { > Oid operator; > Oid opfamily; > RegProcedure regop; > - int pkattno = attoff + 1; > - int mainattno = indkey->values[attoff]; > - Oid optype =

Re: buildfarm + meson

2023-03-07 Thread Andres Freund
On 2023-03-07 18:26:21 -0800, Andres Freund wrote: > On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote: > > Yeah. For touch I think we can probably just get rid of this line in the > > root meson.build: > > > > touch = find_program('touch', native: true) > > Yep. > > > For cp there doesn't

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-03-07 Thread Masahiko Sawada
On Wed, Mar 8, 2023 at 3:30 AM Nathan Bossart wrote: > > On Mon, Mar 06, 2023 at 07:27:59PM +0300, Önder Kalacı wrote: > > On the other hand, we already have a similar problem with > > recovery_min_apply_delay combined with hot_standby_feedback [1]. > > So, that probably is an acceptable

Re: [Proposal] Add foreign-server health checks infrastructure

2023-03-07 Thread vignesh C
On Tue, 7 Mar 2023 at 09:53, Hayato Kuroda (Fujitsu) wrote: > > Dear Katsuragi-san, > > Thank you for reviewing! PSA new version patches. > > > I think we can update the status to ready for committer after > > this fix, if there is no objection. > > That's a very good news for me! How about other

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-03-07 Thread Anton A. Melnikov
Hi, Thomas! On 04.03.2023 00:39, Thomas Munro wrote: It seems a good topic for a separate thread patch. Would you provide a link to the thread you mentioned please? https://www.postgresql.org/message-id/flat/367d01a7-90bb-9b70-4cda-248e81cc475c%40cosium.com Thanks! The important words

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Peter Smith
Here are some review comments for v35-0001 == General 1. Saying the index "should" or "should not" do this or that sounds like it is still OK but just not recommended. TO remove this ambigity IMO most of the "should" ought to be changed to "must" because IIUC this patch will simply not

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Amit Kapila
On Tue, Mar 7, 2023 at 7:17 PM Önder Kalacı wrote: > >> >> > > Let me give an example to demonstrate why I thought something is fishy >> > > here: >> > > >> > > Imagine rel has a (non-default) REPLICA IDENTITY with Oid=. >> > > Imagine the same rel has a PRIMARY KEY with Oid=. >> > > > >

RE: Rework LogicalOutputPluginWriterUpdateProgress

2023-03-07 Thread wangw.f...@fujitsu.com
On Tue, Mar 7, 2023 15:55 PM Kuroda, Hayato/黒田 隼人 wrote: > Dear Wang, > > Thank you for updating the patch! Followings are my comments. Thanks for your comments. > --- > 01. missing comments > You might miss the comment from Peter[1]. Or could you pin the related one? Since I think the

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread shiy.f...@fujitsu.com
On Tue, Mar 7, 2023 9:47 PM Önder Kalacı wrote: > > I'm attaching v35. > I noticed that if the index column only exists on the subscriber side, this index can also be chosen. This seems a bit odd because the index column isn't sent from publisher. e.g. -- pub CREATE TABLE

Re: psql: Add role's membership options to the \du+ command

2023-03-07 Thread David G. Johnston
On Tue, Mar 7, 2023 at 2:02 PM David G. Johnston wrote: > > I'll be looking over your v3 patch sometime this week, if not today. > > Moving the goal posts for this meta-command to >= 9.5 seems like it should be done as a separate patch and thread. The documentation presently states we are

Re: buildfarm + meson

2023-03-07 Thread Andres Freund
Hi, On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote: > Yeah. For touch I think we can probably just get rid of this line in the > root meson.build: > > touch = find_program('touch', native: true) Yep. > For cp there doesn't seem to be a formal requirement, but there is a recipe > in

Re: Add LZ4 compression in pg_dump

2023-03-07 Thread Justin Pryzby
On Wed, Mar 01, 2023 at 04:52:49PM +0100, Tomas Vondra wrote: > Thanks. That seems correct to me, but I find it somewhat confusing, > because we now have > > DeflateCompressorInit vs. InitCompressorGzip > > DeflateCompressorEnd vs. EndCompressorGzip > > DeflateCompressorData - The name

Re: buildfarm + meson

2023-03-07 Thread Andres Freund
Hi, On 2023-03-07 15:47:54 -0500, Andrew Dunstan wrote: > On 2023-03-07 Tu 14:37, Andres Freund wrote: > > The failures are like this: > > > > +ERROR: extension "dummy_index_am" is not available > > +DETAIL: Could not open extension control file > >

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Andres Freund
Hi, On 2023-03-07 04:45:32 +0300, Alexander Korotkov wrote: > The second patch now implements a concept of LazyTupleTableSlot, a slot > which gets allocated only when needed. Also, there is more minor > refactoring and more comments. This patch already is pretty big for what it actually

Re: shoud be get_extension_schema visible?

2023-03-07 Thread Michael Paquier
On Mon, Mar 06, 2023 at 04:44:59PM +0900, Michael Paquier wrote: > I can see why you'd want that, so OK from here to provide this routine > for external consumption. Let's first wait a bit and see if others > have any kind of objections or comments. Done this one as of e20b1ea. -- Michael

Re: add PROCESS_MAIN to VACUUM

2023-03-07 Thread Michael Paquier
On Tue, Mar 07, 2023 at 12:55:08PM -0800, Nathan Bossart wrote: > On Tue, Mar 07, 2023 at 12:39:29PM -0500, Melanie Plageman wrote: >> Yes, sounds clear to me also! > > Here is an updated patch. Fine by me, so done. (I have cut a few words from the comment, without changing its meaning.) --

Re: optimize several list functions with SIMD intrinsics

2023-03-07 Thread David Rowley
On Wed, 8 Mar 2023 at 13:25, Nathan Bossart wrote: > I've attached a work-in-progress patch that implements these optimizations > for both x86 and arm, and I will register this in the July commitfest. I'm > posting this a little early in order to gauge interest. Interesting and quite impressive

optimize several list functions with SIMD intrinsics

2023-03-07 Thread Nathan Bossart
I noticed that several of the List functions do simple linear searches that can be optimized with SIMD intrinsics (as was done for XidInMVCCSnapshot in 37a6e5d). The following table shows the time spent iterating over a list of n elements (via list_member_int) one billion times on my x86 laptop.

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Alexander Korotkov
On Tue, Mar 7, 2023 at 7:26 PM Pavel Borisov wrote: > On Tue, 7 Mar 2023, 10:10 Alexander Korotkov, wrote: >> I don't know what exactly Pavel meant, but average overall numbers for >> low concurrency are. >> master: 420401 (stddev of average 233) >> patchset v11: 420111 (stddev of average 199)

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Peter Smith
On Mon, Mar 6, 2023 at 7:40 PM Amit Kapila wrote: > > On Mon, Mar 6, 2023 at 1:40 PM Peter Smith wrote: > > ... > > > > Anyhow, if you feel those firstterm and FULL changes ought to be kept > > separate from this RI patch, please let me know and I will propose > > those changes in a new thread,

PGDOCS - Replica Identity quotes

2023-03-07 Thread Peter Smith
PGDOCS - Replica Identity quotes Hi, Here are some trivial quote changes to a paragraph describing REPLICA IDENTITY. These changes were previously made in another ongoing R.I. patch v28-0001 [1], but it was decided that since they are not strictly related to that patch they should done

pipe_read_line for reading arbitrary strings

2023-03-07 Thread Daniel Gustafsson
When skimming through pg_rewind during a small review I noticed the use of pipe_read_line for reading arbitrary data from a pipe, the mechanics of which seemed odd. Commit 5b2f4afffe6 refactored find_other_exec() and broke out pipe_read_line() as a static convenience routine for reading a single

Re: Add support for unit "B" to pg_size_pretty()

2023-03-07 Thread David Rowley
On Wed, 8 Mar 2023 at 09:22, Peter Eisentraut wrote: > Ok, I have fixed the original documentation to that effect and > backpatched it. Thanks for fixing that. David

Re: psql: Add role's membership options to the \du+ command

2023-03-07 Thread David G. Johnston
On Mon, Mar 6, 2023 at 12:43 AM Pavel Luzanov wrote: > Indeed, adding ADMIN to pg_has_role looks logical. The function will show > whether one role can manage another directly or indirectly (via SET ROLE). > FWIW I've finally gotten to publishing my beta version of the Role Graph for PostgreSQL

Re: add PROCESS_MAIN to VACUUM

2023-03-07 Thread Nathan Bossart
On Tue, Mar 07, 2023 at 12:39:29PM -0500, Melanie Plageman wrote: > Yes, sounds clear to me also! Here is an updated patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 580f966499..0acc42af2b

Re: buildfarm + meson

2023-03-07 Thread Andrew Dunstan
On 2023-03-07 Tu 14:37, Andres Freund wrote: Hi, On 2023-03-01 13:32:58 -0800, Andres Freund wrote: On 2023-03-01 16:21:32 -0500, Andrew Dunstan wrote: Perhaps the latest version will be more to your taste. I'll check it out. A simple conversion from an existing config failed with: Can't

Re: Add support for unit "B" to pg_size_pretty()

2023-03-07 Thread Peter Eisentraut
On 06.03.23 09:27, David Rowley wrote: On Mon, 6 Mar 2023 at 21:13, Peter Eisentraut wrote: On 02.03.23 20:58, David Rowley wrote: I think I'd prefer to see the size_bytes_unit_alias struct have an index into size_pretty_units[] array. i.e: Ok, done that way. (I had thought about that,

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Andres Freund
Hi, On 2023-03-07 08:22:45 +0530, Amit Kapila wrote: > On Tue, Mar 7, 2023 at 1:34 AM Andres Freund wrote: > > I think even as-is it's reasonable to just use it. The sequential scan > > approach is O(N^2), which, uh, is not good. And having an index over > > thousands > > of non-differing

Re: Add shared buffer hits to pg_stat_io

2023-03-07 Thread Andres Freund
Hi, LGTM. The only comment I have is that a small test wouldn't hurt... Compared to the other things it should be fairly easy... Greetings, Andres Freund

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-07 Thread Regina Obe
> I'm not unsympathetic to the idea of trying to support multiple upgrade paths > in one script. I just don't like this particular design for that, because it > requires the extension author to make promises that nobody is actually going > to deliver on. > > regards, tom

Re: buildfarm + meson

2023-03-07 Thread Andres Freund
Hi, On 2023-03-01 13:32:58 -0800, Andres Freund wrote: > On 2023-03-01 16:21:32 -0500, Andrew Dunstan wrote: > > Perhaps the latest version will be more to your taste. > > I'll check it out. A simple conversion from an existing config failed with: Can't use an undefined value as an ARRAY

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-07 Thread Regina Obe
> The thing that confuses me here is why the PostGIS folks are ending up with > so many files. > We certainly don't have that problem with the extension that > are being maintained in contrib, and I guess there is some difference in > versioning practice that is making it an issue for them but

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-07 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 10, 2023 at 6:50 PM Tom Lane wrote: >> As an example, suppose that a database has foo 4.0 installed, and >> the DBA decides to try to downgrade to 3.0. With the system as it >> stands, if you've provided foo--4.0--3.0.sql then the conversion >> will go through,

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-03-07 Thread Robert Haas
On Thu, Feb 9, 2023 at 4:46 PM Jacob Champion wrote: > On 2/6/23 08:22, Robert Haas wrote: > > I don't think that's quite the right concept. It seems to me that the > > client is responsible for informing the server of what the situation > > is, and the server is responsible for deciding whether

Re: Track IO times in pg_stat_io

2023-03-07 Thread Andres Freund
On 2023-03-07 13:43:28 -0500, Melanie Plageman wrote: > > Now I've a second thought: what do you think about resetting the related > > number > > of operations and *_time fields when enabling/disabling track_io_timing? > > (And mention it in the doc). > > > > That way it'd prevent bad

Re: Track IO times in pg_stat_io

2023-03-07 Thread Melanie Plageman
Thanks for taking another look! On Tue, Mar 7, 2023 at 10:52 AM Drouvot, Bertrand wrote: > On 3/6/23 5:30 PM, Melanie Plageman wrote: > > Thanks for the review! > > > > On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand > > wrote: > >> On 2/26/23 5:03 PM, Melanie Plageman wrote: > >>> The

Re: Track IO times in pg_stat_io

2023-03-07 Thread Andres Freund
Hi, On 2023-03-06 11:30:13 -0500, Melanie Plageman wrote: > > As pgstat_bktype_io_stats_valid() is called only in Assert(), I think that > > would be a good idea > > to also check that if counts are not Zero then times are not Zero. > > Yes, I think adding some validation around the

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-03-07 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 07:27:59PM +0300, Önder Kalacı wrote: > On the other hand, we already have a similar problem with > recovery_min_apply_delay combined with hot_standby_feedback [1]. > So, that probably is an acceptable trade-off for the pgsql-hackers. > If you use this feature, you should

Re: HOT chain validation in verify_heapam()

2023-03-07 Thread Mark Dilger
> On Mar 7, 2023, at 10:16 AM, Robert Haas wrote: > > On Mon, Mar 6, 2023 at 12:36 PM Robert Haas wrote: >> So it seems that we still don't have a patch where the >> value of a variable called lp_valid corresponds to whether or not the >> L.P. is valid. > > Here's a worked-over version of

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-07 Thread Andres Freund
Hi, On 2023-03-06 15:21:14 -0500, Melanie Plageman wrote: > Good point. Attached is what you suggested. I committed the transaction > before the drop table so that the statistics would be visible when we > queried pg_stat_io. Pushed, thanks for the report, analysis and fix, Tom, Horiguchi-san,

Re: HOT chain validation in verify_heapam()

2023-03-07 Thread Robert Haas
On Mon, Mar 6, 2023 at 12:36 PM Robert Haas wrote: > So it seems that we still don't have a patch where the > value of a variable called lp_valid corresponds to whether or not the > L.P. is valid. Here's a worked-over version of this patch. Changes: - I got rid of the code that sets lp_valid in

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-07 Thread Nathan Bossart
On Tue, Mar 07, 2023 at 12:39:13PM +0530, Bharath Rupireddy wrote: > On Tue, Mar 7, 2023 at 3:30 AM Nathan Bossart > wrote: >> Is it possible to memcpy more than a page at a time? > > It would complicate things a lot there; the logic to figure out the > last page bytes that may or may not fit

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-07 Thread Robert Haas
On Tue, Jan 10, 2023 at 6:50 PM Tom Lane wrote: > The script-file-per-upgrade-path aspect solves a problem that you > have, whether you admit it or not; I think you simply aren't realizing > that because you have not had to deal with the consequences of > your proposed feature. Namely that you

Re: add PROCESS_MAIN to VACUUM

2023-03-07 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 10:45 PM Michael Paquier wrote: > > On Mon, Mar 06, 2023 at 04:59:49PM -0800, Nathan Bossart wrote: > > That did cross my mind, but I was worried that trying to explain all that > > here could cause confusion. > > > > If PROCESS_MAIN is set (the default), it's time to

Re: some problem explicit_bzero with building PostgreSQL on linux

2023-03-07 Thread Dimitry Markman
Hi Tom, thanks a lot Adding explicit_bzero.o did the job Thanks a lot dm From: Tom Lane Date: Tuesday, March 7, 2023 at 9:14 AM To: Dimitry Markman Cc: pgsql-hackers@lists.postgresql.org , Bhavya Dabas Subject: Re: some problem explicit_bzero with building PostgreSQL on linux Dimitry

Re: Timeline ID hexadecimal format

2023-03-07 Thread Sébastien Lardière
On 06/03/2023 18:04, Peter Eisentraut wrote: On 03.03.23 16:52, Sébastien Lardière wrote: On 02/03/2023 09:12, Peter Eisentraut wrote: I think here it would be more helpful to show actual examples. Like, here is a possible file name, this is what the different parts mean. So you mean

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-07 Thread Pavel Borisov
Hi, Andres and Alexander! On Tue, 7 Mar 2023, 10:10 Alexander Korotkov, wrote: > On Tue, Mar 7, 2023 at 4:50 AM Andres Freund wrote: > > On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > > > 2. Heap updates with low tuple concurrency: > > > Prepare with pkeys (pgbench -d postgres -i -I

Re: Track IO times in pg_stat_io

2023-03-07 Thread Drouvot, Bertrand
Hi, On 3/6/23 5:30 PM, Melanie Plageman wrote: Thanks for the review! On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand wrote: On 2/26/23 5:03 PM, Melanie Plageman wrote: The timings will only be non-zero when track_io_timing is on That could lead to incorrect interpretation if one wants

Re: Add shared buffer hits to pg_stat_io

2023-03-07 Thread Drouvot, Bertrand
Hi, On 3/6/23 4:38 PM, Melanie Plageman wrote: Thanks for the review! On Tue, Feb 28, 2023 at 7:36 AM Drouvot, Bertrand wrote: BufferDesc * LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum, -bool *foundPtr, IOContext

Re: GUC for temporarily disabling event triggers

2023-03-07 Thread Mikhail Gribkov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed I like it now. * The patch does what it intends to do; *

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-07 Thread Andrew Dunstan
On 2023-03-06 Mo 15:16, Gregory Stark (as CFM) wrote: The CFBot says this patch is failing but I find it hard to believe this is related to this patch... 2023-03-05 20:56:58.705 UTC [33902][client backend] [pg_regress/btree_index][18/750:0] STATEMENT: ALTER INDEX btree_part_idx ALTER COLUMN

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-07 Thread Maxim Orlov
On Tue, 7 Mar 2023 at 15:38, Heikki Linnakangas wrote: > > That is true for pg_multixact/offsets. We will indeed need to add an > epoch and introduce the concept of FullMultiXactIds for that. However, > we can change pg_multixact/members independently of that. We can extend > MultiXactOffset

Re: some problem explicit_bzero with building PostgreSQL on linux

2023-03-07 Thread Tom Lane
Dimitry Markman writes: > how we can guaranty that if HAVE_EXPLICIT_BZERO is not defined then > explicit_bzero function implemented in port/explicit_bzero.c will be used > (just like in Darwin or windows) Did you remember to add explicit_bzero.o to LIBOBJS in the configured Makefile.global? If

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Önder Kalacı
Hi Amit, Peter > > > Let me give an example to demonstrate why I thought something is fishy > here: > > > > > > Imagine rel has a (non-default) REPLICA IDENTITY with Oid=. > > > Imagine the same rel has a PRIMARY KEY with Oid=. > > > > Hmm, alright, this is syntactically possible, but

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-07 Thread Aleksander Alekseev
Hi, > Here's how I see the development path for this [...] > So, in my view, the plan should be [...] > Thoughts? The plan looks great! I would also explicitly include this: > As we start to refactor these things, I also think it would be good to > have more explicit tracking of the valid

Re: Raising the SCRAM iteration count

2023-03-07 Thread Daniel Gustafsson
> On 7 Mar 2023, at 09:26, Daniel Gustafsson wrote: > Right, what I meant was: can a pg_regress sql/expected test drive a psql > interactive prompt? Your comments suggested using password.sql so I was > curious if I was missing a neat trick for doing this. The attached v7 adds a TAP test for

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-07 Thread Heikki Linnakangas
On 07/03/2023 13:38, Maxim Orlov wrote: As for making pg_multixact 64 bit, I spend the last couple of days to make proper pg_upgrade for pg_multixact's and for pg_xact's with wraparound and I've understood, that it is not a simple task compare to pg_xact's. The problem is, we do not have epoch

Re: pg_dump/pg_restore: Fix stdin/stdout handling of custom format on Win32

2023-03-07 Thread Juan José Santamaría Flecha
On Thu, Mar 2, 2023 at 8:01 AM Michael Paquier wrote: > > We had better make sure that this does not break again 10260c7, and > these could not be reproduced with automated tests as they needed a > Windows terminal. Isn't this issue like the other commit, where the > automated testing cannot

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Amit Kapila
On Tue, Mar 7, 2023 at 3:00 PM Peter Smith wrote: > > On Tue, Mar 7, 2023 at 8:01 PM Amit Kapila wrote: > > > > On Tue, Mar 7, 2023 at 1:19 PM Peter Smith wrote: > > > > > > Let me give an example to demonstrate why I thought something is fishy > > > here: > > > > > > Imagine rel has a

Re: Add pg_walinspect function with block info columns

2023-03-07 Thread Matthias van de Meent
On Tue, 7 Mar 2023 at 01:34, Michael Paquier wrote: > > On Mon, Mar 06, 2023 at 04:08:28PM +0100, Matthias van de Meent wrote: > > On Mon, 6 Mar 2023 at 15:40, Bharath Rupireddy > >> IMO, pg_get_wal_records_extended_info as proposed doesn't look good to > >> me as it outputs most of the columns

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-07 Thread Maxim Orlov
On Tue, 17 Jan 2023 at 16:33, Aleksander Alekseev wrote: > Hi hackers, > > Maxim, perhaps you could share with us what your reasoning was here? > > I'm really sorry for late response, but better late than never. Yes, we can not access shared memory without lock. In this particular case, we use

some problem explicit_bzero with building PostgreSQL on linux

2023-03-07 Thread Dimitry Markman
Hi, we got some problem with building PostgreSQL (version 15.1) on linux ldd —version returns ldd (Debian GLIBC 2.31-13+deb11u5.tmw1) 2.31 we can build it all right, however we want to use binaries on different glibc version so we’re detecting usage of the glibc version > 2.17 and we need to

RE: The order of queues in row lock is changed (not FIFO)

2023-03-07 Thread Ryo Yamaji (Fujitsu)
From: Tom Lane > I don't see a bug here, or at least I'm not willing to move the goalposts to > where you want them to be. > I believe that we do guarantee arrival-order locking of individual tuple > versions. However, in the > example you show, a single row is being updated over and over.

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Önder Kalacı
Hi Amit, all > > Few comments: > = > 1. > +get_usable_indexoid(ApplyExecutionData *edata, ResultRelInfo *relinfo) > { > ... > + if (targetrelkind == RELKIND_PARTITIONED_TABLE) > + { > + /* Target is a partitioned table, so find relmapentry of the partition */ > + TupleConversionMap

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Önder Kalacı
Hi Shi Yu, all > Thanks for updating the patch. Here are some comments on v33-0001 patch. > > 1. > + if (RelationReplicaIdentityFullIndexScanEnabled(localrel) && > + remoterel->replident == REPLICA_IDENTITY_FULL) > > RelationReplicaIdentityFullIndexScanEnabled() is introduced

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Önder Kalacı
Hi Andres, Amit, all I think the case in which the patch regresses performance in is irrelevant > in > practice. > This is similar to what I think in this context. I appreciate the effort from Shi Yu, so that we have a clear understanding on the overhead. But the tests we do on [1] where we

Re: RFC: logical publication via inheritance root?

2023-03-07 Thread Aleksander Alekseev
Hi Jacob, > I'm going to register this in CF for feedback. Many thanks for the updated patch. Despite the fact that the patch is still work in progress all in all it looks very good to me. So far I only have a couple of nitpicks, mostly regarding the code coverage [1]: ``` +tablename =

Re: Add pg_walinspect function with block info columns

2023-03-07 Thread Bharath Rupireddy
On Tue, Mar 7, 2023 at 12:48 PM Michael Paquier wrote: > > On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote: > > Ah. Yes, that expansion sounds sensible. > > Okay, so, based on this idea, I have hacked on this stuff and finish > with the attached that shows block data if it

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-07 Thread Alvaro Herrera
On 2023-Mar-07, Julien Rouhaud wrote: > I registered lapwing as a 32b Debian 7 so I thought it would be expected to > keep it as-is rather than upgrading to all newer major Debian versions, > especially since there were newer debian animal registered (no 32b though > AFAICS). I'm not opposed to

Re: Add estimated hit ratio to Memoize in EXPLAIN to explain cost adjustment

2023-03-07 Thread David Rowley
On Sun, 5 Mar 2023 at 13:21, Lukas Fittl wrote: > Alternatively (or in addition) we could consider showing the "ndistinct" > value that is calculated in cost_memoize_rescan - since that's the most > significant contributor to the cache hit ratio (and you can influence that > directly by

Re: using memoize in in paralel query decreases performance

2023-03-07 Thread Pavel Stehule
út 7. 3. 2023 v 10:46 odesílatel David Rowley napsal: > On Tue, 7 Mar 2023 at 22:09, Pavel Stehule > wrote: > > I can live with it. This is an analytical query and the performance is > not too important for us. I was surprised that the performance was about > 25% worse, and so the hit ratio was

Re: using memoize in in paralel query decreases performance

2023-03-07 Thread David Rowley
On Tue, 7 Mar 2023 at 22:09, Pavel Stehule wrote: > I can live with it. This is an analytical query and the performance is not > too important for us. I was surprised that the performance was about 25% > worse, and so the hit ratio was almost zero. I am thinking, but I am not sure > if the

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Peter Smith
On Tue, Mar 7, 2023 at 8:01 PM Amit Kapila wrote: > > On Tue, Mar 7, 2023 at 1:19 PM Peter Smith wrote: > > > > Let me give an example to demonstrate why I thought something is fishy here: > > > > Imagine rel has a (non-default) REPLICA IDENTITY with Oid=. > > Imagine the same rel has a

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-07 Thread Daniel Gustafsson
> On 7 Mar 2023, at 08:33, Alexander Kukushkin wrote: > The "log_directory" GUC must be examined on both, source and target. Agreed, log_directory must be resolved to the configured values. Teaching pg_rewind about those in case they are stored in $PGDATA sounds like a good idea though. --

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-03-07 Thread Daniel Gustafsson
> On 7 Mar 2023, at 09:35, Damir Belyalov wrote: > I felt just logging "Error: %ld" would make people wonder the meaning of > the %ld. Logging something like ""Error: %ld data type errors were > found" might be clearer. > > Thanks. For more clearance change the message to: "Errors were

Re: using memoize in in paralel query decreases performance

2023-03-07 Thread Pavel Stehule
út 7. 3. 2023 v 9:58 odesílatel David Rowley napsal: > /On Tue, 7 Mar 2023 at 21:09, Pavel Stehule > wrote: > > > > po 6. 3. 2023 v 22:52 odesílatel David Rowley > napsal: > >> I wonder if the additional work_mem required for Memoize is just doing > >> something like causing kernel page cache

Re: Add pg_walinspect function with block info columns

2023-03-07 Thread Kyotaro Horiguchi
At Tue, 7 Mar 2023 16:18:21 +0900, Michael Paquier wrote in > On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote: > > Ah. Yes, that expansion sounds sensible. > > Okay, so, based on this idea, I have hacked on this stuff and finish > with the attached that shows block data if it

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-07 Thread Amit Kapila
On Tue, Mar 7, 2023 at 1:19 PM Peter Smith wrote: > > Let me give an example to demonstrate why I thought something is fishy here: > > Imagine rel has a (non-default) REPLICA IDENTITY with Oid=. > Imagine the same rel has a PRIMARY KEY with Oid=. > > --- > > +/* > + * Get replica identity

  1   2   >