Re: [RFC] building postgres with meson - v13

2022-09-21 Thread Andres Freund
Hi, On 2022-09-21 09:46:30 -0700, Andres Freund wrote: > I'm planning to commit this today, unless somebody wants to argue against > that. And done! Changes: - fixed a few typos (thanks Thomas) - less duplication in the CI tasks - removed an incomplete implementation of the target for

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-09-21 Thread Bharath Rupireddy
On Thu, Sep 22, 2022 at 10:07 AM Michael Paquier wrote: > > On Thu, Sep 22, 2022 at 07:16:41AM +0530, Bharath Rupireddy wrote: > > t/010_pg_basebackup.pl ... 134/? > > # Failed test 'pg_basebackup reports checksum mismatch stderr > > /(?^s:^WARNING.*checksum verification failed)/' > > # at

Re: Background writer and checkpointer in crash recovery

2022-09-21 Thread Michael Paquier
On Thu, Sep 15, 2022 at 10:19:01PM -0500, Justin Pryzby wrote: > I don't know that this warrants an Opened Item, but I think some fix > ought to be applied to v15, whether that happens this week or next > month. With RC1 getting close by, I have looked at that again and applied a patch that

Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-09-21 Thread Andrey Chudnovsky
First, My message from corp email wasn't displayed in the thread, That is what Jacob replied to, let me post it here for context: > We can support both passing the token from an upstream client and libpq > implementing OAUTH2 protocol to obtain one. > > Libpq implementing OAUTHBEARER is needed

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Tom Lane
Michael Paquier writes: > On Wed, Sep 21, 2022 at 11:43:56PM -0400, Tom Lane wrote: >> I don't have any opinion on the concrete merits of this change, >> but I want to note that 15rc1 wraps on Monday, and we don't like >> people pushing noncritical changes shortly before a wrap. There >> is not

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread John Naylor
On Thu, Sep 22, 2022 at 1:01 AM Nathan Bossart wrote: > > On Wed, Sep 21, 2022 at 01:17:21PM +0700, John Naylor wrote: > > > In short, this code needs to be lower level so that we still have full > > control while being portable. I will work on this, and also the related > > code for node

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-09-21 Thread Michael Paquier
On Thu, Sep 22, 2022 at 07:16:41AM +0530, Bharath Rupireddy wrote: > t/010_pg_basebackup.pl ... 134/? > # Failed test 'pg_basebackup reports checksum mismatch stderr > /(?^s:^WARNING.*checksum verification failed)/' > # at t/010_pg_basebackup.pl line 769. > # 'unrecognized

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Michael Paquier
On Wed, Sep 21, 2022 at 11:43:56PM -0400, Tom Lane wrote: > I don't have any opinion on the concrete merits of this change, > but I want to note that 15rc1 wraps on Monday, and we don't like > people pushing noncritical changes shortly before a wrap. There > is not a lot of time for fooling

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
On Wed, Sep 21, 2022 at 02:11:36PM -0700, Peter Geoghegan wrote: > On Wed, Sep 21, 2022 at 1:14 PM Nathan Bossart > wrote: >> Presumably a >> generic WAL record compression mechanism could be reused for other large >> records, too. That could be much easier than devising a deduplication >>

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
On Wed, Sep 21, 2022 at 02:41:28PM -0700, Peter Geoghegan wrote: > On Wed, Sep 21, 2022 at 2:11 PM Peter Geoghegan wrote: >> > Presumably a >> > generic WAL record compression mechanism could be reused for other large >> > records, too. That could be much easier than devising a deduplication >>

Re: make additional use of optimized linear search routines

2022-09-21 Thread Nathan Bossart
On Thu, Sep 22, 2022 at 09:52:57AM +0900, Michael Paquier wrote: > On Wed, Sep 21, 2022 at 02:28:08PM +0800, Richard Guo wrote: >> I wonder if there are other code paths we can replace with the linear >> search routines. I tried to search for them but no luck. > > I have been looking at a couple

Re: Fix snapshot name for SET TRANSACTION documentation

2022-09-21 Thread Fujii Masao
On 2022/09/21 14:40, Fujii Masao wrote: On 2022/09/21 12:01, Japin Li wrote: Hi hackers, In 6c2003f8a1bbc7c192a2e83ec51581c018aa162f, we change the snapshot name when exporting snapshot, however, there is one place we missed update the snapshot name in documentation.  Attach a patch to

Re: Query JITing with LLVM ORC

2022-09-21 Thread João Paulo Labegalini de Carvalho
Tom & Thomas: Thank you so much, those a very useful comments. I noticed that I didn't make my intentions very clear. My teams goal is to evaluate if there are any gains in JITing PostgreSQL itself, or at least parts of it, and not the expressions or parts of a query. The rationale to use

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Tom Lane
Justin Pryzby writes: > However the patch ends up, +0.75 to backpatch it to v15 rather than > calling it a new feature in v16. I don't have any opinion on the concrete merits of this change, but I want to note that 15rc1 wraps on Monday, and we don't like people pushing noncritical changes

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Justin Pryzby
On Thu, Sep 22, 2022 at 10:25:11AM +0900, Michael Paquier wrote: > On Wed, Sep 21, 2022 at 07:31:48PM -0500, Justin Pryzby wrote: > > I think at some point (maybe before releasing 1.3.4) the range was > > increased to very large(small), negative levels. It's possible to query > > the library

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-09-21 Thread wangw.f...@fujitsu.com
On Tues, Sep 20, 2022 at 18:30 PM Önder Kalacı wrote: > Thanks for the reviews, attached v12. Thanks for your patch. Here is a question and a comment: 1. In the function GetCheapestReplicaIdentityFullPath. + if (rel->pathlist == NIL) + { + /* +* A

RE: Allow logical replication to copy tables in binary format

2022-09-21 Thread osumi.takami...@fujitsu.com
Hi Few more minor comments. On Tuesday, August 16, 2022 2:04 AM Melih Mutlu wrote: > > > My main concern is to break a scenario that was previously working (14 > -> 15) but after a subscriber upgrade > it won't (14 -> 16). > > Fair concern. Some cases that might break the

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-21 Thread Bharath Rupireddy
On Wed, Sep 21, 2022 at 9:53 PM Ashutosh Sharma wrote: > > Yeah, we can either add this functionality to pg_waldump or maybe add > a new binary itself that would return this information. IMV, a separate tool isn't the way, since pg_waldump already reads WAL files and decodes WAL records, what's

Re: why can't a table be part of the same publication as its schema

2022-09-21 Thread Amit Kapila
On Wed, Sep 21, 2022 at 1:15 AM Jonathan S. Katz wrote: > > (RMT hat on, unless otherwise noted) > > On 9/20/22 9:42 AM, Robert Haas wrote: > > On Mon, Sep 19, 2022 at 11:03 PM Jonathan S. Katz > > wrote: > >> For #1 (allowing calls that have schema/table overlap...), there appears > >> to be

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-09-21 Thread Bharath Rupireddy
On Wed, Sep 21, 2022 at 8:03 AM Bharath Rupireddy wrote: > > On Fri, Aug 19, 2022 at 5:27 PM Bharath Rupireddy > wrote: > > > > Please review the attached v6 patch. > > I'm attaching the v7 patch rebased on to the latest HEAD. v7 patch was failing on Windows [1]. This is because of the full

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Michael Paquier
On Wed, Sep 21, 2022 at 07:31:48PM -0500, Justin Pryzby wrote: > I think at some point (maybe before releasing 1.3.4) the range was > increased to very large(small), negative levels. It's possible to query > the library about the lowest supported compression level, but then > there's a

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread Tom Lane
Michael Paquier writes: > On Wed, Sep 21, 2022 at 05:34:20PM -0700, David G. Johnston wrote: >> What’s the use case for automating pg_basebackup with a named replication >> slot created by the pg_basebackup command? Why can you not leverage a >> temporary replication slot (i.e., omit —slot).

Re: make additional use of optimized linear search routines

2022-09-21 Thread Michael Paquier
On Wed, Sep 21, 2022 at 02:28:08PM +0800, Richard Guo wrote: > I wonder if there are other code paths we can replace with the linear > search routines. I tried to search for them but no luck. I have been looking at a couple of simple patterns across the tree but no luck here either. Well, if

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread Michael Paquier
On Wed, Sep 21, 2022 at 05:34:20PM -0700, David G. Johnston wrote: > What’s the use case for automating pg_basebackup with a named replication > slot created by the pg_basebackup command? Why can you not leverage a > temporary replication slot (i.e., omit —slot). ISTM the create option is >

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread David G. Johnston
On Wednesday, September 21, 2022, Ashwin Agrawal wrote: > Currently, pg_basebackup has > --create-slot option to create slot if not already exists or > --slot to use existing slot > > Which means it needs knowledge on if the slot with the given name already > exists or not before invoking the

Re: pg_basebackup's --gzip switch misbehaves

2022-09-21 Thread Justin Pryzby
On Tue, Sep 13, 2022 at 04:13:20PM +0900, Michael Paquier wrote: > diff --git a/src/common/compression.c b/src/common/compression.c > index da3c291c0f..ac26287d54 100644 > --- a/src/common/compression.c > +++ b/src/common/compression.c > @@ -249,36 +299,49 @@ expect_integer_value(char *keyword,

pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread Ashwin Agrawal
Currently, pg_basebackup has --create-slot option to create slot if not already exists or --slot to use existing slot Which means it needs knowledge on if the slot with the given name already exists or not before invoking the command. If pg_basebackup --create-slot <> command fails for some

Re: [PoC] Let libpq reject unexpected authentication requests

2022-09-21 Thread Jacob Champion
On Wed, Sep 21, 2022 at 3:36 PM Peter Eisentraut wrote: > So let's look at the two TODO comments you have: > > * TODO: how should !auth_required interact with an incomplete > * SCRAM exchange? > > What specific combination of events are you thinking of here? Let's say the

Re: [RFC] building postgres with meson - v13

2022-09-21 Thread Andres Freund
Hi, On 2022-09-21 09:46:30 -0700, Andres Freund wrote: > After that I am planning to split the "ci" commit so that it converts a few of > the CI tasks to use meson, without adding all the other platforms I added for > development. I think that's important to get in soon, given that it'll >

Re: Query JITing with LLVM ORC

2022-09-21 Thread Thomas Munro
On Thu, Sep 22, 2022 at 10:04 AM João Paulo Labegalini de Carvalho wrote: >building with and w/o passing --with-llvm BTW you can also just turn it off with runtime settings, no need to rebuild.

Re: Query JITing with LLVM ORC

2022-09-21 Thread Thomas Munro
On Thu, Sep 22, 2022 at 10:35 AM Tom Lane wrote: > =?UTF-8?Q?Jo=C3=A3o_Paulo_Labegalini_de_Carvalho?= > writes: > > Good to know. I compiled from the REL_14_5 tag and did a simple experiment > > to contrast building with and w/o passing --with-llvm. > > I ran the TPC-C benchmark with 1

Re: [PoC] Let libpq reject unexpected authentication requests

2022-09-21 Thread Peter Eisentraut
On 21.09.22 17:33, Jacob Champion wrote: On Fri, Sep 16, 2022 at 1:29 PM Jacob Champion wrote: I'm happy to implement proofs of concept for that, or any other ideas, given the importance of getting this "right enough" the first time. Just let me know. v8 rebases over the postgres_fdw HINT

Re: Query JITing with LLVM ORC

2022-09-21 Thread Tom Lane
=?UTF-8?Q?Jo=C3=A3o_Paulo_Labegalini_de_Carvalho?= writes: > Good to know. I compiled from the REL_14_5 tag and did a simple experiment > to contrast building with and w/o passing --with-llvm. > I ran the TPC-C benchmark with 1 warehouse, 10 terminals, 20min of ramp-up, > and 120 of measurement

Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-09-21 Thread Jacob Champion
On Wed, Sep 21, 2022 at 3:10 PM Andrey Chudnovskiy wrote: > We can support both passing the token from an upstream client and libpq > implementing OAUTH2 protocol to obtaining one. Right, I agree that we could potentially do both. > Libpq passing toked directly from an upstream client is

Re: Query JITing with LLVM ORC

2022-09-21 Thread João Paulo Labegalini de Carvalho
Hi Thomas, It JITs expressions but not whole queries. Thanks for the clarification. > Query execution at the > tuple-flow level is still done using a C call stack the same shape as > the query plan, but it *could* be transformed to a different control > flow that could be run more efficiently

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Peter Geoghegan
On Wed, Sep 21, 2022 at 2:11 PM Peter Geoghegan wrote: > > Presumably a > > generic WAL record compression mechanism could be reused for other large > > records, too. That could be much easier than devising a deduplication > > strategy for every record type. > > It's quite possible that that's a

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Peter Geoghegan
On Wed, Sep 21, 2022 at 1:14 PM Nathan Bossart wrote: > This idea seems promising. I see that you called this patch a > work-in-progress, so I'm curious what else you are planning to do with it. I really just meant that the patch wasn't completely finished at that point. I hadn't yet convinced

Re: pg_auth_members.grantor is bunk

2022-09-21 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 18, 2022 at 1:26 PM Robert Haas wrote: >> CI is happier with this version, so I've committed 0001. If no major >> problems emerge, I'll proceed with 0002 as well. > Done. Shouldn't the CF entry [1] be closed as committed? regards, tom

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
On Tue, Sep 20, 2022 at 03:12:00PM -0700, Peter Geoghegan wrote: > On Mon, Sep 12, 2022 at 2:01 PM Peter Geoghegan wrote: >> I'd like to talk about one such technique on this thread. The attached >> WIP patch reduces the size of xl_heap_freeze_page records by applying >> a simple deduplication

Re: Query JITing with LLVM ORC

2022-09-21 Thread Thomas Munro
On Thu, Sep 22, 2022 at 4:17 AM João Paulo Labegalini de Carvalho wrote: > I am working on a project with LLVM ORC that led us to PostgreSQL as a target > application. We were surprised by learning that PGSQL already uses LLVM ORC > to JIT certain queries. It JITs expressions but not whole

Re: [RFC] building postgres with meson - v13

2022-09-21 Thread Justin Pryzby
On Wed, Sep 21, 2022 at 09:46:30AM -0700, Andres Freund wrote: > I think we should: > > - convert windows to build with ninja - it builds faster, runs all tests, > parallelizes tests. That means that msbuild based builds don't have coverage > via CI / cfbot, but we don't currently have the

Re: [RFC] building postgres with meson - v13

2022-09-21 Thread Andres Freund
Hi, On 2022-09-21 13:56:37 -0400, Tom Lane wrote: > Andres Freund writes: > > I think we should: > > > - convert windows to build with ninja - it builds faster, runs all tests, > > parallelizes tests. That means that msbuild based builds don't have > > coverage > > via CI / cfbot, but we

Re: Proposal to use JSON for Postgres Parser format

2022-09-21 Thread Matthias van de Meent
On Tue, 20 Sept 2022 at 17:29, Alvaro Herrera wrote: > > On 2022-Sep-20, Matthias van de Meent wrote: > > > Allow me to add: compressability > > > > In the thread surrounding [0] there were complaints about the size of > > catalogs, and specifically the template database. Significant parts of > >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread Nathan Bossart
On Wed, Sep 21, 2022 at 01:17:21PM +0700, John Naylor wrote: > In trying to wrap the SIMD code behind layers of abstraction, the latest > patch (and Nathan's cleanup) threw it away in almost all cases. To explain, > we need to talk about how vectorized code deals with the "tail" that is too >

Re: [RFC] building postgres with meson - v13

2022-09-21 Thread Tom Lane
Andres Freund writes: > I think we should: > - convert windows to build with ninja - it builds faster, runs all tests, > parallelizes tests. That means that msbuild based builds don't have coverage > via CI / cfbot, but we don't currently have the resources to test both. Check. The sooner

Re: why can't a table be part of the same publication as its schema

2022-09-21 Thread Amit Kapila
On Wed, Sep 21, 2022 at 8:24 PM Jonathan S. Katz wrote: > > On 9/21/22 10:24 AM, Alvaro Herrera wrote: > > On 2022-Sep-20, Robert Haas wrote: > > > >>> I don't think we should change this behavior that's already in logical > >>> replication. While I understand the reasons why "GRANT ... ALL

Re: binary version of pg_current_wal_insert_lsn and pg_walfile_name functions

2022-09-21 Thread Ashutosh Sharma
On Tue, Sep 20, 2022 at 5:13 PM Bharath Rupireddy wrote: > > On Mon, Sep 19, 2022 at 8:19 PM Ashutosh Sharma wrote: > > > > Hi All, > > > > Currently, we have pg_current_wal_insert_lsn and pg_walfile_name sql > > functions which gives us information about the next wal insert > > location and the

Query JITing with LLVM ORC

2022-09-21 Thread João Paulo Labegalini de Carvalho
Hi all, I am working on a project with LLVM ORC that led us to PostgreSQL as a target application. We were surprised by learning that PGSQL already uses LLVM ORC to JIT certain queries. I would love to know what motivated this feature and for what it is being currently used for, as it is not

Re: Query Jumbling for CALL and SET utility statements

2022-09-21 Thread Fujii Masao
On 2022/09/19 15:29, Drouvot, Bertrand wrote: Please find attached v6 taking care of the remarks mentioned above. Thanks for updating the patch! +SET pg_stat_statements.track_utility = TRUE; + +-- PL/pgSQL procedure and pg_stat_statements.track = all +-- we drop and recreate the procedures

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-09-21 Thread Jacob Champion
On Tue, Sep 20, 2022 at 4:19 PM Jacob Champion wrote: > > 2. Add support to pass on the OAuth bearer token. In this > > obtaining the bearer token is left to 3rd party application or user. > > > > ./psql -U -d 'dbname=postgres > > oauth_client_id= oauth_bearer_token= > > This hurts,

Re: Is it time to drop fix-old-flex-code.pl?

2022-09-21 Thread Julien Rouhaud
On Wed, Sep 21, 2022 at 10:21:25AM -0400, Tom Lane wrote: > > lapwing uses -Werror, so it will be unhappy, but I don't think it's > unreasonable to say that you should be using fairly modern tools > if you want to use -Werror. I think that the -Werror helped to find problems multiple times

Re: Silencing the remaining clang 15 warnings

2022-09-21 Thread Tom Lane
HEAD and v15 now compile cleanly for me with clang 15.0.0, but I find that there's still work to do in the back branches: * There are new(?) -Wunused-but-set-variable warnings in every older branch, which we evidently cleaned up or rewrote at one point or another. I think this is definitely

Re: [PoC] Let libpq reject unexpected authentication requests

2022-09-21 Thread Jacob Champion
On Fri, Sep 16, 2022 at 1:29 PM Jacob Champion wrote: > I'm happy to implement proofs of concept for that, or any other ideas, > given the importance of getting this "right enough" the first time. > Just let me know. v8 rebases over the postgres_fdw HINT changes; there are no functional

Re: confirmed_flush_lsn shows LSN of the data that has not yet been received by the logical subscriber.

2022-09-21 Thread Ashutosh Sharma
On Wed, Sep 21, 2022 at 7:21 PM Ashutosh Bapat wrote: > > On Mon, Sep 19, 2022 at 8:09 PM Ashutosh Sharma wrote: > > > > On Mon, Sep 19, 2022 at 5:24 PM Ashutosh Bapat > > wrote: > > > > > > On Mon, Sep 19, 2022 at 1:43 PM Ashutosh Sharma > > > wrote: > > > > > > > > On Fri, Sep 9, 2022 at

Re: [PATCH] polish the error message of creating proc

2022-09-21 Thread Junwang Zhao
On Wed, Sep 21, 2022 at 10:53 PM Tom Lane wrote: > > Junwang Zhao writes: > > I noticed that there are some translations under the backend/po directory, > > can we just change > > msgid "function \"%s\" already exists with same argument types" > > to > > msgid "%s \"%s\" already exists with same

Re: why can't a table be part of the same publication as its schema

2022-09-21 Thread Jonathan S. Katz
On 9/21/22 10:24 AM, Alvaro Herrera wrote: On 2022-Sep-20, Robert Haas wrote: I don't think we should change this behavior that's already in logical replication. While I understand the reasons why "GRANT ... ALL TABLES IN SCHEMA" has a different behavior (i.e. it's not applied to future

Re: [PATCH] polish the error message of creating proc

2022-09-21 Thread Tom Lane
Junwang Zhao writes: > I noticed that there are some translations under the backend/po directory, > can we just change > msgid "function \"%s\" already exists with same argument types" > to > msgid "%s \"%s\" already exists with same argument types" ? No. This doesn't satisfy our message

Re: ICU for global collation

2022-09-21 Thread Peter Eisentraut
On 21.09.22 08:50, Marina Polyakova wrote: On 2022-09-20 12:59, Peter Eisentraut wrote: On 17.09.22 10:33, Marina Polyakova wrote: 3. The locale provider is ICU, but it has not yet been set from the template database: $ initdb --locale-provider icu --icu-locale en-US -D data && pg_ctl -D

Re: why can't a table be part of the same publication as its schema

2022-09-21 Thread Alvaro Herrera
On 2022-Sep-20, Robert Haas wrote: > > I don't think we should change this behavior that's already in logical > > replication. While I understand the reasons why "GRANT ... ALL TABLES IN > > SCHEMA" has a different behavior (i.e. it's not applied to future > > objects) and do not advocate to

Re: [PATCH] polish the error message of creating proc

2022-09-21 Thread Julien Rouhaud
On Wed, Sep 21, 2022 at 10:35:47PM +0800, Junwang Zhao wrote: > On Wed, Sep 21, 2022 at 8:17 PM Julien Rouhaud wrote: > > > > You can't put part of the message in parameter, as the resulting string > > isn't > > translatable. You should either use "routine" as a generic term or provide > > 3 >

Re: Removing unused parameter in SnapBuildGetOrBuildSnapshot

2022-09-21 Thread Zhang Mingli
On Sep 21, 2022, 22:22 +0800, Melih Mutlu , wrote: > Hi hackers, > > Sharing a small patch to remove an unused parameter in  > SnapBuildGetOrBuildSnapshot function in snapbuild.c > > With commit 6c2003f8a1bbc7c192a2e83ec51581c018aa162f, SnapBuildBuildSnapshot > no longer needs transaction id.

Re: [PATCH] polish the error message of creating proc

2022-09-21 Thread Junwang Zhao
On Wed, Sep 21, 2022 at 8:17 PM Julien Rouhaud wrote: > > Hi, > > On Wed, Sep 21, 2022 at 07:45:01PM +0800, Junwang Zhao wrote: > > when a error occurs when creating proc, it should point out the > > specific proc kind instead of just printing "function". > > You should have kept the original

Removing unused parameter in SnapBuildGetOrBuildSnapshot

2022-09-21 Thread Melih Mutlu
Hi hackers, Sharing a small patch to remove an unused parameter in SnapBuildGetOrBuildSnapshot function in snapbuild.c With commit 6c2003f8a1bbc7c192a2e83ec51581c018aa162f, SnapBuildBuildSnapshot no longer needs transaction id. This also makes the xid parameter in SnapBuildGetOrBuildSnapshot

Is it time to drop fix-old-flex-code.pl?

2022-09-21 Thread Tom Lane
In view of Author: John Naylor Branch: master [8b878bffa] 2022-09-09 12:55:23 +0700 Bump minimum version of Flex to 2.5.35 I wonder if we should go a little further and get rid of src/tools/fix-old-flex-code.pl (just in HEAD, to be clear). That does nothing when flex is 2.5.36 or newer,

Re: Mingw task for Cirrus CI

2022-09-21 Thread Andres Freund
Hi, On 2022-09-21 16:18:43 +0300, Melih Mutlu wrote: > I've been thinking about how to make the mingw task run only for this patch > on cfbot and not for others. TBH, I couldn't come up with a nice way to > achieve this. > > Does anyone have any suggestions on this? Add a commented-out

Re: Tighten pg_get_object_address argument checking

2022-09-21 Thread Peter Eisentraut
On 21.09.22 12:01, Amit Kapila wrote: On Tue, Sep 20, 2022 at 11:14 PM Peter Eisentraut wrote: For publication schemas (OBJECT_PUBLICATION_NAMESPACE) and user mappings (OBJECT_USER_MAPPING), pg_get_object_address() checked the array length of the second argument, but not of the first

Re: confirmed_flush_lsn shows LSN of the data that has not yet been received by the logical subscriber.

2022-09-21 Thread Ashutosh Bapat
On Mon, Sep 19, 2022 at 8:09 PM Ashutosh Sharma wrote: > > On Mon, Sep 19, 2022 at 5:24 PM Ashutosh Bapat > wrote: > > > > On Mon, Sep 19, 2022 at 1:43 PM Ashutosh Sharma > > wrote: > > > > > > On Fri, Sep 9, 2022 at 5:36 PM Ashutosh Bapat > > > wrote: > > > > > > > > On Thu, Sep 8, 2022 at

Re: On login trigger: take three

2022-09-21 Thread Sergey Shinderuk
On 20.09.2022 17:10, Daniel Gustafsson wrote: On 20 Sep 2022, at 15:43, Sergey Shinderuk wrote: There is a race around setting and clearing of dathasloginevt. Thanks for the report! The whole dathasloginevt mechanism is IMO too clunky and unelegant to go ahead with, I wouldn't be surprised

Re: Mingw task for Cirrus CI

2022-09-21 Thread Melih Mutlu
Hi hackers, Justin Pryzby , 5 Eyl 2022 Pzt, 14:50 tarihinde şunu yazdı: > But cfbot should run the Mingw task for this patch's own commitfest > entry. But right now (because cfbot doesn't include the original commit > message/s), it doesn't get run :( > I've been thinking about how to make the

Re: Summary function for pg_buffercache

2022-09-21 Thread Aleksander Alekseev
Hi Andres, > All you need to do is to read BufferDesc->state into a local variable and > then make decisions based on that You are right, thanks. Here is the corrected patch. -- Best regards, Aleksander Alekseev v9-0001-Added-pg_buffercache_summary-function.patch Description: Binary data

Re: Support pg_attribute_aligned and noreturn in MSVC

2022-09-21 Thread James Coleman
On Tue, Sep 20, 2022 at 9:18 PM Michael Paquier wrote: > > On Tue, Sep 20, 2022 at 08:01:20AM -0400, James Coleman wrote: > > I don't have access to a Windows machine for testing, but re-reading > > the documentation it looks like the issue is that our noreturn macro > > is used after the

Re: [PATCH] polish the error message of creating proc

2022-09-21 Thread Julien Rouhaud
Hi, On Wed, Sep 21, 2022 at 07:45:01PM +0800, Junwang Zhao wrote: > when a error occurs when creating proc, it should point out the > specific proc kind instead of just printing "function". You should have kept the original thread in copy (1), or at least mention it. > diff --git

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2022-09-21 Thread Damir Belyalov
Thank you for reviewing. In the previous patch there was an error when processing constraints. The patch was fixed, but the code grew up and became more complicated (0005-COPY_IGNORE_ERRORS). I also simplified the logic of safeNextCopyFrom(). You asked why we need subtransactions, so the answer is

Re: tweak to a few index tests to hits ambuildempty() routine.

2022-09-21 Thread Alvaro Herrera
On 2022-Sep-21, a.kozhemya...@postgrespro.ru wrote: > After analyzing this, I found out why we don't reach that Assert but we have > coverage shown - firstly, it reached via another test, vacuum; secondly, it > depends on the gcc optimization flag. We reach that Assert only when using > -O0. > If

RE: [Proposal] Add foreign-server health checks infrastructure

2022-09-21 Thread kuroda.hay...@fujitsu.com
Dear Fujii-san, Thanks for checking! > These failed to be applied to the master branch cleanly. Could you update > them? PSA rebased patches. I reviewed my myself and they contain changes. E.g., move GUC-related code to option.c. > + this option relies on kernel events exposed by Linux,

[PATCH] polish the error message of creating proc

2022-09-21 Thread Junwang Zhao
when a error occurs when creating proc, it should point out the specific proc kind instead of just printing "function". diff --git a/src/backend/catalog/pg_proc.c b/src/backend/catalog/pg_proc.c index a9fe45e347..58af4b48ce 100644 --- a/src/backend/catalog/pg_proc.c +++

Re: Hash index build performance tweak from sorting

2022-09-21 Thread Simon Riggs
On Wed, 21 Sept 2022 at 02:32, David Rowley wrote: > > On Tue, 2 Aug 2022 at 03:37, Simon Riggs wrote: > > Using the above test case, I'm getting a further 4-7% improvement on > > already committed code with the attached patch, which follows your > > proposal. > > > > The patch passes info via a

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-21 Thread Bharath Rupireddy
On Wed, Sep 21, 2022 at 12:27 PM Michael Paquier wrote: > > >>> I've also taken help of the error callback mechanism to clean up the > >>> allocated memory in case of a failure. For do_pg_abort_backup() cases, > >>> I think we don't need to clean the memory because that gets called on > >>> proc

Re: Perform streaming logical transactions by background workers and parallel apply

2022-09-21 Thread Amit Kapila
On Wed, Sep 21, 2022 at 2:55 PM Peter Smith wrote: > > == > > 3. .../replication/logical/applyparallelworker.c - parallel_apply_can_start > > +/* > + * Returns true, if it is allowed to start a parallel apply worker, false, > + * otherwise. > + */ > +static bool >

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-21 Thread Bharath Rupireddy
On Wed, Sep 21, 2022 at 12:15 PM Fujii Masao wrote: > > This looks much complicated to me. > > Instead of making allocate_backup_state() or reset_backup_state() > store the label name in BackupState before do_pg_backup_start(), > how about making do_pg_backup_start() do that after checking its

Re: [PATCH]Feature improvement for MERGE tab completion

2022-09-21 Thread Alvaro Herrera
On 2022-Sep-21, Fujii Masao wrote: > How about adding something like PartialMatches() that checks whether > the keywords are included in the input string or not? If so, we can restrict > some tab-completion rules to operating only on MERGE, as follows. I attached > the WIP patch (0002 patch) that

Re: Add common function ReplicationOriginName.

2022-09-21 Thread Peter Smith
On Wed, Sep 21, 2022 at 8:08 PM Amit Kapila wrote: > > On Wed, Sep 21, 2022 at 3:09 PM Peter Smith wrote: > > > > On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila wrote: > > > > > ... > > > > > Can't we use the existing function ReplicationOriginNameForTablesync() > > > by passing relid as

Re: making relfilenodes 56 bits

2022-09-21 Thread Dilip Kumar
On Tue, Sep 20, 2022 at 10:44 PM Robert Haas wrote: Thanks for the review, please see my response inline for some of the comments, rest all are accepted. > On Fri, Sep 9, 2022 at 6:02 AM Dilip Kumar wrote: > > [ new patch ] > > +typedef pg_int64 RelFileNumber; > > This seems really random to

Re: Add common function ReplicationOriginName.

2022-09-21 Thread Amit Kapila
On Wed, Sep 21, 2022 at 3:09 PM Peter Smith wrote: > > On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila wrote: > > > ... > > > Can't we use the existing function ReplicationOriginNameForTablesync() > > by passing relid as InvalidOid for this purpose? We need a check > > inside to decide which name to

Re: Tighten pg_get_object_address argument checking

2022-09-21 Thread Amit Kapila
On Tue, Sep 20, 2022 at 11:14 PM Peter Eisentraut wrote: > > For publication schemas (OBJECT_PUBLICATION_NAMESPACE) and user > mappings (OBJECT_USER_MAPPING), pg_get_object_address() checked the > array length of the second argument, but not of the first argument. > If the first argument was too

Re: Add common function ReplicationOriginName.

2022-09-21 Thread Peter Smith
On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila wrote: > ... > Can't we use the existing function ReplicationOriginNameForTablesync() > by passing relid as InvalidOid for this purpose? We need a check > inside to decide which name to construct, otherwise, it should be > fine. If we agree with this,

Re: SLRUs in the main buffer pool, redux

2022-09-21 Thread Thomas Munro
On Sat, Sep 17, 2022 at 12:41 PM Bagga, Rishu wrote: > While I was working on adding the page headers to SLRU pages on your patch, I > came across this code where it seems like "MultiXactIdToMemberPage" is > mistakenly being used instead of MultiXactIdToOffsetPage in the TrimMultiXact >

Re: Perform streaming logical transactions by background workers and parallel apply

2022-09-21 Thread Peter Smith
Here are some review comments for patch v30-0001. == 1. Commit message In addition, the patch extends the logical replication STREAM_ABORT message so that abort_time and abort_lsn can also be sent which can be used to update the replication origin in parallel apply worker when the streaming

Re: RFC: Logging plan of the running query

2022-09-21 Thread Alena Rybakina
 Ok, I get it. Since GetLockMethodLocalHash() is only used for assertions, this is only defined when USE_ASSERT_CHECKING is enabled. This patch uses GetLockMethodLocalHash() not only for the assertion purpose, so I removed "ifdef USE_ASSERT_CHECKING" for this function. I belive it does not

Multi-insert related comment in CopyFrom()

2022-09-21 Thread Etsuro Fujita
Hi, While working on the “Fast COPY FROM based on batch insert” patch, I noticed this: else if (proute != NULL && resultRelInfo->ri_TrigDesc != NULL && resultRelInfo->ri_TrigDesc->trig_insert_new_table) { /* * For partitioned tables we can't support

Re: tweak to a few index tests to hits ambuildempty() routine.

2022-09-21 Thread a . kozhemyakin
After analyzing this, I found out why we don't reach that Assert but we have coverage shown - firstly, it reached via another test, vacuum; secondly, it depends on the gcc optimization flag. We reach that Assert only when using -O0. If we build with -O2 or -Og that function is not reached (due

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-21 Thread Michael Paquier
On Wed, Sep 21, 2022 at 03:45:49PM +0900, Fujii Masao wrote: > Instead of making allocate_backup_state() or reset_backup_state() > store the label name in BackupState before do_pg_backup_start(), > how about making do_pg_backup_start() do that after checking its length? > Seems this can simplify

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-21 Thread Michael Paquier
On Tue, Sep 20, 2022 at 05:13:49PM +0530, Bharath Rupireddy wrote: > On Tue, Sep 20, 2022 at 7:20 AM Michael Paquier wrote: > I've separated out these variables from the backup state, please see > the attached v9 patch. Thanks, the separation looks cleaner. >>> I've also taken help of the error

Assign TupleTableSlot->tts_tableOid duplicated in tale AM.

2022-09-21 Thread Wenchao Zhang
Hi hackers, Recently, we discover that the field of tts_tableOid of TupleTableSlot is assigned duplicated in table AM's interface which is not necessary. For example, in table_scan_getnextslot, ``` static inline bool table_scan_getnextslot(TableScanDesc sscan, ScanDirection direction,

Re: ICU for global collation

2022-09-21 Thread Marina Polyakova
On 2022-09-20 12:59, Peter Eisentraut wrote: On 17.09.22 10:33, Marina Polyakova wrote: 3. The locale provider is ICU, but it has not yet been set from the template database: $ initdb --locale-provider icu --icu-locale en-US -D data && pg_ctl -D data -l logfile start && createdb

Re: Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

2022-09-21 Thread Fujii Masao
On 2022/09/20 20:43, Bharath Rupireddy wrote: - if (strlen(backupidstr) > MAXPGPATH) + if (state->name_overflowed == true) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("backup label too long (max %d bytes)", It does not

Re: make additional use of optimized linear search routines

2022-09-21 Thread Richard Guo
On Wed, Sep 21, 2022 at 1:40 PM Michael Paquier wrote: > In short, switching those code paths to use the linear search routines > looks like a good thing in the long-term, so I would like to apply > this patch. If you have any comments or objections, please feel > free. Yeah, I agree that the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread John Naylor
On Tue, Sep 20, 2022 at 3:19 PM Masahiko Sawada wrote: > > On Fri, Sep 16, 2022 at 4:54 PM John Naylor > wrote: > > Here again, I'd rather put this off and focus on getting the "large > > details" in good enough shape so we can got towards integrating with > > vacuum. > > Thank you for the