Re: Commitfest 2023-03 starting tomorrow!

2023-03-21 Thread Thomas Munro
On Tue, Mar 21, 2023 at 10:59 PM Alvaro Herrera wrote: > I gave a talk on Friday at a private EDB mini-conference about the > PostgreSQL open source process; and while preparing for that one, I > ran some 'git log' commands to obtain the number of code contributors > for each release, going back

Re: Error "initial slot snapshot too large" in create replication slot

2023-03-21 Thread Kyotaro Horiguchi
At Mon, 20 Mar 2023 13:46:51 -0400, "Gregory Stark (as CFM)" wrote in > On Wed, 12 Oct 2022 at 01:10, Michael Paquier wrote: > > The discussion seems to have stopped here. As this is classified as a > > bug fix, I have moved this patch to next CF, waiting on author for the > > moment. > >

Re: Initial Schema Sync for Logical Replication

2023-03-21 Thread Amit Kapila
On Wed, Mar 22, 2023 at 8:29 AM Masahiko Sawada wrote: > > On Tue, Mar 21, 2023 at 8:18 PM Amit Kapila wrote: > > > > On Tue, Mar 21, 2023 at 7:32 AM Euler Taveira wrote: > > > > > You should > > > exclude them removing these objects from the TOC before running > > > pg_restore or > > > adding

Re: CREATE DATABASE ... STRATEGY WAL_LOG issues

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 09:34:14 -0700, Andres Freund wrote: > On 2023-03-21 11:33:59 -0400, Robert Haas wrote: > > That feels like it would be slightly more rational behavior, > > but I'm not smart enough to guess whether anyone would actually be > > happier (or less happy) after such a change than

Re: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread Peter Smith
Here are some review comments for patch code of HEAD_v19-0001 == doc/src/sgml/ref/create_publication.sgml 1. + + There can be a case where a subscription combines multiple + publications. If a root partitioned table is published by any + subscribed

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-03-21 Thread Michael Paquier
On Wed, Mar 22, 2023 at 11:37:03AM +0900, Kyotaro Horiguchi wrote: > In the original description, "buffer fetches" appears to be a plural > form of a compound noun and correct, similar to "buffer hits" > mentioned later. If we reword it, I think it should be "number of > buffers fetched". Using

Re: Using AF_UNIX sockets always for tests on Windows

2023-03-21 Thread Thomas Munro
Thanks Tom, Andres, Juan José, Andrew for your feedback. I agree that it's a better "OS harmonisation" outcome if we can choose both ways on both OSes. I will leave the 0003 patch aside for now due to lack of time, but here's a rebase of the first two patches. Since this is really just more

Re: Commitfest 2023-03 starting tomorrow!

2023-03-21 Thread Greg Stark
So a week later Status summary: March 15March 22 Needs review: 152 128 Waiting on Author: 42 36 Ready for Committer: 39 32 Committed: 61 82 Moved to next CF: 4 15 Withdrawn: 17

Re: Kerberos delegation support in libpq and postgres_fdw

2023-03-21 Thread Greg Stark
The CFBot says there's a function be_gssapi_get_proxy() which is undefined. Presumably this is a missing #ifdef or a definition that should be outside an #ifdef. [14:05:21.532] dblink.c: In function ‘dblink_security_check’: [14:05:21.532] dblink.c:2606:38: error: implicit declaration of function

RE: Allow logical replication to copy tables in binary format

2023-03-21 Thread shiy.f...@fujitsu.com
On Wed Mar 22, 2023 7:29 AM Peter Smith wrote: > > Thanks for all the patch updates. Patch v19 LGTM. > +1 Regards, Shi Yu

Re: [PATCH] Optional OR REPLACE in CREATE OPERATOR statement

2023-03-21 Thread Gregory Stark (as CFM)
On Tue, 5 Jul 2022 at 11:29, Tom Lane wrote: > > No, that's not acceptable. CREATE OR REPLACE should always produce > exactly the same final state of the object, but in this case we cannot > change the underlying function if the operator already exists. It sounds like this patch isn't the

RE: Question: Do we have a rule to use "PostgreSQL" and "PostgreSQL" separately?

2023-03-21 Thread Hayato Kuroda (Fujitsu)
Dear Daniel, Tom, > > On 20 Mar 2023, at 15:31, Tom Lane wrote: > > > > "Hayato Kuroda (Fujitsu)" writes: > >> While checking documentations, I found that one line notes our product as > >> "PostgreSQL", whereas another line notes > as just > >> "PostgreSQL". > > > > IMO the convention is to

Re: Raising the SCRAM iteration count

2023-03-21 Thread Gregory Stark (as CFM)
On Tue, 14 Mar 2023 at 14:54, Gregory Stark (as CFM) wrote: > > CFBot is failing with this test failure... I'm not sure if this just > represents a timing dependency or a bad test or what? CFBot is now consistently showing these test failures. I think there might actually be a problem here? >

Re: Commitfest 2023-03 starting tomorrow!

2023-03-21 Thread Greg Stark
On Tue, 21 Mar 2023 at 05:59, Alvaro Herrera wrote: > > On 2023-Mar-20, Thomas Munro wrote: > > This led me to suggesting that perhaps we need to be more lenient when > it comes to new contributors. As I said, for seasoned contributors, > it's not a problem to keep up with our requirements,

Re: Initial Schema Sync for Logical Replication

2023-03-21 Thread Masahiko Sawada
On Tue, Mar 21, 2023 at 8:18 PM Amit Kapila wrote: > > On Tue, Mar 21, 2023 at 7:32 AM Euler Taveira wrote: > > > > On Mon, Mar 20, 2023, at 10:10 PM, Kumar, Sachin wrote: > > > > > From: Alvaro Herrera > > > Subject: RE: [EXTERNAL]Initial Schema Sync for Logical Replication > > > On

misplaced GUC in pqcomm.h -- where to put actual common variable though...?

2023-03-21 Thread Greg Stark
So back in 2002 in 7.3 there was a commit 2c6b34d9598 which added a GUC db_user_namespace which is stored in a variable Db_user_namespace. All that seems fine except... The variable this GUC is stored in is Db_user_namespace which... is actually declared in pqcomm.h which is intended to be

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-03-21 Thread Kyotaro Horiguchi
At Wed, 22 Mar 2023 10:16:12 +0900, Michael Paquier wrote in > On Mon, Mar 20, 2023 at 11:57:31AM +0100, Drouvot, Bertrand wrote: > > "Buffer" sounds more appropriate to me, so the attached has been done that > > way. > > This choice is OK for me. > > > + > > +

Re: Improve logging when using Huge Pages

2023-03-21 Thread Justin Pryzby
On Mon, Mar 20, 2023 at 02:03:13PM +0900, Michael Paquier wrote: > On Mon, Mar 20, 2023 at 01:54:46PM +0900, Michael Paquier wrote: > > The main advantage of a read-only GUC over a function is that users > > would not need to start a postmaster to know if huge pages would be > > active or not.

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-21 Thread shiy.f...@fujitsu.com
On Tuesday, March 21, 2023 8:03 PM Önder Kalacı wrote: > > Attached patches again. > Thanks for updating the patch. @@ -408,15 +412,18 @@ $node_subscriber->wait_for_subscription_sync; $node_publisher->safe_psql( 'postgres', qq( ALTER TABLE dropped_cols DROP COLUMN

Re: Show various offset arrays for heap WAL records

2023-03-21 Thread Peter Geoghegan
On Tue, Mar 21, 2023 at 3:37 PM Peter Geoghegan wrote: > One problem that I often run into when performing analysis of VACUUM > using pg_walinspect is the issue of *who* pruned which heap page, for > any given PRUNE record. Was it VACUUM/autovacuum, or was it > opportunistic pruning? There is no

Re: Remove nonmeaningful prefixes in PgStat_* fields

2023-03-21 Thread Michael Paquier
On Mon, Mar 20, 2023 at 10:05:21AM +0100, Drouvot, Bertrand wrote: > Good point and keeping it as it is currently would not > affect the work that is/will be done in [1]. I guess that I'm OK with that to get more of pgstatfuncs.c to use macros for the function definitions there.. Alvaro, Tom,

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

2023-03-21 Thread Michael Paquier
On Mon, Mar 20, 2023 at 11:57:31AM +0100, Drouvot, Bertrand wrote: > "Buffer" sounds more appropriate to me, so the attached has been done that > way. This choice is OK for me. > + > + pg_stat_get_xact_blocks_fetched > + > +pg_stat_get_xact_blocks_fetched ( >

Re: Track IO times in pg_stat_io

2023-03-21 Thread Melanie Plageman
On Mon, Mar 20, 2023 at 10:34 PM Andres Freund wrote: > > Hi, > > On 2023-03-16 17:19:16 -0400, Melanie Plageman wrote: > > > > > I wonder if we should get rid of pgStatBlockReadTime, > > > > > pgStatBlockWriteTime, > > > > > > > > And then have pg_stat_reset_shared('io') reset pg_stat_database

Re: Fix fseek() detection of unseekable files on WIN32

2023-03-21 Thread Michael Paquier
On Mon, Mar 20, 2023 at 07:06:22AM +0900, Michael Paquier wrote: > Not sure about this one. I have considered it and dirmod.c includes > also bits for cygwin, while being aimed for higher-level routines like > rename(), unlink() or symlink(). This patch is only for WIN32, and > aimed for common

Re: allow_in_place_tablespaces vs. pg_basebackup

2023-03-21 Thread Michael Paquier
On Mon, Mar 20, 2023 at 07:56:42AM -0400, Robert Haas wrote: > Anyone want to comment on this? I have not checked the patch in details, but perhaps this needs at least one test? -- Michael signature.asc Description: PGP signature

Re: Missing rules for queryjumblefuncs.{funcs,switch}.c for maintainer-clean

2023-03-21 Thread Michael Paquier
On Mon, Mar 20, 2023 at 04:04:31PM +0800, Richard Guo wrote: > Agreed. +1 to remove the counts. Thanks. Adjusted this way, then. -- Michael signature.asc Description: PGP signature

Re: Allow logical replication to copy tables in binary format

2023-03-21 Thread Peter Smith
Thanks for all the patch updates. Patch v19 LGTM. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: Show various offset arrays for heap WAL records

2023-03-21 Thread Peter Geoghegan
On Mon, Mar 13, 2023 at 6:41 PM Peter Geoghegan wrote: > There are several different things that seem important to me > personally. These are in tension with each other, to a degree. These > are: > > 1. Like Andres, I'd really like to have some way of inspecting things > like heapam PRUNE,

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Matthias van de Meent
On Tue, 21 Mar 2023 at 23:05, Andres Freund wrote: > > Hi, > > On 2023-03-21 21:02:08 +0100, Matthias van de Meent wrote: > > On Tue, 21 Mar 2023 at 20:58, Andres Freund wrote: > > > On 2023-03-21 20:20:40 +0100, Matthias van de Meent wrote: > > > > Yes, attcacheoff is a tremendous performance

Re: doc: add missing "id" attributes to extension packaging page

2023-03-21 Thread Brar Piening
On 17.01.2023 at 23:43, Karl O. Pinc wrote: It's good you asked. After poking about the XSLT 1.0 spec to refresh my memory I abandoned the "mode" approach entirely. The real "right way" is to use the import mechanism. It actually is not. After quite some time trying to figure out why things

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 21:02:08 +0100, Matthias van de Meent wrote: > On Tue, 21 Mar 2023 at 20:58, Andres Freund wrote: > > On 2023-03-21 20:20:40 +0100, Matthias van de Meent wrote: > > > Yes, attcacheoff is a tremendous performance boon in many cases. > > > > Which? We don't use fastgetattr() in

Re: proposal: possibility to read dumped table's name from file

2023-03-21 Thread Pavel Stehule
út 21. 3. 2023 v 16:32 odesílatel Justin Pryzby napsal: > On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote: > > ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby > napsal: > > > > > On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote: > > > > rebase + enhancing about

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Jeff Davis
On Tue, 2023-03-21 at 09:22 -0400, Dave Cramer wrote: > As Jeff mentioned there is a visibility problem if the search path is > changed. The simplest solution IMO is to look up the OID at the time > the format is requested and use the OID going forward to format the > output as binary. If the

Re: pgsql: Stop recommending auto-download of DTD files, and indeed disable

2023-03-21 Thread Robert Haas
On Tue, Mar 21, 2023 at 4:45 PM Robert Haas wrote: > I use MacPorts, rather than Homebrew, but still found it necessary to > do something similar, specifically: > > export XML_CATALOG_FILES=/opt/local/etc/xml/catalog Ah, never mind. I had an incorrect value in my environment. If I unset it

Re: pgsql: Stop recommending auto-download of DTD files, and indeed disable

2023-03-21 Thread Robert Haas
On Wed, Feb 8, 2023 at 5:16 PM Tom Lane wrote: > Stop recommending auto-download of DTD files, and indeed disable it. According to this commit: The Homebrew-supplied programs require the following environment variable to be set: export XML_CATALOG_FILES=/usr/local/etc/xml/catalog

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-03-21 Thread Sergei Kornilov
Hello! The documentation still describes the function pg_stat_statements_reset like this > By default, this function can only be executed by superusers. But unfortunately, this part was lost somewhere. -- Don't want this to be available to non-superusers. REVOKE ALL ON FUNCTION

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 15:26:38 -0400, Tom Lane wrote: > Matthias van de Meent writes: > > ... with that patch we actually don't need the attcacheoff in the > > pg_atttribute struct: it only needs to be present in the derived > > "TupleAttrAlignData" structs which carry the > >

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Matthias van de Meent
On Tue, 21 Mar 2023 at 20:58, Andres Freund wrote: > > Hi, > > On 2023-03-21 20:20:40 +0100, Matthias van de Meent wrote: > > On Tue, 21 Mar 2023 at 19:55, Tom Lane wrote: > > > > > > Andres Freund writes: > > > > FWIW, I think we should consider getting rid of attcacheoff. I doubt > > > >

Re: Comment in preptlist.c

2023-03-21 Thread David Rowley
On Tue, 21 Mar 2023 at 22:41, Etsuro Fujita wrote: > I think that “planner/rewriter” should be parser/rewriter. Attached > is a patch for that. Pushed. David

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 20:20:40 +0100, Matthias van de Meent wrote: > On Tue, 21 Mar 2023 at 19:55, Tom Lane wrote: > > > > Andres Freund writes: > > > FWIW, I think we should consider getting rid of attcacheoff. I doubt it's > > > worth its weight these days, because deforming via slots starts at

Re: Adjust Memoize hit_ratio calculation

2023-03-21 Thread David Rowley
On Tue, 21 Mar 2023 at 09:41, David Rowley wrote: > Because that's being changed in v16, I think it might also be a good > idea to fix the hit_ratio calculation problem reported by David > Johnston in [1]. In the attached, I've adjusted David's calculation > slightly so that we divide by

Re: Add SHELL_EXIT_CODE to psql

2023-03-21 Thread Corey Huinker
> > > As you had it, any nonzero result would prevent backtick substitution. > I'm not really sure how much thought went into the existing behavior, > but I am pretty sure that it's not part of this patch's charter to > change that. > > regards, tom lane > The changes all

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Tom Lane
Matthias van de Meent writes: > ... with that patch we actually don't need the attcacheoff in the > pg_atttribute struct: it only needs to be present in the derived > "TupleAttrAlignData" structs which carry the > length/alignment/storage/byval info. Yeah, I was wondering about that too: keeping

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Matthias van de Meent
On Tue, 21 Mar 2023 at 19:55, Tom Lane wrote: > > Andres Freund writes: > > FWIW, I think we should consider getting rid of attcacheoff. I doubt it's > > worth its weight these days, because deforming via slots starts at the > > beginning anyway. The overhead of maintaining it is not

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Tom Lane
Andres Freund writes: > FWIW, I think we should consider getting rid of attcacheoff. I doubt it's > worth its weight these days, because deforming via slots starts at the > beginning anyway. The overhead of maintaining it is not insubstantial, and > it's just architecturally ugly to to update

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-21 Thread Justin Pryzby
On Thu, Mar 16, 2023 at 07:04:16PM +0400, Ilya Gladyshev wrote: > > 16 марта 2023 г., в 04:07, Justin Pryzby написал(а): > > > > On Tue, Mar 14, 2023 at 06:58:14PM +0400, Ilya Gladyshev wrote: > >>> The only change from the current patch is (3). (1) still calls > >>> count_leaf_partitions(),

Re: PostgreSQL 16 Release Management Team & Feature Freeze

2023-03-21 Thread Jonathan S. Katz
On 3/21/23 1:17 PM, Roberto Mello wrote: On Tue, Mar 21, 2023 at 9:35 AM Jonathan S. Katz wrote: You can track open items for the PostgreSQL 16 release here: https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items The wiki page references April 8th, 2022, btw. Fixed :) Thanks!

Re: Transparent column encryption

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 18:05:15 +0100, Peter Eisentraut wrote: > On 16.03.23 17:36, Andres Freund wrote: > > Maybe a daft question, but why do we need a separate type and typmod for > > encrypted columns? Why isn't the fact that the column is encrypted exactly > > one > > new field, and we use the

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 18:15:40 +0100, Peter Eisentraut wrote: > On 21.03.23 17:43, Andres Freund wrote: > > > The context of my message was to do the proposed change for PG16 to buy > > > back > > > a few bytes that are being added by another feature > > How much would you need to buy back to "reach

Re: PostgreSQL 16 Release Management Team & Feature Freeze

2023-03-21 Thread Roberto Mello
On Tue, Mar 21, 2023 at 9:35 AM Jonathan S. Katz wrote: > > You can track open items for the PostgreSQL 16 release here: > > https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items The wiki page references April 8th, 2022, btw. Roberto

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Peter Eisentraut
On 21.03.23 17:43, Andres Freund wrote: The context of my message was to do the proposed change for PG16 to buy back a few bytes that are being added by another feature How much would you need to buy back to "reach parity"? I don't think we can find enough to make the impact zero bytes. It's

Re: CREATE DATABASE ... STRATEGY WAL_LOG issues

2023-03-21 Thread Robert Haas
On Tue, Mar 21, 2023 at 12:34 PM Andres Freund wrote: > More generally, I still think we need logic to use unused buffers even when > strategies are in use Yep. > (my current theory is that we wouldn't increase the > usagecount when strategies use unused buffers, so they can be replaced more >

Re: Add SHELL_EXIT_CODE to psql

2023-03-21 Thread Tom Lane
Corey Huinker writes: > On Mon, Mar 20, 2023 at 1:01 PM Tom Lane wrote: >> * Why do you have wait_result_to_exit_code defaulting to return 0 >> if it doesn't recognize the code as either WIFEXITED or WIFSIGNALED? >> That seems pretty misleading; again -1 would be a better idea. > That makes

Re: Transparent column encryption

2023-03-21 Thread Peter Eisentraut
On 16.03.23 17:36, Andres Freund wrote: Maybe a daft question, but why do we need a separate type and typmod for encrypted columns? Why isn't the fact that the column is encrypted exactly one new field, and we use the existing type/typmod fields? The way this is implemented is that for an

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 17:36:48 +0100, Peter Eisentraut wrote: > On 21.03.23 00:51, Tom Lane wrote: > > Andres Freund writes: > > > On 2023-03-20 10:37:36 -0400, Tom Lane wrote: > > > > I agree that attinhcount could be narrowed, but I have some concern > > > > about attstattarget. IIRC, the limit

Re: Save a few bytes in pg_attribute

2023-03-21 Thread Peter Eisentraut
On 21.03.23 00:51, Tom Lane wrote: Andres Freund writes: On 2023-03-20 10:37:36 -0400, Tom Lane wrote: I agree that attinhcount could be narrowed, but I have some concern about attstattarget. IIRC, the limit on attstattarget was once 1000 and then we raised it to 1. Is it inconceivable

Re: CREATE DATABASE ... STRATEGY WAL_LOG issues

2023-03-21 Thread Andres Freund
Hi, On 2023-03-21 11:33:59 -0400, Robert Haas wrote: > On Tue, Mar 21, 2023 at 3:01 AM Andres Freund wrote: > > Easy enough to fix and shows clear improvement. One thing I wonder is if > > it's > > worth moving the strategies up one level? Probaly not, but ... > > Hmm, so share a strategy

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Dave Cramer
On Tue, 21 Mar 2023 at 11:52, Jeff Davis wrote: > On Mon, 2023-03-20 at 20:18 -0400, Dave Cramer wrote: > > For JAVA pools it's probably OK, but for pools like pgbouncer we have > > no control of who is going to get the connection next. > > Can pgbouncer use different pools for different

Re: Schema variables - new implementation for Postgres 15

2023-03-21 Thread Peter Eisentraut
On 17.03.23 21:50, Pavel Stehule wrote: rebase + fix-update pg_dump tests I have a few comments on the code: 0001 ExecGrant_Variable() could probably use ExecGrant_common(). The additions to syscache.c should be formatted to the new style. in pg_variable.h: - create_lsn ought to have a

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Jeff Davis
On Mon, 2023-03-20 at 20:18 -0400, Dave Cramer wrote: > For JAVA pools it's probably OK, but for pools like pgbouncer we have > no control of who is going to get the connection next. Can pgbouncer use different pools for different settings of format_binary? Regards, Jeff Davis

PostgreSQL 16 Release Management Team & Feature Freeze

2023-03-21 Thread Jonathan S. Katz
Hi, We are pleased to announce the Release Management Team (RMT) (cc'd) for the PostgreSQL 16 release: - Alvaro Herrera - Amit Kapila - Jonathan Katz You can find information about the responsibilities of the RMT here: https://wiki.postgresql.org/wiki/Release_Management_Team

Re: CREATE DATABASE ... STRATEGY WAL_LOG issues

2023-03-21 Thread Robert Haas
On Tue, Mar 21, 2023 at 3:01 AM Andres Freund wrote: > While hacking on my relation extension patch I found two issues with WAL_LOG: > > 1) RelationCopyStorageUsingBuffer() doesn't free the used strategies. This >means we'll use #relations * ~10k memory Woops. > 2)

Re: proposal: possibility to read dumped table's name from file

2023-03-21 Thread Pavel Stehule
út 21. 3. 2023 v 16:32 odesílatel Justin Pryzby napsal: > On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote: > > ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby > napsal: > > > > > On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote: > > > > rebase + enhancing about

Re: Make EXPLAIN generate a generic plan for a parameterized query

2023-03-21 Thread Christoph Berg
Hi, I reviewed the patch and find the idea of allowing $placeholders with EXPLAIN very useful, it will solve relevant real-world use-cases. (Queries from pg_stat_statements, found in the log, or in application code.) I have some comments: > This allows EXPLAIN to generate generic plans for

Re: proposal: possibility to read dumped table's name from file

2023-03-21 Thread Justin Pryzby
On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote: > ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby napsal: > > > On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote: > > > rebase + enhancing about related option from a563c24 > > > > Thanks. > > > > It looks like this

Making the initial and maximum DSA segment sizes configurable

2023-03-21 Thread Masahiko Sawada
Hi all, I started this new thread from another thread[1] where we're discussing a new storage for TIDs, TidStore, since we found a difficulty about the memory usage limit for TidStores on DSA. TidStore is a new data structure to efficiently store TIDs, backed by a radix tree. In the patch series

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Merlin Moncure
On Tue, Mar 21, 2023 at 8:22 AM Dave Cramer wrote: > > On Tue, 21 Mar 2023 at 07:35, Merlin Moncure wrote: > >> >> >> On Mon, Mar 20, 2023 at 7:11 PM Dave Cramer wrote: >> >>> >>> >>> >>> On Mon, 20 Mar 2023 at 19:10, Merlin Moncure wrote: >>> On Mon, Mar 13, 2023 at 3:33 PM

Re: Ability to reference other extensions by schema in extension scripts

2023-03-21 Thread Tom Lane
"Regina Obe" writes: >> making the no_relocate values visible somehow, I'm not convinced that >> pg_available_extension_versions should be the place to do it. ISTM what's >> relevant is the no_relocate values of *installed* extensions, not those of >> potentially-installable extensions. If we

RE: Ability to reference other extensions by schema in extension scripts

2023-03-21 Thread Regina Obe
> Pushed with some mostly-cosmetic adjustments (in particular I tried to make > the docs and tests neater). > > I did not commit the changes in get_available_versions_for_extension > to add no_relocate as an output column. Those were dead code because you > hadn't done anything to connect them

Re: Comment in preptlist.c

2023-03-21 Thread Tom Lane
Etsuro Fujita writes: > While working on something else, I noticed $SUBJECT added by commit 86dc90056: > * For UPDATE and DELETE queries, the targetlist must also contain "junk" > * tlist entries needed to allow the executor to identify the rows to be > * updated or deleted; for example, the

Re: doc: add missing "id" attributes to extension packaging page

2023-03-21 Thread Brar Piening
On 20.03.2023 at 19:47, Gregory Stark (as CFM) wrote: Looks like a lot of good work was happening on this patch right up until mid-February. Is there a lot of work left? Do you think you'll have a chance to wrap it up this commitfest for this release? Thanks for the ping. I had another look

Re: real/float example for testlibpq3

2023-03-21 Thread Mark Wong
On Mon, Mar 20, 2023 at 01:37:57PM -0400, Gregory Stark (as CFM) wrote: > On Mon, 23 Jan 2023 at 11:54, Mark Wong wrote: > fficient way to communicate useful information. > > > > Yeah, I will try to cover all the data types we ship. :) I'll keep at > > it and drop the code examples. > > I

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Dave Cramer
On Tue, 21 Mar 2023 at 07:35, Merlin Moncure wrote: > > > On Mon, Mar 20, 2023 at 7:11 PM Dave Cramer wrote: > >> >> >> >> On Mon, 20 Mar 2023 at 19:10, Merlin Moncure wrote: >> >>> >>> >>> On Mon, Mar 13, 2023 at 3:33 PM Dave Cramer >>> wrote: >>> OIDs are a pain to deal with IMO.

Re: [BUG] pg_stat_statements and extended query protocol

2023-03-21 Thread Imseih (AWS), Sami
> This indeed feels a bit more natural seen from here, after looking at > the code paths using an Instrumentation in the executor and explain, > for example. At least, this stresses me much less than adding 16 > bytes to EState for something restricted to the extended protocol when > it comes to

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-03-21 Thread Alvaro Herrera
On 2023-Mar-21, Dean Rasheed wrote: > Looking at it with fresh eyes though, I realise that I could have just written > > action->qual = make_and_qual((Node *) ntest, action->qual); > > which is equivalent, but more concise. Nice. I have no further observations about this patch. --

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-03-21 Thread Dean Rasheed
On Tue, 21 Mar 2023 at 10:28, Alvaro Herrera wrote: > > > + /* Combine it with the action's WHEN condition */ > > + if (action->qual == NULL) > > + action->qual = (Node *) ntest; > > + else > > +

Re: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-21 Thread Önder Kalacı
Hi Shi Yu, > > > 1. > $node_publisher->safe_psql( > 'postgres', qq( > ALTER TABLE dropped_cols DROP COLUMN b_drop; > + ALTER TABLE generated_cols DROP COLUMN b_gen; > )); > $node_subscriber->safe_psql( > 'postgres', qq( > ALTER

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-03-21 Thread Aleksander Alekseev
Hi John, > Thanks for picking up this badly-needed topic again! Many thanks for the review! > 0001: > > +In this condition the system can still execute read-only transactions. > +The active transactions will continue to execute and will be able to > +commit. > > This is ambiguous.

Re: Request for comment on setting binary format output per session

2023-03-21 Thread Merlin Moncure
On Mon, Mar 20, 2023 at 7:11 PM Dave Cramer wrote: > > > > On Mon, 20 Mar 2023 at 19:10, Merlin Moncure wrote: > >> >> >> On Mon, Mar 13, 2023 at 3:33 PM Dave Cramer wrote: >> >>> >>> OIDs are a pain to deal with IMO. They will not survive a dump style >> restore, and are hard to keep

Re: Initial Schema Sync for Logical Replication

2023-03-21 Thread Amit Kapila
On Tue, Mar 21, 2023 at 7:32 AM Euler Taveira wrote: > > On Mon, Mar 20, 2023, at 10:10 PM, Kumar, Sachin wrote: > > > From: Alvaro Herrera > > Subject: RE: [EXTERNAL]Initial Schema Sync for Logical Replication > > On 2023-Mar-15, Kumar, Sachin wrote: > > > > > 1. In CreateSubscription() when

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-03-21 Thread Alvaro Herrera
On 2023-Mar-19, Dean Rasheed wrote: > diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y > new file mode 100644 > index efe88cc..e1ebc8d > --- a/src/backend/parser/gram.y > +++ b/src/backend/parser/gram.y > +merge_when_tgt_matched: > + WHEN MATCHED

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-21 Thread shiy.f...@fujitsu.com
On Tue, Mar 21, 2023 4:51 PM Önder Kalacı wrote: > > Hi Amit, Shi Yu > > Now attaching the similar patches for generated columns. > Thanks for your patches. Here are some comments. 1. $node_publisher->safe_psql( 'postgres', qq( ALTER TABLE dropped_cols DROP COLUMN

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-21 Thread Ants Aasma
On Mon, 20 Mar 2023 at 00:59, Melanie Plageman wrote: > > On Wed, Mar 15, 2023 at 6:46 AM Ants Aasma wrote: > > > > On Wed, 15 Mar 2023 at 02:29, Melanie Plageman > > wrote: > > > As for routine vacuuming and the other buffer access strategies, I think > > > there is an argument for

Re: Comment in preptlist.c

2023-03-21 Thread Richard Guo
On Tue, Mar 21, 2023 at 5:41 PM Etsuro Fujita wrote: > While working on something else, I noticed $SUBJECT added by commit > 86dc90056: > > * For UPDATE and DELETE queries, the targetlist must also contain "junk" > * tlist entries needed to allow the executor to identify the rows to be > *

Re: Commitfest 2023-03 starting tomorrow!

2023-03-21 Thread Alvaro Herrera
On 2023-Mar-20, Thomas Munro wrote: > I realised that part of Alvaro's complaint was probably caused by > cfbot's refusal to show any useful information just because it > couldn't apply a patch the last time it tried. A small improvement > today: now it shows a ♲ symbol (with hover text "Rebase

Re: [PATCH] Add initial xid/mxid/mxoff to initdb

2023-03-21 Thread Maxim Orlov
On Mon, 20 Mar 2023 at 22:31, Gregory Stark (as CFM) wrote: > > So is that other thread tracked in a different commitfest entry and > this one completely redundant? I'll mark it Rejected then? > Yep, it appears so. -- Best regards, Maxim Orlov.

Comment in preptlist.c

2023-03-21 Thread Etsuro Fujita
While working on something else, I noticed $SUBJECT added by commit 86dc90056: * For UPDATE and DELETE queries, the targetlist must also contain "junk" * tlist entries needed to allow the executor to identify the rows to be * updated or deleted; for example, the ctid of a heap row. (The

Re: ICU locale validation / canonicalization

2023-03-21 Thread Peter Eisentraut
On 17.03.23 18:55, Jeff Davis wrote: On Wed, 2023-03-15 at 15:18 -0700, Jeff Davis wrote: I left out the validation patch for now, and I'm evaluating a different approach that will attempt to match to the locales retrieved with uloc_countAvailable()/uloc_getAvailable(). I like this approach,

Re: Allow logical replication to copy tables in binary format

2023-03-21 Thread Melih Mutlu
Hi, Peter Smith , 21 Mar 2023 Sal, 04:33 tarihinde şunu yazdı: > Here are my review comments for v18-0001 > > == > doc/src/sgml/logical-replication.sgml > > 1. > + target table. However, logical replication in binary format is more > + restrictive. See the binary option of > + CREATE >

Re: Allow logical replication to copy tables in binary format

2023-03-21 Thread Amit Kapila
On Tue, Mar 21, 2023 at 2:16 PM Melih Mutlu wrote: > > Amit Kapila , 21 Mar 2023 Sal, 09:03 tarihinde şunu > yazdı: >> >> On Tue, Mar 21, 2023 at 7:03 AM Peter Smith wrote: >> > >> > >> > == >> > src/test/subscription/t/014_binary.pl >> > >> > 4. >> > #

Re: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-21 Thread Önder Kalacı
Hi Amit, Shi Yu Now attaching the similar patches for generated columns. Thanks, Onder KALACI Amit Kapila , 21 Mar 2023 Sal, 09:07 tarihinde şunu yazdı: > On Mon, Mar 20, 2023 at 6:28 PM Amit Kapila > wrote: > > > > On Mon, Mar 20, 2023 at 2:58 PM Önder Kalacı > wrote: > > > > > > > > > I

Re: Allow logical replication to copy tables in binary format

2023-03-21 Thread Melih Mutlu
Amit Kapila , 21 Mar 2023 Sal, 09:03 tarihinde şunu yazdı: > On Tue, Mar 21, 2023 at 7:03 AM Peter Smith wrote: > > > > > > == > > src/test/subscription/t/014_binary.pl > > > > 4. > > # - > > # Test mismatched column types with/without

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 21:18 PM Kuroda, Hayato/黒田 隼人 wrote: > Dear Wang, > > I have tested about multilevel partitions, and it worked well. > Followings are my comments for v18-0001. Thanks for your comments and testing. > 01. pg_get_publication_tables > > ``` > + ListCell

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 15:32 PM Peter Smith wrote: > Here are some review comments for v17-0001. Thanks for your comments. > == > src/backend/catalog/pg_publication.c > > 1. filter_partitions > > -static List * > -filter_partitions(List *relids) > +static void > +filter_partitions(List

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Sat, Mar 18, 2023 at 7:37 AM Jacob Champion wrote: > On Thu, Mar 16, 2023 at 11:28 PM wangw.f...@fujitsu.com > wrote: > > Attach the new patch set. Thanks for your comments and testing. > For example, the corner case mentioned in 0003, with multiple > publications having conflicting

RE: Data is copied twice when specifying both child and parent table in publication

2023-03-21 Thread wangw.f...@fujitsu.com
On Mon, Mar 20, 2023 at 18:15 PM Amit Kapila wrote: > Thanks for your comments. > On Mon, Mar 20, 2023 at 1:02 PM Peter Smith > wrote: > > > > > > == > > src/include/catalog/pg_proc.dat > > > > 4. > > +{ oid => '6119', > > + descr => 'get information of the tables in the given publication

Re: Timeline ID hexadecimal format

2023-03-21 Thread Sébastien Lardière
On 21/03/2023 08:15, Peter Eisentraut wrote: On 20.03.23 10:40, Sébastien Lardière wrote: About option_parse_int(), actually, strtoint() is used, do we need a option_parse_ul() fonction ? For the option parsing, I propose the attached patch.  This follows the structure of option_parse_int(),

Hash table scans outside transactions

2023-03-21 Thread Ashutosh Bapat
Hi, Hash table scans (seq_scan_table/level) are cleaned up at the end of a transaction in AtEOXact_HashTables(). If a hash seq scan continues beyond transaction end it will meet "ERROR: no hash_seq_search scan for hash table" in deregister_seq_scan(). That seems like a limiting the hash table

Re: Timeline ID hexadecimal format

2023-03-21 Thread Peter Eisentraut
On 20.03.23 10:40, Sébastien Lardière wrote: About option_parse_int(), actually, strtoint() is used, do we need a option_parse_ul() fonction ? For the option parsing, I propose the attached patch.  This follows the structure of option_parse_int(), so in the future it could be extracted and

CREATE DATABASE ... STRATEGY WAL_LOG issues

2023-03-21 Thread Andres Freund
Hi, While hacking on my relation extension patch I found two issues with WAL_LOG: 1) RelationCopyStorageUsingBuffer() doesn't free the used strategies. This means we'll use #relations * ~10k memory 2) RelationCopyStorageUsingBuffer() gets the buffer for the target relation with

  1   2   >