Re: SQL/JSON revisited

2022-12-27 Thread Amit Langote
On Wed, Dec 28, 2022 at 4:28 PM Amit Langote wrote: > > Hi, > > Rebased the SQL/JSON patches over the latest HEAD. I've decided to > keep the same division of code into individual commits as that > mentioned in the revert commit 2f2b18bd3f, squashing fixup commits in > that list into the

Re: Error-safe user functions

2022-12-27 Thread Amul Sul
On Tue, Dec 27, 2022 at 11:17 PM Tom Lane wrote: > > Andrew Dunstan writes: > > Here's a patch that covers the ltree and intarray contrib modules. > > I would probably have done this a little differently --- I think > the added "res" parameters aren't really necessary for most of > these. But

CFM for 2023-01

2022-12-27 Thread vignesh C
Hi All, If no one has volunteered for the upcoming (January 2023) commitfest. I would like to volunteer for it. Regards, Vignesh

Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

2022-12-27 Thread Amit Kapila
On Tue, Dec 27, 2022 at 5:49 PM Michail Nikolaev wrote: > > > Probably a small part of WAL was somehow skipped by logical worker in > all that mess. > None of these entries are from the point mentioned by you [1] yesterday where you didn't find the corresponding data in the subscriber. How did

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-12-27 Thread Tom Lane
Justin Pryzby writes: > This fails when run more than once: > time meson test --setup running --print > test_pg_db_role_setting-running/regress Ah. > It didn't fail for you because it says: > ./src/test/modules/test_pg_db_role_setting/Makefile > +# disable installcheck for now >

Re: POC, WIP: OR-clause support for indexes

2022-12-27 Thread Andrey Lepikhov
On 12/26/15 23:04, Teodor Sigaev wrote: I'd like to present OR-clause support for indexes. Although OR-clauses could be supported by bitmapOR index scan it isn't very effective and such scan lost any order existing in index. We (with Alexander Korotkov) presented results on Vienna's conference

Re: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Amit Kapila
On Wed, Dec 28, 2022 at 8:19 AM Hayato Kuroda (Fujitsu) wrote: > > > In logical replication, it can happen today as well without > > time-delayed replication. Basically, say apply worker is waiting to > > acquire some lock that is already acquired by some backend then it > > will have the same

RE: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > In logical replication, it can happen today as well without > time-delayed replication. Basically, say apply worker is waiting to > acquire some lock that is already acquired by some backend then it > will have the same behavior. I have not verified this, so you may want > to check

RE: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > Firstly I considered 2, but I thought 1 seemed to be better. > > PSA the updated patch. > > > > I think even for logical replication we should check whether there is > any pending WAL (via pq_is_send_pending()) to be sent. Otherwise, what > is the point to send the done message?

Re: Force streaming every change in logical decoding

2022-12-27 Thread Amit Kapila
On Wed, Dec 28, 2022 at 1:42 AM Andres Freund wrote: > > On 2022-12-26 14:04:28 +0530, Amit Kapila wrote: > > Pushed. > > I did not follow this thread but saw the commit. Could you explain why a GUC > is the right mechanism here? The commit message didn't explain why a GUC was > chosen. > > To me

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-12-27 Thread Justin Pryzby
On Tue, Dec 27, 2022 at 01:58:14AM -0500, Tom Lane wrote: > Justin Pryzby writes: > > FYI: this causes meson test running ("installcheck") fail when run > > twice. I guess that's expected to work, per: > > We do indeed expect that to work ... but I don't see any problem > with repeat "make

Re: recovery modules

2022-12-27 Thread Andres Freund
Hi, On 2022-12-27 15:04:28 -0800, Nathan Bossart wrote: > I'm sorry, I'm still lost here. Wouldn't restoration via library tend to > improve latency? Is your point that clusters may end up depending on this > improvement so much that a shell command would no longer be able to keep > up? Yes.

Re: recovery modules

2022-12-27 Thread Michael Paquier
On Tue, Dec 27, 2022 at 02:11:11PM -0800, Andres Freund wrote: > On 2022-12-27 11:24:49 -0800, Nathan Bossart wrote: >> * Unlike archive modules, recovery libraries cannot be changed at runtime. >> There isn't a safe way to unload a library, and archive libraries work >> around this restriction by

Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2022-12-27 Thread Michael Paquier
On Tue, Dec 27, 2022 at 03:54:46PM +, Jelte Fennema wrote: > This change makes it much easier to have a certain database > administrator peer or cert authentication, that allows connecting as > any user. Without this change you would need to add a line to > pg_ident.conf for every user that is

Re: Removing redundant grouping columns

2022-12-27 Thread Tom Lane
I wrote: > This patch is aimed at being smarter about cases where we have > redundant GROUP BY entries, for example > SELECT ... WHERE a.x = b.y GROUP BY a.x, b.y; The cfbot didn't like this, because of a variable that wasn't used in non-assert builds. Fixed in v2.

Re: build gcc warning

2022-12-27 Thread Tom Lane
Andres Freund writes: > On 2022-12-27 01:55:06 -0500, Tom Lane wrote: >> A couple of buildfarm animals are warning about that too ... but >> only a couple. > I'm a bit confused by gcc getting confused here - the condition for > sub_rteperminfos getting initialized and used are the same. Most of

Re: recovery modules

2022-12-27 Thread Nathan Bossart
On Tue, Dec 27, 2022 at 02:45:30PM -0800, Andres Freund wrote: > On 2022-12-27 14:37:11 -0800, Nathan Bossart wrote: >> On Tue, Dec 27, 2022 at 02:11:11PM -0800, Andres Freund wrote: >> > On 2022-12-27 11:24:49 -0800, Nathan Bossart wrote: >> >> * pg_rewind uses restore_command, but there isn't a

Re: recovery modules

2022-12-27 Thread Andres Freund
Hi, On 2022-12-27 14:37:11 -0800, Nathan Bossart wrote: > On Tue, Dec 27, 2022 at 02:11:11PM -0800, Andres Freund wrote: > > On 2022-12-27 11:24:49 -0800, Nathan Bossart wrote: > >> I've attached a patch set that adds the restore_library, > >> archive_cleanup_library, and recovery_end_library

Re: Make EXPLAIN generate a generic plan for a parameterized query

2022-12-27 Thread Michel Pelletier
On Wed, Dec 7, 2022 at 3:23 AM Laurenz Albe wrote: > On Tue, 2022-12-06 at 10:17 -0800, Andres Freund wrote: > > On 2022-10-29 10:35:26 +0200, Laurenz Albe wrote: > > > > > Here is a patch that > > > > > implements it with an EXPLAIN option named GENERIC_PLAN. > > > > This fails to build the

Re: recovery modules

2022-12-27 Thread Nathan Bossart
On Tue, Dec 27, 2022 at 02:11:11PM -0800, Andres Freund wrote: > On 2022-12-27 11:24:49 -0800, Nathan Bossart wrote: >> I've attached a patch set that adds the restore_library, >> archive_cleanup_library, and recovery_end_library parameters to allow >> archive recovery via loadable modules. This

Removing redundant grouping columns

2022-12-27 Thread Tom Lane
This patch is aimed at being smarter about cases where we have redundant GROUP BY entries, for example SELECT ... WHERE a.x = b.y GROUP BY a.x, b.y; It's clearly not necessary to perform grouping using both columns. Grouping by either one alone would produce the same results, assuming compatible

Re: build gcc warning

2022-12-27 Thread Andres Freund
Hi, On 2022-12-27 01:55:06 -0500, Tom Lane wrote: > Pavel Stehule writes: > > I got new warning > > analyze.c: In function ‘transformStmt’: > > analyze.c:550:21: warning: ‘sub_rteperminfos’ may be used uninitialized > > [-Wmaybe-uninitialized] > > A couple of buildfarm animals are warning about

Re: recovery modules

2022-12-27 Thread Andres Freund
Hi, On 2022-12-27 11:24:49 -0800, Nathan Bossart wrote: > I've attached a patch set that adds the restore_library, > archive_cleanup_library, and recovery_end_library parameters to allow > archive recovery via loadable modules. This is a follow-up to the > archive_library parameter added in v15

Re: Patch: Global Unique Index

2022-12-27 Thread David Zhang
On 2022-12-19 7:51 a.m., Nikita Malakhov wrote: Sorry to bother - but is this patch used in IvorySQL? Here: https://www.ivorysql.org/docs/Global%20Unique%20Index/create_global_unique_index According to syntax it definitely looks like this patch. The global unique index is one of the features

Re: Force streaming every change in logical decoding

2022-12-27 Thread Andres Freund
Hi, On 2022-12-26 14:04:28 +0530, Amit Kapila wrote: > Pushed. I did not follow this thread but saw the commit. Could you explain why a GUC is the right mechanism here? The commit message didn't explain why a GUC was chosen. To me an option like this should be passed in when decoding rather

Re: Error-safe user functions

2022-12-27 Thread Tom Lane
Andrew Dunstan writes: > On Dec 27, 2022, at 12:47 PM, Tom Lane wrote: >> Andrew Dunstan writes: >>> I think that would leave just hstore to be done. >> Yeah, that matches my scoreboard. Are you going to look at >> hstore, or do you want me to? > Go for it. Done.

recovery modules

2022-12-27 Thread Nathan Bossart
I've attached a patch set that adds the restore_library, archive_cleanup_library, and recovery_end_library parameters to allow archive recovery via loadable modules. This is a follow-up to the archive_library parameter added in v15 [0] [1]. The motivation behind this change is similar to that of

Re: Error-safe user functions

2022-12-27 Thread Andrew Dunstan
> On Dec 27, 2022, at 12:47 PM, Tom Lane wrote: > > Andrew Dunstan writes: >> Here's a patch that covers the ltree and intarray contrib modules. > > I would probably have done this a little differently --- I think > the added "res" parameters aren't really necessary for most of > these.

Re: Error-safe user functions

2022-12-27 Thread Tom Lane
Andrew Dunstan writes: > Here's a patch that covers the ltree and intarray contrib modules. I would probably have done this a little differently --- I think the added "res" parameters aren't really necessary for most of these. But it's not worth arguing over. > I think that would leave just

Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2022-12-27 Thread Isaac Morland
On Tue, 27 Dec 2022 at 10:54, Jelte Fennema wrote: This change makes it much easier to have a certain database > administrator peer or cert authentication, that allows connecting as > any user. Without this change you would need to add a line to > pg_ident.conf for every user that is in the

[PATCH] Support using "all" for the db user in pg_ident.conf

2022-12-27 Thread Jelte Fennema
While pg_hba.conf has supported the "all" keyword since a very long time, pg_ident.conf doesn't have this same functionality. This changes permission checking in pg_ident.conf to handle "all" differently from any other value in the database-username column. If "all" is specified and the

False positive warning in verify_heapam.c with GCC 03

2022-12-27 Thread Maxim Orlov
Hi! While playing with some unrelated to the topic stuff, I've noticed a strange warning from verify_heapam.c:730:25: warning: ‘xmax_status’ may be used uninitialized in this function. This happens only when get_xid_status is inlined, and only in GCC with O3. I use a GCC version 11.3.0. For the

Re: Making Vars outer-join aware

2022-12-27 Thread Tom Lane
Richard Guo writes: > For 0012, I'm still trying to understand JoinDomain. AFAIU all EC > members of the same EC should have the same JoinDomain, because for > constants we match EC members only within the same JoinDomain, and for > Vars if they come from different join domains they will have

Re: Underscores in numeric literals

2022-12-27 Thread Justin Pryzby
On Tue, Dec 27, 2022 at 09:55:32AM -0500, Tom Lane wrote: > Peter Eisentraut writes: > > Here is a patch to add support for underscores in numeric literals, for > > visual grouping, like > > > 1_500_000_000 > > 0b10001000_ > > 0o_1_755 > > 0x_ > >

Re: Underscores in numeric literals

2022-12-27 Thread Tom Lane
Peter Eisentraut writes: > Here is a patch to add support for underscores in numeric literals, for > visual grouping, like > 1_500_000_000 > 0b10001000_ > 0o_1_755 > 0x_ > 1.618_034 > per SQL:202x draft. > This adds support in the lexer as well as in

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-27 Thread Anton A. Melnikov
On 27.12.2022 16:50, Michael Paquier wrote: If there are no other considerations could you close the corresponding record on the January CF, please? Indeed, now marked as committed. - Thanks a lot! Merry Christmas! -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-27 Thread Michael Paquier
On Tue, Dec 27, 2022 at 03:26:10PM +0300, Anton A. Melnikov wrote: > I would like to try realize this, better in a separate thread. I don't think that this should be added into the tree, but if you have per-version filtering rules, of course feel free to publish that to the lists. I am sure that

Re: Error-safe user functions

2022-12-27 Thread Andrew Dunstan
On 2022-12-26 Mo 18:00, Tom Lane wrote: > I wrote: >> (Perhaps we should go further than this, and convert all these >> functions to just be DirectInputFunctionCallSafe wrappers >> around the corresponding input functions? That would save >> some duplicative code, but I've not done it here.) >

Re: Error-safe user functions

2022-12-27 Thread Andrew Dunstan
On 2022-12-26 Mo 14:12, Andrew Dunstan wrote: > On 2022-12-26 Mo 12:47, Tom Lane wrote: >> Here's a proposed patch for making tsvectorin and tsqueryin >> report errors softly. We have to take the changes down a >> couple of levels of subroutines, but it's not hugely difficult. > > Great! > > >>

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-27 Thread Anton A. Melnikov
Hello! On 27.12.2022 08:44, Michael Paquier wrote: It is worth noting that perlcritic was complaining here, as eval is getting used with a string. I have spent a few days looking at that, and I really want a maximum of flexibility in the rules that can be applied so I have put a "no critic"

Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

2022-12-27 Thread Michail Nikolaev
Hello, Amit! > IUC, this is the time when only table B's initial sync was > in-progress. Table A's initial sync was finished by that time and for > Table C, it is yet not started. Yes, it is correct. C was started too, but unsuccessfully (restarted after, see below). > During the time of > the

Re: Passing relation metadata to Exec routine

2022-12-27 Thread Nikita Malakhov
Hi Tom! Thank you for your feedback. I agree that for complex columns created with joins, grouping, etc considering properties of the base table does not make sense at all. But for CREATE TABLE LIKE and simple columns that are inherited from some existing relations - it does, if we consider some

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-27 Thread Bharath Rupireddy
On Mon, Dec 26, 2022 at 4:18 PM Bharath Rupireddy wrote: > > On Mon, Dec 26, 2022 at 12:59 PM Michael Paquier wrote: > > +1. I think this feature will also be useful in pg_walinspect. Just for the record - here's the pg_walinspect function to extract FPIs from WAL records -

Add a new pg_walinspect function to extract FPIs from WAL records

2022-12-27 Thread Bharath Rupireddy
Hi, Here's a patch that implements the idea of extracting full page images from WAL records [1] [2] with a function in pg_walinspect. This new function accepts start and end lsn and returns full page image info such as WAL record lsn, tablespace oid, database oid, relfile number, block number,

Re: Perform streaming logical transactions by background workers and parallel apply

2022-12-27 Thread Amit Kapila
On Tue, Dec 27, 2022 at 10:24 AM wangw.f...@fujitsu.com wrote: > > Attach the new version patch which addressed all above comments and part of > comments from [1] except one comment that are being discussed. > 1. +# Test that the deadlock is detected among leader and parallel apply workers. +

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

2022-12-27 Thread Melih Mutlu
Hi, Having FORCE_NULL(*) and FORCE_NOT_NULL(*) sounds good, since postgres already has FORCE_QUOTE(*). I just quickly tried out your patch. It worked for me as expected. One little suggestion: + if (cstate->opts.force_notnull_all) > + { > + int i; > + for(i = 0; i < num_phys_attrs;

RE: Time delayed LR (WAS Re: logical replication restrictions)

2022-12-27 Thread Hayato Kuroda (Fujitsu)
Dear Dilip, Thanks for reviewing our patch! PSA new version patch set. Again, 0001 is not made by us, brought from [1]. > I have done some review for the patch and I have a few comments. > > 1. > A. > + wal_sender_timeout on the publisher. Otherwise, the > + walsender repeatedly

Re: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Amit Kapila
On Tue, Dec 27, 2022 at 2:50 PM Amit Kapila wrote: > > On Tue, Dec 27, 2022 at 1:44 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Thanks for checking my proposal! > > > > > - * Note that if we determine that there's still more data to send, this > > > - * function will return control to the

Re: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Amit Kapila
On Tue, Dec 27, 2022 at 1:44 PM Hayato Kuroda (Fujitsu) wrote: > > Thanks for checking my proposal! > > > - * Note that if we determine that there's still more data to send, this > > - * function will return control to the caller. > > + * Note that if we determine that there's still more data to

Underscores in numeric literals

2022-12-27 Thread Peter Eisentraut
Here is a patch to add support for underscores in numeric literals, for visual grouping, like 1_500_000_000 0b10001000_ 0o_1_755 0x_ 1.618_034 per SQL:202x draft. This adds support in the lexer as well as in the integer type input functions. TODO:

Refactor recordExtObjInitPriv()

2022-12-27 Thread Peter Eisentraut
Another aclchk.c refactoring patch, similar to [0] and [1]. Refactor recordExtObjInitPriv(): Instead of half a dozen of mostly-duplicate conditional branches, write one common one that can handle most catalogs. We already have all the information we need, such as which system catalog

Re: Making Vars outer-join aware

2022-12-27 Thread Richard Guo
On Sat, Dec 24, 2022 at 2:20 AM Tom Lane wrote: > I shoved some preliminary refactoring into the 0001 patch, > notably splitting deconstruct_jointree into two passes. > 0002-0009 cover the same ground as they did before, though > with some differences in detail. 0010-0012 are new work > mostly

RE: Exit walsender before confirming remote flush in logical replication

2022-12-27 Thread Hayato Kuroda (Fujitsu)
Dear Dilip, Thanks for checking my proposal! > - * Note that if we determine that there's still more data to send, this > - * function will return control to the caller. > + * Note that if we determine that there's still more data to send or we are > in > + * the physical replication more, this