More pgindent tweaks

2023-01-25 Thread Andrew Dunstan
After I committed 1249371632 I thought that I should really go ahead and do what I suggested and allow multiple exclude pattern files for pgindent. One obvious case is to exclude an in tree meson build directory. I also sometimes have other in tree objects I'd like to be able exclude. The

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-25 Thread Takamichi Osumi (Fujitsu)
On Wednesday, January 25, 2023 3:27 PM Kyotaro Horiguchi wrote: > At Tue, 24 Jan 2023 12:19:04 +, "Takamichi Osumi (Fujitsu)" > wrote in > > Attached the patch v20 that has incorporated all comments so far. > > Thanks! I looked thourgh the documentation part. Thank you for your review !

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Matthias van de Meent
On Tue, 24 Jan 2023 at 23:50, Peter Geoghegan wrote: > > On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote: > > 0001 (the freezing strategies patch) is now committable IMV. Or at > > least will be once I polish the docs a bit more. I plan on committing > > 0001 some time next week, barring

Re: Fix to enum hashing for dump and restore

2023-01-25 Thread Andrew
Those are excellent points. We will investigate adjusting pg_dump behavior, as this is primarily a dump+restore issue. Thank you! -Andrew J Repp (VMware) On Tue, Jan 24, 2023, at 9:56 PM, Tom Lane wrote: > Andrew writes: > > I have discovered a bug in one usage of enums. If a table with hash

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Pavel Stehule
Hi st 25. 1. 2023 v 15:18 odesílatel songjinzhou <2903807...@qq.com> napsal: > Hello, this usage scenario is from Oracle's PL/SQL language (I have been > doing the function development of PL/SQL language for some time). I think > this patch is very practical and will expand our for loop

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-01-25 Thread Andrei Zubkov
Hi, I've updated this patch for the current master. Also I have some additional explanations.. On Wed, 2023-01-18 at 17:29 +0100, Tomas Vondra wrote: > 1) I'm not sure why the patch is adding tests of permissions on the > pg_stat_statements_reset function? I've fixed that > > 2) If we want

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jim/Jacob, > > I do not think it is worth it to change the current behavior of > PostgreSQL > > in that sense. > > Well, I am not suggesting to change the current behavior of PostgreSQL in > that matter. Quite the contrary, I find this feature very convenient, > specially when you need to

Syncrep and improving latency due to WAL throttling

2023-01-25 Thread Jakub Wartak
Hi, attached is proposal idea by Tomas (in CC) for protecting and prioritizing OLTP latency on syncrep over other heavy WAL hitting sessions. This is the result of internal testing and research related to the syncrep behavior with Tomas, Alvaro and me. The main objective of this

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread songjinzhou
Hello, this usage scenario is from Oracle's PL/SQL language (I have been doing the function development of PL/SQL language for some time). I think this patch is very practical and will expand our for loop scenario. In short, I look forward to your reply. Happy Chinese New Year!

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-25 Thread houzj.f...@fujitsu.com
On Wednesday, January 25, 2023 7:30 AM Peter Smith wrote: > > Here are my review comments for patch v87-0002. Thanks for your comments. > == > doc/src/sgml/config.sgml > > 1. > > -Allows streaming or serializing changes immediately in > logical decoding. > The

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-25 Thread Robert Haas
On Tue, Jan 24, 2023 at 3:33 PM Peter Geoghegan wrote: > Sure, it's possible that such a cancellable aggressive autovacuum was > indeed cancelled, and that that factor made the crucial difference. > But I find it far easier to believe that there simply was no such > aggressive autovacuum in the

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
>>> Also, I intentionally dropped the GUC_NO_SHOW_ALL check in >>> get_explain_guc_options, because it seems redundant given >>> the preceding GUC_EXPLAIN check. It's unlikely we'd ever have >>> a variable that's marked both GUC_EXPLAIN and GUC_NO_SHOW_ALL ... >>> but if we did, shouldn't the

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Robert Haas
On Tue, Jan 24, 2023 at 11:37 PM Tom Lane wrote: > Andres Freund writes: > > Patches attached. > > +1 for 0001. I'm still nervous about 0002. However, maybe the > cases that we had trouble with are legacy issues that nobody cares > about anymore in 2023. We can always look for another answer

Re: CREATE ROLE bug?

2023-01-25 Thread David G. Johnston
On Wed, Jan 25, 2023 at 7:35 AM Bruce Momjian wrote: > > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission? Are we adding any security by > preventing this? > > As an encouraged design choice you wouldn't. You'd create a new group and

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
David Rowley writes: > Does anyone know of any reason why we shouldn't ditch the nomovement > code in heapgettup/heapgettup_pagemode? AFAICS, the remaining actual use-case for NoMovementScanDirection is that defined by ExecutorRun: *If direction is NoMovementScanDirection then nothing

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
> After looking at this, it seemed to me that the factorization > wasn't quite right after all: specifically, the new function > could be used in several more places if it confines itself to > being a privilege check and doesn't consider GUC_NO_SHOW_ALL. > So more like the attached. > > You could

Re: More pgindent tweaks

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 08:59:44AM -0500, Andrew Dunstan wrote: > After I committed 1249371632 I thought that I should really go ahead and > do what I suggested and allow multiple exclude pattern files for > pgindent. One obvious case is to exclude an in tree meson build > directory. I also

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > Of course there is. This is easy to code in C using unsigned ints, > without resorting to abs() (yes, I'm aware that abs() is undefined for > INT_MIN). So in your opinion what is the expected result of to_hex(INT_MIN, with_sign => true)? -- Best regards, Aleksander Alekseev

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-25 Thread Hayato Kuroda (Fujitsu)
Dear Katsuragi-san, Thank you for reading the patch! PSA new version. > Thank you for updating the patch! > > +/* Check whether the postgres server is still alive or not */ > +extern int PQConnCheck(PGconn *conn); > +extern int PQCanConnCheck(void); > > Aren't these PQconnCheck and

Re: drop postmaster symlink

2023-01-25 Thread Devrim Gündüz
Hi, On Wed, 2023-01-25 at 08:54 +0100, Peter Eisentraut wrote: > > Apart from your concerns, it appears there is consensus for making > this change.  The RPM packaging scripts can obviously be fixed > easily for this.  Do you have an objection to making this change? I'm inclined to create the

Re: old_snapshot_threshold bottleneck on replica

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 3:52 AM Maxim Orlov wrote: > But in TransactionIdLimitedForOldSnapshots these variable is using > conjointly. So, I'm not > sure, is it completely safe to remove mutex. Well, that's something we - and ideally you, as the patch author - need to analyze and figure out. We

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Tom Lane
Nitin Jadhav writes: > I agree that the developer can use both GUC_NO_SHOW_ALL and > GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by > mistake then according to the existing code (without patch), > GUC_NO_SHOW_ALL takes higher precedence whether it is marked first or > last in

CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
This works in PG 15: CREATE ROLE service CREATEROLE; CREATE ROLE service1 WITH LOGIN IN ROLE service; SET SESSION AUTHORIZATION service; CREATE ROLE service2 WITH LOGIN IN ROLE service; but generates an error in git master: CREATE ROLE service CREATEROLE;

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-25 Thread Takamichi Osumi (Fujitsu)
On Wednesday, January 25, 2023 3:55 PM Amit Kapila wrote: > On Wed, Jan 25, 2023 at 11:23 AM Takamichi Osumi (Fujitsu) > wrote: > > > > > > Thank you for checking the patch ! > > On Wednesday, January 25, 2023 10:17 AM Kyotaro Horiguchi > wrote: > > > In short, I'd like to propose renaming the

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 08:47:14AM -0500, Robert Haas wrote: > > I am not sure if the behavior is wrong, the error message is wrong, or > > it is working as expected. > > It is indeed related to that discussion and change. In existing > released branches, a CREATEROLE user can make any role a

[PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi hackers, Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: ``` CREATE TABLE t (a INT UNIQUE, b INT); INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; -- succeeds, inserting the first row and ignoring the second ``` ... but not for ON CONFLICT .. DO UPDATE: ```

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread songjinzhou
>Hi >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> napsal: >Hello, my personal understanding is that you can use multiple iterative >controls (as a merge) in a fo loop, otherwise we can only separate these >iterative controls, but in fact, they may do the same thing. >1.

Re: to_hex() for negative inputs

2023-01-25 Thread Dean Rasheed
On Wed, 25 Jan 2023 at 10:57, Aleksander Alekseev wrote: > > > Of course there is. This is easy to code in C using unsigned ints, > > without resorting to abs() (yes, I'm aware that abs() is undefined for > > INT_MIN). > > So in your opinion what is the expected result of to_hex(INT_MIN, >

Re: Logical replication timeout problem

2023-01-25 Thread Amit Kapila
On Tue, Jan 24, 2023 at 8:15 AM wangw.f...@fujitsu.com wrote: > > Attach the new patch. > I think the patch missed to handle the case of non-transactional messages which was previously getting handled. I have tried to address that in the attached. Is there a reason that shouldn't be handled?

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-25 Thread shveta malik
On Mon, Jan 23, 2023 at 6:30 PM Melih Mutlu wrote: > > Hi, > > Thanks for your review. > Attached updated versions of the patches. > Hello, I am still in the process of reviewing the patch, before that I tried to run below test: --publisher create table tab1(id int , name varchar); create table

Re: CREATE ROLE bug?

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 8:29 AM Bruce Momjian wrote: > This works in PG 15: > > CREATE ROLE service CREATEROLE; > CREATE ROLE service1 WITH LOGIN IN ROLE service; > SET SESSION AUTHORIZATION service; > CREATE ROLE service2 WITH LOGIN IN ROLE service; > > but

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 07:38:51AM -0700, David G. Johnston wrote: > On Wed, Jan 25, 2023 at 7:35 AM Bruce Momjian wrote: > > > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission?  Are we adding any security by > preventing

Re: More pgindent tweaks

2023-01-25 Thread Andrew Dunstan
On 2023-01-25 We 09:41, Bruce Momjian wrote: > On Wed, Jan 25, 2023 at 08:59:44AM -0500, Andrew Dunstan wrote: >> After I committed 1249371632 I thought that I should really go ahead and >> do what I suggested and allow multiple exclude pattern files for >> pgindent. One obvious case is to

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: >> Rename contrib module basic_archive to basic_wal_module > > FWIW, I find this new name much less clear than the old one. > > If we want to provide a basic_archive module and

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 19.01.23 21:48, Jacob Champion wrote: I like the existing "caveats" documentation, and I've attached a sample patch with some more caveats documented, based on some of the upthread conversation: - text format makes fixed-length columns leak length information too - you only get partial

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Andres, > I don't think I agree with this being a bug. Perhaps that's not a bug especially considering the fact that the documentation describes this behavior, but in any case the fact that: ``` INSERT INTO t VALUES (1,1) ON CONFLICT (a) DO UPDATE SET b = 0; INSERT INTO t VALUES (1,2) ON

pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Dimos Stamatakis
Hi hackers, I attempted to perform an upgrade from PG-14.5 to PG-15.1 with pg_upgrade and unfortunately it errors out because of a function that does not exist anymore in PG-15.1. The function is ‘pg_catalog.close_lb’ and it exists in 14.5 but not in 15.1. In our scenario we changed the

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 6:28 PM, Tomas Vondra wrote: > > > > On 1/25/23 16:37, gkokola...@pm.me wrote: > > > --- Original Message --- > > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > > pry...@telsasoft.com wrote: >

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Jacob Champion
On Wed, Jan 25, 2023 at 7:47 AM Israel Barth Rubio wrote: > I imagine more people might have already hit a similar situation too. While > the > workaround can seem a bit weird, in my very humble opinion the user/client is > somehow still the one to blame in this case as it is providing the

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:12:00PM +0900, Kyotaro Horiguchi wrote: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else

Re: What object types should be in schemas?

2023-01-25 Thread Peter Eisentraut
On 12.01.23 18:41, Alvaro Herrera wrote: I think one important criterion to think about is how does encryption work when you have per-customer (or per-whatever) schemas. Is the concept of a column encryption [objtype] a thing that you would like to set up per customer? In that case, you will

Re: pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Christoph Moench-Tegeder
## Dimos Stamatakis (dimos.stamata...@servicenow.com): > In our scenario we changed the permissions of this function in PG14.5 > (via an automated tool) and then pg_upgrade tries to change the > permissions in PG15.1 as well. Given that this function wasn't even documented and did nothing but

Re: Syncrep and improving latency due to WAL throttling

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 14:32:51 +0100, Jakub Wartak wrote: > In other words it allows slow down of any backend activity. Any feedback on > such a feature is welcome, including better GUC name proposals ;) and > conditions in which such feature should be disabled even if it would be > enabled globally

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Peter, > It also makes DO UPDATE not work the same way as either UPDATE itself > (which will silently skip a second or subsequent update of the same > row by the same UPDATE statement in RC mode), or MERGE (which has > similar cardinality violations). That's true. On the flip side, UPDATE and

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > > So in your opinion what is the expected result of to_hex(INT_MIN, > > with_sign => true)? > > > > "-8000" or "-0x8000", depending on whether the prefix is > requested. Whether this is the right result is very debatable. 0x8000 is a binary representation of -2147483648:

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 21:42 -0500, Robert Haas wrote: > I find it a bit premature to include this comment in the very first > email what if other people don't like the idea? The trust_strxfrm GUC was pulled from the larger collation refactoring patch, which has been out for a while. The

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 10:02:28 -0500, Tom Lane wrote: > David Rowley writes: > > Does anyone know of any reason why we shouldn't ditch the nomovement > > code in heapgettup/heapgettup_pagemode? +1 Because I dug it up yesterday. There used to be callers of heap* with NoMovement. But they were

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jacob, > I'm not sure how helpful it is to assign "blame" here. I think the > requested improvement is reasonable -- it should be possible to > override the default for a particular connection, without having to > pick a junk value that you hope doesn't match up with an actual file > on the

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 12.01.23 17:32, Peter Eisentraut wrote: Can we do anything about the attack vector wherein a malicious DBA simply copies the encrypted datum from one row to another? We discussed this earlier [0].  This patch is not that feature.  We could get there eventually, but it would appear to be an

Re: CREATE ROLE bug?

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 9:35 AM Bruce Momjian wrote: > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission? Are we adding any security by > preventing this? They can't, because a role can't ever have ADMIN OPTION on itself, and you need

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Tomas Vondra
On 1/25/23 16:37, gkokola...@pm.me wrote: > > > > > > --- Original Message --- > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > wrote: > > >> >> >> On Tue, Jan 24, 2023 at 03:56:20PM +, gkokola...@pm.me wrote: >> >>> On Monday, January 23rd, 2023 at 7:00 PM,

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart wrote: > On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier > > wrote: > >> Rename contrib module basic_archive to basic_wal_module > > > > FWIW, I find this new name much less clear

Re: Proposal: Support custom authentication methods using hooks

2023-01-25 Thread Andrey Chudnovsky
Greetings, Want to resurface the OAUTH support topic in the context of the concerns raised here. > How about- if we just added OAUTH support directly into libpq and the > backend, would that work with Azure's OIDC provider? If not, why not? > If it does, then what's the justification for trying

Re: Implement missing join selectivity estimation for range types

2023-01-25 Thread Mahmoud Sakr
Hi Tomas, > I finally had time to properly read the paper today - the general > approach mostly matches how I imagined the estimation would work for > inequalities, but it's definitely nice to see the algorithm properly > formalized and analyzed. Awesome, thanks for this interest! > What seems

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread Andres Freund
Hi, On 2023-01-14 12:34:03 -0800, Andres Freund wrote: > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > Please review the attached v2 patch further. > > > > I'm still unclear on the performance goals of this patch. I see that

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Tom Lane
Kyotaro Horiguchi writes: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else that needed adjusting. > There seems to

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 12:21:14PM -0500, Robert Haas wrote: > But it does seem pretty important to keep talking about these things, > because there's definitely no guarantee whatsoever that all of the > commits I've made to master in this area are without problems. If we > find important cases

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Justin Pryzby
On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > Of course, one can throw the error before entering the loop, yet I think > that it does not help the readability of the code. IMHO it is easier to > follow if the error is thrown once during that check. > If anything, I can

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 07.01.23 01:34, Justin Pryzby wrote: "ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END = t.oid)\n" This breaks interoperability with older servers: ERROR: column a.attrealtypid does not exist Same in describe.c Find attached some typos and bad indentation. I'm

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 11:01 AM Aleksander Alekseev wrote: > Just to make sure we are on the same page. The patch doesn't break the > current DO NOTHING behavior but rather makes DO UPDATE work the same > way DO NOTHING does. It also makes DO UPDATE not work the same way as either UPDATE itself

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 7:00 PM, Justin Pryzby wrote: > > > On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > > While looking at this, I realized that commit 5e73a6048 introduced a > regression: > > @@ -3740,19 +3762,24

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Pavel Stehule
st 25. 1. 2023 v 17:22 odesílatel songjinzhou <2903807...@qq.com> napsal: > > >Hi > > >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> > napsal: Hello, my personal understanding is that you can use multiple > iterative controls (as a merge) in a fo loop, otherwise we can only >

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: > Rename contrib module basic_archive to basic_wal_module FWIW, I find this new name much less clear than the old one. If we want to provide a basic_archive module and a basic_recovery module, that seems fine. Why merge them? -- Robert

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 11:49:27AM -0500, Tom Lane wrote: > Right. I fixed some other infelicities and pushed it. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 18:45:12 +0300, Aleksander Alekseev wrote: > Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: > > ``` > CREATE TABLE t (a INT UNIQUE, b INT); > INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; > -- succeeds, inserting the first row and ignoring

Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
The attached patch responds to the discussion at [1] about how we ought to offer a way to set any server GUC from the initdb command line. Currently, if for some reason the server won't start with default parameters, the only way to get through initdb is to change the installed version of

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > I wanted to merge basic_archive and basic_recovery because there's a decent > chunk of duplicated code. Would said code likely be duplicated into non-test uses of this feature? If so, maybe you ought to factor it out into a common location. I agree with Robert's point

Re: Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 16:25:19 -0500, Tom Lane wrote: >> The attached patch responds to the discussion at [1] about how >> we ought to offer a way to set any server GUC from the initdb >> command line. > Are you thinking of backpatching this, to offer the people affected by the >

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 02:41:18PM -0800, Nathan Bossart wrote: > This is my preference. If Michael and Robert are okay with it, I think > this is what we should do. Else, I'll create separate basic_archive and > basic_restore modules. Grouping both things into the same module has the advantage

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-25 Thread Jacob Champion
On 1/24/23 12:04, Robert Haas wrote: > I find the concept of "ambient authentication" problematic. I don't > know exactly what you mean by it. I hope you'll tell me, Sure: Ambient authority [1] means that something is granted access based on some aspect of its existence that it can't remove (or

[BUG] pg_stat_statements and extended query protocol

2023-01-25 Thread Imseih (AWS), Sami
Doing some work with extended query protocol, I encountered the same issue that was discussed in [1]. It appears when a client is using extended query protocol and sends an Execute message to a portal with max_rows, and a portal is executed multiple times, pg_stat_statements does not correctly

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote: > On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote: > > 0001 (the freezing strategies patch) is now committable IMV. Or at > > least will be once I polish the docs a bit more. I plan on committing > > 0001 some time next week, barring

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote: > Another bad scenario: Some longrunning / hung transaction caused us to get > close to the xid wraparound. Problem was resolved, autovacuum runs. Previously > we wouldn't have frozen the portion of the table that was actively changing, > now

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 17:22:32 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote: > > I unfortunately haven't been able to keep up with the thread and saw this > > just > > now. But I've expressed the concern below several times before, so it > > shouldn't come as

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hk, On 2023-01-25 18:31:16 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote: > > Sure. But significantly regressing plausible if not common workloads is > > different than knowing that there'll be some edge case where we'll do > > something worse. > > That's

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 18:43:10 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote: > > As far as I can tell, with the eager strategy, the only thing > > vacuum_freeze_min_age really influences is whether we'll block waiting for a > > cleanup lock. IOW, VACUUM on a

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Andres Freund
Hi, Pushed the patches. So far no fallout, and hoverfly recovered. I just checked a few of the more odd animals (Illumos, Solaris, old OpenBSD, AIX) that already ran without finding new warnings. There's a few more animals to run before I'll fully relax though. On 2023-01-25 08:31:23 -0500,

improving user.c error messages

2023-01-25 Thread Nathan Bossart
moving this discussion to a new thread... On Thu, Jan 19, 2023 at 10:20:33AM -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 6:17 PM Nathan Bossart > wrote: >> However, as the attribute >> system becomes more sophisticated, I think we ought to improve the error >> messages in user.c. IMHO

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Nathan Bossart
On Thu, Jan 26, 2023 at 01:23:41PM +1300, Thomas Munro wrote: > Can we also use TimestampDifferenceMilliseconds()? It knows about > rounding up for WaitLatch(). I think we might risk overflowing "long" when all the wakeup times are DT_NOEND: * This is typically used to calculate a wait

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote: > From e41d3f45fcd6f639b768c22139006ad11422575f Mon Sep 17 00:00:00 2001 > From: Peter Geoghegan > Date: Thu, 24 Nov 2022 18:20:36 -0800 > Subject: [PATCH v17 1/3] Add eager and lazy freezing strategies to VACUUM. > > Eager freezing

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote: > I unfortunately haven't been able to keep up with the thread and saw this just > now. But I've expressed the concern below several times before, so it > shouldn't come as a surprise. You missed the announcement 9 days ago, and the similar

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 17:37:17 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote: > > Another bad scenario: Some longrunning / hung transaction caused us to get > > close to the xid wraparound. Problem was resolved, autovacuum runs. > > Previously > > we wouldn't

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 7:11 PM Andres Freund wrote: > > > I switched between vacuum_freeze_strategy_threshold=0 and > > > vacuum_freeze_strategy_threshold=too-high, because it's quicker/takes less > > > warmup to set up something with smaller tables. > > > > This makes no sense to me, at all. >

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 19:43 -0600, Justin Pryzby wrote: > I think "an optimization, if applicable" is either too terse, or > somehow > wrong.  Maybe: > > > Enables or disables the use of abbreviated keys, a sort > > optimization... Done. > > +    optimization could return wrong results. Set

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 10:02:28 -0500, Tom Lane wrote: >> We must have the NoMovementScanDirection option because count = 0 >> does not mean "do nothing", and I noted at least two call sites >> that require it. > I wonder if we'd be better off removing NoMovementScanDirection, and

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound (stop telling users to "vacuum that database in single-user mode")

2023-01-25 Thread Justin Pryzby
On Mon, Jan 16, 2023 at 03:50:57PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > The proposed patchset changes the documentation and the error messages > > accordingly, making them less misleading. 0001 corrects the > > documentation but doesn't touch the code. 0002 and 0003 correct the >

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:15 PM Andres Freund wrote: > However, it significantly increases the overall work when rows have a somewhat > limited lifetime. The documented reason why vacuum_freeze_min_age exist - > although I think it doesn't really achieve its documented goal anymore, after > the

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-25 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thank you for updating the patch! Followings are comments. 1. config.sgml ``` +the changes till logical_decoding_work_mem is reached. It can also be ``` I think it should be sandwiched by . 2. config.sgml ``` +On the publisher side, logical_replication_mode allows

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 8:12 PM John Naylor wrote: > That was followed by several paragraphs that never got around to explaining > why table size should drive freezing strategy. You were talking about the system level view of freeze debt, and how the table view might not be a sufficient proxy

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 02:05:39PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart > wrote: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. Perhaps that is okay, but I would rather just >> have one test

Re: drop postmaster symlink

2023-01-25 Thread Karl O. Pinc
Hello, Somehow I missed the email changing the status of this back to "needs review". Buried in https://www.postgresql.org/message-id/20230107165942.748ccf4e%40slate.karlpinc.com is the one change I see that should be made. > In doc/src/sgml/ref/allfiles.sgml at line 222 there is an ENTITY >

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Thomas Munro
On Thu, Jan 26, 2023 at 12:50 PM Nathan Bossart wrote: > I did this in v3. I noticed that many of your comments also applied to the > similar patch that was recently applied to walreceiver.c, so I created > another patch to fix that up. Can we also use TimestampDifferenceMilliseconds()? It

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 16:43:47 -0800, Andres Freund wrote: > I think, as committed, this will cause serious issues for some reasonably > common workloads, due to substantially increased WAL traffic. > > > The most common problematic scenario I see are tables full of rows with > limited lifetime.

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote: > Sure. But significantly regressing plausible if not common workloads is > different than knowing that there'll be some edge case where we'll do > something worse. That's very vague. Significant to whom, for what purpose? > prep: > CREATE

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 26, 2023 at 3:28 PM Tom Lane wrote: >> It'd probably be reasonable to file down that sharp edge by instead >> specifying that TimestampDifferenceMilliseconds will clamp overflowing >> differences to LONG_MAX. Maybe there should be a clamp on the underflow >>

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:50:22PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. > > Would said code likely be duplicated into non-test uses of this feature? > If so, maybe you ought to

Re: improving user.c error messages

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > On Thu, Jan 19, 2023 at 10:20:33AM -0500, Robert Haas wrote: >> That would be great. I agree that it's good to try to improve the >> error messages. It hasn't been entirely clear to me how to do that. >> For instance, I don't think we want to say something like: >> >>

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote: > My point was the other way round. That vacuum_freeze_min_age *prevented* us > from freezing rows "too soon" - obviously a very blunt instrument. Yes, not freezing at all until aggressive vacuum is definitely good when you don't really need

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Thomas Munro
On Thu, Jan 26, 2023 at 3:28 PM Tom Lane wrote: > Nathan Bossart writes: > > I think we might risk overflowing "long" when all the wakeup times are > > DT_NOEND: > > >* This is typically used to calculate a wait timeout for WaitLatch() > >* or a related function. The choice of

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread John Naylor
On Thu, Jan 26, 2023 at 10:11 AM Andres Freund wrote: > I am. Just not every tradeoff. I just don't see any useful tradeoffs purely > based on the relation size. I expressed reservations about relation size six weeks ago: On Wed, Dec 14, 2022 at 12:16 AM Peter Geoghegan wrote: > > On Tue, Dec

  1   2   >