Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-25 Thread John Naylor
On Tue, Jan 24, 2023 at 1:17 PM Dilip Kumar wrote: > > On Mon, Jan 23, 2023 at 6:00 PM John Naylor > wrote: > > > > Attached is a rebase to fix conflicts from recent commits. > > I have reviewed v22-0022* patch and I have some comments. > > 1. > >It also changes to the column names

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-25 Thread John Naylor
On Wed, Jan 25, 2023 at 8:42 AM Masahiko Sawada wrote: > > On Mon, Jan 23, 2023 at 8:20 PM John Naylor > wrote: > > > > On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > > > > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > > > wrote: > > > > In v21, all of your v20 improvements to

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 1:19 AM Andres Freund wrote: > Here's a draft of what I am thinking of. Not perfect yet, but I think it looks > better. I'm afraid that I will be unable to do any more work on this project. I have withdrawn it from the CF app. If you would like to complete some or all of

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 8:24 PM Peter Geoghegan wrote: > > I think we're on a very dangerous path here. I want VACUUM to be > > better as the next person, but I really don't believe that's the > > direction we're headed. I think if we release like this, we're going > > to experience more VACUUM

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Justin Pryzby
On Thu, Jan 26, 2023 at 02:49:27PM +0900, Michael Paquier wrote: > On Wed, Jan 25, 2023 at 12:00:20PM -0600, Justin Pryzby wrote: > > While looking at this, I realized that commit 5e73a6048 introduced a > > regression: > > > > @@ -3740,19 +3762,24 @@ ReadHead(ArchiveHandle *AH) > > > > -

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 12:00:20PM -0600, Justin Pryzby wrote: > While looking at this, I realized that commit 5e73a6048 introduced a > regression: > > @@ -3740,19 +3762,24 @@ ReadHead(ArchiveHandle *AH) > > - if (AH->compression != 0) > - pg_log_warning("archive is

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread Bharath Rupireddy
On Thu, Jan 26, 2023 at 2:45 AM Andres Freund wrote: > > Hi, > > On 2023-01-14 12:34:03 -0800, Andres Freund wrote: > > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > > Please review the attached v2 patch further. > > > > >

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 07:57:18PM +, gkokola...@pm.me wrote: > Nice catch! Let me see.. -- Michael signature.asc Description: PGP signature

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 8:12 PM John Naylor wrote: > That was followed by several paragraphs that never got around to explaining > why table size should drive freezing strategy. You were talking about the system level view of freeze debt, and how the table view might not be a sufficient proxy

Re: fix and document CLUSTER privileges

2023-01-25 Thread Jeff Davis
On Sat, 2023-01-14 at 14:40 -0800, Nathan Bossart wrote: > On Sat, Jan 14, 2023 at 10:40:40AM +0100, Gilles Darold wrote: > > Nathan, please confirm and fix the status of this commit fest > > entry. > > Yes, thank you for taking care of this.  I believe the only changes > in this > patch that

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 7:41 PM Robert Haas wrote: > Both Andres and I have repeatedly expressed concern about how much is > being changed in the behavior of vacuum, and how quickly, and IMHO on > the basis of very limited evidence that the changes are improvements. > The fact that Andres was

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread John Naylor
On Thu, Jan 26, 2023 at 10:11 AM Andres Freund wrote: > I am. Just not every tradeoff. I just don't see any useful tradeoffs purely > based on the relation size. I expressed reservations about relation size six weeks ago: On Wed, Dec 14, 2022 at 12:16 AM Peter Geoghegan wrote: > > On Tue, Dec

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 26, 2023 at 3:28 PM Tom Lane wrote: >> It'd probably be reasonable to file down that sharp edge by instead >> specifying that TimestampDifferenceMilliseconds will clamp overflowing >> differences to LONG_MAX. Maybe there should be a clamp on the underflow >>

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Thomas Munro
On Thu, Jan 26, 2023 at 3:28 PM Tom Lane wrote: > Nathan Bossart writes: > > I think we might risk overflowing "long" when all the wakeup times are > > DT_NOEND: > > >* This is typically used to calculate a wait timeout for WaitLatch() > >* or a related function. The choice of

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 18:43:10 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote: > > As far as I can tell, with the eager strategy, the only thing > > vacuum_freeze_min_age really influences is whether we'll block waiting for a > > cleanup lock. IOW, VACUUM on a

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 7:11 PM Andres Freund wrote: > > > I switched between vacuum_freeze_strategy_threshold=0 and > > > vacuum_freeze_strategy_threshold=too-high, because it's quicker/takes less > > > warmup to set up something with smaller tables. > > > > This makes no sense to me, at all. >

Re: Non-superuser subscription owners

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 17:00 -0500, Robert Haas wrote: > It seems to me that the relevant > question isn't "are the servers tightly coupled?" but rather "could > some user make a mess if we let them use any arbitrary connection > string?". The split I created is much easier for an admin to answer:

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 8:49 PM Andres Freund wrote: > The concrete setting of vacuum_freeze_strategy_threshold doesn't matter. > Table size simply isn't a usable proxy for whether eager freezing is a good > idea or not. I strongly agree. I can't imagine how a size-based threshold can make any

Re: Record queryid when auto_explain.log_verbose is on

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 04:46:36PM +0900, Michael Paquier wrote: > Thanks. Will check and probably apply on HEAD. Done, after adding one test case with compute_query_id=regress and applying some indentation. -- Michael signature.asc Description: PGP signature

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-25 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thank you for updating the patch! Followings are comments. 1. config.sgml ``` +the changes till logical_decoding_work_mem is reached. It can also be ``` I think it should be sandwiched by . 2. config.sgml ``` +On the publisher side, logical_replication_mode allows

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hk, On 2023-01-25 18:31:16 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote: > > Sure. But significantly regressing plausible if not common workloads is > > different than knowing that there'll be some edge case where we'll do > > something worse. > > That's

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote: > My point was the other way round. That vacuum_freeze_min_age *prevented* us > from freezing rows "too soon" - obviously a very blunt instrument. Yes, not freezing at all until aggressive vacuum is definitely good when you don't really need

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 17:28:48 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 5:15 PM Andres Freund wrote: > > However, it significantly increases the overall work when rows have a > > somewhat > > limited lifetime. The documented reason why vacuum_freeze_min_age exist - > > although I

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote: > Sure. But significantly regressing plausible if not common workloads is > different than knowing that there'll be some edge case where we'll do > something worse. That's very vague. Significant to whom, for what purpose? > prep: > CREATE

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > I think we might risk overflowing "long" when all the wakeup times are > DT_NOEND: >* This is typically used to calculate a wait timeout for WaitLatch() >* or a related function. The choice of "long" as the result type >* is to harmonize with

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 17:37:17 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote: > > Another bad scenario: Some longrunning / hung transaction caused us to get > > close to the xid wraparound. Problem was resolved, autovacuum runs. > > Previously > > we wouldn't

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 17:22:32 -0800, Peter Geoghegan wrote: > On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote: > > I unfortunately haven't been able to keep up with the thread and saw this > > just > > now. But I've expressed the concern below several times before, so it > > shouldn't come as

Re: improving user.c error messages

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > On Thu, Jan 19, 2023 at 10:20:33AM -0500, Robert Haas wrote: >> That would be great. I agree that it's good to try to improve the >> error messages. It hasn't been entirely clear to me how to do that. >> For instance, I don't think we want to say something like: >> >>

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote: > Another bad scenario: Some longrunning / hung transaction caused us to get > close to the xid wraparound. Problem was resolved, autovacuum runs. Previously > we wouldn't have frozen the portion of the table that was actively changing, > now

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 5:15 PM Andres Freund wrote: > However, it significantly increases the overall work when rows have a somewhat > limited lifetime. The documented reason why vacuum_freeze_min_age exist - > although I think it doesn't really achieve its documented goal anymore, after > the

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 16:43:47 -0800, Andres Freund wrote: > I think, as committed, this will cause serious issues for some reasonably > common workloads, due to substantially increased WAL traffic. > > > The most common problematic scenario I see are tables full of rows with > limited lifetime.

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote: > I unfortunately haven't been able to keep up with the thread and saw this just > now. But I've expressed the concern below several times before, so it > shouldn't come as a surprise. You missed the announcement 9 days ago, and the similar

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote: > From e41d3f45fcd6f639b768c22139006ad11422575f Mon Sep 17 00:00:00 2001 > From: Peter Geoghegan > Date: Thu, 24 Nov 2022 18:20:36 -0800 > Subject: [PATCH v17 1/3] Add eager and lazy freezing strategies to VACUUM. > > Eager freezing

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote: > On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote: > > 0001 (the freezing strategies patch) is now committable IMV. Or at > > least will be once I polish the docs a bit more. I plan on committing > > 0001 some time next week, barring

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Nathan Bossart
On Thu, Jan 26, 2023 at 01:23:41PM +1300, Thomas Munro wrote: > Can we also use TimestampDifferenceMilliseconds()? It knows about > rounding up for WaitLatch(). I think we might risk overflowing "long" when all the wakeup times are DT_NOEND: * This is typically used to calculate a wait

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Thomas Munro
On Thu, Jan 26, 2023 at 12:50 PM Nathan Bossart wrote: > I did this in v3. I noticed that many of your comments also applied to the > similar patch that was recently applied to walreceiver.c, so I created > another patch to fix that up. Can we also use TimestampDifferenceMilliseconds()? It

improving user.c error messages

2023-01-25 Thread Nathan Bossart
moving this discussion to a new thread... On Thu, Jan 19, 2023 at 10:20:33AM -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 6:17 PM Nathan Bossart > wrote: >> However, as the attribute >> system becomes more sophisticated, I think we ought to improve the error >> messages in user.c. IMHO

Re: drop postmaster symlink

2023-01-25 Thread Karl O. Pinc
Hello, Somehow I missed the email changing the status of this back to "needs review". Buried in https://www.postgresql.org/message-id/20230107165942.748ccf4e%40slate.karlpinc.com is the one change I see that should be made. > In doc/src/sgml/ref/allfiles.sgml at line 222 there is an ENTITY >

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 06:45:08PM -0500, Tom Lane wrote: > I took a look through this, and have a number of mostly-cosmetic > issues: Thanks for the detailed review. > * It seems wrong that next_sync_start isn't handled as one of the > wakeup[NUM_LRW_WAKEUPS] entries. I see that it needs to be

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound (stop telling users to "vacuum that database in single-user mode")

2023-01-25 Thread Justin Pryzby
On Mon, Jan 16, 2023 at 03:50:57PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > The proposed patchset changes the documentation and the error messages > > accordingly, making them less misleading. 0001 corrects the > > documentation but doesn't touch the code. 0002 and 0003 correct the >

[BUG] pg_stat_statements and extended query protocol

2023-01-25 Thread Imseih (AWS), Sami
Doing some work with extended query protocol, I encountered the same issue that was discussed in [1]. It appears when a client is using extended query protocol and sends an Execute message to a portal with max_rows, and a portal is executed multiple times, pg_stat_statements does not correctly

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-25 Thread Jacob Champion
On 1/24/23 12:04, Robert Haas wrote: > I find the concept of "ambient authentication" problematic. I don't > know exactly what you mean by it. I hope you'll tell me, Sure: Ambient authority [1] means that something is granted access based on some aspect of its existence that it can't remove (or

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 02:41:18PM -0800, Nathan Bossart wrote: > This is my preference. If Michael and Robert are okay with it, I think > this is what we should do. Else, I'll create separate basic_archive and > basic_restore modules. Grouping both things into the same module has the advantage

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 01:58:01PM -0800, Andres Freund wrote: > I think it'd be just fine to keep the name as basic_archive and use it for > both archiving and restoring. Restoring from an archive still deals with > archiving. This is my preference. If Michael and Robert are okay with it, I

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:50:22PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. > > Would said code likely be duplicated into non-test uses of this feature? > If so, maybe you ought to

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 22:00:50 +0300, Aleksander Alekseev wrote: > Perhaps that's not a bug especially considering the fact that the > documentation describes this behavior, but in any case the fact that: > > ``` > INSERT INTO t VALUES (1,1) ON CONFLICT (a) DO UPDATE SET b = 0; > INSERT INTO t

Re: Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 16:25:19 -0500, Tom Lane wrote: >> The attached patch responds to the discussion at [1] about how >> we ought to offer a way to set any server GUC from the initdb >> command line. > Are you thinking of backpatching this, to offer the people affected by the >

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Isaac Morland
On Wed, 25 Jan 2023 at 12:02, Pavel Stehule wrote: > > > st 25. 1. 2023 v 17:22 odesílatel songjinzhou <2903807...@qq.com> napsal: > >> >> As follows, we can only repeat the for statement before we use such SQL: >> >> begin >> for i in 10..20 loop >> raise notice '%', i; -- Things to do >> end

Re: Set arbitrary GUC options during initdb

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 16:25:19 -0500, Tom Lane wrote: > The attached patch responds to the discussion at [1] about how > we ought to offer a way to set any server GUC from the initdb > command line. Are you thinking of backpatching this, to offer the people affected by the issue in [1] a way out?

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 14:05:39 -0500, Robert Haas wrote: > > I wanted to merge basic_archive and basic_recovery because there's a decent > > chunk of duplicated code. Perhaps that is okay, but I would rather just > > have one test module. AFAICT the biggest reason to split it is because we > >

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread SATYANARAYANA NARLAPURAM
On Sat, Jan 14, 2023 at 12:34 PM Andres Freund wrote: > Hi, > > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > Please review the attached v2 patch further. > > > > I'm still unclear on the performance goals of this patch. I

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > I wanted to merge basic_archive and basic_recovery because there's a decent > chunk of duplicated code. Would said code likely be duplicated into non-test uses of this feature? If so, maybe you ought to factor it out into a common location. I agree with Robert's point

Re: to_hex() for negative inputs

2023-01-25 Thread Peter Eisentraut
On 24.01.23 14:10, Dean Rasheed wrote: I also think it might be useful for it to gain a couple of boolean options: 1). An option to output a signed value (defaulting to false, to preserve the current two's complement output). I find the existing behavior so strange, I would rather give up and

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 02:05:39PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart > wrote: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. Perhaps that is okay, but I would rather just >> have one test

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 19:43 -0600, Justin Pryzby wrote: > I think "an optimization, if applicable" is either too terse, or > somehow > wrong.  Maybe: > > > Enables or disables the use of abbreviated keys, a sort > > optimization... Done. > > +    optimization could return wrong results. Set

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 10:02:28 -0500, Tom Lane wrote: >> We must have the NoMovementScanDirection option because count = 0 >> does not mean "do nothing", and I noted at least two call sites >> that require it. > I wonder if we'd be better off removing NoMovementScanDirection, and

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Andres Freund
Hi, Pushed the patches. So far no fallout, and hoverfly recovered. I just checked a few of the more odd animals (Illumos, Solaris, old OpenBSD, AIX) that already ran without finding new warnings. There's a few more animals to run before I'll fully relax though. On 2023-01-25 08:31:23 -0500,

Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
The attached patch responds to the discussion at [1] about how we ought to offer a way to set any server GUC from the initdb command line. Currently, if for some reason the server won't start with default parameters, the only way to get through initdb is to change the installed version of

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 21:42 -0500, Robert Haas wrote: > I find it a bit premature to include this comment in the very first > email what if other people don't like the idea? The trust_strxfrm GUC was pulled from the larger collation refactoring patch, which has been out for a while. The

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread Andres Freund
Hi, On 2023-01-14 12:34:03 -0800, Andres Freund wrote: > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > Please review the attached v2 patch further. > > > > I'm still unclear on the performance goals of this patch. I see that

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 7:00 PM, Justin Pryzby wrote: > > > On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > > While looking at this, I realized that commit 5e73a6048 introduced a > regression: > > @@ -3740,19 +3762,24

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > > So in your opinion what is the expected result of to_hex(INT_MIN, > > with_sign => true)? > > > > "-8000" or "-0x8000", depending on whether the prefix is > requested. Whether this is the right result is very debatable. 0x8000 is a binary representation of -2147483648:

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 6:28 PM, Tomas Vondra wrote: > > > > On 1/25/23 16:37, gkokola...@pm.me wrote: > > > --- Original Message --- > > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > > pry...@telsasoft.com wrote: >

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Peter, > It also makes DO UPDATE not work the same way as either UPDATE itself > (which will silently skip a second or subsequent update of the same > row by the same UPDATE statement in RC mode), or MERGE (which has > similar cardinality violations). That's true. On the flip side, UPDATE and

Re: Implement missing join selectivity estimation for range types

2023-01-25 Thread Mahmoud Sakr
Hi Tomas, > I finally had time to properly read the paper today - the general > approach mostly matches how I imagined the estimation would work for > inequalities, but it's definitely nice to see the algorithm properly > formalized and analyzed. Awesome, thanks for this interest! > What seems

Re: Proposal: Support custom authentication methods using hooks

2023-01-25 Thread Andrey Chudnovsky
Greetings, Want to resurface the OAUTH support topic in the context of the concerns raised here. > How about- if we just added OAUTH support directly into libpq and the > backend, would that work with Azure's OIDC provider? If not, why not? > If it does, then what's the justification for trying

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 11:01 AM Aleksander Alekseev wrote: > Just to make sure we are on the same page. The patch doesn't break the > current DO NOTHING behavior but rather makes DO UPDATE work the same > way DO NOTHING does. It also makes DO UPDATE not work the same way as either UPDATE itself

Re: pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Christoph Moench-Tegeder
## Dimos Stamatakis (dimos.stamata...@servicenow.com): > In our scenario we changed the permissions of this function in PG14.5 > (via an automated tool) and then pg_upgrade tries to change the > permissions in PG15.1 as well. Given that this function wasn't even documented and did nothing but

Re: What object types should be in schemas?

2023-01-25 Thread Peter Eisentraut
On 12.01.23 18:41, Alvaro Herrera wrote: I think one important criterion to think about is how does encryption work when you have per-customer (or per-whatever) schemas. Is the concept of a column encryption [objtype] a thing that you would like to set up per customer? In that case, you will

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart wrote: > On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier > > wrote: > >> Rename contrib module basic_archive to basic_wal_module > > > > FWIW, I find this new name much less clear

Re: Syncrep and improving latency due to WAL throttling

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 14:32:51 +0100, Jakub Wartak wrote: > In other words it allows slow down of any backend activity. Any feedback on > such a feature is welcome, including better GUC name proposals ;) and > conditions in which such feature should be disabled even if it would be > enabled globally

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Andres, > I don't think I agree with this being a bug. Perhaps that's not a bug especially considering the fact that the documentation describes this behavior, but in any case the fact that: ``` INSERT INTO t VALUES (1,1) ON CONFLICT (a) DO UPDATE SET b = 0; INSERT INTO t VALUES (1,2) ON

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 19.01.23 21:48, Jacob Champion wrote: I like the existing "caveats" documentation, and I've attached a sample patch with some more caveats documented, based on some of the upthread conversation: - text format makes fixed-length columns leak length information too - you only get partial

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 12.01.23 17:32, Peter Eisentraut wrote: Can we do anything about the attack vector wherein a malicious DBA simply copies the encrypted datum from one row to another? We discussed this earlier [0].  This patch is not that feature.  We could get there eventually, but it would appear to be an

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 07.01.23 01:34, Justin Pryzby wrote: "ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END = t.oid)\n" This breaks interoperability with older servers: ERROR: column a.attrealtypid does not exist Same in describe.c Find attached some typos and bad indentation. I'm

pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Dimos Stamatakis
Hi hackers, I attempted to perform an upgrade from PG-14.5 to PG-15.1 with pg_upgrade and unfortunately it errors out because of a function that does not exist anymore in PG-15.1. The function is ‘pg_catalog.close_lb’ and it exists in 14.5 but not in 15.1. In our scenario we changed the

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 18:45:12 +0300, Aleksander Alekseev wrote: > Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: > > ``` > CREATE TABLE t (a INT UNIQUE, b INT); > INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; > -- succeeds, inserting the first row and ignoring

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 10:02:28 -0500, Tom Lane wrote: > David Rowley writes: > > Does anyone know of any reason why we shouldn't ditch the nomovement > > code in heapgettup/heapgettup_pagemode? +1 Because I dug it up yesterday. There used to be callers of heap* with NoMovement. But they were

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jacob, > I'm not sure how helpful it is to assign "blame" here. I think the > requested improvement is reasonable -- it should be possible to > override the default for a particular connection, without having to > pick a junk value that you hope doesn't match up with an actual file > on the

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: >> Rename contrib module basic_archive to basic_wal_module > > FWIW, I find this new name much less clear than the old one. > > If we want to provide a basic_archive module and

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Justin Pryzby
On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > Of course, one can throw the error before entering the loop, yet I think > that it does not help the readability of the code. IMHO it is easier to > follow if the error is thrown once during that check. > If anything, I can

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 11:49:27AM -0500, Tom Lane wrote: > Right. I fixed some other infelicities and pushed it. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:12:00PM +0900, Kyotaro Horiguchi wrote: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: > Rename contrib module basic_archive to basic_wal_module FWIW, I find this new name much less clear than the old one. If we want to provide a basic_archive module and a basic_recovery module, that seems fine. Why merge them? -- Robert

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 12:21:14PM -0500, Robert Haas wrote: > But it does seem pretty important to keep talking about these things, > because there's definitely no guarantee whatsoever that all of the > commits I've made to master in this area are without problems. If we > find important cases

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Tomas Vondra
On 1/25/23 16:37, gkokola...@pm.me wrote: > > > > > > --- Original Message --- > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > wrote: > > >> >> >> On Tue, Jan 24, 2023 at 03:56:20PM +, gkokola...@pm.me wrote: >> >>> On Monday, January 23rd, 2023 at 7:00 PM,

Re: CREATE ROLE bug?

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 9:35 AM Bruce Momjian wrote: > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission? Are we adding any security by > preventing this? They can't, because a role can't ever have ADMIN OPTION on itself, and you need

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Jacob Champion
On Wed, Jan 25, 2023 at 7:47 AM Israel Barth Rubio wrote: > I imagine more people might have already hit a similar situation too. While > the > workaround can seem a bit weird, in my very humble opinion the user/client is > somehow still the one to blame in this case as it is providing the

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Pavel Stehule
st 25. 1. 2023 v 17:22 odesílatel songjinzhou <2903807...@qq.com> napsal: > > >Hi > > >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> > napsal: Hello, my personal understanding is that you can use multiple > iterative controls (as a merge) in a fo loop, otherwise we can only >

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Tom Lane
Kyotaro Horiguchi writes: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else that needed adjusting. > There seems to

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread songjinzhou
>Hi >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> napsal: >Hello, my personal understanding is that you can use multiple iterative >controls (as a merge) in a fo loop, otherwise we can only separate these >iterative controls, but in fact, they may do the same thing. >1.

Re: More pgindent tweaks

2023-01-25 Thread Andrew Dunstan
On 2023-01-25 We 09:41, Bruce Momjian wrote: > On Wed, Jan 25, 2023 at 08:59:44AM -0500, Andrew Dunstan wrote: >> After I committed 1249371632 I thought that I should really go ahead and >> do what I suggested and allow multiple exclude pattern files for >> pgindent. One obvious case is to

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Tom Lane
Nitin Jadhav writes: > I agree that the developer can use both GUC_NO_SHOW_ALL and > GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by > mistake then according to the existing code (without patch), > GUC_NO_SHOW_ALL takes higher precedence whether it is marked first or > last in

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Matthias van de Meent
On Tue, 24 Jan 2023 at 23:50, Peter Geoghegan wrote: > > On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote: > > 0001 (the freezing strategies patch) is now committable IMV. Or at > > least will be once I polish the docs a bit more. I plan on committing > > 0001 some time next week, barring

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jim/Jacob, > > I do not think it is worth it to change the current behavior of > PostgreSQL > > in that sense. > > Well, I am not suggesting to change the current behavior of PostgreSQL in > that matter. Quite the contrary, I find this feature very convenient, > specially when you need to

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-01-25 Thread Andrei Zubkov
Hi, I've updated this patch for the current master. Also I have some additional explanations.. On Wed, 2023-01-18 at 17:29 +0100, Tomas Vondra wrote: > 1) I'm not sure why the patch is adding tests of permissions on the > pg_stat_statements_reset function? I've fixed that > > 2) If we want

[PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi hackers, Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: ``` CREATE TABLE t (a INT UNIQUE, b INT); INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; -- succeeds, inserting the first row and ignoring the second ``` ... but not for ON CONFLICT .. DO UPDATE: ```

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 07:38:51AM -0700, David G. Johnston wrote: > On Wed, Jan 25, 2023 at 7:35 AM Bruce Momjian wrote: > > > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission?  Are we adding any security by > preventing

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-25 Thread Robert Haas
On Tue, Jan 24, 2023 at 3:33 PM Peter Geoghegan wrote: > Sure, it's possible that such a cancellable aggressive autovacuum was > indeed cancelled, and that that factor made the crucial difference. > But I find it far easier to believe that there simply was no such > aggressive autovacuum in the

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
David Rowley writes: > Does anyone know of any reason why we shouldn't ditch the nomovement > code in heapgettup/heapgettup_pagemode? AFAICS, the remaining actual use-case for NoMovementScanDirection is that defined by ExecutorRun: *If direction is NoMovementScanDirection then nothing

  1   2   >