Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Noah Misch
On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > I've attached an updated patch. The patch includes a regression test > to detect the new violation as we discussed. I've confirmed that > Cirrus CI tests pass. Please confirm on AIX and review the patch. When the context of a "git

Re: Add LZ4 compression in pg_dump

2022-04-05 Thread gkokolatos
--- Original Message --- On Tuesday, April 5th, 2022 at 3:34 AM, Michael Paquier wrote: > On Fri, Apr 01, 2022 at 03:06:40PM +, [email protected] wrote: > Splitting the program and its arguments makes sense. Great. > At the end I am finishing with the attached. I also saw an ove

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-05 Thread Kyotaro Horiguchi
At Tue, 05 Apr 2022 11:16:44 +0900 (JST), Kyotaro Horiguchi wrote in > So, I have the following points in my mind for now. > > - We create the directory "since we know it is just tentative state". > > - Then, check that no directory in pg_tblspc when reaching consistency > when allow_in_plac

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
Hi David: I just came to the office today to double check this patch. I probably can > finish it very soon. > I would share my current review result first and more review is still in progress. There is a lot of amazing stuff there but I'd save the simple +1 and just share something I'm not full

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
> > The root cause is even ExecQual(winstate->runcondition, econtext) return > false, we > still return the slot to the upper node. A simple hack can avoid it. > Forget to say 0002 shows what I mean. -- Best Regards Andy Fan

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 4:08 PM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > > I've attached an updated patch. The patch includes a regression test > > to detect the new violation as we discussed. I've confirmed that > > Cirrus CI tests pass. Please confi

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-05 Thread Kyotaro Horiguchi
At Tue, 05 Apr 2022 16:38:06 +0900 (JST), Kyotaro Horiguchi wrote in > > However, while working on it, I found that I found that recovery faces > > missing tablespace directories *after* reaching consistency. I'm > > examining that further. > > Okay, it was my thinko. But I faced another obst

Re: Pluggable toaster

2022-04-05 Thread Nikita Malakhov
Hi, Thanks for advices. We have 4 branches, for each patch provided, you can check them out - (come copy-paste from the very fist email, where the patches were proposed) 1) 1_toaster_interface https://github.com/postgrespro/postgres/tree/toaster_interface Introduces syntax for storage and formal to

Re: pg_rewind copies

2022-04-05 Thread Daniel Gustafsson
> On 4 Apr 2022, at 15:08, Peter Eisentraut > wrote: > We use snprintf() like that countless times, and approximately none of them > check for overflow. So while you are right, this might not be the place to > start a new policy. Fair enough, I'll remove these hunks before committing and wil

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-05 Thread Kyotaro Horiguchi
At Tue, 05 Apr 2022 16:38:06 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 05 Apr 2022 11:16:44 +0900 (JST), Kyotaro Horiguchi > wrote in > > So, I have the following points in my mind for now. > > > > - We create the directory "since we know it is just tentative state". > > > > - Then,

Re: Unit tests for SLRU

2022-04-05 Thread Aleksander Alekseev
Hi Daniel, > It also doesn't seem all that appealing that SimpleLruInit can > return false on successful function invocation, it makes for a confusing API. Agree. I think using an additional `bool *found` argument would be less confusing. Noah, Pavel, >> The default place for this kind of test

Re: Extensible Rmgr for Table AMs

2022-04-05 Thread Bharath Rupireddy
On Tue, Apr 5, 2022 at 5:55 AM Jeff Davis wrote: > > On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote: > > 1) Why can't rmid be chosen by the extensions in sequential order > > from > > (129 till 255), say, to start with a columnar extension can choose > > 129, another extension can regi

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-04-05 Thread Michael Paquier
On Fri, Mar 18, 2022 at 05:15:56PM -0700, Andres Freund wrote: > Have you measured the performance effects of this? On fast storage with large > shared_buffers I've seen these loops in profiles. It's probably fine, but it'd > be good to verify that. I am wondering if we could make the function inl

Re: Frontend error logging style

2022-04-05 Thread Daniel Gustafsson
> On 30 Mar 2022, at 00:38, Tom Lane wrote: > > Daniel Gustafsson writes: >> On 29 Mar 2022, at 16:38, Peter Eisentraut >> wrote: >>> Most of these should probably be addressed separately from Tom's patch. > >> Yeah, I think so too. > > Agreed. I tried to confine my patch to mechanical chan

Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
I spent some time thinking about a special case of evaluation of the row filter and wrote a comment that might be useful (see the attachment). However now I think that it's not perfect if the code really relies on the fact that value of an indexed column cannot be TOASTed due to size restrictions.

Re: generic plans and "initial" pruning

2022-04-05 Thread Alvaro Herrera
On 2022-Apr-05, Amit Langote wrote: > While at it, maybe it's better to rename ExecInitPruningContext() to > InitPartitionPruneContext(), which I've done in the attached updated > patch. Good call. I had changed that name too, but yours seems a better choice. I made a few other cosmetic changes

Re: logical decoding and replication of sequences

2022-04-05 Thread Amit Kapila
On Mon, Apr 4, 2022 at 3:10 AM Tomas Vondra wrote: > > I did some experiments over the weekend, exploring how to rework the > sequence decoding in various ways. Let me share some WIP patches, > hopefully that can be useful for trying more stuff and moving this > discussion forward. > > I tried two

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Dilip Kumar
On Fri, Apr 1, 2022 at 11:34 PM Robert Haas wrote: > > On Fri, Apr 1, 2022 at 12:08 AM Dilip Kumar wrote: > > After thinking more about this I see there is some value of > > remembering the dead tids in the conveyor belt. Basically, the point > > is if there are multiple indexes and we do the in

Re: Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
Antonin Houska wrote: > I spent some time thinking about a special case of evaluation of the row > filter and wrote a comment that might be useful (see the attachment). However > now I think that it's not perfect if the code really relies on the fact that > value of an indexed column cannot be TO

Re: Logical replication row filtering and TOAST

2022-04-05 Thread Amit Kapila
On Tue, Apr 5, 2022 at 3:52 PM Antonin Houska wrote: > > Antonin Houska wrote: > > > I spent some time thinking about a special case of evaluation of the row > > filter and wrote a comment that might be useful (see the attachment). > > However > > now I think that it's not perfect if the code re

Re: Add LZ4 compression in pg_dump

2022-04-05 Thread Michael Paquier
On Tue, Apr 05, 2022 at 07:13:35AM +, [email protected] wrote: > Thank you. I agree with the attached and I will carry it forward to the > rest of the patchset. No need to carry it forward anymore, I think ;) -- Michael signature.asc Description: PGP signature

Re: Add LZ4 compression in pg_dump

2022-04-05 Thread gkokolatos
--- Original Message --- On Tuesday, April 5th, 2022 at 12:55 PM, Michael Paquier wrote: > On Tue, Apr 05, 2022 at 07:13:35AM +, [email protected] wrote: > No need to carry it forward anymore, I think ;) Thank you for committing! Cheers, //Georgios > -- > Michael

Re: Unit tests for SLRU

2022-04-05 Thread Maxim Orlov
> > Again, I don't have a strong opinion here. If you insist, I will place the > tests to regress.c. > It is up to committer to decide, but I think it would be good to place tests in regression. In my opinion, many things from core may be used by extensions. And then it is up to extension authors

Re: Window Function "Run Conditions"

2022-04-05 Thread David Rowley
On Tue, 5 Apr 2022 at 19:38, Andy Fan wrote: > 1. We can do more on PASSTHROUGH, we just bypass the window function > currently, but IIUC we can ignore all of the following tuples in current > partition > once we go into this mode. patch 0001 shows what I mean. Yeah, there is more performance

Re: Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
Amit Kapila wrote: > > Antonin Houska wrote: > > > > Nevertheless, a comment in pgoutput_row_filter(), saying that TOASTed values > > are not expected if old_slot is NULL, might be useful. > > > > How about something like the attached? Yes, that'd be sufficient. Thanks. -- Antonin Houska Web

Re: Unit tests for SLRU

2022-04-05 Thread Aleksander Alekseev
Hi hackers, >> Again, I don't have a strong opinion here. If you insist, I will place the >> tests to regress.c. > > It is up to committer to decide, but I think it would be good to place tests > in regression. In my opinion, many things from core may be used by extensions. > And then it is up to

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 6:19 AM Dilip Kumar wrote: > I don't think that saving extra index passes is really a small gain. > I think this will save a lot of IO if indexes pages are not in shared > buffers because here we are talking about we can completely avoid the > index passes for some of the in

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-05 Thread Andrew Dunstan
On 4/3/22 22:10, Peter Geoghegan wrote: > On Sun, Apr 3, 2022 at 11:53 AM Andres Freund wrote: >> We've had bugs in pg_upgrade where post-upgrade xid horizons weren't >> correctly >> set. We've had bugs were indexes were corrupted during replay. >> >> The latter can be caught by wal_consistency

Re: generic plans and "initial" pruning

2022-04-05 Thread Amit Langote
On Tue, Apr 5, 2022 at 7:00 PM Alvaro Herrera wrote: > On 2022-Apr-05, Amit Langote wrote: > > While at it, maybe it's better to rename ExecInitPruningContext() to > > InitPartitionPruneContext(), which I've done in the attached updated > > patch. > > Good call. I had changed that name too, but y

Re: Window Function "Run Conditions"

2022-04-05 Thread Andy Fan
On Tue, Apr 5, 2022 at 7:49 PM David Rowley wrote: > On Tue, 5 Apr 2022 at 19:38, Andy Fan wrote: > > 1. We can do more on PASSTHROUGH, we just bypass the window function > > currently, but IIUC we can ignore all of the following tuples in > current partition > > once we go into this mode. pat

API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Markus Wanner
On 24.03.22 20:32, Robert Haas wrote: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint. This patch changed the delayChkpt field of struct PGPROC from bool to int. Back-porting this change could be considered an API breaking change for extensions using this field. I'm not c

A fastpath for TransactionIdIsCurrentTransactionId

2022-04-05 Thread Andy Fan
Hi: In my recent work, I want to check if the xmin for all the tuples is CurrentTransactioniId, then I found we can improve the fastpath for TransactionIdIsCurrentTransactionId like below, would it be safe? This would be helpful if we have lots of sub transactionId. diff --git a/src/backend/a

How to generate a WAL record spanning multiple WAL files?

2022-04-05 Thread Bharath Rupireddy
Hi, I wanted to have a WAL record spanning multiple WAL files of size, say 16MB. I'm wondering if the Full Page Images (FPIs) of a TOAST table would help here. Please let me know if there's any way to generate such large WAL records. Thoughts? Regards, Bharath Rupireddy.

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-05 Thread Andrew Dunstan
On 3/30/22 14:37, Robert Haas wrote: > On Fri, Feb 18, 2022 at 7:02 PM Andres Freund wrote: >> On 2022-02-15 08:06:58 -0800, Andres Freund wrote: >>> The more I think about it the more I'm convinced that if we want to do this, >>> we should do it for variables and functions. >> Btw, if we were t

Re: Handle infinite recursion in logical replication setup

2022-04-05 Thread Ashutosh Bapat
On Tue, Apr 5, 2022 at 6:21 AM Peter Smith wrote: > > Here are my comments for the latest patch v6-0001. > > (I will post my v6-0002 review comments separately) > > PATCH v6-0001 comments > == > > 1.1 General - Option name > > I still feel like the option name is not ideal. Unf

Re: How to generate a WAL record spanning multiple WAL files?

2022-04-05 Thread Matthias van de Meent
On Tue, 5 Apr 2022 at 15:13, Bharath Rupireddy wrote: > > Hi, > > I wanted to have a WAL record spanning multiple WAL files of size, say > 16MB. I'm wondering if the Full Page Images (FPIs) of a TOAST table > would help here. Please let me know if there's any way to generate > such large WAL recor

Re: How to generate a WAL record spanning multiple WAL files?

2022-04-05 Thread Alvaro Herrera
On 2022-Apr-05, Bharath Rupireddy wrote: > Hi, > > I wanted to have a WAL record spanning multiple WAL files of size, say > 16MB. I'm wondering if the Full Page Images (FPIs) of a TOAST table > would help here. Please let me know if there's any way to generate > such large WAL records. It's easi

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 9:02 AM Markus Wanner wrote: > And for this specific case: Is it worth reverting this change and > applying a fully backwards compatible fix, instead? I think it's normally our policy to avoid changing definitions of accessible structs in back branches, except that we allow

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-05 Thread Tom Lane
Andrew Dunstan writes: > On 3/30/22 14:37, Robert Haas wrote: >> @RMT: Andres proposed upthread that we should plan to do this just >> after feature freeze. Accordingly I propose to commit at least 0002 >> and perhaps 0001 if people want it just after feature freeze. I >> therefore ask that the RM

Re: How to generate a WAL record spanning multiple WAL files?

2022-04-05 Thread Andy Fan
Hi, On Tue, Apr 5, 2022 at 9:46 PM Alvaro Herrera wrote: > On 2022-Apr-05, Bharath Rupireddy wrote: > > > Hi, > > > > I wanted to have a WAL record spanning multiple WAL files of size, say > > 16MB. I'm wondering if the Full Page Images (FPIs) of a TOAST table > > would help here. Please let me

Re: Mingw task for Cirrus CI

2022-04-05 Thread Andrew Dunstan
On 4/4/22 16:41, Andres Freund wrote: > Hi, > > On 2022-03-30 17:26:18 -0700, Andres Freund wrote: >> Hi, >> >> On 2022-03-22 19:00:42 +0300, Melih Mutlu wrote: >>> Rebased it. >>> I also removed the temp installation task and >>> used NoDefaultCurrentDirectoryInExePath env variable instead. >> H

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 5, 2022 at 9:02 AM Markus Wanner > wrote: >> And for this specific case: Is it worth reverting this change and >> applying a fully backwards compatible fix, instead? > I think it's normally our policy to avoid changing definitions of > accessible structs in back

Re: ExecRTCheckPerms() and many prunable partitions

2022-04-05 Thread Greg Stark
This is failing regression tests. I don't understand how this patch could be affecting this test though. Perhaps it's a problem with the json patches that were committed recently -- but they don't seem to be causing other patches to fail. diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/js

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 10:17 AM Tom Lane wrote: > Renaming it would constitute an API break, which is if anything worse > than an ABI break. I don't think so, because an API break will cause a compilation failure, which an extension author can easily fix. > While we're complaining at you, let me

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 5, 2022 at 10:17 AM Tom Lane wrote: >> Renaming it would constitute an API break, which is if anything worse >> than an ABI break. > I don't think so, because an API break will cause a compilation > failure, which an extension author can easily fix. My point is

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 10:32 AM Tom Lane wrote: > Robert Haas writes: > > On Tue, Apr 5, 2022 at 10:17 AM Tom Lane wrote: > >> Renaming it would constitute an API break, which is if anything worse > >> than an ABI break. > > > I don't think so, because an API break will cause a compilation > > f

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 8:41 AM Thom Brown wrote: > I know it's been 8 years, but I still think it would be a useful note > to add to the docs. Makes sense. I will do this soon if nobody objects. I'm mildly uncomfortable with the phrase "WAL records generated over the delay period" because it see

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-05 Thread Magnus Hagander
On Tue, Apr 5, 2022 at 4:54 PM Robert Haas wrote: > On Tue, Apr 5, 2022 at 8:41 AM Thom Brown wrote: > > I know it's been 8 years, but I still think it would be a useful note > > to add to the docs. > Many points for bringing that one back :) Makes sense. I will do this soon if nobody objects

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 10:58 AM Magnus Hagander wrote: >> Makes sense. I will do this soon if nobody objects. >> >> I'm mildly uncomfortable with the phrase "WAL records generated over >> the delay period" because it seems a bit imprecise, but I'm not sure >> what would be better and I think the m

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 10:07 AM Tom Lane wrote: > Yeah, the frontend error message rework in [1]. That has exactly > the same constraint that it's likely to break other open patches, > so it'd be better to do it after the CF cutoff. I think that doing > that concurrently with Robert's thing shou

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-05 Thread Stephen Frost
Greetings, * David Steele ([email protected]) wrote: > On 4/4/22 11:42 AM, Nathan Bossart wrote: > >I noticed a couple of other things that can be removed. Since we no longer > >wait on exclusive backup mode during smart shutdown, we can change > >connsAllowed (in postmaster.c) to a boolean and

Re: Proposal for internal Numeric to Uint64 conversion function.

2022-04-05 Thread Robert Haas
On Thu, Mar 17, 2022 at 4:02 PM Tom Lane wrote: > Pretty much, yeah. I'm way more interested in cleaning up the code > we have than in making things prettier for hypothetical future > call sites. In particular, the problem with writing an API in a > vacuum is that you have little evidence that i

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2022-04-05 Thread Yura Sokolov
Good day, David. I'm looking on patch and don't get some moments. `GrantLockLocal` allocates `LOCALLOCKOWNER` and links it into `locallock->locallockowners`. It links it regardless `owner` could be NULL. But then `RemoveLocalLock` does `Assert(locallockowner->owner != NULL);`. Why it should not f

Re: LogwrtResult contended spinlock

2022-04-05 Thread Alvaro Herrera
Here's a v8, where per my previous comment I removed some code that I believe is no longer necessary. I've omitted the patch that renames LogwrtResult subvariables into LogWriteResult/LogWriteFlush; I still think the end result is better after that one, but it's a pretty trivial change that can be

Re: Printing backtrace of postgres processes

2022-04-05 Thread Robert Haas
On Wed, Mar 30, 2022 at 12:03 PM Justin Pryzby wrote: > On Wed, Mar 30, 2022 at 11:53:52AM -0400, Greg Stark wrote: > > Sadly the cfbot is showing a patch conflict again. It's just a trivial > > conflict in the regression test schedule so I'm not going to update > > the status but it would be good

Re: shared-memory based stats collector - v68

2022-04-05 Thread David G. Johnston
On Mon, Apr 4, 2022 at 7:36 PM Andres Freund wrote: > > I think all this is going to achieve is to making code more complicated. > There > is a *single* non-assert use of accessed_across_databases and now a single > assertion involving it. > > What would having PGSTAT_KIND_CLUSTER and PGSTAT_KIND

Re: Separate the result of \watch for each query execution (psql)

2022-04-05 Thread Robert Haas
On Mon, Mar 21, 2022 at 9:15 PM Andres Freund wrote: > On 2022-02-25 13:23:31 +0900, Noboru Saito wrote: > > I have created a patch that allows you to turn it on and off in \pset. > > The patch unfortunately causes tests to fail: It doesn't seem like the originally proposed design here will be ac

How to simulate sync/async standbys being closer/farther (network distance) to primary in core postgres?

2022-04-05 Thread Bharath Rupireddy
Hi, I'm thinking if there's a way in core postgres to achieve $subject. In reality, the sync/async standbys can either be closer/farther (which means sync/async standbys can receive WAL at different times) to primary, especially in cloud HA environments with primary in one Availability Zone(AZ)/Re

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-05 Thread David Steele
On 4/5/22 11:25 AM, Stephen Frost wrote: Please find attached an updated patch + commit message. Mostly, I just went through and did a bit more in terms of updating the documentation and improving the comments (there were some places that were still worrying about the chance of a 'stray' backup

Re: [PATCH] pg_stat_toast v10

2022-04-05 Thread Robert Haas
On Thu, Mar 31, 2022 at 9:16 AM Gunnar "Nick" Bluth wrote: > That was meant to say "v10", sorry! Hi, >From my point of view, at least, it would be preferable if you'd stop changing the subject line every time you post a new version. Based on the test results in http://postgr.es/m/42bfa680-7998-

Re: How to generate a WAL record spanning multiple WAL files?

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 10:10 AM Andy Fan wrote: >> > I wanted to have a WAL record spanning multiple WAL files of size, say >> > 16MB. I'm wondering if the Full Page Images (FPIs) of a TOAST table >> > would help here. Please let me know if there's any way to generate >> > such large WAL records.

Re: Add index scan progress to pg_stat_progress_vacuum

2022-04-05 Thread Imseih (AWS), Sami
>I nevertheless think that's not acceptable. The whole premise of the > progress >reporting infrastructure is to be low overhead. It's OK to require locking > to >initialize parallel progress reporting, it's definitely not ok to require >locking to report progress. Fair point. >

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-05 Thread Magnus Hagander
On Tue, Apr 5, 2022 at 5:25 PM Stephen Frost wrote: > Greetings, > > * David Steele ([email protected]) wrote: > > On 4/4/22 11:42 AM, Nathan Bossart wrote: > > >I noticed a couple of other things that can be removed. Since we no > longer > > >wait on exclusive backup mode during smart shutdow

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-04-05 Thread Peter Geoghegan
On Mon, Apr 4, 2022 at 8:25 PM Peter Geoghegan wrote: > Right. The reason I used WARNINGs was because it matches vaguely > related WARNINGs in vac_update_relstats()'s sibling function, > vacuum_set_xid_limits(). Okay, pushed the relfrozenxid warning patch. Thanks -- Peter Geoghegan

Re: PostgreSQL 15 Release Management Team (RMT) + Feature Freeze

2022-04-05 Thread Jonathan S. Katz
On 3/26/22 11:10 AM, Jonathan S. Katz wrote: Additionally, the RMT has set the feature freeze date to be April 7, 2022. This is the last day to commit features for PostgreSQL 15. In other words, no new PostgreSQL 15 feature can be committed after April 8 0:00, 2022 AoE[1]. [1] https://en.wik

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-05 Thread Robert Haas
On Mon, Apr 4, 2022 at 10:20 PM Thomas Munro wrote: > > The checkpointer never takes heavyweight locks, so the opportunity > > you're describing can't arise. > > Hmm, oh, you probably meant the buffer interlocking > in SyncOneBuffer(). It's true that my most recent patch throws away > more requ

Re: Add index scan progress to pg_stat_progress_vacuum

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 12:42 PM Imseih (AWS), Sami wrote: > >Why isn't the obvious thing to do here to provide a way to associate > > workers > >with their leaders in shared memory, but to use the existing progress > > fields > >to report progress? Then, when querying progress, the l

Re: shared-memory based stats collector - v68

2022-04-05 Thread Andres Freund
Hi, On 2022-04-05 08:49:36 -0700, David G. Johnston wrote: > On Mon, Apr 4, 2022 at 7:36 PM Andres Freund wrote: > > > > > I think all this is going to achieve is to making code more complicated. > > There > > is a *single* non-assert use of accessed_across_databases and now a single > > asserti

Re: Add index scan progress to pg_stat_progress_vacuum

2022-04-05 Thread Andres Freund
On 2022-04-05 16:42:28 +, Imseih (AWS), Sami wrote: > >Why isn't the obvious thing to do here to provide a way to associate > > workers > >with their leaders in shared memory, but to use the existing progress > > fields > >to report progress? Then, when querying progress, the lead

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-05 Thread John Naylor
On Tue, Apr 5, 2022 at 10:06 PM Robert Haas wrote: > > On Tue, Apr 5, 2022 at 10:07 AM Tom Lane wrote: > > Yeah, the frontend error message rework in [1]. That has exactly > > the same constraint that it's likely to break other open patches, > > so it'd be better to do it after the CF cutoff. I

Re: LogwrtResult contended spinlock

2022-04-05 Thread Alvaro Herrera
Apologies -- I selected the wrong commit to extract the commit message from. Here it is again. I also removed an obsolete /* XXX */ comment. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ >From 0b26f90f1b95f8e9b932eb34bbf9c2a50729cf60 Mon Sep 17 00:00:00 2001 F

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-04-05 Thread Jacob Champion
On Tue, 2022-04-05 at 15:13 +0900, Michael Paquier wrote: > On Wed, Mar 30, 2022 at 04:02:09PM +, Jacob Champion wrote: > > Whether that's a problem in the future entirely depends on whether > > there's some authentication method that considers the empty string a > > sane and meaningful identit

Re: shared-memory based stats collector - v68

2022-04-05 Thread David G. Johnston
On Tuesday, April 5, 2022, Andres Freund wrote: > Hi, > > On 2022-04-05 08:49:36 -0700, David G. Johnston wrote: > > On Mon, Apr 4, 2022 at 7:36 PM Andres Freund wrote: > > > > > > > > I think all this is going to achieve is to making code more > complicated. > > > There > > > is a *single* non-

Re: Add index scan progress to pg_stat_progress_vacuum

2022-04-05 Thread Imseih (AWS), Sami
>Can't the progress data trivially be inferred by the fact that the worker >completed? Yes, at some point, this idea was experimented with in 0004-Expose-progress-for-the-vacuuming-indexes-cleanup-ph.patch. This patch did the calculation in system_views.sql However, the view is complex an

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-05 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 5, 2022 at 10:32 AM Tom Lane wrote: >> My point is that we want that to happen in HEAD, but it's not okay >> for it to happen in a minor release of a stable branch. > I understand, but I am not sure that I agree. I think that if an > extension stops compiling ag

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-05 Thread Tom Lane
Robert Haas writes: > Do you care whether your commit > or mine goes in first? I do not. If they're not independent, at least one of us has messed up. I have family commitments on Saturday, so if I don't get mine in on Friday it'll likely not happen before Sunday. regar

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Peter Geoghegan
On Tue, Apr 5, 2022 at 5:44 AM Robert Haas wrote: > The whole idea of decoupling table and index vacuum > supposes that there are situations in which it's worth performing the > first heap pass where we gather the dead line pointers but where it's > not necessary to follow that up as quickly as po

Re: Temporary file access API

2022-04-05 Thread Robert Haas
On Tue, Mar 8, 2022 at 6:12 AM Antonin Houska wrote: > Thanks for your comments, the initial version is attached here. I've been meaning to look at this thread for some time but have not found enough time to do that until just now. And now I have questions... 1. Supposing we accepted this, how w

Re: Preventing indirection for IndexPageGetOpaque for known-size page special areas

2022-04-05 Thread Robert Haas
On Fri, Apr 1, 2022 at 11:12 AM Matthias van de Meent wrote: > Here's a new 0001 to keep CFBot happy. This seems like it would conflict with the proposal from http://postgr.es/m/ca+tgmoad8wmn6i1mmuo+4znege3hd57ys8uv8uzm7cneqy3...@mail.gmail.com which I still hope to advance in some form at an app

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-05 Thread Nathan Bossart
On Tue, Apr 05, 2022 at 11:25:36AM -0400, Stephen Frost wrote: > Please find attached an updated patch + commit message. Mostly, I just > went through and did a bit more in terms of updating the documentation > and improving the comments (there were some places that were still > worrying about the

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 3:22 PM Peter Geoghegan wrote: > It's not just an enabler of more frequent index vacuuming (for those > indexes that need it the most), though. It's also an enabler of more > frequent lazy_scan_prune processing (in particular setting hint bits > and freezing), which is proba

Re: ExecRTCheckPerms() and many prunable partitions

2022-04-05 Thread David Rowley
On Wed, 6 Apr 2022 at 02:27, Greg Stark wrote: > > This is failing regression tests. I don't understand how this patch > could be affecting this test though. Perhaps it's a problem with the > json patches that were committed recently -- but they don't seem to be > causing other patches to fail. I

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Peter Geoghegan
On Tue, Apr 5, 2022 at 1:10 PM Robert Haas wrote: > I had assumed that this would not be the case, because if the page is > being accessed by the workload, it can be pruned - and probably frozen > too, if we wanted to write code for that and spend the cycles on it - > and if it isn't, pruning and

Re: shared-memory based stats collector - v66

2022-04-05 Thread Andres Freund
Hi, On 2022-04-02 01:16:48 -0700, Andres Freund wrote: > I just noticed that the code doesn't appear to actually work like that right > now. Whenever the timeout is reached, pgstat_report_stat() is called with > force = true. > > And even if the backend is busy running queries, once there's conte

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Mon, Apr 4, 2022 at 8:05 PM Andres Freund wrote: > - added an architecture overview comment to the top of pgstat.c - not sure > if > it makes sense to anybody but me (and perhaps Horiguchi-san)? > > I took a look at this, diff attached. Some typos and minor style stuff, plus trying to bring

Re: [PATCH] Add extra statistics to explain for Nested Loop

2022-04-05 Thread Greg Stark
This is not passing regression tests due to some details of the plan output - marking Waiting on Author: diff -w -U3 c:/cirrus/src/test/regress/expected/partition_prune.out c:/cirrus/src/test/recovery/tmp_check/results/partition_prune.out --- c:/cirrus/src/test/regress/expected/partition_prune.out

Re: shared-memory based stats collector - v69

2022-04-05 Thread Andres Freund
Hi, On 2022-04-05 13:51:12 -0700, David G. Johnston wrote: > On Mon, Apr 4, 2022 at 8:05 PM Andres Freund wrote: > > > - added an architecture overview comment to the top of pgstat.c - not sure > > if > > it makes sense to anybody but me (and perhaps Horiguchi-san)? > > > > > I took a look at

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 2:23 PM Andres Freund wrote: > > On 2022-04-05 13:51:12 -0700, David G. Johnston wrote: > > >, but rather add to the shared queue > > Queue? Maybe you mean the hashtable? > Queue implemented by a list...? Anyway, I think I mean this: /* * List of PgStat_EntryRefs with u

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Robert Haas
On Tue, Apr 5, 2022 at 4:30 PM Peter Geoghegan wrote: > On Tue, Apr 5, 2022 at 1:10 PM Robert Haas wrote: > > I had assumed that this would not be the case, because if the page is > > being accessed by the workload, it can be pruned - and probably frozen > > too, if we wanted to write code for th

Re: [PATCH] pg_stat_toast

2022-04-05 Thread Gunnar "Nick" Bluth
Am 05.04.22 um 18:17 schrieb Robert Haas: > On Thu, Mar 31, 2022 at 9:16 AM Gunnar "Nick" Bluth > wrote: >> That was meant to say "v10", sorry! > > Hi, Hi Robert, and thx for looking at this. > From my point of view, at least, it would be preferable if you'd stop > changing the subject line ev

MERGE bug report

2022-04-05 Thread Joe Wildish
Hello Hackers, Reporting a bug with the new MERGE statement. Tested against 75edb919613ee835e7680e40137e494c7856bcf9. psql output as follows: ... psql:merge.sql:33: ERROR: variable not found in subplan target lists ROLLBACK [local] joe@joe=# \errverbose ERROR: XX000: variable not found in sub

Re: should vacuum's first heap pass be read-only?

2022-04-05 Thread Peter Geoghegan
On Tue, Apr 5, 2022 at 2:53 PM Robert Haas wrote: > On Tue, Apr 5, 2022 at 4:30 PM Peter Geoghegan wrote: > > On Tue, Apr 5, 2022 at 1:10 PM Robert Haas wrote: > > > I had assumed that this would not be the case, because if the page is > > > being accessed by the workload, it can be pruned - and

Re: MERGE bug report

2022-04-05 Thread Zhihong Yu
On Tue, Apr 5, 2022 at 3:18 PM Joe Wildish wrote: > Hello Hackers, > > Reporting a bug with the new MERGE statement. Tested against > 75edb919613ee835e7680e40137e494c7856bcf9. > > psql output as follows: > > ... > psql:merge.sql:33: ERROR: variable not found in subplan target lists > ROLLBACK >

Re: MERGE bug report

2022-04-05 Thread Zhihong Yu
On Tue, Apr 5, 2022 at 3:35 PM Zhihong Yu wrote: > > > On Tue, Apr 5, 2022 at 3:18 PM Joe Wildish wrote: > >> Hello Hackers, >> >> Reporting a bug with the new MERGE statement. Tested against >> 75edb919613ee835e7680e40137e494c7856bcf9. >> >> psql output as follows: >> >> ... >> psql:merge.sql:3

Re: Atomic rename feature for Windows.

2022-04-05 Thread Victor Spirin
Hi Updated patch: we use the posix semantic features in Windows build 17763 and up. We found an issue with this feature on Windows Server 2016 without updates (Windows 1607 Build 14393) Victor Spirin Postgres Professional:http://www.postgrespro.com The Russian Postgres Company 05.07.2021 16:

Re: shared-memory based stats collector - v69

2022-04-05 Thread Andres Freund
On 2022-04-05 14:43:49 -0700, David G. Johnston wrote: > On Tue, Apr 5, 2022 at 2:23 PM Andres Freund wrote: > > > > > On 2022-04-05 13:51:12 -0700, David G. Johnston wrote: > > > > >, but rather add to the shared queue > > > > Queue? Maybe you mean the hashtable? > > > > Queue implemented by a

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-05 Thread Thom Brown
On Tue, 5 Apr 2022 at 16:02, Robert Haas wrote: > > On Tue, Apr 5, 2022 at 10:58 AM Magnus Hagander wrote: > >> Makes sense. I will do this soon if nobody objects. > >> > >> I'm mildly uncomfortable with the phrase "WAL records generated over > >> the delay period" because it seems a bit imprecis

Re: Race condition in server-crash testing

2022-04-05 Thread Tom Lane
Andres Freund writes: > On 2022-04-04 00:50:27 -0400, Tom Lane wrote: >> It's hard to be totally sure, but I think what happened is that >> gaur hit the in-hindsight-obvious race condition in this code: >> we managed to execute a successful iteration of poll_query_until >> before the postmaster ha

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-05 Thread Tatsuo Ishii
>> I would suggest to reorder the last chunk to: >> >>... retried retries failures serfail dlfail >> >> because I intend to add connection failures handling at some point, >> and it would make more sense to add the corresponding count at the end >> with other fails. > > Ok, I have adjusted t

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-05 Thread Thom Brown
On Wed, 6 Apr 2022 at 01:42, Thom Brown wrote: > > On Tue, 5 Apr 2022 at 16:02, Robert Haas wrote: > > > > On Tue, Apr 5, 2022 at 10:58 AM Magnus Hagander wrote: > > >> Makes sense. I will do this soon if nobody objects. > > >> > > >> I'm mildly uncomfortable with the phrase "WAL records generat

  1   2   >