Re: Add sub-transaction overflow status in pg_stat_activity

Hi, On 2022-11-15 09:04:25 -0500, Robert Haas wrote: > On Mon, Nov 14, 2022 at 4:17 PM Andres Freund wrote: > > Perhaps this would better be tackled by a new "visibility" view. It could > > show > > - number of sessions with a snapshot > > - max age of backend xmin > > - pid with max backend

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

Hi, On 2022-11-15 23:36:53 +1300, David Rowley wrote: > On Tue, 15 Nov 2022 at 11:11, Andres Freund wrote: > > Couldn't we reduce the amount of over-allocation by a small amount by > > special > > casing the already-aligned case? That's not going to be relevant for page > > size > > aligne

Re: Moving forward with TDE

On Mon, Oct 24, 2022 at 9:29 AM David Christensen wrote: > I would love to open a discussion about how to move forward and get > some of these features built out. The historical threads here are > quite long and complicated; is there a "current state" other than the > wiki that reflects the

Re: meson oddities

Hi, On 2022-11-15 08:04:29 -0500, Andrew Dunstan wrote: > On 2022-11-14 Mo 18:24, Andres Freund wrote: > > Hi, > > > > On 2022-11-14 17:41:54 -0500, Andrew Dunstan wrote: > >> Here's a couple of things I've noticed. > >> > >> > >> andrew@ub22:HEAD $ inst.meson/bin/pg_config --libdir --ldflags >

Re: Distinct tightly coupled with Agg

On 16/11/22 00:26, Tom Lane wrote: Ankit Kumar Pandey writes: I am looking at todo item (#1) /Implement DISTINCT clause in window aggregates/ and while looking at code, I found distinct tightly coupled with Agg function. Looking at another todo item(#2) /Do we really need so much duplicated

Re: Distinct tightly coupled with Agg

Ankit Kumar Pandey writes: > I am looking at todo item (#1) /Implement DISTINCT clause in window > aggregates/ and while looking at code, I found distinct tightly coupled > with Agg function. Looking at another todo item(#2) /Do we really need > so much duplicated code between Agg and

Distinct tightly coupled with Agg

Hello, I am looking at todo item (#1) /Implement DISTINCT clause in window aggregates/ and while looking at code, I found distinct tightly coupled with Agg function. Looking at another todo item(#2) /Do we really need so much duplicated code between Agg and WindowAgg/?  I was wondering what

closing file in adjust_data_dir

Hi, I was looking at the commit: commit 2fe3bdbd691a5d11626308e7d660440be6c210c8 Author: Peter Eisentraut Date: Tue Nov 15 15:35:37 2022 +0100 Check return value of pclose() correctly In src/bin/pg_ctl/pg_ctl.c : if (fd == NULL || fgets(filename, sizeof(filename), fd) == NULL ||

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

On Fri, Nov 11, 2022 at 10:38 AM Peter Geoghegan wrote: > Attached is v4, which removes the old comments you pointed out were > now out of place (they weren't adding much anyway). Also fixed bitrot > against HEAD from today's visibility map commit from Jeff Davis. Pushed something like this

Re: List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

Amit Langote writes: > On Mon, Nov 14, 2022 at 11:57 PM Tom Lane wrote: >> + * The new member is identified by the zero-based index of the List >> + * element it should go into, and the bit number to be set therein. > The comment sounds a bit ambiguous, especially the ", and the bit > number

Standardizing how pg_waldump presents recovery conflict XID cutoffs

Most recovery conflicts are generated in REDO routines using a standard approach these days: they all call ResolveRecoveryConflictWithSnapshot() with a latestRemovedXid argument taken directly from the WAL record. Right now we don't quite present this information in a uniform way, even though REDO

Re: List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

Alvaro Herrera writes: > On 2022-Nov-14, Tom Lane wrote: >> For discussion's sake, here's my current version of that 0004 patch, >> rewritten to use list-of-bitmapset as the data structure. > I feel that there should be more commentary that explains what a > multi-bms is. Not sure where, maybe

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

On Tue, Nov 15, 2022 at 4:41 AM Bharath Rupireddy wrote: > > On Tue, Nov 15, 2022 at 1:29 AM David Christensen > wrote: > > > > Enclosed is v8, which uses the replication slot method to retain WAL > > as well as fsync'ing the output directory when everything is done. > > Thanks. It mostly is in

Re: when the startup process doesn't (logging startup delays)

On Tue, Nov 15, 2022 at 8:33 AM Bharath Rupireddy wrote: > Please review the v2 patch. It seems to me that this will call disable_startup_progress_timeout once per WAL record, which seems like an unnecessary expense. How about leaving the code inside the loop just as we have it, and putting if

Re: archive modules

On 2022-Nov-15, Nathan Bossart wrote: > On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > > The surrounding code has changed a bit between PG15 and master, so if we > > wanted to backpatch this, we'd need another patch from you. However, at > > this point, I'm content to just

Re: allowing for control over SET ROLE

Bump. Discussion has trailed off here, but I still don't see that we have a better way forward here than what I proposed on September 30th. Two people have commented. Nathan said that he wasn't sure this was best (neither am I) but that he didn't have a better idea either (neither do I). Stephen

Re: archive modules

On Tue, Nov 15, 2022 at 10:31:44AM +0100, Peter Eisentraut wrote: > I have committed this to master. Thanks! > The surrounding code has changed a bit between PG15 and master, so if we > wanted to backpatch this, we'd need another patch from you. However, at > this point, I'm content to just

Re: HOT chain validation in verify_heapam()

On Mon, Nov 14, 2022 at 5:02 PM Andres Freund wrote: > On 2022-11-14 14:27:54 -0500, Robert Haas wrote: > > On Wed, Nov 9, 2022 at 5:08 PM Andres Freund wrote: > > > I don't really understand this logic - why can't we populate the > > > predecessor > > > array, if we can construct a successor

Re: Error-safe user functions

On 2022-10-07 Fr 13:37, Tom Lane wrote: [ lots of detailed review ] > Basically, this patch set should be a lot smaller and not have ambitions > beyond "get the API right" and "make one or two datatypes support COPY > NULL_ON_ERROR". Add more code once that core functionality gets reviewed >

Re: meson oddities

On 15.11.22 00:48, Andres Freund wrote: We effectively do that with autoconf as well, except that we don't mention that in pg_config --ldflags. Our linking rules include CFLAGS, see e.g.: %: %.o $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X) postgres: $(OBJS)

Re: Vacuumdb --force-index-cleanup option not available in postgres 12.9

On Tue, Nov 15, 2022 at 03:40:58PM +, Karthik Jagadish (kjagadis) wrote: > Thanks Justin for prompt response. Could you please provide the full email > for pgsql-user? pgsql-u...@postgresql.org > is not working Of course, I intended to say

Re: Vacuumdb --force-index-cleanup option not available in postgres 12.9

Thanks Justin for prompt response. Could you please provide the full email for pgsql-user? pgsql-u...@postgresql.org is not working From: Justin Pryzby Date: Tuesday, 15 November 2022 at 9:02 PM To: Karthik Jagadish (kjagadis) Cc: pgsql-hack...@postgresql.org

Index not getting cleaned even though vacuum is running

Hi, We notice that vacuum is happening at regular intervals but the space occupied by indexes is always increasing. Any pointers as to why would this happen? Some outputs below. Auto vacuum is enabled but we notice index size is growing. $ psql -U postgres -d cgms -c "SELECT

Re: dynamic result sets support in extended query protocol

On 14.10.22 19:22, Pavel Stehule wrote: 1. there can be possibility to set "dynamic result sets" to unknown. The behaviour of the "dynamic result sets" option is a little bit confusing. I expect the number of result sets should be exactly the same as this number. But the warning is raised only

Re: Check return value of pclose() correctly

On 02.11.22 16:26, Ankit Kumar Pandey wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi Peter,

Re: Avoid overhead open-close indexes (catalog updates)

Em ter., 15 de nov. de 2022 às 04:02, Michael Paquier escreveu: > On Tue, Nov 15, 2022 at 09:57:26AM +0900, Michael Paquier wrote: > > Anyway, multi-inserts are going to be solution better than > > CatalogTupleInsertWithInfo() in some cases, because we would just > > generate one WAL record of N

Re: Meson doesn't define HAVE_LOCALE_T for mscv

On Tue, Nov 15, 2022 at 1:49 AM Andres Freund wrote: > > Hm. Is it right that the changes are only done for msvc? win32_port.h > defines > the types for mingw as well afaict. > > Yes, it does, but configure does nothing with them, so adding those defines is a new feature for MinGW but a

Re: when the startup process doesn't (logging startup delays)

On Tue, 15 Nov 2022 at 13:33, Bharath Rupireddy wrote: > > On Mon, Nov 14, 2022 at 9:31 PM Robert Haas wrote: > > > > On Mon, Nov 14, 2022 at 7:37 AM Simon Riggs > > wrote: > > > > Whilte at it, I noticed that we report redo progress for PITR, but we > > > > don't report when standby enters

Re: Add sub-transaction overflow status in pg_stat_activity

On Mon, Nov 14, 2022 at 4:17 PM Andres Freund wrote: > Perhaps this would better be tackled by a new "visibility" view. It could show > - number of sessions with a snapshot > - max age of backend xmin > - pid with max backend xmin > - number of sessions that suboverflowed > - pid of the session

Re: plpgsql-trigger.html: Format TG_ variables as table (patch)

> On 2 Sep 2022, at 11:19, Daniel Gustafsson wrote: > The patch missed to update the corresponding list for TG_ event trigger vars, > fixed in the attached. I took another look at this, and pushed it with a few small tweaks. Thanks! -- Daniel Gustafsson https://vmware.com/

Re: when the startup process doesn't (logging startup delays)

On Mon, Nov 14, 2022 at 9:31 PM Robert Haas wrote: > > On Mon, Nov 14, 2022 at 7:37 AM Simon Riggs > wrote: > > > Whilte at it, I noticed that we report redo progress for PITR, but we > > > don't report when standby enters archive recovery mode, say due to a > > > failure in the connection to

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

On Tue, 15 Nov 2022 at 17:34, Simon Riggs wrote: > On Mon, 7 Nov 2022 at 21:14, Simon Riggs wrote: > >> These results are compelling, thank you. >> >> Setting this to Ready for Committer. > > New version attached. Take a quick look, I think it should be PGPROC instead of PG_PROC, right? +

Re: psql: Add command to use extended query protocol

On 09.11.22 00:12, Corey Huinker wrote: As for the docs, they're very clear and probably sufficient as-is, but I wonder if we should we explicitly state that the bind-state and bind parameters do not "stay around" after the query is executed? Suggestions in bold:          This command

Re: meson oddities

On 2022-11-14 Mo 18:24, Andres Freund wrote: > Hi, > > On 2022-11-14 17:41:54 -0500, Andrew Dunstan wrote: >> Here's a couple of things I've noticed. >> >> >> andrew@ub22:HEAD $ inst.meson/bin/pg_config --libdir --ldflags >> /home/andrew/pgl/pg_head/root/HEAD/inst.meson/lib/x86_64-linux-gnu >>

Re: Error on missing Python module in Meson setup

> On 15 Nov 2022, at 01:25, Andres Freund wrote: > On 2022-11-14 14:23:02 +0100, Daniel Gustafsson wrote >> Installing the distutils package fixes it, but it seems harsh to fail setup >> on >> a missing package. Would something like the attached make sense? > > The error is a bit better in

Re: Assertion failure in SnapBuildInitialSnapshot()

On Tue, Nov 15, 2022 at 6:55 AM Andres Freund wrote: > > On 2022-11-10 16:04:40 +0530, Amit Kapila wrote: > > I don't have any good ideas on how to proceed with this. Any thoughts > > on this would be helpful? > > One thing worth doing might be to convert the assertion path into an elog(), >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

> v6 was not applying anymore, due to a change in > doc/src/sgml/ref/checkpoint.sgml done by b9eb0ff09e (Rename > pg_checkpointer predefined role to pg_checkpoint). > > Please find attached a rebase in v7. > > While working on this rebase, I also noticed that "pg_checkpointer" is > still mentioned

Re: Add non-blocking version of PQcancel

Ugh, it indeed seems like I somehow messed up sending the new patch. Here's the correct one. 0001-Add-non-blocking-version-of-PQcancel.patch Description: 0001-Add-non-blocking-version-of-PQcancel.patch

Re: Assertion failure in SnapBuildInitialSnapshot()

On Tue, Nov 15, 2022 at 8:08 AM Andres Freund wrote: > > On 2022-11-14 17:25:31 -0800, Andres Freund wrote: > > Hm, also, shouldn't the patch adding CRS_USE_SNAPSHOT have copied more of > > SnapBuildExportSnapshot()? Why aren't the error checks for > > SnapBuildExportSnapshot() needed? Why don't

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

On Mon, 14 Nov 2022 at 15:25, John Naylor wrote: > - Assert((char *) chunk > (char *) block); > + Assert((char *) chunk >= (char *) block); > > Is this related or independent? It's related. Because the code is doing: MemoryChunkSetHdrMask(alignedchunk, unaligned, alignto,

Re: Unit tests for SLRU

Hi, Alexander! > > I have reworked that as per the attached, that provides basically the > > same coverage, going through a SQL interface for the whole thing. > > Like all the other tests of its kind, this does not use a TAP test, > > relying on a custom configuration file instead. This still

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

On Tue, Nov 15, 2022 at 1:29 AM David Christensen wrote: > > Enclosed is v8, which uses the replication slot method to retain WAL > as well as fsync'ing the output directory when everything is done. Thanks. It mostly is in good shape. However, few more comments: 1. +if it does not

Re: Unit tests for SLRU

> On 15 Nov 2022, at 11:15, Aleksander Alekseev > wrote: >> What do you think? > > It looks much better than before. I replaced strcpy() with strncpy() > and pgindent'ed the code. + /* write given data to the page */ + strncpy(TestSlruCtl->shared->page_buffer[slotno], data, BLCKSZ

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

On Tue, 15 Nov 2022 at 11:11, Andres Freund wrote: > Couldn't we reduce the amount of over-allocation by a small amount by special > casing the already-aligned case? That's not going to be relevant for page size > aligne allocations, but for smaller alignment values it could matter. I don't

Re: Non-decimal integer literals

On 14.11.22 08:25, John Naylor wrote: Regarding the patch, it looks good overall. My only suggestion would be to add a regression test for just below and just above overflow, at least for int2. ok Minor nits: - * Process {integer}.  Note this will also do the right thing with {decimal}, +

Re: libpq support for NegotiateProtocolVersion

On 14.11.22 19:11, Jacob Champion wrote: If we want to address this, maybe this should be handled in the polling loop before we pass off the input buffer to the per-message-type handlers. I thought it was supposed to be handled by this code: /* * Can't process if message

Re: New docs chapter on Transaction Management and related changes

On Tue, 8 Nov 2022 at 03:41, Bruce Momjian wrote: > > On Mon, Nov 7, 2022 at 10:58:05AM +, Simon Riggs wrote: > > What I've posted is the merged patch, i.e. your latest patch, plus > > changes to RELEASE SAVEPOINT from you on Oct 16, plus changes based on > > the later comments from Robert

Re: Unit tests for SLRU

Hi Michael, > I have reworked that as per the attached, that provides basically the > same coverage, going through a SQL interface for the whole thing. > Like all the other tests of its kind, this does not use a TAP test, > relying on a custom configuration file instead. This still needs some >

Re: pg_basebackup's --gzip switch misbehaves

> On 15 Nov 2022, at 00:58, Michael Paquier wrote: > > On Mon, Nov 14, 2022 at 03:27:14PM +0100, Daniel Gustafsson wrote: >> Ugh, yes, that's what it should say. > > A split sounds fine by me. On top of what Tom has mentioned, I have > spotted two small-ish things. > > -This module is

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

On Fri, Nov 11, 2022 at 5:52 PM Maxim Orlov wrote: > > Hi! > > I've watched over the patch and consider it useful. Applies without > conflicts. The functionality of the patch itself is > meets declared functionality. Thanks for reviewing. > But, in my view, some improvements may be proposed.

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

On Mon, 7 Nov 2022 at 21:14, Simon Riggs wrote: > These results are compelling, thank you. > > Setting this to Ready for Committer. New version attached. -- Simon Riggshttp://www.EnterpriseDB.com/ 002_minimize_calls_to_SubTransSetParent.v12.patch Description: Binary data

Re: archive modules

On 05.11.22 21:51, Nathan Bossart wrote: On Fri, Nov 04, 2022 at 12:05:26PM +0900, Ian Lawrence Barwick wrote: cfbot reports the patch no longer applies [1]. As CommitFest 2022-11 is currently underway, this would be an excellent time to update the patch. Indeed. Here is a new version of

Re: Allow single table VACUUM in transaction block

On Mon, 14 Nov 2022 at 19:52, Simon Riggs wrote: > > On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs > > wrote: > > > > > Temp tables are actually easier, since we don't need any of the > > > concurrency features we get with lazy vacuum. > > >

Re: List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

On 2022-Nov-14, Tom Lane wrote: > For discussion's sake, here's my current version of that 0004 patch, > rewritten to use list-of-bitmapset as the data structure. I feel that there should be more commentary that explains what a multi-bms is. Not sure where, maybe just put it near the function

Re: Synchronizing slots from primary to standby

Hi, On 2/11/22 3:26 PM, Peter Eisentraut wrote: On 10.02.22 22:47, Bruce Momjian wrote: On Tue, Feb  8, 2022 at 08:27:32PM +0530, Ashutosh Sharma wrote: Which means that if e.g. the standby_slot_names GUC differs from synchronize_slot_names on the physical replica, the slots synchronized on

Re: Making Vars outer-join aware

On Sun, Nov 6, 2022 at 5:53 AM Tom Lane wrote: > I wrote: > > I've been working away at this patch series, and here is an up-to-date > > version. > > This needs a rebase after ff8fa0bf7 and b0b72c64a. I also re-ordered > the patches so that the commit messages' claims about when regression >

<    1   2