Re: Add non-blocking version of PQcancel

2022-03-30 Thread Justin Pryzby
Note that the patch is still variously failing in cirrus. https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/37/3511 You may already know that it's possible to trigger the cirrus ci tasks using a github branch. See src/tools/ci/README.

Re: In-placre persistance change of a relation

2022-03-30 Thread Justin Pryzby
On Thu, Mar 31, 2022 at 01:58:45PM +0900, Kyotaro Horiguchi wrote: > Thanks! Version 20 is attached. The patch failed an all CI tasks, and seems to have caused the macos task to hang. http://cfbot.cputube.org/kyotaro-horiguchi.html Would you send a fixed patch, or remove this thread from the

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-03-30 Thread Andres Freund
On 2022-03-31 01:00:14 -0400, Tom Lane wrote: > How well does this patch work with pre-14 buildfarm clients? Looks to me like it'll just run the test twice, once via TestUpgrade, once via taptest. It's possible that there could be trouble somehow due to duplicated log files or something?

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-03-30 Thread Tom Lane
I wrote: > There's still about a third of the buildfarm running older > client releases --- I count > 2 REL_8 > 2 REL_10 > 13 REL_11 > 6 REL_12 > 16 REL_13.1 > 89 REL_14 Wait a minute ... actually, what's most relevant here is the population running TAP tests,

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-31 00:08:00 -0400, Tom Lane wrote: > Andres Freund writes: > > I assume SSPI doesn't work over unix sockets? Oh. Maybe it's not even that - > > we only enable it when not using unix sockets: > > Duh. But can it work over unix sockets? I wonder if we should go the other way and

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-03-30 Thread Tom Lane
Michael Paquier writes: > So, any particular feelings about this patch? This has been around > for a couple of months/years now, so it could be a good time to do the > switch now rather than wait an extra year, or even the beginning of > the next release cycle. And the buildfarm is already able

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 21:29:16 -0700, Peter Geoghegan wrote: > On Wed, Mar 30, 2022 at 9:20 PM Andres Freund wrote: > > But the debug elog reports that > > > > relfrozenxid updated 714 -> 717 > > relminmxid updated 1 -> 6 > > > > Tthe problem is that the crashing backend reads the

Re: In-placre persistance change of a relation

2022-03-30 Thread Kyotaro Horiguchi
Thanks! Version 20 is attached. At Wed, 30 Mar 2022 08:44:02 -0500, Justin Pryzby wrote in > On Tue, Mar 01, 2022 at 02:14:13PM +0900, Kyotaro Horiguchi wrote: > > Rebased on a recent xlog refactoring. > > It'll come as no surprise that this neds to be rebased again. > At least a few typos I

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 9:29 PM Peter Geoghegan wrote: > > Perhaps we should just fetch the horizons from the "local" catalog for > > shared > > rels? > > Not sure what you mean. Wait, you mean use vacrel->relfrozenxid directly? Seems kind of ugly... -- Peter Geoghegan

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 9:20 PM Andres Freund wrote: > But the debug elog reports that > > relfrozenxid updated 714 -> 717 > relminmxid updated 1 -> 6 > > Tthe problem is that the crashing backend reads the relfrozenxid/relminmxid > from the shared relcache init file written by another backend:

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 21:11:48 -0700, Peter Geoghegan wrote: > On Wed, Mar 30, 2022 at 9:04 PM Andres Freund wrote: > > (gdb) p vacrel->NewRelfrozenXid > > $3 = 717 > > (gdb) p vacrel->relfrozenxid > > $4 = 717 > > (gdb) p OldestXmin > > $5 = 5112 > > (gdb) p aggressive > > $6 = false > > Does this

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 21:04:07 -0700, Andres Freund wrote: > On 2022-03-30 20:35:25 -0700, Peter Geoghegan wrote: > > On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote: > > > I triggered twice now, but it took a while longer the second time. > > > > Great. > > > > I wonder if you can get an RR

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-30 Thread Dilip Kumar
On Thu, Mar 31, 2022 at 5:07 AM Andres Freund wrote: > > Hi, > > On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > > I committed v6 instead. > > I was just discussing the WAL prefetching patch with Thomas. A question in > that discussion made me look at the coverage of REDO for CREATE DATABASE:

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 9:04 PM Andres Freund wrote: > (gdb) p vacrel->NewRelfrozenXid > $3 = 717 > (gdb) p vacrel->relfrozenxid > $4 = 717 > (gdb) p OldestXmin > $5 = 5112 > (gdb) p aggressive > $6 = false Does this OldestXmin seem reasonable at this point in execution, based on context? Does

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-03-30 Thread Michael Paquier
On Thu, Mar 03, 2022 at 02:03:38PM +0900, Michael Paquier wrote: > Indeed. I have reworked the whole, rather than just those three > sentences. So, any particular feelings about this patch? This has been around for a couple of months/years now, so it could be a good time to do the switch now

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Andres Freund writes: > On 2022-03-30 22:07:48 -0400, Tom Lane wrote: >> So ... none of the Windows buildfarm members actually like this >> test script. > On windows CI sets > # Avoids port conflicts between concurrent tap test runs > PG_TEST_USE_UNIX_SOCKETS: 1 Ok ... > I assume SSPI

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 20:35:25 -0700, Peter Geoghegan wrote: > On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote: > > I triggered twice now, but it took a while longer the second time. > > Great. > > I wonder if you can get an RR recording... Started it, but looks like it's too slow. (gdb) p

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-30 Thread Michael Paquier
On Wed, Mar 30, 2022 at 08:23:25PM +1300, Thomas Munro wrote: > That leads to the attached patches, the first of which I'd want to back-patch. Makes sense. - if ((fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0) - d->ret.d_type = DT_DIR; /* For reparse points dwReserved0 field

Re: Handle infinite recursion in logical replication setup

2022-03-30 Thread Amit Kapila
On Wed, Mar 30, 2022 at 7:40 PM Ashutosh Bapat wrote: > > > > The changes for the same are available int the v5 patch available at [1]. > > [1] - > > https://www.postgresql.org/message-id/CALDaNm3wCf0YcvVo%2BgHMGpupk9K6WKJxCyLUvhPC2GkPKRZUWA%40mail.gmail.com > > > > cb->truncate_cb =

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 20:28:44 -0700, Andres Freund wrote: > I was able to trigger the crash. > > cat ~/tmp/pgbench-createdb.sql > CREATE DATABASE pgb_:client_id; > DROP DATABASE pgb_:client_id; > > pgbench -n -P1 -c 10 -j10 -T100 -f ~/tmp/pgbench-createdb.sql > > while I was also running > > for

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote: > I triggered twice now, but it took a while longer the second time. Great. I wonder if you can get an RR recording... -- Peter Geoghegan

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, I was able to trigger the crash. cat ~/tmp/pgbench-createdb.sql CREATE DATABASE pgb_:client_id; DROP DATABASE pgb_:client_id; pgbench -n -P1 -c 10 -j10 -T100 -f ~/tmp/pgbench-createdb.sql while I was also running for i in $(seq 1 100); do echo iteration $i; make -Otarget -C contrib/ -s

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 7:37 PM Peter Geoghegan wrote: > Yeah, a WARNING would be good here. I can write a new version of my > patch series with a separation patch for that this evening. Actually, > better make it a PANIC for now... Attached is v14, which includes a new patch that PANICs like

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 7:00 PM Andres Freund wrote: > Something vaguely like EXPECT_EQ_U32 in regress.c. Maybe > AssertCmp(type, a, op, b), > > Then the assertion could have been something like >AssertCmp(int32, diff, >, 0) I'd definitely use them if they were there. > Does the line number

Re: Add LZ4 compression in pg_dump

2022-03-30 Thread Michael Paquier
On Wed, Mar 30, 2022 at 03:32:55PM +, gkokola...@pm.me wrote: > On Wednesday, March 30th, 2022 at 7:54 AM, Michael Paquier > wrote: >> While moving on with 0002, I have noticed the following in >> >> _StartBlob(): >> if (AH->compression != 0) >> sfx = ".gz"; >> else >>

RE: Logical replication timeout problem

2022-03-30 Thread shiy.f...@fujitsu.com
On Wed, Mar 30, 2022 3:54 PM wangw.f...@fujitsu.com wrote: > > Rebase the patch because the commit d5a9d86d in current HEAD. > Thanks for your patch, I tried this patch and confirmed that there is no timeout problem after applying this patch, and I could reproduce this problem on HEAD.

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 22:07:48 -0400, Tom Lane wrote: > So ... none of the Windows buildfarm members actually like this > test script. They're all showing failures along the lines of > > not ok 2 - fails if basebackup_to_shell.command is not set: matches > > # Failed test 'fails if

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
So ... none of the Windows buildfarm members actually like this test script. They're all showing failures along the lines of not ok 2 - fails if basebackup_to_shell.command is not set: matches # Failed test 'fails if basebackup_to_shell.command is not set: matches' # at t/001_basic.pl line

Re: Commitfest Update

2022-03-30 Thread Julien Rouhaud
On Wed, Mar 30, 2022 at 02:41:26PM -0400, Greg Stark wrote: > > Patches that are Waiting on Author and haven't had activity in months > -- traditionally they were set to Returned with Feedback. It seems the > feeling these days is to not lose state on them and just move them to > the next CF. I'm

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 17:50:42 -0700, Peter Geoghegan wrote: > I tried several times to recreate this issue on CI. No luck with that, > though -- can't get it to fail again after 4 attempts. It's really annoying that we don't have Assert variants that show the compared values, that might make it

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-30 Thread Peter Geoghegan
On Wed, Mar 30, 2022 at 12:01 AM Peter Geoghegan wrote: > Perhaps something is amiss inside vac_update_relstats(), where the > boolean flag that indicates that pg_class.relfrozenxid was advanced is > set: > > if (frozenxid_updated) > *frozenxid_updated = false; > if

Re: Mingw task for Cirrus CI

2022-03-30 Thread Andres Freund
Hi, On 2022-03-22 19:00:42 +0300, Melih Mutlu wrote: > Rebased it. > I also removed the temp installation task and > used NoDefaultCurrentDirectoryInExePath env variable instead. Hm. But you're still using a separate build directory, from what I can see? The NoDefaultCurrentDirectoryInExePath

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 16:35:50 -0700, Andres Freund wrote: > On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > > Separate from the minutia in [1] I'd like to discuss a few questions of more > > general interest. I'll post another question or two later. > > 4) What to do with the

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Nathan Bossart writes: > I think we should give this module a .gitignore file. Patch attached. Indeed, before somebody accidentally commits the cruft that check-world is leaving around. Pushed. regards, tom lane

Re: ORDER BY pushdowns seem broken in postgres_fdw

2022-03-30 Thread Tom Lane
Ronan Dunklau writes: > [ v8-0001-Fix-orderby-handling-in-postgres_fdw.patch ] I looked through this patch. It's going in the right direction, but I have a couple of nitpicks: 1. There are still some more places that aren't checking shippability of the relevant opfamily. 2. The existing usage

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-30 Thread Andres Freund
Hi, On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > I committed v6 instead. I was just discussing the WAL prefetching patch with Thomas. A question in that discussion made me look at the coverage of REDO for CREATE DATABASE:

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > Separate from the minutia in [1] I'd like to discuss a few questions of more > general interest. I'll post another question or two later. 4) What to do with the stats_temp_directory GUC / PG_STAT_TMP_DIR define / pg_stats_temp

Re: POC: GROUP BY optimization

2022-03-30 Thread Tomas Vondra
Pushed, after going through the patch once more, running check-world under valgrind, and updating the commit message. Thanks to everyone who reviewed/tested this patch! -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Nathan Bossart
I think we should give this module a .gitignore file. Patch attached. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 7466f18b7cb781f4db4919faf15b1b1d3cd7bc7a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 30 Mar 2022 15:28:38 -0700 Subject: [PATCH 1/1] add

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 4:22 PM Tom Lane wrote: > Robert Haas writes: > > I'm going to go ahead and commit this test script later on this > > afternoon unless there are vigorous objections real soon now, and then > > if somebody wants to improve it, great! > > I see you did that, but the CF

Re: Correct docs re: rewriting indexes when table rewrite is skipped

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 4:33 PM James Coleman wrote: > Hmm, having it match the way it works makes sense. Would you feel > comfortable with an intermediate step (queueing up that as a larger > change) changing the clause to something like "indexes will still have > to be rebuilt unless the system

Re: [Proposal] vacuumdb --schema only

2022-03-30 Thread Nathan Bossart
I took a look at the v4 patch. 'git-apply' complains about whitespace errors: 0001-vacuumdb-schema-only-v4.patch:17: tab in indent. 0001-vacuumdb-schema-only-v4.patch:18: tab in indent. 0001-vacuumdb-schema-only-v4.patch:19: tab in

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 14:42:23 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > > I think it's reasonably rare because in cases there'd be corruption, we'd > > typically not even have written them out / throw them away explicitly - we > > only read stats when

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 1:39 PM Andres Freund wrote: > Hi, > > On 2022-03-30 12:29:51 -0700, David G. Johnston wrote: > > On Wednesday, March 30, 2022, Andres Freund wrote: > > > My current proposal is to just have two reset times. One for the > contents > > > of > > > pg_stat_database (i.e.

Re: CLUSTER on partitioned index

2022-03-30 Thread Alvaro Herrera
On 2022-Feb-23, Justin Pryzby wrote: > I hope that Alvaro will comment on the simplified patches. If multiple people > think the patch isn't worth it, feel free to close it. But I don't see how > complexity could be the reason. I gave your patch a look and it seems a reasonable thing to do.

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 21:44:20 +0200, Peter Eisentraut wrote: > On 29.03.22 23:01, Andres Freund wrote: > > > I think what's actually most important here is the error reporting. We > > > need > > > to make it clear, at least via log messages, that something bad has > > > happened. > > The message

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 12:29:51 -0700, David G. Johnston wrote: > On Wednesday, March 30, 2022, Andres Freund wrote: > > My current proposal is to just have two reset times. One for the contents > > of > > pg_stat_database (i.e. not affected by pg_stat_reset_single_*_counters()), > > and > > one for

Re: Correct docs re: rewriting indexes when table rewrite is skipped

2022-03-30 Thread James Coleman
On Wed, Mar 30, 2022 at 11:41 AM Robert Haas wrote: > > On Wed, Mar 30, 2022 at 10:04 AM James Coleman wrote: > > Admittedly I hadn't thought of that case. But isn't it already covered > > in the existing docs by the phrase "or an unconstrained domain over > > the new type"? I don't love the

Re: refactoring basebackup.c (zstd workers)

2022-03-30 Thread Justin Pryzby
On Wed, Mar 30, 2022 at 04:14:47PM -0400, Tom Lane wrote: > Robert Haas writes: > >> Maybe if I just put that last sentence into the comment it's clear enough? > > > Done that way, since I thought it was better to fix the bug than wait > > for more feedback on the wording. We can still adjust

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Robert Haas writes: > I'm going to go ahead and commit this test script later on this > afternoon unless there are vigorous objections real soon now, and then > if somebody wants to improve it, great! I see you did that, but the CF entry is still open? regards, tom lane

Re: refactoring basebackup.c (zstd workers)

2022-03-30 Thread Tom Lane
Robert Haas writes: >> Maybe if I just put that last sentence into the comment it's clear enough? > Done that way, since I thought it was better to fix the bug than wait > for more feedback on the wording. We can still adjust the wording, or > the coding, if it's not clear enough. FWIW, I

Re: refactoring basebackup.c (zstd workers)

2022-03-30 Thread Robert Haas
On Tue, Mar 29, 2022 at 8:51 AM Robert Haas wrote: > On Mon, Mar 28, 2022 at 8:11 PM Tom Lane wrote: > > I suspect Robert wrote it that way intentionally --- but if so, > > I agree it could do with more than zero commentary. > > Well, the point is, we stop advancing kwend when we get to the end

Re: Higher level questions around shared memory stats

2022-03-30 Thread Peter Eisentraut
On 29.03.22 23:01, Andres Freund wrote: I think what's actually most important here is the error reporting. We need to make it clear, at least via log messages, that something bad has happened. The message currently (on HEAD, but similarly on the path) is:

Re: Commitfest Update

2022-03-30 Thread Peter Eisentraut
On 30.03.22 20:41, Greg Stark wrote: Patches that are Waiting on Author and haven't had activity in months -- traditionally they were set to Returned with Feedback. It seems the feeling these days is to not lose state on them and just move them to the next CF. I'm not sure that's wise, it ends

Re: Possible fails in pg_stat_statements test

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 2:20 AM Anton A. Melnikov wrote: > > Can the test failures be encountered without such an elaborate setup? If > > not, > > then I don't really see why we need to do anything here? > > There was a real bug report from our test department. They do long time > repetitive

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread David G. Johnston
On Wednesday, March 30, 2022, Andres Freund wrote: > > My current proposal is to just have two reset times. One for the contents > of > pg_stat_database (i.e. not affected by pg_stat_reset_single_*_counters()), > and > one for stats within the entire database. > > What IS it affected by? And

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 14:57:25 -0400, Robert Haas wrote: > On Wed, Mar 23, 2022 at 8:55 PM Andres Freund wrote: > > This behaviour can be trivially (and is) implemented for the shared memory > > stats patch. But every time I read over that part of the code it feels just > > profoundly wrong to me.

Re: explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-03-30 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 03:07:20PM -0600, Justin Pryzby wrote: > Rebased over ebf6c5249b7db525e59563fb149642665c88f747. > It looks like that patch handles only query_id, and this patch also tries to > handle a bunch of other stuff. > > If it's helpful, feel free to kick this patch to a future CF.

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Andres Freund writes: > On 2022-03-30 13:16:47 -0400, Robert Haas wrote: >> On Wed, Mar 30, 2022 at 1:11 PM Andres Freund wrote: >>> My concern is about the quote in the middle of the path, not about quoting >>> at all... I.e. the ' should be after /tmp_install, not before. >> Makes no

Re: Patch to avoid orphaned dependencies

2022-03-30 Thread Nathan Bossart
On Wed, Mar 23, 2022 at 12:49:06PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Bertand, do you think this has any chance of making it into v15? If not, >> are you alright with adjusting the commitfest entry to v16 and moving it to >> the next commitfest? > > I looked this over briefly,

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 13:16:47 -0400, Robert Haas wrote: > On Wed, Mar 30, 2022 at 1:11 PM Andres Freund wrote: > > On March 30, 2022 9:58:26 AM PDT, Tom Lane wrote: > > >Andres Freund writes: > > >> Random aside: Am I the only one bothered by a bunch of places in > > >> Makefile.global.in quoting

Re: proposal: enhancing plpgsql debug API - returns text value of variable content

2022-03-30 Thread Tom Lane
Greg Stark writes: > It looks like this is -- like a lot of plpgsql patches -- having > difficulty catching the attention of reviewers and committers. I was hoping that someone with more familiarity with pldebugger would comment on the suitableness of this patch for their desires. But nobody's

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread Robert Haas
On Wed, Mar 23, 2022 at 8:55 PM Andres Freund wrote: > This behaviour can be trivially (and is) implemented for the shared memory > stats patch. But every time I read over that part of the code it feels just > profoundly wrong to me. Way worse than *not* resetting > pg_stat_database.stats_reset.

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-03-30 Thread Greg Stark
It looks like this patch received a review from Andres and hasn't been updated since. I'm not sure but the review looks to me like it's not ready to commit and needs some cleanup or at least to check on a few things. I guess it's not going to get bumped in the next few days so I'll move it to the

Re: Commitfest Update

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 2:42 PM Greg Stark wrote: > Patches that are Waiting on Author and haven't had activity in months > -- traditionally they were set to Returned with Feedback. It seems the > feeling these days is to not lose state on them and just move them to > the next CF. I'm not sure

Re: range_agg with multirange inputs

2022-03-30 Thread Peter Eisentraut
This patch has been committed. I split it into a few pieces. On 12.03.22 04:18, Paul Jungwirth wrote: On 3/10/22 14:07, Chapman Flack wrote: When I apply this patch, I get a func.sgml with two entries for range_intersect_agg(anymultirange). Arg, fixed. In range_agg_transfn, you've changed

Re: Higher level questions around shared memory stats

2022-03-30 Thread Robert Haas
On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > I think it's reasonably rare because in cases there'd be corruption, we'd > typically not even have written them out / throw them away explicitly - we > only read stats when starting without crash recovery. > > So the "expected" case of

Commitfest Update

2022-03-30 Thread Greg Stark
Since the previous update 30 additional patches have been committed from the CF. This leaves us with 120 Needs Review and 20 Ready for Committer. There's only a few days left until the end of the month. * Add comment about startup process getting a free procState array slot always * Consistent

Re: Mark all GUC variable as PGDLLIMPORT

2022-03-30 Thread Robert Haas
On Fri, Feb 18, 2022 at 7:02 PM Andres Freund wrote: > On 2022-02-15 08:06:58 -0800, Andres Freund wrote: > > The more I think about it the more I'm convinced that if we want to do this, > > we should do it for variables and functions. > > Btw, if we were to do this, we should just use

Re: Logical insert/update/delete WAL records for custom table AMs

2022-03-30 Thread Jeff Davis
On Thu, 2022-02-24 at 20:35 +, Simon Riggs wrote: > The approach is perfectly fine, it just needs to be finished and > rebased. Attached a new version. The scope expanded, so this is likely to slip v15 at this late time. For 15, I'll focus on my extensible rmgr work, which can serve similar

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Thomas Munro
On Thu, Mar 31, 2022 at 5:23 AM Andres Freund wrote: > On 2022-03-26 13:51:35 -0700, Andres Freund wrote: > > Prototype patch attached. > > Because I forgot about cfbot when attaching the patch, cfbot actually ran with > it under this thread's cf entry. It does look like an improvement to me: >

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-03-30 Thread Greg Stark
This patch is now failing in the sqljson regression test. It looks like it's just the ordering of the elements in json_arrayagg() calls which may actually be a faulty test that needs more ORDER BY clauses rather than any issues with the code. Nonetheless it's something that needs to be addressed

Re: real/float example for testlibpq3

2022-03-30 Thread Greg Stark
On Mon, 28 Feb 2022 at 17:50, Tom Lane wrote: > > Chapman Flack writes: > > In the current state of affairs, what's considered the ur-source of that > > information? > > The source code for the type's send/receive functions :-(. One could > wish for something better, but no one has stepped up

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 1:11 PM Andres Freund wrote: > On March 30, 2022 9:58:26 AM PDT, Tom Lane wrote: > >Andres Freund writes: > >> Random aside: Am I the only one bothered by a bunch of places in > >> Makefile.global.in quoting like > >> $(MAKE) -C '$(top_builddir)'

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On March 30, 2022 9:58:26 AM PDT, Tom Lane wrote: >Andres Freund writes: >> Random aside: Am I the only one bothered by a bunch of places in >> Makefile.global.in quoting like >> $(MAKE) -C '$(top_builddir)' DESTDIR='$(abs_top_builddir)'/tmp_install >> install

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 12:54 PM Andres Freund wrote: > There are some commandline utilities (including copy) where backward slashes > in arguments are necessary, to separate options from paths :/. Those are the > extent of backslash use in Cluster.pm that I could see quickly. I just copied this

Re: multithreaded zstd backup compression for client and server

2022-03-30 Thread Dagfinn Ilmari Mannsåker
Robert Haas writes: > On Wed, Mar 30, 2022 at 8:00 AM Dagfinn Ilmari Mannsåker > wrote: >> Robert Haas writes: >> > This patch contains a trivial adjustment to >> > PostgreSQL::Test::Cluster::run_log to make it return a useful value >> > instead of not. I think that should be pulled out and

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Andres Freund writes: > On 2022-03-30 12:34:34 -0400, Tom Lane wrote: >> One refinement that comes to mind as I look at the patch is to distinguish >> between "check" and "installcheck". Not sure that's worthwhile, but not >> sure it isn't, either. > As it's just about "free" to do so, I see no

Re: basebackup/lz4 crash

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 10:35 AM Justin Pryzby wrote: > It looks like maybe this code was copied from > bbstreamer_lz4_compressor_finalize() which has an Assert rather than expanding > the buffer like in bbstreamer_lz4_compressor_new() Hmm, this isn't great. On the server side, we set up a chain

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 12:42:50 -0400, Robert Haas wrote: > On Wed, Mar 30, 2022 at 12:30 PM Andres Freund wrote: > > # Reconfigure to restrict access and require a detail. > > $shell_command = > > $PostgreSQL::Test::Utils::windows_os > > ? qq{$gzip --fast >

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 12:34:34 -0400, Tom Lane wrote: > Andres Freund writes: > > Unless somebody speaks in favor of doing this across branches, I'd just go > > for > > HEAD. > > +1 for HEAD only, especially if we think we might change it some more > later. It seems possible this might break

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Robert Haas
On Wed, Mar 30, 2022 at 12:30 PM Andres Freund wrote: > # Reconfigure to restrict access and require a detail. > $shell_command = > $PostgreSQL::Test::Utils::windows_os > ? qq{$gzip --fast > "$escaped_backup_path%d.%f.gz"} > : qq{$gzip --fast >

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Tom Lane
Andres Freund writes: > Because I forgot about cfbot when attaching the patch, cfbot actually ran with > it under this thread's cf entry. It does look like an improvement to me: > https://cirrus-ci.com/task/6397292629458944?logs=test_world#L900 > We certainly can do better, but it's sufficiently

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 09:28:58 +0530, Dilip Kumar wrote: > On Wed, Mar 30, 2022 at 6:47 AM Andres Freund wrote: > > > > > > du -s /tmp/initdb/ > > WAL_LOG: 35112 > > FILE_COPY: 29288 > > > > So it seems we should specify a strategy in initdb? It kind of makes sense - > > we're not going to read

Re: make MaxBackends available in _PG_init

2022-03-30 Thread Nathan Bossart
On Tue, Mar 29, 2022 at 12:22:21PM -0400, Robert Haas wrote: > It's not, though, because the original proposal was to change things > around so that the value of MaxBackends would have been reliable in > _PG_init(). If we'd done that, then extensions that are using it in > _PG_init() would have

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
On 2022-03-30 08:53:43 -0400, Robert Haas wrote: > Here's a new series, adjusted to use 'gzip' instead of 'cat' and 'type'. Seems to have done the trick: https://cirrus-ci.com/task/6474955838717952?logs=test_contrib_basebackup_to_shell#L1 # Reconfigure to restrict access and require a detail.

Re: Identify missing publications from publisher while create/alter subscription.

2022-03-30 Thread vignesh C
On Wed, Mar 30, 2022 at 5:42 PM Bharath Rupireddy wrote: > > On Wed, Mar 30, 2022 at 5:37 PM Bharath Rupireddy > wrote: > > > > On Wed, Mar 30, 2022 at 4:29 PM Amit Kapila wrote: > > > > > > On Wed, Mar 30, 2022 at 12:22 PM vignesh C wrote: > > > > > > > > I have made the changes for this,

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-30 Thread Andres Freund
Hi, On 2022-03-26 13:51:35 -0700, Andres Freund wrote: > Prototype patch attached. Because I forgot about cfbot when attaching the patch, cfbot actually ran with it under this thread's cf entry. It does look like an improvement to me:

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-03-30 Thread Nathan Bossart
On Tue, Mar 29, 2022 at 03:48:32PM -0700, Nathan Bossart wrote: > On Thu, Mar 24, 2022 at 01:17:01PM +1300, Thomas Munro wrote: >> /* we're only handling directories here, skip if it's not ours */ >> -if (lstat(path, ) == 0 && !S_ISDIR(statbuf.st_mode)) >> +if (lstat(path, ) != 0) >>

Re: Frontend error logging style

2022-03-30 Thread Tom Lane
Greg Stark writes: > FYI this patch no longer applies. No surprise :-( > Given it's a Tom patch and due to its nature it'll bitrot rapidly > anyways I'm don't see a point in updating the status though. We now seem to have buy-in on the concept, so my plan is to let this sit till the end of the

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-03-30 Thread Jacob Champion
On Wed, 2022-03-30 at 13:37 +0200, Peter Eisentraut wrote: > On 28.03.22 22:21, Jacob Champion wrote: > > On Mon, 2022-03-28 at 11:17 +0200, Daniel Gustafsson wrote: > > > Fixing up the switch_server_cert() calls and using default_ssl_connstr > > > makes > > > the test pass for me. The required

Re: Returning multiple rows in materialized mode inside the extension

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 9:01 AM Piotr Styczyński wrote: > I don’t know if this mailing list is a good place to ask this question, > but if it’s not, just correct me. > pgsql-general is probably better > *The problem:* > > We currently have a one-to-many function (an operation that produces >

Re: Add non-blocking version of PQcancel

2022-03-30 Thread Jelte Fennema
I attached a new version of this patch. Which does three main things: 1. Change the PQrequestCancel implementation to use the regular connection establishement code, to support all connection options including encryption. 2. Add PQrequestCancelStart which is a thread-safe and

Re: Printing backtrace of postgres processes

2022-03-30 Thread Justin Pryzby
On Wed, Mar 30, 2022 at 11:53:52AM -0400, Greg Stark wrote: > Sadly the cfbot is showing a patch conflict again. It's just a trivial > conflict in the regression test schedule so I'm not going to update > the status but it would be good to rebase it so we continue to get > cfbot testing. Done.

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-03-30 Thread Jacob Champion
On Tue, 2022-03-29 at 16:53 -0700, Andres Freund wrote: > > We'd also need to guess whether the GUC system's serialization of NULL > > as an empty string is likely to cause problems for any future auth > > methods. > > You can't represent a NULL in a postgres 'text' datum, independent of >

Returning multiple rows in materialized mode inside the extension

2022-03-30 Thread Piotr Styczyński
Hi, I represent a small group of developers. We are working on an open-source PostgreSQL extension to enable processing of genomic data inside Postgres. We have an extensive knowledge of molecular biology or data science and none of the Postgres internals. I don’t know if this mailing list is a

Re: Printing backtrace of postgres processes

2022-03-30 Thread Greg Stark
Sadly the cfbot is showing a patch conflict again. It's just a trivial conflict in the regression test schedule so I'm not going to update the status but it would be good to rebase it so we continue to get cfbot testing.

Re: Frontend error logging style

2022-03-30 Thread Greg Stark
FYI this patch no longer applies. Given it's a Tom patch and due to its nature it'll bitrot rapidly anyways I'm don't see a point in updating the status though.

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 8:46 AM Tom Lane wrote: > I don't want to do that with > a blunderbuss, but perhaps there's an argument to do it for specific > cases (search_path comes to mind, though the performance cost could be > significant, since I think setting that in function SET clauses is >

Re: Adding CI to our tree

2022-03-30 Thread Andres Freund
Hi, Now that zstd is used, enable it in CI. I plan to commit this shortly, unless somebody sees a reason not to do so. Greetings, Andres Freund >From 51cc830e2e82516966fe1c84cd134b1858a89bab Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Sat, 26 Mar 2022 12:36:04 -0700 Subject: [PATCH v1]

  1   2   >