Re: New docs chapter on Transaction Management and related changes

2022-11-22 Thread Laurenz Albe
On Tue, 2022-11-22 at 13:50 -0500, Bruce Momjian wrote: > Agreed, updated patch attached. I cannot find any more problems, and I shouldn't mention the extra empty line at the end of the patch. I'd change the commitfest status to "ready for committer" now if it were not already in that status.

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Julien Rouhaud
Hi, Sorry for the very late answer, I had quite a lot of other things going on recently. And thanks for taking care of the patchset! On Wed, Nov 23, 2022 at 03:05:18PM +0900, Michael Paquier wrote: > On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > > + /* XXX: this should

drop postmaster symlink

2022-11-22 Thread Peter Eisentraut
A little while ago we discussed briefly over in the meson thread whether we could remove the postmaster symlink [0]. The meson build system currently does not install a postmaster symlink. (AFAICT, the MSVC build system does not either.) So if we want to elevate the meson build system, we

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread David Rowley
On Wed, 23 Nov 2022 at 20:29, sirisha chamarthi wrote: > I ran your test1 exactly like your setup except the row count is 300 > (with 13275 blocks). Shared_buffers is 128MB and the hardware configuration > details at the bottom of the mail. It appears Master + 0001 + 0005 regressed >

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread sirisha chamarthi
On Tue, Nov 22, 2022 at 1:58 PM David Rowley wrote: > On Thu, 3 Nov 2022 at 06:25, Andres Freund wrote: > > Attached is an experimental patch/hack for that. It ended up being more > > beneficial to make the access ordering more optimal than prefetching the > tuple > > contents, but I'm not at

Re: Documentation for building with meson

2022-11-22 Thread Justin Pryzby
On Mon, Nov 14, 2022 at 10:41:21AM -0800, samay sharma wrote: > You need LZ4, if you want to support compression of data with that > method; see default_toast_compression and wal_compression. => The first comma is odd. Maybe it should say "LZ4 is needed to support .." > You need Zstandard, if

Help running 010_tab_completion.pl on windows

2022-11-22 Thread Kirk Wolak
I have psql working with readline (roughly) in windows 10! in my attempt to test it... >> 1..0 # SKIP IO::Pty is needed to run this test I would like to run these tests to see how far off I am... (Randomly typing sql and squealing like a child has its limits) I have built this using VS 2022

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2022-11-22 Thread Michael Paquier
On Tue, Nov 22, 2022 at 05:42:24PM -0800, Andres Freund wrote: > The failure has to be happening in wait_for_postmaster_promote(), because the > standby2 is actually successfully promoted. That's the one under -fsanitize=address. It really smells to me like a bug with a race condition all over

Re: Introduce a new view for checkpointer related stats

2022-11-22 Thread Bharath Rupireddy
On Wed, Nov 23, 2022 at 2:23 AM Andres Freund wrote: > > On 2022-11-22 18:08:28 +0530, Bharath Rupireddy wrote: > > > > CREATE VIEW pg_stat_bgwriter AS > > SELECT > > -pg_stat_get_bgwriter_timed_checkpoints() AS checkpoints_timed, > > -

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > + /* XXX: this should stick to elevel for some cases? */ > + ereport(LOG, > + (errmsg("skipping missing authentication file \"%s\"", > + inc_fullname))); > Should we always issue a LOG here?

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-11-22 Thread Justin Pryzby
Note that 001 fails to compile without 002: ../src/backend/storage/buffer/bufmgr.c:1257:43: error: ‘from_ring’ undeclared (first use in this function) 1257 | StrategyRejectBuffer(strategy, buf, from_ring)) My "warnings" script informed me about these gripes from MSVC: [03:42:30.607]

Re: Collation version tracking for macOS

2022-11-22 Thread Thomas Munro
On Tue, Nov 22, 2022 at 7:34 PM Jeff Davis wrote: > On Sat, 2022-10-22 at 14:22 +1300, Thomas Munro wrote: > > Problem 2: If ICU 67 ever decides to report a different version for > > a > > given collation (would it ever do that? I don't expect so, but ...), > > we'd be unable to open the

Re: Non-decimal integer literals

2022-11-22 Thread John Naylor
On Tue, Nov 22, 2022 at 8:36 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > On 15.11.22 11:31, Peter Eisentraut wrote: > > On 14.11.22 08:25, John Naylor wrote: > >> Regarding the patch, it looks good overall. My only suggestion would > >> be to add a regression test for just

RE: Ability to reference other extensions by schema in extension scripts

2022-11-22 Thread Regina Obe
> > "Regina Obe" writes: > >> I have a distinct sense of deja vu here. I think this idea, or > >> something isomorphic to it, was previously discussed with some other > syntax details. > > > I found the old discussion I recalled having and Stephen had suggested > > using @extschema{'postgis'}@

Re: [PoC] configurable out of disk space elog level

2022-11-22 Thread Greg Stark
On Thu, 17 Nov 2022 at 14:56, Robert Haas wrote: > > Having a switch for one particular kind of error (out of many that > could possibly occur) that triggers one particular coping strategy > (out of many that could possibly be used) seems far too specific a > thing to add as a core feature. And

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Amit Kapila
On Wed, Nov 23, 2022 at 1:40 AM Stavros Koureas wrote: > > Reading more carefully what you described, I think you are interested in > getting something you call origin from publishers, probably some metadata > from the publications. > > This identifier in those metadata maybe does not have

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Amit Kapila
On Tue, Nov 22, 2022 at 6:22 PM Stavros Koureas wrote: > > Sure, this can be implemented as a subscription option, and it will cover > this use case scenario as each subscriber points only to one database. > I also have some more analytical/reporting use-cases which need additions in >

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-22 Thread Amit Kapila
On Tue, Nov 22, 2022 at 10:33 PM Maxim Orlov wrote: >> >> >> Regarding the tests, the patch includes a new scenario to >> reproduce this issue. However, since the issue can be reproduced also >> by the existing scenario (with low probability, though), I'm not sure >> it's worth adding the new

Re: ssl tests aren't concurrency safe due to get_free_port()

2022-11-22 Thread Andrew Dunstan
> On Nov 22, 2022, at 8:36 PM, Tom Lane wrote: > > Andres Freund writes: >> While looking into a weird buildfarm failure ([1]), I noticed this: > >> # Checking port 62707 >> Use of uninitialized value $pid in scalar chomp at >>

Re: Logical replication missing information

2022-11-22 Thread Peter Smith
On Fri, Nov 18, 2022 at 4:50 AM PG Doc comments form wrote: > > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/15/logical-replication-row-filter.html > Description: Hi, FYI - I have forwarded this post to the hacker's list, where I

odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2022-11-22 Thread Andres Freund
Hi, My buildfarm animal grassquit just showed an odd failure [1] in REL_11_STABLE: ok 10 - standby is in recovery # Running: pg_ctl -D /mnt/resource/bf/build/grassquit/REL_11_STABLE/pgsql.build/src/bin/pg_ctl/tmp_check/t_003_promote_standby2_data/pgdata promote waiting for server to

Re: ssl tests aren't concurrency safe due to get_free_port()

2022-11-22 Thread Tom Lane
Andres Freund writes: > While looking into a weird buildfarm failure ([1]), I noticed this: > # Checking port 62707 > Use of uninitialized value $pid in scalar chomp at > /mnt/resource/bf/build/grassquit/REL_11_STABLE/pgsql.build/../pgsql/src/test/perl/PostgresNode.pm > line 1247. > Use of

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread John Naylor
On Wed, Nov 23, 2022 at 5:00 AM David Rowley wrote: > > On Thu, 3 Nov 2022 at 22:09, John Naylor wrote: > > I tried a similar test, but with text fields of random length, and there is improvement here: > > Thank you for testing that. Can you share which CPU this was on? That was an Intel Core

Re: ssl tests aren't concurrency safe due to get_free_port()

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 10:57:41 -0500, Andrew Dunstan wrote: > On 2022-11-20 Su 14:05, Andres Freund wrote: > >> If it works ok I will backpatch in couple of days. > > +1 > Done. While looking into a weird buildfarm failure ([1]), I noticed this: # Checking port 62707 Use of uninitialized value $pid

Re: More efficient build farm animal wakeup?

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 17:35:12 -0500, Andrew Dunstan wrote: > The server side appears to be working well. > > The new client side code is being tested on crake and working fine - the > all-up-to-date case takes just a second or two, almost all of which is > taken with getting the json file from the

Re: TAP output format in pg_regress

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 23:17:44 +0100, Daniel Gustafsson wrote: > The attached v10 attempts to address the points raised above. Notes and > diagnostics are printed to stdout/stderr respectively and the TAP emitter is > changed to move more of the syntax into it making it less painful on the >

Re: More efficient build farm animal wakeup?

2022-11-22 Thread Tom Lane
Andrew Dunstan writes: > The new client side code is being tested on crake and working fine - the > all-up-to-date case takes just a second or two, almost all of which is > taken with getting the json file from the server. No git calls at all > are done on the client in this case. Nice! I

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Peter Smith
On Wed, Nov 23, 2022 at 7:38 AM Stavros Koureas wrote: > > Reading more carefully what you described, I think you are interested in > getting something you call origin from publishers, probably some metadata > from the publications. > > This identifier in those metadata maybe does not have

Re: Make ON_ERROR_STOP stop on shell script failure

2022-11-22 Thread Matheus Alcantara
--- Original Message --- On Tuesday, November 22nd, 2022 at 20:10, bt22nakamorit wrote: > There was a mistake in the error message for \! so I updated the patch. > > Best, > Tatsuhiro Nakamori Hi I was checking your patch and seems that it failed to be applied into the master

Re: CI and test improvements

2022-11-22 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 02:45:42PM -0800, Andres Freund wrote: > > > > +ninja -C build |tee build/meson-logs/build.txt > > > > +REM Since pipes lose exit status of the preceding command, rerun > > > > compilation, > > > > +REM without the pipe exiting now if it fails, rather than

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-11-22 Thread Steve Chavez
Hey Alexander, Looks like your latest patch addresses the original issue I posted! So now I can create a placeholder with the USERSET modifier without a superuser, while non-USERSET placeholders still require superuser: ```sql create role foo noinherit; set role to foo; alter role foo set

Re: fixing CREATEROLE

2022-11-22 Thread Mark Dilger
> On Nov 22, 2022, at 2:02 PM, Robert Haas wrote: > >> Patch 0004 feels like something that won't get committed. The >> INHERITCREATEDROLES and SETCREATEDROLES in 0004 seems clunky. > > I think role properties are kind of clunky in general, the way we've > implemented them in PostgreSQL,

Re: More efficient build farm animal wakeup?

2022-11-22 Thread Andrew Dunstan
On 2022-11-22 Tu 13:04, Magnus Hagander wrote: > > > On Tue, Nov 22, 2022 at 12:10 AM Magnus Hagander > wrote: > > > > On Mon, Nov 21, 2022 at 11:42 PM Andrew Dunstan > wrote: > > > On 2022-11-21 Mo 16:20, Magnus Hagander wrote: > > n Mon, Nov 21, 2022 at 9:58 PM Tom

Re: pgstattuple: add test for coverage

2022-11-22 Thread Andres Freund
Hi, On 2022-10-03 00:42:27 +0900, Dong Wook Lee wrote: > > Which indeed is the case, e.g. on 32bit systems it fails: > > > > https://cirrus-ci.com/task/4619535222308864?logs=test_world_32#L253 > > > >

Re: TAP output format in pg_regress

2022-11-22 Thread Daniel Gustafsson
> On 21 Nov 2022, at 14:42, Dagfinn Ilmari Mannsåker wrote: > > Andres Freund writes: > >> But either way, it seems nicer to output the # inside a helper function? > > Note that the helper function should inject '# ' at the start of every > line in the message, not just the first line. It

Re: PGDOCS - Logical replication GUCs - added some xrefs

2022-11-22 Thread Peter Smith
On Wed, Nov 16, 2022 at 10:24 PM vignesh C wrote: > ... > One suggestion: > The format of subscribers includes the data type and default values, > the format of publishers does not include data type and default > values. We can try to maintain the consistency for both publisher and > subscriber

Re: Make mesage at end-of-recovery less scary.

2022-11-22 Thread Justin Pryzby
On Fri, Nov 18, 2022 at 05:25:37PM +0900, Kyotaro Horiguchi wrote: > + while (*p == 0 && p < pe) > + p++; The bug reported by Andres/cfbot/ubsan is here. Fixed in attached. I didn't try to patch the test case to output the failing stderr, but that might be good.

Re: fixing CREATEROLE

2022-11-22 Thread Robert Haas
On Tue, Nov 22, 2022 at 3:01 PM Mark Dilger wrote: > > On Nov 21, 2022, at 12:39 PM, Robert Haas wrote: > > I have drafted a few patches to try to improve the situation. > > The 0001 and 0002 patches appear to be uncontroversial refactorings. Patch > 0003 looks on-point and a move in the right

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread David Rowley
On Thu, 3 Nov 2022 at 22:09, John Naylor wrote: > I tried a similar test, but with text fields of random length, and there is > improvement here: Thank you for testing that. Can you share which CPU this was on? My tests were all on AMD Zen 2. I'm keen to see what the results are on intel

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread David Rowley
On Thu, 3 Nov 2022 at 06:25, Andres Freund wrote: > Attached is an experimental patch/hack for that. It ended up being more > beneficial to make the access ordering more optimal than prefetching the tuple > contents, but I'm not at all sure that's the be-all-end-all. Thanks for writing that

Re: Slow standby snapshot

2022-11-22 Thread Michail Nikolaev
Hello, everyone. I have tried to put it all together. > In the absence of that approach, falling back to a counter that > compresses every N xids would be best, in addition to the two new > forced compression events. Done. > Also, if we add more forced compressions, it seems like we should

Re: Introduce a new view for checkpointer related stats

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 18:08:28 +0530, Bharath Rupireddy wrote: > diff --git a/src/backend/catalog/system_views.sql > b/src/backend/catalog/system_views.sql > index 2d8104b090..131d949dfb 100644 > --- a/src/backend/catalog/system_views.sql > +++ b/src/backend/catalog/system_views.sql > @@ -1105,18

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Stavros Koureas
Reading more carefully what you described, I think you are interested in getting something you call origin from publishers, probably some metadata from the publications. This identifier in those metadata maybe does not have business value on the reporting side. The idea is to use a value which

Re: fixing CREATEROLE

2022-11-22 Thread Mark Dilger
> On Nov 21, 2022, at 12:39 PM, Robert Haas wrote: > > I have drafted a few patches to try to improve the situation. The 0001 and 0002 patches appear to be uncontroversial refactorings. Patch 0003 looks on-point and a move in the right direction. The commit message in that patch is well

Re: [PATCH] Support % wildcard in extension upgrade filenames

2022-11-22 Thread Regina Obe
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed I reviewed this patch

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Robert Haas
On Tue, Nov 22, 2022 at 1:44 PM Tom Lane wrote: > I wrote: > > Still wondering if there's really no CHECK_FOR_INTERRUPT anywhere > > else in this loop. > > I did some experimentation using the test case Jakub presented > to start with, and verified that that loop does respond promptly > to

Re: New docs chapter on Transaction Management and related changes

2022-11-22 Thread Bruce Momjian
On Tue, Nov 22, 2022 at 07:47:26PM +0100, Erik Rijkers wrote: > Op 22-11-2022 om 19:00 schreef Bruce Momjian: > > On Mon, Nov 21, 2022 at 11:15:36AM +0100, Laurenz Albe wrote: > > >..., while both columns will be set in read-write transactions. > > > > Agreed, changed. Updated patch

Re: New docs chapter on Transaction Management and related changes

2022-11-22 Thread Erik Rijkers
Op 22-11-2022 om 19:00 schreef Bruce Momjian: On Mon, Nov 21, 2022 at 11:15:36AM +0100, Laurenz Albe wrote: ..., while both columns will be set in read-write transactions. Agreed, changed. Updated patch attached. In func.sgml: 'Only top-level transaction ID are' should be 'Only

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Tom Lane
I wrote: > Still wondering if there's really no CHECK_FOR_INTERRUPT anywhere > else in this loop. I did some experimentation using the test case Jakub presented to start with, and verified that that loop does respond promptly to control-C even in HEAD. So there are CFI(s) in the loop as I

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 22, 2022 at 11:35 AM Tom Lane wrote: >> Is it appropriate to count distinct pages, rather than just the >> number of times we have to visit a heap tuple? That seems to >> complicate the logic a good deal, and I'm not sure it's buying >> much, especially since

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Robert Haas
On Tue, Nov 22, 2022 at 11:35 AM Tom Lane wrote: > Simon Riggs writes: > > New patch version reporting for duty, sir. Please take it from here! > > Why the CHECK_FOR_INTERRUPTS? I'd supposed that there's going to be > one somewhere down inside the index or heap access --- do you have > reason

Re: New docs chapter on Transaction Management and related changes

2022-11-22 Thread Bruce Momjian
On Mon, Nov 21, 2022 at 11:35:09AM +0100, Álvaro Herrera wrote: > Agreed on not using "unaborted", per previous discussion. > > On 2022-Nov-21, Laurenz Albe wrote: > > > Perhaps we should also avoid the term "transaction block". Even without > > speaking > > of a "block", way too many people

Re: More efficient build farm animal wakeup?

2022-11-22 Thread Magnus Hagander
On Tue, Nov 22, 2022 at 12:10 AM Magnus Hagander wrote: > > > On Mon, Nov 21, 2022 at 11:42 PM Andrew Dunstan > wrote: > >> >> On 2022-11-21 Mo 16:20, Magnus Hagander wrote: >> > n Mon, Nov 21, 2022 at 9:58 PM Tom Lane wrote: >> > >> > Andrew Dunstan writes: >> b> > The buildfarm

Re: New docs chapter on Transaction Management and related changes

2022-11-22 Thread Bruce Momjian
On Mon, Nov 21, 2022 at 11:15:36AM +0100, Laurenz Albe wrote: > > --- a/doc/src/sgml/ref/release_savepoint.sgml > > +++ b/doc/src/sgml/ref/release_savepoint.sgml > > > + RELEASE SAVEPOINT releases the named savepoint and > > + all active savepoints that were created after the named savepoint,

Re: [PATCH] Const'ify the arguments of ilist.c/ilist.h functions

2022-11-22 Thread Andres Freund
Hi, On 2022-11-07 12:03:23 +0300, Aleksander Alekseev wrote: > During the [1] discussion it was suggested to constify the arguments > of ilist.c/ilist.h functions. Bharath (cc'ed) pointed out that it's > better to start a new thread in order to attract more hackers that may > be interested in

Re: Make mesage at end-of-recovery less scary.

2022-11-22 Thread Andres Freund
Hi, On 2022-11-18 17:25:37 +0900, Kyotaro Horiguchi wrote: > Just rebased. Fails with address sanitizer: https://cirrus-ci.com/task/5632986241564672 Unfortunately one of the failures is in pg_waldump and we don't seem to capture its output in 011_crash_recovery. So we don't see the nice

Re: Allow single table VACUUM in transaction block

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:43, Justin Pryzby wrote: > > On Mon, Nov 21, 2022 at 03:07:25PM +, Simon Riggs wrote: > > Attached patch implements VACUUM (BACKGROUND). > > > > There are quite a few small details to consider; please read the docs > > and comments. > > > > There is a noticeable

Re: Postgres picks suboptimal index after building of an extended statistics

2022-11-22 Thread Andres Freund
Hi, On 2022-07-11 12:57:36 +0500, Andrey Lepikhov wrote: > On 7/8/22 03:07, Tom Lane wrote: > > Andrey Lepikhov writes: > > > On 12/8/21 04:26, Tomas Vondra wrote: > > > > I wonder if we should teach clauselist_selectivity about UNIQUE indexes, > > > > and improve the cardinality estimates

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-22 Thread Andres Freund
On 2022-11-21 17:06:56 +0900, Masahiko Sawada wrote: > Sure. I've attached the v10 patches. 0004 is the pure refactoring > patch and 0005 patch introduces the pointer tagging. This failed on cfbot, with som many crashes that the VM ran out of disk for core dumps. During testing with 32bit, so

Re: Slow standby snapshot

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:53, Tom Lane wrote: > > Simon Riggs writes: > > On Tue, 22 Nov 2022 at 16:28, Tom Lane wrote: > >> If we do those things, do we need a wasted-work counter at all? > > > The wasted work counter works well to respond to heavy read-only > > traffic and also avoids wasted

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-22 Thread Maxim Orlov
> I've attached a draft patch. To fix it, I think we can reset > InitialRunningXacts and NInitialRunningXacts at FreeSnapshotBuilder() > and add an assertion in AllocateSnapshotBuilder() to make sure both > are reset. > Thanks for the patch. It works fine. I've tested this patch for 15 and 11

Re: proposal: possibility to read dumped table's name from file

2022-11-22 Thread Pavel Stehule
Hi út 22. 11. 2022 v 8:39 odesílatel Andres Freund napsal: > Hi, > > On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote: > > updated patch attached > > It fails with address sanitizer that's now part of CI: > > https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659 > > [06:33:11.271]

Re: Slow standby snapshot

2022-11-22 Thread Tom Lane
Simon Riggs writes: > On Tue, 22 Nov 2022 at 16:28, Tom Lane wrote: >> If we do those things, do we need a wasted-work counter at all? > The wasted work counter works well to respond to heavy read-only > traffic and also avoids wasted compressions for write-heavy workloads. > So I still like it

Re: Allow single table VACUUM in transaction block

2022-11-22 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 03:07:25PM +, Simon Riggs wrote: > Attached patch implements VACUUM (BACKGROUND). > > There are quite a few small details to consider; please read the docs > and comments. > > There is a noticeable delay before the background vacuum starts. You disallowed some

Re: fixing CREATEROLE

2022-11-22 Thread Joe Conway
On 11/21/22 15:39, Robert Haas wrote: I'm curious to hear what other people think of these proposals, but let me first say what I think about them. First, I think it's clear that we need to do something, because things right now are pretty badly broken and in a way that affects security.

Re: Slow standby snapshot

2022-11-22 Thread Simon Riggs
On Tue, 22 Nov 2022 at 16:28, Tom Lane wrote: > > Simon Riggs writes: > > We seem to have replaced one magic constant with another, so not sure > > if this is autotuning, but I like it much better than what we had > > before (i.e. better than my prev patch). > > Yeah, the magic constant is still

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Tom Lane
Simon Riggs writes: > New patch version reporting for duty, sir. Please take it from here! Why the CHECK_FOR_INTERRUPTS? I'd supposed that there's going to be one somewhere down inside the index or heap access --- do you have reason to think there isn't? Is it appropriate to count distinct

Re: fixing CREATEROLE

2022-11-22 Thread Tom Lane
walt...@technowledgy.de writes: >> No, we don't support partial indexes on catalogs, and I don't think >> we want to change that. Partial indexes would require expression >> evaluations occurring at very inopportune times. > I see. Is that the same for indexes *on* an expression? Or would those

Re: Slow standby snapshot

2022-11-22 Thread Tom Lane
Simon Riggs writes: > We seem to have replaced one magic constant with another, so not sure > if this is autotuning, but I like it much better than what we had > before (i.e. better than my prev patch). Yeah, the magic constant is still magic, even if it looks like it's not terribly sensitive to

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
On Mon, 21 Nov 2022 at 20:55, Robert Haas wrote: > > On Mon, Nov 21, 2022 at 1:17 PM Andres Freund wrote: > > On November 21, 2022 9:37:34 AM PST, Robert Haas > > wrote: > > >On Mon, Nov 21, 2022 at 12:30 PM Andres Freund wrote: > > >> This can't quite be right - isn't this only applying the

Re: fixing CREATEROLE

2022-11-22 Thread walther
Wolfgang Walther: Tom Lane: No, we don't support partial indexes on catalogs, and I don't think we want to change that.  Partial indexes would require expression evaluations occurring at very inopportune times. I see. Is that the same for indexes *on* an expression? Or would those be ok?

Re: Partial aggregates pushdown

2022-11-22 Thread Alexander Pyhalov
fujii.y...@df.mitsubishielectric.co.jp писал 2022-11-22 04:01: Hi Mr.Vondra, Mr.Pyhalov, Everyone. I discussed with Mr.Pyhalov about the above draft by directly sending mail to him(outside of pgsql-hackers). Mr.Pyhalov allowed me to update his patch along with the above draft. So I update

Re: fixing CREATEROLE

2022-11-22 Thread walther
Tom Lane: No, we don't support partial indexes on catalogs, and I don't think we want to change that. Partial indexes would require expression evaluations occurring at very inopportune times. I see. Is that the same for indexes *on* an expression? Or would those be ok? With a custom

Re: Code checks for App Devs, using new options for transaction behavior

2022-11-22 Thread Simon Riggs
On Mon, 7 Nov 2022 at 14:25, Simon Riggs wrote: > > On Wed, 2 Nov 2022 at 07:40, Simon Riggs wrote: > > > > What will be the behavior if someone declares a savepoint with this > > > name ("_internal_nested_xact"). Will this interfere with this new > > > functionality? > > > > Clearly! I guess

Re: ssl tests aren't concurrency safe due to get_free_port()

2022-11-22 Thread Andrew Dunstan
On 2022-11-20 Su 14:05, Andres Freund wrote: >> If it works ok I will backpatch in couple of days. > +1 Done. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com

Re: dynamic result sets support in extended query protocol

2022-11-22 Thread Peter Eisentraut
On 14.10.22 09:11, Peter Eisentraut wrote: Now that the psql support for multiple result sets exists, I want to revive this patch.  It's the same as the last posted version, except now it doesn't require any psql changes or any weird test modifications anymore. (Old news: This patch allows

Re: psql: Add command to use extended query protocol

2022-11-22 Thread Peter Eisentraut
On 21.11.22 23:02, Corey Huinker wrote: I got thinking about this, and while things may be fine as-is, I would like to hear some opinions as to whether this behavior is correct: This is all psql syntax, nothing specific to this command. The only leeway is choosing the appropriate enum

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Stavros Koureas
Sure, this can be implemented as a subscription option, and it will cover this use case scenario as each subscriber points only to one database. I also have some more analytical/reporting use-cases which need additions in logical-replication, I am not sure if I need to open different discussions

Re: fixing CREATEROLE

2022-11-22 Thread Tom Lane
walt...@technowledgy.de writes: > Robert Haas: >> 2. There are some serious implementation challenges because the >> constraints on duplicate object names must be something which can be >> enforced by unique constraints on the relevant catalogs. Off-hand, I >> don't see how to do that. > For each

Re: fixing CREATEROLE

2022-11-22 Thread walther
Robert Haas: 2. There are some serious implementation challenges because the constraints on duplicate object names must be something which can be enforced by unique constraints on the relevant catalogs. Off-hand, I don't see how to do that. It would be easy to make the cluster roles all have

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Dilip Kumar
On Tue, Nov 22, 2022 at 7:44 PM Aleksander Alekseev wrote: > > Hi hackers, > > [ Excluding my personal e-mail from cc:, not sure how it got there. > Please don't cc: to afis...@gmail.com, I'm not using it for reading > pgsql-hackers@. ] > > > I agree with Alexander, that notifications for DBA are

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
On Mon, 21 Nov 2022 at 22:15, Tom Lane wrote: > > Andres Freund writes: > > On 2022-11-21 16:17:56 -0500, Robert Haas wrote: > >> But ... what if they're not? Could the index contain a large number of > >> pages containing just 1 tuple each, or no tuples at all? If so, maybe > >> we can read ten

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Aleksander Alekseev
Hi hackers, [ Excluding my personal e-mail from cc:, not sure how it got there. Please don't cc: to afis...@gmail.com, I'm not using it for reading pgsql-hackers@. ] > I agree with Alexander, that notifications for DBA are a little bit > outside the scope of the activity in this thread unless

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-22 Thread Pavel Borisov
Hi, Alexander! On Tue, 22 Nov 2022 at 13:01, Aleksander Alekseev wrote: > > Hi Chris, > > > Right now the way things work is: > > 1. Database starts throwing warnings that xid wraparound is approaching > > 2. Database-owning team initiates an emergency response, may take downtime > > or

Re: On login trigger: take three

2022-11-22 Thread Mikhail Gribkov
Hi hackers, Since the original authors, as Daniel said, seems to have retired from the patch, I have allowed myself to continue the patch polishing. Attached v32 includes fresh rebase and the following fixes: - Copying dathasloginevt flag during DB creation from template; - Restoring

RE: Perform streaming logical transactions by background workers and parallel apply

2022-11-22 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thanks for updating the patch! I tested the case whether the deadlock caused by foreign key constraint could be detected, and it worked well. Followings are my review comments. They are basically related with 0001, but some contents may be not. It takes time to understand 0002

Re: fixing CREATEROLE

2022-11-22 Thread Robert Haas
On Tue, Nov 22, 2022 at 3:02 AM wrote: > My suggestion to $subject and the namespace problem would be to > introduce database-specific roles, i.e. add a database column to > pg_authid. Having this column set to 0 will make the role a cluster-wide > role ("cluster role") just as currently the

Re: Non-decimal integer literals

2022-11-22 Thread Peter Eisentraut
On 15.11.22 11:31, Peter Eisentraut wrote: On 14.11.22 08:25, John Naylor wrote: Regarding the patch, it looks good overall. My only suggestion would be to add a regression test for just below and just above overflow, at least for int2. ok This was a valuable suggestion, because this found

Re: postgres_fdw binary protocol support

2022-11-22 Thread Ashutosh Bapat
Hi Illya, On Mon, Nov 21, 2022 at 8:50 PM Ilya Gladyshev wrote: > > Hi everyone, > > I have made a patch that introduces support for libpq binary protocol > in postgres_fdw. The idea is simple, when a user knows that the foreign > server is binary compatible with the local and his workload could

Re: Introduce a new view for checkpointer related stats

2022-11-22 Thread Bharath Rupireddy
On Tue, Nov 22, 2022 at 1:26 PM Drouvot, Bertrand wrote: > > Hi, > > On 11/17/22 1:51 PM, Bharath Rupireddy wrote: > > Hi, > > > > pg_stat_bgwriter view currently reports checkpointer stats as well. It > > is that way because historically checkpointer was part of bgwriter > > until the commits

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-22 Thread Masahiko Sawada
Hi, On Tue, Nov 22, 2022 at 6:37 PM Amit Kapila wrote: > > On Mon, Nov 21, 2022 at 6:17 PM Maxim Orlov wrote: > > > > PROBLEM > > > > After some investigation, I think, the problem is in the snapbuild.c > > (commit 272248a0c1b1, see [0]). We do allocate InitialRunningXacts > > array in the

Re: Logical Replication Custom Column Expression

2022-11-22 Thread Amit Kapila
On Mon, Nov 21, 2022 at 5:05 PM Ashutosh Bapat wrote: > > On Sat, Nov 19, 2022 at 6:47 PM Stavros Koureas > wrote: > > > > What does not support is the option for defining custom column expressions, > > as keys or values, into the upstream (publication). This will give more > > flexibility

Fix for visibility check on 14.5 fails on tpcc with high concurrency

2022-11-22 Thread Dimos Stamatakis
Hi hackers, When running tpcc on sysbench with high concurrency (96 threads, scale factor 5) we realized that a fix for visibility check (introduced in PG-14.5) causes sysbench to fail in 1 out of 70 runs. The error is the following: SQL error, errno = 0, state = 'XX000': new multixact has

Re: Fix comments atop pg_get_replication_slots

2022-11-22 Thread Amit Kapila
On Mon, Nov 21, 2022 at 1:22 PM sirisha chamarthi wrote: > > Looks good to me. Attached a patch for the same. > Pushed. -- With Regards, Amit Kapila.

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-22 Thread Amit Kapila
On Tue, Nov 22, 2022 at 6:11 AM Nathan Bossart wrote: > > While working on avoiding unnecessary wakeups in logical/worker.c (as was > done for walreceiver.c in 05a7be9), I noticed that the tests began taking > much longer. This seems to be caused by the reduced frequency of calls to >

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-22 Thread Simon Riggs
On Mon, 21 Nov 2022 at 23:34, Robert Haas wrote: > > On Mon, Nov 21, 2022 at 6:17 PM Tom Lane wrote: > > evidence that it's a live problem. API warts are really hard to > > get rid of once instituted. > > Yeah, agreed. Agreed, happy not to; that version was just a WIP to see how it might work.

Re: when the startup process doesn't (logging startup delays)

2022-11-22 Thread Bharath Rupireddy
On Mon, Nov 21, 2022 at 10:37 PM Robert Haas wrote: > > On Sun, Nov 20, 2022 at 9:20 PM Kyotaro Horiguchi > wrote: > > I prefer Robert's approach as it is more robust for future changes and > > simple. I prefer to avoid this kind of piggy-backing and it doesn't > > seem to be needed in this

Re: [PATCH] Add initial xid/mxid/mxoff to initdb

2022-11-22 Thread Maxim Orlov
Hi! CF bot says patch does not apply. Rebased. Your reviews are very much welcome! -- Best regards, Maxim Orlov. v2-0001-Add-initdb-option-to-initialize-cluster-with-non-.patch Description: Binary data

Re: pg_upgrade, tables_with_oids.txt -> tables_with_oids.sql?

2022-11-22 Thread Daniel Gustafsson
> On 6 Nov 2022, at 09:48, Daniel Westermann (DWE) > wrote: > as I've just upgraded an instance which contained tables "WITH OIDS" I wonder > if it would make sense if pg_upgrade directly creates a script to fix those. > I know it is easy to that with e.g. sed over tables_with_oids.txt but it

Re: [CAUTION!! freemail] Re: Partial aggregates pushdown

2022-11-22 Thread Ted Yu
On Tue, Nov 22, 2022 at 1:11 AM fujii.y...@df.mitsubishielectric.co.jp < fujii.y...@df.mitsubishielectric.co.jp> wrote: > Hi Mr.Yu. > > Thank you for comments. > > > + * Check that partial aggregate agg has compatibility > > > > If the `agg` refers to func parameter, the parameter name is aggform

  1   2   >