Re: psql - factor out echo code

2022-11-29 Thread Michael Paquier
On Sun, Jul 24, 2022 at 10:23:39PM +0200, Pavel Stehule wrote: > I had just one question - with this patch, the format of output of modes > ECHO ALL and ECHO QUERIES will be different, and that can be a little bit > messy. On second hand, the prefix --QUERY can be disturbing in echo queries >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread Masahiko Sawada
On Fri, Nov 25, 2022 at 6:47 PM John Naylor wrote: > > > > On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > > > [v11] > > There is one more thing that just now occurred to me: In expanding the use of > size classes, that makes rebasing and reworking the shared memory piece more >

Re: Large Pages and Super Pages for PostgreSQL

2022-11-29 Thread ZHU XIAN WEN
Hi Thomas Thank you very much for the work. I just got latest FreeBSD 13.1 environment, and I'm going to test and verify it. so would you please rebase latest patch? best wishes Tony On 2022/11/8 06:59, Thomas Munro wrote: On Sun, Jan 16, 2022 at 8:32 PM Thomas Munro wrote: On Sun, Jan

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread Masahiko Sawada
On Fri, Nov 25, 2022 at 5:00 PM John Naylor wrote: > > On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > > > So it seems that there are two candidates of rt_node structure: (1) > > all nodes except for node256 are variable-size nodes and use pointer > > tagging, and (2) node32 and

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Drouvot, Bertrand
Hi, On 11/30/22 7:34 AM, Bharath Rupireddy wrote: On Wed, Nov 30, 2022 at 6:01 AM Andres Freund wrote: Hi, On 2022-11-28 12:58:48 -0500, Robert Haas wrote: On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: I think we should consider deprecating the pg_stat_bgwriter columns but leaving

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Noah Misch
On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > On Fri, Sep 10, 2021 at 2:39 AM Noah Misch wrote: > > Revoke PUBLIC CREATE from public schema, now owned by pg_database_owner. > > > > This switches the default ACL to what the documentation has recommended > > since CVE-2018-1058.

pg_dump: Remove "blob" terminology

2022-11-29 Thread Peter Eisentraut
For historical reasons, pg_dump refers to large objects as "BLOBs". This term is not used anywhere else in PostgreSQL, and it also means something different in the SQL standard and other SQL systems. This patch renames internal functinos, code comments, documentation, etc. to use the "large

Re: Making pg_rewind faster

2022-11-29 Thread Michael Paquier
On Thu, Oct 06, 2022 at 04:08:45PM +0900, Michael Paquier wrote: > file_entry_t has an entry to track if a file is a relation file. I > think that it would be much cleaner to track if we are handling a WAL > segment when inserting an entry in insert_filehash_entry(), so > isrelfile could be

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 01:51, Bruce Momjian wrote: > Thanks to Simon for getting this important > information in our docs, and for the valuable feedback from others that > made this even better. And thanks to you for pulling that all together Bruce. -- Simon Riggs

Re: real/float example for testlibpq3

2022-11-29 Thread Michael Paquier
On Wed, Nov 09, 2022 at 12:47:23PM -0800, Mark Wong wrote: > I'll take a stab at adding a new chapter and share how that looks. This thread has stalled for three weeks, so I have marked it as returned with feedback. Once you have a new patch, please feel free to submit it. -- Michael

Re: pg_get_constraintdef: Schema qualify foreign tables unless pretty printing is enabled

2022-11-29 Thread Michael Paquier
On Wed, Oct 12, 2022 at 02:19:25PM +0900, Michael Paquier wrote: > Lukas, please note that this patch is waiting for your input for a few > weeks now. Could you reply to the reviews provided? This has stalled for six weeks, so I have marked the patch as returned with feedback. -- Michael

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 22:31:50 -0800, Andres Freund wrote: > On 2022-11-30 00:55:42 -0500, Tom Lane wrote: > > Andres Freund writes: > > > What libraries is postgres linked against? I don't know whether -z now > > > only > > > affects the "top-level" dependencies of postgres, or also the > > >

Re: Query Jumbling for CALL and SET utility statements

2022-11-29 Thread Michael Paquier
On Wed, Oct 12, 2022 at 09:13:20AM +0900, Michael Paquier wrote: > Thanks for mentioning that. With an automated way to generate this > code, cursors would be handled, at the advantage of making sure that > no fields are missing in the jumbled structures (is_local was missed > for example on

Re: Slow standby snapshot

2022-11-29 Thread Simon Riggs
On Tue, 29 Nov 2022 at 20:46, Tom Lane wrote: > > I wrote: > > That seems like a fairly bad idea: it will add extra contention > > on ProcArrayLock, and I see no real strong argument that the path > > can't get traversed often enough for that to matter. It would > > likely be better for

Re: Support tls-exporter as channel binding for TLSv1.3

2022-11-29 Thread Michael Paquier
On Fri, Oct 14, 2022 at 11:00:10AM +0900, Michael Paquier wrote: > Oh, okay. That's an argument in favor of not doing that, then. > Perhaps we'd better revisit the introduction of tls-exporter once we > know more about all that, and it looks like we would need a way to be > able to negotiate

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-29 Thread Amit Kapila
On Tue, Nov 29, 2022 at 6:03 PM Amit Kapila wrote: > > 12. Apart from the above, I have made a few changes in the comments > and some other cosmetic changes in the attached patch. > I have made some additional changes in the comments at various places. Kindly check the attached and let me know

Re: ExecRTCheckPerms() and many prunable partitions

2022-11-29 Thread Amit Langote
On Wed, Nov 30, 2022 at 11:56 AM Amit Langote wrote: > On Wed, Nov 30, 2022 at 3:04 AM Alvaro Herrera > wrote: > > On 2022-Nov-29, Amit Langote wrote: > > > > > Maybe, we should have the following there so that the PlannedStmt's > > > contents don't point into the Query? > > > > > >

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:50, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 1:32 AM Simon Riggs > wrote: > > Re-attaching patch for bgwriter and walwriter, so it is clear this is > > not yet committed. > > I'm just curious, and not suggesting that 60s wakeups are a problem > for the polar ice

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Bharath Rupireddy
On Wed, Nov 30, 2022 at 6:01 AM Andres Freund wrote: > > Hi, > > On 2022-11-28 12:58:48 -0500, Robert Haas wrote: > > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > > > I think we should consider deprecating the pg_stat_bgwriter columns but > > > leaving them in place for a few years.

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Peter Eisentraut
On 30.11.22 02:51, Bruce Momjian wrote: On Fri, Nov 18, 2022 at 02:11:50PM -0500, Bruce Momjian wrote: On Mon, Nov 7, 2022 at 11:04:46PM +0100, Laurenz Albe wrote: On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: Agreed; new compilation patch attached, including mine and then Robert's

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-30 00:55:42 -0500, Tom Lane wrote: > Andres Freund writes: > > What libraries is postgres linked against? I don't know whether -z now only > > affects the "top-level" dependencies of postgres, or also the dependencies > > of > > shared libraries that haven't been built with -z

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Simon Riggs
On Wed, 30 Nov 2022 at 03:56, Nathan Bossart wrote: > > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: > > The last important point for me is tests, in src/test/modules > > probably. It might be possible to reuse the final state of other > > modules' tests to test cleanup, or at

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
Andres Freund writes: > On 2022-11-29 20:44:34 -0500, Tom Lane wrote: >> It's also strange that we're apparently running with signals enabled >> whereever it is that rtld_bind is getting called from. Could it be that >> sigaction is failing to install the requested signal mask, so that one >>

Re: Printing backtrace of postgres processes

2022-11-29 Thread Bharath Rupireddy
On Fri, Nov 11, 2022 at 6:04 PM Bharath Rupireddy wrote: > > On Fri, Nov 11, 2022 at 7:59 AM Kyotaro Horiguchi > wrote: > > > > At Thu, 10 Nov 2022 15:56:35 +0530, Bharath Rupireddy > > wrote in > > > On Mon, Apr 18, 2022 at 9:10 AM vignesh C wrote: > > > > > > > > The attached v21 patch has

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 20:44:34 -0500, Tom Lane wrote: > It's also strange that we're apparently running with signals enabled > whereever it is that rtld_bind is getting called from. Could it be that > sigaction is failing to install the requested signal mask, so that one > postmaster signal handler

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
Andres Freund writes: > On 2022-11-29 20:44:34 -0500, Tom Lane wrote: >> Backtrace stopped: frame did not save the PC > Do you have any idea why the stack can't be unwound further here? Is it > possibly indicative of a corrupted stack? I guess we'd need to dig into > the netbsd libc code :( I

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread John Naylor
There are a few things up in the air, so I'm coming back to this list to summarize and add a recent update: On Mon, Nov 14, 2022 at 7:59 PM John Naylor wrote: > > - See how much performance we actually gain from tagging the node kind. Needs a benchmark that has enough branch mispredicts and

Re: Tests for psql \g and \o

2022-11-29 Thread Michael Paquier
On Wed, Nov 23, 2022 at 09:18:57PM +0100, Daniel Verite wrote: > PFA a new patch addressing these issues. Thanks, the tests part of the main regression test suite look good to me, so I have applied them after fixing a few typos and tweaking the style of the test. Regarding the tests with pipes,

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Wed, 23 Nov 2022 at 22:19, John Naylor wrote: > > > On Wed, Nov 23, 2022 at 3:54 PM David Rowley wrote: > > > > Going by [1], clang will actually use multiplication by 16 to > > implement the former. gcc is better and shifts left by 4, so likely > > won't improve things for gcc. It seems

Re: Strange failure on mamba

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 20:44:34 -0500, Tom Lane wrote: > Thanks to commit 51b5834cd I've now been able to capture some info > from mamba's last couple of failures [1][2]. Sure enough, what is > happening is that postmaster children are getting stuck in recursive > rtld symbol resolution. A couple of

Re: pg_dump bugs reported as pg_upgrade bugs

2022-11-29 Thread Tom Lane
Bruce Momjian writes: > FYI, you might wonder why so many bugs reported on pg_upgrade eventually > are bugs in pg_dump. Well, of course, partly is it because pg_upgrade > relies on pg_dump, but a bigger issue is that pg_upgrade will fail if > pg_dump or its restoration generate _any_ errors. My

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 07:56:53PM -0800, Nathan Bossart wrote: > On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: >> The last important point for me is tests, in src/test/modules >> probably. It might be possible to reuse the final state of other >> modules' tests to test cleanup, or

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
On Wed, Nov 30, 2022 at 05:27:40PM +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 5:23 PM Thomas Munro wrote: >> On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart >> wrote: >> > I spent some more time on the prevent-unnecessary-wakeups patch for >> > logical/worker.c that I've been alluding

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Hayato Kuroda (Fujitsu)
Dear Nathan, > I spent some more time on the prevent-unnecessary-wakeups patch for > logical/worker.c that I've been alluding to in this thread, and I found a > few more places where we depend on the worker periodically waking up. This > seems to be a common technique, so I'm beginning to wonder

pg_dump bugs reported as pg_upgrade bugs

2022-11-29 Thread Bruce Momjian
FYI, you might wonder why so many bugs reported on pg_upgrade eventually are bugs in pg_dump. Well, of course, partly is it because pg_upgrade relies on pg_dump, but a bigger issue is that pg_upgrade will fail if pg_dump or its restoration generate _any_ errors. My guess is that many people are

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 5:23 PM Thomas Munro wrote: > On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart > wrote: > > I spent some more time on the prevent-unnecessary-wakeups patch for > > logical/worker.c that I've been alluding to in this thread, and I found a > > few more places where we depend

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 5:10 PM Nathan Bossart wrote: > I spent some more time on the prevent-unnecessary-wakeups patch for > logical/worker.c that I've been alluding to in this thread, and I found a > few more places where we depend on the worker periodically waking up. This > seems to be a

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-29 Thread Nathan Bossart
I spent some more time on the prevent-unnecessary-wakeups patch for logical/worker.c that I've been alluding to in this thread, and I found a few more places where we depend on the worker periodically waking up. This seems to be a common technique, so I'm beginning to wonder whether these changes

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Nathan Bossart
On Tue, Nov 29, 2022 at 12:02:44PM +, Simon Riggs wrote: > The last important point for me is tests, in src/test/modules > probably. It might be possible to reuse the final state of other > modules' tests to test cleanup, or at least integrate a custodian test > into each module. Of course.

Re: Reducing power consumption on idle servers

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:32 AM Simon Riggs wrote: > Re-attaching patch for bgwriter and walwriter, so it is clear this is > not yet committed. I'm just curious, and not suggesting that 60s wakeups are a problem for the polar ice caps, but why even time out at all? Are the latch protocols

RE: Partial aggregates pushdown

2022-11-29 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Mr.Pyhalov. Thank you for comments. > I've looked through the patch. Overall I like this approach, but have > the following comments. > > 1) Why should we require partialaggfn for min()/max()/count()? We could > just use original functions for a lot of aggregates, and so it would be >

Re: ExecRTCheckPerms() and many prunable partitions

2022-11-29 Thread Amit Langote
On Wed, Nov 30, 2022 at 3:04 AM Alvaro Herrera wrote: > On 2022-Nov-29, Amit Langote wrote: > > > Maybe, we should have the following there so that the PlannedStmt's > > contents don't point into the Query? > > > > newperminfo = copyObject(perminfo); > > Hmm, I suppose if we want a separate

Re: Make mesage at end-of-recovery less scary.

2022-11-29 Thread Kyotaro Horiguchi
At Tue, 22 Nov 2022 16:04:56 -0600, Justin Pryzby wrote in > On Fri, Nov 18, 2022 at 05:25:37PM +0900, Kyotaro Horiguchi wrote: > > + while (*p == 0 && p < pe) > > + p++; > > The bug reported by Andres/cfbot/ubsan is here. > > Fixed in attached. Ur..ou.. -

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-11-29 Thread Justin Pryzby
On Mon, Nov 28, 2022 at 09:08:36PM -0500, Melanie Plageman wrote: > > +pgstat_io_op_stats_collected(BackendType bktype) > > +{ > > + return bktype != B_INVALID && bktype != B_ARCHIVER && bktype != > > B_LOGGER && > > + bktype != B_WAL_RECEIVER && bktype != B_WAL_WRITER; > > >

Re: Strange failure on mamba

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 3:43 PM Thomas Munro wrote: > sigaction(0, NULL, ) s/sigaction/sigprocmask/

Re: Strange failure on mamba

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 2:44 PM Tom Lane wrote: > Now, we certainly cannot think that these are occurring early in > postmaster startup. In the wake of 8acd8f869, we should expect > that there's no further need to call rtld_bind at all in the > postmaster, but seemingly that's not so. It's very

Re: Patch: Global Unique Index

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 09:16:23PM -0500, Tom Lane wrote: > Assuming that you are inserting into index X, and you've checked > index Y to find that it has no conflicts, what prevents another > backend from inserting a conflict into index Y just after you look? > AIUI the idea is to prevent that by

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
On Tue, Nov 29, 2022 at 5:57 PM Bruce Momjian wrote: > On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote: > > My argument is that removing xidStopLimit is totally fine, because it > > only serves to stop the database. What to do about xidWarnLimit is a > > slightly more complex

Re: Patch: Global Unique Index

2022-11-29 Thread Tom Lane
Bruce Momjian writes: > On Tue, Nov 29, 2022 at 06:13:56PM -0500, Tom Lane wrote: >> ... not to mention creating a high probability of deadlocks between >> concurrent insertions to different partitions. If they each >> ex-lock their own partition's index before starting to look into >> other

Re: Patch: Global Unique Index

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 06:13:56PM -0500, Tom Lane wrote: > Greg Stark writes: > > If I understand correctly you're going to insert into the local index > > for the partition using the normal btree uniqueness implementation. > > Then while holding an exclusive lock on the index do lookups on

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Bruce Momjian
On Fri, Nov 18, 2022 at 02:11:50PM -0500, Bruce Momjian wrote: > On Mon, Nov 7, 2022 at 11:04:46PM +0100, Laurenz Albe wrote: > > On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > > > Agreed; new compilation patch attached, including mine and then > > > Robert's suggested rewordings. > > >

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
I wrote: > Thomas Munro writes: >> On Fri, Nov 18, 2022 at 11:08 AM Tom Lane wrote: >>> mamba has been showing intermittent failures in various replication >>> tests since day one. >> I wonder if it's a runtime variant of the other problem. We do >> load_file("libpqwalreceiver", false) before

Re: User functions for building SCRAM secrets

2022-11-29 Thread Michael Paquier
On Tue, Nov 29, 2022 at 09:32:34PM +0100, Daniel Gustafsson wrote: > On the whole I tend to agree with Jacob upthread, while this does provide > consistency it doesn't seem to move the needle for best practices. Allowing > scram_build_secret_sha256('password', 'a', 1); with the password

Re: Collation version tracking for macOS

2022-11-29 Thread Michael Paquier
On Wed, Nov 30, 2022 at 01:50:51PM +1300, Thomas Munro wrote: > The new character added to Version 12.1 is: > > U+32FF SQUARE ERA NAME REIWA > > Version 12.1 adds that single character to enable software to be > rapidly updated to support the new Japanese era name in calendrical > systems and

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:25 PM Jeff Davis wrote: > On Wed, 2022-11-30 at 10:52 +1300, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > > > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > > https://unicode-org.atlassian.net/browse/ICU-22216 > > > > I'm no

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:32 PM Jeff Davis wrote: > On Wed, 2022-11-30 at 10:29 +1300, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > > > Here's what I found for the 'ar' locale (firstminor/lastminor are > > > the > > > icu library versions,

Re: Add LZ4 compression in pg_dump

2022-11-29 Thread Michael Paquier
On Tue, Nov 29, 2022 at 12:10:46PM +, gkokola...@pm.me wrote: > Thank you. Please advice if is preferable to split 0002 in two parts. > I think not but I will happily do so if you think otherwise. This one makes me curious. What kind of split are you talking about? If it makes the code

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 10:29 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > > Here's what I found for the 'ar' locale (firstminor/lastminor are > > the > > icu library versions, firstcollversion/lastcollversion are their > > respective collation versions for the

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Andres Freund
Hi, On 2022-11-28 12:58:48 -0500, Robert Haas wrote: > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > > I think we should consider deprecating the pg_stat_bgwriter columns but > > leaving them in place for a few years. New stuff should only be added to > > pg_stat_checkpointer, but we

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 10:52 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > > One bit of weirdness is that I may have found another ICU > > > problem. > > > > Reported as: > > > >

Re: Patch: Global Unique Index

2022-11-29 Thread Tom Lane
Greg Stark writes: > If I understand correctly you're going to insert into the local index > for the partition using the normal btree uniqueness implementation. > Then while holding an exclusive lock on the index do lookups on every > partition for the new key. Effectively serializing inserts to

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Greg Stark
On Mon, 28 Nov 2022 at 16:53, Peter Geoghegan wrote: > > Imagine if we actually had 64-bit XIDs -- let's assume for a moment > that it's a done deal. This raises a somewhat awkward question: do you > just let the system get further and further behind on freezing, > forever? We can all agree that

Re: Patch: Global Unique Index

2022-11-29 Thread Greg Stark
On Fri, 25 Nov 2022 at 20:03, David Zhang wrote: > > Hi Bruce, > > Thank you for helping review the patches in such detail. > > On 2022-11-25 9:48 a.m., Bruce Momjian wrote: > > Looking at the patch, I am unclear how the the patch prevents concurrent > duplicate value insertion during the

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Greg Stark
On Mon, 28 Nov 2022 at 13:00, Robert Haas wrote: > > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > I vote to just remove them. I think that most people won't update > their queries until they are forced to do so. I don't think it > matters very much when we force them to do that. I

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > One bit of weirdness is that I may have found another ICU problem. > > Reported as: > > https://unicode-org.atlassian.net/browse/ICU-22216 I'm no expert on loader/linker arcana but I have

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 09:00 +1300, Thomas Munro wrote: > I'm struggling to understand what's new about proposal #6. Perhaps it's just a slight variant; I'm not sure. It's not a complete proposal yet. The difference I had in mind is that it would treat the built-in ICU differently from what is

Re: Slow standby snapshot

2022-11-29 Thread Tom Lane
Michail Nikolaev writes: > The small thing I was thinking to add in KnownAssignedXidsCompress is > the assertion like > Assert(MyBackendType == B_STARTUP); Mmm ... given where the call sites are, we have got lots more problems than this if some non-startup process reaches them. I'm not sure

Re: Removing another gen_node_support.pl special case

2022-11-29 Thread Tom Lane
I wrote: > I notice that EquivalenceClass is already marked as no_copy_equal, > which means that gen_node_support.pl can know that emitting a > recursive node-copy or node-compare request is a bad idea. What > do you think of using the patch as it stands, plus a cross-check > that we don't emit

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > Here's what I found for the 'ar' locale (firstminor/lastminor are the > icu library versions, firstcollversion/lastcollversion are their > respective collation versions for the given locale): > > firstminor | lastminor | firstcollversion |

Re: Slow standby snapshot

2022-11-29 Thread Michail Nikolaev
Hello, Tom. > Since we're running out of time in the current commitfest, > I went ahead and changed that, and made the cosmetic fixes > I wanted, and pushed. Great, thanks! The small thing I was thinking to add in KnownAssignedXidsCompress is the assertion like Assert(MyBackendType ==

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-11-29 Thread Jacob Champion
On 11/24/22 00:20, mahendrakar s wrote: > I had validated Github by skipping the discovery mechanism and letting > the provider extension pass on the endpoints. This is just for > validation purposes. > If it needs to be supported, then need a way to send the discovery > document from extension.

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-11-29 Thread Jacob Champion
On 11/23/22 19:45, Andrey Chudnovsky wrote: > Mostly terminology questions here. OAUTHBEARER SASL appears to be the > spec about using OAUTH2 tokens for Authentication. > While any OAUTH2 can generally work, we propose to specifically > highlight that only OIDC providers can be supported, as we

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 08:41 +1300, Thomas Munro wrote: > In terms of user experience, I think that might mean that users of > 'zh' who encounter warnings after a minor upgrade would therefore > really only have the options of REFRESHing and rebuilding, or > downgrading the package, because there's

Re: Slow standby snapshot

2022-11-29 Thread Tom Lane
I wrote: > That seems like a fairly bad idea: it will add extra contention > on ProcArrayLock, and I see no real strong argument that the path > can't get traversed often enough for that to matter. It would > likely be better for KnownAssignedXidsCompress to obtain the lock > for itself, only

Re: Missing update of all_hasnulls in BRIN opclasses

2022-11-29 Thread Justin Pryzby
On Mon, Nov 28, 2022 at 01:13:14AM +0100, Tomas Vondra wrote: > Opinions? Considering this will need to be backpatches, it'd be good to > get some feedback on the approach. I think it's fine, but it would be > unfortunate to fix one issue but break BRIN in a different way. > ---

Re: User functions for building SCRAM secrets

2022-11-29 Thread Daniel Gustafsson
> On 27 Nov 2022, at 06:21, Jonathan S. Katz wrote: > On 11/26/22 2:53 PM, Jonathan S. Katz wrote: >> On 11/16/22 10:09 PM, Michael Paquier wrote: > >>> git diff --check reports some whitespaces. >> Ack. Will fix on the next pass. (I've been transitioning editors, which >> could have resulted

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Tue, 29 Nov 2022 at 03:00, Peter Eisentraut wrote: > Fixed in new patch. There seems to be a small bug in the pg_strtointXX functions in the code that checks that there's at least 1 digit. This causes 0x to be a valid representation of zero. That does not seem to be allowed by the parser,

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 14:34 -0500, Joe Conway wrote: > I understand that it is not easily done, but if the combination of > collprovider + collversion does not represent specific immutable > ordering behavior for a given locale Given the u_versionToString() bug, we know the version string could

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Tue, 29 Nov 2022 at 23:11, Dean Rasheed wrote: > > On Wed, 23 Nov 2022 at 08:56, David Rowley wrote: > > > > On Wed, 23 Nov 2022 at 21:54, David Rowley wrote: > > > I wonder if you'd be better off with something like: > > > > > > while (*ptr && isxdigit((unsigned char) *ptr)) > > >

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:52 AM Robert Haas wrote: > On Tue, Nov 29, 2022 at 1:59 PM Jeff Davis wrote: > > 6. Create a new concept of a "locked down collation" that points at > > some specific collation code (identified by some combination of library > > version and collation version or whatever

Re: Collation version tracking for macOS

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 1:59 PM Jeff Davis wrote: > 6. Create a new concept of a "locked down collation" that points at > some specific collation code (identified by some combination of library > version and collation version or whatever else can be used to identify > it). If a collation is

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:03 AM Jeff Davis wrote: > On Wed, 2022-11-30 at 07:18 +1300, Thomas Munro wrote: > > I think it also includes the CLDR version for *some* locales. From a > > quick look, that includes 'ar', 'ru', 'tr', 'zh'. Jeff, would you > > mind sharing the same table for one of

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > One bit of weirdness is that I may have found another ICU problem. Reported as: https://unicode-org.atlassian.net/browse/ICU-22216 -- Jeff Davis PostgreSQL Contributor Team - AWS

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 11:20 AM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > > > On Tue, Nov 29, 2022 at 10:52 AM Andrey Borodin > wrote: > >> On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: >> > >> > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM >>

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Mon, 2022-11-28 at 19:36 -0800, Jeff Davis wrote: > On Mon, 2022-11-28 at 21:57 -0500, Robert Haas wrote: > > That is ... astonishingly bad. > > https://unicode-org.atlassian.net/browse/CLDR-16175 Oops, reported in CLDR instead of ICU. Moved to:

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 2:32 PM Justin Pryzby wrote: > On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > > Here's a proposed patch to do that. > > If I'm not wrong, you message includes a diffstat but without the patch > itself. D'oh. -- Robert Haas EDB:

Re: Collation version tracking for macOS

2022-11-29 Thread Joe Conway
On 11/29/22 13:59, Jeff Davis wrote: On Tue, 2022-11-29 at 11:27 -0500, Joe Conway wrote: My vote is for something like #5. The collversion should indicate a specific immutable ordering behavior. Easier said than done:

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Justin Pryzby
On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > Here's a proposed patch to do that. If I'm not wrong, you message includes a diffstat but without the patch itself.

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Robert Haas
On Fri, Sep 10, 2021 at 2:39 AM Noah Misch wrote: > Revoke PUBLIC CREATE from public schema, now owned by pg_database_owner. > > This switches the default ACL to what the documentation has recommended > since CVE-2018-1058. Upgrades will carry forward any old ownership and > ACL. Sites that

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 10:52 AM Andrey Borodin wrote: > On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > > > > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > > > 2. Process proc die immediately when a backend is waiting for sync > > > replication

Re: Add non-blocking version of PQcancel

2022-11-29 Thread Daniel Gustafsson
> On 15 Nov 2022, at 12:38, Jelte Fennema wrote: > Here's the correct one.<0001-Add-non-blocking-version-of-PQcancel.patch> This version of the patch no longer applies, a rebased version is needed. -- Daniel Gustafsson https://vmware.com/

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 07:18 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 7:03 AM Jeremy Schneider > wrote: > > It seems to me that the collator_version field is not a good > > version > > identifier to use. > > > > Just taking a quick glance at the ICU home page right now, it shows > >

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 11:27 -0500, Joe Conway wrote: > My vote is for something like #5. The collversion should indicate a > specific immutable ordering behavior. Easier said than done: https://www.postgresql.org/message-id/abddc35a7a447d93e2b8371a1a9052cb48866070.ca...@j-davis.com Even

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread Andrey Borodin
On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > > 2. Process proc die immediately when a backend is waiting for sync > > replication acknowledgement, as it does today, however, upon restart, > > don't

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 12:32 -0500, Robert Haas wrote: > You know more about this than I do, for sure, so don't let my vote > back the project into a bad spot. I'm going back and forth myself. I haven't found a great answer here yet. > But, yeah, the thing you mention > here is what I'm worried

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 1:35 PM Andres Freund wrote: > On 2022-11-29 13:30:02 -0500, Tom Lane wrote: > > I still wonder whether we'll regret losing information about the > > subtransaction tree structure, as discussed in the other thread [1]. > > That seems like the main barrier to proceeding

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 13:30:02 -0500, Tom Lane wrote: > I still wonder whether we'll regret losing information about the > subtransaction tree structure, as discussed in the other thread [1]. > That seems like the main barrier to proceeding with this. Yea, this has me worried. I suspect that we have

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Tom Lane
Simon Riggs writes: > New version of patch, now just a one-line patch! Of course, there's all the documentation and comments that you falsified. Also, what of the SubTransSetParent call in ProcessTwoPhaseBuffer? (The one in ProcArrayApplyXidAssignment is actually okay, though the comment making

Re: Fix for visibility check on 14.5 fails on tpcc with high concurrency

2022-11-29 Thread Dimos Stamatakis
[External Email] On 2022-Nov-25, Dimos Stamatakis wrote: > So does this mean there is no race condition in this case and that > this error is redundant? No, it means I believe a bug exists but that I haven't spent enough time on it to understand what it is. Great! Please keep me posted and

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 7:03 AM Jeremy Schneider wrote: > It seems to me that the collator_version field is not a good version > identifier to use. > > Just taking a quick glance at the ICU home page right now, it shows that > all of the last 5 versions of ICU have included "additions and >

  1   2   >