Re: archive modules

2022-08-23 Thread Nathan Bossart
e a natural place for such a note. But on the other hand, this is generic behavior for any library that is dynamically loaded in a separate process. Overall, I think I'm +1 for this patch. I haven't thought too much about the exact wording, but provided others support it as well, I will try to take a deeper look soon. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-08-23 Thread Nathan Bossart
CheckpointStats.l_dec_ops_end_t = GetCurrentTimestamp(); > + > CheckPointReplicationOrigin(); Shouldn't we include CheckPointReplicationSlots() and CheckPointReplicationOrigin() in this new stat? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-23 Thread Nathan Bossart
false, LOG) != > PGFILETYPE_DIR) > + continue; IIUC an error in get_dirent_type() could cause slots to be skipped here, which is a behavior change. [0] https://postgr.es/m/20220329224832.GA560657%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: O(n) tasks cause lengthy startups and checkpoints

2022-08-24 Thread Nathan Bossart
On Thu, Aug 11, 2022 at 04:09:21PM -0700, Nathan Bossart wrote: > Here is a rebased patch set for cfbot. There are no other differences > between v7 and v8. Another rebase for cfbot. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6810355cb3d1a03326b152aebe3c907

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-24 Thread Nathan Bossart
g in > the earlier patch *_has_zero. That seems reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-24 Thread Nathan Bossart
seem to exist anywhere, so unless someone can > demonstrate otherwise, I'm going to rip it out soon. This is what I found, too, so +1. I've attached a patch for this. [0] https://gcc.gnu.org/onlinedocs/gcc/AArch64-Options.html -- Nathan Bossart Amazon Web Services: https://aws.am

Re: archive modules

2022-08-24 Thread Nathan Bossart
to a user-defined C function). For example, since the archive_library is only loaded by the archiver process, this view will not display any customized options defined by archive modules unless special action is taken to load them into the backend process executin

Re: pg_upgrade failing for 200+ million Large Objects

2022-08-24 Thread Nathan Bossart
and passes the tests. I am aiming to spend some more time on this in the near future. [0] https://commitfest.postgresql.org/39/3841/ -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c index c8a70d9bc1..faf1953e18 10064

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-24 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 10:38:34AM +0700, John Naylor wrote: > On Thu, Aug 25, 2022 at 1:01 AM Nathan Bossart > wrote: >> On Wed, Aug 24, 2022 at 11:07:03AM +0700, John Naylor wrote: >> > - Can a user on ARM64 ever get a runtime fault if the machine attempts >> >

Re: archive modules

2022-08-25 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 03:29:41PM +0200, talk to ben wrote: > Here is a patch with the proposed wording. Here is the same patch with a couple more links. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 92a6d8669d9e5b527a7ac9af7eb359a86526775b Mon Sep 17 00:00:00 200

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-25 Thread Nathan Bossart
<= '0x%x'", > key - 1); > + if (key < 0xFF && !pg_lfind8(key, charbuf, len_no_tail)) > + elog(ERROR, "pg_lfind8() did not find existing element <= > '0x%x'", key); > + if (key < 0xFE && pg_lfind8(key + 1, charbuf, len_no_tail)) > + elog(ERROR, "pg_lfind8() found nonexistent element <= '0x%x'", > key + 1); > +} nitpick: Shouldn't the elog() calls use "==" instead of "<=" for this one? Otherwise, 0001 looks good to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: use SSE2 for is_valid_ascii

2022-08-25 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 04:41:53PM +0700, John Naylor wrote: > v3 applies on top of the v9 json_lex_string patch in [1] and adds a > bit more to that, resulting in a simpler patch that is more amenable > to additional SIMD-capable platforms. LGTM -- Nathan Bossart Amazon Web Servic

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-25 Thread Nathan Bossart
On Fri, Aug 26, 2022 at 10:45:10AM +0700, John Naylor wrote: > On Thu, Aug 25, 2022 at 11:57 AM Nathan Bossart > wrote: >> The ARM literature appears to indicate that Neon support is pretty standard >> on aarch64, and AFAICT it's pretty common to just assume it's

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-25 Thread Nathan Bossart
%3DApXbF02zrc0G0oT9CPof0Q%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 5f973a39d67a744d514ee80e05a1c7f40bc0ebc6 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 25 Aug 2022 22:18:30 -0700 Subject: [PATCH v3 1/2] abstract architecture-specific implementation deta

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-26 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 11:13:47PM -0700, Nathan Bossart wrote: > Here is a new patch set that applies on top of v9-0001 in the > json_lex_string patch set [0] and v3 of the is_valid_ascii patch [1]. Here is a rebased patch set that applies to HEAD. -- Nathan Bossart Amazon Web Services:

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-27 Thread Nathan Bossart
ub; > > +#elif defined(USE_NEON) > + > + /* use the same approach as the USE_SSE2 block above */ > + sub = vqsubq_u8(v, vector8_broadcast(c)); > + result = vector8_has_zero(sub); > > I think we should invent a helper that does saturating subtraction and > call that,

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-27 Thread Nathan Bossart
ut a run-time check, > + * which seems not worth the trouble for now. > + */ > +#include > +#define USE_NEON > ... Thank you for the analysis! I'll do it this way in the next patch set. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-27 Thread Nathan Bossart
On Sun, Aug 28, 2022 at 10:39:09AM +1200, Thomas Munro wrote: > On Sun, Aug 28, 2022 at 10:12 AM Nathan Bossart > wrote: >> Yup. The problem is that AFAICT there's no equivalent to >> _mm_movemask_epi8() on aarch64, so you end up with something like >> &g

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-27 Thread Nathan Bossart
Here is a new patch set in which I've attempted to address all feedback. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From a5f381097819db05b6e47418597cd56bab411fad Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 25 Aug 2022 22:18:30 -0700 Subject: [PATC

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-28 Thread Nathan Bossart
d before "After." Otherwise, LGTM. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-08-28 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 10:12:00AM +0530, Bharath Rupireddy wrote: > Please review the v12 patch attached. LGTM. I've marked this as ready-for-committer. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-28 Thread Nathan Bossart
On Sat, Aug 27, 2022 at 02:06:32PM +0530, Bharath Rupireddy wrote: > PSA v17 patch with reorderbuffer.c changes reverted. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Slight refactoring of state check in pg_upgrade check_ function

2022-08-28 Thread Nathan Bossart
y, just code consistency. The patch looks reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: effective_multixact_freeze_max_age issue

2022-08-28 Thread Nathan Bossart
e cases (maybe when vacuum_freeze_min_age is set very high and there are long-running transactions), ISTM it might not have tremendously much impact. Is the intent to create some sort of long-term behavior change for autovacuum, or is this mostly aimed towards consistency among the cutoff calculations? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-28 Thread Nathan Bossart
rkaround SIMD architectures frequent lack of unsigned comparison. Done. > + * Return the result of subtracting the respective elements of the input > + * vectors using saturation. > > I wonder if we should explain briefly what saturating arithmetic is. I > had never encountered it

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-29 Thread Nathan Bossart
On Mon, Aug 29, 2022 at 05:49:46PM +0700, John Naylor wrote: > Bowerbird just reported the same error, so I went ahead and pushed a > fix with this. Thanks! I've attached a follow-up patch with a couple of small suggestions. -- Nathan Bossart Amazon Web Services: https://aws.amaz

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-29 Thread Nathan Bossart
ssible grantors > > ...which is never supposed to happen. The grantor column in the expected test output refers to "rhaas", so I imagine the test only passes on your machines. Should we create a new grantor role for this test? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

introduce bufmgr hooks

2022-08-29 Thread Nathan Bossart
page. * bufmgr_invalidate_hook: called within InvalidateBuffer(). The attached patch is a first attempt at introducing these hooks with acceptable names, placements, arguments, etc. Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From d1683d73df8930927a464555f256c8c91c7cf24e

Re: effective_multixact_freeze_max_age issue

2022-08-29 Thread Nathan Bossart
On Mon, Aug 29, 2022 at 10:25:50AM -0700, Peter Geoghegan wrote: > On Sun, Aug 28, 2022 at 4:14 PM Nathan Bossart > wrote: >> The idea seems sound to me, and IMO your patch simplifies things nicely, >> which might be reason enough to proceed with it. > > It is primarily

Re: archive modules

2022-08-29 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 01:06:00PM -0700, Nathan Bossart wrote: > On Thu, Aug 25, 2022 at 03:29:41PM +0200, talk to ben wrote: >> Here is a patch with the proposed wording. > > Here is the same patch with a couple more links. I would advise that you create a commitfest entry fo

Re: identifying the backend that owns a temporary schema

2022-08-29 Thread Nathan Bossart
_get_backend_*() functions even if we do end up adjusting pg_stat_activity. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-08-30 Thread Nathan Bossart
On Tue, Aug 30, 2022 at 09:49:20AM +0200, Benoit Lobréau wrote: > Ok done, https://commitfest.postgresql.org/39/3856/ (is that fine if I > added you as a reviewer ?) Of course. I've marked it as ready-for-committer. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-30 Thread Nathan Bossart
another role, the current role effectively does not have ADMIN OPTION on the other role. Is this correct? As I'm writing this down, I think it's beginning to make more sense to me, so this might just be a matter of under-caffeination. But perhaps some extra comments or documentation wouldn't be out of the question... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-30 Thread Nathan Bossart
ental model I was beginning to form. Okay, now to take a closer look at the patch... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-30 Thread Nathan Bossart
On Tue, Aug 30, 2022 at 08:33:46AM -0400, Robert Haas wrote: > Third try. Now that I have this grantor stuff fresh in my mind, this patch looks good to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: introduce bufmgr hooks

2022-08-30 Thread Nathan Bossart
turn whether smgrread() needs to be called. Would that address your concern? > I think that the INSTR_* section should enclose the hook call as it is > still an I/O operation in the view of the core. Okay, will do. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: effective_multixact_freeze_max_age issue

2022-08-30 Thread Nathan Bossart
or with Multis anymore (not since > the function was taught to deal with the question of whether caller's > VACUUM will be aggressive). > > Going to push this in the next day or so. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-30 Thread Nathan Bossart
e, the vectorized less-equal check compared to 0x1F, but > the byte-wise path did so with < 32. I made them both "less-equal 31" > for consistency. I'll commit this by the end of the week unless anyone > has a better idea about testing. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-31 Thread Nathan Bossart
on to get_dirent_type(). I don't at all > understand why e.g. the above change to make copydir() silently skip over > files it can't stat is ok? In this example, the call to get_dirent_type() should ERROR if the call to lstat() fails (the "elevel" argument is set to ERROR). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

make additional use of optimized linear search routines

2022-09-01 Thread Nathan Bossart
_lfind*() suite of functions (and anywhere else where SIMD might be useful, really). If you have any ideas in mind, I'm all ears. [0] https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/port/pg_lfind.h [1] https://postgr.es/m/20220802221301.GA742739%40nathanxps13 -- N

Re: introduce bufmgr hooks

2022-09-01 Thread Nathan Bossart
requisite for this change, I'm happy to give it a try. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: allowing for control over SET ROLE

2022-09-01 Thread Nathan Bossart
However, that seems like it'd create a weird discrepancy. If you have ADMIN but not INHERIT or SET, you'd still be able to grant membership with or without INHERIT, but you wouldn't be able to grant SET. In the end, I guess I agree with you that it's "fundamentally difficult to allow people to administer a set of privileges without giving them the ability to usurp those privileges..." -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: introduce bufmgr hooks

2022-09-01 Thread Nathan Bossart
On Tue, Aug 30, 2022 at 03:22:43PM -0700, Nathan Bossart wrote: > Okay, will do. v2 attached. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From df1556c6da69f8c0aae9f8878f24e21907cf4d89 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 11 Aug 2022 16:24:26 -0700 S

Re: postgres_fdw hint messages

2022-09-01 Thread Nathan Bossart
I will work on polishing it up a bit. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c index e323fdd0e6..7fe42cb732 100644 --- a/contrib/dblink/dblink.c +++ b/contrib/dblink/dblink.c @@ -2006,28 +2006,31 @@ dblink_fdw_va

Re: postgres_fdw hint messages

2022-09-01 Thread Nathan Bossart
ate that the string is too long without ERROR-ing (e.g., by returning -1). If the user-provided string is too long, we'd just omit the hint. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: postgres_fdw hint messages

2022-09-02 Thread Nathan Bossart
On Thu, Sep 01, 2022 at 04:31:20PM -0700, Nathan Bossart wrote: > On Thu, Sep 01, 2022 at 07:08:49PM -0400, Tom Lane wrote: >> (1) there probably needs to be some threshold of closeness, so we don't >> offer "foobar" when the user wrote "huh" > > Ag

Re: O(n) tasks cause lengthy startups and checkpoints

2022-09-02 Thread Nathan Bossart
On Wed, Aug 24, 2022 at 09:46:24AM -0700, Nathan Bossart wrote: > Another rebase for cfbot. And another. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 63a470be1ac8af3b12684f136f70b2d7b6f87b81 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 5 Jan 2022 19

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-09-02 Thread Nathan Bossart
On Fri, Sep 02, 2022 at 05:09:14PM +0900, Michael Paquier wrote: > I had a few hours and I've spent them looking at what you had here in > details, and there were a few things I have tweaked before applying > the patch. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: make additional use of optimized linear search routines

2022-09-02 Thread Nathan Bossart
ized with pg_lfind*(). > But 1) these places have issues that arguments differ in signedness, and > 2) I'm not sure whether they are performance-sensitive or not. Yeah, I doubt that these typically deal with many elements or are performance-sensitive enough to bother with. -- Natha

Re: introduce bufmgr hooks

2022-09-02 Thread Nathan Bossart
On Thu, Sep 01, 2022 at 05:34:03PM -0700, Andres Freund wrote: > On 2022-09-01 13:11:50 -0700, Nathan Bossart wrote: >> On Wed, Aug 31, 2022 at 08:29:31AM -0700, Andres Freund wrote: >> > I also think it'll >> > make it harder to improve things in this area, whic

Re: Backpatching nbtree VACUUM (page deletion) hardening

2022-09-02 Thread Nathan Bossart
On Fri, Sep 02, 2022 at 02:13:15PM -0700, Peter Geoghegan wrote: > I think that this should be backpatched now, to protect users from > particularly nasty problems that hitting the error eventually leads > to. +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: postgres_fdw hint messages

2022-09-02 Thread Nathan Bossart
the distance limit to 4. This seems to prevent the silly suggestions (e.g., "host" for "foo") while retaining the more believable ones (e.g., "passfile" for "password"), at least for the small set of examples covered in the tests. -- Nathan Bossart Amazon W

Re: First draft of the PG 15 release notes

2022-09-04 Thread Nathan Bossart
@@ -1255,7 +1255,7 @@ Author: Jeff Davis Add predefined role pg_checkpointer + linkend="predefined-roles-table">pg_checkpoint that allows members to run CHECKPOINT (Jeff Davis) -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-05 Thread Nathan Bossart
6 privilege bits. I was able to get this approach building and passing tests in a few minutes, but there might be performance/space concerns. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From f19eea3f3148916a79d002094ca4eb4aa98af753 Mon Sep 17 00:00:00 2001 From: Nathan B

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-06 Thread Nathan Bossart
I don't think this work needs to be tied to the VACUUM/ANALYZE stuff, but I am interested in it and hope to take it on at some point. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-06 Thread Nathan Bossart
On Mon, Sep 05, 2022 at 11:56:30AM -0700, Nathan Bossart wrote: > Here is a first attempt at allowing users to grant VACUUM or ANALYZE > per-relation. Overall, this seems pretty straightforward. I needed to > adjust the permissions logic for VACUUM/ANALYZE a bit, which causes som

Re: Switching XLog source from archive to streaming when primary available

2022-09-06 Thread Nathan Bossart
retry_interval should be the lower bound for this GUC, too. Or maybe WaitForWALToBecomeAvailable() should make sure that the standby makes at least once attempt to restore the file from archive before switching to streaming replication. [0] https://www.postgresql.org/docs/current/warm-standby.html#STANDBY-SERVER-OPERATION -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-07 Thread Nathan Bossart
eneral interest in the patches. From the previous discussion, it sounds like we want to first use a distinct set of bits for each catalog table. Is that what I should proceed with? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-07 Thread Nathan Bossart
84c80348c5d93f4.camel%40j-davis.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-07 Thread Nathan Bossart
.es/m/18391.1521419120%40sss.pgh.pa.us [1] https://postgr.es/m/11414.1526422062%40sss.pgh.pa.us -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From d1ec82eeb12a15bb4f39bbf0d88ae32bd554418d Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 7 Sep 2022 22:25:29 -0700 Subject: [PAT

Re: Switching XLog source from archive to streaming when primary available

2022-09-08 Thread Nathan Bossart
On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote: > On Wed, Sep 7, 2022 at 3:27 AM Nathan Bossart > wrote: >> It's not clear to me how this is expected to interact with the pg_wal phase >> of standby recovery. As the docs note [0], standby servers

Re: pg_upgrade failing for 200+ million Large Objects

2022-09-08 Thread Nathan Bossart
going discussion about changing aclitem. I still think this is a problem worth fixing, but it's not yet clear how to proceed. [0] https://postgr.es/m/227228.1616259220%40sss.pgh.pa.us [1] https://postgr.es/m/20181122001415.ef5bncxqin2y3esb%40alap3.anarazel.de -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_upgrade failing for 200+ million Large Objects

2022-09-08 Thread Nathan Bossart
On Thu, Sep 08, 2022 at 04:29:10PM -0700, Jacob Champion wrote: > On Thu, Sep 8, 2022 at 4:18 PM Nathan Bossart > wrote: >> IIUC the main benefit of this approach is that it isn't dependent on >> binary-upgrade mode, which seems to be a goal based on the discussion &

Re: Avoid overhead with fprintf related functions

2022-09-09 Thread Nathan Bossart
bset of these changes in hot paths. [0] https://postgr.es/m/CAApHDvp2THseLvCc%2BTcYFBC7FKHpHTs1JyYmd2JghtOVhb5WGA%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Switching XLog source from archive to streaming when primary available

2022-09-09 Thread Nathan Bossart
On Fri, Sep 09, 2022 at 12:14:25PM +0530, Bharath Rupireddy wrote: > On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi > wrote: >> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart >> wrote in >> > My general point is that we should probably offer some basic preven

Re: Possible crash on standby

2022-09-09 Thread Nathan Bossart
ctive to false in an 'else' block, but that doesn't seem necessary if XLogShutdownWalRcv() is safe to call unconditionally. So, unless there is a bigger problem that I'm not seeing, +1 for your patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Summary function for pg_buffercache

2022-09-09 Thread Nathan Bossart
t we focus on saving the memory first and then think about the > performance, if necessary. +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Possible crash on standby

2022-09-09 Thread Nathan Bossart
https://commitfest.postgresql.org/39/3814 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Switching XLog source from archive to streaming when primary available

2022-09-09 Thread Nathan Bossart
On Fri, Sep 09, 2022 at 11:07:00PM +0530, Bharath Rupireddy wrote: > On Fri, Sep 9, 2022 at 10:29 PM Nathan Bossart > wrote: >> IMO the timeout approach would be more intuitive for users. When it comes >> to archive recovery, "WAL segment" isn't a standard unit

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-09 Thread Nathan Bossart
uces both. [0] https://postgr.es/m/20220908055035.GA2100193%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid overhead with fprintf related functions

2022-09-09 Thread Nathan Bossart
On Fri, Sep 09, 2022 at 05:53:54PM -0400, Tom Lane wrote: > Ranier Vilela writes: >> Em sex., 9 de set. de 2022 às 13:20, Nathan Bossart < >> nathandboss...@gmail.com> escreveu: >>> I agree with David [0]. But if you can demonstrate a performance gain, >>>

Re: Support for Rust

2022-09-10 Thread Nathan Bossart
at > I'm doing something wrong, and the Rust compiler has been excellent at > preventing bugs. There was some discussion about Rust back in 2017 [0] that you might be interested in. [0] https://www.postgresql.org/message-id/flat/CAASwCXdQUiuUnhycdRvrUmHuzk5PsaGxr54U4t34teQjcjb%3DAQ%40m

Re: archive modules

2022-09-10 Thread Nathan Bossart
On Sat, Sep 10, 2022 at 04:44:16PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Of course. I've marked it as ready-for-committer. > > Pushed with a bit of additional wordsmithing. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: postgres_fdw hint messages

2022-09-13 Thread Nathan Bossart
the patch that uses the column phrasing. I wasn't sure whether "reference" was the right word to use in this context, but I used it for now for consistency with the column hints. I think "specify" or "use" would work as well. -- Nathan Bossart Amazon Web Services: htt

Re: archive modules

2022-09-14 Thread Nathan Bossart
postmaster startup anyway. Nonetheless it's likely * that we have odd behaviors such as unexpected GUC ordering dependencies. [0] https://postgr.es/m/CA%2BTgmoaf4Y7_U%2B_W%2BSg5DoAta_FMssr%3D52mx7-_tJnfaD1VubQ%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-09-14 Thread Nathan Bossart
believe this applies to archive_library, too. > - for segments to complete like does. > + for segments to complete like and > +does. nitpick: s/does/do -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 04:47:23PM -0400, Tom Lane wrote: > Peter Eisentraut writes: >> On 14.09.22 22:03, Nathan Bossart wrote: >>> I originally did it this way, but changed it based on this feedback [0]. I >>> have no problem with the general idea, but the recovery_t

Re: archive modules

2022-09-14 Thread Nathan Bossart
On Wed, Sep 14, 2022 at 06:12:09PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Wed, Sep 14, 2022 at 04:47:23PM -0400, Tom Lane wrote: >>> Yeah, the objection there is only to trying to enforce such >>> interrelationships in GUC hooks. In this case it se

Re: postgres_fdw hint messages

2022-09-16 Thread Nathan Bossart
On Fri, Sep 16, 2022 at 03:54:53PM +0200, Peter Eisentraut wrote: > I don't think we need a verb there at all. I committed it without a verb. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

introduce optimized linear search functions that return index of matching element

2022-09-16 Thread Nathan Bossart
ector8_highbit_mask() can be removed as well. * I simplified the test for pg_lfind8_ge_idx() by making it look a little more like the test for pg_lfind32(). I wasn't sure about the use of rand() and qsort(), and overall it just felt a little too complicated to me. I've tested all three

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-17 Thread Nathan Bossart
hread can remain focused on the radix tree stuff. [0] https://www.postgresql.org/message-id/20220917052903.GA3172400%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: remove more archiving overhead

2022-09-17 Thread Nathan Bossart
ation change only covers archive_library. How are users of > archive_command supposed to handle this? I believe users of archive_command need to do something similar to what is described here. However, it might be more reasonable to expect archive_command users to simply return false when there is a pre-existing file, as the deleted text notes. IIRC that is why I added that sentence originally. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-19 Thread Nathan Bossart
of pg_walfile_name_offset(). Perhaps we could add a function like pg_walfile_offset_lsn() that accepts a WAL file name and byte offset and returns the LSN. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-19 Thread Nathan Bossart
On Mon, Sep 19, 2022 at 03:16:42PM -0700, Nathan Bossart wrote: > It seems like you want the opposite of pg_walfile_name_offset(). Perhaps > we could add a function like pg_walfile_offset_lsn() that accepts a WAL > file name and byte offset and returns the LSN. Like so... -- Natha

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-19 Thread Nathan Bossart
ow we're trying to dump a problem > that's a decade in the making on Nathan. I'm not exactly sure what to > propose as an alternative, but that doesn't seem quite fair. Are there any concerns with simply expanding AclMode to 64 bits, as done in v5 [0]? [0] https://postgr.es/m/20220908055035.GA2100193%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-20 Thread Nathan Bossart
hange. I might be missing something in my patch, or maybe I am misunderstanding how arrays of aclitems are stored on disk. > Some comments in utils/acl.h need a refresh as the number of lower and > upper bits looked at from ai_privs changes. Oops, I missed that one. I fixed it in the attached patch

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-09-20 Thread Nathan Bossart
Here is a rebased patch for cfbot. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 9ae1f5409ddeee17b99a9565247da885cbb86be9 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 22 Mar 2022 15:35:34 -0700 Subject: [PATCH v6 1/1] Disallow setting XMAX_COMMITTED

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-09-20 Thread Nathan Bossart
be more + * performant. + */ +#define PWRITEV_BLCKSZ XLOG_BLCKSZ This seems like something we should sort out now instead of leaving as future work. Given your recent note, I think we should just use XLOG_BLCKSZ and PGAlignedXLogBlock and add a comment about the performance findings with different buf

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-09-20 Thread Nathan Bossart
On Tue, Sep 20, 2022 at 04:00:26PM -0700, Nathan Bossart wrote: > On Mon, Aug 08, 2022 at 06:10:23PM +0530, Bharath Rupireddy wrote: >> I'm attaching v5 patch-set. I've addressed review comments received so >> far and fixed a compiler warning that CF bot complained about

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-20 Thread Nathan Bossart
On Tue, Sep 20, 2022 at 11:05:33AM -0700, Nathan Bossart wrote: > On Tue, Sep 20, 2022 at 02:45:52PM +0900, Michael Paquier wrote: >> Any impact for the column sizes of the catalogs holding ACL >> information? Just asking while browsing the patch set. > > Since each acli

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-20 Thread Nathan Bossart
On Tue, Sep 20, 2022 at 04:31:17PM -0700, Nathan Bossart wrote: > On Tue, Sep 20, 2022 at 11:05:33AM -0700, Nathan Bossart wrote: >> On Tue, Sep 20, 2022 at 02:45:52PM +0900, Michael Paquier wrote: >>> Any impact for the column sizes of the catalogs holding ACL >>> info

Re: predefined role(s) for VACUUM and ANALYZE

2022-09-20 Thread Nathan Bossart
r a few tests, I haven't discovered any changes to the output of pg_column_size(). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread Nathan Bossart
is allocated for all of the chunks, so there wouldn't be any danger in searching all them and discarding any results >= node->count. Granted, we're depending on the number of chunks always being a multiple of elements-per-vector in order to avoid the tail path, but that seem

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
also yield some rather substantial improvements for this kind of data. Presumably a generic WAL record compression mechanism could be reused for other large records, too. That could be much easier than devising a deduplication strategy for every record type. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: make additional use of optimized linear search routines

2022-09-21 Thread Nathan Bossart
n looking at a couple of simple patterns across the tree but > no luck here either. Well, if someone spots something, it could > always be done later. For now I have applied the bits discussed on > this thread. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
offset based on the maximum in the array. Furthermore, we might be able to make use of SIMD instructions to mitigate any performance penalty. I'm tempted to start by just using single-byte offsets when possible since that should be relatively simple while still yielding a decent improvement for many workloads. What do you think? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2022-09-21 Thread Nathan Bossart
On Wed, Sep 21, 2022 at 02:11:36PM -0700, Peter Geoghegan wrote: > On Wed, Sep 21, 2022 at 1:14 PM Nathan Bossart > wrote: >> Presumably a >> generic WAL record compression mechanism could be reused for other large >> records, too. That could be much easier than

Re: archive modules

2022-09-22 Thread Nathan Bossart
arget_xxx > settings. I don't think silently turning off one setting by setting another > is a good behavior. Peter, would you like to proceed with something like [0] to resolve this? If so, I will work on cleaning the patch up. [0] https://postgr.es/m/20220914222736.GA3042279%40nathanxps13

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Nathan Bossart
son really doesn't like it when there are things leftover from configure/make. Whenever I switch from make to meson, I have to run 'make maintainer-clean'. Otherwise, all of my usual build options, ccache, etc. are working just like before. Nice work! [0] https://github.com/mes

Re: [RFC] building postgres with meson - v13

2022-09-22 Thread Nathan Bossart
On Thu, Sep 22, 2022 at 01:28:09PM -0700, Andres Freund wrote: > On 2022-09-22 13:05:33 -0700, Nathan Bossart wrote: >> * I'm using an Ubuntu-based distribution, and the version of meson that apt >> installed was not new enough for Postgres. I ended up cloning meson [0] >

<    7   8   9   10   11   12   13   14   15   16   >