Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Nathan Bossart
he stuff in common/int.h) to help prevent this sort of thing in the future. But that'd require you to realize that your code is at risk of overflow, at which point it's probably just as easy to restructure the logic like you've done here. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: backtrace_on_internal_error

2023-12-05 Thread Nathan Bossart
y to turn on backtraces for everything, or at least everything above a certain logging level. That would primarily be useful for when you don't know exactly which C function is producing the error. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: CRC32C Parallel Computation Optimization on ARM

2023-12-04 Thread Nathan Bossart
On Mon, Dec 04, 2023 at 07:27:01AM +, Xiang Gao wrote: > This is the latest patch. Looking forward to your feedback, thanks! Thanks for the new patch. I am hoping to spend much more time on this in the near future... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

introduce dynamic shared memory registry

2023-12-04 Thread Nathan Bossart
I thought it might be useful for extensions for caching, etc. and wanted to see whether there was any interest in the feature. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From b63c28303384636699f2f514e71b62829346be4b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 11 Oct

Re: optimize atomic exchanges

2023-12-04 Thread Nathan Bossart
On Mon, Dec 04, 2023 at 12:18:05PM -0600, Nathan Bossart wrote: > Barring objections or additional feedback, I think I'm inclined to press > forward with this one and commit it in the next week or two. I'm currently > planning to keep the inline assembly, but I'm considering

Re: optimize atomic exchanges

2023-12-04 Thread Nathan Bossart
On Thu, Nov 30, 2023 at 10:35:22PM -0600, Nathan Bossart wrote: > One thing on my mind is whether we should bother with the inline assembly > versions. It looks like gcc has had __atomic since 4.7.0 (2012), so I'm > not sure we gain much from them. OTOH they are pretty simple and seem &

Re: [PATCH] pg_convert improvement

2023-12-04 Thread Nathan Bossart
Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Is WAL_DEBUG related code still relevant today?

2023-12-02 Thread Nathan Bossart
ody using it, I think we need to fix the > compilation and test failure issues, and start testing this code > (perhaps I can think of setting up a buildfarm member to help here). +1 for at least fixing the code and tests, provided we decide to keep it. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-02 Thread Nathan Bossart
pt_arg +{ +$$ = makeDefElem("format", $2, @1); +} ; copy_generic_opt_arg: -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-01 Thread Nathan Bossart
it's unfortunate that this further complicates the branching in CopyOneRowTo(), but after some quick glances at the code, I'm not sure it's worth refactoring a bunch of stuff to make this nicer. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] pg_convert improvement

2023-12-01 Thread Nathan Bossart
me, so I will plan on committing it shortly. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: optimize atomic exchanges

2023-11-30 Thread Nathan Bossart
On Thu, Nov 30, 2023 at 07:56:27PM -0800, Andres Freund wrote: > On 2023-11-30 21:18:15 -0600, Nathan Bossart wrote: >> Some rudimentary tests show a >40% speedup with this patch on x86_64. > > On bigger machines, with contention, the wins are likely much higher. I see > tw

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-11-30 Thread Nathan Bossart
NIL && \ var##__state.i < var##__state.l->length && \ (var = lfirst_node(type, ##__state.l->elements[var##__state.i]), true));\ var##__state.i++) There might be a way to use foreach_internal for foreach_node, too, but this is

Re: optimize atomic exchanges

2023-11-30 Thread Nathan Bossart
On Wed, Nov 29, 2023 at 03:29:05PM -0600, Nathan Bossart wrote: > I haven't done any sort of performance testing on this yet. Some > preliminary web searches suggest that there is unlikely to be much > difference between cmpxchg and xchg, but presumably there's some difference >

Re: CRC32C Parallel Computation Optimization on ARM

2023-11-30 Thread Nathan Bossart
On Thu, Nov 23, 2023 at 08:05:26AM +, Xiang Gao wrote: > On Date: Wed, 22 Nov 2023 15:06:18PM -0600, Nathan Bossart wrote: >>pg_crc32c_armv8.o: CFLAGS += ${CFLAGS_CRC} ${CFLAGS_CRYPTO} > > It does not work correctly. CFLAGS ='-march=armv8-a+crc, > -march=armv8-a+crypto', w

postgres_fdw test timeouts

2023-11-30 Thread Nathan Bossart
, and I didn't find any previous reports about this in the archives after lots of searching, so I thought I'd at least park these notes here in case anyone else has ideas. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [HACKERS] Changing references of password encryption to hashing

2023-11-29 Thread Nathan Bossart
s are better than those ones and what they protect against > and what risks remain." That would be awesome. +1. IMO the "Password Authentication" section [0] does this pretty well already. [0] https://www.postgresql.org/docs/devel/auth-password.html -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

optimize atomic exchanges

2023-11-29 Thread Nathan Bossart
On Fri, Nov 10, 2023 at 08:55:29PM -0600, Nathan Bossart wrote: > On Fri, Nov 10, 2023 at 06:48:39PM -0800, Andres Freund wrote: >> Yes. We should optimize pg_atomic_exchange_u32() one of these days - it can >> be >> done *far* faster than a cmpxchg. When I was adding t

Re: Fix assertion in autovacuum worker

2023-11-29 Thread Nathan Bossart
On Tue, Nov 28, 2023 at 06:48:59PM -0800, Andres Freund wrote: > On 2023-11-28 20:42:47 -0600, Nathan Bossart wrote: >> Right. Perhaps we could add a LWLockReleaseAll() to >> pgstat_shutdown_hook() instead of the autovacuum code, but I'm afraid that >> is still just a hack

add AVX2 support to simd.h

2023-11-29 Thread Nathan Bossart
On Wed, Nov 22, 2023 at 12:49:35PM -0600, Nathan Bossart wrote: > On Wed, Nov 22, 2023 at 02:54:13PM +0200, Ants Aasma wrote: >> For reference, executing the page checksum 10M times on a AMD 3900X CPU: >> >> clang-14 -O2 4.292s (17.8 GiB/s) >> clang-14

Re: common signal handler protection

2023-11-28 Thread Nathan Bossart
r > we've called InitProcessGlobals(). But perhaps that should be commented upon > somewhere? Good call. I expanded on the MyProcPid assertion in wrapper_handler() a bit. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 8c36c9470bee558bcab3fa70d456ca976dc02ac2 Mon Se

Re: Fix assertion in autovacuum worker

2023-11-28 Thread Nathan Bossart
On Tue, Nov 28, 2023 at 04:03:49PM -0800, Andres Freund wrote: > On 2023-11-28 16:05:16 -0600, Nathan Bossart wrote: >> From a glance, it looks to me like the problem is that pgstat_shutdown_hook >> is registered as a before_shmem_exit callback, while ProcKill is registered >>

Re: Fix assertion in autovacuum worker

2023-11-28 Thread Nathan Bossart
particular issue, but I'm wondering whether there's a bigger problem here. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: common signal handler protection

2023-11-28 Thread Nathan Bossart
Here is a new patch set with feedback addressed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From e4bea5353c2685457545b67396095e9b96156982 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 28 Nov 2023 14:58:20 -0600 Subject: [PATCH v3 1/3] Check that MyProc

Re: archive modules loose ends

2023-11-28 Thread Nathan Bossart
Here is a new version of the patch with feedback addressed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 3cda5bb87c82738ad6f8a082ef5dfeb49cd51392 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 28 Nov 2023 11:17:13 -0600 Subject: [PATCH v3 1/1] archiver except

Re: autovectorize page checksum code included elsewhere

2023-11-27 Thread Nathan Bossart
On Mon, Nov 27, 2023 at 03:21:53PM -0600, Nathan Bossart wrote: > On Sat, Nov 25, 2023 at 02:09:14PM +0700, John Naylor wrote: >> Sorry, I wasn't clear, I mean: detect that a packager passed >> "-march=x86_64-v2" in the CFLAGS, so that a symbol in header files >> wo

Re: common signal handler protection

2023-11-27 Thread Nathan Bossart
Thanks for taking a look. On Wed, Nov 22, 2023 at 02:59:45PM -0800, Andres Freund wrote: > On 2023-11-22 15:59:44 -0600, Nathan Bossart wrote: >> +/* >> + * Except when called with SIG_IGN or SIG_DFL, pqsignal() sets up this >> function >> + * as the handler for

Re: autovectorize page checksum code included elsewhere

2023-11-27 Thread Nathan Bossart
On Sat, Nov 25, 2023 at 02:09:14PM +0700, John Naylor wrote: > On Thu, Nov 23, 2023 at 11:51 PM Nathan Bossart > wrote: >> >> On Thu, Nov 23, 2023 at 05:50:48PM +0700, John Naylor wrote: >> > On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart >> > wrot

Re: locked reads for atomics

2023-11-27 Thread Nathan Bossart
a pg_atomic_flag instead for 0002, but that code seems intended for a slightly different use-case and has more complicated barrier semantics.) -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 53f717e84a3aa912f9d89dce4be7f77d9bffb3c2 Mon Sep 17 00:00:00 2001 From: Nathan Bossart D

Re: Do away with a few backwards compatibility macros

2023-11-27 Thread Nathan Bossart
ng said that, I'm all +1 if the v1 patch proposed in this thread gets in. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: autovectorize page checksum code included elsewhere

2023-11-23 Thread Nathan Bossart
On Thu, Nov 23, 2023 at 05:50:48PM +0700, John Naylor wrote: > On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart > wrote: >> One half-formed idea I have is to introduce some sort of ./configure flag >> that enables all the newer instructions that your CPU understands. > >

Re: common signal handler protection

2023-11-22 Thread Nathan Bossart
On Tue, Nov 21, 2023 at 04:40:06PM -0600, Nathan Bossart wrote: > cfbot seems unhappy with this on Windows. IIUC we need to use > PG_SIGNAL_COUNT there instead, but I'd like to find a way to have just one > macro for all platforms. Here's an attempt at fixing the Windows build. -

Re: CRC32C Parallel Computation Optimization on ARM

2023-11-22 Thread Nathan Bossart
On Wed, Nov 22, 2023 at 10:16:44AM +, Xiang Gao wrote: > On Date: Fri, 10 Nov 2023 10:36:08AM -0600, Nathan Bossart wrote: >>+__attribute__((target("+crc+crypto"))) >> >>I'm not sure we can assume that all compilers will understand this, and I'm >>

Re: autovectorize page checksum code included elsewhere

2023-11-22 Thread Nathan Bossart
pr08mb6991.eurprd08.prod.outlook.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: common signal handler protection

2023-11-21 Thread Nathan Bossart
On Tue, Nov 21, 2023 at 03:20:08PM -0600, Nathan Bossart wrote: > +#ifdef NSIG > +#define PG_NSIG (NSIG) > +#else > +#define PG_NSIG (64) /* XXX: wild guess */ > +#endif > + Assert(signo < PG_NSIG); cfbot seems unhappy with this on Windows.

common signal handler protection

2023-11-21 Thread Nathan Bossart
signal() into a void function, but IIUC that would require an SONAME bump. For now, I've just documented the bogosity of the return values. Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 5f46dc0150aba3ef75053e91f7c9a4d6624e2c4f Mon Sep 17 00:00:00 2001 From: Na

Re: simplehash: SH_OPTIMIZE_REPEAT for optimizing repeated lookups of the same key

2023-11-21 Thread Nathan Bossart
On Mon, Nov 20, 2023 at 10:37:47PM -0800, Jeff Davis wrote: > It would be interesting to know how often it's a good idea to turn it > on, though. I could try turning it on for various other uses of > simplehash, and see where it tends to win. That seems worthwhile to me. -- Natha

Re: Do away with a few backwards compatibility macros

2023-11-21 Thread Nathan Bossart
ounds > about right to me, but maybe somebody has an argument for thinking about > it differently. That seems reasonable to me. I don't think we need to mandate that backwards-compatibility support be removed as soon as it is eligible, but it can be considered fair game at that point. --

Re: Do away with a few backwards compatibility macros

2023-11-20 Thread Nathan Bossart
On Thu, Nov 16, 2023 at 09:46:22AM -0600, Nathan Bossart wrote: > On Thu, Nov 16, 2023 at 07:11:41PM +0530, Bharath Rupireddy wrote: >> After a recent commit 6a72c42f (a related discussion [1]) which >> removed MemoryContextResetAndDeleteChildren(), I think there are a >> cou

Re: simplehash: SH_OPTIMIZE_REPEAT for optimizing repeated lookups of the same key

2023-11-20 Thread Nathan Bossart
t implements the same thing in the caller). I'm mostly thinking out loud here, but could we just always do this? I guess you might want to avoid it if your SH_EQUAL is particularly expensive and you know repeated lookups are rare, but maybe that's uncommon enough that we don't really care. -- Na

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Nathan Bossart
On Mon, Nov 20, 2023 at 05:14:17PM -0800, Jubilee Young wrote: > On Mon, Nov 20, 2023 at 2:52 PM Nathan Bossart > wrote: >> Does pgrx use ascii.h at all? > > We don't use utils/ascii.h, no. Alright. The next minor release isn't until February, so I'll let this one sit a l

Re: archive modules loose ends

2023-11-20 Thread Nathan Bossart
On Mon, Nov 13, 2023 at 03:35:28PM -0800, Andres Freund wrote: > On 2023-11-13 16:42:31 -0600, Nathan Bossart wrote: >> There seems to be no interest in this patch, so I plan to withdraw it from >> the commitfest system by the end of the month unless such interest >> mater

Re: Hide exposed impl detail of wchar.c

2023-11-20 Thread Nathan Bossart
hould probably end up living somewhere near the UTF-8 support, and > the easiest way to make it not go into something pgrx currently > includes would be > to make it a new header file, though there's a fair amount of API we > don't touch. Does pgrx use ascii.h at all? -- Nathan Boss

Re: Hide exposed impl detail of wchar.c

2023-11-16 Thread Nathan Bossart
asible to keep simd.h out of all headers that third-party code might want to use forever, but that's not an argument against doing this right now for pgrx. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Hide exposed impl detail of wchar.c

2023-11-16 Thread Nathan Bossart
s. [0] https://postgr.es/m/CAFBsxsHG%3Dg6W8Mie%2B_NO8dV6O0pO2stxrnS%3Dme5ZmGqk--fd5g%40mail.gmail.com [1] https://postgr.es/m/CAFBsxsH1Yutrmu%2B6LLHKK8iXY%2BvG--Do6zN%2B2900spHXQNNQKQ%40mail.gmail.com [2] https://github.com/pgcentralfoundation/pgrx/issues/1298 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Do away with a few backwards compatibility macros

2023-11-16 Thread Nathan Bossart
o remove them. I'm fine with this because all of these macros are no-ops for all supported versions of Postgres. Even if an extension is using them today, you'll get the same behavior as before if you remove the uses and rebuild against v12-v16. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Popcount optimization using AVX512

2023-11-15 Thread Nathan Bossart
farm coverage for this kind of architecture-specific stuff. [0] https://postgr.es/m/20230726043707.GB3211130%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: typo in fallback implementation for pg_atomic_test_set_flag()

2023-11-15 Thread Nathan Bossart
On Wed, Nov 15, 2023 at 09:52:34AM -0600, Nathan Bossart wrote: > On Tue, Nov 14, 2023 at 07:17:32PM -0800, Andres Freund wrote: >> Are you planning to apply the fix? > > Yes, I'll take care of it. Committed and back-patched. I probably could've skipped back-patching this one s

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-15 Thread Nathan Bossart
Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-15 Thread Nathan Bossart
/backend/access/brin/brin.c: patch does not apply I wonder if your mail client is modifying the patch. Do you have the same issue if you download it from the archives? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: typo in fallback implementation for pg_atomic_test_set_flag()

2023-11-15 Thread Nathan Bossart
T_FLAG is not defined. I haven't traced through all the #ifdefs that lead to this point exhaustively, though, so perhaps this is still unlikely. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
identally enabled "digraph" in my .vimrc at some point... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
it 1833f1a 7 years ago. Debian code search shows very minimal > usages of the above macros. Can we do away with > tuplestore_donestoring? Can we take these other things to a separate thread? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
on't mind going the __attribute__((deprecated)) route, but otherwise, I'd probably just remove it completely. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
On Tue, Nov 14, 2023 at 11:01:15AM -0600, Nathan Bossart wrote: > On Tue, Nov 14, 2023 at 04:36:44PM +, Dagfinn Ilmari Mannsåker wrote: >> There's also __attribute__((deprecated)) (and and __declspec(deprecated) >> for MSVC), but that can AFAIK only be attached to functions a

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
tic inline > function. It might be worth introducing pg_attribute_deprecated() in c.h. I'm not too worried about this particular macro, but it seems handy in general. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
On Tue, Nov 14, 2023 at 05:20:16PM +0100, Alvaro Herrera wrote: > Let's leave the macro around and just remove its uses in PGDG-owned > code. Having the macro around hurts nothing, and we can remove it in 15 > years or so. WFM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
On Tue, Nov 14, 2023 at 10:59:04AM -0500, Tom Lane wrote: > Nathan Bossart writes: >> On Tue, Nov 14, 2023 at 04:25:24PM +0530, Amul Sul wrote: >>> Changes looks pretty much straight forward, but patch failed to apply on the >>> latest master head(b41b1a7f490) at

Re: retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-14 Thread Nathan Bossart
and my machine, and check-world continues to pass. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

typo in fallback implementation for pg_atomic_test_set_flag()

2023-11-13 Thread Nathan Bossart
/architecture combinations. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/include/port/atomics/generic.h b/src/include/port/atomics/generic.h index cb5804adbf..95d99dd0be 100644 --- a/src/include/port/atomics/generic.h +++ b/src/include/port/atomics/generic.h @@ -83,7

Re: archive modules loose ends

2023-11-13 Thread Nathan Bossart
There seems to be no interest in this patch, so I plan to withdraw it from the commitfest system by the end of the month unless such interest materializes. On Fri, Feb 17, 2023 at 01:56:24PM -0800, Nathan Bossart wrote: > The first one is the requirement that archive module authors cre

retire MemoryContextResetAndDeleteChildren backwards compatibility macro

2023-11-13 Thread Nathan Bossart
it, but I don't think it would be too burdensome for extension authors to fix if we removed it completely. Patch attached. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c index 25338a90e2..f0eac078e0 100644

Re: autovectorize page checksum code included elsewhere

2023-11-11 Thread Nathan Bossart
On Sat, Nov 11, 2023 at 07:38:59PM +0700, John Naylor wrote: > On Tue, Nov 7, 2023 at 9:47 AM Nathan Bossart > wrote: >> Separately, I'm wondering whether we should consider using CFLAGS_VECTORIZE >> on the whole tree. Commit fdea253 seems to be responsible for introducin

Re: locked reads for atomics

2023-11-10 Thread Nathan Bossart
ought it might've been hinting at something beyond the compare/exchange implications. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: locked reads for atomics

2023-11-10 Thread Nathan Bossart
On Fri, Nov 10, 2023 at 03:11:50PM -0800, Andres Freund wrote: > On 2023-11-10 14:51:28 -0600, Nathan Bossart wrote: >> + * This read is guaranteed to read the current value, > > It doesn't guarantee that *at all*. What it guarantees is solely that the > current CPU won't

Re: locked reads for atomics

2023-11-10 Thread Nathan Bossart
mics also volatile? The PostgreSQL atomics support appears to ensure they are volatile. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Atomic ops for unlogged LSN

2023-11-10 Thread Nathan Bossart
On Thu, Nov 09, 2023 at 03:27:33PM -0600, Nathan Bossart wrote: > I wonder if it's worth providing a set of "locked read" functions. Those > could just do a compare/exchange with 0 in the generic implementation. For > patches like this one where the overhead really sh

locked reads for atomics

2023-11-10 Thread Nathan Bossart
Moving this to a new thread and adding it to the January commitfest. On Thu, Nov 09, 2023 at 03:27:33PM -0600, Nathan Bossart wrote: > On Tue, Nov 07, 2023 at 04:58:16PM -0800, Andres Freund wrote: >> However, even if there's likely some other implied memory barrier that we >>

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-11-10 Thread Nathan Bossart
me. I think it's a good idea to keep it simple to start with. Since this is really only intended for special tests that run in special builds, it seems like we ought to be able to change it easily in the future as needed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-11-10 Thread Nathan Bossart
On Tue, Oct 03, 2023 at 03:17:57PM +0530, vignesh C wrote: > Few comments: Pierre, do you plan to submit a new revision of this patch set for the November commitfest? If not, the commitfest entry may be marked as returned-with-feedback soon. -- Nathan Bossart Amazon Web Services: ht

Re: SET ROLE documentation improvement

2023-11-10 Thread Nathan Bossart
he current session user must have the SET for the + specified role_name, either + directly or indirectly via a chain of memberships with the + SET option. (If the session user is a superuser, any role can be selected.) -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-10 Thread Nathan Bossart
ey have a performance >> problems; it's not going to be the same one for everybody. > > +1 +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: CRC32C Parallel Computation Optimization on ARM

2023-11-10 Thread Nathan Bossart
v8_crypto_available()) pg_comp_crc32c = pg_comp_crc32c_armv8_parallel; else pg_comp_crc32c = pg_comp_crc32c_armv8; else pc_comp_crc32c = pg_comp_crc32c_sb8; -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Atomic ops for unlogged LSN

2023-11-09 Thread Nathan Bossart
On Tue, Nov 07, 2023 at 04:58:16PM -0800, Andres Freund wrote: > On 2023-11-07 11:02:49 -0600, Nathan Bossart wrote: >> Is there something special about all other backends being shut down that >> ensures this returns the most up-to-date value and not something from "some

Re: Popcount optimization using AVX512

2023-11-07 Thread Nathan Bossart
On Mon, Nov 06, 2023 at 09:53:15PM -0800, Noah Misch wrote: > On Mon, Nov 06, 2023 at 09:59:26PM -0600, Nathan Bossart wrote: >> On Mon, Nov 06, 2023 at 07:15:01PM -0800, Noah Misch wrote: >> > The glibc/gcc "ifunc" mechanism was designed to solve this problem of &

Re: Atomic ops for unlogged LSN

2023-11-07 Thread Nathan Bossart
as the stated contract for this function seems to suggest? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Commitfest: older Waiting on Author entries

2023-11-07 Thread Nathan Bossart
> vector search support I haven't had a chance to follow up on these in some time, so I went ahead and marked them returned-with-feedback. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: GUC names in messages

2023-11-07 Thread Nathan Bossart
On Tue, Nov 07, 2023 at 10:33:03AM +0100, Alvaro Herrera wrote: > On 2023-Nov-01, Nathan Bossart wrote: >> +1, IMHO quoting GUC names makes it abundantly clear that they are special >> identifiers. In de4d456, we quoted the role names in a bunch of messages. >> We didn'

Re: Popcount optimization using AVX512

2023-11-06 Thread Nathan Bossart
On Mon, Nov 06, 2023 at 07:15:01PM -0800, Noah Misch wrote: > On Mon, Nov 06, 2023 at 09:52:58PM -0500, Tom Lane wrote: >> Nathan Bossart writes: >> > Like I said, I don't have any proposals yet, but assuming we do want to >> > support newer intrinsics, either open-code

Re: Atomic ops for unlogged LSN

2023-11-06 Thread Nathan Bossart
re time looking at those... [0] https://postgr.es/m/20231102034006.GA85609%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

autovectorize page checksum code included elsewhere

2023-11-06 Thread Nathan Bossart
to do the analysis/benchmarking? [0] https://postgr.es/m/1367013190.11576.249.camel%40sussancws0025 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/bin/pg_checksums/Makefile b/src/bin/pg_checksums/Makefile index ac736b2260..3f946ee9d6 100644 --- a/src/bin

Re: Popcount optimization using AVX512

2023-11-06 Thread Nathan Bossart
thus far. One complication for visiblitymap_count() is that the data passed to pg_popcount64() is masked, which requires a couple more intructions when you're using the intrinsics. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Atomic ops for unlogged LSN

2023-11-06 Thread Nathan Bossart
On Wed, Nov 01, 2023 at 10:40:06PM -0500, Nathan Bossart wrote: > Since this isn't a tremendously performance-sensitive area, IMHO we should > code defensively to eliminate any doubts about correctness and to make it > easier to reason about. Concretely, like this. -- Nathan Bossart A

Re: CRC32C Parallel Computation Optimization on ARM

2023-11-06 Thread Nathan Bossart
On Fri, Nov 03, 2023 at 10:46:57AM +, Xiang Gao wrote: > On Date: Thu, 2 Nov 2023 09:35:50AM -0500, Nathan Bossart wrote: >> The idea is that we don't want to start forcing runtime checks on builds >> where we aren't already doing runtime checks. IOW if the compiler can use &

Re: Three commit tips

2023-11-02 Thread Nathan Bossart
> I would annotate commits with "partial", e.g.: > > Author: Andy Jackson (partial) I tend to use "Co-authored-by" for this purpose. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: CRC32C Parallel Computation Optimization on ARM

2023-11-02 Thread Nathan Bossart
fi My current thinking is that we'll want to add USE_ARMV8_VMULL and USE_ARMV8_VMULL_WITH_RUNTIME_CHECK and use those to decide exactly what to compile. I'll admit I haven't fully thought through every detail yet, but I'm cautiously optimistic that we can avoid too much complexity in the aut

Re: Don't pass NULL pointer to strcmp().

2023-11-01 Thread Nathan Bossart
On Wed, Nov 01, 2023 at 10:39:04PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> What if we disallowed NULL string GUCs in v17? > > Well, we'd need to devise some other solution for hacks like the > one used by timezone_abbreviations (see comment in > check_timezone_abb

Re: Atomic ops for unlogged LSN

2023-11-01 Thread Nathan Bossart
ince this isn't a tremendously performance-sensitive area, IMHO we should code defensively to eliminate any doubts about correctness and to make it easier to reason about. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Don't pass NULL pointer to strcmp().

2023-11-01 Thread Nathan Bossart
proof against similar bugs, but it might also break a fair number of extensions. If there aren't any other reasons to continue supporting it, maybe it's the right long-term approach, though. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: GUC names in messages

2023-11-01 Thread Nathan Bossart
/option names, but those are in all-caps, so they already stand out nicely. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-31 Thread Nathan Bossart
On Mon, Oct 30, 2023 at 11:21:43AM -0500, Nathan Bossart wrote: > On Fri, Oct 27, 2023 at 07:01:10AM +, Xiang Gao wrote: >> On Thu, 26 Oct, 2023 11:37:52AM -0500, Nathan Bossart wrote: >>>> We consider that a runtime check needs to be done in any scenario. >

Re: always use runtime checks for CRC-32C instructions

2023-10-31 Thread Nathan Bossart
On Tue, Oct 31, 2023 at 03:38:17PM -0500, Nathan Bossart wrote: > On Tue, Oct 31, 2023 at 04:12:40PM -0400, Tom Lane wrote: >> This seems like a reasonable proposal. > > Great. I think that leaves us with nothing left to do for this thread, so > I'll withdraw it from the c

Re: always use runtime checks for CRC-32C instructions

2023-10-31 Thread Nathan Bossart
On Tue, Oct 31, 2023 at 04:12:40PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Okay. With that in mind, I think the path forward for new instructions is >> as follows: > >> * If the special CRC instructions can be used with the default compiler >>

Re: always use runtime checks for CRC-32C instructions

2023-10-31 Thread Nathan Bossart
ons are a big portion of that pg_waldump -z command and that 2) the CRC instructions provide significant performance gains. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: always use runtime checks for CRC-32C instructions

2023-10-31 Thread Nathan Bossart
on (assuming we proceed with the proposed optimization). An alternate approach could be to only use newer instructions if they are available with the default compiler flags, but given the current state of the buildfarm, such optimizations might not get much uptake for a while. -- Nathan Bossart Amaz

Re: Fix search_path for all maintenance commands

2023-10-31 Thread Nathan Bossart
Given the experience gained from the 2018 security fixes [0], I think this is okay. [0] https://postgr.es/m/20230715211333.GB3675150%40rfd.leadboat.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [17] Special search_path names "!pg_temp" and "!pg_catalog"

2023-10-31 Thread Nathan Bossart
t; "!pg_temp" is by typing it out, and I'm not sure a lot of people will > actually do that. I thought it sounded generally useful, but if we're not going to proceed with the primary use-case for this feature, then perhaps it's not worth going through this particular one-way door at this

Re: always use runtime checks for CRC-32C instructions

2023-10-31 Thread Nathan Bossart
On Mon, Oct 30, 2023 at 10:36:01PM -0500, Nathan Bossart wrote: > I tested pg_waldump -z with 50M 65-byte records for the following > implementations on an ARM system: > > * slicing-by-8: ~3.08s > * proposed patches applied (runtime check): ~

Re: always use runtime checks for CRC-32C instructions

2023-10-30 Thread Nathan Bossart
mentation compiled : ~2.42s * forced inlining : ~2.38s Avoiding the runtime check produced a 0.8% improvement, and forced inlining produced another 1.7% improvement. In comparison, even the runtime check implementation produced a 20.8% improvement over the slicing-by-

Re: always use runtime checks for CRC-32C instructions

2023-10-30 Thread Nathan Bossart
-time-per-process probe should > be negligible; it's the per-use cost that I worry about. If you can > do some measurements proving that that worry is ill-founded, then > I'm good with test-first. Will do. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

<    1   2   3   4   5   6   7   8   9   10   >