Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-12-21 Thread David Rowley
On Wed, 16 Nov 2022 at 23:56, David Rowley wrote: > I've attached an updated patch. The 0002 is just intended to exercise > these allocations a little bit, it's not intended for commit. I was > using that to ensure valgrind does not complain about anything. It > seems happy now. After making

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-16 Thread David Rowley
On Wed, 16 Nov 2022 at 08:19, Andres Freund wrote: > We already rely on memory context returning MAXIMUM_ALIGNOF aligned > allocations. Adding the special case, I think, means that the we could safely > over-allocate by "only" > alignto + sizeof(MemoryChunk) - MAXIMUM_ALIGNOF > > Which would be

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-15 Thread Andres Freund
Hi, On 2022-11-15 16:58:10 -0500, Greg Stark wrote: > So I think it's kind of cute that you've implemented these as agnostic > wrappers that work with any allocator ... but why? > > I would have expected the functionality to just be added directly to > the allocator to explicitly request whole

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-15 Thread Greg Stark
So I think it's kind of cute that you've implemented these as agnostic wrappers that work with any allocator ... but why? I would have expected the functionality to just be added directly to the allocator to explicitly request whole aligned pages which IIRC it's already capable of doing but just

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-15 Thread Andres Freund
Hi, On 2022-11-15 23:36:53 +1300, David Rowley wrote: > On Tue, 15 Nov 2022 at 11:11, Andres Freund wrote: > > Couldn't we reduce the amount of over-allocation by a small amount by > > special > > casing the already-aligned case? That's not going to be relevant for page > > size > > aligne

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-15 Thread David Rowley
On Mon, 14 Nov 2022 at 15:25, John Naylor wrote: > - Assert((char *) chunk > (char *) block); > + Assert((char *) chunk >= (char *) block); > > Is this related or independent? It's related. Because the code is doing: MemoryChunkSetHdrMask(alignedchunk, unaligned, alignto,

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-15 Thread David Rowley
On Tue, 15 Nov 2022 at 11:11, Andres Freund wrote: > Couldn't we reduce the amount of over-allocation by a small amount by special > casing the already-aligned case? That's not going to be relevant for page size > aligne allocations, but for smaller alignment values it could matter. I don't

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-14 Thread Andres Freund
Hi, On 2022-11-08 14:57:35 +1300, David Rowley wrote: > On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > > Should we handle the case where we get a suitably aligned pointer from > > MemoryContextAllocExtended() differently? > > Maybe it would be worth the extra check. I'm trying to imagine

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-13 Thread John Naylor
On Tue, Nov 8, 2022 at 8:57 AM David Rowley wrote: > On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > > I doubtthere's ever a need to realloc such a pointer? Perhaps we could just > > elog(ERROR)? > > Are you maybe locked into just thinking about your current planned use > case that we want

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Tue, 8 Nov 2022 at 15:17, John Naylor wrote: > > > On Tue, Nov 8, 2022 at 8:57 AM David Rowley wrote: > > Is there anything we could align to CPU cacheline size that would > > speed something up? > > InitCatCache() already has this, which could benefit from simpler notation. Thanks. I wasn't

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Tue, 8 Nov 2022 at 14:57, David Rowley wrote: > On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > > Should we handle the case where we get a suitably aligned pointer from > > MemoryContextAllocExtended() differently? > > Maybe it would be worth the extra check. I'm trying to imagine future

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread Andres Freund
Hi, On 2022-11-08 15:01:18 +1300, David Rowley wrote: > Andres has suggested I remove the repalloc stuff I added but see my reply to > that. I'm fine with keeping it, I just couldn't really think of cases that have strict alignment requirements but also requires resizing. Greetings, Andres

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread John Naylor
On Tue, Nov 8, 2022 at 8:57 AM David Rowley wrote: > Is there anything we could align to CPU cacheline size that would > speed something up? InitCatCache() already has this, which could benefit from simpler notation. /* * Allocate a new cache structure, aligning to a cacheline boundary * *

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
On Fri, 4 Nov 2022 at 04:20, Maxim Orlov wrote: > I've done a quick look and the patch is looks good to me. > Let's add tests for these functions, should we? If you think this is an > overkill, feel free to trim tests for your taste. Thanks for doing that. I'm keen to wait a bit and see if we

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread David Rowley
Thanks for having a look. On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > FWIW, to me this is really more part of mcxt.c than its own > allocator... Particularly because MemoryContextAllocAligned() et al are > implemented there. I'm on the fence about this one. I thought it was nice that we

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread Andres Freund
Hi, On 2022-11-02 00:28:46 +1300, David Rowley wrote: > Part of the work that Thomas mentions in [1], regarding Direct I/O, > diff --git a/src/backend/utils/mmgr/alignedalloc.c > b/src/backend/utils/mmgr/alignedalloc.c > new file mode 100644 > index 00..e581772758 > --- /dev/null > +++

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-03 Thread Maxim Orlov
Hi! Part of the work that Thomas mentions in [1], regarding Direct I/O, > has certain requirements that pointers must be page-aligned. > > I've attached a patch which implements palloc_aligned() and > MemoryContextAllocAligned() ... > I've done a quick look and the patch is looks good to me.

Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-01 Thread David Rowley
Part of the work that Thomas mentions in [1], regarding Direct I/O, has certain requirements that pointers must be page-aligned. I've attached a patch which implements palloc_aligned() and MemoryContextAllocAligned() which accept an 'alignto' parameter which must be a power-of-2 value. The memory