Re: copy.c allocation constant

2018-01-24 Thread Tom Lane
Bruce Momjian writes: > On Thu, Jan 25, 2018 at 09:30:54AM +1300, Thomas Munro wrote: >> On Thu, Jan 25, 2018 at 7:19 AM, Robert Haas wrote: >>> My guess is that a fairly common pattern for larger chunks will be to >>> round the size up to a multiple of 4kB, the usual memory page size. >> >> See

Re: copy.c allocation constant

2018-01-24 Thread Thomas Munro
On Thu, Jan 25, 2018 at 9:35 AM, Bruce Momjian wrote: > The BSD memory allocator used to allocate in powers of two, and keep the > header in a separate location. They did this so they could combine two > free, identically-sized memory blocks into a single one that was double > the size. I have n

Re: copy.c allocation constant

2018-01-24 Thread Bruce Momjian
On Thu, Jan 25, 2018 at 09:30:54AM +1300, Thomas Munro wrote: > On Thu, Jan 25, 2018 at 7:19 AM, Robert Haas wrote: > > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > > wrote: > >> At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > >> with similar ideas (freelists, ...) so

Re: copy.c allocation constant

2018-01-24 Thread Thomas Munro
On Thu, Jan 25, 2018 at 7:19 AM, Robert Haas wrote: > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > wrote: >> At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator >> with similar ideas (freelists, ...) so hopefully it's fine too. >> >> And then there are the systems without gl

Re: copy.c allocation constant

2018-01-24 Thread Andres Freund
Hi, On 2018-01-24 17:07:01 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > glibc's malloc does add a header. My half-informed suspicion is that > > most newer malloc backing allocators will have a header, because > > maintaining a shared lookup-by-address table is pretty expensive to > > m

Re: copy.c allocation constant

2018-01-24 Thread Alvaro Herrera
Andres Freund wrote: > On 2018-01-24 14:25:37 -0500, Robert Haas wrote: > > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > > > platforms though. From man mallopt: > > > Balancing these factors leads to a defa

Re: copy.c allocation constant

2018-01-24 Thread Andres Freund
On 2018-01-24 14:25:37 -0500, Robert Haas wrote: > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > > platforms though. From man mallopt: > > Balancing these factors leads to a default setting of 128*1024 for the

Re: copy.c allocation constant

2018-01-24 Thread Robert Haas
On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > platforms though. From man mallopt: > Balancing these factors leads to a default setting of 128*1024 for the > M_MMAP_THRESHOLD parameter. > Additionally, even when

Re: copy.c allocation constant

2018-01-24 Thread Andres Freund
On 2018-01-24 13:19:19 -0500, Robert Haas wrote: > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > wrote: > > At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > > with similar ideas (freelists, ...) so hopefully it's fine too. > > > > And then there are the systems without gl

Re: copy.c allocation constant

2018-01-24 Thread Robert Haas
On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra wrote: > At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > with similar ideas (freelists, ...) so hopefully it's fine too. > > And then there are the systems without glibc, or with other libc > implementations. No idea about thos

Re: copy.c allocation constant

2018-01-24 Thread Bruce Momjian
On Wed, Jan 24, 2018 at 06:25:01PM +0100, Tomas Vondra wrote: > > > > I think there there are two things to consider - aset.c and glibc. > > In AllocSet this is handled as over-sized chunk, that is chunk greater > than ALLOC_CHUNK_LIMIT (which ends up being 8kB). Which means it's > allocated as

Re: copy.c allocation constant

2018-01-24 Thread Tomas Vondra
On 01/24/2018 04:14 AM, Bruce Momjian wrote: > On Tue, Nov 28, 2017 at 11:51:28AM -0500, Andrew Dunstan wrote: >> >> >> While reading copy.c I noticed this line: >> >> >> #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ >> >> >> Doesn't that seem rather odd? If we're adding

Re: copy.c allocation constant

2018-01-23 Thread Bruce Momjian
On Tue, Nov 28, 2017 at 11:51:28AM -0500, Andrew Dunstan wrote: > > > While reading copy.c I noticed this line: > > > #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ > > > Doesn't that seem rather odd? If we're adding 1 wouldn't it be better as > 65535 so we palloc a po

copy.c allocation constant

2017-11-28 Thread Andrew Dunstan
While reading copy.c I noticed this line: #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ Doesn't that seem rather odd? If we're adding 1 wouldn't it be better as 65535 so we palloc a power of 2? I have no idea if this affects performance, but it did strike me as stra