On 27 November 2017 at 04:46, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> On 26 November 2017 at 08:46, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> I've confirmed that the attached is sufficient to stop the valgrind crash >>> on my machine. But as I said, I think we should be more aggressive at >>> resizing the buffer, to reduce resize cycles. I'm inclined to start out >>> with a buffer size of 128 or 256 or so bytes and double it when needed. >>> Anybody have a feeling for a typical size for the "main data" part >>> of a WAL record? > >> We reuse the buffer and only pfree/palloc when we need to enlarge the >> buffer, so not sure we need to do the doubling thing and it probably >> doesn't matter what the typical size is. > > Well, I'm concerned about the possibility of a lot of palloc thrashing > if the first bunch of records it reads happen to have steadily increasing > sizes. However, rather than doubling, it might be sufficient to set a > robust minimum on the first allocation, ie use something along the lines > of Max(1024, MAXALIGN(state->main_data_len)).
Agreed. I was just researching what that number should be... and I was thinking that we should use the maximum normal tuple size, which I think is TOAST_TUPLE_THRESHOLD + SizeOfXLogRecord + SizeOfXLogRecordDataHeaderLong -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
