Re: detoast datum into the given buffer as a optimization.

Hi! Andy, I'm truly sorry, I removed quoted messages just to not make replies to walls of text. On detoast_attr_buffer topic - and again agree with reply above that supposes adding required data length as parameter, at least by doing so we should explicitly make the user of this API pay attention

Re: detoast datum into the given buffer as a optimization.

Nikita Malakhov writes: > Hi! > > Sorry for misguiding you, I've overlooked va_rawsize with va_extinfo. > You're right, va_rawsize holds uncompressed size, and extinfo actual > storage size. This was not intentional. That's OK, so we are in the same page here. > I'd better not count on caller's

Re: detoast datum into the given buffer as a optimization.

Hi! Sorry for misguiding you, I've overlooked va_rawsize with va_extinfo. You're right, va_rawsize holds uncompressed size, and extinfo actual storage size. This was not intentional. I'd better not count on caller's do know detoasted data length, and much more the buffer is correctly initialized,

Re: detoast datum into the given buffer as a optimization.

Nikita Malakhov writes: Hello Nikita, > > For EXPANDED attributes va_rawsize is the size of the compressed attribute, > not original size. > You can check toast_save_datum for that. Here is some quota from toast_save_datum, which is different from what you are saying here. * Get the

Re: detoast datum into the given buffer as a optimization.

Hi Andy, For EXPANDED attributes va_rawsize is the size of the compressed attribute, not original size. You can check toast_save_datum for that. This thread looks like the second take of the shared detoast datum patch. Have you checked my proposals in that thread? -- Regards, Nikita Malakhov Po

Re: detoast datum into the given buffer as a optimization.

Hi, I find some independent improvement in this area. In the detoast_attr, VARATT_IS_EXTERNAL_ONDISK and VARATT_IS_EXTERNAL_INDIRECT are checked first, and then VARATT_IS_EXTERNAL_EXPANDED is checked. However when VARATT_IS_EXTERNAL_EXPANDED is true, detoast_external_attr is called which would ch

Re: detoast datum into the given buffer as a optimization.

Hi Tom, > Andy Fan writes: >> * Note if caller provides a non-NULL buffer, it is the duty of caller >> * to make sure it has enough room for the detoasted format (Usually >> * they can use toast_raw_datum_size to get the size) > > This is a pretty awful, unsafe API design. Sorry that I ex

Re: detoast datum into the given buffer as a optimization.

Jubilee Young writes: > On Wed, Sep 18, 2024 at 2:23 PM Nathan Bossart > wrote: >> >> On Wed, Sep 18, 2024 at 05:35:56PM +0800, Andy Fan wrote: >> > Currently detoast_attr always detoast the data into a palloc-ed memory >> > and then if user wants the detoast data in a different memory, user h

Re: detoast datum into the given buffer as a optimization.

Thank you all for the double check. > Andy Fan writes: >> * Note if caller provides a non-NULL buffer, it is the duty of caller >> * to make sure it has enough room for the detoasted format (Usually >> * they can use toast_raw_datum_size to get the size) > > ..., It puts it on the caller

Re: detoast datum into the given buffer as a optimization.

Andy Fan writes: > * Note if caller provides a non-NULL buffer, it is the duty of caller > * to make sure it has enough room for the detoasted format (Usually > * they can use toast_raw_datum_size to get the size) This is a pretty awful, unsafe API design. It puts it on the caller to know h

Re: detoast datum into the given buffer as a optimization.

On Wed, Sep 18, 2024 at 2:23 PM Nathan Bossart wrote: > > On Wed, Sep 18, 2024 at 05:35:56PM +0800, Andy Fan wrote: > > Currently detoast_attr always detoast the data into a palloc-ed memory > > and then if user wants the detoast data in a different memory, user has to > > copy them, I'm thinking

Re: detoast datum into the given buffer as a optimization.

On Wed, Sep 18, 2024 at 05:35:56PM +0800, Andy Fan wrote: > Currently detoast_attr always detoast the data into a palloc-ed memory > and then if user wants the detoast data in a different memory, user has to > copy them, I'm thinking if we could provide a buf as optional argument for > detoast_attr

detoast datum into the given buffer as a optimization.

Hi, Currently detoast_attr always detoast the data into a palloc-ed memory and then if user wants the detoast data in a different memory, user has to copy them, I'm thinking if we could provide a buf as optional argument for detoast_attr to save such wastage. current format: /* -- *