On Fri Nov 20, 2026 at 3:34 PM UTC, Peter Eisentraut wrote:
> On 16.11.26 18:49, Tristan Partin wrote:
>>> - TRGM in contrib/pg_trgm/ is an instance of what [0] talks about: It
>>> has a trailing flexible array member, but it also has trailing padding,
>>> so allocating palloc(TRGMHDRSIZE) causes a warning, because it's less
>>> than sizeof(TRGM).  In my patch I work around this by allocating like
>>> palloc(Max(TRGMHDRSIZE, sizeof(TRGM))).
>> 
>> Maybe we should add an allocator function or macro for this? In the
>> counted_by discussion, I talked a bit about how core does a pretty good
>> job of providing allocator helpers for structs with flexible array
>> members. I realize this is contrib/, but it might still be valuable.
>> 
>>> - There is a bit of a maze of different structs and unions that underlie
>>> the numeric type.  The allocation (Numeric) palloc(NUMERIC_HDRSZ_SHORT)
>>> gets flagged by the compiler.  In my patch I change this to (Numeric)
>>> palloc(sizeof(struct NumericData)), which allocates a bit more but makes
>>> the sizes and types match.  Maybe some more judicious reshuffling of the
>>> underlying data structures could avoid this, or maybe it's okay like this.
>> 
>> Like the previous comment, I wonder if some helper allocator
>> functions/macros could be useful. But as you said, maybe some
>> reshuffling could avoid that all together, not sure.
>
> In the attached patch series, I have an idea for a more generalized 
> approach: A macro that computes the required size and then a wrapper 
> macro around it for allocating.  This then can be applied to any 
> structure with a flexible array member, and it eliminates some 
> duplication with the offsetof calls and gets you the kind of type safety 
> that palloc_object and palloc_array provide.
>
> I have also added a patch that makes some more uses of this macro. 
> These are the same kinds of problems as the one I showed for pg_trgm and 
> numeric, except that the size isn't constant and so the compiler won't 
> flag it.

Overall, I think the concept looks good! Maybe adding 
a palloc_empty_flex_array() macro could be useful to make the 
0 allocations a little bit nicer to read.

> @@ -1226,6 +1226,24 @@ pg_noreturn extern void ExceptionalCondition(const 
> char *conditionName,
>   */
>  #define Min(x, y)            ((x) < (y) ? (x) : (y))
>  
> +/*
> + * flex_size
> + *           Return the size of a struct "type" whose flexible array member
> + *           "member" holds "count" elements.
> + *
> + * The result is never less than sizeof(type), so that any trailing padding
> + * of the struct is covered.  "count" is evaluated only once.
> + */
> +#define flex_size(type, member, count) \
> +     flex_size_impl(sizeof(type), offsetof(type, member), \
> +                                sizeof(((type *) 0)->member[0]), (count))
> +
> +static inline size_t
> +flex_size_impl(size_t structsz, size_t offset, size_t elemsz, size_t count)
> +{
> +     return Max(structsz, offset + count * elemsz);
> +}
> +
>  
>  /* Get a bit mask of the bits set in non-size_t aligned addresses */
>  #define SIZE_T_ALIGN_MASK (sizeof(size_t) - 1)

It might behoove us to use the same safe math techniques that we use in 
palloc_mul() here. Maybe I am being too cautious. It is not clear to me 
if any potential "count" values would ever be derived from user input. 
It would protect from us _succeeding_ to allocate a potentially smaller 
amount of memory, and then writing or reading past the end.

>>> - As of commit 6a3da13d376 from this week, FormData_pg_attribute has 4
>>> bytes of trailing padding, and so sizeof(FormData_pg_attribute) is now
>>> different from ATTRIBUTE_FIXED_PART_SIZE.  This causes a warning in one
>>> place.  We had a discussion about this some time ago [1], where the
>>> opinions diverged between like, it's important to keep this distinction
>>> in case we ever have padding bytes, and, it's already incorrect and
>>> would be even more incorrect if we had padding bytes. ;-)  Well, we're
>>> here now, so perhaps this should be revisited.
>
> I have adjusted the description of this patch a bit.  There are a lot of 
> details to be figured out in this area, but at least we can fix this 
> particular issue easily and in a consistent way.  tupdesc.c already uses 
> sizeof(FormData_pg_attribute) consistently for allocating pg_attribute 
> structures.  It only uses ATTRIBUTE_FIXED_PART_SIZE for things like 
> memcpy().  With the proposed patch, we make bootstrap.c consistent in 
> this regard, and we make the warning go away.  The deeper issues can 
> then be researched independent of this.

I agree that this problem is likely orthogonal, and we can move it to 
a separate discussion.

>> In [1], Tom said:
>> 
>>> What I'm trying to figure out is if we need to put in place some
>>> provisions to prevent there from being pad bytes at the end of any
>>> catalog struct.
>> 
>> I figured I could propose at least one way to flag padding bytes in the
>> attached patch. I am currently not proposing a solution.
>> 
>> Two options proposed in the thread[1]:
>> 
>> - Placeholder fields exactly matching the size of the padding bytes.
>> 
>>      #ifdef CATALOG_VARLEN                   /* variable-length fields start 
>> here */
>>              pg_node_tree ev_qual;
>>              pg_node_tree ev_action;
>>      #else
>>              uint8       dummy;
>>      #endif
>> 
>> - Use different structs for on-media and in-memory representations.
>> 
>> To not rock the boat too much, maybe I would throw my hat in the ring
>> for option 1? It seems like the simplest and most immediate solution.
>
> I think we could get the same check simpler and more portably by having 
> genbki.pl generate some static assertions that check like
>
> static_assert(sizeof(FormData_pg_attribute) == 
> offsetof(FormData_pg_attribute, attcollation) + 
> sizeof(FormData_pg_attribute.attcollation));
>
> But this flags a bunch of issues.  Sorting that out might be a big project.

I had a similar idea as well.

Overall, I think that the non-WIP patches look good, but it would 
probably be useful to get another committer's thoughts before moving 
forward.

-- 
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)


Reply via email to