Hi, On 2021-08-18 10:21:03 -0400, Tom Lane wrote: > Anyway, this whole argument could be rendered moot if we could convert > name to a variable-length type. That would satisfy *both* sides of > the argument, since those who need long names could have them, while > those who don't would see net reduction instead of growth in space usage.
Yes, I think that's the best direction to go. We're loosing a fair bit of in-memory efficiency with current NAMEDATALEN already, it'd imo be beneficial to go for variable length encoding from that angle alone. > Of course, this is far far easier said than done; else we would have > done it years ago. But maybe it's not entirely out of reach. > I do not think it'd be hard to change "name" to have the same on-disk > storage representation as cstring; the hard part is what about its > usage in fixed-width catalog structures. Indeed. ISTM that the hardest part of that is dealing with copying around Form* structs? I wonder if we're getting closer to the time where we should just give up on the struct / ondisk layout mirroring for catalog tables, and generate explicit transformation routines via Catalog.pm. If we have to touch places handling Form_* structs anyway, we could make that more worthwhile by doing away with CATALOG_VARLEN etc - the transformation routines would just make those fields pointers which then can be accessed normally. > Maybe we could finesse that > by decreeing that the name column always has to be the last > non-CATALOG_VARLEN field. (This would require fixing up the couple of > places where we let some other var-width field have that distinction; > but I suspect that would be small in comparison to the other work this > implies. If there are any catalogs having two name columns, one of them > would become more difficult to reach from C code.) I wish we could find a way to make "relative pointers" (i.e. a variable offset from some base struct) work on the C level in a transparent and portable manner. Just about any instruction set has them natively anyway, and for a lot of tasks like variable length members it can be considerably more efficient... It'd also make DSM using code simpler and faster. Oh well, one can dream. > Another fun thing --- and, I fear, another good argument against just > raising NAMEDATALEN --- is what about TupleDescs, which last I checked > used an array of fixed-width pg_attribute images. But maybe we could > replace that with an array of pointers. Andres already did a lot of > the heavy code churn required to hide that data structure behind > TupleDescAttr() macros, so changing the representation should be much > less painful than it would once have been. I was recently wondering if we shouldn't go to a completely bespoke datastructure for TupleDesc->attrs, rather than reusing FormData_pg_attribute. Right now every attribute uses nearly two cachelines (112 bytes). Given how frequent a task tuple [de]forming is, and how often it's a bottleneck, increasing the cache efficiency of tupledescs would worth quite a bit of effort - I do see tupledesc attr cache misses in profiles. A secondary benefit would be that we do create a lot of short-lived descs in the executor, slimming those down obviously would be good on its own. A third benefit would be that we could get rid of attcacheoff in pg_attribute, that always smelled funny to me. One possible way to structure such future tupledescs would be to have multiple arrays in struct TupleDescData. With an array of just the data necessary for [de]forming at the place ->attrs is, and other stuff in one or more separate arrays. The other option could perhaps be omitted for some tupledescs or computed lazily. For deforming we just need attlen (2byte), attbyval (1 byte), attalign (1byte) and optionally attcacheoff (4 byte), for forming we also need attstorage (1 byte). Naively that ends up being 12 bytes - 5 attrs / cacheline is a heck of a lot better than ~0.5. Greetings, Andres Freund
