Hi! Widening of a TOAST pointer is possible if we keep backward compatibility with old-fashioned TOAST tables - I mean differ 'long' and 'short' TOAST pointers and process them accordingly on insert and delete cases, and vacuum with logical replication. It is not very difficult, however it takes some effort. Recently I've found out that I have not overseen all compatibility cases, so the provided patch is functional but limited in compatibility.
We already have a flag byte in the TOAST pointer which is responsible for the type of the pointer - va_flag field. It was explained in the Pluggable TOAST patch. One is enough, there is no need to add another one. On Wed, Apr 26, 2023 at 3:55 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi, > > > I agree that we can't simply widen varatt_external to use 8 bytes for > > the toast ID in all cases. > > +1 > > Note that the user may have a table with multiple TOASTable > attributes. If we simply widen the TOAST pointer it may break the > existing tables in the edge case. Also this may be a reason why > certain users may prefer to continue using narrow pointers. IMO wide > TOAST pointers should be a table option. Whether the default for new > tables should be wide or narrow pointers is debatable. > > In another discussion [1] we seem to agree that we also want to have > an ability to include a 32-bit dictionary_id to the TOAST pointers and > perhaps support more compression methods (ZSTD to name one). Besides > that it would be nice to have an ability to extend TOAST pointers in > the future without breaking the existing pointers. One possible > solution would be to add a varint feature bitmask to every pointer. So > we could add flags like TOAST_IS_WIDE, TOAST_HAS_DICTIONARY, > TOAST_UNKNOWN_FEATURE_FROM_2077, etc indefinitely. > > I suggest we address all the current and future needs once and > completely refactor TOAST pointers rather than solving one problem at > a time. I believe this will be more beneficial for the community in > the long term. > > Thoughts? > > [1]: https://commitfest.postgresql.org/43/3626/ > > -- > Best regards, > Aleksander Alekseev > -- Regards, Nikita Malakhov Postgres Professional The Russian Postgres Company https://postgrespro.ru/
