> In the long term, I suppose we may want to think about storing immutable state in entities as a first-class feature of the codebase (something more than just a bag of properties).
I don’t see a strong reason not to do this, especially since the idea of eventually getting rid of metadata.json is always tempting. We could persist the full metadata in the metadata store, as Eric suggested, and use it as an index to significantly reduce IO costs, while still keeping metadata.json in object storage as required by the Iceberg spec. Yufei On Tue, Feb 3, 2026 at 2:17 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Prashant, > > As you know I approved this change in GH. Commenting here for reference. > > In the long term, I suppose we may want to think about storing immutable > state in entities as a first-class feature of the codebase (something more > than just a bag of properties). > > Cheers, > Dmitri. > > On Tue, Jan 27, 2026 at 12:55 PM Prashant Singh via dev < > [email protected]> wrote: > > > Hello everyone, > > > > Presently since not all the location properties are stored in the entity > > one has to do complete loadTable (i.e read the table from the object > > store), even for existing use cases such as credential endpoint i.e > > vend the cred of the table. This is super expensive, we can optimize this > > if we start keeping these properties of the table in the table entity > > (polaris persistence). > > > > With this we will have a way to do cred vending, in future (to remote > sign) > > without going to object store. > > > > Note if we take dependency on this we would have to think of *backfill* > but > > for step 1 it seems a reasonable step as i personally see this as an > > extension of storing yet another set of props like we did [1], while we > > still think / debate on if we wanna store the whole table metadata in > > persistence, would love to know what other folks think, if anyone have > > *objection* to it. > > > > I proactively have created a pr [2] for the same. > > > > Huge thanks to Alex for proposing this in their remote signing effort in > > the first place. > > > > [1] https://github.com/apache/polaris/pull/2735 > > > > [2] https://github.com/apache/polaris/pull/3226 > > > > [3] https://github.com/apache/polaris/pull/2280#discussion_r2519568503 > > > > > > Best, > > > > Prashant Singh > > >
