Hi Prithvi, Thanks for the analysis! It is very helpful.
The grant record lookups appear to be specific to the internal RBAC model. I wonder if they are necessary with external Authorizers (e.g. Ranger). I suppose those queries will return empty results in the external case anyway. What do people think about refactoring the resolution code path to take the Authorizer "kind" into account? I believe Sung's prior work was moving in that direction. After PR [5194] we should be able to avoid the grant record lookup for authorizers that do not use them, I hope. [5194] https://github.com/apache/polaris/pull/5194 Cheers, Dmitri. On Fri, Sep 4, 2026 at 12:46 PM Prithvi S <[email protected]> wrote: > Hi Dmitri, > > Thanks for raising this. > > You are right that even on a cache hit, resolve still talks to JDBC. That > is by design: InMemoryEntityCache is not a "skip the database" cache. For > JDBC it is a "skip the expensive load" cache, with a version check as the > invalidation path. > I traced Resolver.resolveAll() with JDBC and compared a warmed > InMemoryEntityCache against cache == null on the same catalog. The workload > was a loadTable-like resolve (principal P1 with PRINCIPAL_ROLE:ALL, catalog > "test", path N1/N2/T2). > > On a warm cache, that resolve issued 1 SELECT: > > SELECT id, catalog_id, entity_version, grant_records_version > FROM ENTITIES > WHERE (catalog_id, id) IN ((?, ?), ... ) AND realm_id = ? > > Without the cache, the same resolve issued 23 SELECTs: 9 full entity rows > (including properties / internal_properties) and 14 grant-record lookups. > Scenario Cache SELECTs Payload > N1/N2/T2 (small properties) no-cache 23 947 B > N1/N2/T2 cold-cache 23 947 B > N1/N2/T2 warm-cache 1 144 B > N1/N2/T1 (64KiB properties) no-cache 23 66 KB > N1/N2/T1 warm-cache 1 144 B > deep path (6 segments) no-cache 29 959 B > deep path (6 segments) warm-cache 1 192 B > > So the version check is real, but it replaces the full entity JSON and the > grant-record queries. Cold cache looks the same as no cache; the gain is on > subsequent resolves of the same entities. > > I have not measured a full REST request or a stale-cache / high-churn path, > so I do not want to claim an end-to-end speedup from this :) > For the resolve path itself, the query shape is clear. > > WDYT? > > Thanks, > Prithvi > > On Fri, Sep 4, 2026 at 6:50 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi All, > > > > InMemoryEntityCache is currently used in Apache Polaris with JDBC > > Persistence. > > > > However, as far as I can tell, even for cached entities, the resolve will > > still issue a JDBC query to confirm that the latest version was returned > > from the cache. > > > > So, the question is: does this cache provide any considerable efficiency > > gain? > > > > Has anyone measured the performance difference with and without this > cache? > > > > Would anyone be willing to carry out such a test? > > > > WDYT? > > > > Thanks, > > Dmitri. > > >
