Thanks for raising this Alex. I think it's a good idea to avoid sync in general. Will take a close look.
Yufei On Fri, Jul 24, 2026 at 9:35 AM Alexandre Dutra <[email protected]> wrote: > Hi all, > > I have recently submitted a PR to enable support for external > principals, that is, principals that are not backed by an entity in > the Polaris metastore: > > https://github.com/apache/polaris/pull/5119 > > This implementation finalizes the concepts originally introduced in > this design document [1], which we have previously discussed during > our Polaris sync meetings. > > The key benefit of this change is the elimination of an expensive > database call that is currently triggered for every request, which is > especially impactful since not all metastores use a caching layer. > > It also brings another considerable benefit: not having to synchronize > principals externally anymore. > > Please take a look. Essentially, when a realm uses external auth and > is further configured in "external principal" mode, the system simply > bypasses the metastore lookup for the principal entity. > > You might wonder why we don't eliminate this lookup entirely, > considering that the Resolver is its only true consumer and ends up > re-resolving the entity anyway. However, removing it completely would > fundamentally reshape our internal authentication flow. I believe it > is safer to keep the internal authentication flow intact while > offering the flexibility to skip this costly overhead when external > IDPs are used. > > You may also wonder why not conflate external auth with external > principals and eliminate the metastore lookup whenever external auth > is used: that would be a clean approach, but would also introduce a > big behavioral change. I think it is safer to let users decide if they > want to keep > the current behavior (lookup on) or the new one (lookup off). > > In summary: > > 1) for realms using internal auth: this change doesn't apply at all > (nothing changes) > > 2) for realms using external auth: nothing changes by default, but the > new mode allows skipping metastore lookup and makes principal > synchronization unnecessary. > > 3) for realms using mixed auth: > - If the token is issued by Polaris (internal auth), see #1 > - Otherwise, see #2 > > I look forward to hearing your thoughts and feedback on this approach. > > Thanks, > Alex > > [1]: > https://docs.google.com/document/d/1VSoN1-QsAJGaM40oWTxeLIoYTp-XlfQM9nG-jqfb_-E/edit >
