Hi David, Thanks for stating this thread!
> 1. Is provider-specific configuration validation desirable for generic > `ICEBERG_REST` federation? >From POV, having validation can help users detect mistakes early. It is nice to have. In this particular case, the validation is based purely on pattern matching against existing catalog properties. It does not involve I/O or external dependencies. I believe this kind of validation should be ok to keep in the main codebase (runtime/service). I wonder what other people think about this. Side note: Please consider making Python changes in separate PRs as they usually need different reviewers ;) > 3. What ownership and dispatch boundary would keep the shared admin > workflow provider-neutral and avoid running provider-specific logic for > every catalog operation? Regarding abstraction, I would suggest using CDI to find multiple implementations of a (new) common validation interface. Admin Service code will stay lean and only call the validation interface. That call will be dispatched to every matching CDI bean, each running logic specific to a use case (e.g. BigLake). Since validation code is involved only on updates (unfrequently), and valation logic likely runs in-memory (fast), I do not think we need to worry about calling every validators on every change. The PR [5513] shows some nice (recent) examples of leveraging CDI for pluggable components. [5513] https://github.com/apache/polaris/pull/5513 Cheers, Dmitri. On Wed, Sep 23, 2026 at 12:26 AM David Chaava via dev < [email protected]> wrote: > Hi Polaris community, > > > We would like guidance on whether Polaris should support provider-specific > configuration validation for external Iceberg REST catalogs, and, if so, > what the appropriate extension model should be. > > > Context: > - Polaris supports the generic `ICEBERG_REST` connection type. > - The initial BigLake implementation added provider-specific validation in > the shared `PolarisAdminService` create/update path. > - Review feedback on PR #5196 raised concerns about putting > provider-specific logic in the main admin workflow, introducing a validator > without a shared abstraction, and coupling provider configuration rules to > Polaris releases. > > > We have closed that implementation and would like to agree on the design > before proposing a replacement. > > > Questions: > 1. Is provider-specific configuration validation desirable for generic > `ICEBERG_REST` federation? > 2. If so, should Polaris first define a shared abstraction or extension > model for such validation? > 3. What ownership and dispatch boundary would keep the shared admin > workflow provider-neutral and avoid running provider-specific logic for > every catalog operation? > 4. Under what conditions, if any, would a separate connection type be > preferable to extending the generic `ICEBERG_REST` flow? > > > Links: > - Tracking issue: https://github.com/apache/polaris/issues/5195 > - Closed initial PR: https://github.com/apache/polaris/pull/5196 > - Architecture feedback: > https://github.com/apache/polaris/pull/5196#discussion_r3846267047 > > > Thanks, > David >
