I think it would be great to bring this functionality into Iceberg. I'm curious about your plan for getting it in. It sounds like you're suggesting adding the Dremio project to the Iceberg repo and making it optional. Why not contribute the functionality directly to the AuthManager already in Iceberg? Is this incompatible or is there a reason the current one can't be extended through contributions?
On Tue, Jun 17, 2025 at 11:23 AM Christian Thiel <christian.t.b...@gmail.com> wrote: > Hey Alex, > > Thanks for the Initiative — I really appreciate the effort here! > > Having good auth compatibility in the Catalog ecosystem is key to > establish secure standards by making them easy to use. While Iceberg should > stay open to other means of Authentication, OAuth2 is the most widely > adopted interoperable auth standard, and its role in Iceberg REST reflects > that. But with human-centric flows like Auth Code (with PKCE 😉) and Device > Code missing from most standard clients, users often default to handing out > personal Client ID/secret pairs—which is really bad from a security > perspective. > > While I can’t speak to the Java details, I fully support bringing the > functionality into Iceberg. I have tested the proposed code successfully > with Spark and different IdPs, including Auth & Device Code flows with > token refresh, as well as token refresh for Client Credential flows. > > Thanks! > > Christian > > > On Mon, 16 Jun 2025 at 20:33, Alex Dutra <alex.du...@dremio.com.invalid> > wrote: > >> Hi all, >> >> Dremio recently open-sourced a new implementation of the Auth Manager >> API for OAuth2: >> >> https://github.com/dremio/iceberg-auth-manager >> >> I wrote a blog post about it a while ago [1]. >> >> Built on top of the Auth Manager API introduced in Iceberg 1.9.0, this >> project provides a more flexible and extensible OAuth2 manager >> compared to the built-in equivalent in Iceberg Core. It follows OAuth2 >> standards strictly, but also provides compatibility with any existing >> Apache Iceberg REST catalog, and contains no Dremio-specific >> functionality. To date, this is the only OAuth2 manager fully >> compliant with external identity providers. >> >> Dremio would like to contribute this code to the Apache Iceberg >> project. I am therefore initiating this discussion to determine the >> community's interest in accepting this donation. >> >> This project is beneficial to the community because it addresses >> well-known limitations, such as token refresh problems [2][3][4], and >> also because it introduces highly anticipated features like the >> Authorization Code grant support [5]. Fixing these limitations or >> adding support for such large features in the built-in manager, while >> avoiding any risk of regressions, would have been a lot harder. >> >> Also worth mentioning: this project adheres to the "Iceberg OAuth2 >> Client Authentication Guide", proposed by Christian Thiel [6]. >> >> This project could initially serve as a runtime-selectable alternative >> to the current built-in implementation. Upon reaching sufficient >> maturity however, it could potentially replace the existing manager. >> >> Please share your thoughts by replying to this email. Alternatively, >> we can discuss this topic at the Catalog Sync meeting this Wednesday, >> June 18th, if that is a more comfortable option to everyone. >> >> Thanks, >> >> Alex >> >> [1] >> https://medium.com/data-engineering-with-dremio/introducing-dremio-auth-manager-for-apache-iceberg-223827342d19 >> [2]: https://github.com/apache/iceberg/issues/12196 >> [3]: https://github.com/apache/iceberg/issues/12363 >> [4]: https://github.com/apache/iceberg/issues/13030 >> [5]: https://github.com/apache/iceberg/issues/10677 >> [6]: >> https://docs.google.com/document/d/1buW9PCNoHPeP7Br5_vZRTU-_3TExwLx6bs075gi94xc/edit?tab=t.0#heading=h.hufqidg1ij89 >> >
