Sounds good to me. Let's discuss that in a separate thread. Yufei
On Tue, Jun 9, 2026 at 7:53 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > When locations are not randomized, clients are able (at their discretion) > to stage (long) table creation operations in the same location multiple > times. > > I do not mean to discuss here whether it is wise to do so. It is largely a > concern of the Polaris Administrator, I think. My point is simply that > Polaris behaviour will change in a way that will prevent certain use cases > that were possible before. > > As for actual credential refresh mechanism, let's discuss that in the more > specific thread [1] > > [1] https://lists.apache.org/thread/by053ptrxrxqx9fm57q64bl3bv41vy46 > > Cheers, > Dmitri. > > On Tue, Jun 9, 2026 at 2:00 AM Yufei Gu <[email protected]> wrote: > > > Hi Dmitri, > > > > Just to make sure I understand, is the concern that for long running > staged > > create operations (for example CTAS), clients won't be able to refresh > > vended credentials because the final randomized location isn't known > ahead > > of time? > > > > If so, how is that different from the broader credential refresh problem > > discussed in the staged create thread? Couldn't a refresh mechanism be > tied > > to the assigned location or a refresh token instead of the table > > identifier? > > > > Yufei > > > > > > On Mon, Jun 8, 2026 at 11:37 AM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > Hi All, > > > > > > I support using unique table locations for the reasons stated in > > > previous emails. > > > > > > However, I'd like to highlight the fact that currently, with randomized > > new > > > table locations, it will not be possible for clients executing > > long-running > > > Staged Create operations (usually CTAS [1]) to refresh vended > > credentials. > > > > > > I believe we ought to advise users about that, especially after making > > > randomized locations the default. > > > > > > [1] https://lists.apache.org/thread/by053ptrxrxqx9fm57q64bl3bv41vy46 > > > > > > Cheers, > > > Dmitri. > > > > > > On Mon, Jun 8, 2026 at 12:48 PM Yufei Gu <[email protected]> wrote: > > > > > > > Hi folks, > > > > > > > > I've been working on a change [1] and would like to get feedback. The > > > > change introduces two related capabilities: > > > > > > > > 1. > > > > > > > > Tables and views created without an explicit location will use > > unique > > > > table locations by default. Please note that this is a behavior > > > change. > > > > 2. > > > > > > > > Operators can optionally disable client-specified locations in > > create > > > > table/view requests via a feature flag > > > > ALLOW_CLIENT_SPECIFIED_TABLE_LOCATION. The flag is disabled by > > default > > > > to > > > > preserve the current behavior. > > > > > > > > Existing tables and views are unaffected. Federated catalogs and > > register > > > > table/view workflows are also unaffected. > > > > > > > > Using unique table locations by default has a few benefits: > > > > > > > > - It avoids accidental overlap between tables sharing the same > > > namespace > > > > or storage location. > > > > - It provides stronger isolation between tables, making table > > > lifecycle > > > > operations such as drop, purge, and maintenance (e.g., removing > > orphan > > > > files) safer. > > > > - It makes the overlapping check logic easier and faster. We don't > > > have > > > > to worry about location conflicts under a structured table > location > > > (the > > > > default setting). > > > > > > > > This direction also aligns with the intent of recent Iceberg > > discussions > > > > [2] around unique table locations and catalog managed storage. > > > > > > > > Personally, I think enabling unique table locations by default is the > > > right > > > > choice. For organizations that want stricter control over storage > > > > locations, disabling ALLOW_CLIENT_SPECIFIED_TABLE_LOCATION provides a > > > > straightforward way to enforce catalog managed placement. > > > > > > > > I'd be interested in hearing the community's thoughts on both the > > overall > > > > direction and the proposed defaults. > > > > > > > > 1. https://github.com/apache/polaris/pull/4606 > > > > > > > > 2. https://github.com/apache/iceberg/pull/12892 > > > > > > > > Thanks, > > > > > > > > Yufei > > > > > > > > > >
