Hi all,
As we're now very close to releasing 1.9.0, I think we can resume
the discussion about the command naming and detailed output format for a
read-only admin report of tables and views that are incompatible with
structured locations (tracked in [5917]).

The documentation for this behavior has already landed in [5922] and I'd
like to start working on the command, and I need some consensus/pointers on
the command name and output format. Here is a concrete proposal.

Command name

polaris-admin-tool.jar check-structured-locations \
  --realm <realm> [--catalog <catalog>] \
  [--assume-structured | --as-if-flag-false] \
  [--format text|json]


   - --realm: required. The command walks every catalog in the realm unless
   --catalog narrows it to one.
   - --assume-structured: evaluates the structured-location rules as if
   ALLOW_UNSTRUCTURED_TABLE_LOCATION=false, even while the flag is still
   true. This is the pre-flight check before disabling the flag.
   - Without that option, the command uses the effective flag value and
   still reports affected entities when the flag is already false.
   - The command is read-only. It never changes configuration, catalog
   metadata or stored data. It does not block disabling the flag and does not
   repair anything.
   - I'm open to alternatives such as validate-locations or
   report-unstructured-locations.

Output format

The default is human-readable text. It prints the current effective setting
per catalog, then one line per violation, then a summary:

Realm: my-realm
Catalog: prod   ALLOW_UNSTRUCTURED_TABLE_LOCATION=true (catalog
override)   evaluating as: structured

TYPE   ENTITY          LOCATION                        REASON
TABLE  ns2.orders      s3://bucket/prod/ns1/orders     outside
namespace location s3://bucket/prod/ns2
VIEW   ns2.orders_v    s3://bucket/legacy/orders_v     outside
namespace location s3://bucket/prod/ns2

Summary: catalogs=1 entities_checked=412 incompatible=2

--format json prints one object per finding, so operators can script
remediation:

{"realm":"my-realm","catalog":"prod","namespace":["ns2"],"name":"orders","type":"TABLE",
 "location":"s3://bucket/prod/ns1/orders",
 "namespaceLocations":["s3://bucket/prod/ns2"],
 "reason":"OUTSIDE_NAMESPACE_LOCATION"}

Exit codes

   - 0: no incompatible entities found.
   - 1: the report completed and found incompatible entities.
   - A different non-zero code for errors.

Implementation

The command will reuse the server's structured-location validation (the
check introduced in [5856]). That way the report matches what the server
enforces. The Admin Tool docs will get usage and sample output.

Feedback and guidance are really welcome. Please let me know if I’m missing
anything or if there’s anything I should consider.

Thanks

Arnab


[5856]: https://github.com/apache/polaris/pull/5856

[5917]: https://github.com/apache/polaris/issues/5917
[5922]: https://github.com/apache/polaris/pull/5922

On Thu, Oct 29, 2026 at 10:27 PM Yufei Gu <[email protected]> wrote:

> Two issues were filed:
> - #5916: Document the impact of disabling ALLOW_UNSTRUCTURED_TABLE_LOCATION
> <https://github.com/apache/polaris/issues/5916> — assigned to milestone
> *1.9.0*.
> - #5917: Add an admin report for incompatible table/view locations
> <https://github.com/apache/polaris/issues/5917> — optional read-only
> tooling, planned as a follow-up after 1.9.0.
> Yufei
>
>
> On Thu, Oct 29, 2026 at 12:20 AM Jean-Baptiste Onofré <[email protected]>
> wrote:
>
> > Hi guys,
> >
> > I agree with Alex and Russell: we should not block true -> false.
> >
> > Let's start with prep step before the 1.9.0 release as you proposed:
> > 1. We document the caveat in the configuration docs and upgrade notes.
> > 2. The report command to the admin tool makes sense. A read-only
> > command that walks the catalog (or realm) and lists the tables/views
> > whose locations would fail the structured check. That can come in a
> > follow-up after 1.9.0 release.
> >
> > Regards
> > JB
> >
> > On Wed, Oct 28, 2026 at 5:40 PM Yufei Gu <[email protected]> wrote:
> > >
> > > I got that a flawed feature isn't a bug :-). My main concern is that it
> > > breaks other important features, like credential vending and table
> > updates,
> > > which occur much more frequently than updating the flag. Are we OK with
> > > this? Maybe it's fine given that only the catalog admin has the
> > permission.
> > > Let's document this caveat first. Having a tool to check all
> > > locations under a catalog is also a good idea.
> > >
> > > Yufei
> > >
> > >
> > > On Wed, Oct 28, 2026 at 8:23 AM Russell Spitzer <
> > [email protected]>
> > > wrote:
> > >
> > > > I beat you to it Alex :) I see we are thinking alike.
> > > >
> > > > On Wed, Oct 28, 2026 at 10:21 AM Alex Dutra <[email protected]>
> wrote:
> > > >
> > > > > Hi Yufei,
> > > > >
> > > > > I agree that the situation is tricky.
> > > > >
> > > > > Option 1 is indeed expensive, but option 2 would actually prevent
> > > > > someone who cleaned up their warehouse so that all tables comply,
> > from
> > > > > genuinely switching to structured mode. Conceptually, this feels
> > wrong
> > > > > to me, because we are preventing a transition from a less secure
> > setup
> > > > > to a more secure setup.
> > > > >
> > > > > Another concern I have is a practical one: if the flag is set at
> the
> > > > > realm level, how is Polaris going to detect the change?
> > > > >
> > > > > Instead, how about a mix of documentation + a dedicated tool that
> > > > > could report all the tables that *would* fail if structured mode is
> > > > > switched on?
> > > > >
> > > > > Thanks,
> > > > > Alex
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Oct 28, 2026 at 1:19 AM Yufei Gu <[email protected]>
> > wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > While reviewing #5856 (
> https://github.com/apache/polaris/pull/5856),
> > I
> > > > > > noticed that we allow changing ALLOW_UNSTRUCTURED_TABLE_LOCATION
> > from
> > > > > true
> > > > > > to false without checking existing table/view locations. This can
> > break
> > > > > > tables that were valid under the previous setting.
> > > > > >
> > > > > > For example:
> > > > > >
> > > > > >    - Create ns1.t1 at its default location with the flag enabled.
> > > > > >    - Rename it to ns2.t2. The data stays under ns1.
> > > > > >    - Turn the flag off. Loading the table with credentials can
> now
> > fail
> > > > > >    with a 403 because its location is outside ns2.
> > > > > >
> > > > > > The catalog update succeeds, but the problem shows up when
> someone
> > > > > accesses
> > > > > > the table later.
> > > > > >
> > > > > > I see two options:
> > > > > >
> > > > > >    1. Verify every table/view before allowing the change, and
> > reject it
> > > > > if
> > > > > >    any location is incompatible. This could be very expensive for
> > large
> > > > > >    catalogs. We can consider it as phase 2.
> > > > > >    2. Block true → false altogether for now. This should also
> cover
> > > > > >    removing the property when that makes the effective value
> > false. We
> > > > > can
> > > > > >    still allow false → true.
> > > > > >
> > > > > > I’d lean toward option 2 for now. It seems better to reject the
> > > > > > configuration change than let it succeed and break existing
> tables
> > > > > > afterward.
> > > > > >
> > > > > > Any concerns with this approach?
> > > > >
> > > >
> >
>

Reply via email to