Hi Yufei, Thanks for writing this up. I went through your points and I agree: let's skip Phase 1 and make storage configurations their own entities.
On your points: 1. Default base location. This goes beyond create-table. With default settings a namespace can't have a custom location, so every namespace location is derived from the catalog's default-base-location. A namespace bound to an Azure configuration in an S3 catalog would get an s3:// location, and every table in it would fail the allowed-locations check. The only workaround is turning on ALLOW_NAMESPACE_CUSTOM_LOCATION and setting each namespace's location by hand. An entity doesn't fix this by itself either. I'd give each configuration an optional base location, use it for namespaces and tables bound before they're created (more below), and refuse to bind an existing namespace or table whose location isn't inside the configuration's allowed locations. 2, 3 and 5. Agreed. On 3: #5721 already moved to set/remove fields, but it's still one catalog entity with one version, so two admins editing different configurations conflict. 4. Access control. Agreed, and this is the real cost of skipping Phase 1: we deferred AuthZ in June to unblock it, and now we need it up front. I'd follow the policy model: create, read, write, drop and list privileges on the configuration, and attach as a pair (STORAGE_CONFIG_ATTACH on the configuration plus a dedicated attach privilege on the namespace or table), all granted to catalog roles. It shouldn't be TABLE_WRITE_PROPERTIES, because TABLE_WRITE_DATA includes it and every writer could move a table to another identity. I'd leave out detach. Unlike a policy, unbinding a table that has data only points it at a configuration that can't reach its files. Changing a binding is a rebind, with the same location check as above. On your questions: 1. Skipping Phase 1: +1. 2. Option B. Policies already do most of what we need, but they fit storage poorly. A policy lives under a namespace, and POLICY_CREATE is a namespace privilege. Its content is an untyped JSON string. Applicable policies are returned to anyone with TABLE_READ_PROPERTIES, so role ARNs and external IDs would be visible to every table reader. And resolving them costs a lookup per path level on every loadTable. I'd copy the patterns instead (attach checked on both sides, versioned updates) into a STORAGE_CONFIG entity, and keep the binding as a configuration name on the namespace or table, as you and Dmitri suggested in June. 3. Scope. Catalog-scoped. A configuration holds an identity (role ARN and external ID, Azure tenant, GCS service account) and a fence (allowed locations, base location). The identity is the part worth sharing. The fence has to stay per catalog, because the cross-catalog overlap check is what stops one catalog's admin from getting credentials for another catalog's files. A configuration shared across catalogs, locations included, would fail that check, and the only way around it is ALLOW_OVERLAPPING_CATALOG_URLS, which is realm-wide. So I'd start catalog-scoped with the identity inline. Later the identity can become a realm-level object, managed by service_admin, that configurations reference by name. That's additive, and close to how Unity Catalog separates credentials from external locations. One more thing worth changing while we're at it: how a table ends up on a configuration other than its parent's. A binding needs an existing table, but Polaris writes the first metadata.json (and, for CTAS, vends the first credentials) with the parent's configuration. A Polaris-specific create-and-bind call would get around, but Spark, Trino and dbt can't use it directly. Instead (or in addition), an admin could bind a name ahead of time: "finance.ledger uses configuration finance". When a client then creates finance.ledger through the normal createTable, Polaris uses that configuration for the location, the first write and the credentials, and moves the binding onto the new table. Namespaces work the same way, which is what fixes point 1. The obvious concerns are stale bindings, renames, drop and re-create, and who owns a name that doesn't exist yet. I think each has an answer: a pending binding is used up at create and from then on behaves like any other binding, dropping the configuration deletes its pending bindings, and the namespace owns the names under it, so binding ahead needs the same privileges as binding the namespace. It does need a small table for pending bindings. If that's too much, we could allow it for namespaces only; tables then follow their namespace. If this sounds right, I'll update the design doc and share it here before opening a spec PR. I'll keep #5721 open until then.
