Hi Vignesh, I agree that we have a difference here. It makes sense to align Azure and GCS with S3.
I have a few comments: - For S3, we distinguish endpoint (what clients see) from endpointInternal (what Polaris uses), plus stsEndpoint. In private link or Kubernetes deployments, the host Polaris reaches is often not the one clients use. I think the Azure/GCS fields should follow the same logic, with the same naming, so users won't be surprised. - The security gate is the main point for me. For Azure and GCS, Polaris sends bearer token to these hosts (the Entra token for the user delegation key request, the source GCP token for IAM/STS). Unlike SigV4 on S3, a bearer token can be replayed. As storage config can be set by catalog admins, the new fields should be guarded by a realm feature flag, similar to ALLOW_SETTING_S3_ENDPOINTS. Given the bearer token aspect, I would go with disabled by default (but happy to discuss that :)). I think it's also aligned with our thread model: configured endpoints must not redirect credentials outside the intended trust relationship. - The Entra authority and the GCP STS/IAM hosts related to the server identity, not to a specific catalog. I would prefer them as server level configuration rather than catalog storage config fields. For Entra, I think DefaultAzureCredential already honors the AZURE_AUTHORITY_HOST env variable, so it's worth to check whether that's enough before adding a new Polaris config. - In the Google Auth version we use (1.55.0), I checked and DownscopedCredentials buids the STS URL from the domain, and the builder has no endpoint setter. So a custom STS endpoint can't just be passed through. We would need either a domain approach or our own token exchange behind GcpCredentialOps. The IAM credentials side is easier, as IamCredentialsSettings supports setEndpoint(). - AzureLocation currently rejects any host other than blob/ dfs.core.windows.net. My preference would be to keep locations unchanged and have the endpoint override only change the network target. The way, location validation and allowed locations keep working as they do today. On the client side, we could use Iceberg adls.connection-string-xxx. As this changes the Management API, I suggest to split the work into two PRs: one for the spec, and the follow up. Happy to review the PRs :) Regards JB On Sat, Nov 7, 2026 at 4:46 PM vignesh a <[email protected]> wrote: > Hi all, > > Polaris can already vend S3 credentials against a non-public > endpoint. Catalog storage configuration includes endpoint and > stsEndpoint, and the Floci S3 integration tests run > vended-credentials on that path. > > Azure and GCS have no equivalent. Polaris mints Azure > user-delegation SAS against > https://{account}.dfs.core.windows.net (or blob), derived from > the table location. GCS downscoping always uses the public IAM > and STS hosts. Private-link and emulator deployments cannot > direct that minting at a local or private endpoint. > > The in-tree Floci Azure and GCS integration tests show the > gap. Iceberg clients receive static shared-key or OAuth tokens > plus a client host extra, because Polaris cannot vend through > Floci. > > Therefore I propose optional catalog endpoint fields for Azure > and GCS, analogous to S3 endpoint and stsEndpoint. Polaris > would use them when minting Azure SAS and GCS downscoped > tokens, and would pass the storage endpoint to Iceberg clients > as FileIO extras. Azure token fetch uses the default credential > chain against public Entra, so an emulator that also issues > tokens may need an authority override as well. > > The Floci Azure and GCS integration tests could then use > vended-credentials instead of static tokens. > > If this direction looks reasonable, I'll put up a PR & > adjust the shape based on feedback here.. > > Thanks, > Vignesh >
