Thanks for raising this, Vignesh. I think this makes sense for private
link use cases. Is configuring different endpoints all we need to
support private links?

Yufei

On Sun, Nov 8, 2026 at 11:07 PM Jean-Baptiste Onofré <[email protected]> wrote:
>
> Hi Vignesh,
>
> I agree that we have a difference here. It makes sense to align Azure and
> GCS with S3.
>
> I have a few comments:
> - For S3, we distinguish endpoint (what clients see) from endpointInternal
> (what Polaris uses), plus stsEndpoint. In private link or Kubernetes
> deployments, the host Polaris reaches is often not the one clients use. I
> think the Azure/GCS fields should follow the same logic, with the same
> naming, so users won't be surprised.
> - The security gate is the main point for me. For Azure and GCS, Polaris
> sends bearer token to these hosts (the Entra token for the user delegation
> key request, the source GCP token for IAM/STS). Unlike SigV4 on S3, a
> bearer token can be replayed. As storage config can be set by catalog
> admins, the new fields should be guarded by a realm feature flag, similar
> to ALLOW_SETTING_S3_ENDPOINTS. Given the bearer token aspect, I would go
> with disabled by default (but happy to discuss that :)). I think it's also
> aligned with our thread model: configured endpoints must not redirect
> credentials outside the intended trust relationship.
> - The Entra authority and the GCP STS/IAM hosts related to the server
> identity, not to a specific catalog. I would prefer them as server level
> configuration rather than catalog storage config fields. For Entra, I think
> DefaultAzureCredential already honors the AZURE_AUTHORITY_HOST env
> variable, so it's worth to check whether that's enough before adding a new
> Polaris config.
> - In the Google Auth version we use (1.55.0), I checked and
> DownscopedCredentials buids the STS URL from the domain, and the builder
> has no endpoint setter. So a custom STS endpoint can't just be passed
> through. We would need either a domain approach or our own token exchange
> behind GcpCredentialOps. The IAM credentials side is easier, as
> IamCredentialsSettings supports setEndpoint().
> - AzureLocation currently rejects any host other than blob/
> dfs.core.windows.net. My preference would be to keep locations unchanged
> and have the endpoint override only change the network target. The way,
> location validation and allowed locations keep working as they do today. On
> the client side, we could use Iceberg adls.connection-string-xxx.
>
> As this changes the Management API, I suggest to split the work into two
> PRs: one for the spec, and the follow up.
>
> Happy to review the PRs :)
>
> Regards
> JB
>
> On Sat, Nov 7, 2026 at 4:46 PM vignesh a <[email protected]> wrote:
>
> > Hi all,
> >
> > Polaris can already vend S3 credentials against a non-public
> > endpoint. Catalog storage configuration includes endpoint and
> > stsEndpoint, and the Floci S3 integration tests run
> > vended-credentials on that path.
> >
> > Azure and GCS have no equivalent. Polaris mints Azure
> > user-delegation SAS against
> > https://{account}.dfs.core.windows.net (or blob), derived from
> > the table location. GCS downscoping always uses the public IAM
> > and STS hosts. Private-link and emulator deployments cannot
> > direct that minting at a local or private endpoint.
> >
> > The in-tree Floci Azure and GCS integration tests show the
> > gap. Iceberg clients receive static shared-key or OAuth tokens
> > plus a client host extra, because Polaris cannot vend through
> > Floci.
> >
> > Therefore I propose optional catalog endpoint fields for Azure
> > and GCS, analogous to S3 endpoint and stsEndpoint. Polaris
> > would use them when minting Azure SAS and GCS downscoped
> > tokens, and would pass the storage endpoint to Iceberg clients
> > as FileIO extras. Azure token fetch uses the default credential
> > chain against public Entra, so an emulator that also issues
> > tokens may need an authority override as well.
> >
> > The Floci Azure and GCS integration tests could then use
> > vended-credentials instead of static tokens.
> >
> > If this direction looks reasonable, I'll put up a PR &
> > adjust the shape based on feedback here..
> >
> > Thanks,
> > Vignesh
> >

Reply via email to