Hi Alex, Yufei and Dmitri,

Thanks for the detailed answers, Alex. Replies to the open points:

- DuckDB: from what I can see it does not support remote signing today. The 
Iceberg extension's ACCESS_DELEGATION_MODE option only accepts 
vended_credentials (the default) and none [1]. Support is tracked in 
duckdb-iceberg #670 (open since January; a maintainer said it is wanted but 
lower priority) [2], and a community pull request, #1318, adds a remote_signing 
mode through an Iceberg-owned filesystem; it is still open, and the maintainers 
asked whether it can be done in httpfs instead [3].

- Persistence (Yufei): we use the relational JDBC persistence on PostgreSQL. If 
it helps your Kubernetes benchmark, that is the setup I can test against.

- PyIceberg: we run PyIceberg 0.11.1 against our catalog, with the s3fs extra 
installed, so the fsspec FileIO is available. Once a build exists I will test 
it with py-io-impl=pyiceberg.io.fsspec.FsspecFileIO on existing tables and 
report back what works with an S3-compatible store.

- Spark: our compaction runs on Spark with the Iceberg 1.10 runtime, so it 
would be in compat mode (existing tables only). Good to know that works; I can 
try the 1.12 runtime for a test if that is useful.

- Decision cache: from a user's side, a configurable TTL that is off by 
default, with the maximum revocation delay documented as part of the contract, 
is fine for us.

- Drop with purge: we run with table purge disabled and delete files outside 
the catalog, so we have no stake in the post-drop signing question and no 
preference between the token-recorded decision and the soft-delete approach.

Best,
Tommy

[1] https://duckdb.org/docs/stable/core_extensions/iceberg/iceberg_options 
(ACCESS_DELEGATION_MODE: "Allowed values are vended_credentials and none")
[2] https://github.com/duckdb/duckdb-iceberg/issues/670
[3] https://github.com/duckdb/duckdb-iceberg/pull/1318


________________________________________
From: Dmitri Bourlatchkov <[email protected]>
Sent: Friday, November 13, 2026 18:19
To: [email protected] <[email protected]>
Cc: Yufei Gu <[email protected]>; Tommy Bollhofer 
<[email protected]>; Alex Dutra <[email protected]>
Subject: Re: [DISCUSS] Resume remote signing work


Hi Alex,

Re: the DROP_TABLE_WITH_PURGE proposal:

It looks like it should work with the location overlap checks that you 
mentioned. However, can those checks slow down the signing process?

Assuming the performance is acceptable, I think it is an interesting approach 
and is worth exploring.

DROP_TABLE_WITH_PURGE protects server-side actions invoked via dedicated API 
methods / parameters.

This use case is different in that all deletes are performed on the client 
side. Any client with normal write access can perform these deletes before 
dropping the table from the catalog. So extending the validity of the signing 
token beyond the lifetime of the catalog entity does not look like an 
authorization decision to me. Since we validate that locations do not overlap, 
signed requests will stay within the scope of the original entity anyway.

This feels more like a catalog-level flag to me with the value being the 
maximum time period allowed for use on deleted entities. However, using the 
authorizer for this with different operation name (e.g. 
DELETE_FILES_AFTER_DROP) works too.

Re: soft delete:

I think leveraging the soft-delete feature to support Spark use cases is more 
robust and easier for users to manage.

The soft-deleted entity still maintains a hold on table locations. I think this 
offers a stronger protection against overlaps than one-off checks in each 
signing request (check/create races are still possible).

Undropping the table could check if (some key) files are still present.

And yes, we can indeed offer the option of denying engines all access to 
soft-deleted table files.

Cheers,
Dmitri.

On Fri, Nov 13, 2026 at 5:15 PM Alex Dutra via dev <[email protected]> 
wrote:
Hi all,

One more remote signing question, raised while testing with Spark.

Spark implements DROP TABLE ... PURGE client-side: it loads the table,
drops it without purging it, then deletes its files itself, with the
FileIO of the loaded table [1]. With vended credentials this works, as
the credentials outlive the drop. With per-request authorization, each
sign request is authorized against a table that no longer exists, so
the files are left behind.

Proposal: when a table is loaded, created or registered, Polaris also
asks the authorizer whether the caller is allowed to
DROP_TABLE_WITH_PURGE, and records the outcome in the remote
signing token, along with the table's ID and locations. Once the table is
dropped, sign requests carrying a valid token are allowed if the recorded
decision allows it, the request only reads or deletes objects at the
recorded locations,
the table was dropped rather than renamed, and no other table or
namespace now uses those locations. Requests without a token are
rejected.

Requests for tables that exist are still authorized per request, with
or without a token. The decision is only used once the table's grants
are gone with it, and the authorizer (including OPA and Ranger) makes
it, at load time. Spark loads the table right before purging it, so
the decision is recent.

The soft-delete proposal [2] would let Polaris authorize such requests
against the retained table instead, when enabled. Note that Spark's
purge is a drop without purge followed by client-side deletes, so with
soft delete on, Spark would delete the files of a table that can still
be undropped, unless the catalog refuses it. Remote signing would make
refusing possible.

Thoughts?

Thanks,
Alex

[1] 
https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
[2] https://lists.apache.org/thread/1djyz0x3k7m71d71bqhw11c3syyshtdr


On Wed, Nov 11, 2026 at 9:21 AM Alex Dutra <[email protected]> wrote:
>
> Hi Yufei,
>
> Yes I think the cache is reasonable as long as it becomes part of the
> security constraints and is clearly documented.
>
> However, I would not take that -19% as granted: my benchmarks were on
> my local laptop only, where network calls to Postgres are cheap. I
> still need to reproduce the benchmark on k8s on some cloud provider,
> which I will shortly. But I expect those -19% to become worse, in
> which case, the cache may turn out to be an essential part of the
> setup.
>
> Thanks,
> Alex
>
> On Tue, Nov 10, 2026 at 7:24 PM Yufei Gu <[email protected]> wrote:
> >
> > On second thought, I believe invalidating the decision cache is hard even 
> > with native RBAC, potentially making the system really complicated. Given 
> > that OPA/Ranger cannot even support it, we'd prefer the simpler 
> > solution(TTL).
> >
> > It looks like other systems provide the same decision cache with certain 
> > constraints. However, A cache is acceptable only when its maximum 
> > revocation delay is an explicit part of the security contract. AWS Verified 
> > Permissions[1] caches authorization decisions for 120 seconds by default, 
> > and says policy changes may not take effect until the cache expires. 
> > OpenFGA has caching off by default and offers a higher consistency mode 
> > that bypasses it.
> >
> > I think we should make the decision cache configurable and off by default. 
> > When it is on, we must explicitly state the maximum revocation delay as 
> > part of the security contract.
> >
> > Tommy, would you mind sharing which persistence you are using? I think -19% 
> > for JDBC isn't too bad as the initial version, given that both 
> > communities(Iceberg, Polaris) are actively working on batch signing.
> >
> > 1. 
> > https://docs.aws.amazon.com/verifiedpermissions/latest/userguide/policy-stores-api-userpool-considerations-troubleshooting.html
> >
> > Yufei
> >
> >
> >
> > On Mon, Nov 9, 2026 at 11:36 AM Alex Dutra via dev <[email protected]> 
> > wrote:
> >>
> >> Hi Tommy,
> >>
> >> Thanks for offering to test!
> >>
> >> Before answering your questions, I forgot to say that I am also
> >> planning to introduce a "compatibility mode" for older clients: this
> >> mode is automatically activated when there is no signing token in the
> >> request. Then Polaris would proceed as follows:
> >>
> >> - Sign requests without a token, on an existing table: accepted.
> >> - Sign requests without a token, on a missing table: rejected.
> >>
> >> In this mode, older clients can read and write existing tables, but
> >> get a clear error on CTAS (staged-creates). I think that strikes a
> >> good balance between support for older clients, while also taking
> >> security seriously by rejecting staged-creates.
> >>
> >> With that in mind:
> >>
> >> - Spark: with the Iceberg 1.12 runtime, all operations will be
> >> supported. With the 1.10 runtime, we would be in compat mode, and
> >> reads and writes on existing tables should work too, but not CTAS.
> >>
> >> - PyIceberg: should work for existing tables too, with the same
> >> limitation, but only with its fsspec FileIO
> >> (py-io-impl=pyiceberg.io.fsspec.FsspecFileIO); the default PyArrow
> >> FileIO does not support remote signing. I haven't tested it yet, so
> >> your feedback would be welcome.
> >>
> >> - Trino: no. Trino accesses S3 through its own file system, which does
> >> not implement remote signing, whatever the Iceberg library version.
> >>
> >> - DuckDB: no; as far as I know, its Iceberg extension does not support
> >> remote signing. Could you confirm please?
> >>
> >> I'll let you know when a build is available.
> >>
> >> Thanks,
> >> Alex
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Nov 9, 2026 at 4:13 PM Tommy Bollhofer <[email protected]> 
> >> wrote:
> >> >
> >> > Hi Alex, Yufei and Dmitri,
> >> >
> >> > Thanks for the write-up and the numbers, Alex. Some input from a user of 
> >> > a FlashBlade (S3-compatible, no AssumeRole) deployment, where every 
> >> > client today holds static storage keys:
> >> >
> >> > - Per-request authorization with the short decision cache as the 
> >> > default, and signing tokens only where staged creates need them, sounds 
> >> > right to us. Our workloads are short, and a revocation taking effect 
> >> > within the cache TTL (30 seconds) is fine for us. We would not want a 
> >> > no-cache mode to be the default, given the numbers.
> >> > - On TABLE_REMOTE_SIGN: either way works for us, because our data-writer 
> >> > roles already hold CATALOG_MANAGE_CONTENT (which would imply it). We 
> >> > have no objection to dropping it and relying on TABLE_READ_DATA / 
> >> > TABLE_WRITE_DATA plus the catalog-level REMOTE_SIGNING_ENABLED flag, 
> >> > which keeps the privilege model simpler.
> >> > - A 24-hour default for the signing token, configurable, is fine for us.
> >> >
> >> > One question: Do you have a view on which engines and clients will be 
> >> > able to use remote signing in the first release (Trino, Spark, 
> >> > PyIceberg, DuckDB)? Our stack is Trino with Iceberg 1.12, Spark on the 
> >> > Iceberg 1.10 runtime (we can move it up for a test), PyIceberg and 
> >> > DuckDB. We would test whichever of them support it as soon as a build is 
> >> > available, and report back what works on FlashBlade.
> >> >
> >> > Best,
> >> > Tommy
> >> >
> >> >
> >> >
> >> >
> >> > ________________________________________
> >> > From: Alex Dutra via dev <[email protected]>
> >> > Sent: Monday, November 09, 2026 09:16
> >> > To: [email protected] <[email protected]>
> >> > Cc: Dmitri Bourlatchkov <[email protected]>; Alex Dutra 
> >> > <[email protected]>
> >> > Subject: Re: [DISCUSS] Resume remote signing work
> >> >
> >> >
> >> > Hi Yufei, Dmitri,
> >> >
> >> >
> >> >
> >> > Thanks for the feedback!
> >> >
> >> >
> >> >
> >> > @Yufei:
> >> >
> >> >
> >> >
> >> > The NoSQL overhead is due to cache deserialization. The NoSQL cache
> >> >
> >> > was designed to be GC friendly and holds serialized entities; but in a
> >> >
> >> > path that is mostly CPU bound like the sign endpoint, deserialization
> >> >
> >> > quickly becomes the bottleneck, even with zero database calls. We are
> >> >
> >> > aware of that, and already have a few ideas to improve its efficiency.
> >> >
> >> >
> >> >
> >> > Cached decisions: we could invalidate the cache on permission changes,
> >> >
> >> > but that would only work for built-in RBAC indeed. For OPA or Ranger,
> >> >
> >> > revocations would take effect within the cache TTL. That's why I think
> >> >
> >> > the cache TTL should be low: 30 seconds by default. I can benchmark
> >> >
> >> > shorter TTLs. That's also why the cache should only be applied to the
> >> >
> >> > sign endpoint, not to others.
> >> >
> >> >
> >> >
> >> > “Signing tokens”: the benchmarks showed that per-request authz remains
> >> >
> >> > doable performance-wise (no token needed), but for staged creates, the
> >> >
> >> > authorizer needs some extra info that only a token can carry. With
> >> >
> >> > this context in mind, I would like to repurpose signing tokens as
> >> >
> >> > follows: every table load, create, register and staged create would
> >> >
> >> > return a token carrying only the table context (operation, catalog,
> >> >
> >> > table; locations as well for staged-creates) and the principal hash,
> >> >
> >> > but not its privileges. Each sign request would still be authorized
> >> >
> >> > individually: on the table for existing tables, or on the namespace
> >> >
> >> > for staged-creates. Permission changes therefore take effect
> >> >
> >> > immediately (modulo the decision cache TTL). The token allows
> >> >
> >> > staged-creates to operate optimally, and ties each sign request to its
> >> >
> >> > initial operation, which improves auditing. Also, since it carries
> >> >
> >> > nothing secret anymore, it could be signed rather than encrypted,
> >> >
> >> > which is cheaper to verify.
> >> >
> >> >
> >> >
> >> > Batch sign: agreed, batch signing would help both models; but note
> >> >
> >> > that once it lands in the spec, it still needs to get support at least
> >> >
> >> > in the Java signer client, so it's not something we can count on
> >> >
> >> > today.
> >> >
> >> >
> >> >
> >> > Permissions: my rationale is that Polaris signs requests with its own
> >> >
> >> > credentials, and I didn’t want this to be possible without an explicit
> >> >
> >> > permission. But the catalog-level REMOTE_SIGNING_ENABLED flag already
> >> >
> >> > lets admins opt in, so I'm OK to drop TABLE_REMOTE_SIGN and rely on
> >> >
> >> > TABLE_READ_DATA / TABLE_WRITE_DATA, if everybody is on board.
> >> >
> >> >
> >> >
> >> > @Dmitri:
> >> >
> >> >
> >> >
> >> > The new token I'm proposing doesn’t need to expire for security
> >> >
> >> > reasons (it's not a security token), only for housekeeping sake (allow
> >> >
> >> > key rotation, for instance). Therefore, I think one validity window is
> >> >
> >> > enough, and one that is relatively generous is fine: I'd go with 24
> >> >
> >> > hours by default, configurable.
> >> >
> >> >
> >> >
> >> > Thanks,
> >> >
> >> > Alex
> >> >
> >> >
> >> >
> >> > On Sat, Nov 7, 2026 at 2:18 AM Dmitri Bourlatchkov via dev
> >> >
> >> > <[email protected]> wrote:
> >> >
> >> > >
> >> >
> >> > > Hi Alex,
> >> >
> >> > >
> >> >
> >> > > It's great to see progress on remote signing.
> >> >
> >> > >
> >> >
> >> > > I think the dual approach you proposed is reasonable.
> >> >
> >> > >
> >> >
> >> > > Re: staged creates: the long-lived signing token is similar to a vended
> >> >
> >> > > credential. I believe it has the same problem that if the staging 
> >> > > operation
> >> >
> >> > > takes a very long time, the token (or credential) can expire [1].
> >> >
> >> > >
> >> >
> >> > > However, with signing tokens the Polaris administrator has more control
> >> >
> >> > > over assigning the appropriate credential validity window, so I think 
> >> > > it is
> >> >
> >> > > a useful feature to have.
> >> >
> >> > >
> >> >
> >> > > It might be worth supporting different validity window configurations 
> >> > > and
> >> >
> >> > > different authorizable operations for normal table access and staged 
> >> > > create
> >> >
> >> > > requests. WDYT?
> >> >
> >> > >
> >> >
> >> > > [1] 
> >> > > https://lists.apache.org/thread.html/by053ptrxrxqx9fm57q64bl3bv41vy46
> >> >
> >> > >
> >> >
> >> > > Cheers,
> >> >
> >> > > Dmitri.
> >> >
> >> > >
> >> >
> >> > > On Fri, Nov 6, 2026 at 12:04 PM Alex Dutra via dev 
> >> > > <[email protected]>
> >> >
> >> > > wrote:
> >> >
> >> > >
> >> >
> >> > > > Hi Tommy, hi all,
> >> >
> >> > > >
> >> >
> >> > > > Your assistance in testing and evaluating this feature is very 
> >> > > > welcome!
> >> >
> >> > > >
> >> >
> >> > > > Here are a few updates on my work:
> >> >
> >> > > >
> >> >
> >> > > > Since the old design doc [1] is mostly out of date, I'm currently
> >> >
> >> > > > exploring two design choices for authorizing sign requests that we
> >> >
> >> > > > briefly discussed, but never settled on:
> >> >
> >> > > >
> >> >
> >> > > > 1. A signing token, issued when the table is loaded and validated by
> >> >
> >> > > > the sign endpoint; this only works with Iceberg 1.12+. This is 
> >> > > > similar
> >> >
> >> > > > to how Project Nessie does remote signing. In this mode, the
> >> >
> >> > > > authorization decision is expected to be fast, because it is
> >> >
> >> > > > pre-computed in the token.
> >> >
> >> > > >
> >> >
> >> > > > 2. Per-request authorization, with or without a short-lived decision
> >> >
> >> > > > cache; this works for any Iceberg client. In this mode, authorization
> >> >
> >> > > > is done the normal way, which may incur in calls to the database
> >> >
> >> > > > and/or to the external authorizer, if any. The decision cache is a
> >> >
> >> > > > small enhancement where an authorization decision is cached for 30
> >> >
> >> > > > seconds by default.
> >> >
> >> > > >
> >> >
> >> > > > To compare them, I created a new Gatling benchmark for remote signing
> >> >
> >> > > > [2], and I'm collecting preliminary results by running it locally
> >> >
> >> > > > against my working branch. A few early numbers (single machine,
> >> >
> >> > > > Polaris capped at 4 CPUs, 1000 sign requests per table load, so take
> >> >
> >> > > > them as indicative only), in max sign requests/s per node:
> >> >
> >> > > >
> >> >
> >> > > > - JDBC: token 12.7k, per-request without cache 10.3k (-19%),
> >> >
> >> > > > per-request with cache 12.4k
> >> >
> >> > > >
> >> >
> >> > > > - NoSQL: token 11.8k, per-request without cache 4.8k (-60%),
> >> >
> >> > > > per-request with cache 12.0k
> >> >
> >> > > >
> >> >
> >> > > > Using per-request authz without a decision cache is clearly the worst
> >> >
> >> > > > scenario, as expected: the gap is mostly due to entity resolution
> >> >
> >> > > > costs, and also cache deserialization for the NoSQL backend. I also
> >> >
> >> > > > tested external principals (credential-mode=external) and OPA: the
> >> >
> >> > > > numbers are similar or slightly worse due to repeated calls to OPA.
> >> >
> >> > > >
> >> >
> >> > > > However, per-request authz with a decision cache is roughly on par
> >> >
> >> > > > with signing tokens. That's an interesting observation and begs the
> >> >
> >> > > > question: which method should Polaris implement? Both?
> >> >
> >> > > >
> >> >
> >> > > > Each approach has pros and cons:
> >> >
> >> > > >
> >> >
> >> > > > - Signing tokens:
> >> >
> >> > > >   - Pros: works out of the box with staged table creations: we 
> >> > > >capture
> >> >
> >> > > > the authz decision at load time
> >> >
> >> > > >   - Cons: tokens are valid for X hours, and cannot be revoked or
> >> >
> >> > > > refreshed (at least not for now - maybe later)
> >> >
> >> > > >
> >> >
> >> > > > - Per-request authz with decision cache:
> >> >
> >> > > >   - Pros: cache is short-lived (a few seconds) compared to token 
> >> > > >lifetime
> >> >
> >> > > >   - Cons: does not work well with staged table creations (the sign
> >> >
> >> > > > request arrives before the table commit)
> >> >
> >> > > >
> >> >
> >> > > > I am inclined to use a mix of both: per-request authz by default,
> >> >
> >> > > > except for staged table creations where we'd use signing tokens.
> >> >
> >> > > >
> >> >
> >> > > > What are your thoughts?
> >> >
> >> > > >
> >> >
> >> > > > Thanks,
> >> >
> >> > > > Alex
> >> >
> >> > > >
> >> >
> >> > > > [1]:
> >> >
> >> > > > https://docs.google.com/document/d/1ygdia7u4bUHUt6n8XhZo48aKoIyyrCvKqan3XP25iB8/edit?usp=sharing
> >> >
> >> > > > [2]: https://github.com/apache/polaris-tools/pull/304
> >> >
> >> > > >
> >> >
> >> > > > On Mon, Nov 2, 2026 at 9:27 PM Tommy Bollhofer 
> >> > > > <[email protected]>
> >> >
> >> > > > wrote:
> >> >
> >> > > > >
> >> >
> >> > > > > Hi Alex,
> >> >
> >> > > > >
> >> >
> >> > > > > Thank you for the update, and for linking the original thread.
> >> >
> >> > > > >
> >> >
> >> > > > > Great to hear that you're planning to revisit the previous PR. 
> >> > > > > We'd be
> >> >
> >> > > > happy to test an experimental build against our on-premises 
> >> > > > FlashBlade
> >> >
> >> > > > S3-compatible storage and share our findings, including compatibility
> >> >
> >> > > > across the various engines and versions we're currently using.
> >> >
> >> > > > >
> >> >
> >> > > > > Our client stack includes Trino with Iceberg 1.12, Spark with the
> >> >
> >> > > > Iceberg 1.10 runtime, PyIceberg, and DuckDB. Given the relatively 
> >> > > > short
> >> >
> >> > > > duration of our workloads, we should be able to begin testing without
> >> >
> >> > > > requiring credential refresh support.
> >> >
> >> > > > >
> >> >
> >> > > > > Please let us know when there's a build available to evaluate. 
> >> > > > > We'd be
> >> >
> >> > > > glad to help validate the implementation and provide feedback.
> >> >
> >> > > > >
> >> >
> >> > > > > Thanks again for your work on this!
> >> >
> >> > > > >
> >> >
> >> > > > > Best,
> >> >
> >> > > > > Tommy
> >> >
> >> > > > >
> >> >
> >> > > > >
> >> >
> >> > > > > ________________________________________
> >> >
> >> > > > > From: Alex Dutra <[email protected]>
> >> >
> >> > > > > Sent: Tuesday, October 27, 2026 13:19
> >> >
> >> > > > > To: [email protected] <[email protected]>
> >> >
> >> > > > > Cc: Tommy Bollhofer <[email protected]>
> >> >
> >> > > > > Subject: Re: [DISCUSS] Resume remote signing work
> >> >
> >> > > > >
> >> >
> >> > > > >
> >> >
> >> > > > > Hi Tommy,
> >> >
> >> > > > >
> >> >
> >> > > > > Thanks for chiming in! Your reply got disconnected from the thread
> >> >
> >> > > > though, so here is the link to my first message just in case:
> >> >
> >> > > > >
> >> >
> >> > > > > https://lists.apache.org/thread.html/vk5zj7qtrztl36c9c1qt6d09lryjltdk
> >> >
> >> > > > >
> >> >
> >> > > > > The details you provide are very valuable: since FlashBlade doesn't
> >> >
> >> > > > support AssumeRole, remote signing becomes indeed a good 
> >> > > > alternative. And
> >> >
> >> > > > since you can live for now without auth-credentials refresh, it 
> >> > > > seems to me
> >> >
> >> > > > we have enough evidence that we should move forward and provide a 
> >> > > > first,
> >> >
> >> > > > experimental implementation of remote signing.
> >> >
> >> > > > >
> >> >
> >> > > > > I will start preparing a revisited PR, built on top of the old one.
> >> >
> >> > > > >
> >> >
> >> > > > > Thanks,
> >> >
> >> > > > > Alex
> >> >
> >> > > > >
> >> >
> >> > > > >
> >> >
> >> > > > >
> >> >
> >> > > > >
> >> >
> >> > > > > On Thu, Oct 22, 2026 at 7:06 PM Tommy Bollhofer via dev <
> >> >
> >> > > > [email protected]> wrote:
> >> >
> >> > > > > Hi all,
> >> >
> >> > > > >
> >> >
> >> > > > > +1 to resuming the remote signing work now, rather than waiting 
> >> > > > > for the
> >> >
> >> > > > > spec on refreshing auth data. Alex asked on the issue (#32) for 
> >> > > > > our use
> >> >
> >> > > > > case, so here it is.
> >> >
> >> > > > >
> >> >
> >> > > > > We run Polaris against an on-prem S3-compatible store (a 
> >> > > > > FlashBlade).
> >> >
> >> > > > > Its STS support is limited to AssumeRoleWithWebIdentity, so Polaris
> >> >
> >> > > > > cannot vend credentials through its current AssumeRole flow. As a
> >> >
> >> > > > > result we turn vending off and distribute static storage keys to 
> >> > > > > every
> >> >
> >> > > > > service in every environment: Polaris, Trino, Spark (table 
> >> > > > > maintenance),
> >> >
> >> > > > > and PyIceberg and DuckDB clients. That is a rotation and 
> >> > > > > blast-radius
> >> >
> >> > > > > burden that grows with each environment we add. Remote signing 
> >> > > > > would
> >> >
> >> > > > > remove the static keys from the clients, which is why we would 
> >> > > > > like to
> >> >
> >> > > > > see it move.
> >> >
> >> > > > >
> >> >
> >> > > > > On the auth-refresh gap: our workloads are mostly short queries and
> >> >
> >> > > > > pipeline runs, and our longest jobs are Spark maintenance 
> >> > > > > operations
> >> >
> >> > > > > that take minutes, so as far as we can tell we could start without
> >> >
> >> > > > > refresh support and pick it up incrementally, as Alex suggested.
> >> >
> >> > > > >
> >> >
> >> > > > > Client versions are mixed on our side (Trino on Iceberg 1.12, 
> >> > > > > Spark on
> >> >
> >> > > > > the Iceberg 1.10 runtime, PyIceberg and DuckDB with their own 
> >> > > > > Iceberg
> >> >
> >> > > > > implementations), so we expect client-side support to vary. We 
> >> > > > > would be
> >> >
> >> > > > > happy to test early builds against FlashBlade and report which 
> >> > > > > engines
> >> >
> >> > > > > and versions work.
> >> >
> >> > > > >
> >> >
> >> > > > > (On #32 we also described a smaller option, 
> >> > > > > AssumeRoleWithWebIdentity
> >> >
> >> > > > > as an alternative STS flow. Remote signing is the fuller fix and 
> >> > > > > the
> >> >
> >> > > > > one we would rather see.)
> >> >
> >> > > > >
> >> >
> >> > > > > Best,
> >> >
> >> > > > > Tommy
> >> >
> >> > > > >
> >> >
> >> > > > > On 2026-10-02 15:00 UTC Alex Dutra wrote:
> >> >
> >> > > > > > Hi all,
> >> >
> >> > > > > >
> >> >
> >> > > > > > Polaris will soon be updated to Iceberg 1.12.
> >> >
> >> > > > > >
> >> >
> >> > > > > > This release includes the new remote-signing-config field within
> >> >
> >> > > > > > LoadTableResult, which addresses nearly all remaining 
> >> > > > > > requirements for
> >> >
> >> > > > > > Polaris to complete its remote signing implementation, abandoned 
> >> > > > > > a
> >> >
> >> > > > > > while ago due to impending spec changes.
> >> >
> >> > > > > >
> >> >
> >> > > > > > However, one detail remains: when we added the 
> >> > > > > > remote-signing-config
> >> >
> >> > > > > > structure, refreshing the contained authentication or 
> >> > > > > > authorization
> >> >
> >> > > > > > data was intentionally left out of scope. I plan to start a 
> >> > > > > > separate
> >> >
> >> > > > > > discussion and spec change proposal with the Iceberg community
> >> >
> >> > > > > > regarding this shortly.
> >> >
> >> > > > > >
> >> >
> >> > > > > > Despite this, the core remote signing functionality is again 
> >> > > > > > viable in
> >> >
> >> > > > > > Polaris today, with Iceberg 1.12. The only missing piece being 
> >> > > > > > that
> >> >
> >> > > > > > auth data cannot be refreshed yet, though this capability could 
> >> > > > > > be
> >> >
> >> > > > > > added incrementally in a future update.
> >> >
> >> > > > > >
> >> >
> >> > > > > > With that in mind, I would like to ask the Polaris community:
> >> >
> >> > > > > >
> >> >
> >> > > > > > Should we resume the remote signing work where we left [1]? Or 
> >> > > > > > would
> >> >
> >> > > > > > it be preferable to wait until the specification for refreshing 
> >> > > > > > auth
> >> >
> >> > > > > > data is finalized?
> >> >
> >> > > > > >
> >> >
> >> > > > > > For additional background, you can refer to the original DISCUSS
> >> >
> >> > > > > > thread [2], but keep in mind that much of it is now outdated.
> >> >
> >> > > > > >
> >> >
> >> > > > > > Thanks,
> >> >
> >> > > > > > Alex
> >> >
> >> > > > > >
> >> >
> >> > > > > > [1]: https://github.com/apache/polaris/pull/3995
> >> >
> >> > > > > > [2]:
> >> >
> >> > > > https://lists.apache.org/thread.html/xpf60ko823c7mlg5bhnddq3btvdsq5s1
> >> >
> >> > > > > >
> >> >
> >> > > > >
> >> >
> >> > > > > Tommy Bollhofer
> >> >
> >> > > > > Senior Data Engineer
> >> >
> >> > > > > Voloridge.com
> >> >
> >> > > > > [email protected]
> >> >
> >> > > > > 561.231.5770 <(561)%20231-5770>
> >> >
> >> > > > >
> >> >
> >> > > > > This email and any attachments are intended only for the intended
> >> >
> >> > > > recipient(s) and may contain information that is privileged and/or
> >> >
> >> > > > confidential. If you are not an intended recipient, or have received 
> >> > > > this
> >> >
> >> > > > email in error, please notify the sender immediately and permanently 
> >> > > > delete
> >> >
> >> > > > this email, its attachments, and any copies. Any disclosure, copying,
> >> >
> >> > > > distribution, or reliance on the contents by anyone other than an 
> >> > > > intended
> >> >
> >> > > > recipient is unauthorized. This email message (including any 
> >> > > > attachments)
> >> >
> >> > > > is not to be construed as investment advice or the solicitation of 
> >> > > > an offer
> >> >
> >> > > > to buy or sell interests in any securities. Any information provided 
> >> > > > in
> >> >
> >> > > > this message must be considered in conjunction with the appropriate
> >> >
> >> > > > disclosure documents. PAST PERFORMANCE IS NOT INDICATIVE OF FUTURE 
> >> > > > RESULTS.
> >> >
> >> > > >
> >> >

Reply via email to