Hi JB and Dan, Thanks for the reviews. I agree we should be clear on how authorization for the FILE type works for a REST catalog server, and that the interface supports those semantics.
The initial batch api proposal assumes, like the existing access delegation endpoints, that the server authorizes the request's principal against the table. Regarding Dan's point on including more granular auth context into the request: since an external FILE reference originates from a column in a data file, I can see why we'd want that context to be in the authorization input. I think JB made some great observations on this topic: 1. The context would be an optional, spec-defined input (data file location and field ID), kept separate from the server-provided `properties`. 2. Use field-id over column names 3. It is a claim from the client. It informs the server's decision but is not proof, so for shared storage the server still needs its own control and verification. Because the context is optional, adding it later does not change the behavior of clients and servers that don't use it, and a server that needs it can refuse requests without it. So I think we can treat it as an enhancement rather than block the batch API on it. I think where the context lives is still an open question. JB makes a good case for defining it at the element level in `RemoteSignBatchElement`, since a batch can span several data files and FILE columns. But keeping it at `RemoteSignBatchRequest` would keep elements smaller, at the cost of clients grouping by data file and field. Either way we could add it as an optional context, and if we do support it, I think it's also worth considering the feature across other access delegation modes (like vended credentials). That's my thought on this specific requirement. But that being said, I'm happy to keep this thread open while the FILE implementation settles, so we capture authorization semantics we haven't thought of yet. Sung On 2026/11/09 19:58:13 Jean-Baptiste Onofré wrote: > Hi guys, > > Thanks for bumping this thread (I just saw that thanks to Dan :)). > > I agree with Dan: authorization should drive the request shape, and it's > easier to settle it now. > > A few thoughts: > - If the server needs context to authorize a FILE reference, it has to be > per element, as a batch can span several data files and columns. I would > keep the top level properties as it is and add an optional context on > RemoteSignBatchElement. Mixing client provided context with server provided > static properties in the same map would be confusing imho. > - If servers rely on this context to autorize, all clients (Java, Python, > Rust, ...) have to send the same thing, so I think it should be defined in > the spec (for instance data file location and field ID) and not as opaque > keys in a map. I would also use the field ID rather than the column name: > names change on rename and are ambiguous for nested fields. > - The context is a claim from the client. The server can check that the > caller has access to the column (and maybe that the data file is part of > the table), but it can't verify that the URI is actually referenced in that > file without reading it. So for shared storage, the server still needs its > own control (for instance allowed locations for a FILE column), and the > spec should be clear that the context helps the decision but is not a proof > by itself. On the cost side, elements sharing the same (data file, field > ID) can be authorized once, so the cost scales with the number of distinct > pairs, not with the number of rows. > > Per element authorization also answers the partial results question from > the PR review: with shared storage, some elements of a batch can be signed > while others are denied, so the completed/failed split makes sense. > > I would wait for the File type impl before a vote. As Sung said, #18080 > should land first, so the X-Iceberg-Access-Delegation handling is defined > once for /sign and /sign/batch. > > Regards > JB > > On Mon, Nov 9, 2026 at 5:12 PM Daniel Weeks via dev <[email protected]> > wrote: > > > Hey Sung, > > > > Thanks for starting this thread, I wanted to bump it because I'd like to > > wait for the file implementation before settling on this. > > > > One of the primary uses of this endpoint is for File type access, but we > > haven't yet discussed the overhead of the authorization cost. > > > > I know auth isn't part of the REST spec, but we should be considerate of > > the challenges. In the simple case, we can likely authorize file access > > that lives under the table, but for scenarios where files reference shared > > storage, we need to determine whether a request can be signed or not. This > > might require additional information (like the file containing the > > reference and the column name from the source table) in order for the > > server to validate access. > > > > This might require changing the request design. Originally, we (or I) > > assumed that the properties would be the same for all paths, but if we need > > to add contextual information like above, then we would need properties at > > the request level. > > > > -Dan > > > > On Mon, Oct 26, 2026 at 2:32 PM Sung Yun <[email protected]> wrote: > > > >> Hi folks, > >> > >> I'd like to start a discussion on adding a batch signing endpoint to the > >> REST catalog spec. This follows the earlier threads on File-level access > >> delegation[1], and Access Delegation for Iceberg FILE type in the REST > >> Catalog[2]. > >> > >> PR: https://github.com/apache/iceberg/pull/18254 > >> Design doc: https://s.apache.org/irc-batch-signing > >> > >> Today a client signs one object request per call to signRequest. For > >> table data this is bounded by the number of data and delete files touched > >> by a scan. However, with the recently adopted FILE type[3], where each row > >> may reference an external object, the number of signatures scales with > >> rows. > >> > >> The PR proposes a new batch signing endpoint to support that scale. > >> > >> Reviews on the PR or replies here are both welcome. > >> > >> Thank you, > >> Sung > >> > >> [1] https://lists.apache.org/thread.html/4fxylxkfj0ngp0ksfysgkhn8s6o9x4n5 > >> [2] https://lists.apache.org/thread.html/s283f6obyqf6c14bn37sw7vtf9y1c9mj > >> [3] https://github.com/apache/iceberg/pull/17918 > >> > > >
