JarroVGIT commented on issue #12263:
URL: https://github.com/apache/iceberg/issues/12263#issuecomment-3730570582
@RussellSpitzer I think I may have found an avenue that does not require any
change to the Table spec, nor the REST spec, but with limitations. Like
mentioned above, the key requirements are:
- Possibility to separate RBAC (in the catalog)
- Separate location (preferred, due to vended credentials)
- Completely independent lifecycle (e.g. schema changes/drops/maintenance
should not interfere).
Limitations: I haven't thought it through for clone-of-clone scenario's.
The key learning today was that the REST catalog specifies that multiple
credentials[>1] (for different path prefixes) can be returned in a `GET
<table>` (or `loadTable`) request. In the implementation of a catalog I looked
at, it (logically) only returned a single credential, but this is something
that could be changed (on the implementation side). Not sure if readers can
deal with it, but that is a tomorrow-problem for now.
Then, on the catalog side, I could:
- Add an endpoint to do a clone, which does quite a bit, e.g.:
- copy metadata to new location and strip it from history, new UUID, etc.
- add some custom property like `table.clone.from`,
- put tags on source snapshot and target snapshot (prevent maintenance)
- register clone table.
- Change logic of creating credentials based on existence of
`table.clone.from` to generate more than one vended credential.
- Additional checks and logic on `DROP` requests
It isn't perfect, and it puts a lot of burden on the catalog implementation,
but I think this could work. The most important thing is that it doesn't
require a specification change and (assuming readers can handle multiple
credentials for a table) doesn't change the reader requirements.
[>1]: I did find the document where this specification of the REST spec was
discussed, but I couldn't find a rationale of why this was an array and not
just a single `credential` field. I'll have to look tomorrow how readers would
handle this though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
