sungwy opened a new pull request, #18079: URL: https://github.com/apache/iceberg/pull/18079
Downscoped from @williamhyun's work in #17457. This PR supports the client reading pre-signed URLs if the server returns pre-signed URLs in place of `file-path` in a `planTableScan` response. `PreSignedUrlInputFile` reads an `http` or `https` location as given, with a shared HTTP client. Note for reviewers: A few assumptions were made regarding the intended user workflow for this initial scope: 1. An `http` or `https` location in a scan task, in place of the native location, is a pre-signed URL, signed for GET, and only read. 2. Expiry is communicated out of band; the client does not track it and uses a URL until it fails. 3. The URL replaces the location and flows through every path that treats `file_path` as a plain location, including the `files` metadata table and engine logs. I think that is acceptable for a first step; carrying the URL beside a native location would be a spec change to the task. Out of scope: the client asking the server to pre-sign a location. That is a separate spec change and best addressed as a follow up change/implementation to this PR. The change was initially drafted with Claude. The design, scope and review are mine. In addition to the tests included in this PR, the read path was verified against S3 and GCS objects that were signed out of band. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
