DerGut opened a new pull request, #1368: URL: https://github.com/apache/iceberg-rust/pull/1368
## Which issue does this PR close? - Closes #1360. ## What changes are included in this PR? This PR adds an integration for the Azure Datalake storage service. It mainly adds parsing logic for configuration properties. The finished config struct is simply passed down to OpenDAL. It also creates a new `Storage::Azdls` enum variant based on OpenDAL's existing `Scheme::Azdls` enum variant. It then fits the parsing logic into the existing framework to build the storage integration from an `io::FileIOBuilder`. --- Note, OpenDAL has only [recently added](https://github.com/apache/opendal/pull/6205) configuration fields for more secure authentication properties. This PR is built on an unreleased version of OpenDAL. In addition, I have another [OpenDAL PR](https://github.com/apache/opendal/pull/6212) pending to add ADLS connection string support. I suggest to wait for at least the first PR to be released. In its current form, OpenDAL only supports basic auth via directly passed credentials. But I'm also happy to adapt the changes to support what is already out. For the time being, I'll keep the PR in draft until we either have a new OpenDAL release, or we take the decision to release what is already out. ## Are these changes tested? I added minor unit tests to validate the configuration property parsing logic. I decided not to add integration tests because 1. ADLS is not S3-compatible which means that we can't reuse our Minio setup 2. the Azure-specific alternative to local testing - Azurite - doesn't support ADLS I have yet to test it in a functioning environment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
