dwsmith1983 commented on PR #5872: URL: https://github.com/apache/datafusion-comet/pull/5872#issuecomment-5683995971
> Please preserve virtual hosting for this HTTP case and cover it through `extract_s3_config_options`, including the resulting URL. Done. The dotted-bucket rule now runs in `extract_s3_config_options` only when no custom endpoint is configured, since the default AWS endpoint is HTTPS; a custom endpoint decides by its own scheme inside `normalize_endpoint`. The test now goes through `extract_s3_config_options` and asserts `http://storage.example.test` with `review.dotted.bucket` yields virtual hosting and the endpoint `http://review.dotted.bucket.storage.example.test`, next to the HTTPS and default-endpoint cases. > Could we pass Hadoop's resolved default file into this branch instead of deriving it from the native process's `HOME`? Done. `NativeConfig.extractObjectStoreOptions` now forwards `fs.s3a.comet.default.profile.file`, resolved on the JVM as `AWS_SHARED_CREDENTIALS_FILE` or the `user.home` `.aws/credentials`, for every scheme on the `fs.s3a.*` surface. Native's Hadoop-provider arm takes a configured `fs.s3a.auth.profile.file` first, then that forwarded path; the native `HOME` fallback only remains for callers that forward nothing. Tests on both sides: the Scala one checks the forwarded value and that a blank env override falls back to `user.home` rather than `HOME`, and the native one checks the forwarded file is taken, a configured file wins, and the SDK spellings ignore both. Merged with `main` at a8e8157ea first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
