zhangfengcdt opened a new pull request, #5935: URL: https://github.com/apache/datafusion-comet/pull/5935
## Which issue does this PR close? Closes #5637. ## Rationale for this change The native Iceberg write path accepts `gs://` data locations, but the only bridge from the Hadoop `Configuration` into the native `FileIO` translates `fs.s3a.*` keys. A `HadoopFileIO` takes its GCS credentials, endpoint and project from `fs.gs.*`, so none of that reached the native writer and it could resolve a different storage identity or endpoint than the JVM writer would. Following the issue's recommendation, this change fails closed for that combination rather than attempting an `fs.gs.*` to `gcs.*` bridge, which is not a simple key rename. ## What changes are included in this PR? 1. `CometIcebergNativeWrite`: a new trigger rule that declines a `gs://` data location unless the `FileIO` opening it is a `GCSFileIO`. The decision is a small package-visible function so it can be unit-tested directly. 2. `IcebergReflection`: a helper that returns the effective `FileIO` class for a location. A `ResolvingFileIO` is asked for its delegate via `ioClass(location)`, so a table that resolves to `GCSFileIO` stays eligible. Reflection failures fail closed. 3. `iceberg-writes.md`: documents the new condition. ## How are these changes tested? New and updated tests in `CometIcebergWriteDetectionSuite` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
