ErikBPF commented on PR #5786: URL: https://github.com/apache/datafusion-comet/pull/5786#issuecomment-5788698738
Thanks for the detailed review. I pushed the follow-up in `2d082c054` and rewrote the PR description to match the current design. - The Spark-only inference assertion now reads the duplicate-bearing file alone; the two-file explicit-schema controls remain. The affected case passed three times, including the full `CometNativeReaderSuite` run (85 succeeded, 0 failed, 1 existing cancellation). - A native regression exercises both Variant adapter paths with duplicate physical children: a direct `Column` and a default-adapter `CastExpr`. This proves the paths at the adapter boundary, but does not establish that a real Parquet Variant file produces that layout. I kept the guards because either path can decode the full physical subtree. - The seven guarded expression outcomes now pass through `checked_decoded_expr` before decoding. The duplicate-field message has one constructor, and `id_duplicate_roots` now documents its key and precedence. - `create` reuses the physical-name fold when both schemas are the same `Arc` and allocates index vectors only for actual folded-name collisions. I have not claimed a measured speedup. - The scan docs now describe Spark's variable sibling resolution and link #5884 and #5964. Local native Parquet tests passed (202, with 5 ignored), as did the full Spark reader suite and formatting checks. New-head CI is pending. I could not add the test labels: GitHub denied `AddLabelsToLabelable` for my account. Could a maintainer add `run-spark-4.1-tests`, `run-iceberg-tests`, and `run-benchmark-check` so the Spark SQL, Iceberg, and all-target Clippy checks run before merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
