peterxcli commented on PR #5407: URL: https://github.com/apache/datafusion-comet/pull/5407#issuecomment-5455039017
Both confirmed @sunchao — addressed in 4925cc296. **Fold ambiguity:** `check_column_unicode_fold` now computes the full Unicode-fold match set the way Spark's resolver groups names. More than one match raises Spark's own duplicate-field error (`DuplicateFieldCaseInsensitive`) instead of serving the ASCII match; exactly one match invisible to the ASCII matcher keeps the fail-closed #5495 error; one ASCII-visible match stays the ordinary path. An e2e regression writes ASCII `K` + Kelvin `K` VARIANT columns and asserts both engines reject `SELECT k`. **ID shadowing:** the remap now collects logical names that resolve by field ID and hides any non-ID-matched physical field carrying such a name behind a fake name (the same scheme as the unmatched-requested-ID guard, applied to the mirror hazard), so the ID-matched column is the only name match the adapter can find. Exact collisions in case-sensitive mode, ASCII-insensitive otherwise — a genuinely distinct case-sensitive `V` sibling is unaffected. Covered by an adapter-level Rust test of your exact layout (`v BINARY`/ID 2 before `other VARIANT`/ID 1) and an e2e that reads `42` natively, matching Spark. Validated on Spark 4.0 and 4.1 (all 8 Variant reader regressions plus the pre-existing no-ID duplicate test). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
