andygrove opened a new pull request, #4478: URL: https://github.com/apache/datafusion-comet/pull/4478
## Which issue does this PR close? Closes #. ## Rationale for this change Continuation of the per-category expression audit. Same pattern as #4476 (hash), #4475 (conditional), #4474 (misc), #4473 (collection), #4470 (json), #4469 (struct), using the updated `audit-comet-expression` skill in #4468. ## What changes are included in this PR? ### Support-doc audit notes Add per-version audit sub-bullets to `element_at`, `map_contains_key`, `map_entries`, `map_from_arrays`, `map_from_entries`, `map_keys`, `map_values`, and `str_to_map`. Most of the category is byte-for-byte identical across the four versions. Spark 4.0 adds the usual collation widening (`StringTypeNonCSAICollation` on `StringToMap`) and the `nullIntolerant: Boolean` field refactor. Spark 4.1 adds a `legacySplitTruncate` flag to `StringToMap`. ### Support-level consistency fixes (in `maps.scala`) - `CometMapFromEntries`: rephrase the BinaryType reasons to read consistently (`` `BinaryType` is not supported as a map key/value in `map_from_entries` ``) and backtick the type name and function name. - `CometStrToMap`: drop the no-op `getSupportLevel` override (it returned `Compatible(None)`, which is also the default). Update the affected test references in `CometMapExpressionSuite` (via the shared constants) and the `expressions/map/map_from_entries.sql` fallback substring matchers. ### Tracking issues filed for follow-up - #4477 `str_to_map` does not honour Spark 4.1.1 `legacy.truncateForEmptyRegexSplit` flag. The previously closed #3327 (`map_from_arrays` null-input crash) is referenced from the `map_from_arrays` sub-bullet. ### Audit process Audited directly using the `audit-comet-expression` skill (4 Spark versions per #4468). Eight serdes plus existing test coverage. ## How are these changes tested? - `./mvnw test -Dsuites="org.apache.comet.CometSqlFileTestSuite expressions/map/" -Dtest=none` (9 tests pass after updating the substring matchers) - `./mvnw test -Dsuites="org.apache.comet.CometMapExpressionSuite" -Dtest=none` (8 succeeded, 1 already-`ignore`d) - `make core` succeeds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
