srielau commented on PR #59316: URL: https://github.com/apache/spark/pull/59316#issuecomment-6093099179
Follow-up for the SQL review of this PR: 1. Map-key length checks now live next to `applyTextParseSemantics` as `CharVarcharUtils.applyXmlMapKeySemantics`. They are not codegen `StaticInvoke` targets, so they are no longer on `CharVarcharCodegenUtils`. The error class stays `UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY` until a shared JSON/XML helper can land with SPARK-60108; this PR does not take a dependency on that change. 2. `isLengthCheckedKeyType` is inlined into that helper. `convertXmlMapKey` is a one-line call. 3. I did not rebase/squash the earlier assign-after-parse commit (already pushed). Happy to rebase onto current `master` if a reviewer wants that; `git diff master...HEAD` for `SQLConf.scala` is only the `MAP_KEY_DEDUP_POLICY` sentence. 4. Tests now cover PERMISSIVE key failure keeping `tail`, nested `MAP<STRING, MAP<CHAR(2), INT>>` too-short keys, default `valueTag` `_VALUE` vs `CHAR(2)`, and `ignoreCorruptFiles` not skipping the rest of the file. Unwrapped datasource key errors wrap as `BadRecordException` instead of matching the corrupt-file skip path. 5. The collated-keys assertion uses `Map` equality rather than `map_entries` order. 6. `MAP_KEY_DEDUP_POLICY` docs name both `from_xml` and the XML datasource. 7. `unsupportedXmlCharVarcharMapKey` now has scaladoc explaining why 0A000 is a `SparkRuntimeException`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
