ulysses-you commented on PR #60078: URL: https://github.com/apache/spark/pull/60078#issuecomment-6626968350
Updated the branch (95b3517f907): - Dropped the `require` from `InternalRowComparableWrapper`. It was a new behavior change for a connector whose `partitionKey()` row is wider than its key expressions, and it ran once per record in the shuffle key extractor; the narrower-row case already fails in the sort before the wrapper. - Key values now render through `Literal.create(value, dataType).toString`, so DATE, TIMESTAMP, TIME, DECIMAL and BINARY keys print as values and `null` prints as `null`. - `KeyedPartitioning` now overrides `toString`/`simpleString` instead of `stringArgs`: the key list takes the fields left by the expressions and the trailing flags, and both levels go through `truncatedString`, so at most one `...` appears. This replaces the hand-rolled slot arithmetic and the double-ellipsis corner. - @pan3793 the key list now uses `truncatedString(partitionKeys, ...)`, rendering `[[1], [2], ..., [7], ... 5 more fields]` with the same format as the rest of the argument list. The last-key boundary is not shown. Tests cover the scalar key types (DATE, TIMESTAMP, TIMESTAMP_NTZ, TIME, DECIMAL, BINARY, CHAR, VARCHAR, UTF8_LCASE), null, `UnsafeRow`, the budget boundaries, an empty key list, and a deserialized partitioning: catalyst 30 passed, the SPJ/DSv2 suites 504 passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
