Visorgood commented on PR #6110: URL: https://github.com/apache/datafusion-comet/pull/6110#issuecomment-5818988294
Both docs updated. `jvm_shuffle.md`: added a fourth case to "When JVM Shuffle is Used" – native shuffle serializes the partitioning expressions, so a key it has no serde for keeps the exchange off the native path while JVM shuffle takes it, with the Spark 4.0+ mapsort wrapper over an array or struct map key as the example. Followed by a note that a collated hash or range key is declined by both Comet paths and stays a plain Spark Exchange, which is what the new collation test pins. `understanding-comet-plans.md`: collated strings were given as the example for CometColumnarExchange, which is wrong – they go to Spark's shuffle. Replaced with the mapsort key, and said explicitly that a collated key is not one of these cases. Thanks for adding run-spark-4.1-tests; this push should pick the suites up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
