Smallfu666 opened a new pull request, #12661: URL: https://github.com/apache/gluten/pull/12661
### What changes Spark represents `array_sort(array)` using a generated null-aware default comparator. Gluten currently passes this comparator to Velox as a lambda, but Velox cannot rewrite it and falls back to Spark execution. This change detects Spark's default comparator using the bound lambda arguments and omits the lambda only for an exact semantic match (`semanticEquals`), allowing Velox to use its native one-argument `array_sort`. Custom comparators remain unchanged and continue using the two-argument path. This revives and revalidates the approach from #12095, which was auto-closed as stale. Fixes #12094 ### Tests - Spark 4.0.2 targeted native test passed (`MiscOperatorSuite - "array_sort with default comparator"`; full suite blocked by unrelated Velox HashTable SIGFPE) - Spark 4.1.1 targeted native test passed (`MiscOperatorSuite - "array_sort with default comparator"`, 1/1 PASS, BUILD SUCCESS) - Spark 3.5 clean compile and test-compile passed (`./build/mvn -ntp clean test-compile -Pspark-3.5 -Pscala-2.12 -Pjava-17 -Pbackends-velox -Pspark-ut`) - Test coverage: integer arrays, null elements, empty array, null array, double arrays with NaN, descending custom-comparator negative control - Spark oracle comparison and `ProjectExecTransformer` plan assertion verified ### AI Tooling Disclosure Was this patch authored or co-authored using generative AI tooling? Generated-by: Gemini 3.6 Flash -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
