viirya commented on PR #6130: URL: https://github.com/apache/datafusion-comet/pull/6130#issuecomment-6011005089
I also addressed the exchange-identity issue. Equality and hashing no longer compare the full Arrow UDF protobuf, which contains generated argument expression IDs. They compare the Catalyst UDF expressions that Spark canonicalizes, while retaining the batch limits in plan identity. The new regression checks `sameResult` for equivalent UDF plans with different generated IDs, and checks that different functions and batch limits remain distinct. These changes are in `b0dab2f53`. Local Spark 4.1 and 4.2 test compilation, the plan-equality suite, and Python syntax checks passed. The exact-head CI run passed: 34 successful checks and 14 skipped, including the Spark 4.1 and 4.2 native PyArrow UDF suites. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
