akshaytayal commented on issue #13181: URL: https://github.com/apache/gluten/issues/13181#issuecomment-6139770501
@rdtr , Thanks for coordinating! No overlap: #13176, #13178, #13179 and #13180 are all yours. I've only worked on the other three: - #13174: #13243 (SPJ) and #13246 (SPJ sorted merge) - #13175: #13244 (DELETE), #13247 (MERGE) and #13248 (NOT NULL) - #13177: #13245 (golden files) Plus #13164 (enhanced DSv2 filter suites). They're all test-only with green CI, waiting for review. I also checked them against your open PRs and none of them conflict. > [@akshaytayal](https://github.com/akshaytayal) Thanks for setting up the Spark 4.2 UT module and splitting the disabled tests into these issues! To avoid duplicate work, here is what I'm working on: > > * [[Spark 4.2 UT] Floating-point collection semantics: -0.0/NaN normalization & RESPECT NULLS #13176](https://github.com/apache/gluten/issues/13176): the -0.0/NaN normalization of array set functions and `collect_set` ([fix(sparksql): Normalize -0.0 and NaN in array set function results facebookincubator/velox#19358](https://github.com/facebookincubator/velox/pull/19358), [fix(sparksql): Normalize -0.0 and NaN in array_union results facebookincubator/velox#19359](https://github.com/facebookincubator/velox/pull/19359), [fix(sparksql): Normalize -0.0 and NaN in collect_set results facebookincubator/velox#19361](https://github.com/facebookincubator/velox/pull/19361)) and `collect_set` RESPECT NULLS ([[GLUTEN-13176][VL] Pass RESPECT NULLS of collect_set to Velox #13258](https://github.com/apache/gluten/pull/13258), with [fix(exec): Forward constant inputs from companion aggregates facebookincubator/velox#19464](https://github.com/facebookincubator/velox/pull/19464)). > * [[Spark 4.2 UT] Velox expression & operator gaps (math/TIME/date/broadcast/hash-agg) #13178](https://github.com/apache/gluten/issues/13178): asinh/acosh, the broadcast size test and the sort-aggregate test ([[GLUTEN-13178][VL] Re-enable the Spark 4.2 acosh, broadcast size and sort-aggregate tests #13259](https://github.com/apache/gluten/pull/13259)). "TIME type with arrays" fails in Spark's conversion of arrays of TIME, so I left it disabled. > * [[Spark 4.2 UT] Spark 4.2 framework/API changes breaking Gluten tests (non-velox) #13179](https://github.com/apache/gluten/issues/13179): `MiscOperatorSuite` and `GlutenBloomFilterFallbackSuite` ([[GLUTEN-13179][VL] Register test functions through SessionCatalog for Spark 4.2 #13195](https://github.com/apache/gluten/pull/13195)), `GlutenWholeStageCodegenSuite` ([[GLUTEN-13179][CORE] Build balanced And trees for scan and filter conditions #13205](https://github.com/apache/gluten/pull/13205)), the null-named attribute test ([[GLUTEN-13179][CORE] Fix NPE when a column has a null name #13206](https://github.com/apache/gluten/pull/13206), re-enabled in [[GLUTEN-13179][VL] Re-enable the Spark 4.2 null-named attribute test #13260](https://github.com/apache/gluten/pull/13260)), and the session extension and DEFAULT length tests ([[GLUTEN-13179][VL] Fix the Spark 4.2 session extension and DEFAULT length tests #13262](https://github.com/apache/gluten/pull/13262)). > * [[Spark 4.2 UT] File-source & Hive scan (Parquet/ORC ignoreMissingFiles/NullType, bucketed-scan AE, ADD JAR) #13180](https://github.com/apache/gluten/issues/13180): bucketed scan and ADD JAR ([[GLUTEN-13180][VL] Re-enable the Spark 4.2 bucketed scan and ADD JAR tests #13261](https://github.com/apache/gluten/pull/13261)), `ignoreMissingFiles` ([[GLUTEN-13180][CORE] Fall back the scan when the ignoreMissingFiles read option differs from the conf #13207](https://github.com/apache/gluten/pull/13207)) and NullType ([fix(parquet): Read a column with the UNKNOWN annotation as any requested type facebookincubator/velox#19461](https://github.com/facebookincubator/velox/pull/19461)). > > If you have already started on any of these, please let me know and I'll leave them to you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
