loudongfeng commented on PR #5311: URL: https://github.com/apache/incubator-gluten/pull/5311#issuecomment-2058483332
Find that spark also return Infinity in some cases, so convert Infinity to null does not work. spark-sql --master local[4] --conf spark.sql.shuffle.partitions=3 ```sql drop table if exists example_table1; CREATE TABLE example_table1(id int, value double) USING parquet; INSERT INTO example_table1 VALUES (1, 4.9E-322), (2, 1.7976931348623157E308), (3, 1.7976931348623157E308); SELECT var_samp(value) FROM example_table1; ``` the result will be Infinity., while return NaN for the following: spark-sql --master local[2] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
