nchammas commented on code in PR #45036: URL: https://github.com/apache/spark/pull/45036#discussion_r1491158277
########## core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala: ########## @@ -110,6 +110,18 @@ class OpenHashSet[@specialized(Long, Int, Double, Float) T: ClassTag]( this } + /** + * Check if a key exists at the provided position using object equality rather than + * cooperative equality. Otherwise, hash sets will mishandle values for which `==` + * and `equals` return different results, like 0.0/-0.0 and NaN/NaN. Review Comment: Yes, the differences are subtle: ```scala scala> 0.0 == -0.0 val res0: Boolean = true scala> 0.0 equals -0.0 val res1: Boolean = false scala> Double.NaN == Double.NaN val res2: Boolean = false scala> Double.NaN equals Double.NaN val res3: Boolean = true ``` There is a long discussion on the Scala forums from 2017 about this difference and some of the problems it causes: [Can we get rid of cooperative equality?](https://contributors.scala-lang.org/t/can-we-get-rid-of-cooperative-equality/1131) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org