coderfender commented on code in PR #3541:
URL: https://github.com/apache/datafusion-comet/pull/3541#discussion_r2820104117
##########
spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala:
##########
@@ -332,6 +332,38 @@ abstract class CometTestBase
}
}
+// inspired from spark-testing-base
+ protected def assertDataFrameEquals(
Review Comment:
Thank you for the suggestion . The issue is rather on the `castTest`
framework in which we have a `collect()` statement. This collect would convert
the spark's timestamp (microseconds) to java's timestamp (which is basically a
date) causing error / overflow. In order to avoid that I have had to implement
`assertDataFrameEquals` (inspired from spark-testing-base) to make sure we have
data and schema parity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
