Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19607 I have no idea about the reason but seems like there is a difference of handling DST in `spark.createDataFrame(data, schema=schema)` between Jenkins and my local environments. The debug print by `df.show()` from the test [tests.py#L3487](https://github.com/ueshin/apache-spark/blob/9101a3a12f17b5bd633756139eaa2cb3ee9bb33c/python/pyspark/sql/tests.py#L3487) was: ``` +---+-------------------+ |idx| timestamp| +---+-------------------+ | 0|1969-01-01 01:01:01| | 1|2012-02-02 02:02:02| | 2| null| | 3|2100-04-04 04:04:04| +---+-------------------+ ``` but in my local: ``` +---+-------------------+ |idx| timestamp| +---+-------------------+ | 0|1969-01-01 01:01:01| | 1|2012-02-02 02:02:02| | 2| null| | 3|2100-04-04 05:04:04| +---+-------------------+ ``` Could you please let me know if I miss something or you have any ideas?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org