Sean is right, casting timestamps to strings (which is what show() does)
uses the local timezone, either the Java default zone `user.timezone`,
the Spark default zone `spark.sql.session.timeZone` or the default
DataFrameWriter zone `timeZone`(when writing to file).
You say you are in PST,
You sure it is not just that it's displaying in your local TZ? Check the
actual value as a long for example. That is likely the same time.
On Thu, Jun 8, 2023, 5:50 PM karan alang wrote:
> ref :
>
ref :
https://stackoverflow.com/questions/76436159/apache-spark-not-reading-utc-timestamp-from-mongodb-correctly
Hello All,
I've data stored in MongoDB collection and the timestamp column is not
being read by Apache Spark correctly. I'm running Apache Spark on GCP
Dataproc.
Here is sample data :
