Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18933 Ping. I ran into this exact issue with pandas_udf on a simple data set with a timestamp type column. As far as I can tell, there is no way to around this since pandas code is running deep inside pyspark and the only workaround is to make the column a string? @BryanCutler @ueshin @icexelloss @HyukjinKwon any thought on how to fix this?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org