Li Jin created SPARK-28003: ------------------------------ Summary: spark.createDataFrame with Arrow doesn't work with pandas.NaT Key: SPARK-28003 URL: https://issues.apache.org/jira/browse/SPARK-28003 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 2.4.0 Reporter: Li Jin
{code:java} import pandas as pd dt1 = [pd.NaT, pd.Timestamp('2019-06-11')] * 100 pdf1 = pd.DataFrame({'time': dt1}) df1 = self.spark.createDataFrame(pdf1) {code} The example above doesn't work with arrow enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org