[ https://issues.apache.org/jira/browse/SPARK-43194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-43194: --------------------------------- Parent: SPARK-42618 Issue Type: Sub-task (was: Bug) > PySpark 3.4.0 cannot convert timestamp-typed objects to pandas with pandas 2.0 > ------------------------------------------------------------------------------ > > Key: SPARK-43194 > URL: https://issues.apache.org/jira/browse/SPARK-43194 > Project: Spark > Issue Type: Sub-task > Components: PySpark > Affects Versions: 3.4.0 > Environment: {code} > In [4]: import pandas as pd > In [5]: pd.__version__ > Out[5]: '2.0.0' > In [6]: import pyspark as ps > In [7]: ps.__version__ > Out[7]: '3.4.0' > {code} > Reporter: Phillip Cloud > Priority: Major > > {code} > In [1]: from pyspark.sql import SparkSession > In [2]: session = SparkSession.builder.appName("test").getOrCreate() > 23/04/19 09:21:42 WARN Utils: Your hostname, albatross resolves to a loopback > address: 127.0.0.2; using 192.168.1.170 instead (on interface enp5s0) > 23/04/19 09:21:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to > another address > Setting default log level to "WARN". > To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use > setLogLevel(newLevel). > 23/04/19 09:21:42 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > In [3]: session.sql("select now()").toPandas() > {code} > Results in: > {code} > ... > TypeError: Casting to unit-less dtype 'datetime64' is not supported. Pass > e.g. 'datetime64[ns]' instead. > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org