ueshin commented on a change in pull request #32847:
URL: https://github.com/apache/spark/pull/32847#discussion_r650219350
##########
File path: python/pyspark/pandas/data_type_ops/binary_ops.py
##########
@@ -53,3 +57,34 @@ def radd(self, left, right) -> Union["Series", "Index"]:
raise TypeError(
"Concatenation can not be applied to %s and the given type." %
self.pretty_name
)
+
+ def astype(
+ self, index_ops: Union["Index", "Series"], dtype: Union[str, type,
Dtype]
+ ) -> Union["Index", "Series"]:
+ dtype, spark_type = pandas_on_spark_type(dtype)
+ if not spark_type:
+ raise ValueError("Type {} not understood".format(dtype))
Review comment:
Does this happen?
##########
File path: python/pyspark/pandas/data_type_ops/binary_ops.py
##########
@@ -53,3 +57,34 @@ def radd(self, left, right) -> Union["Series", "Index"]:
raise TypeError(
"Concatenation can not be applied to %s and the given type." %
self.pretty_name
)
+
+ def astype(
+ self, index_ops: Union["Index", "Series"], dtype: Union[str, type,
Dtype]
+ ) -> Union["Index", "Series"]:
+ dtype, spark_type = pandas_on_spark_type(dtype)
+ if not spark_type:
+ raise ValueError("Type {} not understood".format(dtype))
Review comment:
Does this happen? I guess the exception is thrown in
`pandas_on_spark_type`.
##########
File path: python/pyspark/pandas/tests/data_type_ops/test_binary_ops.py
##########
@@ -147,6 +147,14 @@ def test_from_to_pandas(self):
self.assert_eq(pser, psser.to_pandas())
self.assert_eq(ps.from_pandas(pser), psser)
+ def test_astype(self):
+ pser = self.pser
+ psser = self.psser
+ self.assert_eq(pd.Series(["1", "2", "3"]), psser.astype(str))
+ self.assert_eq(pser.astype("category"), psser.astype("category"))
+ cat_type = pd.api.types.CategoricalDtype(categories=[b"2", b"3", b"1"])
Review comment:
nit: Shall we import `CategoricalDtype` at the header? And ditto in the
other files.
##########
File path: python/pyspark/pandas/data_type_ops/base.py
##########
@@ -39,9 +41,8 @@
TimestampType,
UserDefinedType,
)
-
import pyspark.sql.types as types
Review comment:
I'm wondering why we import some types from `pyspark.sql.types` but also
import `types` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
