[
https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389243#comment-17389243
]
Yikun Jiang edited comment on SPARK-36000 at 7/29/21, 4:15 AM:
---------------------------------------------------------------
[~XinrongM] I did some investigation on this, I found the problem is triggered
in the python to Java unpickling step, because Decimal('NaN') is not supported
by net.razorvine.pickle, see more in
https://issues.apache.org/jira/browse/SPARK-36337
was (Author: yikunkero):
[~XinrongM] I did some investigation on this, I found the problem is triggered
in python to Java unpickling, because Decimal('NaN') is not supported by
net.razorvine.pickle, see more in
https://issues.apache.org/jira/browse/SPARK-36337
> Support creation and operations of ps.Series/Index with Decimal('NaN')
> ----------------------------------------------------------------------
>
> Key: SPARK-36000
> URL: https://issues.apache.org/jira/browse/SPARK-36000
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 3.2.0
> Reporter: Xinrong Meng
> Priority: Major
>
> The creation and operations of ps.Series/Index with Decimal('NaN') doesn't
> work as expected.
> That might be due to the underlying PySpark limit.
> Please refer to sub-tasks for issues detected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
