[
https://issues.apache.org/jira/browse/SPARK-55242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18071017#comment-18071017
]
Mohd Azmat commented on SPARK-55242:
------------------------------------
Hi Tian Gao,
I've submitted a fix for this issue.
substitution, so `ps.from_pandas()` works correctly.
PR: https://github.com/apache/spark/pull/55196
A regression test (`test_from_pandas_with_np_array_elements`) has been
added in `pyspark/pandas/tests/data_type_ops/test_complex_ops.py`.
Would appreciate your review. Thanks!
> pyspark.pandas create np.array from list in pandas
> --------------------------------------------------
>
> Key: SPARK-55242
> URL: https://issues.apache.org/jira/browse/SPARK-55242
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 4.2.0
> Reporter: Tian Gao
> Priority: Major
> Labels: pull-request-available
>
> {code:java}
> # Different element types
> pdf = pd.DataFrame(
> {"a": [1, 2, 3, 4, 5, 6, 7, 8, 9], "b": [[e] for e in [4, 5, 6, 3, 2, 1,
> 0, 0, 0]]},
> index=np.random.rand(9),
> )
> psdf = ps.from_pandas(pdf) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
