[
https://issues.apache.org/jira/browse/SPARK-38822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-38822:
--------------------------------
Description:
[https://github.com/apache/spark/blob/becda3339381b3975ed567c156260eda036d7a1b/python/pyspark/pandas/tests/indexes/test_base.py#L2179]
we need to raise indexError when out of bounds for axis, and also change test
case
- Related changes:
- panda 1.4.0+ is using numpy insert:
https://github.com/pandas-dev/pandas/commit/c021d33ecf0e096a186edb731964767e9288a875
- Since numpy 1.8 (10 years ago
https://github.com/numpy/numpy/commit/908e06c3c465434023649b0ca522836580c5cfdc)
: [`out-of-bound indices will generate an
error.`](https://numpy.org/devdocs/release/1.8.0-notes.html#changes)
was:
[https://github.com/apache/spark/blob/becda3339381b3975ed567c156260eda036d7a1b/python/pyspark/pandas/tests/indexes/test_base.py#L2179]
we need to raise indexError when out of bounds for axis, and also change test
case
related:
- numpy 1.20+ fix a issue:
[https://github.com/numpy/numpy/commit/be11c06de7cf128c364c33dafbdecd2372f322e6]
- panda 1.4.x using numpy insert:
https://github.com/pandas-dev/pandas/commit/c021d33ecf0e096a186edb731964767e9288a875
> Raise indexError when insert loc is out of bounds
> -------------------------------------------------
>
> Key: SPARK-38822
> URL: https://issues.apache.org/jira/browse/SPARK-38822
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.4.0
> Reporter: Yikun Jiang
> Priority: Major
>
>
> [https://github.com/apache/spark/blob/becda3339381b3975ed567c156260eda036d7a1b/python/pyspark/pandas/tests/indexes/test_base.py#L2179]
>
> we need to raise indexError when out of bounds for axis, and also change test
> case
>
> - Related changes:
> - panda 1.4.0+ is using numpy insert:
> https://github.com/pandas-dev/pandas/commit/c021d33ecf0e096a186edb731964767e9288a875
> - Since numpy 1.8 (10 years ago
> https://github.com/numpy/numpy/commit/908e06c3c465434023649b0ca522836580c5cfdc)
> : [`out-of-bound indices will generate an
> error.`](https://numpy.org/devdocs/release/1.8.0-notes.html#changes)
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
