[
https://issues.apache.org/jira/browse/SPARK-41718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652009#comment-17652009
]
Phillip Cloud commented on SPARK-41718:
---------------------------------------
If 1.24 can't be supported in 3.4.0, it'd be nice if an upper bound could be
placed on the numpy dependency since 1.24 is a hard break for pyspark.
> Numpy 1.24 breaks PySpark due to use of `np.bool` instead of `np.bool_` in
> many places
> --------------------------------------------------------------------------------------
>
> Key: SPARK-41718
> URL: https://issues.apache.org/jira/browse/SPARK-41718
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Phillip Cloud
> Priority: Major
>
> In numpy 1.24, `numpy.bool` was removed (it was deprecated prior to 1.24).
> This causes many APIs in pyspark to stop working because an AttributeError is
> raised. The alternative is to use `numpy.bool_` (trailing underscore).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
