This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 52d52a158c9 [SPARK-40376][PYTHON] Avoid Numpy deprecation warning
52d52a158c9 is described below
commit 52d52a158c9a169d97122523142b7c3026ee92bb
Author: ELHoussineT <elhoussineta...@gmail.com>
AuthorDate: Mon Sep 12 20:46:15 2022 -0500
[SPARK-40376][PYTHON] Avoid Numpy deprecation warning
### What changes were proposed in this pull request?
Use `bool` instead of `np.bool` as `np.bool` will be deprecated (see:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations)
Using `np.bool` generates this warning:
```
UserWarning: toPandas attempted Arrow optimization because
'spark.sql.execution.arrow.pyspark.enabled' is set to true, but has reached the
error below and can not continue. Note that
'spark.sql.execution.arrow.pyspark.fallback.enabled' does not have an effect on
failures in the middle of computation.
3070E `np.bool` is a deprecated alias for the builtin
`bool`. To silence this warning, use `bool` by itself. Doing this will not
modify any behavior and is safe. If you specifically wanted the numpy scalar
type, use `np.bool_` here.
3071E Deprecated in NumPy 1.20; for more details and
guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
```
### Why are the changes needed?
Deprecation soon:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations.
### Does this PR introduce _any_ user-facing change?
The warning will be suppressed
### How was this patch tested?
Existing tests should suffice.
Closes #37817 from ELHoussineT/patch-1.
Authored-by: ELHoussineT <elhoussineta...@gmail.com>
Signed-off-by: Sean Owen <sro...@gmail.com>
---
python/pyspark/sql/pandas/conversion.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/pyspark/sql/pandas/conversion.py
b/python/pyspark/sql/pandas/conversion.py
index fff0bac5480..22717241fde 100644
--- a/python/pyspark/sql/pandas/conversion.py
+++ b/python/pyspark/sql/pandas/conversion.py
@@ -295,7 +295,7 @@ class PandasConversionMixin:
elif type(dt) == DoubleType:
return np.float64
elif type(dt) == BooleanType:
- return np.bool # type: ignore[attr-defined]
+ return bool
elif type(dt) == TimestampType:
return np.datetime64
elif type(dt) == TimestampNTZType:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
