[
https://issues.apache.org/jira/browse/SPARK-52332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinrong Meng updated SPARK-52332:
---------------------------------
Description:
{{>>> ps.set_option("compute.fail_on_ansi_mode", False)}}
{{>>> spark.conf.set("spark.sql.ansi.enabled", False)}}
{{>>> }}
{{>>> import pandas as pd}}
{{>>> import numpy as np}}
{{>>> pdf = pd.DataFrame(}}
{{... {}}
{{... "a": [1.0, -1.0, 0.0, np.nan],}}
{{... "b": [0.0, 0.0, 0.0, 0.0],}}
{{... },}}
{{... dtype=np.float32,}}
{{... )}}
{{>>> }}
{{>>> psdf = ps.from_pandas(pdf)}}
{{>>> }}
{{>>> psdf["a"] / psdf["b"]}}
{{0 inf
}}
{{1 -inf}}
{{2 NaN}}
{{3 NaN}}
{{dtype: float64}}
{{>>> }}
{{>>> pdf["a"] / pdf["b"]}}
{{0 inf}}
{{1 -inf}}
{{2 NaN}}
{{3 NaN}}
{{dtype: float32}}
> Fix promotion from float32 to float64 during division
> -----------------------------------------------------
>
> Key: SPARK-52332
> URL: https://issues.apache.org/jira/browse/SPARK-52332
> Project: Spark
> Issue Type: Sub-task
> Components: PS
> Affects Versions: 4.1.0
> Reporter: Xinrong Meng
> Priority: Major
>
> {{>>> ps.set_option("compute.fail_on_ansi_mode", False)}}
> {{>>> spark.conf.set("spark.sql.ansi.enabled", False)}}
> {{>>> }}
> {{>>> import pandas as pd}}
> {{>>> import numpy as np}}
> {{>>> pdf = pd.DataFrame(}}
> {{... {}}
> {{... "a": [1.0, -1.0, 0.0, np.nan],}}
> {{... "b": [0.0, 0.0, 0.0, 0.0],}}
> {{... },}}
> {{... dtype=np.float32,}}
> {{... )}}
> {{>>> }}
> {{>>> psdf = ps.from_pandas(pdf)}}
> {{>>> }}
> {{>>> psdf["a"] / psdf["b"]}}
> {{0 inf
> }}
> {{1 -inf}}
> {{2 NaN}}
> {{3 NaN}}
> {{dtype: float64}}
> {{>>> }}
> {{>>> pdf["a"] / pdf["b"]}}
> {{0 inf}}
> {{1 -inf}}
> {{2 NaN}}
> {{3 NaN}}
> {{dtype: float32}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
