[
https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaya Krishna updated SPARK-26054:
---------------------------------
Description: When a computed column that rounds the value is added to a
data frame, it is affecting the value of the original column as well. The
behavior depends on the database column type - If it is either float or double,
the result is as expected - the original column will have its own formatting
and the computed column will be rounded as per the rounding definition
specified for it. However if the column type in the database is decimal, then
Spark SQL is applying the rounding even to the original column. Attached image
has the spark sql code that shows the issue. (was: When a computed column that
rounds the value is added to a dataframe, it is affecting the value of the
original column as well. The behavior depends on the database column type - If
it is either float or double, the result is as expected - the original column
will have its own formatting and the computed column will be rounded as per the
rounding definition specified for it. However if the column type in the
database is decimal, then Spark SQL is applying the rounding even to the
original column.)
> Creating a computed column applying the spark sql rounding on a column of
> type decimal affects the orginal column as well.
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-26054
> URL: https://issues.apache.org/jira/browse/SPARK-26054
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: Jaya Krishna
> Priority: Minor
> Attachments: sparksql-rounding.png
>
>
> When a computed column that rounds the value is added to a data frame, it is
> affecting the value of the original column as well. The behavior depends on
> the database column type - If it is either float or double, the result is as
> expected - the original column will have its own formatting and the computed
> column will be rounded as per the rounding definition specified for it.
> However if the column type in the database is decimal, then Spark SQL is
> applying the rounding even to the original column. Attached image has the
> spark sql code that shows the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
