[
https://issues.apache.org/jira/browse/SPARK-40616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611328#comment-17611328
]
Yuming Wang commented on SPARK-40616:
-------------------------------------
Please try to set spark.sql.legacy.exponentLiteralAsDecimal.enabled=true.
> Loss of precision using SparkSQL shell on high-precision DECIMAL types
> ----------------------------------------------------------------------
>
> Key: SPARK-40616
> URL: https://issues.apache.org/jira/browse/SPARK-40616
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.1
> Reporter: xsys
> Priority: Major
>
> h3. Describe the bug
> We are trying to save {{DECIMAL}} values with high precision in a table using
> the SparkSQL shell. When we {{INSERT}} decimal values with precision higher
> than the standard double precision, precision is lost.
> (8.8888888888888888888e9 interpreted as 8888888888.8888900000 instead of
> 8888888888.8888888888).
> This seems to be caused by type inference at shell parsing inferring that the
> value is a double type.
> h3. To reproduce
> On Spark 3.2.1 (commit {{{}4f25b3f712{}}}), using {{{}spark-sql{}}}:
> {code:java}
> $SPARK_HOME/bin/spark-sql{code}
> In the shell:
> {code:java}
> CREATE TABLE t(c0 DECIMAL(20,10));
> INSERT INTO t VALUES (8.8888888888888888888e9);
> SELECT * FROM t;{code}
> Executing the above gives this:
> {code:java}
> spark-sql> CREATE TABLE t(c0 DECIMAL(20,10));
> 22/09/29 11:28:41 WARN ResolveSessionCatalog: A Hive serde table will be
> created as there is no table provider specified. You can set
> spark.sql.legacy.createHiveTableByDefault to false so that native data source
> table will be created instead.
> Time taken: 0.118 seconds
> spark-sql> INSERT INTO t VALUES (8.8888888888888888888e9);
> Time taken: 0.392 seconds
> spark-sql> SELECT * FROM t;
> 8888888888.8888900000
> Time taken: 0.197 seconds, Fetched 1 row(s){code}
> h3. Expected behavior
> We expect the inserted value to retain the precision as determined by the
> parameters for the {{DECIMAL}} type. For example, we expect the example above
> to return {{{}8888888888.8888888888{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
