[ https://issues.apache.org/jira/browse/SPARK-40616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611328#comment-17611328 ]
Yuming Wang commented on SPARK-40616: ------------------------------------- Please try to set spark.sql.legacy.exponentLiteralAsDecimal.enabled=true. > Loss of precision using SparkSQL shell on high-precision DECIMAL types > ---------------------------------------------------------------------- > > Key: SPARK-40616 > URL: https://issues.apache.org/jira/browse/SPARK-40616 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.2.1 > Reporter: xsys > Priority: Major > > h3. Describe the bug > We are trying to save {{DECIMAL}} values with high precision in a table using > the SparkSQL shell. When we {{INSERT}} decimal values with precision higher > than the standard double precision, precision is lost. > (8.8888888888888888888e9 interpreted as 8888888888.8888900000 instead of > 8888888888.8888888888). > This seems to be caused by type inference at shell parsing inferring that the > value is a double type. > h3. To reproduce > On Spark 3.2.1 (commit {{{}4f25b3f712{}}}), using {{{}spark-sql{}}}: > {code:java} > $SPARK_HOME/bin/spark-sql{code} > In the shell: > {code:java} > CREATE TABLE t(c0 DECIMAL(20,10)); > INSERT INTO t VALUES (8.8888888888888888888e9); > SELECT * FROM t;{code} > Executing the above gives this: > {code:java} > spark-sql> CREATE TABLE t(c0 DECIMAL(20,10)); > 22/09/29 11:28:41 WARN ResolveSessionCatalog: A Hive serde table will be > created as there is no table provider specified. You can set > spark.sql.legacy.createHiveTableByDefault to false so that native data source > table will be created instead. > Time taken: 0.118 seconds > spark-sql> INSERT INTO t VALUES (8.8888888888888888888e9); > Time taken: 0.392 seconds > spark-sql> SELECT * FROM t; > 8888888888.8888900000 > Time taken: 0.197 seconds, Fetched 1 row(s){code} > h3. Expected behavior > We expect the inserted value to retain the precision as determined by the > parameters for the {{DECIMAL}} type. For example, we expect the example above > to return {{{}8888888888.8888888888{}}}. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org