[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753728#comment-15753728 ]
Navya Krishnappa commented on SPARK-18877: ------------------------------------------ Precision and scale vary depending on the decimal values in the column. Suppose if source file contains Amount(column name) 9.03E+12 1.19E+11 24335739714 1.71E+11 then spark consider Amount column as decimal(3,-9). and throws an below mentioned exception Caused by: java.lang.IllegalArgumentException: requirement failed: Decimal precision 4 exceeds max precision 3 at scala.Predef$.require(Predef.scala:224) at org.apache.spark.sql.types.Decimal.set(Decimal.scala:112) at org.apache.spark.sql.types.Decimal$.apply(Decimal.scala:425) at org.apache.spark.sql.execution.datasources.csv.CSVTypeCast$.castTo(CSVInferSchema.scala:264) at org.apache.spark.sql.execution.datasources.csv.CSVRelation$$anonfun$csvParser$3.apply(CSVRelation.scala:116) > Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: > requirement failed: Decimal precision 28 exceeds max precision 20 > ---------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-18877 > URL: https://issues.apache.org/jira/browse/SPARK-18877 > Project: Spark > Issue Type: Bug > Affects Versions: 2.0.2 > Reporter: Navya Krishnappa > > When reading below mentioned csv data, even though the maximum decimal > precision is 38, following exception is thrown > java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 > exceeds max precision 20 > Decimal > 2323366225312000000000000000 > 24335739714000000 > 23233662253000 > 232336622530000 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org