[ https://issues.apache.org/jira/browse/SPARK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li resolved SPARK-20143. ----------------------------- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0 > DataType.fromJson should throw an exception with better message > --------------------------------------------------------------- > > Key: SPARK-20143 > URL: https://issues.apache.org/jira/browse/SPARK-20143 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.2.0 > Reporter: Hyukjin Kwon > Assignee: Hyukjin Kwon > Priority: Minor > Fix For: 2.2.0 > > > Currently, > {code} > scala> import org.apache.spark.sql.types.DataType > import org.apache.spark.sql.types.DataType > scala> DataType.fromJson( """"abcd"""") > java.util.NoSuchElementException: key not found: abcd > at scala.collection.MapLike$class.default(MapLike.scala:228) > at scala.collection.AbstractMap.default(Map.scala:59) > at scala.collection.MapLike$class.apply(MapLike.scala:141) > at scala.collection.AbstractMap.apply(Map.scala:59) > at org.apache.spark.sql.types.DataType$.nameToType(DataType.scala:118) > at org.apache.spark.sql.types.DataType$.parseDataType(DataType.scala:132) > at org.apache.spark.sql.types.DataType$.fromJson(DataType.scala:104) > ... 48 elided > scala> DataType.fromJson( """{"abcd":"a"}""") > scala.MatchError: JObject(List((abcd,JString(a)))) (of class > org.json4s.JsonAST$JObject) > at org.apache.spark.sql.types.DataType$.parseDataType(DataType.scala:130) > at org.apache.spark.sql.types.DataType$.fromJson(DataType.scala:104) > ... 48 elided > scala> DataType.fromJson( """{"fields": [{"a":123}], "type": "struct"}""") > scala.MatchError: JObject(List((a,JInt(123)))) (of class > org.json4s.JsonAST$JObject) > at > org.apache.spark.sql.types.DataType$.org$apache$spark$sql$types$DataType$$parseStructField(DataType.scala:169) > at > org.apache.spark.sql.types.DataType$$anonfun$parseDataType$1.apply(DataType.scala:150) > at > org.apache.spark.sql.types.DataType$$anonfun$parseDataType$1.apply(DataType.scala:150) > at scala.collection.immutable.List.map(List.scala:273) > at org.apache.spark.sql.types.DataType$.parseDataType(DataType.scala:150) > at org.apache.spark.sql.types.DataType$.fromJson(DataType.scala:104) > ... 48 elided > {code} > {{DataType.fromJson}} throws non-readable error messages for the json input. > We could improve this rather than throwing {{scala.MatchError}} or > {{java.util.NoSuchElementException}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org