Reynold Xin created SPARK-7961: ---------------------------------- Summary: Redesign SQLConf to report better error messages Key: SPARK-7961 URL: https://issues.apache.org/jira/browse/SPARK-7961 Project: Spark Issue Type: Improvement Components: SQL Reporter: Reynold Xin Priority: Blocker
Right now, we don't validate config values and as a result will throw exceptions when queries or DataFrame operations are run. Imagine if one user sets config variable "spark.sql.retainGroupColumns" (requires "true", "false") to "hello". The set action itself will complete fine. When another user runs a query, it will throw the following exception: {code} java.lang.IllegalArgumentException: For input string: "hello" at scala.collection.immutable.StringLike$class.parseBoolean(StringLike.scala:238) at scala.collection.immutable.StringLike$class.toBoolean(StringLike.scala:226) at scala.collection.immutable.StringOps.toBoolean(StringOps.scala:31) at org.apache.spark.sql.SQLConf.dataFrameRetainGroupColumns(SQLConf.scala:265) at org.apache.spark.sql.GroupedData.toDF(GroupedData.scala:74) at org.apache.spark.sql.GroupedData.agg(GroupedData.scala:227) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org