[ https://issues.apache.org/jira/browse/SPARK-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-14839: ------------------------------------ Assignee: Apache Spark > Support for other types as option in OPTIONS clause > --------------------------------------------------- > > Key: SPARK-14839 > URL: https://issues.apache.org/jira/browse/SPARK-14839 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.0.0 > Reporter: Hyukjin Kwon > Assignee: Apache Spark > Priority: Minor > > This was found in https://github.com/apache/spark/pull/12494. > Currently, Spark SQL does not support other types and {{null}} as a value of > an options. > For example, > {code} > CREATE ... > USING csv > OPTIONS (path "your-path", quote null) > {code} > throws an exception below > {code} > Unsupported SQL statement > == SQL == > CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, > modelName string, comments string, grp string) USING csv OPTIONS (path > "your-path", quote null) > org.apache.spark.sql.catalyst.parser.ParseException: > Unsupported SQL statement > == SQL == > CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, > modelName string, comments string, grp string) USING csv OPTIONS (path > "your-path", quote null) > at > org.apache.spark.sql.catalyst.parser.AbstractSqlParser.nativeCommand(ParseDriver.scala:66) > at > org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:56) > at > org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:53) > at > org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:86) > at > org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53) > at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:195) > at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:764) > ... > {code} > Currently, Scala API supports to take options with the types, {{String}}, > {{Long}}, {{Double}} and {{Boolean}} and Python API also supports other > types. I think in this way we can support data sources in a consistent way. > It looks it is okay to to provide other types as arguments just like > [Microsoft SQL|https://msdn.microsoft.com/en-us/library/ms190322.aspx] > because [SQL-1992|http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt] > standard mentions options as below: > {quote} > An implementation remains conforming even if it provides user op- > tions to process nonconforming SQL language or to process conform- > ing SQL language in a nonconforming manner. > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org