[jira] [Created] (SPARK-14800) Dealing with null as a value in options for each internal data source

[Hyukjin Kwon (JIRA)]

Hyukjin Kwon created SPARK-14800:
------------------------------------

             Summary: Dealing with null as a value in options for each internal 
data source
                 Key: SPARK-14800
                 URL: https://issues.apache.org/jira/browse/SPARK-14800
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Hyukjin Kwon
            Priority: Minor


This was found in https://github.com/apache/spark/pull/12494.

Currently, most of options in data sources throws {{NullPointerException}} when 
given value is {{null}}.

For example, the codes below:

{code}
sqlContext.read
      .format("csv")
      .option("compression", null)
{code}

throws an exception below:

{code}
java.lang.NullPointerException was thrown.
java.lang.NullPointerException
    at 
org.apache.spark.sql.execution.datasources.csv.CSVOptions.getChar(CSVOptions.scala:32)
    at 
org.apache.spark.sql.execution.datasources.csv.CSVOptions.<init>(CSVOptions.scala:68)
    at 
org.apache.spark.sql.execution.datasources.csv.DefaultSource.prepareWrite(DefaultSource.scala:86)
...
{code}

while some of options such as {{nullValue}} in CSV data source accepts {{null}} 
.

So, the options throwing {{NullPointerException}} when it sets to {{null}} 
might have to be handled separately for each data source.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org