[ https://issues.apache.org/jira/browse/SPARK-16903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408900#comment-15408900 ]
Liwei Lin commented on SPARK-16903: ----------------------------------- [~hyukjin.kwon] thanks for pinging me! I believe the cause is {{nullValue}} does not apply to {{StringType}} today. I'll update {{https://github.com/apache/spark/pull/14118}} to fix the {{StringType}} thing. > nullValue in first field is not respected by CSV source when read > ----------------------------------------------------------------- > > Key: SPARK-16903 > URL: https://issues.apache.org/jira/browse/SPARK-16903 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.0 > Reporter: Hossein Falaki > > file: > {code} > a,- > -,10 > {code} > Query: > {code} > create temporary table test(key string, val decimal) > using com.databricks.spark.csv > options (path "/tmp/hossein2/null.csv", header "false", delimiter ",", > nullValue "-"); > {code} > Result: > {code} > select count(*) from test where key is null > 0 > {code} > But > {code} > select count(*) from test where val is null > 1 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org