[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892679#comment-15892679 ] Apache Spark commented on SPARK-18699: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865959#comment-15865959 ] Apache Spark commented on SPARK-18699: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863426#comment-15863426 ] Hyukjin Kwon commented on SPARK-18699: -- Thanks for cc'ing me. For me, it is reasonable to me too

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863122#comment-15863122 ] Takeshi Yamamuro commented on SPARK-18699: -- This fix makes some sensible to me and what do you

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756175#comment-15756175 ] Apache Spark commented on SPARK-18699: -- User 'kubatyszko' has created a pull request for this issue:

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756170#comment-15756170 ] Kuba Tyszko commented on SPARK-18699: - Fixed in PR https://github.com/apache/spark/pull/16319 Enjoy

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750163#comment-15750163 ] Rishi Kamaleswaran commented on SPARK-18699: Thanks for the reply! Unfortunately neither of

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750050#comment-15750050 ] Hyukjin Kwon commented on SPARK-18699: -- BTW, maybe you could try to set {{nullValue}} to {{" "}} or

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748886#comment-15748886 ] Rishi Kamaleswaran commented on SPARK-18699: This issue is also seen in cases where a

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734318#comment-15734318 ] Takeshi Yamamuro commented on SPARK-18699: -- yea, I'm also working on large csv files now and,

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720710#comment-15720710 ] Jakub Nowacki commented on SPARK-18699: --- While I don't argue that some other packages have similar

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720128#comment-15720128 ] Hyukjin Kwon commented on SPARK-18699: -- Thank you for cc'ing me. Yup, I noticed this but I decided

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720069#comment-15720069 ] Takeshi Yamamuro commented on SPARK-18699: -- Anyway, we can easily fix this like this:

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720033#comment-15720033 ] Takeshi Yamamuro commented on SPARK-18699: -- Additionally, in our basic stance, it seems this csv

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719993#comment-15719993 ] Takeshi Yamamuro commented on SPARK-18699: -- `DROPMALFORMED` works well in this query though,

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719887#comment-15719887 ] Jakub Nowacki commented on SPARK-18699: --- Yes, my understanding was that it should put nullify the

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15719863#comment-15719863 ] Takeshi Yamamuro commented on SPARK-18699: -- you mean a query below and you'd like to load the