[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2020-09-02 Thread Jeff Harrison (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189634#comment-17189634 ] Jeff Harrison commented on SPARK-28079: --- I would also like something along these lines - it would

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868582#comment-16868582 ] F Jimenez commented on SPARK-28079: --- When an explicit user-defined schema is not passed in, one will

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868577#comment-16868577 ] Liang-Chi Hsieh commented on SPARK-28079: - {{columnNameOfCorruptRecord}} currently applied only

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868367#comment-16868367 ] F Jimenez commented on SPARK-28079: --- I can confirm there is no change using 2.4.4 built from latest

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868345#comment-16868345 ] F Jimenez commented on SPARK-28079: --- No, or not yet ;). I'm trying to build 2.4.4 from the branch to

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868341#comment-16868341 ] Hyukjin Kwon commented on SPARK-28079: -- Did you check the doc added by the fix of SPARK-28058

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-20 Thread F Jimenez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868327#comment-16868327 ] F Jimenez commented on SPARK-28079: --- Hi, sorry for the late response I'll paste the relevant part in

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868235#comment-16868235 ] Hyukjin Kwon commented on SPARK-28079: -- I think he's saying the last row is not in the corrupt row

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

2019-06-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866777#comment-16866777 ] Liang-Chi Hsieh commented on SPARK-28079: - Isn't it the expected behavior as documented in