[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 Currently, the corrupted record field should be set explicitly if I haven't missed some changes in the related code path. Please refer the test here -

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-04-06 Thread hhamalai
Github user hhamalai commented on the issue: https://github.com/apache/spark/pull/17315 Just to clarify, with these changes in place, should the corrupted record field be visible when the data contains errors? To me the existing behaviour is still the same as it used to be, so no

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17315 @HyukjinKwon yes please go ahead! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 @cloud-fan, do you mind if I work on a followup if you are currently not? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17315 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74918/ Test PASSed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74918/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74918/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17315 LGTM except a major comment about FAIL_FAST_MODE. Looks great! Glad to see such a code clean! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74882/ Test PASSed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74879/ Test PASSed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74882/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74879 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74879/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74879/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74882/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17315 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 For me it looks good as targrted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74867/ Test FAILed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74867/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 Sorry, my PR got merged and made a conflict with this. These are all from me. Probably, let me try to leave some more trivial comments while it's open. --- If your project is set up for it,

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74867/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17315 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74857/ Test FAILed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74857/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74743/ Test PASSed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74743/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74743/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74738/ Test FAILed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74738/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74738/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17315 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 (The test failure seems not related with the last commit. There was a related discussion about that -9 https://github.com/apache/spark/pull/16305#issuecomment-267540707) --- If your project

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74728/ Test FAILed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74728/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17315 I support this idea. Let me try to take a close look within tomorrow to help. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74666/ Test FAILed. ---

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74666/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17315 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17315 **[Test build #74666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74666/testReport)** for PR 17315 at commit

[GitHub] spark issue #17315: [SPARK-19949][SQL] unify bad record handling in CSV and ...

2017-03-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17315 cc @sameeragarwal @HyukjinKwon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled