[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22528 We should stop returning a wrong result. Please fix it. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22528 @MaxGekk, please feel free to proceed based upon your idea. From a cursory look, the idea makes sense to me. --- - To

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-28 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22528 Since there is no consensus, I am going to close the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-27 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22528 > Another concern here is, we have another place to control the compression codec (where we usually delegate to HDFS libraries). I was considering using Compressor API but its streaming

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22528 The limitation is quite clear - multi line mode only and single file. We don't have to rush about this - looks @MaxGekk has a clear idea on this. Another concern here is, we have another place

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22528 Generally, this is useful for reading the zip archives containing a single CSV. The extra support JSON file is not very useful. The major limitation is we have to read the zipped CSV

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-25 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22528 @gatorsmile Please, take a look at the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22528 @HyukjinKwon One of main use cases which the PR addresses is reading zipped CSV files. Currently it is possible to read such files by Spark but the approach looks over-complicated, for example:

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96477/ Test PASSed. ---

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22528 **[Test build #96477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96477/testReport)** for PR 22528 at commit

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22528 **[Test build #96477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96477/testReport)** for PR 22528 at commit

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22528 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96476/ Test FAILed. ---

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22528: [SPARK-25513][SQL] Read zipped CSV and JSON

2018-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22528 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional