[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-152095666 You could create dataframe per path and then union them. On Oct 28, 2015 19:14, "Jon Edvald" wrote: > Hey all. Just ran into

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user edvald commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-151805526 Hey all. Just ran into this bug when upgrading to 1.5.1, very glad it was resolved! That said, I may not be able to run the updated code in my scenario - is there a

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on a diff in the pull request: https://github.com/apache/spark/pull/8416#discussion_r42308755 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -123,6 +124,24 @@ class DataFrameReader private[sql](sqlContext:

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148940616 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148940556 [Test build #43883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43883/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148940615 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148926431 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148927469 [Test build #43883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43883/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148926418 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8416 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148954533 Merged into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148798288 Actually, this one has not been merged yet (weird thing happened while merging it), @koertkuipers please go ahead to address the comment. --- If your project is set up

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/8416#discussion_r42274201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -123,6 +124,24 @@ class DataFrameReader private[sql](sqlContext:

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148629714 LGTM, merging into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343732 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343731 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343576 [Test build #43780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43780/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316257 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316285 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316840 [Test build #43780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43780/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148589922 i believe this is done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148512582 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148512580 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148512389 [Test build #43797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43797/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148455862 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148460264 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148455905 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148459073 [Test build #43792 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43792/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148460269 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148460253 [Test build #43792 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43792/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148478062 [Test build #43797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43797/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148476797 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148476827 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-147840468 LGTM except for the new API, cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-147872102 Thanks for working on this, I spent some time debating the API with @rxin and here is what we came up with: - calling the function `load(paths:

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-145902167 goals (copied over from SPARK-5741 comments by @marmbrus ): It was originally just parquet that would support more than one file, but now all HadoopFSRelations

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138771060 [Test build #42182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42182/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138770801 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138770812 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138790959 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138790960 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-138790889 [Test build #42182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42182/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-136034683 i updated this pullreq based on the conversation at https://issues.apache.org/jira/browse/SPARK-5741 --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135935355 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135935360 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135937690 [Test build #41779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41779/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135939465 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135939464 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135944143 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135944142 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135944133 [Test build #41778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41778/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135937501 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135937496 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135935546 [Test build #41778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41778/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135939456 [Test build #41779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41779/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135486744 Can you point me to the jira where that decision was made? Hadoop globbing only covers a small subset of all use cases. For example for timeseries analysis

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-135488186 I am not sure Union is a good idea at all, since i would have to union DataFrames for hundreds of partitions and the Union logical operator only takes left and

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134679875 We actually explicitly stopped supporting this because `,` is a valid character in file names and people were trying to use it. Instead we support standard hadoop

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134663598 [Test build #41535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41535/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134663995 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134663990 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134600515 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134600558 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134603095 [Test build #41535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41535/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

GitHub user koertkuipers opened a pull request: https://github.com/apache/spark/pull/8416 [SPARK-10185] [SQL] Feat sql comma separated paths Make sure comma-separated paths get processed correcly in ResolvedDataSource for a HadoopFsRelationProvider You can merge this pull request

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134586268 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134586306 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134588446 [Test build #41533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41533/consoleFull) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134589741 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134589729 [Test build #41533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41533/console) for PR 8416 at commit

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-134589745 Test FAILed. Refer to this link for build results (access rights to CI server needed):