[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user koertkuipers commented on the issue: https://github.com/apache/spark/pull/13933 For parquet, json etc. path not being put in options is not an issue since they don't retrieve it from the options On Jun 29, 2016 2:31 AM, "Xiao Li" wrote:

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13933 @zsxwing If we just provide one `path` in the function input, it will not put `path` into the `options`. The API `parquet(path: String)`still calls `load(paths : _*)`, instead of `load(path)`.

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13933 @gatorsmile parquet, json or other file formats support both `path` and `paths` options. So that's not a problem. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13933 The problem also exists in the other APIs: like parquet, json and others.

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13933 @zsxwing Let me try to fix it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13933 I noticed that the Python API is inconsistent here: https://github.com/apache/spark/blob/1aad8c6e59c1e8b18a3eaa8ded93ff6ad05d83df/python/pyspark/sql/readwriter.py#L147 It always calls `def

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/13933 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13933 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/13933 LGTM -- cc @tdas to take a look since he wrote the original patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13933 cc @rxin The code is ready for review. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13933 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61342/ Test PASSed. ---

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13933 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13933 **[Test build #61342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61342/consoleFull)** for PR 13933 at commit

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13933 **[Test build #61342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61342/consoleFull)** for PR 13933 at commit

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13933 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61333/ Test FAILed. ---

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13933 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13933 **[Test build #61333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61333/consoleFull)** for PR 13933 at commit

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13933 **[Test build #61333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61333/consoleFull)** for PR 13933 at commit

[GitHub] spark issue #13933: [SPARK-16236] [SQL] Add Path Option back to Load API in ...

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13933 cc @rxin @tdas @koertkuipers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled