Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/19861
Thanks for the example, @cloud-fan.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
Basically we want per-query options, which can be specified via
`DataFrameReader/Writer.option`, and we also want per-session options, which
can save users from typing same options again and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
@rdblue This is also what we already have for built-in data sources, e.g.
`spark.sql.parquet.compression.codec`, `spark.sql.parquet.filterPushdown`, etc.
---
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/19861
@jiangxb1987, I understand what this does. I just wanted an example use
case where it was necessary. What was the motivating *use case*?
---
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
With `SessionConfigSupport`, you can use the datasource session configs
more easily for that:
1. All configs with the name prefix `spark.datasource.$keyPrefix` will be
imported in to the
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/19861
@jiangxb1987, @cloud-fan, what was the use case you needed to add this for?
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85060/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #85060 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85060/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #85060 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85060/testReport)**
for PR 19861 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85049/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #85049 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85049/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #85049 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85049/testReport)**
for PR 19861 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84995/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84995 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84995/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84995 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84995/testReport)**
for PR 19861 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19861
Hm ..
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84988/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84988 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84988/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84988 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84988/testReport)**
for PR 19861 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84965/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84965 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84965/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84965 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84965/testReport)**
for PR 19861 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
SparkR test is pretty flaky recently...
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84961/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84961 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84961/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84961 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84961/testReport)**
for PR 19861 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84952/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84952 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84952/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84952 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84952/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84941/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84941 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84941/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84941 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84941/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84921/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84921 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84921/testReport)**
for PR 19861 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84921 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84921/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84914/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84914 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84914/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84914 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84914/testReport)**
for PR 19861 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19861
we should also do it for write path
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84902/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84902 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84902/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84902 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84902/testReport)**
for PR 19861 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84899 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84899/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84899/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84899 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84899/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84776/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84776 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84776/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84776 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84776/testReport)**
for PR 19861 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84745/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84745 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84745/testReport)**
for PR 19861 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84745 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84745/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84712/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84712 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84712/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84712 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84712/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84607/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84607 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84607/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84607 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84607/testReport)**
for PR 19861 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19861
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84603/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84603 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84603/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84603 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84603/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84565/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84565 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84565/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84565 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84565/testReport)**
for PR 19861 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84563 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84563/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84563/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19861
**[Test build #84563 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84563/testReport)**
for PR 19861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84433/
Test PASSed.
---
1 - 100 of 107 matches
Mail list logo
