Github user tedyu commented on a diff in the pull request:
https://github.com/apache/spark/pull/13160#discussion_r64677095
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
---
@@ -771,7 +777,11 @@ object SparkSession {
val sparkConf =
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/13160
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221652457
LGTM2. Merging into master 2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221462569
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221462571
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221462448
**[Test build #59243 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59243/consoleFull)**
for PR 13160 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221453746
LGTM pending Jenkins.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221442569
**[Test build #59243 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59243/consoleFull)**
for PR 13160 at commit
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221442596
Thanks @rxin, update the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13160#discussion_r64436157
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2245,6 +2245,7 @@ object SparkContext extends Logging {
if
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221342399
cc @andrewor14
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221300038
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221300037
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221299742
**[Test build #59201 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59201/consoleFull)**
for PR 13160 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221262642
**[Test build #59201 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59201/consoleFull)**
for PR 13160 at commit
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221253912
test it again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221251504
**[Test build #59200 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59200/consoleFull)**
for PR 13160 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221251537
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221251535
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221249792
**[Test build #59200 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59200/consoleFull)**
for PR 13160 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221243250
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221243247
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221243103
**[Test build #59199 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59199/consoleFull)**
for PR 13160 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-221224496
**[Test build #59199 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59199/consoleFull)**
for PR 13160 at commit
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-220970728
@rxin Seems it didn't resolve the issue described in the jira, I don't have
time to check the changes, may take a look at it tomorrow. Another issue I
found is that
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-220730659
@zjffdu did https://github.com/apache/spark/pull/13200 solve the problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-220463295
I've done it here: https://github.com/apache/spark/pull/13200
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-220201723
SharedState is constructed from SparkContext, so if there's already an
existing SparkContext, I can not pass additional session conf to SharedState
since the conf of
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-220196423
Can you explain more why is it a problem? I think it is OK to not change
the undelying SparkContext, since the configs are supposed to be mutable, but
only change the
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219946777
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219946772
**[Test build #58755 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58755/consoleFull)**
for PR 13160 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219946774
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219946536
**[Test build #58755 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58755/consoleFull)**
for PR 13160 at commit
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219946569
It seems changing sqlconf won't work. Because SharedState is coupled with
SparkContext closely while SparkContext.conf is immutable. That means if
SparkContext is
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219911211
logging a warning seems like a good idea!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zjffdu commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219910757
Changing sql conf setting seems a better approach. Besides that some other
setting in SparkSession#Build also won't take effect if there's already an
existing
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219909680
hm I don't think it is safe to create two contexts and stop the existing
one. Maybe we can go ahead and change the sql conf setting, but not the spark
context setting?
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219909346
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219909350
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219909325
**[Test build #58734 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58734/consoleFull)**
for PR 13160 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13160#issuecomment-219908222
**[Test build #58734 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58734/consoleFull)**
for PR 13160 at commit
GitHub user zjffdu opened a pull request:
https://github.com/apache/spark/pull/13160
[SPARK-15345][SQL][PYSPARK]. SparkSession's conf doesn't take effect when
this already an existing SparkContext
## What changes were proposed in this pull request?
Override the existing
42 matches
Mail list logo
