Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-217014407
Oh, thank you, @andrewor14 .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12809
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-217009927
Merging into master 2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-217009910
I'm going to merge this one first to avoid conflicts since it's such a big
patch.
---
If your project is set up for it, you can reply to this email and have your
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62118419
--- Diff: python/pyspark/context.py ---
@@ -952,6 +952,11 @@ def dump_profiles(self, path):
"""
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-217005321
Thank you for review, again!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62116382
--- Diff: python/pyspark/context.py ---
@@ -952,6 +952,11 @@ def dump_profiles(self, path):
"""
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62116077
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/RegressionMetricsExample.scala
---
@@ -18,22 +18,22 @@
package
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62115862
--- Diff:
examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
---
@@ -21,23 +21,19 @@
import
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62115755
--- Diff:
examples/src/main/python/mllib/binary_classification_metrics_example.py ---
@@ -27,7 +27,7 @@
if __name__ == "__main__":
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-217003710
Thanks @dongjoon-hyun. This is mostly straightforward and it LGTM. I
pointed out a few parts that were not totally straightforward for other
reviewers.
---
If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62115390
--- Diff:
examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
---
@@ -21,23 +21,19 @@
import
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62115305
--- Diff: python/pyspark/context.py ---
@@ -952,6 +952,11 @@ def dump_profiles(self, path):
"""
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62115163
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/RegressionMetricsExample.scala
---
@@ -18,22 +18,22 @@
package
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62114725
--- Diff:
examples/src/main/scala/org/apache/spark/examples/ml/DecisionTreeExample.scala
---
@@ -167,17 +167,17 @@ object DecisionTreeExample {
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62114550
--- Diff:
examples/src/main/scala/org/apache/spark/examples/ml/CountVectorizerExample.scala
---
@@ -51,6 +48,8 @@ object CountVectorizerExample {
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62114466
--- Diff:
examples/src/main/python/mllib/binary_classification_metrics_example.py ---
@@ -27,7 +27,7 @@
if __name__ == "__main__":
sc
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62114371
--- Diff:
examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
---
@@ -21,23 +21,19 @@
import
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62114250
--- Diff:
examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
---
@@ -21,23 +21,19 @@
import
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12809#discussion_r62113498
--- Diff:
examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
---
@@ -21,23 +21,19 @@
import
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216959655
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216959651
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216959504
**[Test build #57779 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57779/consoleFull)**
for PR 12809 at commit
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216953628
Hi, @rxin and @andrewor14 .
Now, Scala/Java/Python examples use new style.
Could you review this PR?
---
If your project is set up for it, you can reply
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216952906
**[Test build #57779 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57779/consoleFull)**
for PR 12809 at commit
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216952309
Rebased to resolve conflicts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216951343
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216951346
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216951209
**[Test build #57775 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57775/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216825368
**[Test build #57752 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57752/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216825441
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216825445
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216823930
**[Test build #57752 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57752/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216823364
**[Test build #57751 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57751/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216823367
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216823366
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216823085
**[Test build #57751 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57751/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216821606
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216821608
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216821601
**[Test build #57750 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57750/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216821335
**[Test build #57750 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57750/consoleFull)**
for PR 12809 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216617154
By the way @dongjoon-hyun we don't want to expose the constructor to the
public (see #12873), so a lot of the changes here will have to be rewritten to
use the
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216390963
Oh! I misunderstood. :)
Thanks. I'll take that issue happily.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216389674
I was suggesting if you have time go ahead and do the python builder api :)
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216389243
Okay, I'll remove Python-related stuff here first and keep them for later.
When do you expect for merging new builder api for python?
---
If your project is
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216388122
Actually one thing is that we don't yet have a builder api for python. Can
you do that in a separate pr first?
---
If your project is set up for it, you can reply to
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216386899
Sure! I'm waiting for it. :) Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216384165
I've merged https://github.com/apache/spark/pull/12830
want to update the example using that?
---
If your project is set up for it, you can reply to this email
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216080409
Thank you for feedback, @rxin . Sure!
Also, congratulation for branching 2.0. :)
---
If your project is set up for it, you can reply to this email and have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216080058
Hey @dongjoon-hyun Give me a day or two to think about the API for creating
SparkSession. It'd be great to finalize that and then update the examples to
reflect that.
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216057809
Hi, @rxin .
Could you merge this PR please?
Or, please let me know if there is something to do more.
---
If your project is set up for it, you can reply
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216017057
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216017058
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216017020
**[Test build #57465 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57465/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216014636
**[Test build #57465 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57465/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216012761
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216012759
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216012754
**[Test build #57462 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57462/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216012528
**[Test build #57462 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57462/consoleFull)**
for PR 12809 at commit
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216012424
Rebased.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216000775
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216000776
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-216000744
**[Test build #57454 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57454/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215999659
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215999660
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215999614
**[Test build #57453 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57453/consoleFull)**
for PR 12809 at commit
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215996680
Now, I addressed all comments so far.
Thank you for fast reviews, @rxin .
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215996651
**[Test build #57454 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57454/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215995374
**[Test build #57453 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57453/consoleFull)**
for PR 12809 at commit
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215995243
I updated Java examples with SparkSession(JavaSparkContext). For
`SparkSession(SparkConf)`, I'll handle soon.
---
If your project is set up for it, you can
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215994403
Yea that makes sense.
For this one, I think we should also have a SparkSession ctor that takes in
SparkConf, which calls SparkContext.getOrCreate to load a
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215992910
Hi, @rxin . For this issue, I'll add new constructor for `SparkSession` and
proceed to Java examples.
```
def this(sparkContext: JavaSparkContext) =
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215957101
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215957102
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215957054
**[Test build #57441 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57441/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215956371
**[Test build #57441 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57441/consoleFull)**
for PR 12809 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215955469
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215955468
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215955422
**[Test build #57440 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57440/consoleFull)**
for PR 12809 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12809#issuecomment-215954989
**[Test build #57440 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57440/consoleFull)**
for PR 12809 at commit
80 matches
Mail list logo
