Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155232384
@jkbradley I send the PR at https://github.com/apache/spark/pull/9577.
---
If your project is set up for it, you can reply to this email and have your
reply appear on Gi
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155175314
@jkbradley sure!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featu
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155148314
@yu-iskw Thanks for this! Quick request: Could you please send a little
follow-up PR to document (in the Python doc) what is being returned?
---
If your project is s
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154643744
Thank you for merging it and your great support!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8643
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154642699
LGTM, merging this into master and 1.6 branch, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154640689
@davies thanks for the review. I fixed them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proj
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154640453
**[Test build #2002 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2002/consoleFull)**
for PR 8643 at commit
[`e91c65a`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154632203
**[Test build #2002 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2002/consoleFull)**
for PR 8643 at commit
[`e91c65a`](https://git
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154629369
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154618704
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154617157
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154615987
LGTM, but a few minor comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207597
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -773,10 +776,10 @@ def train(cls, rdd, k=10, maxIterations=20,
docConcentration=-1.0,
:param o
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207604
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -745,9 +749,8 @@ def load(cls, sc, path):
raise TypeError("sc should be a SparkContext, got
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207559
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -687,9 +687,14 @@ class LDAModel(JavaModelWrapper):
... [2, SparseVector(2, {0: 1.0})],
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154585403
@jkbradley @davies could you review it? I modified the type conversion
using `SerDe.dumps`.
---
If your project is set up for it, you can reply to this email and have y
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154558405
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154558189
**[Test build #45258 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45258/consoleFull)**
for PR 8643 at commit
[`81ee096`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154546841
**[Test build #45258 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45258/consoleFull)**
for PR 8643 at commit
[`81ee096`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154544398
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154544425
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user shaneknapp commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154543591
jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not hav
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154537228
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154532750
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154531317
Build finished. No test results found.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154531334
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45252/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154529543
Build triggered. sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154529620
Build started sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154527959
**[Test build #45250 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/consoleFull)**
for PR 8643 at commit
[`56b69f9`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154527979
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154527973
Build finished. No test results found.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526569
Build started sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526739
**[Test build #45250 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/consoleFull)**
for PR 8643 at commit
[`56b69f9`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526527
Build triggered. sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44185186
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44172629
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44171965
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44169316
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44168908
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44164309
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43823836
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43723787
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43723078
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43722360
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720929
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -690,6 +690,21 @@ class LDAModel(JavaModelWrapper):
>>> model = LDA.train(rdd, k=2)
>>> mo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720867
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720248
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247670
I reverted DataFrame serialization to Java Any types.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247107
**[Test build #44881 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44881/consoleFull)**
for PR 8643 at commit
[`0bc114e`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247342
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247346
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237999
**[Test build #44881 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44881/consoleFull)**
for PR 8643 at commit
[`0bc114e`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237516
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237504
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43658405
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fo
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43657655
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Fou
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976267
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976265
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does n
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976149
**[Test build #44799 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44799/consoleFull)**
for PR 8643 at commit
[`353a6b0`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152939997
**[Test build #44799 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44799/consoleFull)**
for PR 8643 at commit
[`353a6b0`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152938719
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have thi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152938736
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392863
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392862
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392759
**[Test build #44182 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44182/consoleFull)**
for PR 8643 at commit
[`2f70193`](https://git
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150372929
@jkbradley sorry for the delay of my update. I tried to use DataFrame
serialization at
https://github.com/yu-iskw/spark/commit/2f701930677c7d359c0a706d16c2438509528b77.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150372009
**[Test build #44182 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44182/consoleFull)**
for PR 8643 at commit
[`2f70193`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150371481
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150371516
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-141212929
@yu-iskw I think it's OK to use DataFrame internally in spark.mllib. It
already has the dependency, and it would be a private API.
---
If your project is set up for
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-140981031
@jkbradley thank you for the comment. Just to be sure,
`LDAModelWrapper`.`describeTopics()` should return a DataFrame and then extract
the return value from the DataFram
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-139610362
@yu-iskw Rather than using Java Any types and the old serialization
patterns, would it be easier to convert to a local DataFrame? We should be
able to take advantage
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327571
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327569
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327468
[Test build #42099 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42099/console)
for PR 8643 at commit
[`97e78b6`](https://github.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138319885
[Test build #42099 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42099/consoleFull)
for PR 8643 at commit
[`97e78b6`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138318637
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138318657
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306191
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306187
[Test build #42096 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42096/console)
for PR 8643 at commit
[`f300798`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306190
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138305856
[Test build #42096 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42096/consoleFull)
for PR 8643 at commit
[`f300798`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138304711
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138304729
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user yu-iskw opened a pull request:
https://github.com/apache/spark/pull/8643
[SPARK-8467][MLlib][PySpark] Add LDAModel.describeTopics() in Python
Could @jkbradley and @davies review it?
- Create a wrapper class: `LDAModelWrapper` for `LDAModel`. Because we
can't dea
86 matches
Mail list logo
