Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley Sure! Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
@GayathriMurali No problem; you've had to wait quite a while. I'll be
happy to take it over.
Could you please close this PR for now?
I'll send a new PR based on your commits (so
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley I am so sorry I couldn't respond to this on time! I am in a
transition process and might not be able to drive this JIRA to completion at
this point in time. Thanks!
---
If your
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
@GayathriMurali Btw, I know it's been a long time. If you no longer have
time to work on this, please say so, and I'd be happy to take it over. Thanks!
---
If your project is set up for it,
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
Done with review for now; I'll check back for updates. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #3234 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3234/consoleFull)**
for PR 14112 at commit
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
@GayathriMurali Apologies for the long delay! It slipped past the release,
and I'm trying to catch up on PRs now. I'll make a final review pass ASAP
---
If your project is set up for it, you
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #3234 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3234/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley Can you please help review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley Please let me know if I can do anything to help get this merged
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley Can you please help review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62568/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62568 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62568/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62568 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62568/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62567 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62567/consoleFull)**
for PR 14112 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62567/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62567 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62567/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley I implemented model loading logic for DistributedLDA as well. I
am using a versionRegex for robustness in version checking. Using
`as[Data].head()` is producing a scala match
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62337/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62337 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62337/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62337 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62337/consoleFull)**
for PR 14112 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62298 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62298/consoleFull)**
for PR 14112 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62298/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62298 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62298/consoleFull)**
for PR 14112 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62295/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62295 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62295/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62295 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62295/consoleFull)**
for PR 14112 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62293/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62293 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62293/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62293 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62293/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley I am sorry, I have been held up with something else. I am
looking on ways to add this to DistribtedLDA model. I will have something by
EOD today.
---
If your project is set up
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
@GayathriMurali I'd like to accelerate getting this merged. Please let me
know if you'd like help with it, especially with adding the fix for other LDA
classes. (We can collaborate on the same
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
@hhbyyh I agree about not changing the other models' loading code unless it
becomes necessary. I hope we can design a better long-term solution during 2.1.
We should definitely cover the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62125/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62125 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62125/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62125 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62125/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
+1 for separate loading logic. The recent commit includes separate code
paths depending on sparkVersion
---
If your project is set up for it, you can reply to this email and have your
reply
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/14112
I agree that it will make thing easier to separate the loading logic here
in LDA due to its extra complexity. Yet maybe we should not extend the pattern
to previous changes about loading
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
Thank for the PR!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
What will happen in the future if the model changes, such as storing
another val? We will need to update the model loading logic and should follow
different code paths for models saved in 1.6
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/14112
I agree it will be hard to do a proper unit test, but we should definitely
test offline. I've made a JIRA for designing backwards compatibility for the
long term:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62108/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62108 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62108/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62108 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62108/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@hhbyyh Thanks for helping out. Updated commit includes logic to include
topicDistributionCol @yanboliang
---
If your project is set up for it, you can reply to this email and have your
Github user yanboliang commented on the issue:
https://github.com/apache/spark/pull/14112
@hhbyyh I think offline test should be OK for now, since we don't have
unified save/load compatibility test framework until now. It's better we can
get this feature in the next RC.
---
If your
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/14112
@jkbradley I find it not easy to add a unit test to cover the logic.
Appreciate your thoughts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/14112
PR created. Sorry I got something else that I need to turn to. Ideally, the
overriding `getAndSetParams` should be in LDAParams, thus it can be reused by
LDA and LDA Local/Distributed Model. Please
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/14112
Thanks @GayathriMurali for the PR. I think we'll need to override the
default behavior of getAndSetParams. Meanwhile, we need to invoke both
convertVectorColumnsToML and convertMatrixColumnsToML.
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
retest this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14112
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62012/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62012 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62012/consoleFull)**
for PR 14112 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14112
**[Test build #62012 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62012/consoleFull)**
for PR 14112 at commit
Github user GayathriMurali commented on the issue:
https://github.com/apache/spark/pull/14112
@hhbyyh Can you please help review? I am not sure if this is the right way
to do it, as topicDistributionCol is not included in the MLWriter or load.
---
If your project is set up for it,
63 matches
Mail list logo
