Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
[SPARK-17201](https://issues.apache.org/jira/browse/SPARK-17201)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does n
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/13796
@dbtsai
in this jira
https://issues.apache.org/jira/browse/SPARK-17163
it consider to unify interface for binary logistic regression and softmax.
but I think they are not equival
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
The solution of this overparameterized problem in the link is just adding
the regularization, and users may not want it. I think we need to optimize it
on (k-1) parameters, and then put the final or
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/13796
@dbtsai
simply say, when reg==0, the minimizer is not a point but becomes a plane.
so the second derivative on the plane will turn into zero. so ...
and I give a reference here:
ht
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@WeichenXu123 Do you run into this potential issue with any dataset? If so,
we may need to consider optimize softmax with pivoting when `reg == 0`.
---
If your project is set up for it, you can repl
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/13796
I found a problem in the merged code: when reg == 0 the minimizer of
softmax cost is not unique.
In such case, it will cause Hessian matrix non-invertible, and I thinks it
may cause quasi-
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah I also have a concern to have a separate MLOR class, and I prefer
consolidate them into one so we can maintain them easier. This has to be done
before the release of 2.1 otherwise, we can not
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
@dbtsai Thanks for all of your meticulous review. Very much appreciated!
Glad we can have MLOR in Spark ML now.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah Thank you for this great weighted MLOR work in Spark 2.1. I merged
this PR into master, and let's discuss/work on the followups in separate JIRAs.
Thanks.
---
If your project is set up for
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64033/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #64033 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64033/consoleFull)**
for PR 13796 at commit
[`fc2aa95`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #64033 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64033/consoleFull)**
for PR 13796 at commit
[`fc2aa95`](https://github.com/apache/spark/commit/f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64007/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #64007 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64007/consoleFull)**
for PR 13796 at commit
[`ffc64d4`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #64007 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64007/consoleFull)**
for PR 13796 at commit
[`ffc64d4`](https://github.com/apache/spark/commit/f
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
I also created an umbrella JIRA for tracking follup items -
[SPARK-17133](https://issues.apache.org/jira/browse/SPARK-17133)
---
If your project is set up for it, you can reply to this email and hav
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63996/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63996 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63996/consoleFull)**
for PR 13796 at commit
[`0c851d7`](https://github.com/apache/spark/commit/
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
Thanks @dbtsai for the detailed review! I addressed most comments. We still
need to:
* Decide whether to handle numClasses being specified in metadata
* Decide what happens when numClasse
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63996 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63996/consoleFull)**
for PR 13796 at commit
[`0c851d7`](https://github.com/apache/spark/commit/0
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
I go through the PR again, and it's in a very good shape. Only couple minor
issues needed to be addressed. Thank you @sethah for the great work. This will
be a big feature in Spark 2.1
---
If your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63951/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63951 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63951/consoleFull)**
for PR 13796 at commit
[`f02b226`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63951 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63951/consoleFull)**
for PR 13796 at commit
[`f02b226`](https://github.com/apache/spark/commit/f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63938/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63938 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63938/consoleFull)**
for PR 13796 at commit
[`61521e6`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63938 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63938/consoleFull)**
for PR 13796 at commit
[`61521e6`](https://github.com/apache/spark/commit/6
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63878/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63878 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63878/consoleFull)**
for PR 13796 at commit
[`b4203a5`](https://github.com/apache/spark/commit/
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
I checked that the derivations in the `LogisticAggregator` doc render
nicely with scala doc, even though it's a private class and so will not show up
in the API. There was a slight complication with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63878 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63878/consoleFull)**
for PR 13796 at commit
[`b4203a5`](https://github.com/apache/spark/commit/b
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63849/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63849 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63849/consoleFull)**
for PR 13796 at commit
[`96c52a5`](https://github.com/apache/spark/commit/
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
I added the derivation and other details to the scala doc for
`LogisticAggregator`. I have not had a chance to run the doc build yet, but the
content is there now for review. I'll build the docs toda
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63849 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63849/consoleFull)**
for PR 13796 at commit
[`96c52a5`](https://github.com/apache/spark/commit/9
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah Thank you for great work. I'll make another pass tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
@dbtsai Thanks for taking the time to review this! Major items right now:
* Adding derivation to the aggregator doc (this is mostly finished, just
fighting scala doc with Latex)
* Decidin
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
Maybe we could move the shared code `MultiClassSummarizer`,
`LogisticAggregator` and `LogisticCostFun` into
`LogisticRegressionUtils.scala`?
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63804/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63804 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63804/consoleFull)**
for PR 13796 at commit
[`51f51e6`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63804 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63804/consoleFull)**
for PR 13796 at commit
[`51f51e6`](https://github.com/apache/spark/commit/5
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63799/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63799 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63799/consoleFull)**
for PR 13796 at commit
[`0d72c6f`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63798/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63798 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63798/consoleFull)**
for PR 13796 at commit
[`2f11bd7`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63799 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63799/consoleFull)**
for PR 13796 at commit
[`0d72c6f`](https://github.com/apache/spark/commit/0
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63798 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63798/consoleFull)**
for PR 13796 at commit
[`2f11bd7`](https://github.com/apache/spark/commit/2
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63704 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63704/consoleFull)**
for PR 13796 at commit
[`026e1f6`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63704/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63702/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63702 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63702/consoleFull)**
for PR 13796 at commit
[`9d4559e`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63704 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63704/consoleFull)**
for PR 13796 at commit
[`026e1f6`](https://github.com/apache/spark/commit/0
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #63702 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63702/consoleFull)**
for PR 13796 at commit
[`9d4559e`](https://github.com/apache/spark/commit/9
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah Please merge master since there is a conflict. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah I apologize for the delay. I just came back to US. Gonna make the
first pass. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user mengxr commented on the issue:
https://github.com/apache/spark/pull/13796
@sethah Thanks for implementing `MultinomialLogisticRegression`! This would
be a major feature for Spark 2.1. @dbtsai is probably the best people to review
this PR. But he is taking a break now. Do y
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #60901 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60901/consoleFull)**
for PR 13796 at commit
[`f4c817f`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60901/
Test PASSed.
---
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
cc @dbtsai @jkbradley @mengxr If you get a chance to review and/or provide
feedback on the approach I'd appreciate it.
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #60901 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60901/consoleFull)**
for PR 13796 at commit
[`f4c817f`](https://github.com/apache/spark/commit/f
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13796
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60900/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #60900 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60900/consoleFull)**
for PR 13796 at commit
[`e4681b0`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13796
**[Test build #60900 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60900/consoleFull)**
for PR 13796 at commit
[`e4681b0`](https://github.com/apache/spark/commit/e
77 matches
Mail list logo
