GitHub user actuaryzhang opened a pull request:
https://github.com/apache/spark/pull/16149
[SPARK-18715][ML]Fix AIC calculations in Binomial GLM
The AIC calculation in Binomial GLM seems to be off when the response or
weight is non-integer: the result is different from that in R
Github user actuaryzhang commented on a diff in the pull request:
https://github.com/apache/spark/pull/16131#discussion_r90912181
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala
---
@@ -505,7 +505,7 @@ object
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/16131
@srowen
Try this example below or the example @sethah had issue with in #15683.
I have tried running the 2.1 version Poisson GLM on our data and it fails
for most of them
Github user actuaryzhang commented on a diff in the pull request:
https://github.com/apache/spark/pull/16131#discussion_r90784498
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala
---
@@ -505,7 +505,7 @@ object
Github user actuaryzhang commented on a diff in the pull request:
https://github.com/apache/spark/pull/15683#discussion_r90771932
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala
---
@@ -88,6 +89,12 @@ class
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/16131
Jenkins, add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user actuaryzhang opened a pull request:
https://github.com/apache/spark/pull/16131
[SPARK-18701][ML] Poisson GLM fails due to wrong initialization
Poisson GLM fails for many standard data sets (see example in test or
JIRA). The issue is incorrect initialization leading
Github user actuaryzhang commented on a diff in the pull request:
https://github.com/apache/spark/pull/15683#discussion_r87662897
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala
---
@@ -88,6 +89,12 @@ class
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/15683
@sethah Thanks for your review and suggestion. I have made a new commit
reflecting your comments.
@srowen Thanks for all the suggestions. When do you think this change could
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/15683
@sethah Thanks for the review and comments. I now created a separate unit
test. It also passed the style test.
I accidentally merged master into a branch... and don't know how
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/15683
@srowen @thunterdb
I just updated the unit test for poisson GLM (only for the log link). The
simulated data are now forced to take values of zero. Existing data generation
Github user actuaryzhang commented on the issue:
https://github.com/apache/spark/pull/15683
@srowen Will add the unit test over the weekend.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user actuaryzhang opened a pull request:
https://github.com/apache/spark/pull/15683
[SPARK-18166][MLib] Fix Poisson GLM bug due to wrong requirement of
response values
## What changes were proposed in this pull request?
The current implementation of Poisson GLM
501 - 513 of 513 matches
Mail list logo
