Re: [MLLib] Logistic Regression and standadization

2018-04-28 Thread Joseph PENG
Hi Valeriy, Let me make sure we are on the same page. "the current mllib implementation returns exactly the same model whether standardization is turned on or off. " This should be corrected as "the current mllib implementation returns exactly the same model whether standardization is turned on

Re: GLM Poisson Model - Deviance calculations

2018-04-18 Thread Joseph PENG
Are you referring this? override def deviance(y: Double, mu: Double, weight: Double): Double = { 2.0 * weight * (y * math.*log(y / mu)* - (y - mu)) } Not sure how does R handle this, but my guess is they may add a small number, e.g. 0.5, to the numerator and denominator. If you can

[Spark Debug] Search vars of interest

2017-10-29 Thread Joseph PENG
I am debug spark (itself not its application) by running `sbt default test suite`. I have successfully set up everything, and IntelliJ debugger have caught every variables. However, I can not find the variables I need, because there are so many variables. For example, I want to find the input