Has any one ran Poisson GLM model and got the
GeneralizedLinearRegressionTrainingSummary object (to access p, t-values,
deviances ,aic etc.,) successfully?
I have tried to fit two datasets to compare Spark vs R outputs, both models
ran fine in Spark and i was able to get the coefficients back. But when i
tried to get the object "GeneralizedLinearRegressionTrainingSummary" i
always get the following error
java.lang.NumberFormatException
at java.math.BigDecimal.<init>(BigDecimal.java:494)
at java.math.BigDecimal.<init>(BigDecimal.java:824)
at scala.math.BigDecimal$.decimal(BigDecimal.scala:52)
at scala.math.BigDecimal$.apply(BigDecimal.scala:249)
at
org.apache.spark.ml.regression.GeneralizedLinearRegressionTrainingSummary.org$apache$spark$ml$regression$GeneralizedLinearRegressionTrainingSummary$$round$1(GeneralizedLinearRegression.scala:1503)
at
org.apache.spark.ml.regression.GeneralizedLinearRegressionTrainingSummary.toString(GeneralizedLinearRegression.scala:1551)
at
scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:332)
at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:337)
Looking into Spark 2.3 source code for
GeneralizedLinearRegressionTrainingSummary's toString method throws error at
rounding the deviances. For some reason the deviances are NaN.
I have ran the same model in spark 2.2.0, i can get the
GeneralizedLinearRegressionTrainingSummary object fine(slightly different
code than Spark 2.3) but the deviances are also NaN.
Does any one know why the deviances are NaN?
Thanks,
Srikar.V
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: [email protected]
