[
https://issues.apache.org/jira/browse/SPARK-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14067342#comment-14067342
]
Michael Yannakopoulos commented on SPARK-2552:
----------------------------------------------
Hi Xiangrui,
>From what I have seen so far, this error affects the prediction made using the
_predict_ method of _LogisticRegressionModel_ defined in
_spark/python/pyspark/mllib/classification.py_ file. Is there any other
occurence of
this issue in another file as well???
I can see two solutions in order to solve this issue:
a) Either check if the dot product between coeffs and data attributes gives a
value in the desired range [ -745, 709 ] and if not to just set it to the one
of these two values.
b) To create specific math functions in Java such as Logistic Function, SoftMax,
etc.. and call them via py4j in python2.7 and store the result in a
'decimal.Decimal' variable.
Thanks,
Michael
> Stabilize the computation of logistic function in pyspark
> ---------------------------------------------------------
>
> Key: SPARK-2552
> URL: https://issues.apache.org/jira/browse/SPARK-2552
> Project: Spark
> Issue Type: Bug
> Components: MLlib, PySpark
> Reporter: Xiangrui Meng
> Labels: Starter
>
> exp(1000) throws an error in python. For logistic function, we can use either
> 1 / ( 1 + exp( -x ) ) or 1 - 1 / (1 + exp( x ) ) to compute its value which
> ensuring exp always takes a negative value.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
