[
https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503217#comment-16503217
]
Teng Peng commented on SPARK-24431:
-----------------------------------
[~Ben2018] The article makes sense to me. It seems the current behavior follows
"Case 2: TP is not 0", but it set precision = 1 if recall = 0. (See
[https://github.com/apache/spark/blob/734ed7a7b397578f16549070f350215bde369b3c/mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala#L110]
)
Have you checked out SPARK-21806 and its discussion on JIRA?
Let's hear [~srowen] 's opinions.
> wrong areaUnderPR calculation in BinaryClassificationEvaluator
> ---------------------------------------------------------------
>
> Key: SPARK-24431
> URL: https://issues.apache.org/jira/browse/SPARK-24431
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 2.2.0
> Reporter: Xinyong Tian
> Priority: Major
>
> My problem, I am using CrossValidator(estimator=LogisticRegression(...), ...,
> evaluator=BinaryClassificationEvaluator(metricName='areaUnderPR')) to
> select best model. when the regParam in logistict regression is very high, no
> variable will be selected (no model), ie every row 's prediction is same ,eg.
> equal event rate (baseline frequency). But at this point,
> BinaryClassificationEvaluator set the areaUnderPR highest. As a result best
> model seleted is a no model.
> the reason is following. at time of no model, precision recall curve will be
> only two points: at recall =0, precision should be set to zero , while the
> software set it to 1. at recall=1, precision is the event rate. As a result,
> the areaUnderPR will be close 0.5 (my even rate is very low), which is
> maximum .
> the solution is to set precision =0 when recall =0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
