[
https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109
]
Teng Peng edited comment on SPARK-24431 at 6/2/18 6:46 PM:
-----------------------------------------------------------
I am trying to understand this description. What's your definition of event
rate? Is it defined as (TP+FN)/(TP + FP + TN + FN).
Also, 0.5 is not the maximum of AreaunderPR, which could attain 1.0.
was (Author: teng peng):
I am trying to understand this description. What's your definition of event
rate? Is it defined as (TP+FN)/(TP + FP + TN + FN).
> wrong areaUnderPR calculation in BinaryClassificationEvaluator
> ---------------------------------------------------------------
>
> Key: SPARK-24431
> URL: https://issues.apache.org/jira/browse/SPARK-24431
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 2.2.0
> Reporter: Xinyong Tian
> Priority: Major
>
> My problem, I am using CrossValidator(estimator=LogisticRegression(...), ...,
> evaluator=BinaryClassificationEvaluator(metricName='areaUnderPR')) to
> select best model. when the regParam in logistict regression is very high, no
> variable will be selected (no model), ie every row 's prediction is same ,eg.
> equal event rate (baseline frequency). But at this point,
> BinaryClassificationEvaluator set the areaUnderPR highest. As a result best
> model seleted is a no model.
> the reason is following. at time of no model, precision recall curve will be
> only two points: at recall =0, precision should be set to zero , while the
> software set it to 1. at recall=1, precision is the event rate. As a result,
> the areaUnderPR will be close 0.5 (my even rate is very low), which is
> maximum .
> the solution is to set precision =0 when recall =0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
