[ https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133483#comment-16133483 ]
Joseph K. Bradley commented on SPARK-21770: ------------------------------------------- I vaguely recall discussing this before but forget where that discussion was. Overall, I'd vote for the uniform distribution: * The "probability" column has a clear meaning: It should provide a predicted probability distribution over class labels. An all-0 vector is not a valid probability distribution. * It does not really make sense to say all classes are impossible. When fitting a statistical model to predict from n classes, one makes the implicit assumption that there exist "true" classes to be predicted. However, I can see the argument for not changing current behavior (from a software engineering standpoint). That said, if people are relying on this behavior, their application logic is probably incorrect from a statistical modeling perspective. Any opinions [~sethah], [~yanboliang], [~dbtsai] ? > ProbabilisticClassificationModel: Improve normalization of all-zero raw > predictions > ----------------------------------------------------------------------------------- > > Key: SPARK-21770 > URL: https://issues.apache.org/jira/browse/SPARK-21770 > Project: Spark > Issue Type: Improvement > Components: ML > Affects Versions: 2.3.0 > Reporter: Siddharth Murching > Priority: Minor > > Given an n-element raw prediction vector of all-zeros, > ProbabilisticClassifierModel.normalizeToProbabilitiesInPlace() should output > a probability vector of all-equal 1/n entries -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org