Re: Confusion matrix for binary classification

[CD Athuraliya]

Hi Burak,

Thanks for the response. I am using Spark version 1.3.0 through Java API.

Regards,
CD

On Tue, Jun 23, 2015 at 5:11 AM, Burak Yavuz  wrote:

> Hi,
>
> In Spark 1.4, you may use DataFrame.stat.crosstab to generate the
> confusion matrix. This would be very simple if you are using the ML
> Pipelines Api, and are working with DataFrames.
>
> Best,
> Burak
>
> On Mon, Jun 22, 2015 at 4:21 AM, CD Athuraliya 
> wrote:
>
>> Hi,
>>
>> I am looking for a way to get confusion matrix for binary classification.
>> I was able to get confusion matrix for multiclass classification using this
>> [1]. But I could not find a proper way to get confusion matrix in similar
>> class available for binary classification [2]. Later I found this class [3]
>> which corresponds to my requirement but I am not sure about the way I
>> should use that class to get evaluation metrics for binary classification.
>> e.g. Given the constructor BinaryConfusionMatrixImpl(BinaryLabelCounter
>> count, BinaryLabelCounter totalCount), from where I can get this count and
>> totalCount? Appreciate any help on this.
>>
>> [1]
>> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/MulticlassMetrics.html#confusionMatrix()
>> [2]
>> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.html
>> [3]
>> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrixImpl.html
>>
>> Thanks
>>
>> --
>> *CD Athuraliya*
>> Software Engineer
>> WSO2, Inc.
>> Mobile: +94 716288847 <94716288847>
>> LinkedIn  | Twitter
>>  | Blog
>> 
>>
>
>


-- 
*CD Athuraliya*
Software Engineer
WSO2, Inc.
Mobile: +94 716288847 <94716288847>
LinkedIn  | Twitter
 | Blog 

Re: Confusion matrix for binary classification

[Burak Yavuz]

Hi,

In Spark 1.4, you may use DataFrame.stat.crosstab to generate the confusion
matrix. This would be very simple if you are using the ML Pipelines Api,
and are working with DataFrames.

Best,
Burak

On Mon, Jun 22, 2015 at 4:21 AM, CD Athuraliya 
wrote:

> Hi,
>
> I am looking for a way to get confusion matrix for binary classification.
> I was able to get confusion matrix for multiclass classification using this
> [1]. But I could not find a proper way to get confusion matrix in similar
> class available for binary classification [2]. Later I found this class [3]
> which corresponds to my requirement but I am not sure about the way I
> should use that class to get evaluation metrics for binary classification.
> e.g. Given the constructor BinaryConfusionMatrixImpl(BinaryLabelCounter
> count, BinaryLabelCounter totalCount), from where I can get this count and
> totalCount? Appreciate any help on this.
>
> [1]
> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/MulticlassMetrics.html#confusionMatrix()
> [2]
> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.html
> [3]
> http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrixImpl.html
>
> Thanks
>
> --
> *CD Athuraliya*
> Software Engineer
> WSO2, Inc.
> Mobile: +94 716288847 <94716288847>
> LinkedIn  | Twitter
>  | Blog
> 
>