I'm missing something here: where are the "class 1" and "class 2"
numbers for each person coming from?
I agree that describing the results of a multiclass classification is
tricky, especially when using something naturally two-class like svms.
Reporting the multiclass accuracy rate, then the pairwise ones seems
most prudent.
Jo
On 5/24/2011 2:09 AM, Vadim Axel wrote:
Attached an output of one such pathological case (completely real data).
I even do not talk about splits. It's sufficient that for half subjects
you have 0.8/0.4 prediction and for other half you have 0.4/0.8...
For more than two classes it really becomes hardly maintainable. I
recently had 5 classes experiment and I ended up reporting each classes
separately.
On Mon, May 23, 2011 at 11:01 PM, Yaroslav Halchenko
<[email protected] <mailto:[email protected]>> wrote:
it is a good thesis indeed, especially for the case of multiclass
classification where people make claims about unraveling complex
categorical structure, whenever it is only few categories which get
"significantly" well classified.
And your illustration goes even further than your verbal description --
at first I thought that there is an error, since I expected at least one
class to be significant when "average" accuracy becomes significant.
But indeed it might be not the case, e.g. if a classifier favors one
class over another across splits, thus none of the classes come out with
a consistently "significant" performance while mean accuracy does (could
you check if that is indeed the case by looking on per split
diagonals?). Cool. I always thought that digging in the mud is
very entertaining ;)
On Mon, 23 May 2011, Vadim Axel wrote:
> Attached an illustration for my thesis.
> The average classification rate can be considered significant,
while we
> clearly see that it is not exactly true...
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
