Attached an illustration for my thesis. The average classification rate can be considered significant, while we clearly see that it is not exactly true...
On Mon, May 23, 2011 at 1:26 PM, Vadim Axel <axel.va...@gmail.com> wrote: > Hi, > > One more issue, which I had never seen to be mentioned in the papers: > Two classes classification rate is the confusion matrix diagonal average. > Things are fine as soon as both values are above 0.5. But what if one of > them is let's say 0.7 and the other one is 0.45? All significance procedures > (permutation, group t-test whatever) work on average values. Thus, my > obscure average mean would be treated as a first class citizen and > contribute to beyond chance prediction. Clearly, one have to check the > results before averaging, but they are never reported (at I least I have > never seen). For example, I can require that p-values for both classes > should be significant. However, then, I am afraid, I may get a chart with > two ROIs with similar average prediction rate when one is highly significant > and another one is completely not. Nobody would understand what is going > on... > > Any suggestions / thoughts? > > Thanks, > Vadim > > > > >
<<attachment: class_rate.JPG>>
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
