To inject a slightly different topic: while I and many others have done
t-tests on accuracies (equivalently, error rates) for group-level
analyses and gotten sensible-looking results, this is probably not
ideal: accuracies are bounded by 0 and 1, which violates the assumptions
of t-tests (and similar statistics).
Logit mixed models may be a better parametric test for the group level.
I've found some readable introductions to these issues in the
linguistics literature, such as this one:
http://dx.doi.org/10.1016%2Fj.jml.2007.11.007
Categorical Data Analysis: Away from ANOVAs (transformation or not) and
towards Logit Mixed Models
T. Florian Jaeger J Mem Lang. 2008 Nov; 59(4): 434–446.
I haven't yet started playing with these for MVPA datasets; anyone tried?
I don't want to imply that logit mixed models (or t-tests) are better
than permutation methods; permutation-based tests are almost certainly
preferable, particularly with cross-validated statistics. However, it is
sometimes useful to have a quick parametric statistic as well; but this
should perhaps not be a t-test.
Jo
On 5/9/2015 11:49 AM, Jingwen Jin wrote:
Hi MVPA experts,
I have a general question about conducting group-level analysis on the
subjects' classification accuracy maps. Let's say I am doing a
one-sample t-test to find the voxels that have high classification
accuracy across subjects. Essentially, I am doing a t-test on percentage
numbers (SVM classification accuracy measured as percentage correct).
Since percentage is highly affected by the testing example numbers, and
in general would probably not meet the normal distribution assumption
for t-test.
So my question is if people adjust for testing trial numbers or any sort
of transformation? For example, I converted each voxel's percentage
number to a z score at the individual subject's classification map
level, and then do group-level t-test on these z score maps. I wonder if
this is valid?
Thank you very much!
Best,
Frances
--
Jingwen Frances Jin
Department of Psychology, PhD candidate in Clinical area
Stony Brook University
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
