Actually, I'm working with dog fMRI data, and I have a constant issue with normalization/registration. So you might be right.
Thank you for the input. Regards Raúl On Tue, Nov 27, 2018 at 2:31 PM Roberto Guidotti <[email protected]> wrote: > I think that it could be a registration problem or a > normalization/detrending problem. > I used to have below-chance accuracy in across-subject analyses, but this > is not your case! > > Bests, > R > > On Mon, 26 Nov 2018 at 16:29, Raúl Hernández <[email protected]> wrote: > >> Thank you for the link, I will look into it carefully. >> Sorry for not being clear, yes I have 4 acquisitions from each >> participant. I calculate an accuracy for each participant by calculating >> the mean across all cross validation folds. Then I take the this calculated >> mean from each participant and run a t test in which each participant >> contributes with a single number. >> >> Regards, >> >> Raul >> >> On Mon, Nov 26, 2018 at 4:13 PM Etzel, Jo <[email protected]> wrote: >> >>> I agree with Patil that consistent below-chance accuracy is a sign that >>> something is not working properly. >>> >>> I collected some thoughts in >>> >>> http://mvpa.blogspot.com/2013/04/below-chance-classification-accuracy.html >>> (and a few other posts tagged "below-chance"). >>> >>> Also, be careful with terminology; I assume by "leave-one-run-out >>> cross-validation on 4 acquisitions" you mean each person completed four >>> scanning runs (each with the same fMRI acquisition parameters)? And a >>> t-test can be fine for a quick significance test, but it should be done >>> at the group level, testing if the subjects' accuracies are above chance >>> (i.e., each person contributing one number to the t-test), not on the >>> cross-validation folds within each person. >>> >>> Jo >>> >>> >>> On 11/26/2018 7:05 AM, Raúl Hernández wrote: >>> > I also consider that option, but when I try the very same thing with a >>> > different, region (not related to the task). I get accuracies of 50%. >>> So >>> > that makes me think that it is affected by the task, but I don't know >>> > what to think of it. >>> > >>> > Regards >>> > >>> > On Mon, Nov 26, 2018 at 1:34 PM Kaustubh Patil < >>> [email protected] >>> > <mailto:[email protected]>> wrote: >>> > >>> > I suspect that there might be something wrong in the code/how the >>> > data is handled. >>> > >>> > If you 30% accuracy then that would mean that you will get 70% if >>> > you use a simple rule to predict the "other class" after your >>> > classifier. This is a sign that something is not right in data >>> > handling/evaluation. >>> > >>> > Best >>> > >>> > On Mon, Nov 26, 2018 at 1:27 PM Raúl Hernández < >>> [email protected] >>> > <mailto:[email protected]>> wrote: >>> > >>> > No, it is balanced. It has the same number of observations for >>> > each class. >>> > >>> > On Mon, Nov 26, 2018 at 12:52 PM Kaustubh Patil >>> > <[email protected] <mailto:[email protected]>> >>> wrote: >>> > >>> > Just for clarification. >>> > >>> > Is that data imbalanced, i.e. many more observations from >>> > one class? >>> > >>> > Best, >>> > Kaustubh >>> > >>> > On Mon, Nov 26, 2018 at 12:50 PM Raúl Hernández >>> > <[email protected] <mailto:[email protected]>> >>> wrote: >>> > >>> > Dear PyMVPA community, >>> > >>> > I'm doing classification in ROI's, I'm performing a >>> > simple 2 way classification using LSVM, and a >>> > leave-one-run-out cross-validation on 4 acquisitions. >>> On >>> > some ROI's, I get a good accuracy for the number of >>> > participants (60%), but in others I get consistently >>> bad >>> > accuracy (30%). To test whether the performance is >>> above >>> > chance, I use a one sample t test (I know that it is >>> not >>> > the best test for this type of data, I just use it as >>> > quick overview). When I test the bad accuracies, those >>> > are also significant. >>> > >>> > What does it mean a consistently bad accuracy? >>> > >>> > Regards, >>> > >>> > Raul >>> > _______________________________________________ >>> > Pkg-ExpPsy-PyMVPA mailing list >>> > [email protected] >>> > <mailto:[email protected]> >>> > >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >>> > >>> > _______________________________________________ >>> > Pkg-ExpPsy-PyMVPA mailing list >>> > [email protected] >>> > <mailto:[email protected]> >>> > >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >>> > >>> > _______________________________________________ >>> > Pkg-ExpPsy-PyMVPA mailing list >>> > [email protected] >>> > <mailto:[email protected]> >>> > >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >>> > >>> > _______________________________________________ >>> > Pkg-ExpPsy-PyMVPA mailing list >>> > [email protected] >>> > <mailto:[email protected]> >>> > >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >>> > >>> > >>> > _______________________________________________ >>> > Pkg-ExpPsy-PyMVPA mailing list >>> > [email protected] >>> > >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >>> > >>> _______________________________________________ >>> Pkg-ExpPsy-PyMVPA mailing list >>> [email protected] >>> >>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa >> >> _______________________________________________ >> Pkg-ExpPsy-PyMVPA mailing list >> [email protected] >> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa > > > > -- > Ing. Roberto Guidotti, PhD. > PostDoc Fellow > Institute for Advanced Biomedical Technologies - ITAB > Department of Neuroscience and Imaging > University of Chieti "G. D'Annunzio" > Via dei Vestini, 33 > 66013 Chieti, Italy > tel: +39 0871 3556919 > e-mail: [email protected]; [email protected] > linkedin: http://it.linkedin.com/in/robertogui/ > twitter: @robbisg > github: https://github.com/robbisg > _______________________________________________ > Pkg-ExpPsy-PyMVPA mailing list > [email protected] > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
