Hmm, I'm trying to follow the same pattern as the tuturial at http://www.pymvpa.org/tutorial_significance.html:
>>> distr_est = MCNullDist(repeater, tail='left', ... measure=null_cv, ... enable_ca=['dist_samples']) Can you give me a hint as to what I'd need to change? The cross validation folds should be leave-two-out (one from each subject group), hence the partitioner and sifter. Then, we should do the same analysis PERM_COUNT times, permuting the labels of each training set for each fold. Here, it looks like dist_est is starting with the same dataset PERM_COUNT times, then building its folds by applying the partitioner, then the sifter, then a single iteration of the permutator to each resulting fold. This sounds like what I want, but isn't coming out that way. Ben On Aug 31, 2014, at 10:08 AM, Nick Oosterhof <[email protected]> wrote: > On Aug 31, 2014, at 4:24 PM, Ben Acland <[email protected]> wrote: > >> Can anyone see a mistake in the following code that would cause all of my >> permutation tests to return the same mean error value? >> >> # cross-validated linear svm with permutation testing (leave out 1 from each >> sub group) >> rep = mp.Repeater(count=PERM_COUNT) > > [...] > >> distr_est = mp.MCNullDist(rep, >> tail='left', >> measure=null_cv, >> enable_ca=['dist_samples']) > > [...] > >> ... and at the end of all of this, cvmcc.null_dist.ca.dist_samples contains >> PERM_COUNT repetitions of a single value. > > You are feeding your Repeater as the permutator argument. It would seem that > the MCNullDist has no access to unpermuted datasets. > _______________________________________________ > Pkg-ExpPsy-PyMVPA mailing list > [email protected] > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
