On Thu, 28 Jan 2010, Matthias Ekman wrote: > > how does your cv.states.null_dist.dist_samples would look?
> looks pretty 'okay' with mean about .5 -- so I guess everything is fine. gaussianish? ;) > mmh... but still wondering why p-values are so low for RFE (compared to > feature selection with fixed number of elements). as you have mentioned, it does perform better, so you can expect lower p values. If you look at the distribution of random performances with so many samples you should get it quite narrow, so tails would be very light and probability of getting 0.8 just by chance -- reeealy small, as it should. as for how much RFE could influence generalization - see for instance figure 1 of http://psychology.rutgers.edu/~jose/hanson_halchenko_inpress.pdf you can see how some subjects went from random (on full brain data) to low error (or even to 0). So, do not undervalue the importance/effect of feature selection and congrats on obtaining good results ;) -- .-. =------------------------------ /v\ ----------------------------= Keep in touch // \\ (yoh@|www.)onerussian.com Yaroslav Halchenko /( )\ ICQ#: 60653192 Linux User ^^-^^ [175555] _______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
