A separate but related issue: Does anyone know what could cause a negative shift in the searchlight accuracy distribution for a 2-category Linear SVM classifier of single subject? I'm seeing this in a couple of my subjects following a switch to using zscore(dataset, chunks_attr='chunks', dtype='float32') …. so zscoring against the mean and not against a baseline condition. (I'm actually zscoring after removing the rest trials from the timeseries.) While the vast majority of my subjects show a normal looking curve with a peak right at 50%, a couple of subjects, while having a normal-esque curve, show accuracy peaks at 40-45%.
Generally speaking, I'm seeing group data accuracy peaks in more-or-less our hypothesized areas of the brain, but I'm still a bit worried that I've done something wrong somewhere along the line. I can't figure out why I'm seeing the negative peak (in a couple of subjects) or why using zscore against rest has such a detrimental effect on my accuracies… Thanks and best, Mike On Fri, Dec 16, 2011 at 11:53 PM, Mike E. Klein <michaelekl...@gmail.com>wrote: > Thanks for the response! > > I just took a really quick first look (just a single 2-way comparison for > a single subject): > > Looking between my experimental conditions, it looks like setting C=-5 > and/or using *zscore(dataset, chunks_attr='chunks', dtype='float32') *leads > to accuracies that are a bit higher than before, but still worse than > without any zscoring at all. (I hope that I've done the zscoring against > the full time series correctly...) > > Looking at sounds vs. silence: > > *With old z-scoring, C=-5:* > Accuracies are around 60% with a heavy selection bias towards the rest > condition (chooses "rest" correctly 27/27 times, but also chooses rest for > 21/27 sound conditions). > > *With old z-scoring, C=-1:* > Same as above, except with accuracies around 54% > > *With NO z-scoring, C=-5 or C=-1:* > Accuracies are 98-100% > > *With C=-5 (or C=-1) and zscore(dataset, chunks_attr='chunks', > dtype='float32'):* > Accuracies are about 98% > > > So it looks like: > > (a) Using C=-5 (as opposed to C=-1) helps a little with the zscore against > rest method. Although it might help across the board, but there's a ceiling > effect with the other combinations. > (b) There's a huge difference between whether I zscore against rest or > with the whole time series. I'm not sure what's up... running sounds > > silence GLMs in FSL show obvious responses in the expected brain regions. > > > Thanks again, > Mike > > > > > > On Fri, Dec 16, 2011 at 11:14 PM, Yaroslav Halchenko < > deb...@onerussian.com> wrote: > >> before discussion kicks in -- out of curiosity... what happens if you >> either do nested cross-validation to choose C parameter or just set it >> a bit higher (e.g. C=-5 to still be scaled according to the data), what >> if you do zscoring across full time series (not just baseline condition) >> -- for both of those there are explanations, I just wondered to get >> better idea of what might be happening in your case >> >> >> On Fri, 16 Dec 2011, Mike E. Klein wrote: >> > where my baseline condition is silence. Without zscoring, SVMs can >> tell >> > any of the sound conditions vs. the silence condition at 98-100% >> > accuracy...which makes sense. With zscoring, this drops to the 80-90% >> > level. The experiment has a good amount of samples, is well-balanced, >> > motion-corrected, etc., so I can't find other obvious confounds.)� >> -- >> =------------------------------------------------------------------= >> Keep in touch www.onerussian.com >> Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic >> >> _______________________________________________ >> Pkg-ExpPsy-PyMVPA mailing list >> Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org >> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa > > >
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa