@ Dr. Halchenko
I already tried to do what you suggest: that is, set up CrossValidation with
BayesConfusionHypothesis, as you show below, and use that as the data measure
within a sphere_searchlight. However, I do not get the results on a per-feature
level. It's possible that it's there, and I cannot figure out how to access it,
but a look at res.sa.hypothesis doesn't give me results per feature, which is
what I'm after.
@ Susanne
I'm working in 2.2.0, inside neurodebian.
Our experiment is focused around object properties. We have participants make
judgments on how similar two concepts are along a given dimension. We have
broken those dimensions into abstract (thematic context, function) and
perceptual/concrete features (color/shape). What I've done so far is look at
regions that we expect to be able to support a 4-way classification, and we've
been quite successful. However, we're interested in finding regions that might
be able to encode abstract versus concrete, without being able to separate
between two abstract or concrete features - that is [[0, 1], [2, 3]] versus
[[0], [1], [2], [3]]. We have already identified some such regions within some
a priori ROIs, but while we have the data, we'd like to get a better idea about
how this plays out throughout the brain, which is where this searchlight/bayes
system comes in.
As I mentioned above, when I plug a cross validation w/ the
BayesConfusionHypothesis node properly set up into a searchlight, it doesn't
return that nice bayesian hypothesis test result for each feature, which is
what I'm after.
Thank you both for getting back to me so quickly!
- Gavin
On Nov 25, 2013, at 2:15 PM, Yaroslav Halchenko wrote:
>
> On Mon, 25 Nov 2013, Hanson, Gavin Keith wrote:
>
>> Hi:
>> I am reasonably new to PyMVPA, but have not had any trouble with it so far.
>> However, I am interested in running an analysis similar to that performed in
>> Connolly, 2012, where dissimilarity matrices are computed per searchlight. I
>> would like to know if anyone knows of a straightforward way to do with
>> within PyMVPA - that is, get the simple confusion matrix per searchlight.
>> Even better, I would like to be able to look at the results of the
>> BayesConfusionHypothesis node at a per-searchlight level, as our hypothesis
>> has to do with how the specificity of information encoding changes across
>> the brain, and that tool seems perfect for that. Results with ROIs have been
>> promising, but I'd like to see how this plays out across the entire brain,
>> and the ability to use the searchlight tool in conjunction with the Bayes
>> Confusion node would be very helpful. If anyone could help me with this, I'd
>> much appreciate it!
>
> From your description it sounds you are looking for this recipe present
> in our unittests:
>
> mvpa2/tests/test_transerror.py
>
> 602 def test_confusion_as_node():
> 603 from mvpa2.misc.data_generators import normal_feature_dataset
> 604 from mvpa2.clfs.gnb import GNB
> 605 from mvpa2.clfs.transerror import Confusion
> 606 ds = normal_feature_dataset(snr=2.0, perlabel=42, nchunks=3,
> 607 nonbogus_features=[0,1], nfeatures=2)
> 608 clf = GNB()
> 609 cv = CrossValidation(
> 610 clf, NFoldPartitioner(),
> 611 errorfx=None,
> 612 postproc=Confusion(labels=ds.UT),
> 613 enable_ca=['stats'])
> 614 res = cv(ds)
> 615 # needs to be identical to CA
> 616 assert_array_equal(res.samples, cv.ca.stats.matrix)
> 617 assert_array_equal(res.sa.predictions, ds.UT)
> 618 assert_array_equal(res.fa.targets, ds.UT)
>
> 619 skip_if_no_external('scipy')
>
> 620 from mvpa2.clfs.transerror import BayesConfusionHypothesis
> 621 from mvpa2.base.node import ChainNode
> 622 # same again, but this time with Bayesian hypothesis testing at
> the end
> 623 cv = CrossValidation(
> 624 clf, NFoldPartitioner(),
> 625 errorfx=None,
> 626 postproc=ChainNode((Confusion(labels=ds.UT),
> 627 BayesConfusionHypothesis())))
> 628 res = cv(ds)
> 629 # only two possible hypothesis with two classes
> 630 assert_equals(len(res), 2)
> 631 # the first hypothesis is the can't discriminate anything
> 632 assert_equal(len(res.sa.hypothesis[0]), 1)
> 633 assert_equal(len(res.sa.hypothesis[0][0]), 2)
> 634 # and the hypothesis is actually less likely than the other one
> 635 # (both classes can be distinguished)
> 636 assert(np.e**res.samples[0,0] < np.e**res.samples[1,0])
> ....
>
> so by using such a CrossValidation construct as input measure for your
> Searchlight construct should achieve what you aim, or am I wrong?
>
> --
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate, Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
> WWW: http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> [email protected]
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
