Thank you very much! It works like a charm. Best regards, Wouter De Baene
On 20 Mar 2014, at 13:18, Michael Hanke <[email protected]<mailto:[email protected]>> wrote: Hey, On Wed, Mar 19, 2014 at 10:44 PM, Wouter De Baene <[email protected]<mailto:[email protected]>> wrote: Hi all, I’m running my first pymvpa-analysis after attending a very nice pymvpa workshop in Delmenhorst. Glad you liked it! Although there are a few related questions in the archive, I don’t seem to get this working. I’m setting up a searchlight analysis with Nfold cross-validation. I'm trying to look at the confusion matrix for each cross-validation for each searchlight. This information seems to be lost when using searchlights. As I understood there were some plans to aggregate the output per searchlight into the output of the total results. Is there any news about this? OR could I get to this information in another way for now? The easiest way to aggregate this kind of non-scalar information within a searchlight is to get the confusion matrix to come out of the CrossValidation as a vector (flattened matrix). For the first part of a possible solution we have "Confusion", a node that turns raw targets predictions into a confusion table. Here is a sketch: no error computing in the classifier, return raw predictions, use Confusion as a post-processing node to yield matrices. CrossValidation(some_classifier, some_partitioner, errorfx=None, postproc=Confusion()) Now the last bit is to flatten the confusion matrix into a vector. You could write a simple Node implementation that does that. Here is some pseudo code: class MyFlatMatrix(Node): def _call(ds): return Dataset(ds.samples.ravel()) and now use this class as a post-processing node for Confusion itself. CrossValidation(some_classifier, some_partitioner, errorfx=None, postproc=Confusion(postproc=MyFlatMatrix())) stick this into a searchlight and it should harvest what you want. This is all written as a sketch and wasn't tested... Instead of a custom Node class you could also use a ChainMapper comprised of a FlattenMapper and a TransposeMapper to achieve the same thing.... Good luck! Michael -- Michael Hanke http://mih.voxindeserto.de<http://mih.voxindeserto.de/> _______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected]<mailto:[email protected]> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
