Hi all,
Tying in to another discussion, could it be beneficial to first average
every 5 trials or so? In a way this reduces noise, so the performance
would most likely go up - as might the informativeness of feature
sensitivity. The downside is that you no longer have predictions on a
trial by trial basis.
Cheers,
Marius
On 14-01-17 03:57 AM, Nick Oosterhof wrote:
On Jan 16, 2014, at 10:22 PM, Marius 't Hart wrote:
On 14-01-14 12:48 PM, Nick Oosterhof wrote:
On Jan 12, 2014, at 9:40 PM, Marius 't Hart wrote:
My toy problem is to classify the two most extreme conditions (two targets)
based on averaged Cz and Pz activity [...]
Did you provide an equal number of samples for each class (target)? Because if
you didn't then 60% could, in principle, be due to chance. That is, if 60% of
the samples are in one class in the training set, then a 'naive' classifier
that is just guessing that class all the time will give 60% accuracy.
Yes, after manual artefact rejection the number of trials in each condition is
different. I take the first N trials from each condition, with N being the
number of trials in the condition with the smallest number of trials. This
number is different for each participant.
That's fine then - it means that an accuracy of 60% would be meaningful, if it
is consistent above 50% across participants.
Also: what preprocessing did you do? Any z-scoring, baseline correction etc?
I do baseline correction, but no Z-scoring. Should I do Z-scoring? If so, over
all data, within electrode or within trial?
There are no *absolute* rules for this, and even then there are different ways
to do z-scoring.
In fMRI world there are at least two ways, both are voxel-wise
1) take all data to compute mean and std.
2) take data from baseline periods to compute mean and std.
Translating this to MEEG, you could try a sensor-wise z-scoring. It may be best
to use the pre-stimulus period to compute the mean and std parameters, so that
noisy channels will be scaled less and thus contribute less to classification.
However I don't know whether this as easily done in PyMVPA for MEEG data as in
fMRI data, as I would assume that your features consist of combinations of
time-points and channels, whereas in fMRI data the features are just voxels.
In the CNV plot it looked like the usefulness of Pz and Cz for the
classifier(s) should flip at around 1 second in the preparation interval, so I
wanted to look at sensitivity. [...] That doesn't look like what I expected -
but I find it hard to judge if what I'm doing is actually correct. For example,
on inspecting all the different sensitivity datasets, it looks like the sign
that each feature gets is usually the same... but there are a few exceptions.
Does the sign actually mean anything in terms of a feature's usefulness?
As far as I know the sign is not that important - it's more about the relative
magnitude. If the sensitivity is further away from zero then that means the
feature is more important for discriminating the classes.
OK, so basically, although it looks like there is difference in the usefulness
of the electrodes for classifying the conditions, the classifiers don't reflect
that. Would it make sense to try different classifiers, instead of Linear SVM?
You could try and see if you get consistent results using other classifiers,
but generally linear SVM is a good classifier to start with. For now I would
stay away from non-linear stuff as its results are more difficult to interpret,
and the low number of trials in a typical experiment means it can be prone to
overfitting.
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
