Roberta Nacif wrote:
>
> To validate the logistic regression results, the logistic regression
> classifiers were trained on 9 folds and tested on 1 fold. This was
> performed 10 times, each time using other training and testing folds.
> Hence, I obtained 10 p-values and coefficients for each input.
>
> With this procedure I averaged the several clasification matrix to
> obtain a single meausre (as recomended in Hair et al., p. 275).
> However, my problem is that I want to aggregate these 10 p-values and
> coefficients into one measure of input significance. Unlike the
I don't think there is a simple way to combine your p-values (I am
guessing those p-values are associated with each parameter).
The natural way to test different hypotheses in a cross-validation
framework would be to compare the cross-validated losses (classification
accuracy in your case). If you want to go a bit further, and assuming
you use the same folds for all hypotheses, you could pair the individual
losses (from each fold) and do some test on these. Note however that the
losses are not independant so you need to make some serious
approximation.
Also this may depend on what hypotheses your are testing.
Cyril.
--
Cyril Goutte
INRIA Rhone-Alpes
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
