-------- Original Message --------
Subject: RE: Jackknife tests in CVAGen7a
Date: Thu, 15 Sep 2011 12:25:00 -0400
From: Sheets, H David <[email protected]>
To: [email protected] <[email protected]>
Hi-
The jackknife groupings test in CVAGen7a is a leave-one-out cross
validation. Each specimen in turn is selected as the "test" specimen
and the CVA is fitted to the remaining n-1 specimens. The CV axes
derived from the n-1 specimens in the "learning set" is then used to
assign the test specimen to one of the groups in the analysis. This is
done for all specimens in the set to arrive at a classification table.
This cross-validation procedure is an estimate of how well the CV might
do on a newly collected specimen from the same population. Note that if
the original collection is not an unbiased sample, this will not be a
good estimate.
The Jackknife assignment test is more complex, it attempts to assign a
measure of the strength of the assignment of specimens to groups, ie a
p-value of the assignment, assuming the specimen was a member of the
group. This is an attempt to get at a measure of the "quality" of the
assignment. A CVA might assign a specimen to a group reliably,
although that specimen might not actually be typically of the group, it
is simply closest to that group, and distinct from the other groups in
the study. Please see the complete description in:
Nolte, A.W. and Sheets, H.D. (2005) Shape based assignment tests suggest
transgressive phenotypes in natural sculpin hybrids (Teleostei,
Scorpaeniformes, Cottidae). Frontiers in Zoology 2:11
doi:10.1186/1742-9994-2-11
Available at: http://www.frontiersinzoology.com/content/2/1/11.
For most purpose, the Jack-knife groupings test is probably adequate,
and is simply in operation to the cross-validation procedures seen in
other CVA or discriminant function procedures in software.
The "resubstitution rate" of correct assignments obtained by fitting the
CVA to the data and then classifying the same specimens the CVA was
fitted to can be wildly optimistic as to the performance of the CVA.
If you have a relatively robust analysis, the jack-knife (or cross
validation) rate will be similar to the resubstitution rate.
-Dave
H. David Sheets, PhD
Professor
Dept. of Physics
Canisius College
2001 Main St
Buffalo, NY 14208
-----Original Message-----
From: morphmet [mailto:[email protected]]
Sent: Thursday, September 15, 2011 10:01 AM
To: morphmet
Subject: Jackknife tests in CVAGen7a
-------- Original Message --------
Subject: Jackknife tests in CVAGen7a
Date: Wed, 14 Sep 2011 15:54:46 -0400
From: David Thulman <[email protected]>
To: [email protected]
Hello,
What is the difference in the jackknife tests in CVAGen7a in the
Statistics pull-down menu: Jack-knife Groupings, and Jackknife
Assignment Test? When is it appropriate to use either (or both)? I am
using the CVA to test to effectiveness of group identification for 3
groups of bilaterally symmetric artifacts.
Thanks,
Dave Thulman
Department of Anthropology
George Washington University
