-------- Original Message --------
Subject: Re: Jackknife tests in CVAGen7a
Date: Thu, 15 Sep 2011 13:27:34 -0400
From: David Thulman <[email protected]>
To: [email protected]
Thanks. One clarification, if I may. For my data, the jackknife
groupings test is 90% accurate, while the jackknife assignments test is
84.6% correct and significant and 2.6% correct and non-significant.
Looking at the article you recommended, I should interpret the "2.6%
correct and non-significant" as an outlier that was nonetheless
correctly assigned?
Thanks again,
Dave
On Thu, Sep 15, 2011 at 1:07 PM, morphmet
<[email protected]
<mailto:[email protected]>> wrote:
-------- Original Message --------
Subject: RE: Jackknife tests in CVAGen7a
Date: Thu, 15 Sep 2011 12:25:00 -0400
From: Sheets, H David <[email protected]
<mailto:[email protected]>>
To: [email protected] <mailto:[email protected]>
<[email protected] <mailto:[email protected]>>
Hi-
The jackknife groupings test in CVAGen7a is a leave-one-out cross
validation. Each specimen in turn is selected as the "test"
specimen and the CVA is fitted to the remaining n-1 specimens. The
CV axes derived from the n-1 specimens in the "learning set" is then
used to assign the test specimen to one of the groups in the
analysis. This is done for all specimens in the set to arrive at a
classification table. This cross-validation procedure is an estimate
of how well the CV might do on a newly collected specimen from the
same population. Note that if the original collection is not an
unbiased sample, this will not be a good estimate.
The Jackknife assignment test is more complex, it attempts to assign
a measure of the strength of the assignment of specimens to groups,
ie a p-value of the assignment, assuming the specimen was a member
of the group. This is an attempt to get at a measure of the
"quality" of the assignment. A CVA might assign a specimen to a
group reliably, although that specimen might not actually be
typically of the group, it is simply closest to that group, and
distinct from the other groups in the study. Please see the
complete description in:
Nolte, A.W. and Sheets, H.D. (2005) Shape based assignment tests
suggest transgressive phenotypes in natural sculpin hybrids
(Teleostei, Scorpaeniformes, Cottidae). Frontiers in Zoology 2:11
doi:10.1186/1742-9994-2-11
Available at: http://www.frontiersinzoology.__com/content/2/1/11
<http://www.frontiersinzoology.com/content/2/1/11>.
For most purpose, the Jack-knife groupings test is probably
adequate, and is simply in operation to the cross-validation
procedures seen in other CVA or discriminant function procedures in
software.
The "resubstitution rate" of correct assignments obtained by fitting
the CVA to the data and then classifying the same specimens the CVA
was fitted to can be wildly optimistic as to the performance of the
CVA. If you have a relatively robust analysis, the jack-knife (or
cross validation) rate will be similar to the resubstitution rate.
-Dave
H. David Sheets, PhD
Professor
Dept. of Physics
Canisius College
2001 Main St
Buffalo, NY 14208
-----Original Message-----
From: morphmet [mailto:morphmet_moderator@__morphometrics.org
<mailto:[email protected]>]
Sent: Thursday, September 15, 2011 10:01 AM
To: morphmet
Subject: Jackknife tests in CVAGen7a
-------- Original Message --------
Subject: Jackknife tests in CVAGen7a
Date: Wed, 14 Sep 2011 15:54:46 -0400
From: David Thulman <[email protected] <mailto:[email protected]>>
To: [email protected] <mailto:[email protected]>
Hello,
What is the difference in the jackknife tests in CVAGen7a in the
Statistics pull-down menu: Jack-knife Groupings, and Jackknife
Assignment Test? When is it appropriate to use either (or both)? I am
using the CVA to test to effectiveness of group identification for 3
groups of bilaterally symmetric artifacts.
Thanks,
Dave Thulman
Department of Anthropology
George Washington University
