Another useful measure to compare partitions is the adjusted Rand index which is implemented in the library(e1071) within the classAgreement function. If you have your data partitions to be compared in a matricial form (where each column is a different partition), the syntax is ARI<-classAgreement(table(data[,i],data[,j]))$crand
Other useful measures of goodness-of-fit for clustering are the silhouette index or the c-index or the Goodman-Kruskal index. although they evaluate in general inter/intra-cluster distance distributions. For instance, you can maximise/minimise these indices to find the best partition among a set of candidate ones. Mattia Prosperi. 2010/11/17 Marc Schwartz <marc_schwa...@me.com>: > On Nov 17, 2010, at 7:33 AM, Martin Tomko wrote: > >> Dear all, >> I am having a hard time to figure out a suitable test for the match between >> two nominal classifications of the same set of data. >> I have used hierarchical clustering with multiple methods (ward, >> k-means,...) to classify my dat into a set number of classesa, and I would >> like to compare the resulting automated classification with the actual - >> objective benchmark one. >> So in principle I have a data frame with n columns of nominal >> classifications, and I want to do a mutual comparison and test for >> significance in difference in classification between pairs of columns. >> >> I just need to identify a suitable test, but I fail. I am currently >> exploring the possibility of using Cohen's Kappa, but I am open to other >> suggestions. Especially the fact that kappa seems to be moslty used on >> failible, human annotators seems to bring in limitations taht do not apply >> to my automatic classification. >> Any help will be appreciated, especially if also followed by a pointer to an >> R package that implements it. >> >> Thanks >> Martin > > > In addition to Matt's comments, you might want to consider marginal > homogeneity tests. There are extensions of the pairwise McNemar test to > greater than two categories. Some online information is here: > > http://www.john-uebersax.com/stat/mcnemar.htm > > and there is the ?mh_test implemented in the 'coin' package on CRAN. > > HTH, > > Marc Schwartz > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
