[
https://issues.apache.org/jira/browse/MAHOUT-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414679#comment-13414679
]
Jeff Eastman commented on MAHOUT-1045:
--------------------------------------
I also recall wondering about the density calculation when I wrote it. The
intra-cluster density uses a similar normalization in the book and, well, I
really cannot defend either of them. Here it is just for conversation:
{code}
public double interClusterDensity() {
pruneInvalidClusters();
double max = 0;
double min = Double.MAX_VALUE;
double sum = 0;
int count = 0;
for (int i = 0; i < clusters.size(); i++) {
Cluster clusterI = clusters.get(i);
for (int j = i + 1; j < clusters.size(); j++) {
Cluster clusterJ = clusters.get(j);
double d = measure.distance(clusterI.getCenter(), clusterJ.getCenter());
min = Math.min(d, min);
max = Math.max(d, max);
sum += d;
count++;
}
}
double density = (sum / count - min) / (max - min);
log.info("Inter-Cluster Density = {}", density);
return density;
}
{code}
> Cluster evaluators returning bad results
> ----------------------------------------
>
> Key: MAHOUT-1045
> URL: https://issues.apache.org/jira/browse/MAHOUT-1045
> Project: Mahout
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 0.6, 0.7, 0.8
> Environment: Several environments and data sets
> Reporter: Pat Ferrel
> Fix For: 0.8
>
>
> With real world crawl data the Intra-cluster density from ClusterEvaluator is
> almost always NaN. The CDbw inter-cluster density is almost always 0. I have
> also seen several cases where CDbw fails to return any results but have not
> tracked down why yet.
> I have sent a link to an 8G data set that reproduces these errors to Jeff
> Eastman.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
