[ https://issues.apache.org/jira/browse/MAHOUT-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762167#action_12762167 ]
Shashikant Kore commented on MAHOUT-165: ---------------------------------------- I am trying out this patch. Somehow, I find it extremely slow compared to colt. I am running KMeans on 100k test vectors. With colt, all 5 iterations and clustering finished in 8 minutes. With this patch, it's been an hour and it hasn't even completed 50% of Iteration 1. (Iteration 0 was completed in 3 minutes.) I checked kmeans.Cluster.java and verified that correct method on distance measure is called (one with 3 parameters). It is correct, and it can be verified by the quick completion of Iteration 0. I am not not able to understand this behaviour. > Using better primitives hash for sparse vector for performance gains > -------------------------------------------------------------------- > > Key: MAHOUT-165 > URL: https://issues.apache.org/jira/browse/MAHOUT-165 > Project: Mahout > Issue Type: Improvement > Components: Matrix > Affects Versions: 0.2 > Reporter: Shashikant Kore > Assignee: Grant Ingersoll > Fix For: 0.2 > > Attachments: colt.jar, mahout-165-trove.patch, > MAHOUT-165-updated.patch, mahout-165.patch, MAHOUT-165.patch, mahout-165.patch > > > In SparseVector, we need primitives hash map for index and values. The > present implementation of this hash map is not as efficient as some of the > other implementations in non-Apache projects. > In an experiment, I found that, for get/set operations, the primitive hash of > Colt performance an order of magnitude better than OrderedIntDoubleMapping. > For iteration it is 2x slower, though. > Using Colt in Sparsevector improved performance of canopy generation. For an > experimental dataset, the current implementation takes 50 minutes. Using > Colt, reduces this duration to 19-20 minutes. That's 60% reduction in the > delay. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.