File it for 0.3 ?
Robin On Fri, Feb 19, 2010 at 1:56 AM, Jake Mannix <jake.man...@gmail.com> wrote: > On Thu, Feb 18, 2010 at 11:55 AM, Robin Anil <robin.a...@gmail.com> wrote: > > > I was trying out SeqAccessSparseVector on Canopy Clustering using > Manhattan > > distance. I found performance to be really bad. So I profiled it with > > Yourkit(Thanks a lot for providing us free license) > > > > Since i was trying out manhattan distance, there were a lot of A-B which > > created a lot of clone operation 5% of the total time > > there were also so many A+B for adding a point to the canopy to average. > > this was also creating a lot of clone operations. 90% of the total time > > > > SequentialAccessSparseVector should only be used in a read-only fashion. > If > you are creating an average centroid which is sparse, but it is mutating, > then it should be RandomAccessSparseVector. The points which are being > used > to create it can be SequentialAccessSparseVector (if they themselves never > change), but then the method called should be > SequentialAccessSparseVector.addTo(RandomAccessSparseVector) - this > exploits > the fast sequential iteration of SeqAcc, and the fast random-access > mutatability of RandAcc. > > > > > > So we definitely needs to improve that.. > > > > For a small hack. I made the cluster centers RandomAccess Vector. Things > > are fast again. I dont know whether to commit or not. But something to > look > > into in 0.4? > > > > Yeah, cluster *centers* should indeed be RandomAccess. JIRA / patch so we > can see exactly what the change is? > > -jake >
