Some quick and dirty testing at the time showed that 160 entries per server gave a decent distribution.. The java client just implements what the C library does. I'm in the process of updating libketama to make this configurable and merging a patch that implements FNV hashing, which seems to be almost twice as fast as md5 (still need a java impl of FNV-1a) Patch i was sent is here: svn://svn.audioscrobbler.net/misc/ketama/patches/ I'll post some better benchmarks and updates once i'm done cleaning up libketama. RJ
On Monday 30 July 2007 01:11:26 Hanson Char wrote: > Hi, > > I refer to the ketama SockIOPool > (http://static.last.fm/ketama/ketama-0.1.1.tar.bz2) implementation for > the use of consistent hashing in the memcached java client. In the > initialize method, if the weight of each server is 1, the "factor" > would be 40 and therefore resulting in each server being added to the > TreeMap 160 times (with code extracted below.) > > Curious, why 40 ? Is it optimal ? Why not configurable ? > > Hanson Char > > double factor = > Math.floor(((double)(40*this.servers.length*thisWeight))/(double)this.total >Weight); > > for(long j = 0; j < factor; j++) { > byte[] d = > md5.digest((servers[i]+"-"+j).getBytes()); > for(int h=0;h<4;h++) { > Long k = > ((long)(d[3+h*4]&0xFF) << 24) > > | ((long)(d[2+h*4]&0xFF) << 16) > | ((long)(d[1+h*4]&0xFF) << 8) > | ((long)(d[0+h*4]&0xFF)); > > buckets.put(k, servers[i]); > log.debug( "++++ added " + servers[i] + > " to server bucket" ); > } > } -- Richard Jones Last.fm Ltd. | http://www.last.fm/ Office: +44 (0) 207 780 7080
