On Mon, Apr 4, 2011 at 3:17 PM, Anurag Priyam <[email protected]> wrote: [...] > Mapping Queries to Shards > ------------------------- > > Use a vBucket[1][2] style partitioning scheme coupled with libhashkit[3] for a > pluggable hashing solution.
One of my initial ideas here was to have a pluggable partitioning solution. Libmemcached sort of does that. You can choose from a simple modulo, ketama, weighted ketama, and vbuckets. The most flexible (read, that gives you most control) partitioning scheme you can have is a range based/forwarding table partitioning. vBuckets almost parallels that. The only reason, that I can think of, why anyone would use ketama (or other consistent hashing scheme) is that it is simpler; needs less app side thinking. So, I thought, maybe we should have a pluggable partitioning scheme, with vbuckets as the default. After reading up, and thinking more about vbuckets, I am unable to justify a pluggable partitioning scheme at the cost of overhead involved in maintaining more code. Tools (or, API) to handle addition/removal of nodes, re-partitioning of data, shard groups will have to handle all the cases. Seems a little difficult to make them all pluggable in accordance with the partitioning scheme. Comments ? [...] P.S: Sorry for a lot of TODOs, I just copy pasted from my notes :|. -- Anurag Priyam http://about.me/yeban/ _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
