On Mon, 2011-04-04 at 16:19 +0530, Anurag Priyam wrote: > On Mon, Apr 4, 2011 at 3:17 PM, Anurag Priyam <[email protected]> > wrote: > [...] > > Mapping Queries to Shards > > ------------------------- > > > > Use a vBucket[1][2] style partitioning scheme coupled with libhashkit[3] > > for a > > pluggable hashing solution. > > One of my initial ideas here was to have a pluggable partitioning > solution. Libmemcached sort of does that. You can choose from a simple > modulo, ketama, weighted ketama, and vbuckets. > > The most flexible (read, that gives you most control) partitioning > scheme you can have is a range based/forwarding table partitioning. > vBuckets almost parallels that. The only reason, that I can think of, > why anyone would use ketama (or other consistent hashing scheme) is > that it is simpler; needs less app side thinking. So, I thought, maybe > we should have a pluggable partitioning scheme, with vbuckets as the > default. > > After reading up, and thinking more about vbuckets, I am unable to > justify a pluggable partitioning scheme at the cost of overhead > involved in maintaining more code. Tools (or, API) to handle > addition/removal of nodes, re-partitioning of data, shard groups will > have to handle all the cases. Seems a little difficult to make them > all pluggable in accordance with the partitioning scheme. > > Comments ?
I think at least we should have the option of vbuckets or ketama/weighted ketama (which was the scheme I was dreaming of when I thought of how I would do it originally). Whether that be in a plugin or option, I am undecided and am flexible on it. Kind Regards -- Andrew Hutchings - LinuxJedi - http://www.linuxjedi.co.uk/ _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
