Over the past few months I've been working on an alternative pure-Erlang Key/Value store to act as a backend to Riak KV. This is now open source and available at
https://github.com/martinsumner/leveled The store is a work-in-progress prototype, originally started to better understand the impact of different trade-offs in LSM-Tree design. The aim is to: - provide a fully-featured Riak backend (e.g. secondary index, object expiry support etc) - provide stable throughput with larger object sizes (> 4KB) - provide a simpler and more flexible path to making Riak KV changes end-to-end The prime change in the store when compared to HanoiDB or eleveldb is that storage is split with only Keys & Metadata being placed in the merge tree, and the full object living to the side in a series of CDB-based journals. The intention of this is to: - reduce write amplification and ease page-cache pollution issues on scanning events. - support faster HEAD requests than GET requests, and in parallel an alternative Riak KV branch has been produced to move from an n-GET model to a n-HEAD 1-GET model of fetching data for both KV GET and KV PUT operations The impact of this has been to improve throughput for larger object sizes where disk I/O and not CPU is the current limit on throughput. The advantage increases the greater the object size, and the tighter the constraint on disk. Please visit the github page, I've tried to write up as much about the project as I can. There's the results of various volume tests, information on the research which prompted the design, an overview of the design itself and some hints as to what I expect to try next with leveled. Any feedback, please mail me, raise an issue on github, or ping me @masleeds Cheers Martin
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
