P.S. I failed to include the link to the wiki page that discusses each of Basho's optimizations for the Riak environment:
https://github.com/basho/leveldb/wiki On Jan 27, 2014, at 8:49 PM, Matthew Von-Maszewski <matth...@basho.com> wrote: > Basho's leveldb requirements have lead to different optimizations. Facebook > has a captive hardware environment and usage case that does not match ours. > I am not saying their changes are better or worse, only different. > > Basho needs: > > - multiple databases running simultaneously: 6 to 64 > - support for slow solitary SATA drives all the way to rocking' fast FusionIO > PCI arrays > - fast disaster recovery / repair > - aggressive delete logic to help remove large chunks of deleted data now > instead of next year > - (and a few things still not unleashed in Riak 2.0) > > So our compaction strategy has come a long way: from Google's original > single thread to a thread pool that can run simultaneous companions on the > same database as well as other databases. We have a dedicated thread that > constantly computes the speed of the local array and gradually manipulates > each write request to prevent Google's designed stall … maximizing total > throughput for the simultaneous databases while maintaining a constant user > experience. And in between all that, we have retuned the skip list, the > bloom filter, and integrated Intel's hardware CRC assembly code. > > That is what meets Riak's needs. > > Matthew > > > On Jan 27, 2014, at 6:05 PM, Alexander Sicular <sicul...@gmail.com> wrote: > >> Hey Basho gang, >> >> Have you all taken any sort of look at FB's leveldb fork, rocksdb, >> http://rocksdb.org/? Just watching Dhruba's video it seems like it may be a >> useful upgrade, >> https://code.facebook.com/videos/174548089418739/rocksdb-a-high-performance-embedded-key-value-store-for-flash-storage-data-scale/. >> The main talking points seems to be that rocksdb upgrades leveldb such that >> it is more efficient when storing data more than system ram, has more >> efficient access patternsto take advantage of ssd's and has a better >> compaction story. >> >> Care to share your thoughts? >> >> Thanks, >> >> -Alexander Sicular >> >> @siculars >> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
