Hello, I'm trying to diagnose why we're using so much swap memory for Riak+Search, and derive what settings should be changed such that we can lower our memory utilization. Obviously we can scale up our riak boxes, but I'd like to have better insight into the settings before we just upgrade the machines.
We have a 512 partition 5 node ring with LevelDb Backend, Search enabled. Currently we're seeing 100% of all available physical ram consumed by riak (1.7gb) and a large swap use as well: ~3.6gb: riak 2677 1.7 85.1 3665736 1745696 pts/0 Ssl+ Mar12 109:16 /usr/lib/riak/erts-5.8.4/bin/beam.smp -K true -A 64 -W w -- My my computations according to http://wiki.basho.com/LevelDB.html we should be at around ~850mb of ram for the levelDb cache. I calculate another ~128 for JS VMs, but I have no way to figure out where the rest is used. How do we figure out where the rest of the memory is being consumed, and how can we tune it to use less memory? Excerpt from config: %% Riak KV config {riak_kv, [ {storage_backend, riak_kv_eleveldb_backend}, {pb_ip, "<%= bindRiakAddress %>" }, {pb_port, 8087 }, ... {map_js_vm_count, 8 }, {reduce_js_vm_count, 6 }, {hook_js_vm_count, 2 }, {js_max_vm_mem, 8}, {js_thread_stack, 16}, %{js_source_dir, "/tmp/js_source"}, {http_url_encoding, on}, {riak_kv_stat, true}, {legacy_stats, true}, {vnode_vclocks, true}, {legacy_keylisting, false}, ... ]}, %% Riak Search Config {riak_search, [ {enabled, true} ]}, %% Merge Index Config {merge_index, [ {data_root, "/var/lib/riak/merge_index"}, {data_root_2i, "/var/lib/riak/merge_index_2i"}, {buffer_rollover_size, 1048576}, {max_compact_segments, 20} ]}, %% Bitcask Config {bitcask, [ {data_root, "/var/lib/riak/bitcask"} ]}, %% eLevelDB Config {eleveldb, [ {data_root, "/var/lib/riak/leveldb"} ]}, --Ken clipboard, inc. we're hiring!: www.clipboard.com/jobs
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
