> When you say Riak is reading data continuously, are you referring to the AAE > trees being rebuilt? I just want to confirm that you did remove the > anti_entropy directory, as not removing it can cause additional repair > operations to occur.
I did remove the anti_entropy directories on all (6) nodes. I assumed it must be the AAE trees being rebuilt, usually the cluster is very quiet during US night time. I stopped riak, upgrade riak, rm anti_entropy, start riak on the nodes (one by one) and after that on all the nodes the riak process was reading >40Mb/s continuously. The physical nodes each have 2TB+ of data holding largish values (2-5Mb each) and a separate disk with the metadata only a few GB. > AAE can be enabled / disabled via riak attach with > > riak_kv_entropy_manager:enable(). > > and > > riak_kv_entropy_manager:disable(). Thanks. I have disable the entropy manager on one of the nodes but after a few minutes it’s still reading data fast. Nothing funny showing up in the log files, other then the occasional [ riak_kv_vnode:do_get_object:1299 Reading large object of size 5076278 from ] for values that are larger then 500000 bytes. Actually the entropy manager seems to be VERY quiet regarding logging… riak-admin awe-status does indicate rebuilding going on: ================================== Exchanges ================================== Index Last (ago) All (ago) ------------------------------------------------------------------------------- 205523667749658222872393179600727299639115513856 -- -- 319703483166135013357056057156686910549735243776 -- -- 433883298582611803841718934712646521460354973696 -- -- 662242929415565384811044689824565743281594433536 -- -- 776422744832042175295707567380525354192214163456 -- -- 890602560248518965780370444936484965102833893376 -- -- 1004782375664995756265033322492444576013453623296 -- -- 1118962191081472546749696200048404186924073353216 -- -- 1233142006497949337234359077604363797834693083136 -- -- 1347321821914426127719021955160323408745312813056 -- -- ================================ Entropy Trees ================================ Index Built (ago) ------------------------------------------------------------------------------- 205523667749658222872393179600727299639115513856 -- 319703483166135013357056057156686910549735243776 6.7 hr 433883298582611803841718934712646521460354973696 -- 662242929415565384811044689824565743281594433536 32.9 min 776422744832042175295707567380525354192214163456 1.9 hr 890602560248518965780370444936484965102833893376 -- 1004782375664995756265033322492444576013453623296 -- 1118962191081472546749696200048404186924073353216 4.7 hr 1233142006497949337234359077604363797834693083136 4.4 hr 1347321821914426127719021955160323408745312813056 -- ================================ Keys Repaired ================================ Index Last Mean Max ——————————————————————————————————————— Kind regards, Timo
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
