Hi Edgar, I'm pleased to hear you've resolved the issue. https://github.com/basho/riak_kv/issues/666 will track adding some validation code to protect against similar incidents.
Jon On Fri, Sep 20, 2013 at 8:59 AM, Edgar Veiga <edgarmve...@gmail.com> wrote: > Problem solved. > > The n_val = "3" caused the crash! I had a window of time while starting a > node to send a new PUT command and restore the correct value. > > Best regards, thanks Jon > > > On 20 September 2013 15:42, Edgar Veiga <edgarmve...@gmail.com> wrote: > >> Yes I did, via CURL command: >> >> curl -v -XPUT http://dat7:8098/riak/visitors -H "Content-Type: >> application/json" -d '{"props":{"n_val":"3"}}' >> >> How can I update the n_val of an already existing bucket, while the >> instance is down? >> >> >> On 20 September 2013 15:32, Jon Meredith <jmered...@basho.com> wrote: >> >>> Looks like the nval is set to a binary <<"3">> rather than an integer. >>> Have you changed it recently and how? >>> >>> On Sep 20, 2013, at 8:25 AM, Edgar Veiga <edgarmve...@gmail.com> wrote: >>> >>> Hello everyone, >>> >>> Please lend me a hand here... I'm running a riak cluster of 6 machines >>> (version 1.4.1). >>> >>> Suddenly all the nodes in the cluster went down and they are refusing to >>> go up again. It keeps crashing all the the time, this is just a sample of >>> what I get when starting a node: >>> >>> 2013-09-20 15:16:14.016 [info] <0.7.0> Application lager started on node >>> 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.024 [info] <0.7.0> Application sasl started on node ' >>> riak@192.168.20.107' >>> 2013-09-20 15:16:14.026 [info] <0.7.0> Application crypto started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.026 [info] <0.7.0> Application public_key started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.029 [info] <0.7.0> Application ssl started on node ' >>> riak@192.168.20.107' >>> 2013-09-20 15:16:14.031 [info] <0.7.0> Application riak_sysmon started >>> on node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.036 [info] <0.7.0> Application os_mon started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.044 [info] <0.7.0> Application runtime_tools started >>> on node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.047 [info] <0.7.0> Application erlang_js started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.058 [info] <0.7.0> Application inets started on node >>> 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.059 [info] <0.7.0> Application mochiweb started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.063 [info] <0.7.0> Application webmachine started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.064 [info] <0.7.0> Application basho_stats started >>> on node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.067 [info] <0.7.0> Application bitcask started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.152 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_core,vnode_routing} = proxy >>> 2013-09-20 15:16:14.160 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_core,staged_joins} = true >>> 2013-09-20 15:16:14.166 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_core,resizable_ring} = true >>> 2013-09-20 15:16:14.172 [info] <0.7.0> Application riak_core started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:14.177 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_pipe,trace_format} = ordsets >>> 2013-09-20 15:16:14.270 [info] <0.522.0>@riak_kv_env:doc_env:42 >>> Environment and OS variables: >>> 2013-09-20 15:16:14.446 [notice] <0.41.0>@lager_app:119 Deprecated >>> lager_file_backend config detected, please consider updating it >>> 2013-09-20 15:16:14.786 [info] <0.573.0>@riak_core:wait_for_service:470 >>> Waiting for service riak_kv to start (0 seconds) >>> 2013-09-20 15:16:14.821 [info] <0.590.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.590.0>) >>> 2013-09-20 15:16:14.823 [info] <0.591.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.591.0>) >>> 2013-09-20 15:16:14.825 [info] <0.592.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.592.0>) >>> 2013-09-20 15:16:14.826 [info] <0.593.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.593.0>) >>> 2013-09-20 15:16:14.827 [info] <0.594.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.594.0>) >>> 2013-09-20 15:16:14.829 [info] <0.595.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.595.0>) >>> 2013-09-20 15:16:14.831 [info] <0.596.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.596.0>) >>> 2013-09-20 15:16:14.833 [info] <0.597.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_map) >>> host starting (<0.597.0>) >>> 2013-09-20 15:16:14.837 [info] <0.600.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.600.0>) >>> 2013-09-20 15:16:14.841 [info] <0.601.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.601.0>) >>> 2013-09-20 15:16:14.845 [info] <0.602.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.602.0>) >>> 2013-09-20 15:16:14.848 [info] <0.603.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.603.0>) >>> 2013-09-20 15:16:14.850 [info] <0.605.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.605.0>) >>> 2013-09-20 15:16:14.852 [info] <0.606.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: >>> riak_kv_js_reduce) host starting (<0.606.0>) >>> 2013-09-20 15:16:14.853 [info] <0.608.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_hook) >>> host starting (<0.608.0>) >>> 2013-09-20 15:16:14.855 [info] <0.609.0>@riak_kv_js_vm:init:76 >>> Spidermonkey VM (thread stack: 16MB, max heap: 8MB, pool: riak_kv_js_hook) >>> host starting (<0.609.0>) >>> 2013-09-20 15:16:14.855 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,vnode_vclocks} = true >>> 2013-09-20 15:16:14.867 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,legacy_keylisting} = false >>> 2013-09-20 15:16:14.873 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,listkeys_backpressure} = true >>> 2013-09-20 15:16:14.884 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,index_backpressure} = true >>> 2013-09-20 15:16:14.889 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,mapred_system} = pipe >>> 2013-09-20 15:16:14.900 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,mapred_2i_pipe} = true >>> 2013-09-20 15:16:14.905 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,anti_entropy} = enabled_v1 >>> 2013-09-20 15:16:14.911 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,handoff_data_encoding} = encode_raw >>> 2013-09-20 15:16:14.917 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,object_format} = v1 >>> 2013-09-20 15:16:14.927 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,secondary_index_version} = v2 >>> 2013-09-20 15:16:14.943 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,vclock_data_encoding} = encode_zlib >>> 2013-09-20 15:16:14.953 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_kv,crdt} = [pncounter] >>> 2013-09-20 15:16:15.011 [warning] <0.6.0> lager_error_logger_h dropped >>> 17 messages in the last second that exceeded the limit of 100 messages/sec >>> 2013-09-20 15:16:15.012 [info] >>> <0.162.0>@riak_core_capability:process_capability_changes:530 New >>> capability: {riak_control,member_info_version} = v1 >>> 2013-09-20 15:16:15.022 [info] <0.7.0> Application riak_control started >>> on node 'riak@192.168.20.107' >>> 2013-09-20 15:16:15.022 [info] <0.7.0> Application erlydtl started on >>> node 'riak@192.168.20.107' >>> 2013-09-20 15:16:16.013 [error] <0.1209.0> gen_fsm <0.1209.0> in state >>> started terminated with reason: no function clause matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,...]],...},...]}, >>> undefined) line 851 >>> 2013-09-20 15:16:16.013 [error] <0.1209.0> CRASH REPORT Process >>> <0.1209.0> with 0 neighbours exited with reason: no function clause >>> matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,...]],...},...]}, >>> undefined) line 851 in gen_fsm:terminate/7 line 589 >>> 2013-09-20 15:16:16.014 [error] <0.140.0> Supervisor riak_core_vnode_sup >>> had child undefined started with {riak_core_vnode,start_link,undefined} at >>> <0.1209.0> exit with reason no function clause matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,...]],...},...]}, >>> undefined) line 851 in context child_terminated >>> 2013-09-20 15:16:16.014 [error] <0.739.0> CRASH REPORT Process >>> proxy_riak_kv_vnode_376793390874373408599387495934666716005045108736 with 0 >>> neighbours exited with reason: >>> {{{function_clause,[{riak_kv_vnode,terminate,[{badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,365375409332725729550921208179070754913983135744,359666418561901890026688064301272774368452149248,353957427791078050502454920423474793822921162752,348248437020254210978221776545676813277390176256,342539446249430371453988632667878832731859189760,336830455478606531929755488790080852186328203264,331121464707782692405522344912282871640797216768,...]],...},...]},...],...},...]},...},...} >>> in gen_server:call/3 line 188 >>> 2013-09-20 15:16:16.014 [error] <0.146.0> Supervisor >>> riak_core_vnode_proxy_sup had child >>> {riak_kv_vnode,376793390874373408599387495934666716005045108736} started >>> with riak_core_vnode_proxy:start_link(riak_kv_vnode, >>> 376793390874373408599387495934666716005045108736) at <0.739.0> exit with >>> reason >>> {{{function_clause,[{riak_kv_vnode,terminate,[{badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,365375409332725729550921208179070754913983135744,359666418561901890026688064301272774368452149248,353957427791078050502454920423474793822921162752,348248437020254210978221776545676813277390176256,342539446249430371453988632667878832731859189760,336830455478606531929755488790080852186328203264,331121464707782692405522344912282871640797216768,...]],...},...]},...],...},...]},...},...} >>> in context child_terminated >>> 2013-09-20 15:16:16.015 [error] <0.138.0> Supervisor riak_core_sup had >>> child riak_core_vnode_manager started with >>> riak_core_vnode_manager:start_link() at <0.161.0> exit with reason >>> {{function_clause,[{riak_kv_vnode,terminate,[{badarg,[{lists,split,[<<"3">>,[376793390874373408599387495934666716005045108736,371084400103549569075154352056868735459514122240,365375409332725729550921208179070754913983135744,359666418561901890026688064301272774368452149248,353957427791078050502454920423474793822921162752,348248437020254210978221776545676813277390176256,342539446249430371453988632667878832731859189760,336830455478606531929755488790080852186328203264,331121464707782692405522344912282871640797216768,...]],...},...]},...],...},...]},...} >>> in context child_terminated >>> 2013-09-20 15:16:24.147 [error] <0.2034.0> gen_fsm <0.2034.0> in state >>> started terminated with reason: no function clause matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[445301280124259482890185222468242482551416946688,439592289353435643365952078590444502005885960192,...]],...},...]}, >>> undefined) line 851 >>> 2013-09-20 15:16:24.147 [error] <0.2034.0> CRASH REPORT Process >>> <0.2034.0> with 0 neighbours exited with reason: no function clause >>> matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[445301280124259482890185222468242482551416946688,439592289353435643365952078590444502005885960192,...]],...},...]}, >>> undefined) line 851 in gen_fsm:terminate/7 line 589 >>> 2013-09-20 15:16:24.147 [error] <0.2019.0> gen_fsm <0.2019.0> in state >>> started terminated with reason: no function clause matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[34253944624943037145398863266787883273185918976,28544953854119197621165719388989902727654932480,...]],...},...]}, >>> undefined) line 851 >>> 2013-09-20 15:16:24.147 [error] <0.2019.0> CRASH REPORT Process >>> <0.2019.0> with 0 neighbours exited with reason: no function clause >>> matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[34253944624943037145398863266787883273185918976,28544953854119197621165719388989902727654932480,...]],...},...]}, >>> undefined) line 851 in gen_fsm:terminate/7 line 589 >>> 2013-09-20 15:16:24.153 [error] <0.2029.0> gen_fsm <0.2029.0> in state >>> started terminated with reason: no function clause matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[308285501624487334308589769401090949458673270784,302576510853663494784356625523292968913142284288,...]],...},...]}, >>> undefined) line 851 >>> 2013-09-20 15:16:24.153 [error] <0.2029.0> CRASH REPORT Process >>> <0.2029.0> with 0 neighbours exited with reason: no function clause >>> matching >>> riak_kv_vnode:terminate({badarg,[{lists,split,[<<"3">>,[308285501624487334308589769401090949458673270784,302576510853663494784356625523292968913142284288,...]],...},...]}, >>> undefined) line 851 in gen_fsm:terminate/7 line 589 >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> > -- Jon Meredith VP, Engineering Basho Technologies, Inc. jmered...@basho.com
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
