Hi there riak users I've been running some tests on riak-cs on a 3 node riak cluster ( 3 servers , each host riak and riak-cs pointing at the local riak . One of the instance is also hosting the stanchion ) Software versions : - Riak is 1.4.1 - Riak-CS is 1.4.0 - s3cmd is 1.5.0 alpha 3
n_val was either set to be 2 or 3 ( tried both )for default bucket , yet
whenever I kill a single riak node ( simulate a server outage )
I come across the following error when try to use s3cmd get path:
*
*
*s3cmd ls s3://riak.test/test/1*
*
*
*WARNING: Retrying failed request: /?prefix=test/1&delimiter=/ ()*
*WARNING: Waiting 3 sec...*
*WARNING: Retrying failed request: /?prefix=test/1&delimiter=/ ()*
*WARNING: Waiting 6 sec...*
*
*
Looking into Riak-CS error logs , I find these errors
*
*
*
2013-08-20 23:39:40.240 [error] <0.14674.0> gen_fsm <0.14674.0> in state
waiting_object_list terminated with reason:
<<"{error,insufficient_vnodes_available}">>
2013-08-20 23:39:40.240 [error] <0.14674.0> CRASH REPORT Process
<0.14674.0> with 1 neighbours exited with reason:
<<"{error,insufficient_vnodes_available}">> in gen_fsm:terminate/7 line 611
This of course implies of missing v_nodes found on the single dead riak
node... :/
*
Writing to riak using s3cmd keeps working flawlessly . And of course after
I revive the dead node
misplaced data is re-distributed ( ring of vnodes are redistributed to
reflect requested distribution ).
I might be missing something here , but ain't the whole point of
replication is to avoid such issues with a single node failure ? ( assuming
n_val is greater the 1 )
Isn't a 3 Node Cluster with n_val =>2 supposed to withstand a single node
failure ?
Have I missed something in configuring Riak-CS ?
Any help will be appreciated , this is really the final missing piece of
our RiAK puzzle :)
Regards,
Idan Shinberg
System Architect
Idomoo Ltd.
Mob +972.54.562.2072
email idan.shinb...@idomoo.com
web www.idomoo.com
[image: Description: cid:FC66336E-E750-4C2D-B6E3-985D5A06B5BE@idomoo.co.il]
<<image001.jpg>>
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
