I've been running riak for about a month and I'm using it as a
persistent cache. Everything has been fine until tonight that is where
it appears to have had a catastrophic failure. Basically when I make a
request using the http interface I get a failure. I cannot be more
specific than that as there is no response code back; I think it's
getting a TCP reset ...
... That was about 10 minutes ago. When I tried to find out if I was
getting a TCP reset I restarted th program and everything was fine.
Anyway most of the errors look like this:
=ERROR REPORT==== 9-Aug-2010::13:23:27 ===
{mochiweb_socket_server,256,{acceptor_error,{error,accept_failed}}}
=ERROR REPORT==== 9-Aug-2010::13:23:27 ===
application: mochiweb
"Accept failed error"
"{error,emfile}"
And every so often I get:
=ERROR REPORT==== 9-Aug-2010::13:23:56 ===
** State machine <0.17333.0> terminating
** Last event in was {riak_vnode_req_v1,
182687704666362864775460604089535377456991567872,
{fsm,undefined,<0.17335.0>},
{riak_kv_put_req_v1,
{<<"uris">>,
<<"5ea5dc023dd73b5f711efde3d50b9e91da4c5027">>},
{r_object,<<"uris">>,
<<"5ea5dc023dd73b5f711efde3d50b9e91da4c5027">>,
[{r_content,
{dict,5,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],
[[<<"Links">>]],
[],[],[],[],[],[],[],
[[<<"content-type">>,97,112,112,108,105,99,97,
116,105,111,110,47,106,115,111,110],
[<<"X-Riak-VTag">>,49,102,84,99,107,121,67,100,
81,67,83,85,102,67,102,118,68,55,80,50,82,118]],
[],[],
[[<<"X-Riak-Last-Modified">>|
{1281,360236,222890}]],
[],
[[<<"X-Riak-Meta">>]]}}},
<<"{\"uri\":\"http://feedproxy.google.com/~r/time/politics/~3/SGIQlFYkIy8/0,8599,2006898,00.html\",\"download_date\":\"20100809132314\"}";>>}],
[{<<2,235,89,160>>,{1,63448579436}}],
{dict,1,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
[[clean|true]],
[]}}},
undefined},
118194806,63448579436,
[{returnbody,true}]}}
** When State == active
** Data == {state,182687704666362864775460604089535377456991567872,
riak_kv_vnode,
{state,182687704666362864775460604089535377456991567872,
riak_kv_bitcask_backend,
{#Ref<0.0.0.23192>,
"/var/lib/riak/bitcask/182687704666362864775460604089535377456991567872"},
[],false},
undefined,none}
** Reason for termination =
** {bad_return_value,{error,{write_locked,emfile}}}
I've put a full log here:
http://stuff.roughage.com.au/riak-failure.log.gz
It's nearly midnight here and I going to bed but if anyone can shed any
light on this that'd cool.
rgh
--
+61 (0) 410 646 369
[e]: [email protected]
[im]: [email protected]
You're worried criminals will continue to penetrate into cyberspace, and
I'm worried complexity, poor design and mismanagement will be there to meet
them - Marcus Ranum
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
