Howdy,
I am having a problem with Riak crashing (note, this not in production)
with "maximum number of file descriptors exhausted'. The problem is, I
have my file descriptors bumped pretty high. I am on a Mac with 10.9.2,
with 16 GB of memory.
launchctl limit
cpu unlimited unlimited
filesize unlimited unlimited
data unlimited unlimited
stack 8388608 67104768
core 0 unlimited
rss unlimited unlimited
memlock unlimited unlimited
maxproc 709 1064
maxfiles 65536 65536
There are only 372 files in the data bitcask directory:
bryan@yetitron:/opt/riak-1.4.2$ ls data/bitcask/*/* | wc -l
372
Here is the message repeated many times from the error.log
2014-03-01 10:38:55.080 [error] <0.15839.0> gen_server memsup terminated
with reason: maximum number of file descriptors exhausted, check ulimit -n
2014-03-01 10:38:55.080 [error] <0.15839.0> CRASH REPORT Process memsup
with 0 neighbours exited with reason: maximum number of file descriptors
exhausted, check ulimit -n in gen_server:terminate/6 line 747
2014-03-01 10:38:55.080 [error] <0.86.0> Supervisor os_mon_sup had child
memsup started with memsup:start_link() at <0.15839.0> exit with reason
maximum number of file descriptors exhausted, check ulimit -n in context
child_terminated
Here is the crash.log:
2014-03-01 10:38:55 =ERROR REPORT====
** Generic server memsup terminating
** Last message in was
{'EXIT',<0.15840.0>,{emfile,[{erlang,open_port,[{spawn,"/bin/sh -s
unix:cmd
2>&1"},[stream]],[]},{os,start_port_srv_handle,1,[{file,"os.erl"},{line,254}]},{os,start_port_srv_loop,0,[{file,"os.erl"},{line,270}]}]}}
** When Server state ==
{state,{unix,darwin},false,undefined,undefined,false,60000,30000,0.8,0.05,<0.15840.0>,#Ref<0.0.0.137055>,undefined,[reg],[]}
** Reason for termination ==
** {emfile,[{erlang,open_port,[{spawn,"/bin/sh -s unix:cmd
2>&1"},[stream]],[]},{os,start_port_srv_handle,1,[{file,"os.erl"},{line,254}]},{os,start_port_srv_loop,0,[{file,"os.erl"},{line,270}]}]}
2014-03-01 10:38:55 =CRASH REPORT====
crasher:
initial call: memsup:init/1
pid: <0.15839.0>
registered_name: memsup
exception exit: {{emfile,[{erlang,open_port,[{spawn,"/bin/sh -s
unix:cmd
2>&1"},[stream]],[]},{os,start_port_srv_handle,1,[{file,"os.erl"},{line,254}]},{os,start_port_srv_loop,0,[{file,"os.erl"},{line,270}]}]},[{gen_server,terminate,6,[{file,"gen_server.erl"},{line,747}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
ancestors: [os_mon_sup,<0.85.0>]
messages: []
links: [<0.86.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 377
stack_size: 24
reductions: 216
neighbours:
2014-03-01 10:38:55 =SUPERVISOR REPORT====
Supervisor: {local,os_mon_sup}
Context: child_terminated
Reason: {emfile,[{erlang,open_port,[{spawn,"/bin/sh -s unix:cmd
2>&1"},[stream]],[]},{os,start_port_srv_handle,1,[{file,"os.erl"},{line,254}]},{os,start_port_srv_loop,0,[{file,"os.erl"},{line,270}]}]}
Offender:
[{pid,<0.15839.0>},{name,memsup},{mfargs,{memsup,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]
2014-03-01 10:38:55 =SUPERVISOR REPORT====
Supervisor: {local,os_mon_sup}
Context: shutdown
Reason: reached_max_restart_intensity
Offender:
[{pid,<0.15839.0>},{name,memsup},{mfargs,{memsup,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]
2014-03-01 10:38:57 =SUPERVISOR REPORT====
Supervisor: {local,os_mon_sup}
Context: shutdown_error
Reason: killed
Offender:
[{pid,<0.92.0>},{name,cpu_sup},{mfargs,{cpu_sup,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]
Cheers,
Bryan
--
Bryan Hughes
*Go Factory*
http://www.go-factory.net
/"Internet Class, Enterprise Grade"/
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com