Re: [ceph-users] Radosgw (civetweb) hangs once around 850 established connections

Hi Sean, Did you check that the process isn't hitting some ulimits? cat /proc/`pidof radosgw`/limits and compare with the num processes/num FDs in use. Cheers, Dan On Tue, Mar 29, 2016 at 8:35 PM, seapasu...@uchicago.edu wrote: > So an update for anyone else having

Re: [ceph-users] Radosgw (civetweb) hangs once around 850 established connections

So an update for anyone else having this issue. It looks like radosgw either has a memory leak or it spools the whole object into ram or something. root@kh11-9:/etc/apt/sources.list.d# free -m total used free sharedbuffers cached Mem: 64397 63775

Re: [ceph-users] Radosgw (civetweb) hangs once around 850 established connections

Hi Ben! I'm using ubuntu 14.04 I have restarted the gateways with the numthreads line you suggested. I hope this helps. I would think I would get some kind of throttle log or something. 500 seems really strange as well. Do you have a thread for this? RGW still has a weird race condition

Re: [ceph-users] Radosgw (civetweb) hangs once around 850 established connections

What OS are you using? I have a lot more open connections than that. (though i have some other issues, where rgw sometimes returns 500 errors, it doesn't stop like yours) You might try tuning civetweb's num_threads and 'rgw num rados handles': rgw frontends = civetweb num_threads=125

[ceph-users] Radosgw (civetweb) hangs once around 850 established connections

I have a cluster of around 630 OSDs with 3 dedicated monitors and 2 dedicated gateways. The entire cluster is running hammer (0.94.5 (9764da52395923e0b32908d83a9f7304401fee43)). (Both of my gateways have stopped responding to curl right now. root@host:~# timeout 5 curl localhost ; echo $? 124