[ https://issues.apache.org/jira/browse/ZOOKEEPER-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862303#action_12862303 ]
Ted Dunning commented on ZOOKEEPER-759: --------------------------------------- This is a unix specific bean so don't forget to defang the test if the bean isn't available. > Stop accepting connections when close to file descriptor limit > -------------------------------------------------------------- > > Key: ZOOKEEPER-759 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-759 > Project: Zookeeper > Issue Type: Improvement > Components: server > Reporter: Travis Crawford > > Zookeeper always tries to accept new connections, throwing an exception if > out of file descriptors. An improvement would be denying new client > connections when close to the limit. > Additionally, file-descriptor limits+usage should be exported to the > monitoring four-letter word, should that get implemented (see ZOOKEEPER-744). > DETAILS > A Zookeeper ensemble I administer recently suffered an outage when one node > was restarted with the low system-default ulimit of 1024 file descriptors and > later ran out. File descriptor usage+max are already being monitored by the > following MBeans: > - java.lang.OperatingSystem.MaxFileDescriptorCount > - java.lang.OperatingSystem.OpenFileDescriptorCount > They're described (rather tersely) at: > http://java.sun.com/javase/6/docs/jre/api/management/extension/com/sun/management/UnixOperatingSystemMXBean.html > This feature request is for the following: > (a) Stop accepting new connections when OpenFileDescriptorCount is close to > MaxFileDescriptorCount, defaulting to 95% FD usage. New connections should be > denied, logged to disk at debug level, and increment a > ``ConnectionDeniedCount`` MBean counter. > (b) Begin accepting new connections when usage drops below some configurable > threshold, defaulting to 90% of FD usage, basically the high/low watermark > model. > (c) Update the administrators guide with a comment about using an appropriate > FD limit. > (d) Extra credit: if ZOOKEEPER-744 is implemented export statistics for: > zookeeper_open_file_descriptor_count > zookeeper_max_file_descriptor_count > zookeeper_max_file_descriptor_mismatch - boolean, exported by leader, if not > all zk's have the same max FD value -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.