On Thu, Sep 23, 2010 at 7:05 AM, Michael Segel <michael_se...@hotmail.com> wrote: > > > 4000 xcievers is a lot. > > I'm wondering if there's a correlation between the number of xcievers and > ulimit -n. Should they be configured on a 1 to 1 ratio? >
2:1 ratio of file descriptors to xceivers. 4000 xceivers is quite normal on a heavily loaded HBase cluster in my experience. The cost is the RAM of the extra threads, but there's not much you can do about it, given the current design of the datanode. -Todd > -Mike > >> Date: Thu, 23 Sep 2010 08:04:40 -0400 >> Subject: Xcievers Load >> From: marnan...@gmail.com >> To: u...@hbase.apache.org; common-user@hadoop.apache.org >> >> Hi, >> We have a job that writes many small files (using MultipleOutputFormat) >> and its exceeding the 4000 xcievers that we have configured. What is the >> effect on the cluster of increasing this count to some higher number? >> Many thanks, >> Martin >> >> PD: Hbase is also running on the cluster. > -- Todd Lipcon Software Engineer, Cloudera
