I agree. The tests should not need more than 32k open files.
----- Original Message ----- From: Todd Lipcon <[email protected]> To: [email protected] Cc: Sent: Saturday, November 26, 2011 10:42 AM Subject: Re: max number of open files when testing HBase on the Jenkins server I disagree that we should raise the limit. If we're using up 32k descriptors in unit tests, it's probably a real leak we should be concerned about. Maybe we can add a temporary patch upstream which does: catch (IOException ioe) { if (ioe.getMessage().contains("Too many open files")) { Runtime.getRuntime().exec("lsof -u " + System.getProperty("user.name")); } } or something like that? -Todd On Thu, Nov 24, 2011 at 6:27 PM, Ted Yu <[email protected]> wrote: > Mikhail: > If you look at > https://builds.apache.org/job/PreCommit-HBASE-Build/364/console, you would > see: > > max memory size (kbytes, -m) unlimited > open files (-n) 32768 > > It is not clear why 32768 was not enough for TestAdmin. > > Normally we don't regard test failure resulted from 'Too many open files' > as real test failure. > > I agree the effective limit should be raised. > > Cheers > > On Thu, Nov 24, 2011 at 5:03 PM, Mikhail Bautin < > [email protected]> wrote: > >> Hello, >> >> I saw this error when testing my patch on Jenkins: >> >> Caused by: java.io.IOException: Too many open files >> at sun.nio.ch.IOUtil.initPipe(Native Method) >> at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49) >> at >> sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) >> at java.nio.channels.Selector.open(Selector.java:209) >> at >> org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:42) >> at sun.reflect.GeneratedConstructorAccessor40.newInstance(Unknown >> Source) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> at java.lang.Class.newInstance0(Class.java:355) >> at java.lang.Class.newInstance(Class.java:308) >> at >> org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1737) >> ... 55 more >> >> (Full log: >> https://builds.apache.org/job/PreCommit-HBASE-Build/364//testReport/org.apache.hadoop.hbase.client/TestAdmin/testCheckHBaseAvailableClosesConnection/ >> ) >> >> I am not sure who maintains the Jenkins server, but it would be nice to >> increase the maximum number of open files for HBase unit tests. >> >> Thanks, >> --Mikhail >> > -- Todd Lipcon Software Engineer, Cloudera
