Re: Too many open files with DirectUpdateHandlerOptimizeTest

[Tricia Williams]

This same thing happens to me since DirectUpdateHandlerOptimizeTest was 
added to the repository.

How does one increase the file handle limit in ubuntu?
Thanks,
Tricia

Shalin Shekhar Mangar wrote:
Yes, it happens on a fresh checkout too.

cat /proc/sys/fs/file-max gives 204979 on my box. The test which fails is
testOptimize.

I increased the file handle limit to 4096 but it still fails. Then I
increased it to 16384 and the test passed.

On Mon, Jul 21, 2008 at 2:58 AM, Yonik Seeley <[EMAIL PROTECTED]> wrote:

  
Does it happen on a fresh checkout?
I use windows (and cygwin for the familiar command line interface),
which has higher limits.

Also watch out for the system-wide limit:
http://www.cs.wisc.edu/condor/condorg/linux_scalability.html

If it happens with a fresh checkout, perhaps we could lower the merge
factor for that test.

-Yonik

On Sun, Jul 20, 2008 at 2:59 PM, Shalin Shekhar Mangar
<[EMAIL PROTECTED]> wrote:
    
Hi,

The DirectUpdateHandlerOptimizeTest fails on my local box due to too many
open files. The nightly build does not fail so I'm assuming it must be
something specific to my setup. Has anyone else seen this problem on
      
their
    
local environment?

ulimit -n gives 1024 on my Ubuntu Hardy Heron.

<testcase
      
classname="org.apache.solr.update.DirectUpdateHandlerOptimizeTest"
    
name="testOptimize" time="5.687">
   <error message="java.io.FileNotFoundException:

      
/tmp/org.apache.solr.update.DirectUpdateHandlerOptimizeTest-1216580007333/index/_9u.tii
    
(Too many open files)"
type="java.lang.RuntimeException">java.lang.RuntimeException:
java.io.FileNotFoundException:

      
/tmp/org.apache.solr.update.DirectUpdateHandlerOptimizeTest-1216580007333/index/_9u.tii
    
(Too many open files)
       at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:806)
       at

      
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:368)
    
       at

      
org.apache.solr.update.DirectUpdateHandlerOptimizeTest.testOptimize(DirectUpdateHandlerOptimizeTest.java:62)
    
Caused by: java.io.FileNotFoundException:

      
/tmp/org.apache.solr.update.DirectUpdateHandlerOptimizeTest-1216580007333/index/_9u.tii
    
(Too many open files)
       at java.io.RandomAccessFile.open(Native Method)
       at
      
java.io.RandomAccessFile.&lt;init&gt;(RandomAccessFile.java:212)
    
       at

      
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.&lt;init&gt;(FSDirectory.java:539)
    
       at

      
org.apache.lucene.store.FSDirectory$FSIndexInput.&lt;init&gt;(FSDirectory.java:569)
    
       at
org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:478)
       at

      
org.apache.lucene.index.TermInfosReader.&lt;init&gt;(TermInfosReader.java:80)
    
       at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:319)
       at
      
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:264)
    
       at
      
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:199)
    
       at

      
org.apache.lucene.index.MultiSegmentReader.&lt;init&gt;(MultiSegmentReader.java:55)
    
       at

      
org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexReader.java:93)
    
       at

      
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:649)
    
       at

      
org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.java:81)
    
       at org.apache.lucene.index.IndexReader.open(IndexReader.java:209)
       at org.apache.lucene.index.IndexReader.open(IndexReader.java:173)
       at

      
org.apache.solr.search.SolrIndexSearcher.&lt;init&gt;(SolrIndexSearcher.java:93)
    
       at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:797)
</error>
 </testcase>


--
Regards,
Shalin Shekhar Mangar.