Hi, I have an issue when optimizing a large index (40GB). Basically the optimize falls over with an IOException stating there are not enough system resources, within a RandomAccessFile.readBytes call followed back to the CompoundFileWriter close method. I assume this is when creating a new CFS file from the separate other files. Unfortunately there is no hint at which system resource, and I have no access to the machine this fails on, otherwise I would run some testing to see what resource usage is going on. The process running this, has 1.3GB of memory allowance.
If anyone has any insights they would be much appreciated. Stack trace included now: Exception in thread "Lucene Merge Thread #0" org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: Insufficient system resources exist to complete the requested service at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Co ncurrentMergeScheduler.java:309) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr entMergeScheduler.java:286) Caused by: java.io.IOException: Insufficient system resources exist to complete the requested service at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(Unknown Source) at org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector y.java:596) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput. java:136) at org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j ava:228) at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java :184) at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j ava:204) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge Scheduler.java:205) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr entMergeScheduler.java:260) Exception in thread "Thread-12957887" java.lang.RuntimeException: java.io.IOException: background merge hit exception: _cx:c3072736 _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 _dn:c4045 _do:c3643 _dp:c3197 into _dq [optimize] at a.b.c.d.e.f.ImportThread$ImportRunner.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.IOException: background merge hit exception: _cx:c3072736 _d8:c55434 _d9:c9791 _da:c10097 _dl:c54262 _dm:c4791 _dn:c4045 _do:c3643 _dp:c3197 into _dq [optimize] at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2258) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2203) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2183) ... 2 more Caused by: java.io.IOException: Insufficient system resources exist to complete the requested service at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(Unknown Source) at org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector y.java:596) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput. java:136) at org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.j ava:228) at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java :184) at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.j ava:204) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4260) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3877) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMerge Scheduler.java:205) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr entMergeScheduler.java:260) Thanks Carl Austin This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies within the Detica Group Limited group of companies. Detica Group Limited is registered in England under No: 03328242. Registered offices: Warwick House, PO Box 87, Farnborough Aerospace Centre, Farnborough, Hampshire, GU14 6YU, England.