Thanks for the response. Whenever this happens it seems to take however much space is available. We were able to reproduce the issue and fill up a 250GB hard drive, which should be more than enough for a 20GB repository index.
Thanks, David -----Original Message----- From: Fabián Mandelbaum [mailto:[email protected]] Sent: Wednesday, July 23, 2014 7:55 AM To: [email protected] Subject: Re: Large index merge loop "2014-07-16 16:22:08,545 ERROR [org.apache.jackrabbit.core.query.lucene.IndexMerger] Error while merging indexes: java.io.IOException: There is not enough space on the disk" "There is not enough space on the disk" means... well... that you have run out of disk space somehow... or that JackRabbit, the Java VM, or the OS (or all three) cannot claim/use more disk space. I'd start looking there, if the disk fills up, and try with a bigger disk/partition size. Good luck. On Mon, Jul 21, 2014 at 8:39 PM, Hsu, David <[email protected]> wrote: > Hi, our app is based on Drools Guvnor and our repository is currently > about 20GB. Every once in a while, we will build up enough indexes > that it has a to do a very big merge (around 4GB). However, at a > certain point the TransientRepository shuts down for whatever reason > (I'm new to Jackrabbit and haven't been able to track down why...I am > still logged into the app when this occurs) and tries to stop the > IndexMerger. When the IndexMerger finally does finish, it says merging > has been canceled and that it is deleting the index, but looking at > the index directory I can see that the folder is still present. Then > the TransientRepository starts up again it retries and starts the > whole cycle over again, resulting in a bunch of leftover 4GB index folders > and eventually a full hard drive. > > We initially encountered this problem on Jackrabbit 1.4.5 (came with > Guvnor when this project was started) and have recently upgraded to > 2.8.0 thinking it would help, but we were still able to reproduce the > issue. Is this known and/or expected behavior with the > TransientRepository? Is there anything in Jackrabbit itself that > causes all Sessions to spontaneously close and shut down the repository? If > so how do we work around this? > > Below is an excerpt from our most recent run on 2.8.0. I forgot to > enable debug statements on TransientRepository, but if needed I can > provide the > 1.4.5 logs that have them. > > 2014-07-16 16:00:38,890 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] requesting merge > for [_10w6:99896, _10ze:99781, _115x:99915, _11qc:99849, _12kx:99861, > _13mh:43521, _14ha:34129, _15c5:34256, _1672:34254, _16ur:29993] > 2014-07-16 16:00:38,891 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] # of busy merge > workers: 1 > 2014-07-16 16:00:54,095 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:00:54,097 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > 2014-07-16 16:06:25,673 INFO > [org.apache.jackrabbit.core.query.lucene.IndexMerger] merged 675455 > documents in 344310 ms into _16uy. > 2014-07-16 16:06:25,674 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] index merging > canceled > 2014-07-16 16:06:25,675 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] deleting index > _16uy > 2014-07-16 16:06:25,675 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] > IndexMerger.Worker thread stopped > 2014-07-16 16:06:25,676 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] Worker finished > 2014-07-16 16:06:26,539 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:06:26,540 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > > 2014-07-16 16:06:34,731 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] requesting merge > for [_10w6:99896, _10ze:99781, _115x:99915, _11qc:99849, _12kx:99861, > _13mh:43521, _14ha:34129, _15c5:34256, _1672:34254, _16ur:29993] > 2014-07-16 16:06:34,732 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] # of busy merge > workers: 1 > 2014-07-16 16:06:54,192 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:06:54,193 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > 2014-07-16 16:12:28,267 INFO > [org.apache.jackrabbit.core.query.lucene.IndexMerger] merged 675455 > documents in 352127 ms into _16uz. > 2014-07-16 16:12:28,270 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] index merging > canceled > 2014-07-16 16:12:28,270 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] deleting index > _16uz > 2014-07-16 16:12:28,271 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] Worker finished > 2014-07-16 16:12:28,271 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] > IndexMerger.Worker thread stopped > 2014-07-16 16:12:28,607 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:12:28,608 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > > 2014-07-16 16:12:34,629 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] requesting merge > for [_10w6:99896, _10ze:99781, _115x:99915, _11qc:99849, _12kx:99861, > _13mh:43521, _14ha:34129, _15c5:34256, _1672:34254, _16ur:29993] > 2014-07-16 16:12:34,631 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] # of busy merge > workers: 1 > 2014-07-16 16:12:54,220 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:12:54,221 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > 2014-07-16 16:18:23,807 INFO > [org.apache.jackrabbit.core.query.lucene.IndexMerger] merged 675455 > documents in 348109 ms into _16v0. > 2014-07-16 16:18:23,810 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] index merging > canceled > 2014-07-16 16:18:23,810 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] deleting index > _16v0 > 2014-07-16 16:18:23,810 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] > IndexMerger.Worker thread stopped > 2014-07-16 16:18:23,812 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] Worker finished > 2014-07-16 16:18:24,165 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:18:24,166 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > > 2014-07-16 16:18:31,090 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] requesting merge > for [_10w6:99896, _10ze:99781, _115x:99915, _11qc:99849, _12kx:99861, > _13mh:43521, _14ha:34129, _15c5:34256, _1672:34254, _16ur:29993] > 2014-07-16 16:18:31,091 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] # of busy merge > workers: 1 > 2014-07-16 16:18:54,236 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:18:54,238 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > 2014-07-16 16:22:07,872 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] deleting index > _16v1 > 2014-07-16 16:22:08,545 ERROR > [org.apache.jackrabbit.core.query.lucene.IndexMerger] Error while > merging > indexes: > java.io.IOException: There is not enough space on the disk > at java.io.RandomAccessFile.setLength(Native Method) > at > org.apache.lucene.store.FSDirectory$FSIndexOutput.setLength(FSDirectory.java:489) > at > org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:195) > at > org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:137) > at > org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3219) > at > org.apache.jackrabbit.core.query.lucene.PersistentIndex.addIndexes(PersistentIndex.java:114) > at > org.apache.jackrabbit.core.query.lucene.IndexMerger$Worker.run(IndexMerger.java:526) > at > java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) > at > java.util.concurrent.FutureTask$Sync.innerRun(Unknown > Source) > at java.util.concurrent.FutureTask.run(Unknown Source) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.a > ccess$301(Unknown > Source) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.r > un(Unknown > Source) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > 2014-07-16 16:22:08,551 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] Worker finished > 2014-07-16 16:22:08,552 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] > IndexMerger.Worker thread stopped > 2014-07-16 16:22:08,890 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] dispose > IndexMerger > 2014-07-16 16:22:08,891 DEBUG > [org.apache.jackrabbit.core.query.lucene.IndexMerger] quit flag set > > Any help would be greatly appreciated. > > Thanks, > > David Hsu > -- Fabián Mandelbaum IS Engineer
