[
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617880#action_12617880
]
Matthew Mastracci commented on LUCENE-753:
------------------------------------------
I just tried out the latest NIOFSDirectory patch and I'm seeing a bug. If I go
back to the regular FSDirectory, everything works fine.
I can't reproduce it on a smaller testcase. It only happens with the live
index.
Any ideas on where to debug?
Caused by: java.lang.IndexOutOfBoundsException: Index: 24444, Size: 4
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:260)
at org.apache.lucene.index.FieldInfos.fieldName(FieldInfos.java:249)
at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:68)
at
org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:123)
at
org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:154)
at
org.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:223)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:217)
at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:678)
at
org.apache.lucene.index.MultiSegmentReader.docFreq(MultiSegmentReader.java:373)
at org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:87)
at org.apache.lucene.search.Similarity.idf(Similarity.java:457)
at
org.apache.lucene.search.TermQuery$TermWeight.<init>(TermQuery.java:44)
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:146)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:187)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:362)
at org.apache.lucene.search.Query.weight(Query.java:95)
at org.apache.lucene.search.Searcher.createWeight(Searcher.java:171)
at org.apache.lucene.search.Searcher.search(Searcher.java:132)
The index is not using the compound file format:
{{
7731499698 Jul 28 03:46 _6zk.fdt
232014520 Jul 28 03:50 _6zk.fdx
32 Jul 28 03:50 _6zk.fnm
3775713450 Jul 28 04:06 _6zk.frq
58003634 Jul 28 04:07 _6zk.nrm
2944298834 Jul 28 04:18 _6zk.prx
432418 Jul 28 04:18 _6zk.tii
30784106 Jul 28 04:19 _6zk.tis
217354711 Jul 28 08:18 _76i.fdt
6509864 Jul 28 08:18 _76i.fdx
32 Jul 28 08:18 _76i.fnm
144348761 Jul 28 08:18 _76i.frq
1627470 Jul 28 08:18 _76i.nrm
295528445 Jul 28 08:19 _76i.prx
52622 Jul 28 08:19 _76i.tii
3858378 Jul 28 08:19 _76i.tis
199621206 Jul 29 13:29 _7cm.fdt
5994720 Jul 29 13:29 _7cm.fdx
32 Jul 29 13:29 _7cm.fnm
136445620 Jul 29 13:29 _7cm.frq
1498684 Jul 29 13:29 _7cm.nrm
284805312 Jul 29 13:30 _7cm.prx
48346 Jul 29 13:30 _7cm.tii
3522117 Jul 29 13:30 _7cm.tis
3914068 Jul 29 13:30 _7cn.fdt
119184 Jul 29 13:30 _7cn.fdx
32 Jul 29 13:30 _7cn.fnm
2993343 Jul 29 13:30 _7cn.frq
29800 Jul 29 13:30 _7cn.nrm
7380878 Jul 29 13:30 _7cn.prx
5277 Jul 29 13:30 _7cn.tii
378816 Jul 29 13:30 _7cn.tis
383147 Jul 29 13:30 _7cq.fdt
11240 Jul 29 13:30 _7cq.fdx
32 Jul 29 13:30 _7cq.fnm
290398 Jul 29 13:30 _7cq.frq
2814 Jul 29 13:30 _7cq.nrm
763135 Jul 29 13:30 _7cq.prx
1581 Jul 29 13:30 _7cq.tii
115971 Jul 29 13:30 _7cq.tis
19 Jul 29 13:30 date
20 Jul 21 01:53 segments.gen
155 Jul 29 13:30 segments_d61
}}
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FSDirectoryPool.patch, FSIndexInput.patch, FSIndexInput.patch,
> lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of
> files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
