Hi Dat, I can reproduce this behavior even with like 50000 records. Is what you said the only reason that make this exception occur?
Thanks, On Sat, Jun 13, 2015 at 5:40 AM, Đạt Cao Mạnh <caomanhdat...@gmail.com> wrote: > Hi, the total number of documents in an index of lucene is > Integer.MAX_VALUE. So using a single lucene index to index billions > documents is not a proper ways. You should consider using Solr Cloud or > Elasticsearch to index your documents. > > On 19:43, Fri, 12 Jun 2015 Gimantha Bandara <giman...@wso2.com> wrote: > > > Hi all, > > > > We are using Lucene 4.10.3 for indexing. Recently we changed our > > implementation so that we give data batchwise to lucene to index. Earlier > > we just query all the data from the data source and index all data at > > once. It works well. But the number of entries can be up to billions. So > > getting all the data entries from the data source causes OutOfMemory > > sometimes. So we changed the implementation to So that Lucene indexes the > > data batchwise. Now we are getting the following exception. Can anyone > tell > > me what that exception means? > > > > java.lang.ArrayIndexOutOfBoundsException: 147 > > at > > > > > org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.advance(Lucene41PostingsReader.java:538) > > at org.apache.lucene.search.TermScorer.advance(TermScorer.java:85) > > at > > > > > org.apache.lucene.search.ConjunctionScorer.doNext(ConjunctionScorer.java:82) > > at > > > > > org.apache.lucene.search.ConjunctionScorer.nextDoc(ConjunctionScorer.java:100) > > at > > > org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192) > > at > > org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163) > > at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35) > > at > > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) > > at > > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309) > > at > > > org.apache.lucene.facet.FacetsCollector.doSearch(FacetsCollector.java:294) > > at > > org.apache.lucene.facet.FacetsCollector.search(FacetsCollector.java:198) > > > > > > -- > > Gimantha Bandara > > Software Engineer > > WSO2. Inc : http://wso2.com > > Mobile : +94714961919 > > > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919
