Can you post you scenario in detail along with your modification please? On 14:09, Sun, 14 Jun 2015 Gimantha Bandara <giman...@wso2.com> wrote:
> Hi Dat, > > I can reproduce this behavior even with like 50000 records. Is what you > said the only reason that make this exception occur? > > Thanks, > > On Sat, Jun 13, 2015 at 5:40 AM, Đạt Cao Mạnh <caomanhdat...@gmail.com> > wrote: > >> Hi, the total number of documents in an index of lucene is >> Integer.MAX_VALUE. So using a single lucene index to index billions >> documents is not a proper ways. You should consider using Solr Cloud or >> Elasticsearch to index your documents. >> >> On 19:43, Fri, 12 Jun 2015 Gimantha Bandara <giman...@wso2.com> wrote: >> >> > Hi all, >> > >> > We are using Lucene 4.10.3 for indexing. Recently we changed our >> > implementation so that we give data batchwise to lucene to index. >> Earlier >> > we just query all the data from the data source and index all data at >> > once. It works well. But the number of entries can be up to billions. So >> > getting all the data entries from the data source causes OutOfMemory >> > sometimes. So we changed the implementation to So that Lucene indexes >> the >> > data batchwise. Now we are getting the following exception. Can anyone >> tell >> > me what that exception means? >> > >> > java.lang.ArrayIndexOutOfBoundsException: 147 >> > at >> > >> > >> org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.advance(Lucene41PostingsReader.java:538) >> > at org.apache.lucene.search.TermScorer.advance(TermScorer.java:85) >> > at >> > >> > >> org.apache.lucene.search.ConjunctionScorer.doNext(ConjunctionScorer.java:82) >> > at >> > >> > >> org.apache.lucene.search.ConjunctionScorer.nextDoc(ConjunctionScorer.java:100) >> > at >> > >> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192) >> > at >> > org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163) >> > at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35) >> > at >> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) >> > at >> > org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309) >> > at >> > >> org.apache.lucene.facet.FacetsCollector.doSearch(FacetsCollector.java:294) >> > at >> > org.apache.lucene.facet.FacetsCollector.search(FacetsCollector.java:198) >> > >> > >> > -- >> > Gimantha Bandara >> > Software Engineer >> > WSO2. Inc : http://wso2.com >> > Mobile : +94714961919 >> > >> > > > > -- > Gimantha Bandara > Software Engineer > WSO2. Inc : http://wso2.com > Mobile : +94714961919 >
