I use StandardAnalyzer.the records daily ranges from 5 crore to 6 crore. for
every second i am updating my Index. i instantiate IndexSearcher object one
time for all the searches. for an hour can i see the updated records in the
indexstore by reinstantiating IndexSearcher object.but the problem when i
reinstantiate IndexSearcher ,my RAM memory gets appended.is there any
possibility to control the memory at that time without affecting
performance.
testn wrote:
>
> A couple things to make sure:
> 1. When you open IndexWriter, what is the analyzer you use?
> StandardAnalyzer?
> 2. How many records are there?
> 3. Could you also check number of terms in your indices? If there are too
> many terms, you could consider chop something in smaller piece for
> example... store area code and phone number separately if the numbers are
> pretty distributed.
>
>
> Sebastin wrote:
>>
>> Hi testn,
>> here is my index details:
>> Index fields :5 fields
>> Store Fileds:10 fields
>>
>>
>> Index code:
>>
>> contents=new StringBuilder().append(compCallingPartyNumber).append("
>> ").append(compCalledPartyNumber).append("
>> ").append(compImsiNumber).append(" ").append(callType).toString();
>>
>>
>> records=new StringBuilder().append(compCallingPartyNumber).append("
>> ").append(compCalledPartyNumber).append("
>> ").append(compchargDur).append(" ").append(compTimeSc).append("
>> ").append(compImsiNumber).append(" ").append(outgoingRoute).append("
>> ").append(incomingRoute).append(" ").append(cgiLocation).toString();
>>
>>
>> Document document = new Document();
>> document.add(new Field("contents",
>> contents,
>>
>> Field.Store.NO,
>>
>> Field.Index.TOKENIZED));
>> document.add(new Field("fil", filen,
>>
>> Field.Store.NO,
>>
>> Field.Index.TOKENIZED));
>> document.add(new Field("records",
>> records,
>>
>> Field.Store.YES,
>>
>> Field.Index.NO));
>> document.add(new Field("dateSc",
>> dateSc,
>>
>> Field.Store.YES,
>>
>> Field.Index.TOKENIZED));
>>
>> indexWriter.addDocument(document);
>>
>> inputs for the document:
>>
>> compCallingPartyNumber="9840836588";
>> compCalledPartyNumber="9840861114";
>> compImsiNumber="984510005469874";
>> callType="1";
>> compChargDur="98456";
>> compTimeSc="984";
>> outgoingRoute="i987j";
>> incomingRoute="poi09";
>> cgiLocation="dft1234567";
>>
>> here is my search code:
>>
>> Directory indexDir2 =
>> FSDirectory.getDirectory(indexSourceDir02,false);
>> IndexReader indexSource2 =
>> IndexReader.open(indexDir2);
>> Directory indexDir3 =
>>
>> FSDirectory.getDirectory(indexSourceDir03,false);
>> IndexReader indexSource3 =
>> IndexReader.open(indexDir3);
>> Directory indexDir4 =
>>
>> FSDirectory.getDirectory(indexSourceDir04,false);
>> IndexReader indexSource4 =
>> IndexReader.open(indexDir4);
>>
>>
>>
>> IndexReader[] readArray =
>> {indexSource2,indexSource3,indexSource4};
>> //merged reader
>> IndexReader mergedReader = new MultiReader(readArray);
>> IndexSearcher is = new IndexSearcher(mergedReader);
>>
>>
>> QueryParser parser =
>> new QueryParser("contents" ,new
>> StandardAnalyzer());
>>
>>
>> String searchQuery=
>> new
>> StringBuffer().append(inputNo).append(" AND dateSc:["
>> ).append(fromDate).append(" TO ").append(toDate).append("]").append("
>> ").append("AND").append(" ").append(callTyp).toString();
>>
>>
>>
>> Query callDetailquery =
>> parser.parse(searchQuery);
>>
>> hits = is.search(callDetailquery);
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> testn wrote:
>>>
>>> Can you provide more info about your index? How many documents, fields
>>> and what is the average document length?
>>>
>>>
>>> Sebastin wrote:
>>>>
>>>> Hi testn,
>>>> i index the dateSc as 070904(2007/09/04) format.i am not
>>>> using any timestamp here.how can we effectively reopen the
>>>> IndexSearcher for an hour and save the memory because my index gets
>>>> updated every minute.
>>>>
>>>> testn wrote:
>>>>>
>>>>> Check out Wiki for more information at
>>>>> http://wiki.apache.org/jakarta-lucene/LargeScaleDateRangeProcessing
>>>>>
>>>>>
>>>>>
>>>>> Sebastin wrote:
>>>>>>
>>>>>> Hi All,
>>>>>> i used to search 3 Lucene Index store of size 6 GB,10 GB,10 GB
>>>>>> of records using MultiReader class.
>>>>>>
>>>>>> here is the following code snippet:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Directory indexDir2 =
>>>>>>
>>>>>> FSDirectory.getDirectory(indexSourceDir02,false);
>>>>>> IndexReader indexSource2 =
>>>>>> IndexReader.open(indexDir2);
>>>>>> Directory indexDir3 =
>>>>>>
>>>>>> FSDirectory.getDirectory(indexSourceDir03,false);
>>>>>> IndexReader indexSource3 =
>>>>>> IndexReader.open(indexDir3);
>>>>>> Directory indexDir4 =
>>>>>>
>>>>>> FSDirectory.getDirectory(indexSourceDir04,false);
>>>>>> IndexReader indexSource4 =
>>>>>> IndexReader.open(indexDir4);
>>>>>>
>>>>>>
>>>>>>
>>>>>> IndexReader[] readArray =
>>>>>> {indexSource2,indexSource3,indexSource4};
>>>>>> //merged reader
>>>>>> IndexReader mergedReader = new MultiReader(readArray);
>>>>>> IndexSearcher is = new IndexSearcher(mergedReader);
>>>>>>
>>>>>>
>>>>>> QueryParser parser =
>>>>>> new QueryParser("contents" ,new
>>>>>> StandardAnalyzer());
>>>>>>
>>>>>>
>>>>>> String searchQuery=
>>>>>> new
>>>>>> StringBuffer().append(inputNo).append(" AND dateSc:["
>>>>>> ).append(fromDate).append(" TO ").append(toDate).append("]").append("
>>>>>> ").append("AND").append(" ").append(callTyp).toString();
>>>>>>
>>>>>>
>>>>>>
>>>>>> Query callDetailquery =
>>>>>> parser.parse(searchQuery);
>>>>>>
>>>>>> hits = is.search(callDetailquery);
>>>>>>
>>>>>>
>>>>>> it takes 300 MB of RAM for every search and it is very very slow is
>>>>>> there any other way to control the Memory and to make search faster.i
>>>>>> use SINGLETON to use the IndexSearcher as a one time used object for
>>>>>> all the instances.
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12500824
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
