Re: Java Heap Space -Out Of Memory Error

Hi testn, here is my index details: Index fields :5 fields Store Fileds:10 fields Index code: contents=new StringBuilder().append(compCallingPartyNumber).append(" ").append(compCalledPartyNumber).appen

Re: Extract terms not by reader, but by documents

Not sure if I am understanding what you are trying to do. I think you are trying to find out which terms occurred in a particular document, correct? I also am not sure about your first example. My understanding of extractTerms is that it just gives you back the set of all terms that occ

Extract terms not by reader, but by documents

Hi all, In some custom highlighting, I often write a code like this: Set matchedTerms = new HashSet(); query.rewrite(reader).extractTerms(matchedTerms); With this code the Term Set gets populated by the matched query in your whole index. Is it possible to this with a docume

Re: open file descriptors for deleted index files

Closing old IndexSearcher should take care of this problem for you. Take a look at Solr. It opens a new IndexSearcher and direct all requests to the new one. It then closes the old IndexSearcher when all the requests that it is serving has completed. Bill On 9/4/07, Tony Qian <[EMAIL PROTECTED]

Re: open file descriptors for deleted index files

On 9/4/07, Tony Qian <[EMAIL PROTECTED]> wrote: > We run a servlet inside Tomcat. We transfer index files to search server > every 5 minutes using rsync with --delete option. Also, we schedule a > TimeTask inside servlet to check if index files are updated. If yes, create > a new IndexSearcher obje

open file descriptors for deleted index files

All, I'm facing an issue in which the file descriptors are not closed for deleted index files. I searched mailing list and didn't find the solution. Here is some info: java 21488 wppd 139r REG8,7 152456865 571208 /data/index/_idx.cfs (deleted) java 21488 w

Re: Java Heap Space -Out Of Memory Error

Can you provide more info about your index? How many documents, fields and what is the average document length? Sebastin wrote: > > Hi testn, >i index the dateSc as 070904(2007/09/04) format.i am not using > any timestamp here.how can we effectively reopen the IndexSearcher for an

Re: Data in the Index [was: JdbcDirectory]

I store Lucene index outside database, and run indexing periodically to get the latest updates, not depending on ORM APIs. In general, search data can be slower to update unless some realtime requirements. Storing data in index saves trips to databases. This usually is a huge difference on renderi

Re: Lockless read-only deletions in IndexReader?

Excellent, a much simpler approach! I think it should work? Maybe override numDocs() as well? Mike "Karl Wettin" <[EMAIL PROTECTED]> wrote: > > 20 aug 2007 kl. 14.33 skrev Michael McCandless: > > > "karl wettin" <[EMAIL PROTECTED]> wrote: > > > >> I want to set documents in my IndexReader as

Look for strange encodings -- tokenization

Hi all, I'd want to know the best way to look for strange encodings on a Lucene index. i have several inputs where input can have been encoded on different sets. I not always know if my guess about the encoding has been ok. Hence, I'd thought of querying the index for some typical strings that wo

Re: Java Heap Space -Out Of Memory Error

Hi testn, i index the dateSc as 070904(2007/09/04) format.i am not using any timestamp here.how can we effectively reopen the IndexSearcher for an hour and save the memory because my index gets updated every minute. testn wrote: > > Check out Wiki for more information at > http://wik

Re: Data in the Index [was: JdbcDirectory]

Hi, At first, we thought we would use a "dual" approach, an Lucene index and a RDBMS for storage. While prototyping, for simplicity sake, we used the Lucene index as storage, thinking we could easily replace it later. So far, speed is satisfying enough that we are going to keep data there util re

Data in the Index [was: JdbcDirectory]

So, Anyone ever stored the data in the index also ? What are your experiences ? Thanks a lot Gui On Sep 3, 2007, at 3:47 PM, Guilherme Barile wrote: Storing the data in the index, mainly for non-structured data. We plan to implement something like this ThingDB from http:// demo.openlibra

Re: Java Heap Space -Out Of Memory Error

Check out Wiki for more information at http://wiki.apache.org/jakarta-lucene/LargeScaleDateRangeProcessing Sebastin wrote: > > Hi All, >i used to search 3 Lucene Index store of size 6 GB,10 GB,10 GB of > records using MultiReader class. > > here is the following code snippet: > > >

Re: Java Heap Space -Out Of Memory Error

I think you store dateSc with full precision i.e. with time. You should consider to index it just date part or to the resolution you really need. It should reduce the memory it use when constructing DateRangeQuery and plus it will improve search performance as well. Sebastin wrote: > > Hi All,

Re: how to implement searching "in time" efficiently

Hi Erick, help me for this search in time efficiently. Erick Erickson wrote: > > This topic has been discussed a number of times, I suggest you > search the mail archives as that will get you very complete answers > more quickly. See > http://www.gossamer-threads.com/lists/lucene/java-u

Java Heap Space -Out Of Memory Error

Hi All, i used to search 3 Lucene Index store of size 6 GB,10 GB,10 GB of records using MultiReader class. here is the following code snippet: Directory indexDir2 = FSDirectory.getDirectory(indexSourceDir02,false);