Re: how to index large number of files?

2010-10-22 Thread Peter Keegan
running eclipse with -Xmx2G parameter. This only affects the Eclipse JVM, not the JVM launched by Eclipse to run your application. Did you add -Xmx2G to the 'VM arguments' of your Debug or Run configuration? Peter On Thu, Oct 21, 2010 at 3:26 PM, Sahin Buyrukbilen sahin.buyrukbi...@gmail.com

Re: Using a TermFreqVector to get counts of all words in a document

2010-10-22 Thread Grant Ingersoll
http://www.lucidimagination.com/blog/2009/05/26/accessing-words-around-a-positional-match-in-lucene/ has an example of implementing a TermVectorMapper. There are also several implementations included in the Lucene codebase. All it really does is give you a callback as it is reading the code

Re: how to index large number of files?

2010-10-22 Thread Sahin Buyrukbilen
thank you for all. I tried Peter's suggestion and it really worked. I added -Xmx2G to the run/debug VM arguments and it worked. However, before getting his advice I had already started to split the files into folders 100.000 by 100.000 and now indexing recursively :) Anyway, I have learned much