Re: Tips on speeding the indexing process

2009-07-01 Thread Noble Paul നോബിള്‍ नोब्ळ्
exing speed by > temporarily removing both of them. > > Thanks, > Kalyan Manepalli > > -Original Message- > From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] > Sent: Wednesday, July 01, 2009 3:31 PM > To: solr-user@lucene.apache.org > Subject: Re: Tips on s

RE: Tips on speeding the indexing process

2009-07-01 Thread Manepalli, Kalyan
: Manepalli, Kalyan [mailto:kalyan.manepa...@orbitz.com] Sent: Wednesday, July 01, 2009 3:41 PM To: solr-user@lucene.apache.org Subject: RE: Tips on speeding the indexing process Regarding the analysis, we do couple of things during indexing. First is use a dictionary text file for stopword filter factory

RE: Tips on speeding the indexing process

2009-07-01 Thread Manepalli, Kalyan
-Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, July 01, 2009 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Tips on speeding the indexing process Kalyan, 150/200 ms per 1 document to index seems too long, but it really depends on how much

Re: Tips on speeding the indexing process

2009-07-01 Thread Otis Gospodnetic
Message > From: "Manepalli, Kalyan" > To: "solr-user@lucene.apache.org" > Sent: Wednesday, July 1, 2009 4:21:30 PM > Subject: RE: Tips on speeding the indexing process > > Here are some specs for my indexer. > Indexer is custom Java code that reads data

RE: Tips on speeding the indexing process

2009-07-01 Thread Manepalli, Kalyan
Manepalli -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, July 01, 2009 3:11 PM To: solr-user@lucene.apache.org Subject: Re: Tips on speeding the indexing process Kalyan, Using SolrJ? Use the StreamingServer, it's nice and fast. Alternat

Re: Tips on speeding the indexing process

2009-07-01 Thread Otis Gospodnetic
xt -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: "Manepalli, Kalyan" > To: "solr-user@lucene.apache.org" > Sent: Wednesday, July 1, 2009 3:42:45 PM > Subject: Tips on speeding the indexing process > > Hi, > I

Re: Tips on speeding the indexing process

2009-07-01 Thread Erik Hatcher
Kalyan, Tell us about your indexer. Is it DIH-powered? Custom Java code, perhaps, using SolrJ indexing over HTTP? Is your indexer doing a lot of work itself to preprocess documents before sending to Solr? Erik On Jul 1, 2009, at 3:42 PM, Manepalli, Kalyan wrote: Hi,

Tips on speeding the indexing process

2009-07-01 Thread Manepalli, Kalyan
Hi, I have a very generic question regarding indexing. In my current app, I have about 450,000 docs each doc size around 2k. The total indexing time is around 2hrs. Now due to multi language support, the number of documents is increasing to 2.0 million. The total indexing time is exc