For SUSS https://issues.apache.org/jira/browse/SOLR-1565 it says no binary support...
When we try to use from solrj binary through SUSS adding document took thousands of milliseconds to million per doc.!! so we turn back normal submit. On Tue, Sep 27, 2011 at 7:14 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > Hello, > > > PS: solr streamindex is not option because we need to submit javabin... > > > If you are referring to StreamingUpdateSolrServer, then the above statement > makes no sense and you should give SUSS a try. > > Are you sure your 16 reducers produce more than 500 docs/second? > I think somebody already suggested increasing the number of reducers to > ~32. > What happens to your CPU load and indexing speed then? > > > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > >________________________________ > >From: Lord Khan Han <khanuniver...@gmail.com> > >To: solr-user@lucene.apache.org > >Sent: Monday, September 26, 2011 7:09 AM > >Subject: SOLR Index Speed > > > >Hi, > > > >We have 500K web document and usind solr (trunk) to index it. We have > >special anaylizer which little bit heavy cpu . > >Our machine config: > > > >32 x cpu > >32 gig ram > >SAS HD > > > >We are sending document with 16 reduce client (from hadoop) to the stand > >alone solr server. the problem is we couldnt get speedier than the 500 doc > / > >per sec. 500K document tooks 7-8 hours to index :( > > > >While indexin the the solr server cpu load is around : 5-6 (32 max) it > >means %20 of the cpu total power. We have plenty ram ... > > > >I turned of auto commit and give 8198 rambuffer .. there is no io wait .. > > > >How can I make it faster ? > > > >PS: solr streamindex is not option because we need to submit javabin... > > > >thanks.. > > > > > > >
