What kind of storage is used for the Solr index files? When I tested it, NFS was 100X slower than local disk.
wunder -----Original Message----- From: Dan A. Dickey [mailto:dan.dic...@savvis.net] Sent: Thursday, September 10, 2009 11:15 AM To: solr-user@lucene.apache.org Cc: Walter Underwood Subject: Re: Solr http post performance seems slow - help? On Thursday 10 September 2009 09:10:27 am Walter Underwood wrote: > How big are your documents? For the most part, I'm just indexing metadata that has been pulled from the documents. I think I have currently about 40 or so fields that I'm setting. When the document is an actual document - pdf, doc, etc... I use the DIH to extract stuff and also set the metadata then. > Is your index on local disk or network- > mounted disk? I'm basically pulling the metadata info from a database and the documents themselves are shared via NFS to the Solr indexer. -Dan > > wunder > > On Sep 10, 2009, at 6:39 AM, Yonik Seeley wrote: > > > On Thu, Sep 10, 2009 at 9:13 AM, Dan A. Dickey > > <dan.dic...@savvis.net> wrote: > >> I'm posting documents to Solr using http (curl) from > >> C++/C code and am seeing approximately 3.3 - 3.4 > >> documents per second being posted. Is this to be expected? > > > > No, that's very slow. > > Are you using libcurl, or actually forking a new process for every > > document? > > Are you committing on every document? > > > > If you can, using Java would make your life much easier since you > > could use the SolrJ client and it's binary protocol for indexing. > > > > -Yonik > > http://www.lucidimagination.com > > > > -- Dan A. Dickey | Senior Software Engineer Savvis 10900 Hampshire Ave. S., Bloomington, MN 55438 Office: 952.852.4803 | Fax: 952.852.4951 E-mail: dan.dic...@savvis.net
