Hi, 2 GB for ramBufferSize is probably too much and not needed, but you could increase it from default 32 MB to something like 128 MB or even 512 MB, if you really have that much data where that would make a difference (you mention only 49 PDF files). I'd leave mergeFactor at 10 for now. The slowness (if there is slowness - how long is it taking?) could be from: * slow DB * suboptimal SQL * PDF content extraction * indexing itself * ...
Otis ---- Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ ----- Original Message ---- > From: Tomás Fernández Löbbe <tomasflo...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Mon, January 31, 2011 10:13:32 AM > Subject: Re: Solr Indexing Performance > > Well, I would say that the best way to be sure is to benchmark different > configurations. > As far as I know, it's usually not recommended such a big RAM Buffer size, > default is 32 MB and probably won't get any improvements using more than 128 > MB. > The same with the mergeFactor, I know that a larger merge factor it's better > for indexing, but 50 sounds like a lot. Anyway, as I said before, the best > thing to do is benchmark different configurations and see which one works > better for you. > > Have you tried assigning less memory to the JVM? That would leave more > memory available to the OS. > > Tomás > > On Sun, Jan 30, 2011 at 1:54 AM, Darx Oman <darxo...@gmail.com> wrote: > > > Hi guys > > > > > > > > I'm running a solr instance (trunk) in my dev. Server to test my > > configuration. I'm doing a DIH full import to index 49 PDF files with > > their > > corresponding database records. Both the PDF files and database are local > > in the server. > > > > *Server : * > > > > · Windows 2008 R2 > > > > · MS SQL server 2008 R2 > > > > · 16 core processor > > > > · 16 GB ram > > > > *Tomcat (7.0.5) : * > > > > · Set JAVA_OPTS = %JAVA_OPTS% -Xms1024M -Xmx8192M > > > > *Solrconfig:* > > > > · Main index configurations > > <ramBufferSize>2048</ramBufferSize> > > <mergeFactor>50</mergeFactor> > > > > *DIH configuration:* > > > > · 2 data sources defined jdbcDataSource and BinFileDataSource > > > > · One main entity with 3 sub entities > > > > <entity dataSource="myJdbc" …> > > > > <entity dataSource="myBinFile" …> </entity> > > > > <entity dataSource=" myJdbc" …> </entity> > > > > <entity dataSource=" myJdbc" …> </entity> > > > > <entity/> > > > > · Total schema fields are 8, three of which are text type and > > multivalued. > > > > *My DIH import Status Messages:* > > > > · Total Requests made to DataSource = 99** > > > > · Total Rows Fetched = 2124** > > > > · Total DocumentsProcessed = 49** > > > > · Time Taken = *0:2:3:880*** > > > > * > > Is this time reasonable or it can be improved?* > > >
