Does anybody know if Solr performs better on Jetty vs Tomcat? -----Original Message----- From: David Smiley (@MITRE.org) [mailto:dsmi...@mitre.org] Sent: Tuesday, July 30, 2013 9:08 PM To: solr-user@lucene.apache.org Subject: Re: Ingesting geo data into Solr very slow
Hi Marta, Presumably you are indexing polygons -- I suspect complex ones. There isn't too much that you can do about this right now other than index them in parallel. I see you are doing this in 2 threads; try 4, or maybe even 6. Also, ensure that maxDistErr is reflective of the smallest distance you need to distinguish between. It may help a little but not much. I can think of some internal code details that might be improved but that doesn't help you now. There's some generic Solr things you can do to improve indexing performance too like increasing the indexing buffer size (100MB -> 200MB) and the mergeFactor (10->20 albeit temporarily and/or issue optimize), both in solrconfig.xml. Changing the servlet engine won't help. Calling server.addBean(item) isn't a problem either. ~ David Simonian, Marta M (US SSA) wrote > Hi, > > We are using Solr 4.4 to ingest geo data and it's really slow. When we > don't index the geo it takes seconds to ingest 100, 000 records but as > soon as we add it takes 2 hours. > > Also we found that when changing the distErrPct from 0.025 to 0.1, > 1000 rows are ingested in 20 sec vs 2 min. But we can't change that > setting as we want our search to be as accurate as possible. > > About the environment we are running Solr on 6 CPUs and 8GB of memory. > We've been monitoring the VMs and they seem to be ok. > > We are running on Tomcat but we might switch to Jetty to see if that > will increase the performance. > > We use ConcurrentUpdateSolrServer(httpSolrServer, 5000, 2); > > We are saving a bean rather than a solr document (server.addBean(item)). > I'm not sure if that could make it slow as it's going to do some > conversion? > > Can you please let me know what are the best settings for Solr? Maybe > some changes in the solrconfig.xml or the schema.xml? > What are the preferred environment settings and resources? > > Thank you! > Marta ----- Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Ingesting-geo-data-into-Solr-very-slow-tp4081484p4081527.html Sent from the Solr - User mailing list archive at Nabble.com.
