Solr resource usage/Clustering
Hi, We have a single solr instance serving queries to the client through out the day and being indexed twice a day using scheduled jobs. During the scheduled jobs, which actually syncs databases from data collection machines to the master database, it can make many indexing calls. It is usually about 50k-100k records that are synced on each iteration of sync and we make calls to solr in batch of 1000 documents. Now, during the sync process, solr throws 503 (service not available message) quite frequently and in fact it responds very slow to index the documents. I have checked the cpu and memory usage during the sync process and it never consumed more than 40-50 % of CPU and 10-20% of RAM. My question is how to increase the performance of indexing to increase the speed up the sync process. -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
Re: High system cpu usage while starting solr
One quick improvement can be to add add -Xm*s*6144m along with -Xmx6144m this causes jvm to acquire all memory before hand and it would not waste time in allocating more memory by requesting to kernel. On restart, I am not sure but I guess solr does some syncing of indexes, so it might be slow to respond in that duration. On Fri, Nov 7, 2014 at 2:58 PM, mizayah miza...@gmail.com wrote: Hello, Im running few solr cores on one pretty good server. After some time i discover that restarting solr makes queries last longer. What i see is that after restart jvm usage is realy low and raise slowly while system cpu ussage is high. My select queries are realy slow during that time. After few days when jvm grab some MORE memory system drops down. java settings -Xmx6144m -XX:+UseConcMarkSweepGC -XX:+PrintGC -XX:+PrintGCDetails I have 8GB ram HELP! -- View this message in context: http://lucene.472066.n3.nabble.com/High-system-cpu-usage-while-starting-solr-tp4168124.html Sent from the Solr - User mailing list archive at Nabble.com. -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
Re: Stopwords in shingles suggester
Is this https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StopFilterFactory what you are looking for? Basically, you can use analyzers for this purpose. You can even write your own analyzer. On Mon, Oct 27, 2014 at 6:26 PM, O. Klein kl...@octoweb.nl wrote: Is there a way in Solr to filter out stopwords in shingles like ES does? http://www.elasticsearch.org/blog/searching-with-shingles/ -- View this message in context: http://lucene.472066.n3.nabble.com/Stopwords-in-shingles-suggester-tp4166057.html Sent from the Solr - User mailing list archive at Nabble.com. -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
