Re: SolrCloud performance

Hi Shawn, Thank you very much for your analysis. I currently don’t have multiple machines to play with. I will try "one Solr instance and one ZK instance would be more efficient on a single server” you suggested. Thanks again, Chuming On Nov 4, 2018, at 7:56 PM, Shawn Heisey wrote: > On

Re: SolrCloud performance

On 11/4/2018 8:38 AM, Chuming Chen wrote: I have shared a tar ball with you (apa...@elyograg.org) from google drive. The tar ball includes logs directories of 4 nodes, solrconfig.xml, solr.in.sh, and screenshot of TOP command. The log files is about 1 day’s log. However, I restarted the solr

Re: SolrCloud performance

Hi Shawn, I have shared a tar ball with you (apa...@elyograg.org) from google drive. The tar ball includes logs directories of 4 nodes, solrconfig.xml, solr.in.sh, and screenshot of TOP command. The log files is about 1 day’s log. However, I restarted the solr cloud several times during that

Re: SolrCloud performance

Please see inline for my thoughts Deepak "The greatness of a nation can be judged by the way its animals are treated. Please consider stopping the cruelty by becoming a Vegan" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool

Re: SolrCloud performance

On 11/2/2018 1:38 PM, Chuming Chen wrote: I am running a Solr cloud 7.4 with 4 shards and 4 nodes (JVM "-Xms20g -Xmx40g”), each shard has 32 million documents and 32Gbytes in size. A 40GB heap is probably completely unnecessary for an index of that size.  Does each machine have one replica

SolrCloud performance

Hi All, I am running a Solr cloud 7.4 with 4 shards and 4 nodes (JVM "-Xms20g -Xmx40g”), each shard has 32 million documents and 32Gbytes in size. For a given query (I use complexphrase query), typically, the first time it took a couple of seconds to return the first 20 docs. However, for the

Re:Re: solrcloud performance problem

Hi Shawn, Thanks for your input and help. what you just guessed is right, we run solr in jetty by using start.jar, the parms is what I sent to you in my last mail. about GC, I will check it carefully, thanks. -- 发自我的网易邮箱手机智能版 在 2016-05-11 21:32:33，"Shawn Heisey" 写道：

Re: solrcloud performance problem

On 5/10/2016 7:46 PM, lltvw wrote: > the args used to start solr are as following, and upload my screen shot to > http://www.yupoo.com/photos/qzone3927066199/96064170/, please help to take a > look, thanks. > > -DSTOP.PORT=7989 > -DSTOP.KEY= > -DzkHost=node1:2181,node2:2181,node3:2181/solr >

Re:Re: solrcloud performance problem

Hi Shawn, Thanks for your help. the args used to start solr are as following, and upload my screen shot to http://www.yupoo.com/photos/qzone3927066199/96064170/, please help to take a look, thanks. -DSTOP.PORT=7989 -DSTOP.KEY= -DzkHost=node1:2181,node2:2181,node3:2181/solr

Re: solrcloud performance problem

On 5/9/2016 11:42 PM, lltvw wrote: > By using jps command double check the parms used to start solr, i found that > the max heap size already set to 10G. So I made a big mistake yesterday. > > But by using solr admin UI, I select the collection with performance problem, > in the overview page I

Re:Re: Re:Re: solrcloud performance problem

Hi Toke, The version i am using is 4.10, i do not know why by setting log to all, and then recycle solr, i still could not get detailed log info. What is wrong. Does debug info from solr admin UI make sense? -- 发自我的网易邮箱手机智能版 在 2016-05-10 16:25:34，"Toke Eskildsen"

Re: Re:Re: solrcloud performance problem

On Tue, 2016-05-10 at 15:33 +0800, lltvw wrote: > What log you mentioned, console log or something else. the version I am using > is 4.10. There should be a solr.log somewhere. If you have not changed the default log levels, it should log all queries. - Toke Eskildsen, State and University

Re:Re: solrcloud performance problem

Hi Toke, What log you mentioned, console log or something else. the version I am using is 4.10. -- 发自我的网易邮箱手机智能版 在 2016-05-10 14:42:34，"Toke Eskildsen" 写道： >On Tue, 2016-05-10 at 00:41 +0800, lltvw wrote: >> Recently we setup a 4.10 solrcloud env with about

Re: solrcloud performance problem

On Tue, 2016-05-10 at 00:41 +0800, lltvw wrote: > Recently we setup a 4.10 solrcloud env with about 9000 doc indexed > in it，this solrcloud with 12 shards, each shard on one separate > machine, but when we try to search some infor on solrcloud, the > response time is about 300ms. Could you

Re:Re: solrcloud performance problem

Shawn, By using jps command double check the parms used to start solr, i found that the max heap size already set to 10G. So I made a big mistake yesterday. But by using solr admin UI, I select the collection with performance problem, in the overview page I find that the heap memory is about

Re: solrcloud performance problem

On 5/9/2016 9:11 PM, lltvw wrote: > You are right, the max heap is 512MB, thanks. 90 million documents split into 12 shards means 7.5 million documents per shard. With that many documents and a 512MB heap, you're VERY lucky if Solr doesn't experience OutOfMemoryError problems -- which will make

Re:Re: solrcloud performance problem

Hi shawn, You are right, the max heap is 512MB, thanks. -- 发自我的网易邮箱手机智能版 在 2016-05-10 10:02:44，"Shawn Heisey" 写道： >On 5/9/2016 4:41 PM, lltvw wrote: >> Shawn, thanks. >> >> Each machine with 48G memory installed, and now with 20G free, I check JVM >> heap size use

Re: solrcloud performance problem

On 5/9/2016 4:41 PM, lltvw wrote: > Shawn, thanks. > > Each machine with 48G memory installed, and now with 20G free, I check JVM > heap size use solr admin UI, the heap size is about 20M. What is the *max* heap? An unmodified install of Solr 5.x or later has a max heap of 512MB. In the admin

Re:Re: solrcloud performance problem

Shawn, thanks. Each machine with 48G memory installed, and now with 20G free, I check JVM heap size use solr admin UI, the heap size is about 20M. -- 发自我的网易邮箱手机智能版 在 2016-05-10 02:04:22，"Shawn Heisey" 写道： >On 5/9/2016 10:52 AM, lltvw wrote: >> Sorry, I missed the

Re: solrcloud performance problem

On 5/9/2016 10:52 AM, lltvw wrote: > Sorry, I missed the size of each shard, the size is about 3G each. Thanks. > > 在 2016-05-10 00:41:13，lltvw 写道： >> Recently we setup a 4.10 solrcloud env with about 9000 doc indexed in >> it，this solrcloud with 12 shards, each shard on one

solrcloud performance problem

Hi all, Recently we setup a 4.10 solrcloud env with about 9000 doc indexed in it，this solrcloud with 12 shards, each shard on one separate machine, but when we try to search some infor on solrcloud, the response time is about 300ms. Seems that the performance is not good, please advice

Re: Solrcloud performance issues

why you use 15 replicas? more replicas more slower. -- View this message in context: http://lucene.472066.n3.nabble.com/Solrcloud-performance-issues-tp4186035p4188738.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solrcloud performance issues

Hi Vijay, We're working on SOLR-6816 ... would love for you to be a test site for any improvements we make ;-) Curious if you've experimented with changing the mergeFactor to a higher value, such as 25 and what happens if you set soft-auto-commits to something lower like 15 seconds? Also, make

Re: Solrcloud performance issues

Hi, Did you say you have 150 servers in this cluster? And 10 shards for just 90M docs? If so, that 150 hosts sounds like too much for all other numbers I see here. I'd love to see some metrics here. e.g. what happens with disk IO around those commits? How about GC time/size info? Are JVM

Solrcloud performance issues

Hi Erick, We have following configuration of our solr cloud 1. 10 Shards 2. 15 replicas per shard 3. 9 GB of index size per shard 4. a total of around 90 mil documents 5. 2 collection viz search1 serving live traffic and search 2 for indexing. We swap collection when indexing

RE: SolrCloud performance issues regarding hardware configuration

search engn dev [sachinyadav0...@gmail.com] wrote: Yes, You are right my facet queries are for text analytic purpose. Does this mean that facet calls are rare (at most one at a time)? Users will send boolean and spatial queries. current performance for spatial queries is 100qps with 150

Re: SolrCloud performance issues regarding hardware configuration

-- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4148222.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud performance issues regarding hardware configuration

.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4148222.html Sent from the Solr - User mailing list archive at Nabble.com. -- Himanshu Mehrotra Download Our App[image: A] https://play.google.com/store/apps/details?id=com.snapdeal.mainutm_source

SolrCloud performance issues regarding hardware configuration

with 32 gb ram each? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: SolrCloud performance issues regarding hardware configuration

From: search engn dev [sachinyadav0...@gmail.com]: 1 collection : 4 shards : each shard has one master and one replica total documents : 700 million Are you using DocValues for your facet fields? What is the approximate number of unique values in your facets and what is their type (string,

RE: SolrCloud performance issues regarding hardware configuration

.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4147871.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud performance issues regarding hardware configuration

as soon as fire it to solr. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-performance-issues-regarding-hardware-configuration-tp4147843p4147871.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: SolrCloud performance issues regarding hardware configuration

search engn dev [sachinyadav0...@gmail.com] wrote: out of 700 million documents 95-97% values are unique approx. That's quite a lot. If you are not already using DocValues for that, you should do so. So, each shard handles ~175M documents. Even with DocValues, there is an overhead of just

Re: SolrCloud performance in VM environment

Be a bit careful here. 128G is lots of memory, you may encounter very long garbage collection pauses. Just be aware that this may be happening later. Best, Erick On Tue, Oct 22, 2013 at 5:04 PM, Tom Mortimer tom.m.f...@gmail.com wrote: Just tried it with no other changes than upping the RAM

Re: SolrCloud performance in VM environment

Boogie, Shawn, Thanks for the replies. I'm going to try out some of your suggestions today. Although, without more RAM I'm not that optimistic.. Tom On 21 October 2013 18:40, Shawn Heisey s...@elyograg.org wrote: On 10/21/2013 9:48 AM, Tom Mortimer wrote: Hi everyone, I've been working

Re: SolrCloud performance in VM environment

Just tried it with no other changes than upping the RAM to 128GB total, and it's flying. I think that proves that RAM is good. =) Will implement suggested changes later, though. cheers, Tom On 22 October 2013 09:04, Tom Mortimer tom.m.f...@gmail.com wrote: Boogie, Shawn, Thanks for the

Re: SolrCloud Performance Issue

really struggling to figure out the bottleneck, any known pain points I should be checking ? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096277.html Sent from the Solr - User mailing list archive at Nabble.com.

SolrCloud performance in VM environment

Hi everyone, I've been working on an installation recently which uses SolrCloud to index 45M documents into 8 shards on 2 VMs running 64-bit Ubuntu (with another 2 identical VMs set up for replicas). The reason we're using so many shards for a relatively small index is that there are complex

RE: SolrCloud performance in VM environment

some sort of MPIO in place (esp if you are using 1GB iscsi) From: Tom Mortimer tom.m.f...@gmail.com Sent: Monday, October 21, 2013 08:48 To: solr-user@lucene.apache.org Subject: SolrCloud performance in VM environment Hi everyone, I've been working

Re: SolrCloud performance in VM environment

On 10/21/2013 9:48 AM, Tom Mortimer wrote: Hi everyone, I've been working on an installation recently which uses SolrCloud to index 45M documents into 8 shards on 2 VMs running 64-bit Ubuntu (with another 2 identical VMs set up for replicas). The reason we're using so many shards for a

Re: SolrCloud Performance Issue

/SolrCloud-Performance-Issue-tp4095971p4096277.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud Performance Issue

Query result cache hit might be low due to using NOW in bf. NOW is always translated to current time and that of course changes from ms to ms... :) Primoz From: Shamik Bandopadhyay sham...@gmail.com To: solr-user@lucene.apache.org Date: 17.10.2013 00:14 Subject:SolrCloud

Re: SolrCloud Performance Issue

://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096123.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud Performance Issue

: http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095971p4096277.html Sent from the Solr - User mailing list archive at Nabble.com.

SolrCloud Performance Issue

: http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Issue-tp4095940.html Sent from the Solr - User mailing list archive at Nabble.com.

SolrCloud Performance Issue

Hi, I'm in the process of transitioning to SolrCloud from a conventional Master-Slave model. I'm using Solr 4.4 and has set-up 2 shards with 1 replica each. I've 3 zookeeper ensemble. All the nodes are running on AWS EC2 instances. Shards are on m1.xlarge and sharing a zookeeper instance

Re: SolrCloud Performance for High Query Volume

: Thursday, January 17, 2013 10:12 AM Subject: Re: SolrCloud Performance for High Query Volume Hello Niran,  Now with the roughly the same schema and solrconfig configuration Can you be more specific about what was changed and how?  * 4 Solr server instances each with 4 CPUs (each 6 cores

SolrCloud Performance for High Query Volume

Hi all, I'm currently in the process of doing some performance testing in preparations for upgrading from Solr 3.6.1 to Solr 4.0. (We're badly in need of NRT functionality) Our existing deployment is not a typical deployment for Solr, as we use it to search and facet on financial data such as

Re: SolrCloud Performance - Indexing

... Calissa yapar... -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Performance-Indexing-tp4022549.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud Performance - Indexing

/SolrCloud-Performance-Indexing-tp4022549.html Sent from the Solr - User mailing list archive at Nabble.com. -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com

Re: SolrCloud Performance - Indexing

.472066.n3.nabble.com/SolrCloud-Performance-Indexing-tp4022549.html Sent from the Solr - User mailing list archive at Nabble.com. -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com

SolrCloud Performance - Indexing

, but for a frequently updated system, does it still useful to set up a cloud environment? and are there some workarounds for indexing speed, other than the known ones for solr, on cloud? - Zeki ama calismiyor... Calissa yapar... -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud