RE: Solr Cloud 5.3.0 Read Time Outs
Thanks Shalin and Eric. That was helpful. I will try reducing the commit frequency and let you know how it goes. Best regards, Adrian -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Sunday, December 13, 2015 2:42 AM To: solr-user <solr-user@lucene.apache.org> Subject: Re: Solr Cloud 5.3.0 Read Time Outs +1 to what Shalin said. You've adjusted maxWarmingSeachers up, probably because you saw warnings in the log files. This is _not_ the solution to the "MaxWarmingSearchers exceeded" error. The solution is, as Shalin says, decrease your commit frequency. Commit can be an expensive operation, see: https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Best, Erick On Sat, Dec 12, 2015 at 9:54 AM, Shalin Shekhar Mangar <shalinman...@gmail.com> wrote: > Yes, that is probably the cause. I think you have very aggressive > commit rates and Solr is not able to keep up. If you are sending > explicit commits, switch to using autoCommit with openSearcher=false > every 5-10 minutes (this depends on your indexing rate) and > autoSoftCommit every 2-5 minutes. Adjust as necessary. > > On Sat, Dec 12, 2015 at 10:08 PM, Adrian Liew <adrian.l...@avanade.com> wrote: >> Hi there, >> >> I am using Solr Cloud 5.3.0 on a multiserver cluster (3 servers to mention) >> whereby each server spec is at 16 core and 32 GB Ram. >> >> I am facing regular errors - Error sending update to >> http://someip:8983/solr - "Timeout occured while waiting response from >> server at server a" ... Caused by java.net.SocketTimeoutException: Read >> Timed out. >> >> I am not sure if this error can be caused due to some preceding >> warnings reported such as >> >> Error sending update to http://someip:8983/solr - >> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: >> Error from server at >> http://someip:8983/solr/sitecore_master_index_shard1_replica3: Error >> opening new searcher. exceeded limit of maxWarmingSearchers=6 >> >> Can the maxWarmingSearchers error possibly cause the read timeouts to occur? >> If yes, when maxWarmingSearchers warning is addressed, will that remove the >> errors for the read timeouts? >> >> Best regards, >> Adrian >> > > > > -- > Regards, > Shalin Shekhar Mangar.
Solr Cloud 5.3.0 Read Time Outs
Hi there, I am using Solr Cloud 5.3.0 on a multiserver cluster (3 servers to mention) whereby each server spec is at 16 core and 32 GB Ram. I am facing regular errors - Error sending update to http://someip:8983/solr - "Timeout occured while waiting response from server at server a" ... Caused by java.net.SocketTimeoutException: Read Timed out. I am not sure if this error can be caused due to some preceding warnings reported such as Error sending update to http://someip:8983/solr - org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://someip:8983/solr/sitecore_master_index_shard1_replica3: Error opening new searcher. exceeded limit of maxWarmingSearchers=6 Can the maxWarmingSearchers error possibly cause the read timeouts to occur? If yes, when maxWarmingSearchers warning is addressed, will that remove the errors for the read timeouts? Best regards, Adrian
RE: SolrCloud Shard + Replica on Multiple servers with SolrCloud
Hi all, Will really like to seek anyone's opinion on my query below. Desperate to know if this is possible or if someone is keen to share their thought experience. Best regards, Adrian -Original Message- From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Saturday, November 28, 2015 10:38 AM To: solr-user@lucene.apache.org Subject: RE: SolrCloud Shard + Replica on Multiple servers with SolrCloud Hi Upaya, I am trying to setup a 3 shard 3 server setup with a replication factor of 2 with SolrCloud 5.3.0. In particular trying to follow this setup described in this blog: http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolkit/ Correction to description below: EC2 Instance 1 Shard 1 - Leader (port 8984 separate drive with 50 GB SSD) Shard 2 - Leader (port 8985 separate drive with 50 GB SSD) - Leader (port 8986 separate drive with 50 GB SSD) EC2 Instance 2 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) - Replica (port 8986 separate drive with 50 GB SSD) EC2 Instance 3 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) - Replica (port 8986 separate drive with 50 GB SSD) To your questions >> Why are you running multiple instances on the same host? This was the architecture best practice provided by Lucidworks. For more info, you can visit this site, http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolkit/ >> You can host your two replicas inside the same Solr instance. I reckon because this avoids the probability of a single shard (its leader and replicas) going down in one hit. What happens if on node that holds one shard goes down altogether? You will lose a chunk of your index. The architecture I mentioned above prevents that from happening. I will want my shards to be spread out for HA. >> Also, you should not concern yourself (too much) with which node is the >> leader as that can change through time. I am not concerned as I know this setup will guarantee a leader is in place for each shard for fault tolerance situation. >> How have you come to the conclusion that you need to shard? I am preparing a use case for my customer. Haven't arrived yet as to when to shard. But I need to setup a demo to show to my customer. I am proposing this as an architecture for the long term to them. > As I know there are two approaches to sharding that is "Custom Sharding" > and "Automatic Sharding". Which approach suits the use case described > above? Do you know this answer? Do you also have your own opinion on setting up a 3 shard 3 server cluster? Regards, Adrian -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Friday, November 27, 2015 9:09 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud Shard + Replica on Multiple servers with SolrCloud Why are you running multiple instances on the same host? You can host your two replicas inside the same Solr instance. Also, you should not concern yourself (too much) with which node is the leader as that can change through time. How have you come to the conclusion that you need to shard? Upayavira On Fri, Nov 27, 2015, at 10:23 AM, Adrian Liew wrote: > Hi all, > > I am trying to figure out how to setup 3 shard 3 server setup with a > replication factor of 2 with SolrCloud 5.3.0. > > In particular trying to follow this setup described in this blog: > http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolk > it/ > > EC2 Instance 1 > > Shard 1 - Leader (port 8984 separate drive with 50 GB SSD) Shard 2 - > Leader (port 8985 separate drive with 50 GB SSD) > > EC2 Instance 2 > > Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - > Replica (port 8985 separate drive with 50 GB SSD) > > EC2 Instance 3 > > Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - > Replica (port 8985 separate drive with 50 GB SSD) > > Can anyone shed some light on how these can be configured using the > SolrCloud collection API or using Solr command line utility to split > them on different instances. > > As I know there are two approaches to sharding that is "Custom Sharding" > and "Automatic Sharding". Which approach suits the use case described > above? > > Is anyone able to provide pointers from past experience or point me to > a good article that describes how this can be setup? > > Regards, > Adrian >
RE: SolrCloud Shard + Replica on Multiple servers with SolrCloud
Hi Upaya, I am trying to setup a 3 shard 3 server setup with a replication factor of 2 with SolrCloud 5.3.0. In particular trying to follow this setup described in this blog: http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolkit/ Correction to description below: EC2 Instance 1 Shard 1 - Leader (port 8984 separate drive with 50 GB SSD) Shard 2 - Leader (port 8985 separate drive with 50 GB SSD) - Leader (port 8986 separate drive with 50 GB SSD) EC2 Instance 2 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) - Replica (port 8986 separate drive with 50 GB SSD) EC2 Instance 3 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) - Replica (port 8986 separate drive with 50 GB SSD) To your questions >> Why are you running multiple instances on the same host? This was the architecture best practice provided by Lucidworks. For more info, you can visit this site, http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolkit/ >> You can host your two replicas inside the same Solr instance. I reckon because this avoids the probability of a single shard (its leader and replicas) going down in one hit. What happens if on node that holds one shard goes down altogether? You will lose a chunk of your index. The architecture I mentioned above prevents that from happening. I will want my shards to be spread out for HA. >> Also, you should not concern yourself (too much) with which node is the >> leader as that can change through time. I am not concerned as I know this setup will guarantee a leader is in place for each shard for fault tolerance situation. >> How have you come to the conclusion that you need to shard? I am preparing a use case for my customer. Haven't arrived yet as to when to shard. But I need to setup a demo to show to my customer. I am proposing this as an architecture for the long term to them. > As I know there are two approaches to sharding that is "Custom Sharding" > and "Automatic Sharding". Which approach suits the use case described > above? Do you know this answer? Do you also have your own opinion on setting up a 3 shard 3 server cluster? Regards, Adrian -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Friday, November 27, 2015 9:09 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud Shard + Replica on Multiple servers with SolrCloud Why are you running multiple instances on the same host? You can host your two replicas inside the same Solr instance. Also, you should not concern yourself (too much) with which node is the leader as that can change through time. How have you come to the conclusion that you need to shard? Upayavira On Fri, Nov 27, 2015, at 10:23 AM, Adrian Liew wrote: > Hi all, > > I am trying to figure out how to setup 3 shard 3 server setup with a > replication factor of 2 with SolrCloud 5.3.0. > > In particular trying to follow this setup described in this blog: > http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolk > it/ > > EC2 Instance 1 > > Shard 1 - Leader (port 8984 separate drive with 50 GB SSD) Shard 2 - > Leader (port 8985 separate drive with 50 GB SSD) > > EC2 Instance 2 > > Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - > Replica (port 8985 separate drive with 50 GB SSD) > > EC2 Instance 3 > > Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - > Replica (port 8985 separate drive with 50 GB SSD) > > Can anyone shed some light on how these can be configured using the > SolrCloud collection API or using Solr command line utility to split > them on different instances. > > As I know there are two approaches to sharding that is "Custom Sharding" > and "Automatic Sharding". Which approach suits the use case described > above? > > Is anyone able to provide pointers from past experience or point me to > a good article that describes how this can be setup? > > Regards, > Adrian >
SolrCloud Shard + Replica on Multiple servers with SolrCloud
Hi all, I am trying to figure out how to setup 3 shard 3 server setup with a replication factor of 2 with SolrCloud 5.3.0. In particular trying to follow this setup described in this blog: http://lucidworks.com/blog/2014/06/03/introducing-the-solr-scale-toolkit/ EC2 Instance 1 Shard 1 - Leader (port 8984 separate drive with 50 GB SSD) Shard 2 - Leader (port 8985 separate drive with 50 GB SSD) EC2 Instance 2 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) EC2 Instance 3 Shard 1 - Replica (port 8984 separate drive with 50 GB SSD) Shard 2 - Replica (port 8985 separate drive with 50 GB SSD) Can anyone shed some light on how these can be configured using the SolrCloud collection API or using Solr command line utility to split them on different instances. As I know there are two approaches to sharding that is "Custom Sharding" and "Automatic Sharding". Which approach suits the use case described above? Is anyone able to provide pointers from past experience or point me to a good article that describes how this can be setup? Regards, Adrian
RE: Solr Cloud 5.3.0 Errors in Logs
Hi Emir, I am working with a third party platform, Sitecore. The product is a black box that encapsulates the internal workings of solr queries and so on. If there are any questions you have with regards with the below, let me know. It will be useful for me to communicate what could cause the issues below. Regards, Adrian -Original Message- From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com] Sent: Monday, November 16, 2015 4:47 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cloud 5.3.0 Errors in Logs Hi Adrian, Can you give us bit more details about warmup queries you use and test that you are running when error occurs. Thanks, Emir On 16.11.2015 08:40, Adrian Liew wrote: > Hi there, > > Will like to get some opinions on the errors encountered below. I have > currently setup a SolrCloud cluster of 3 servers (each server hosting a Solr > instance and a Zookeeper instance). > > I am encountering the errors below in the logs: > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore > org.apache.solr.common.SolrException: Error opening new searcher. exceeded > limit of maxWarmingSearchers=6, try again later. > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore > org.apache.solr.common.SolrException: Error opening new searcher. exceeded > limit of maxWarmingSearchers=6, try again later. > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore > org.apache.solr.common.SolrException: Error opening new searcher. exceeded > limit of maxWarmingSearchers=6, try again later. > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore > org.apache.solr.common.SolrException: Error opening new searcher. exceeded > limit of maxWarmingSearchers=6, try again later. > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at > http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error > opening new searcher. exceeded limit of maxWarmingSearchers=6, try again > later. > Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at > http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error > opening new searcher. exceeded limit of maxWarmingSearchers=6, try again > later. > Monday, November 16, 2015 3:22:54 PM WARN null > DistributedUpdateProcessor Error sending update to > http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM > WARN null DistributedUpdateProcessor Error sending update to > http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM > WARN null DistributedUpdateProcessor Error sending update to > http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM > WARN null DistributedUpdateProcessor Error sending update to > http://172.18.111.112:8983/solr > > 11/16/2015, 3:17:09 PM > > WARN > > null > > DistributedUpdateProcessor > > Error sending update to http://172.18.111.112:8983/solr > > 11/16/2015, 3:17:09 PM > > WARN > > null > > DistributedUpdateProcessor > > Error sending update to http://172.18.111.112:8983/solr > > 11/16/2015, 3:22:26 PM > > ERROR > > null > > SolrCmdDistributor > > org.apache.solr.client.solrj.SolrServerException: Timeout occured > while waiting response from server at: > http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1 > > > > Main errors are Timeout occurred exceptions, maxWarmingSearchers exceeded. Is > anyone able to advise or have experienced something the same as the above in > their SolrCloud setup? > > Regards, > Adrian > > > -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/
RE: Solr Cloud 5.3.0 Errors in Logs
Thanks Eric. Here is my reply >> 1> that commits are happening too often, and especially if the commits >> are happening >> from a client. If commits aren't being sent by a client, then look at >> autoCommit and softAutoCommit in solrconfig.xml (if you can). Understand what you mean. Besides talking to to the folks at Sitecore on where they issue commits, is there a way I can balance these with autoCommit and softAutoCommit in solrconfig.xml? Best is, can you recommend any articles that talk about best practice configuration for a production setup. >> 2> excessive autowarm settings, again in solronfig.xml. >> If, as you say all of Solr is a black box, then talk to the Sitecore folks, >> on the surface Solr is just poorly configured. I will raise with Sitecore guys on this. Particularly asking them why commits are happening faster than searchers can be opened. I have seen overlappingDeckSearchers have exceeded the limit errors as well. Let you know what I get back from the guys. Regards, Adrian -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, November 17, 2015 12:44 AM To: solr-user <solr-user@lucene.apache.org> Subject: Re: Solr Cloud 5.3.0 Errors in Logs Having 6 warming serachers is an anti-pattern. What it means is that commits are happening faster than your searcher can be opened. There is _no_ good reason that I know of for changing it from 2, having changed it in solrconfig.xml to 6 almost always indicates an improper configuration. Places to look: 1> that commits are happening too often, and especially if the commits are happening from a client. If commits aren't being sent by a client, then look at autoCommit and softAutoCommit in solrconfig.xml (if you can). 2> excessive autowarm settings, again in solronfig.xml. If, as you say all of Solr is a black box, then talk to the Sitecore folks, on the surface Solr is just poorly configured. Best, Erick On Mon, Nov 16, 2015 at 4:33 AM, Adrian Liew <adrian.l...@avanade.com> wrote: > Hi Emir, > > I am working with a third party platform, Sitecore. The product is a black > box that encapsulates the internal workings of solr queries and so on. If > there are any questions you have with regards with the below, let me know. It > will be useful for me to communicate what could cause the issues below. > > Regards, > Adrian > > -Original Message- > From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com] > Sent: Monday, November 16, 2015 4:47 PM > To: solr-user@lucene.apache.org > Subject: Re: Solr Cloud 5.3.0 Errors in Logs > > Hi Adrian, > Can you give us bit more details about warmup queries you use and test that > you are running when error occurs. > > Thanks, > Emir > > On 16.11.2015 08:40, Adrian Liew wrote: >> Hi there, >> >> Will like to get some opinions on the errors encountered below. I have >> currently setup a SolrCloud cluster of 3 servers (each server hosting a Solr >> instance and a Zookeeper instance). >> >> I am encountering the errors below in the logs: >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore >> org.apache.solr.common.SolrException: Error opening new searcher. exceeded >> limit of maxWarmingSearchers=6, try again later. >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore >> org.apache.solr.common.SolrException: Error opening new searcher. exceeded >> limit of maxWarmingSearchers=6, try again later. >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore >> org.apache.solr.common.SolrException: Error opening new searcher. exceeded >> limit of maxWarmingSearchers=6, try again later. >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore >> org.apache.solr.common.SolrException: Error opening new searcher. exceeded >> limit of maxWarmingSearchers=6, try again later. >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor >> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error >> from server at >> http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error >> opening new searcher. exceeded limit of maxWarmingSearchers=6, try again >> later. >> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor >> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error >> from server at >> http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error >> opening new searcher. exceeded limit of maxWarmingSearchers=6, try again >> later. >> Monday, November 16, 2015 3:22:54 PM WARN null >> DistributedUpdateProcessor Error sending update to >> http://172.18.111.112:8983/solr Monday, Nov
RE: Solr Cloud 5.3.0 Errors in Logs
Thanks for the tip Eric. Really useful article to know. I will keep you posted on my findings! Regards, Adrian -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, November 17, 2015 8:56 AM To: solr-user <solr-user@lucene.apache.org> Subject: Re: Solr Cloud 5.3.0 Errors in Logs Here's perhaps more than you really want to know about commits https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ The short form is that most setups set autoCommit to a relatively short interval (15-60 seconds at least under heavy indexing loads, I've seen 10-15 minutes under relatively light loads) with openSearcher set to false. And I rarely set maxDocs in that configuration, it's actually not that useful IMO. Then set up an autoSoftCommit to be as long as you can tolerate, but IMO rarely shorter than 60 seconds unless you have very aggressive near-real-time (NRT) requirements. And if your product manager simply insists on very aggressive NRT settings, you should consider making your filterCache and queryResultCache relatively small with minimal autowarming. Any time you exceed maxWarmingSearchers, it indicates poorly configured Solr instances and/or commits happening far too often. Bumping that up to numbers greater than two is almost always a band-aid over that misconfiguration. On Mon, Nov 16, 2015 at 3:45 PM, Adrian Liew <adrian.l...@avanade.com> wrote: > Thanks Eric. > > Here is my reply > >>> 1> that commits are happening too often, and especially if the >>> 1> commits >>> are happening >>> from a client. If commits aren't being sent by a client, then look at >>> autoCommit and softAutoCommit in solrconfig.xml (if you can). > Understand what you mean. Besides talking to to the folks at Sitecore on > where they issue commits, is there a way I can balance these with autoCommit > and softAutoCommit in solrconfig.xml? Best is, can you recommend any articles > that talk about best practice configuration for a production setup. > >>> 2> excessive autowarm settings, again in solronfig.xml. >>> If, as you say all of Solr is a black box, then talk to the Sitecore folks, >>> on the surface Solr is just poorly configured. > I will raise with Sitecore guys on this. Particularly asking them why commits > are happening faster than searchers can be opened. I have seen > overlappingDeckSearchers have exceeded the limit errors as well. Let you know > what I get back from the guys. > > Regards, > Adrian > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, November 17, 2015 12:44 AM > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: Solr Cloud 5.3.0 Errors in Logs > > Having 6 warming serachers is an anti-pattern. What it means is that commits > are happening faster than your searcher can be opened. There is _no_ good > reason that I know of for changing it from 2, having changed it in > solrconfig.xml to 6 almost always indicates an improper configuration. > > Places to look: > 1> that commits are happening too often, and especially if the commits > are happening > from a client. If commits aren't being sent by a client, then look at > autoCommit and softAutoCommit in solrconfig.xml (if you can). > > 2> excessive autowarm settings, again in solronfig.xml. > > If, as you say all of Solr is a black box, then talk to the Sitecore folks, > on the surface Solr is just poorly configured. > > Best, > Erick > > On Mon, Nov 16, 2015 at 4:33 AM, Adrian Liew <adrian.l...@avanade.com> wrote: >> Hi Emir, >> >> I am working with a third party platform, Sitecore. The product is a black >> box that encapsulates the internal workings of solr queries and so on. If >> there are any questions you have with regards with the below, let me know. >> It will be useful for me to communicate what could cause the issues below. >> >> Regards, >> Adrian >> >> -Original Message- >> From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com] >> Sent: Monday, November 16, 2015 4:47 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Solr Cloud 5.3.0 Errors in Logs >> >> Hi Adrian, >> Can you give us bit more details about warmup queries you use and test that >> you are running when error occurs. >> >> Thanks, >> Emir >> >> On 16.11.2015 08:40, Adrian Liew wrote: >>> Hi there, >>> >>> Will like to get some opinions on the errors encountered below. I have >>> currently setup a SolrCloud cluster of 3 servers (each server hosting a >>> Sol
Solr Cloud 5.3.0 Errors in Logs
Hi there, Will like to get some opinions on the errors encountered below. I have currently setup a SolrCloud cluster of 3 servers (each server hosting a Solr instance and a Zookeeper instance). I am encountering the errors below in the logs: Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1: Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later. Monday, November 16, 2015 3:22:54 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr 11/16/2015, 3:17:09 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr 11/16/2015, 3:17:09 PM WARN null DistributedUpdateProcessor Error sending update to http://172.18.111.112:8983/solr 11/16/2015, 3:22:26 PM ERROR null SolrCmdDistributor org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1 Main errors are Timeout occurred exceptions, maxWarmingSearchers exceeded. Is anyone able to advise or have experienced something the same as the above in their SolrCloud setup? Regards, Adrian
RE: Solr.cmd cannot create collection in Solr 5.2.1
Hi Shawn,
You are correct. It works when we don't include the conf folder. I guess it
just something to take note of when uploading solrconfig.xml to Zookeeper.
Thanks for the tip.
- Adrian
-Original Message-
From: Shawn Heisey [mailto:apa...@elyograg.org]
Sent: Tuesday, October 27, 2015 12:11 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr.cmd cannot create collection in Solr 5.2.1
On 10/26/2015 2:23 AM, Adrian Liew wrote:
> {
> "responseHeader":{
> "status":0,
> "QTime":1735},
>
> "failure":{"":"org.apache.solr.client.solrj.impl.HttpSolrClient$Remote
> SolrExce ption:Error from server at http://172.18.111.112:8983/solr:
> Error CREATEing Solr Core 'sitecore_core_index_shard1_replica2':
> Unable to create core [sitecore_core _index_shard1_replica2] Caused
> by: Can't find resource 'solrconfig.xml' in class path or
> '/configs/sitecore_common_config',
> cwd=D:\\Solr-5.2.1-Instance\\server"}
> }
>
> I do a check to see if solrconfig.xml is present in the Zookeeper, if I run
> zkCli.bat -cmd list on the each of the server, I can see that solrconfig.xml
> is listed:
>
> DATA:
>
> /configs (1)
> /configs/sitecore_common_config (1)
>/configs/sitecore_common_config/conf (8)
> /configs/sitecore_common_config/conf/currency.xml (0)
I think the problem is that you included the conf directory in what you
uploaded to zookeeper. The config files (solrconfig.xml, schema.xml,
etc) should be sitting right in the directory you upload, not inside a conf
subdirectory. This is somewhat counterintuitive when compared to what happens
when NOT running in cloud mode, but the logic is fairly
simple: The conf directory is what gets uploaded to zookeeper.
A question for fellow committers: Is it too much handholding for us to look in
a conf directory in zookeeper? My bias is that we should not do that, but I do
not see it as particularly harmful.
Thanks,
Shawn
Solr.cmd cannot create collection in Solr 5.2.1
Hi all,
I have setup a 3 server Zookeeper cluster by following the instructions
provided from Zookeeper site:
I am having experiences trying to zkCli.bat into the Zookeeper services on 3
EC2 instances once after I have started the ZK services on all 3 servers.
For example, I have setup my three servers to have the IPs:
Server1 - 172.18.111.111:2181
Server2 - 172.18.111.112:2182
Server3 - 172.18.112.112:2183
I am using Solr v.5.2.1.
When I have started Solr to run with the Zookeeper services and attempting to
upload the configuration to Zookeeper, I get the following failure message
reported by solr.cmd when attempting to create a collection using solr.cmd.
D:\Solr-5.2.1-Instance\bin>solr.cmd create_collection -c sitecore_core_index -n
sitecore_common_config -shards 1 -replicationFactor 3
Connecting to ZooKeeper at 172.18.111.111:2181,172.18.111.112:2182,172.18.112.11
2:2183
Re-using existing configuration directory sitecore_common_config
Creating new collection 'sitecore_core_index' using command:
http://172.18.112.112:8983/solr/admin/collections?action=CREATE=sitecore_co
re_index=1=3=1
e=sitecore_common_config
{
"responseHeader":{
"status":0,
"QTime":1735},
"failure":{"":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrExce
ption:Error from server at http://172.18.111.112:8983/solr: Error CREATEing Solr
Core 'sitecore_core_index_shard1_replica2': Unable to create core [sitecore_core
_index_shard1_replica2] Caused by: Can't find resource 'solrconfig.xml' in class
path or '/configs/sitecore_common_config', cwd=D:\\Solr-5.2.1-Instance\\server"}
}
I do a check to see if solrconfig.xml is present in the Zookeeper, if I run
zkCli.bat -cmd list on the each of the server, I can see that solrconfig.xml is
listed:
DATA:
/configs (1)
/configs/sitecore_common_config (1)
/configs/sitecore_common_config/conf (8)
/configs/sitecore_common_config/conf/currency.xml (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/protwords.txt (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/solrconfig.xml (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/synonyms.txt (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/stopwords.txt (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/schema.xml (0)
DATA: ...supressed...
/configs/sitecore_common_config/conf/_rest_managed.json (0)
DATA:
{"initArgs":{},"managedList":[]}
/configs/sitecore_common_config/conf/lang (1)
/configs/sitecore_common_config/conf/lang/stopwords_en.txt (0)
DATA: ...supressed...
/zookeeper (1)
DATA:
/overseer (6)
DATA:
Has anyone come across this issue before for Solr 5.2.1?
Regards,
Adrian
RE: Run Solr 5.3.0 as a Service on Windows using NSSM
Hi Edwin, Solr 5.3.0 seems to be working for me using NSSM. I am operating on a Windows Server 2012. I did put start -f -p 8983. Are you getting errors? Is Solr not starting up? Best regards, Adrian -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Thursday, October 15, 2015 11:13 AM To: solr-user@lucene.apache.org Subject: Re: Run Solr 5.3.0 as a Service on Windows using NSSM Hi Anders, Yes, I did put the -f param for running it in foreground. I put start -f -p 8983 in the Arugments parameters in NSSM service installer. Is that the correct place to put for Solr 5.3.0? I did the same way for Solr 5.1 and it was working then. I'm using Windows 8.1. Regards, Edwin On 14 October 2015 at 23:44, Anders Thulin <anders.thu...@comintelli.com> wrote: > Did you add the f param for running it in foreground? > I noticed that the Solr service was restarted indefinetly when running > it as a background service. > its also needed to stop the windows service. > > This test worked well here (on Windows 2012): > > REM Test for running solr 5.3.1 as a windows service > C:\nssm\nssm64.exe install "Solr 5.3.1" > C:\search\solr-5.3.1\bin\solr.cmd "start -f -p 8983" > > On 8 October 2015 at 04:34, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > > > Hi Adrian and Upayavira, > > > > It works fine when I start Solr outside NSSM. > > As for the NSSM, so far I haven't tried the automatic startup yet. I > start > > the services for ZooKeeper and Solr in NSSM manually from the > > Windows Component Services, so the ZooKeeper will have been started > > before I > start > > Solr. > > > > I'll also try to write the script for Solr that can check it can > > access Zookeeper before attempting to start Solr. > > > > Regards, > > Edwin > > > > > > On 7 October 2015 at 19:16, Upayavira <u...@odoko.co.uk> wrote: > > > > > Wrap your script that starts Solr with one that checks it can > > > access Zookeeper before attempting to start Solr, that way, once > > > ZK starts, Solr will come up. Then, hand *that* script to NSSM. > > > > > > And finally, when one of you has got a setup that works with NSSM > > > starting Solr via the default bin\solr.cmd script, create a patch > > > and upload it to JIRA. It would be a valuable thing for Solr to > > > have a > > > *standard* way to start Solr on Windows as a service. I recall > > > checking the NSSM license and it wouldn't be an issue to include > > > it within Solr > - > > > or to have a script that assumes it is installed. > > > > > > Upayavira > > > > > > On Wed, Oct 7, 2015, at 11:49 AM, Adrian Liew wrote: > > > > Hi Edwin, > > > > > > > > You may want to try explore some of the configuration properties > > > > to configure in zookeeper. > > > > > > > > > > > > > > http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html#sc_zkMulitS > erverSetup > > > > > > > > My recommendation is to try run your batch files outside of NSSM > > > > so > it > > is > > > > easier to debug and observe what you see from the command > > > > window. I > > don't > > > > think ZK and Solr can be automated on startup well using NSSM > > > > due to > > the > > > > fact that ZK services need to be running before you start up > > > > Solr services. I just had conversation with Shawn on this topic. > > > > NSSM > cannot > > > > do the magic startup in a cluster setup. In that, you may need > > > > to > write > > > > custom scripting to get it right. > > > > > > > > Back to your original issue, I guess it is worth exploring > > > > timeout values. Then again, I will leave the real Solr experts > > > > to chip in > their > > > > thoughts. > > > > > > > > Best regards, > > > > > > > > Adrian Liew > > > > > > > > > > > > -Original Message- > > > > From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] > > > > Sent: Wednesday, October 7, 2015 1:40 PM > > > > To: solr-user@lucene.apache.org > > > > Subject: Re: Run Solr 5.3.0 as a Service on Windows using NSSM > > > > > > > > Hi Adrian, > > > > > > > > I've waited for more than 5 minutes and most of the time when I > refresh > > &g
Autostart Zookeeper and Solr using scripting
Hi, I am trying to implement some scripting to detect if all Zookeepers have started in a cluster, then restart the solr servers. Has anyone achieved this yet through scripting? I also saw there is the ZookeeperClient that is available in .NET via a nuget package. Not sure if this could be also implemented to check if a zookeeper is running. Any thoughts on anyone using a script to perform this? Regards, Adrian
RE: Run Solr 5.3.0 as a Service on Windows using NSSM
Hi, I am trying to implement some scripting to detect if all Zookeepers have started in a cluster, then restart the solr servers. Has anyone achieved this yet through scripting? I also saw there is the ZookeeperClient that is available in .NET via a nuget package. Not sure if this could be also implemented to check if a zookeeper is running. Any thoughts? Regards, Adrian -Original Message- From: Anders Thulin [mailto:anders.thu...@comintelli.com] Sent: Wednesday, October 14, 2015 11:44 PM To: solr-user@lucene.apache.org Subject: Re: Run Solr 5.3.0 as a Service on Windows using NSSM Did you add the f param for running it in foreground? I noticed that the Solr service was restarted indefinetly when running it as a background service. its also needed to stop the windows service. This test worked well here (on Windows 2012): REM Test for running solr 5.3.1 as a windows service C:\nssm\nssm64.exe install "Solr 5.3.1" C:\search\solr-5.3.1\bin\solr.cmd "start -f -p 8983" On 8 October 2015 at 04:34, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: > Hi Adrian and Upayavira, > > It works fine when I start Solr outside NSSM. > As for the NSSM, so far I haven't tried the automatic startup yet. I > start the services for ZooKeeper and Solr in NSSM manually from the > Windows Component Services, so the ZooKeeper will have been started > before I start Solr. > > I'll also try to write the script for Solr that can check it can > access Zookeeper before attempting to start Solr. > > Regards, > Edwin > > > On 7 October 2015 at 19:16, Upayavira <u...@odoko.co.uk> wrote: > > > Wrap your script that starts Solr with one that checks it can access > > Zookeeper before attempting to start Solr, that way, once ZK starts, > > Solr will come up. Then, hand *that* script to NSSM. > > > > And finally, when one of you has got a setup that works with NSSM > > starting Solr via the default bin\solr.cmd script, create a patch > > and upload it to JIRA. It would be a valuable thing for Solr to have > > a > > *standard* way to start Solr on Windows as a service. I recall > > checking the NSSM license and it wouldn't be an issue to include it > > within Solr - or to have a script that assumes it is installed. > > > > Upayavira > > > > On Wed, Oct 7, 2015, at 11:49 AM, Adrian Liew wrote: > > > Hi Edwin, > > > > > > You may want to try explore some of the configuration properties > > > to configure in zookeeper. > > > > > > > > > http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html#sc_zkMulitS > erverSetup > > > > > > My recommendation is to try run your batch files outside of NSSM > > > so it > is > > > easier to debug and observe what you see from the command window. > > > I > don't > > > think ZK and Solr can be automated on startup well using NSSM due > > > to > the > > > fact that ZK services need to be running before you start up Solr > > > services. I just had conversation with Shawn on this topic. NSSM > > > cannot do the magic startup in a cluster setup. In that, you may > > > need to write custom scripting to get it right. > > > > > > Back to your original issue, I guess it is worth exploring timeout > > > values. Then again, I will leave the real Solr experts to chip in > > > their thoughts. > > > > > > Best regards, > > > > > > Adrian Liew > > > > > > > > > -Original Message- > > > From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] > > > Sent: Wednesday, October 7, 2015 1:40 PM > > > To: solr-user@lucene.apache.org > > > Subject: Re: Run Solr 5.3.0 as a Service on Windows using NSSM > > > > > > Hi Adrian, > > > > > > I've waited for more than 5 minutes and most of the time when I > > > refresh it says that the page cannot be found. Got one or twice > > > the main Admin page is loaded, but none of the cores are loaded. > > > > > > I have 20 cores which I'm loading. The core are of various sizes, > > > but > the > > > maximum one is 38GB. Others ranges from 10GB to 15GB, and there're > > > some which are less than 1GB. > > > > > > My overall core size is about 200GB. > > > > > > Regards, > > > Edwin > > > > > > > > > On 7 October 2015 at 12:11, Adrian Liew <adrian.l...@avanade.com> > wrote: > > > > > > > Hi Edwin, > > > > > > > > I have setup NSSM on Solr 5.3
RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi Shawn, To reiterate, this is the exception I get if unable to connect to Zookeeper service: E:\solr-5.3.0\server\scripts\cloud-scripts>zkcli.bat -z 10.0.0.4:2181 -cmd list Exception in thread "main" org.apache.solr.common.SolrException: java.util.concu rrent.TimeoutException: Could not connect to ZooKeeper 10.0.0.4:2181 within 3000 0 ms at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:18 1) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:11 5) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:10 5) at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:181) Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 10.0.0.4:2181 within 3 ms at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(Conne ctionManager.java:208) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:17 3) ... 3 more For example, in the event if one of the zookeeper services goes down for a few minutes, it may be too late to bring that service back online into the zookeeper cluster due the timeout faced above. In that, all zookeeper services need to be restarted at the same time. Please clarify if there is a configuration that I missed out, an expected behaviour or if this is a bug. Regards, Adrian -Original Message- From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Wednesday, October 7, 2015 11:56 AM To: solr-user@lucene.apache.org Subject: RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd Hi Shawn, Thanks for the reply. Understood your comments and will revert back to the defaults. However, I raised this issue because I realized that Zookeeper becomes impatient if it cannot heartbeat its other peers in time. So for example, if 1 ZK server goes down out of 3 ZK servers, the 1 ZK server will stop pinging other servers and complain about timeout issues to zkCli connect to its service. Will revert back with an update. Regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Tuesday, October 6, 2015 10:16 PM To: solr-user@lucene.apache.org Subject: Re: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd On 10/6/2015 3:38 AM, Adrian Liew wrote: > Thanks for the reply. Looks like this has been resolved by manually starting > the Zookeeper services on each server promptly so that the tickTime value > does not timeout too quickly to heartbeat other peers. Hence, I increased the > tickTime value to about 5 minutes to give some time for a node hosting > Zookeeper to restart and autostart its service. This case seems fixed but I > will double check again once more to be sure. I am using nssm > (non-sucking-service-manager) to autostart Zookeeper. I will need to retest > this once again using nssm to make sure zookeeper services are up and running. That sounds like a very bad idea. A typical tickTime is two *seconds*. Zookeeper is designed around certain things happening very quickly. I don't think you can increase that to five *minutes* (multiplying it by 150) without the strong possibility of something going very wrong and processes hanging for minutes at a time waiting for a timeout that should happen very quickly. I am reasonably certain that tickTime is used for zookeeper operation in several ways, so I believe that this much of an increase will cause fundamental problems with zookeeper's normal operation. I admit that I have not looked at the code, so I could be wrong ... but based on the following information from the Zookeeper docs, I don't think I am wrong: tickTime the length of a single tick, which is the basic time unit used by ZooKeeper, as measured in milliseconds. It is used to regulate heartbeats, and timeouts. For example, the minimum session timeout will be two ticks. Thanks, Shawn
RE: If zookeeper is down, SolrCloud nodes will not start correctly, even if zookeeper is started later
Hi Shawn Thanks for informing me. I guess the worst case scenario is that all 3 ZK services are down and that may be unlikely the case. At this juncture, as you said the viable workaround is a manual approach to start up the services in sequence in ensuring a quorum can take place. So the proper sequence in a 3 ZK + Solr (both ZK and Solr in each server) server setup will be as follows: Downed situation with one or mode ZK services 1. Restart all ZK Services first on all three machines 2. Restart all Solr Services on all three machines Please do clarify if the above is correct and I will be happy to take this approach and communicate to my customer. Many thanks. Regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Wednesday, October 7, 2015 4:09 PM To: solr-user@lucene.apache.org Subject: Re: If zookeeper is down, SolrCloud nodes will not start correctly, even if zookeeper is started later On 10/6/2015 10:22 PM, Adrian Liew wrote: > Hence, the issue is that upon startup of three machines, the startup > of ZK and Solr is out of sequence that causes SolrCloud to behave > unexpectedly. Noting there is Jira ticket addressed here for Solr 4.9 > above to include an improvement to the issue above. > (https://issues.apache.org/jira/browse/SOLR-5129) That issue is unresolved, so it has not been fixed in any Solr version. At this time, if you do not have Zookeeper quorum (a majority of your ZK nodes fully operational), you will not be able to successfully start SolrCloud nodes. The issue has low priority because there is a viable workaround -- ensure that ZK has quorum before starting or restarting any Solr node. Thinking out loud: Until this issue is fixed, I think this means that a 3-node setup where all three nodes use the zookeeper embedded in Solr will require a strange startup sequence if none of the nodes are running: * Start node 1. Solr will not start correctly -- no ZK quorum. * Start node 2. Solr might start correctly, not sure. * Start node 3. This should start correctly. * Restart node 1. With ZK nodes 2 and 3 running, this will work. * Restart node 2 if it did not start properly the first time. I really have no idea whether the second node startup will work properly. Thanks, Shawn
RE: Run Solr 5.3.0 as a Service on Windows using NSSM
Hi Edwin, You may want to try explore some of the configuration properties to configure in zookeeper. http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html#sc_zkMulitServerSetup My recommendation is to try run your batch files outside of NSSM so it is easier to debug and observe what you see from the command window. I don't think ZK and Solr can be automated on startup well using NSSM due to the fact that ZK services need to be running before you start up Solr services. I just had conversation with Shawn on this topic. NSSM cannot do the magic startup in a cluster setup. In that, you may need to write custom scripting to get it right. Back to your original issue, I guess it is worth exploring timeout values. Then again, I will leave the real Solr experts to chip in their thoughts. Best regards, Adrian Liew -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Wednesday, October 7, 2015 1:40 PM To: solr-user@lucene.apache.org Subject: Re: Run Solr 5.3.0 as a Service on Windows using NSSM Hi Adrian, I've waited for more than 5 minutes and most of the time when I refresh it says that the page cannot be found. Got one or twice the main Admin page is loaded, but none of the cores are loaded. I have 20 cores which I'm loading. The core are of various sizes, but the maximum one is 38GB. Others ranges from 10GB to 15GB, and there're some which are less than 1GB. My overall core size is about 200GB. Regards, Edwin On 7 October 2015 at 12:11, Adrian Liew <adrian.l...@avanade.com> wrote: > Hi Edwin, > > I have setup NSSM on Solr 5.3.0 in an Azure VM and can start up Solr > with a base standalone installation. > > You may have to give Solr some time to bootstrap things and wait for > the page to reload. Are you still seeing the page after 1 minute or so? > > What are your core sizes? And how many cores are you trying to load? > > Best regards, > Adrian > > -Original Message- > From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] > Sent: Wednesday, October 7, 2015 11:46 AM > To: solr-user@lucene.apache.org > Subject: Run Solr 5.3.0 as a Service on Windows using NSSM > > Hi, > > I tried to follow this to start my Solr as a service using NSSM. > http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/ > > Everything is fine when I start the services under Component Services. > However, when I tried to point to the Solr Admin page, it says that > the page cannot be found. > > I have tried the same thing in Solr 5.1, and it was able to work. Not > sure why it couldn't work for Solr 5.2 and Solr 5.3. > > Is there any changes required to what is listed on the website? > > Regards, > Edwin >
RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi Shawn, Thanks for the reply. Understood your comments and will revert back to the defaults. However, I raised this issue because I realized that Zookeeper becomes impatient if it cannot heartbeat its other peers in time. So for example, if 1 ZK server goes down out of 3 ZK servers, the 1 ZK server will stop pinging other servers and complain about timeout issues to zkCli connect to its service. Will revert back with an update. Regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Tuesday, October 6, 2015 10:16 PM To: solr-user@lucene.apache.org Subject: Re: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd On 10/6/2015 3:38 AM, Adrian Liew wrote: > Thanks for the reply. Looks like this has been resolved by manually starting > the Zookeeper services on each server promptly so that the tickTime value > does not timeout too quickly to heartbeat other peers. Hence, I increased the > tickTime value to about 5 minutes to give some time for a node hosting > Zookeeper to restart and autostart its service. This case seems fixed but I > will double check again once more to be sure. I am using nssm > (non-sucking-service-manager) to autostart Zookeeper. I will need to retest > this once again using nssm to make sure zookeeper services are up and running. That sounds like a very bad idea. A typical tickTime is two *seconds*. Zookeeper is designed around certain things happening very quickly. I don't think you can increase that to five *minutes* (multiplying it by 150) without the strong possibility of something going very wrong and processes hanging for minutes at a time waiting for a timeout that should happen very quickly. I am reasonably certain that tickTime is used for zookeeper operation in several ways, so I believe that this much of an increase will cause fundamental problems with zookeeper's normal operation. I admit that I have not looked at the code, so I could be wrong ... but based on the following information from the Zookeeper docs, I don't think I am wrong: tickTime the length of a single tick, which is the basic time unit used by ZooKeeper, as measured in milliseconds. It is used to regulate heartbeats, and timeouts. For example, the minimum session timeout will be two ticks. Thanks, Shawn
RE: Run Solr 5.3.0 as a Service on Windows using NSSM
Hi Edwin, I have setup NSSM on Solr 5.3.0 in an Azure VM and can start up Solr with a base standalone installation. You may have to give Solr some time to bootstrap things and wait for the page to reload. Are you still seeing the page after 1 minute or so? What are your core sizes? And how many cores are you trying to load? Best regards, Adrian -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Wednesday, October 7, 2015 11:46 AM To: solr-user@lucene.apache.org Subject: Run Solr 5.3.0 as a Service on Windows using NSSM Hi, I tried to follow this to start my Solr as a service using NSSM. http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/ Everything is fine when I start the services under Component Services. However, when I tried to point to the Solr Admin page, it says that the page cannot be found. I have tried the same thing in Solr 5.1, and it was able to work. Not sure why it couldn't work for Solr 5.2 and Solr 5.3. Is there any changes required to what is listed on the website? Regards, Edwin
If zookeeper is down, SolrCloud nodes will not start correctly, even if zookeeper is started later
Changing subject header. I am encountering this issue in Solr 5.3.0 whereby I am getting haywired leader election using SolrCloud. I am using NSSM 2.24 to startup my solr services with zookeeper set as a dependency. For example, if I have three servers marked as 10.0.0.4, 10.0.0.5 and 10.0.0.6, both 10.0.0.4 and 10.0.0.5 show up as Leaders in Solr Admin Panel. I then decided to manually stop all services, and ensure all Zookeeper services are booted up first prior to starting Solr services on all machines. Then I refreshed my Solr Admin Panel to observe the correct leaders and followers and test node recovery. Everything turned out fine. Hence, the issue is that upon startup of three machines, the startup of ZK and Solr is out of sequence that causes SolrCloud to behave unexpectedly. Noting there is Jira ticket addressed here for Solr 4.9 above to include an improvement to the issue above. (https://issues.apache.org/jira/browse/SOLR-5129) Can someone please advise. Best regards, Adrian -Original Message- From: Alessandro Benedetti [mailto:benedetti.ale...@gmail.com] Sent: Tuesday, October 6, 2015 7:14 PM To: solr-user@lucene.apache.org Subject: Re: Zookeeper HA with 3x ZK with Solr server nodes When you have a ZK Ensemble a quorum of active nodes is necessary to have the entire Ensemble to work ( elect leaders, manage the cluster topology etc etc) . The quorum is 50% living nodes +1 . If you have an ensemble of 3 nodes, the quorum is 3/2 +1 = 2 nodes . With an ensemble of 3 nodes, you can lose 1 and the ZK ensemble will continue to work. If you have an ensemble of 5 nodes, the quorum is 5/2 +1 = 3 nodes With an ensemble of 5 nodes, you can lose 2 and the ZK ensemble will continue to work. ect ect Cheers 2015-10-06 10:55 GMT+01:00 Adrian Liew <adrian.l...@avanade.com>: > Hi there, > > > > I have 3 Solr server Azure VM nodes participating in SolrCloud with ZK > installed on each of these nodes (to avoid a single point of failure > with ZK for leader election). Each Solr server is hosted in a Windows > Server > 2012 R2 environment. I was told by my peer that if one zookeeper > service fails, the entire quorum fails. Hence if a quorum fails, does > that mean it will not be able to elect the leader from the remaining 2 > alive Solr servers, even if ZK services are installed in each node? > > > > I am yet to this out as this defeats the purpose of having a ZK > installed on each server. I am afraid if one node fails, a leader > cannot be elected with the remaining two available nodes. Correct me if I am > wrong. > > > > Regards, > > Adrian > > > -- -- Benedetti Alessandro Visiting card - http://about.me/alessandro_benedetti Blog - http://alexbenedetti.blogspot.co.uk "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi Edwin, Thanks for the reply. Looks like this has been resolved by manually starting the Zookeeper services on each server promptly so that the tickTime value does not timeout too quickly to heartbeat other peers. Hence, I increased the tickTime value to about 5 minutes to give some time for a node hosting Zookeeper to restart and autostart its service. This case seems fixed but I will double check again once more to be sure. I am using nssm (non-sucking-service-manager) to autostart Zookeeper. I will need to retest this once again using nssm to make sure zookeeper services are up and running. Regards, Adrian Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030 -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Monday, October 5, 2015 10:02 AM To: solr-user@lucene.apache.org Subject: Re: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd Hi Adrian, It's unlikely to be the firewall settings if it is failing intermittently. More of a network issues. The error says it's a connection time out, and since you say it happens only intermittently, I'm suspecting it could be network issues. Have you check if the connection to the various servers are always up? Regards, Edwin On 3 October 2015 at 00:22, Erick Erickson <erickerick...@gmail.com> wrote: > Hmmm, there are usually a couple of ports that each ZK instance needs, > is it possible that you've got more than one process using one of > those ports? > > By default (I think), zookeeper uses "peer port + 1000" for its leader > election process, see: > https://zookeeper.apache.org/doc/r3.3.3/zookeeperStarted.html > the "Running Replicated Zookeeper" section. > > I'm not quite clear whether the above ZK2 port and ZK3 port are just > meant to indicate a single Zookeeper instance on a node or not so I > thought I'd check. > > Firewalls should always fail, not intermittently so I'm puzzled about > that > > Best, > Erick > > On Fri, Oct 2, 2015 at 1:33 AM, Adrian Liew <adrian.l...@avanade.com> > wrote: > > Hi Edwin, > > > > I have followed the standards recommended by the Zookeeper article. > > It > seems to be working. > > > > Incidentally, I am facing intermittent issues whereby I am unable to > connect to Zookeeper service via Solr's zkCli.bat command, even after > having setting automatic startup of my ZooKeeper service. I have > basically configured (non-sucking-service-manager) nssm to auto start > Solr with a dependency of Zookeeper to ensure both services are > running on startup for each Solr VM. > > > > Here is an example what I tried to run to connect to the ZK service: > > > > E:\solr-5.3.0\server\scripts\cloud-scripts>zkcli.bat -z > > 10.0.0.6:2183 > -cmd list > > Exception in thread "main" org.apache.solr.common.SolrException: > java.util.concu > > rrent.TimeoutException: Could not connect to ZooKeeper 10.0.0.6:2183 > within 3000 > > 0 ms > > at > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:18 > > 1) > > at > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:11 > > 5) > > at > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:10 > > 5) > > at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:181) > > Caused by: java.util.concurrent.TimeoutException: Could not connect > > to > ZooKeeper > > 10.0.0.6:2183 within 3 ms > > at > org.apache.solr.common.cloud.ConnectionManager.waitForConnected(Conne > > ctionManager.java:208) > > at > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:17 > > 3) > > ... 3 more > > > > > > Further to this I inspected the output shown in console window by > zkServer.cmd: > > > > 2015-10-02 08:24:09,305 [myid:3] - WARN > [WorkerSender[myid=3]:QuorumCnxManager@ > > 382] - Cannot open channel to 2 at election address /10.0.0.5:3888 > > java.net.SocketTimeoutException: connect timed out > > at java.net.DualStackPlainSocketImpl.waitForConnect(Native > Method) > > at java.net.DualStackPlainSocketImpl.socketConnect(Unknown > Source) > > at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) > > at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown > Source) > > at java.net.AbstractPlainSocketImpl.connect(Unknown Source) > > at java.net.PlainSocketImpl.connect(Unknown Source) > > at java.net.SocksSocketImpl.connect
Zookeeper HA with 3x ZK with Solr server nodes
Hi there, I have 3 Solr server Azure VM nodes participating in SolrCloud with ZK installed on each of these nodes (to avoid a single point of failure with ZK for leader election). Each Solr server is hosted in a Windows Server 2012 R2 environment. I was told by my peer that if one zookeeper service fails, the entire quorum fails. Hence if a quorum fails, does that mean it will not be able to elect the leader from the remaining 2 alive Solr servers, even if ZK services are installed in each node? I am yet to this out as this defeats the purpose of having a ZK installed on each server. I am afraid if one node fails, a leader cannot be elected with the remaining two available nodes. Correct me if I am wrong. Regards, Adrian
RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi Edwin, I have followed the standards recommended by the Zookeeper article. It seems to be working. Incidentally, I am facing intermittent issues whereby I am unable to connect to Zookeeper service via Solr's zkCli.bat command, even after having setting automatic startup of my ZooKeeper service. I have basically configured (non-sucking-service-manager) nssm to auto start Solr with a dependency of Zookeeper to ensure both services are running on startup for each Solr VM. Here is an example what I tried to run to connect to the ZK service: E:\solr-5.3.0\server\scripts\cloud-scripts>zkcli.bat -z 10.0.0.6:2183 -cmd list Exception in thread "main" org.apache.solr.common.SolrException: java.util.concu rrent.TimeoutException: Could not connect to ZooKeeper 10.0.0.6:2183 within 3000 0 ms at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:18 1) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:11 5) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:10 5) at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:181) Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 10.0.0.6:2183 within 3 ms at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(Conne ctionManager.java:208) at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:17 3) ... 3 more Further to this I inspected the output shown in console window by zkServer.cmd: 2015-10-02 08:24:09,305 [myid:3] - WARN [WorkerSender[myid=3]:QuorumCnxManager@ 382] - Cannot open channel to 2 at election address /10.0.0.5:3888 java.net.SocketTimeoutException: connect timed out at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(Quorum CnxManager.java:368) at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxM anager.java:341) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$Worke rSender.process(FastLeaderElection.java:449) at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$Worke rSender.run(FastLeaderElection.java:430) at java.lang.Thread.run(Unknown Source) 2015-10-02 08:24:09,305 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElect ion@597] - Notification: 1 (message format version), 3 (n.leader), 0x70011 ( n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x7 (n.peerEpoch) LOOKING (my state) I noticed the error message by zkServer.cmd as Cannot open channel to 2 at election address /10.0.0.5:3888 Can firewall settings be the issue here? I feel this may be a network issue between the individual Solr VMs. I am using a Windows Server 2012 R2 64 bit environment to run Zookeeper 3.4.6 and Solr 5.3.0. Currently, I have setup my firewalls in the Advanced Configuration Firewall Settings as below: As for the Firewall settings I have configured the below for each Azure VM (Phoenix-Solr-0, Phoenix-Solr-1, Phoenix-Solr-2) in the Firewall Advanced Security Settings: For allowed inbound connections: Solr port 8983 ZK1 port 2181 ZK2 port 2888 ZK3 port 3888 Regards, Adrian -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Friday, October 2, 2015 11:03 AM To: solr-user@lucene.apache.org Subject: Re: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd Hi Adrian, How is your setup of your system like? By right it shouldn't be an issue if we use different ports. in fact, if the various zookeeper instance are running on a single machine, they have to be on different ports in order for it to work. Regards, Edwin On 1 October 2015 at 18:19, Adrian Liew <adrian.l...@avanade.com> wrote: > Hi all, > > The problem below was resolved by appropriately setting my server ip > addresses to have the following for each zoo.cfg: > > server.1=10.0.0.4:2888:3888 > server.2=10.0.0.5:2888:3888 > server.3=10.0.0.6:2888:3888 > > as opposed to the following: > > server.1=10.0.0.4:2888:3888 > server.2=10.0.0.5:2889:3889 > server.3=10.0.0.6:2890:3890 > > I am not sure why the above can be an issue (by right it should not), > however I followed the recommendations provided by Zookeeper > administration guide under RunningReplicatedZookeeper ( > https://zookeeper.apache.org/doc/r3.1.2/zookeeperStarted.html#sc_Runni > ngReplicatedZooKeeper > ) > > Given t
RE: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi all, The problem below was resolved by appropriately setting my server ip addresses to have the following for each zoo.cfg: server.1=10.0.0.4:2888:3888 server.2=10.0.0.5:2888:3888 server.3=10.0.0.6:2888:3888 as opposed to the following: server.1=10.0.0.4:2888:3888 server.2=10.0.0.5:2889:3889 server.3=10.0.0.6:2890:3890 I am not sure why the above can be an issue (by right it should not), however I followed the recommendations provided by Zookeeper administration guide under RunningReplicatedZookeeper (https://zookeeper.apache.org/doc/r3.1.2/zookeeperStarted.html#sc_RunningReplicatedZooKeeper) Given that I am testing multiple servers in a mutiserver environment, it will be safe to use 2888:3888 on each server rather than have different ports. Regards, Adrian From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Thursday, October 1, 2015 5:32 PM To: solr-user@lucene.apache.org Subject: Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd Hi there, Currently, I have setup an azure virtual network to connect my Zookeeper clusters together with three Azure VMs. Each VM has an internal IP of 10.0.0.4, 10.0.0.5 and 10.0.0.6. I have also setup Solr 5.3.0 which runs in Solr Cloud mode connected to all three Zookeepers in an external ensemble manner. I am able to connect to 10.0.0.4 and 10.0.0.6 via the zkCli.cmd after starting the Zookeeper services. However for 10.0.0.5, I keep getting the below error even if I started the zookeeper service. [cid:image001.png@01D0FC6E.BDC2D990] I have restarted 10.0.0.5 VM several times and still am unable to connect to Zookeeper via zkCli.cmd. I have checked zoo.cfg (making sure ports, data and logs are all set correctly) and myid to ensure they have the correct configurations. The simple command line I used to connect to Zookeeper is zkCli.cmd -server 10.0.0.5:2182 for example. Any ideas? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030
RE: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout issues
Hi Shawn, Thanks for that. You did mention about starting out with empty collections and already I am experiencing timeout issues. Could this have to do with the hardware or server spec sizing itself. For example, lack of memory allocated, network issues etc. that can possibly cause this? Given Azure has a 99.95 percent SLA, I don't think this could be a network issue. I am currently using a 4 core 7 GB RAM memory machine for an individual Solr Server. I don't quite understand why this is happening as I am just trying to setup a bare bones Solr Cloud setup using Solr 5.3.0 and Zookeeper 3.4.6. Any tips will be much appreciated. Best regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Thursday, October 1, 2015 11:12 PM To: solr-user@lucene.apache.org Subject: Re: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout issues On 10/1/2015 4:43 AM, Adrian Liew wrote: > E:\solr-5.3.0\bin>solr.cmd create_collection -c sitecore_core_index -n > sitecore_ common_configs -shards 1 -replicationFactor 3 > > Connecting to ZooKeeper at 10.0.0.4:2181,10.0.0.5:2182,10.0.0.6:2183 ... > Re-using existing configuration directory sitecore_common_configs > > Creating new collection 'sitecore_core_index' using command: > http://localhost:8983/solr/admin/collections?action=CREATE=siteco > re_core_in > dex=1=3=2 > igName=sit > ecore_common_configs > > ERROR: Failed to create collection 'sitecore_core_index' due to: > create the coll ection time out:180s The timeout, as it mentions, is 180 seconds, or three minutes. This is a the default timeout for the Collections API, and it is a particularly long timeout. When it is exceeded, it is usually an indication of a serious problem. The collection create will likely succeed eventually, after an unknown amount of time ... the collections API just gave up on waiting for the response. There are two things that I know of that can cause this: A very large number of collections, and general performance issues. I did some testing a while back with thousands of empty collections on the Solr 5.x cloud example. It did not turn out well. Many things timed out, and a server restart would throw the whole cloud into chaos for a very long time. If those collections were not empty, then I suspect the problems would be even worse. General performance issues (usually RAM-related) can cause big problems with SolrCloud too. The following wiki page is most of my accumulated knowledge about what causes performance problems with Solr: https://wiki.apache.org/solr/SolrPerformanceProblems Thanks, Shawn
Cannot connect to a zookeeper 3.4.6 instance via zkCli.cmd
Hi there, Currently, I have setup an azure virtual network to connect my Zookeeper clusters together with three Azure VMs. Each VM has an internal IP of 10.0.0.4, 10.0.0.5 and 10.0.0.6. I have also setup Solr 5.3.0 which runs in Solr Cloud mode connected to all three Zookeepers in an external ensemble manner. I am able to connect to 10.0.0.4 and 10.0.0.6 via the zkCli.cmd after starting the Zookeeper services. However for 10.0.0.5, I keep getting the below error even if I started the zookeeper service. [cid:image001.png@01D0FC6E.BDC2D990] I have restarted 10.0.0.5 VM several times and still am unable to connect to Zookeeper via zkCli.cmd. I have checked zoo.cfg (making sure ports, data and logs are all set correctly) and myid to ensure they have the correct configurations. The simple command line I used to connect to Zookeeper is zkCli.cmd -server 10.0.0.5:2182 for example. Any ideas? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030
RE: Performance warning overlapping onDeckSearchers
Thanks very much for the useful info Eric. Sincerely appreciate you pointed out
those questions. In fact, I am currently working with a third party product
called Sitecore Web Content Management System (WCMS) that does the job to issue
updates to the Solr Index.
I need to understand abit more when and how they are committing documents to
Solr. Your question about how the new searchers arise to produce those
performance warning messages is valid.
Your questions will be more for the Sitecore Product Team to investigate in
which I will chase them up for answers.
1. Whether we are overriding the system variable solr.autoSoftCommit.maxTime?
2. Whether we are overriding the solr.autoCommit.maxTime (although this really
shouldn't matter).
3. Why we have too many hard commits?
4. How come new searchers are opened even when this is set with
openSearcherfalse/openSearcher
I shall chase these questions up with them. Will feedback to you where
necessary.
Thanks.
Best regards,
Adrian Liew
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Wednesday, August 12, 2015 11:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Performance warning overlapping onDeckSearchers
Something's not adding up here. Is your _client_ perhaps issuing commits when
you index documents? This is Joel's question, so we need to see how you send
docs to Solr. We really need to know how you're indexing docs to Solr.
My bet (and I suspect Joel's) is that you're either using SolrJ to send docs to
Solr and have something like solrJ while (more docs) {
create a doc
send it to Solr
commit
}
rather than
while (more docs) {
create a bunch of docs (I usually start with 1,000) using the commitwithin
option and make it as long as possible
send the batch to solr
}
Maybe commit at the very end, but that's not necessary if you're willing to
wait for commitWithin.
post.jar
Or, you're using post.jar in some kind of loop which commits every time you use
it by default. You can disable this, try 'java -jar post.jar -help' for all the
options, but the one you want is -Dcommit=no.
NOTE: you have to issue a commit _sometime_ to see the docs, either the
commitWithin option in SolrJ or explicitly if you're using the post.jar tool.
You can even issue a commit (this is suitable for
testing) via curl or a browser with
http://solr_node:8983/solr/collection/update?commit=true
The reason we're focusing here is that:
Soft commits are disabled in your setup, this is the -1 in autoSoftCommit.
Hard commits are not opening searchers, this is the autoCommit,
openSearcherfalse/openSearcher section.
Are, you perhaps overriding the system variable solr.autoSoftCommit.maxTime
when you start up Solr?
What about solr.autoCommit.maxTime (although this really shouldn't matter).
If you're not overriding the above, then no searchers should be being opened at
all after you start Solr, and only one should be opening when you do start
Solr. So you should not be getting the warning about Overlapping
onDeckSearchers.
Forget the static warming queries, they are irrelevant until we understand why
you're getting any new searchers. For future reference, these are the
newSearcher and firstSearcher events in solrconfig.xml. newSearcher is fired
every time one commits, firstSearcher when you start Solr.
The bottom line here is you need to find out why you're committing at all,
which opens a new searcher which, when that happens too often generates the
warning you're seeing.
Best,
Erick
On Wed, Aug 12, 2015 at 6:51 AM, Adrian Liew adrian.l...@avanade.com wrote:
Hi Joel,
I am fairly new to Solr (version which I am using v5.2.1) so I suppose what
you may be asking is referring to the autocommits section:
!-- AutoCommit
Perform a hard commit automatically under certain conditions.
Instead of enabling autoCommit, consider using commitWithin
when adding documents.
http://wiki.apache.org/solr/UpdateXmlMessages
maxDocs - Maximum number of documents to add since the last
commit before automatically triggering a new commit.
maxTime - Maximum amount of time in ms that is allowed to pass
since a document was added before automatically
triggering a new commit.
openSearcher - if false, the commit causes recent index changes
to be flushed to stable storage, but does not cause a new
searcher to be opened to make those changes visible.
If the updateLog is enabled, then it's highly recommended to
have some sort of hard autoCommit to limit the log size.
--
autoCommit
maxTime${solr.autoCommit.maxTime:15000}/maxTime
openSearcherfalse/openSearcher
/autoCommit
!-- softAutoCommit is like autoCommit except it causes a
'soft' commit which only ensures that changes are visible
RE: Performance warning overlapping onDeckSearchers
Additionally, I realized that my autowarmCount is set to zero for the following Cache entries except perSegFilter: filterCache class=solr.FastLRUCache size=512 initialSize=512 autowarmCount=0/ !-- Query Result Cache Caches results of searches - ordered lists of document ids (DocList) based on a query, a sort, and the range of documents requested. Additional supported parameter by LRUCache: maxRamMB - the maximum amount of RAM (in MB) that this cache is allowed to occupy -- queryResultCache class=solr.LRUCache size=512 initialSize=512 autowarmCount=0/ !-- Document Cache Caches Lucene Document objects (the stored fields for each document). Since Lucene internal document ids are transient, this cache will not be autowarmed. -- documentCache class=solr.LRUCache size=512 initialSize=512 autowarmCount=0/ !-- custom cache currently used by block join -- cache name=perSegFilter class=solr.search.LRUCache size=10 initialSize=0 autowarmCount=10 regenerator=solr.NoOpRegenerator / The link, https://wiki.apache.org/solr/FAQ#What_does_.22PERFORMANCE_WARNING:_Overlapping_onDeckSearchers.3DX.22_mean_in_my_logs.3F did suggest to reduce the autowarmCount or reduce warm up cache activity (which I am not sure where to begin doing this). I suspect autowarmCount is not very large as the above. Let me know what you think. Best regards, Adrian Liew -Original Message- From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Wednesday, August 12, 2015 3:32 PM To: solr-user@lucene.apache.org Subject: RE: Performance warning overlapping onDeckSearchers Thanks Shawn. Having said that increasing maxWarmingSearchers is usually wrong to solve this, are there any implications if we set maxWarmingSearchers to zero to resolve this problem? Or do you think there are some other settings that are worthwhile tuning to cater to the above? Best regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Tuesday, August 11, 2015 11:02 PM To: solr-user@lucene.apache.org Subject: Re: Performance warning overlapping onDeckSearchers On 8/11/2015 3:02 AM, Adrian Liew wrote: Has anyone come across this issue, [some_index] PERFORMANCE WARNING: Overlapping onDeckSearchers=2? I am currently using Solr v5.2.1. What does this mean? Does this raise red flags? I am currently encountering an issue whereby my Sitecore system is unable to update the index appropriately. I am not sure if this is linked to the warnings above. https://wiki.apache.org/solr/FAQ#What_does_.22PERFORMANCE_WARNING:_Overlapping_onDeckSearchers.3DX.22_mean_in_my_logs.3F What the wiki page doesn't explicitly state is that increasing maxWarmingSearchers is usually the wrong way to solve this, because that can actually make the problem *worse*. It is implied by the things the page DOES say, but it is not stated. Thanks, Shawn
RE: Performance warning overlapping onDeckSearchers
Thanks Shawn. Having said that increasing maxWarmingSearchers is usually wrong to solve this, are there any implications if we set maxWarmingSearchers to zero to resolve this problem? Or do you think there are some other settings that are worthwhile tuning to cater to the above? Best regards, Adrian -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Tuesday, August 11, 2015 11:02 PM To: solr-user@lucene.apache.org Subject: Re: Performance warning overlapping onDeckSearchers On 8/11/2015 3:02 AM, Adrian Liew wrote: Has anyone come across this issue, [some_index] PERFORMANCE WARNING: Overlapping onDeckSearchers=2? I am currently using Solr v5.2.1. What does this mean? Does this raise red flags? I am currently encountering an issue whereby my Sitecore system is unable to update the index appropriately. I am not sure if this is linked to the warnings above. https://wiki.apache.org/solr/FAQ#What_does_.22PERFORMANCE_WARNING:_Overlapping_onDeckSearchers.3DX.22_mean_in_my_logs.3F What the wiki page doesn't explicitly state is that increasing maxWarmingSearchers is usually the wrong way to solve this, because that can actually make the problem *worse*. It is implied by the things the page DOES say, but it is not stated. Thanks, Shawn
RE: Solr old log files are not archived or removed automatically.
Hi Erick,
1 how did you install/run your Solr? As a service or regular? See
the reference guide, Permanent Logging Settings for some info on the
difference there.
What is the difference between regular or service?
2 what does your log4j.properties file look like?
Here are the contents in the log4j.properties file:
# Logging level
solr.log=logs
log4j.rootLogger=INFO, file, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%-4r [%t] %-5p %c %x
[%X{collection} %X{shard} %X{replica} %X{core}] \u2013 %m%n
#- size rotation with log cleanup.
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.MaxFileSize=4MB
log4j.appender.file.MaxBackupIndex=9
#- File to log to and log format
#log4j.appender.file.File=${solr.log}/solr.log
log4j.appender.file.File=C:/solr_logs/solr.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%-5p - %d{-MM-dd
HH:mm:ss.SSS}; [%X{collection} %X{shard} %X{replica} %X{core}] %C; %m\n
log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.apache.hadoop=WARN
# set to INFO to enable infostream log messages
log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF
I am not sure how best I can limit the size of the solr_logs directory. Does
log4j come with a feature to remove old log files with a given retention period?
Best regards,
Adrian Liew
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Monday, August 10, 2015 11:36 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr old log files are not archived or removed automatically.
1 how did you install/run your Solr? As a service or regular? See
the reference guide, Permanent Logging Settings for some info on the
difference there.
2 what does your log4j.properties file look like?
Best,
Erick
On Mon, Aug 10, 2015 at 12:13 AM, Adrian Liew adrian.l...@avanade.com wrote:
Hi there,
I am using Solr v.5.2.1 on my local machine. I realized that old log files
are not removed in a timely manner by log4j. The logs which I am referring to
are the log files that reside within solr_directory\server\logs. So far I
have previous two months' worth of log files accumulated in the log
directory. Consequently, this causes my directory grow to such large sizes. I
will need to manually remove the old log files which is undesirable.
Is this is a bug with Solr or a missing configuration that needs to be set?
As far as I know, all Solr Logging configuration is done in the
solr_directory\server\resources\log4j.properties
Appreciate the soonest reply.
Thanks.
RE: SolrNet and deep pagination
Thanks Chris. We opted to use v0.5 which is an alpha version. And yes you I should be referring the SolrNet Google Group. Thanks for your help. Regards, Adrian -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: Tuesday, August 11, 2015 5:17 AM To: solr-user@lucene.apache.org Cc: Chong Kah Heng chong.kah.h...@avanade.com Subject: Re: SolrNet and deep pagination : Has anyone worked with deep pagination using SolrNet? The SolrNet : version that I am using is v0.4.0.2002. I followed up with this article, : https://github.com/mausch/SolrNet/blob/master/Documentation/CursorMark.md : , however the version of SolrNet.dll does not expose the a StartOrCursor : property in the QueryOptions class. I don't know anything about SolrNet, but i do know that the URL you list above is for the documentation on the master branch. If i try to look at the the same document on the 0.4.x branch, that document doesn't exist -- suggesting the feature isn't supported in the version of SolrNet you are using... https://github.com/mausch/SolrNet/blob/0.4.x/Documentation/CursorMark.md https://github.com/mausch/SolrNet/tree/0.4.x/Documentation In fact, if i search the repo for StartOrCursor i see a file named StartOrCursor.cs exists on the master branch, but not on the 0.4.x branch... https://github.com/mausch/SolrNet/blob/master/SolrNet/StartOrCursor.cs https://github.com/mausch/SolrNet/blob/0.4.x/SolrNet/StartOrCursor.cs ...so it seems unlikely that this (class?) is supported in the release you are using. Note: according to the docs, there is a SolrNet google group where this question is probably the most appopriate: https://github.com/mausch/SolrNet/blob/master/Documentation/README.md https://groups.google.com/forum/#!forum/solrnet -Hoss http://www.lucidworks.com/
Performance warning overlapping onDeckSearchers
Hi there, Has anyone come across this issue, [some_index] PERFORMANCE WARNING: Overlapping onDeckSearchers=2? I am currently using Solr v5.2.1. What does this mean? Does this raise red flags? I am currently encountering an issue whereby my Sitecore system is unable to update the index appropriately. I am not sure if this is linked to the warnings above. Regards, Adrian
Solr old log files are not archived or removed automatically.
Hi there, I am using Solr v.5.2.1 on my local machine. I realized that old log files are not removed in a timely manner by log4j. The logs which I am referring to are the log files that reside within solr_directory\server\logs. So far I have previous two months' worth of log files accumulated in the log directory. Consequently, this causes my directory grow to such large sizes. I will need to manually remove the old log files which is undesirable. Is this is a bug with Solr or a missing configuration that needs to be set? As far as I know, all Solr Logging configuration is done in the solr_directory\server\resources\log4j.properties Appreciate the soonest reply. Thanks.
SolrNet and deep pagination
Hi there, Has anyone worked with deep pagination using SolrNet? The SolrNet version that I am using is v0.4.0.2002. I followed up with this article, https://github.com/mausch/SolrNet/blob/master/Documentation/CursorMark.md , however the version of SolrNet.dll does not expose the a StartOrCursor property in the QueryOptions class. Does anyone have insight into this? Feel free to let me know if there is a later version that we should be using. Additionally, does anyone know how someone will go about using the code to paginate say about 10 records per page on an index page of 2. This means, I will like to page 10 records from page 2 of the entire recordset. Regards, Adrian
RE: Jetty servlet container in production environment
Thanks Upaya for sharing. I am looking to deploy Solr in a Windows 64 Bit Server environment. Some people do say Jetty works optimally in a Linux based environment. Having said that, I believe Solr will have improved it's stability within a Windows environment. I agree with you on the advice. Shall just leave it as Jetty servlet. Thanks. Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030 -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Wednesday, July 15, 2015 2:57 PM To: solr-user@lucene.apache.org Subject: Re: Jetty servlet container in production environment Use Jetty. Or rather, just use bin/solr or bin\solr.cmd to interact with Solr. In the past, Solr shipped as a war which could be deployed in any servlet container. Since 5.0, it is to be considered a self-contained application, that just happens to use Jetty underneath. If you used something other than the inbuilt Jetty, you might end up with issues later on down the line when developers decide to make an optimisation or improvement that isn't compatible with the Servlet spec. Upayavira On Wed, Jul 15, 2015, at 07:43 AM, Adrian Liew wrote: Hi all, Will like to ask your opinion if it is recommended to use the default Jetty servlet container as a service to run Solr on a multi-server production environment. I hear some places that recommend using Tomcat as a servlet container. Is anyone able to share some thoughts about this? Limitations, advantages or disadvantages of using Jetty servlet in a production environment Regards, Adrian
Jetty servlet container in production environment
Hi all, Will like to ask your opinion if it is recommended to use the default Jetty servlet container as a service to run Solr on a multi-server production environment. I hear some places that recommend using Tomcat as a servlet container. Is anyone able to share some thoughts about this? Limitations, advantages or disadvantages of using Jetty servlet in a production environment Regards, Adrian
RE: Running Solr 5.2.1 on WIndows using NSSM
Hi Edwin, Sorry for the late reply. Was caught up yesterday. Yes I did not use the start.jar command and followed this article using solr.cmd - http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/. I am using a Windows Server 2012 R2 Server. The article example shows that it passes the start -f -p 8983 as arguments to the service. I believe it is important to have the -f. Did you try this example? If it didn't work for you, have you tried to remove the service via nssm and add it again? Best regards, Adrian -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Monday, July 13, 2015 10:51 AM To: solr-user@lucene.apache.org Subject: Re: Running Solr 5.2.1 on WIndows using NSSM Hi Adrian, I got this to work for Solr 5.1, but when I tried this in Solr 5.2.1, it gives the error Windows could not start the solr5.2.1 service on Local Computer. The service did not return an error. This could be an internal Windows error or an internal service error. As Solr 5.2.1 is not using the start.jar command to run Solr, are we still able to use the same arguments to set up the nssm? Regards, Edwin On 8 July 2015 at 17:38, Adrian Liew adrian.l...@avanade.com wrote: Answered my own question. :) It seems to work great for me by following this article. http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/ Regards, Adrian -Original Message- From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Wednesday, July 8, 2015 4:43 PM To: solr-user@lucene.apache.org Subject: Running Solr 5.2.1 on WIndows using NSSM Hi guys, I am looking to run Apache Solr v5.2.1 on a windows machine. I tried to setup a windows service using NSSM (Non-Sucking-Service-Manager) to install the windows service on the machine pointing to the solr.cmd file path itself and installing the service. After installation, I tried to start the windows service but it gives back an alert message. It says \Windows could not start the SolrService service on Local Computer. The service did not return an error. This could be an internal Windows error or an internal service error. Most of the examples of older Apache Solr uses the java -start start.jar command to run Solr and seem to run okay with nssm. I am not sure if this could be the solr.cmd issue or NSSM's issue. Alternatively, I have tried to use Windows Task Scheduler to configure a task to point to the solr.cmd as well and run task whenever the computer starts (regardless a user is logged in or not). The task scheduler seems to report back 'Task Start Failed' with Level of 'Error'. Additionally, after checking Event Viewer, it returns the error with nssm Failed to open process handle for process with PID 3640 when terminating service Solr Service : The parameter is incorrect. Chances this can point back to the solr.cmd file itself. Thoughts? Regards, Adrian
RE: Running Solr 5.2.1 on WIndows using NSSM
Answered my own question. :) It seems to work great for me by following this article. http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/ Regards, Adrian -Original Message- From: Adrian Liew [mailto:adrian.l...@avanade.com] Sent: Wednesday, July 8, 2015 4:43 PM To: solr-user@lucene.apache.org Subject: Running Solr 5.2.1 on WIndows using NSSM Hi guys, I am looking to run Apache Solr v5.2.1 on a windows machine. I tried to setup a windows service using NSSM (Non-Sucking-Service-Manager) to install the windows service on the machine pointing to the solr.cmd file path itself and installing the service. After installation, I tried to start the windows service but it gives back an alert message. It says \Windows could not start the SolrService service on Local Computer. The service did not return an error. This could be an internal Windows error or an internal service error. Most of the examples of older Apache Solr uses the java -start start.jar command to run Solr and seem to run okay with nssm. I am not sure if this could be the solr.cmd issue or NSSM's issue. Alternatively, I have tried to use Windows Task Scheduler to configure a task to point to the solr.cmd as well and run task whenever the computer starts (regardless a user is logged in or not). The task scheduler seems to report back 'Task Start Failed' with Level of 'Error'. Additionally, after checking Event Viewer, it returns the error with nssm Failed to open process handle for process with PID 3640 when terminating service Solr Service : The parameter is incorrect. Chances this can point back to the solr.cmd file itself. Thoughts? Regards, Adrian
Running Solr 5.2.1 on WIndows using NSSM
Hi guys, I am looking to run Apache Solr v5.2.1 on a windows machine. I tried to setup a windows service using NSSM (Non-Sucking-Service-Manager) to install the windows service on the machine pointing to the solr.cmd file path itself and installing the service. After installation, I tried to start the windows service but it gives back an alert message. It says \Windows could not start the SolrService service on Local Computer. The service did not return an error. This could be an internal Windows error or an internal service error. Most of the examples of older Apache Solr uses the java -start start.jar command to run Solr and seem to run okay with nssm. I am not sure if this could be the solr.cmd issue or NSSM's issue. Alternatively, I have tried to use Windows Task Scheduler to configure a task to point to the solr.cmd as well and run task whenever the computer starts (regardless a user is logged in or not). The task scheduler seems to report back 'Task Start Failed' with Level of 'Error'. Additionally, after checking Event Viewer, it returns the error with nssm Failed to open process handle for process with PID 3640 when terminating service Solr Service : The parameter is incorrect. Chances this can point back to the solr.cmd file itself. Thoughts? Regards, Adrian
RE: Setup Solr Multiserver Environment
Thanks Eric.
I have a scenario whereby I will like to setup a minimal 1 shard with 3 replica
all of which each replica sits in its one machine. I also have an external ZK
ensemble setup with zk1, zk2 and zk3.
A simple example: I create say collection1 in the following order to satisfy
the scenario above using just solr.cmd. (note I am using a windows server do
perform the below as this is a requirement)
1. Create collection1 with numshards of 1 and with port 5000 which points to
say example/cloud/node1/solr
2. Next create the replica for collection1 (without specifying numshards this
time as it will tell Solr to create a replica) with port 5001 which points to
say example/cloud/node2/solr
3. Repeat step 2 and point this to say example/cloud/node3/solr with port 5002
I am currently stuck at Step 1 as I only managed to create the collection with
Shard 1 pointing at port 5000. However, how can I go about creating the replica
for that existing collection, collection1?
Best regards,
Adrian
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Tuesday, July 7, 2015 1:25 AM
To: solr-user@lucene.apache.org
Subject: Re: Setup Solr Multiserver Environment
How are you starting this? WARNING: I haven't worked on a windows box for some
time so beware my advice ;).
You shouldn't have to edit these at all if you're using the start scripts (now
preferred).
-h host
-p port
-s dir
All this does is set an env variable that gets substituted in for things like
${jetty.port:5002}. That syntax really says Use the env var 'jetty.port' if
it's defined, otherwise default to 5002
Try bin/solr.cmd --help and see what the options are..
Best,
Erick
On Mon, Jul 6, 2015 at 1:41 AM, Adrian Liew adrian.l...@avanade.com wrote:
Has anyone tried setting up a multiserver environment for Solr on a Windows
machine?
I was just wondering how you will go about say setting up a single shard with
3 replicas (one of them is the leader) using the solr.cmd or infact java.cmd
on three different virtual machines for example. Note, I can only run
solr.cmd on Windows Machine. I won't be able to use shell scripting using the
sh command if I wanted to have an automated script to startup.
This is the way I see it for SolrCloud scenario:
1 replica = 1 machine
So for a three server instance/node setup with a single shard, it will be:
Shard 1:
Replica 1 (this is the Leader for the shard for example) :
solrnode1.cloudapp.net:5000 Replica 2: solrnode1.cloudapp.net:5001
Replica 3: solrnode1.cloudapp.net:5002
May it be possible to go about changing the hostport in the solr.xml of each
of those solr server instance/nodes above? So like each of them will be
pointing to like a directory called solr/node1, solr/node2 and solr/node3. So
the only place which makes sense to configure the host addresses is modifying
the sections below:
The default section that comes with the solr.xml is within solrcloud section
is:
str name=host${host:}/str
int name=hostPort${jetty.port:8983}/int
str name=hostContext${hostContext:solr}/str
So in my case for example, can I do the following?
Replica 1:
str name=hostsolrnode1.cloudapp.net/str
int name=hostPort${jetty.port:5000}/int
str name=hostContext${hostContext:solr}/str
Replica 2:
str name=host solrnode1.cloudapp.net /str
int name=hostPort${jetty.port:5001}/int
str name=hostContext${hostContext:solr}/str
Replica 3:
str name=host solrnode1.cloudapp.net /str
int name=hostPort${jetty.port:5002}/int
str name=hostContext${hostContext:solr}/str
Any suggestions are welcomed.
Regards,
Adrian
Best regards,
Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn.
Bhd..| Consulting Services
(: Direct: +(603) 2382 5668
È: +6010-2288030
RE: Solr 5.2.1 setup zookeeper ensemble problem
Thanks David. That was helpful. I have managed to upload my configuration to ZooKeeper with the below. Best regards, Adrian -Original Message- From: davidphilip cherian [mailto:davidphilipcher...@gmail.com] Sent: Monday, July 6, 2015 4:35 PM To: solr-user@lucene.apache.org Subject: Re: Solr 5.2.1 setup zookeeper ensemble problem Hi Adrian, What is the error that you are getting? In order to upload configs files, you could use zkcli.sh script that will be shipped with solr and use the upconfig command. ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 \ -cmd upconfig -confname my_new_config -confdir server/solr/configsets/basic_configs/conf https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities On Mon, Jul 6, 2015 at 1:43 PM, Adrian Liew adrian.l...@avanade.com wrote: There seems to be an issue running the following command using solr.cmd as below: - bin\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183 Anyone can please advise. Also, Is there a way to upload a configuration file (containing schema.xml and solrconfig.xml) to ZooKeeper easily using solr.cmd in solr 5.2.1? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030
Solr 5.2.1 setup zookeeper ensemble problem
There seems to be an issue running the following command using solr.cmd as below: - bin\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183 Anyone can please advise. Also, Is there a way to upload a configuration file (containing schema.xml and solrconfig.xml) to ZooKeeper easily using solr.cmd in solr 5.2.1? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030
RE: Solr 5.2.1 setup zookeeper ensemble problem
Hi David, When I run the command below on a Windows machine using Powershell window: .\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183 I get the following error: Invalid command-line option: localhost:2182 Somehow it does not recognize comma separated between the localhost. As far as I know if you are trying to run SolrCloud against your ZooKeeper Ensemble, you will need to specifify all three ZK server addresses (according to . https://cwiki.apache.org/confluence/display/solr/Setting+Up+an+External+ZooKeeper+Ensemble) However, some blogs say you can just connect to only just one instance for a ZK Ensemble (http://solr.pl/en/2013/03/11/solrcloud-howto-2/). So I am not sure which one is correct now. Thoughts on the above? Regards, Adrian -Original Message- From: davidphilip cherian [mailto:davidphilipcher...@gmail.com] Sent: Monday, July 6, 2015 4:35 PM To: solr-user@lucene.apache.org Subject: Re: Solr 5.2.1 setup zookeeper ensemble problem Hi Adrian, What is the error that you are getting? In order to upload configs files, you could use zkcli.sh script that will be shipped with solr and use the upconfig command. ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 \ -cmd upconfig -confname my_new_config -confdir server/solr/configsets/basic_configs/conf https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities On Mon, Jul 6, 2015 at 1:43 PM, Adrian Liew adrian.l...@avanade.com wrote: There seems to be an issue running the following command using solr.cmd as below: - bin\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183 Anyone can please advise. Also, Is there a way to upload a configuration file (containing schema.xml and solrconfig.xml) to ZooKeeper easily using solr.cmd in solr 5.2.1? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030
Setup Solr Multiserver Environment
Has anyone tried setting up a multiserver environment for Solr on a Windows
machine?
I was just wondering how you will go about say setting up a single shard with 3
replicas (one of them is the leader) using the solr.cmd or infact java.cmd on
three different virtual machines for example. Note, I can only run solr.cmd on
Windows Machine. I won't be able to use shell scripting using the sh command if
I wanted to have an automated script to startup.
This is the way I see it for SolrCloud scenario:
1 replica = 1 machine
So for a three server instance/node setup with a single shard, it will be:
Shard 1:
Replica 1 (this is the Leader for the shard for example) :
solrnode1.cloudapp.net:5000
Replica 2: solrnode1.cloudapp.net:5001
Replica 3: solrnode1.cloudapp.net:5002
May it be possible to go about changing the hostport in the solr.xml of each of
those solr server instance/nodes above? So like each of them will be pointing
to like a directory called solr/node1, solr/node2 and solr/node3. So the only
place which makes sense to configure the host addresses is modifying the
sections below:
The default section that comes with the solr.xml is within solrcloud section is:
str name=host${host:}/str
int name=hostPort${jetty.port:8983}/int
str name=hostContext${hostContext:solr}/str
So in my case for example, can I do the following?
Replica 1:
str name=hostsolrnode1.cloudapp.net/str
int name=hostPort${jetty.port:5000}/int
str name=hostContext${hostContext:solr}/str
Replica 2:
str name=host solrnode1.cloudapp.net /str
int name=hostPort${jetty.port:5001}/int
str name=hostContext${hostContext:solr}/str
Replica 3:
str name=host solrnode1.cloudapp.net /str
int name=hostPort${jetty.port:5002}/int
str name=hostContext${hostContext:solr}/str
Any suggestions are welcomed.
Regards,
Adrian
Best regards,
Adrian Liew | Consultant Application Developer
Avanade Malaysia Sdn. Bhd..| Consulting Services
(: Direct: +(603) 2382 5668
È: +6010-2288030
Solr Cloud Cluster Info and Clean
Hi Does anyone know where information about SolrCloud cluster is stored? I.E the below info? Is this stored in somewhere in ZooKeeper? In my case I am using an external Zookeeper server setup in an Ensemble. [cid:image001.png@01D0B827.CF87E750] Does anyone know how to clear their SolrCloud cluster. I.E I will like to remove collections initially made to the cluster and start fresh again. I have attempted this stackoverflow http://stackoverflow.com/questions/16579242/solrcloud-delete-collection-bug but managed to get a blank json in which I thought I can edit and then reupload it to ZooKeeper. Regards, Adrian
RE: Migrating from Solr 5.1 to Solr 5.2.1
Hi Edwin, Yes seems to work for me with this command if we use -s instead: -solr.cmd start -p 8983 -s C:\solrsc8_521_in\example\cloud\node1\solr -z localhost:2181,localhost:2182,localhost:2183 -noprompt I noticed you did not put a start in your command? Note the below command won't work with the -e parameter as this will runs the example action. The script will attempt to start two solr nodes on 8983 and 7574 and attach the nodes to zk with -z localhost:2181,localhost:2182,localhost:2183 without quotation marks. So the below line fails. -solr.cmd -e cloud -z localhost:2181,localhost:2182,localh ost:2183 -noprompt (won't work with quotation marks) I can see the Cloud tab on the left side and I do not see any collections yet. I believe this is a good start since I have not added in any collections yet. Have you tried to add a collection to 1 shard and 3 replicas for example? Best regards, Adrian Liew | Consultant Application Developer Avanade Malaysia Sdn. Bhd..| Consulting Services (: Direct: +(603) 2382 5668 È: +6010-2288030 -Original Message- From: Zheng Lin Edwin Yeo [mailto:edwinye...@gmail.com] Sent: Tuesday, July 7, 2015 11:46 AM To: solr-user@lucene.apache.org Subject: Re: Migrating from Solr 5.1 to Solr 5.2.1 I tried to put the 3 ZooKeeper ports together with the double-quotes and it works. - bin\solr.cmd -p 8983 -s example\node1\solr -z localhost:2181,localhost:2182,localhost:2183 Regards, Edwin On 7 July 2015 at 10:59, Zheng Lin Edwin Yeo edwinye...@gmail.com wrote: Thank you Timothy. I've managed to start Solr 5.2.1 by using bin\solr.cmd. However, I could not get it to start it together with 3 instances of external ZooKeeper with this command - bin\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183 I'll get the error *Invalid command-line option: localhost:2182* if I enter the above command. I can only managed to start it if I define only 1 instance of the ZooKeeper Eg: - bin\solr.cmd -e cloud -z localhost:2181 Regards, Edwin On 7 July 2015 at 00:07, Timothy Potter thelabd...@gmail.com wrote: Hi Edwin, You'll need to use the bin\solr.cmd to start Solr as it now requires some additional system properties to be set. Put simply, starting solr using java -jar start.jar is not supported. Please try bin\solr.cmd and let us know if you run into any issues. You can set any additional system properties (-D) you need in the bin\solr.in.cmd script Cheers, Tim On Fri, Jul 3, 2015 at 2:07 AM, Zheng Lin Edwin Yeo edwinye...@gmail.com wrote: Hi, I'm trying to migrate from Solr 5.1 to Solr 5.2.1. However, I faced some problems when I'm trying to migrate my index over, and when I'm trying to link up the external ZooKeeper to Solr. I'm using ZooKeeper 3.4.6 In Solr 5.1, I used this command to start Solr for both Shard1 and Shard2: java -D64 -Dsolr.clustering.enabled=true -Xms512M -Xmx4096M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapDumps -DzkHost=localhost:2181,localhost:2182,localhost:2183 -jar start.jar However, I get the following error in Solr 5.2.1: *Error: Unable to access jarfile start.jar* For ZooKeeper, I've been using the zkCli.bat file under server\scripts\cloud-scripts: zkcli.bat -zkhost localhost:2181 \ -cmd upconfig -confname collection1 -confdir C:\Users\edwin_000\Desktop\edwin\edm-5.2.1\configuration\collection1\ conf However, I get the following error in Solr 5.2.1 *Error: Could not find or load main class org.apache.solr.cloud.ZkCLI* Is there any changes to the code structure in Solr 5.2.1 as compared to the older versions? Regards, Edwin
