Re: Solr 4.0 SolrCloud with AWS Auto Scaling
thanks for pointing me to Solr's Zookeeper servlet. I will look at the source to see how I can use to fulfill my needs. Bill On Thu, Jan 3, 2013 at 6:43 PM, Mark Miller wrote: > Technically, you want to make sure zookeeper reports the node as live and > active. > > You could use the same api that the UI uses for that - the > localhost:port/solr/zookeeper (I think?) servlet. > > If you can't reach it for a node, it's obviously down - if you can reach > it, parse the json and see if it notes the node as active? > > Not quite as clean as you'd like prob. Might be worth a JIRA issue to look > at further options. > > - Mark > > On Jan 3, 2013, at 5:54 PM, Bill Au wrote: > > > Thanks, Mark. > > > > That does remove the node. And it seems to do so permanently. Even > when I > > restart Solr after unloading, it does not join the SolrCloud cluster. > And > > I can get it to re-join the cluster by creating the core. > > > > Anyone know if there is an API to determine the state of a node. When > AWS > > auto scaling add a new node, I need to make sure it has before active > > before I enable it in the load balancer. > > > > Bill > > > > > > > > > > On Thu, Jan 3, 2013 at 9:10 AM, Mark Miller > wrote: > > > >> > >> http://wiki.apache.org/solr/CoreAdmin#UNLOAD > >> > >> - Mark > >> > >> On Jan 3, 2013, at 9:06 AM, Bill Au wrote: > >> > >>> Mark, > >>>What do you mean by "unload them"? > >>> > >>> I am using an AWS load balancer with my auto scaling group in stead of > >>> using Solr's built-in load balancer. I am no sharding my index. I am > >>> using SolrCloud for replication only. I am doing local search on each > >>> instance and sending all updates to the shard leader directly because I > >>> want to minimize traffic between nodes during search and update > >>> > >>> Bill > >>> > >>> > >>> On Wed, Jan 2, 2013 at 6:47 PM, Mark Miller > >> wrote: > >>> > >>>> > >>>> On Jan 2, 2013, at 5:51 PM, Bill Au wrote: > >>>> > >>>>> Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > >>>>> > >>>>> My concern is that as AWS auto scaling add and remove instances to > >>>>> SolrCloud, the number of nodes in SolrCloud Zookeeper config will > grow > >>>>> indefinitely as removed instances will never be used again. AWS auto > >>>>> scaling will keep on adding new instances, and there is no way to > >> remove > >>>>> them from Zookeeper, right? > >>>> > >>>> You can unload them and that removes them. > >>>> > >>>>> What's the effect of have all these phantom > >>>>> nodes? > >>>> > >>>> Unless they are only replicas, they would need to be removed. > >>>> > >>>> Also, unless you are using elastic ips, > >>>> https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. > >>>> > >>>> - Mark > >> > >> > >
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
Technically, you want to make sure zookeeper reports the node as live and active. You could use the same api that the UI uses for that - the localhost:port/solr/zookeeper (I think?) servlet. If you can't reach it for a node, it's obviously down - if you can reach it, parse the json and see if it notes the node as active? Not quite as clean as you'd like prob. Might be worth a JIRA issue to look at further options. - Mark On Jan 3, 2013, at 5:54 PM, Bill Au wrote: > Thanks, Mark. > > That does remove the node. And it seems to do so permanently. Even when I > restart Solr after unloading, it does not join the SolrCloud cluster. And > I can get it to re-join the cluster by creating the core. > > Anyone know if there is an API to determine the state of a node. When AWS > auto scaling add a new node, I need to make sure it has before active > before I enable it in the load balancer. > > Bill > > > > > On Thu, Jan 3, 2013 at 9:10 AM, Mark Miller wrote: > >> >> http://wiki.apache.org/solr/CoreAdmin#UNLOAD >> >> - Mark >> >> On Jan 3, 2013, at 9:06 AM, Bill Au wrote: >> >>> Mark, >>>What do you mean by "unload them"? >>> >>> I am using an AWS load balancer with my auto scaling group in stead of >>> using Solr's built-in load balancer. I am no sharding my index. I am >>> using SolrCloud for replication only. I am doing local search on each >>> instance and sending all updates to the shard leader directly because I >>> want to minimize traffic between nodes during search and update >>> >>> Bill >>> >>> >>> On Wed, Jan 2, 2013 at 6:47 PM, Mark Miller >> wrote: >>> >>>> >>>> On Jan 2, 2013, at 5:51 PM, Bill Au wrote: >>>> >>>>> Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? >>>>> >>>>> My concern is that as AWS auto scaling add and remove instances to >>>>> SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow >>>>> indefinitely as removed instances will never be used again. AWS auto >>>>> scaling will keep on adding new instances, and there is no way to >> remove >>>>> them from Zookeeper, right? >>>> >>>> You can unload them and that removes them. >>>> >>>>> What's the effect of have all these phantom >>>>> nodes? >>>> >>>> Unless they are only replicas, they would need to be removed. >>>> >>>> Also, unless you are using elastic ips, >>>> https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. >>>> >>>> - Mark >> >>
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
Thanks, Mark. That does remove the node. And it seems to do so permanently. Even when I restart Solr after unloading, it does not join the SolrCloud cluster. And I can get it to re-join the cluster by creating the core. Anyone know if there is an API to determine the state of a node. When AWS auto scaling add a new node, I need to make sure it has before active before I enable it in the load balancer. Bill On Thu, Jan 3, 2013 at 9:10 AM, Mark Miller wrote: > > http://wiki.apache.org/solr/CoreAdmin#UNLOAD > > - Mark > > On Jan 3, 2013, at 9:06 AM, Bill Au wrote: > > > Mark, > > What do you mean by "unload them"? > > > > I am using an AWS load balancer with my auto scaling group in stead of > > using Solr's built-in load balancer. I am no sharding my index. I am > > using SolrCloud for replication only. I am doing local search on each > > instance and sending all updates to the shard leader directly because I > > want to minimize traffic between nodes during search and update > > > > Bill > > > > > > On Wed, Jan 2, 2013 at 6:47 PM, Mark Miller > wrote: > > > >> > >> On Jan 2, 2013, at 5:51 PM, Bill Au wrote: > >> > >>> Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > >>> > >>> My concern is that as AWS auto scaling add and remove instances to > >>> SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > >>> indefinitely as removed instances will never be used again. AWS auto > >>> scaling will keep on adding new instances, and there is no way to > remove > >>> them from Zookeeper, right? > >> > >> You can unload them and that removes them. > >> > >>> What's the effect of have all these phantom > >>> nodes? > >> > >> Unless they are only replicas, they would need to be removed. > >> > >> Also, unless you are using elastic ips, > >> https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. > >> > >> - Mark > >
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
http://wiki.apache.org/solr/CoreAdmin#UNLOAD - Mark On Jan 3, 2013, at 9:06 AM, Bill Au wrote: > Mark, > What do you mean by "unload them"? > > I am using an AWS load balancer with my auto scaling group in stead of > using Solr's built-in load balancer. I am no sharding my index. I am > using SolrCloud for replication only. I am doing local search on each > instance and sending all updates to the shard leader directly because I > want to minimize traffic between nodes during search and update > > Bill > > > On Wed, Jan 2, 2013 at 6:47 PM, Mark Miller wrote: > >> >> On Jan 2, 2013, at 5:51 PM, Bill Au wrote: >> >>> Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? >>> >>> My concern is that as AWS auto scaling add and remove instances to >>> SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow >>> indefinitely as removed instances will never be used again. AWS auto >>> scaling will keep on adding new instances, and there is no way to remove >>> them from Zookeeper, right? >> >> You can unload them and that removes them. >> >>> What's the effect of have all these phantom >>> nodes? >> >> Unless they are only replicas, they would need to be removed. >> >> Also, unless you are using elastic ips, >> https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. >> >> - Mark
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
Mark, What do you mean by "unload them"? I am using an AWS load balancer with my auto scaling group in stead of using Solr's built-in load balancer. I am no sharding my index. I am using SolrCloud for replication only. I am doing local search on each instance and sending all updates to the shard leader directly because I want to minimize traffic between nodes during search and update Bill On Wed, Jan 2, 2013 at 6:47 PM, Mark Miller wrote: > > On Jan 2, 2013, at 5:51 PM, Bill Au wrote: > > > Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > > > > My concern is that as AWS auto scaling add and remove instances to > > SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > > indefinitely as removed instances will never be used again. AWS auto > > scaling will keep on adding new instances, and there is no way to remove > > them from Zookeeper, right? > > You can unload them and that removes them. > > > What's the effect of have all these phantom > > nodes? > > Unless they are only replicas, they would need to be removed. > > Also, unless you are using elastic ips, > https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. > > - Mark
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
With AWS auto scaling, one can specify a minimum number of instances for an auto scaling group. So there should never be an insufficient number of replicas. Once can also specify a termination policy so that the newly added nodes are removed first. But with SolrCloud as long as there are enough replicas there is no wrong node to remove, right? AWS Beanstalk seems to be a wrapper for AWS auto scaling and other AWS elastic services. I am not sure if it offers the detail-grained control that you have when using auto scaling directly. On Wed, Jan 2, 2013 at 11:14 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > We've considered using AWS Beanstalk (hmm, what's the difference between > AWS auto scaling and elastic beanstalk? not sure.) for search-lucene.com , > but the idea of something adding and removing nodes seems scary. The > scariest part to me is automatic removal of wrong nodes that ends up in > data loss or insufficient number of replicas. > > But if somebody has done thing and has written up a how-to, I'd love to see > it! > > Otis > -- > Solr & ElasticSearch Support > http://sematext.com/ > > > > > > On Wed, Jan 2, 2013 at 5:51 PM, Bill Au wrote: > > > Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > > > > My concern is that as AWS auto scaling add and remove instances to > > SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > > indefinitely as removed instances will never be used again. AWS auto > > scaling will keep on adding new instances, and there is no way to remove > > them from Zookeeper, right? What's the effect of have all these phantom > > nodes? > > > > Bill > > >
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
We've considered using AWS Beanstalk (hmm, what's the difference between AWS auto scaling and elastic beanstalk? not sure.) for search-lucene.com , but the idea of something adding and removing nodes seems scary. The scariest part to me is automatic removal of wrong nodes that ends up in data loss or insufficient number of replicas. But if somebody has done thing and has written up a how-to, I'd love to see it! Otis -- Solr & ElasticSearch Support http://sematext.com/ On Wed, Jan 2, 2013 at 5:51 PM, Bill Au wrote: > Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > > My concern is that as AWS auto scaling add and remove instances to > SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > indefinitely as removed instances will never be used again. AWS auto > scaling will keep on adding new instances, and there is no way to remove > them from Zookeeper, right? What's the effect of have all these phantom > nodes? > > Bill >
Re: Solr 4.0 SolrCloud with AWS Auto Scaling
On Jan 2, 2013, at 5:51 PM, Bill Au wrote: > Is anyone running Solr 4.0 SolrCloud with AWS auto scaling? > > My concern is that as AWS auto scaling add and remove instances to > SolrCloud, the number of nodes in SolrCloud Zookeeper config will grow > indefinitely as removed instances will never be used again. AWS auto > scaling will keep on adding new instances, and there is no way to remove > them from Zookeeper, right? You can unload them and that removes them. > What's the effect of have all these phantom > nodes? Unless they are only replicas, they would need to be removed. Also, unless you are using elastic ips, https://issues.apache.org/jira/browse/SOLR-4078 may be of interest. - Mark
