Re: Dinamically change master
Back with my master resiliency need, talking with Upayavira we discovered we were proposing the same solution :-) This can be useful if you don't have a VIP with master/backup polling policy. It goes like this: there are 2 host for indexing, one is the main and one is the backup one, the backup one is slave of the main one and the main one is also master of N hosts which will be used for searching. If the main master goes down then the backup one will be used for indexing and/or serving search slaves. This last feature can be done defining an external properties file for each search slave which will contain the URL to master (pointed inside the replication request handler tag of solrconfig.xml), so if these search slaves run on multi core one has only to change properties file URL and issue a http://SLAVEURL/solr/admin/cores?action=RELOAD&core=core0 to get polling the backup master. Cheers, Tommaso 2010/12/1 Tommaso Teofili > Thanks Upayavira, that sounds very good. > > p.s.: > I read that page some weeks ago and didn't get back to check on it. > > > 2010/12/1 Upayavira > >> Note, all extracted from http://wiki.apache.org/solr/SolrReplication >> >> You'd put: >> >> >> >> >>startup >>commit >> >> >> >> into every box you want to be able to act as a master, then use: >> >> http://slave_host:port >> /solr/replication?command=fetchindex&masterUrl=> master URL> >> >> As the above page says better than I can, "It is possible to pass on >> extra attribute 'masterUrl' or other attributes like 'compression' (or >> any other parameter which is specified in the tag) to >> do a one time replication from a master. This obviates the need for >> hardcoding the master in the slave." >> >> HTH, Upayavira >> >> On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili" >> wrote: >> > Hi Upayavira, >> > this is a good start for solving my problem, can you please tell how >> does >> > such a replication URL look like? >> > Thanks, >> > Tommaso >> > >> > 2010/12/1 Upayavira >> > >> > > Hi Tommaso, >> > > >> > > I believe you can tell each server to act as a master (which means it >> > > can have its indexes pulled from it). >> > > >> > > You can then include the master hostname in the URL that triggers a >> > > replication process. Thus, if you triggered replication from outside >> > > solr, you'd have control over which master you pull from. >> > > >> > > Does this answer your question? >> > > >> > > Upayavira >> > > >> > > >> > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler" >> > > wrote: >> > > > Hi Tommaso, >> > > > >> > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: >> > > > >> > > > > Hi all, >> > > > > >> > > > > in a replication environment if the host where the master is >> running >> > > > > goes >> > > > > down for some reason, is there a way to communicate to the slaves >> to >> > > > > point >> > > > > to a different (backup) master without manually changing >> > > > > configuration (and >> > > > > restarting the slaves or their cores)? >> > > > > >> > > > > Basically I'd like to be able to change the replication master >> > > > > dinamically >> > > > > inside the slaves. >> > > > > >> > > > > Do you have any idea of how this could be achieved? >> > > > >> > > > One common approach is to use VIP (virtual IP) support provided by >> > > > load balancers. >> > > > >> > > > Your slaves are configured to use a VIP to talk to the master, so >> that >> > > > it's easy to dynamically change which master they use, via updates >> to >> > > > the load balancer config. >> > > > >> > > > -- Ken >> > > > >> > > > -- >> > > > Ken Krugler >> > > > +1 530-210-6378 >> > > > http://bixolabs.com >> > > > e l a s t i c w e b m i n i n g >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > >> > >
Re: Dinamically change master
Thanks Upayavira, that sounds very good. p.s.: I read that page some weeks ago and didn't get back to check on it. 2010/12/1 Upayavira > Note, all extracted from http://wiki.apache.org/solr/SolrReplication > > You'd put: > > > > >startup >commit > > > > into every box you want to be able to act as a master, then use: > > http://slave_host:port/solr/replication?command=fetchindex&masterUrl= master URL> > > As the above page says better than I can, "It is possible to pass on > extra attribute 'masterUrl' or other attributes like 'compression' (or > any other parameter which is specified in the tag) to > do a one time replication from a master. This obviates the need for > hardcoding the master in the slave." > > HTH, Upayavira > > On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili" > wrote: > > Hi Upayavira, > > this is a good start for solving my problem, can you please tell how does > > such a replication URL look like? > > Thanks, > > Tommaso > > > > 2010/12/1 Upayavira > > > > > Hi Tommaso, > > > > > > I believe you can tell each server to act as a master (which means it > > > can have its indexes pulled from it). > > > > > > You can then include the master hostname in the URL that triggers a > > > replication process. Thus, if you triggered replication from outside > > > solr, you'd have control over which master you pull from. > > > > > > Does this answer your question? > > > > > > Upayavira > > > > > > > > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler" > > > wrote: > > > > Hi Tommaso, > > > > > > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: > > > > > > > > > Hi all, > > > > > > > > > > in a replication environment if the host where the master is > running > > > > > goes > > > > > down for some reason, is there a way to communicate to the slaves > to > > > > > point > > > > > to a different (backup) master without manually changing > > > > > configuration (and > > > > > restarting the slaves or their cores)? > > > > > > > > > > Basically I'd like to be able to change the replication master > > > > > dinamically > > > > > inside the slaves. > > > > > > > > > > Do you have any idea of how this could be achieved? > > > > > > > > One common approach is to use VIP (virtual IP) support provided by > > > > load balancers. > > > > > > > > Your slaves are configured to use a VIP to talk to the master, so > that > > > > it's easy to dynamically change which master they use, via updates to > > > > the load balancer config. > > > > > > > > -- Ken > > > > > > > > -- > > > > Ken Krugler > > > > +1 530-210-6378 > > > > http://bixolabs.com > > > > e l a s t i c w e b m i n i n g > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: Dinamically change master
Note, all extracted from http://wiki.apache.org/solr/SolrReplication You'd put: startup commit into every box you want to be able to act as a master, then use: http://slave_host:port/solr/replication?command=fetchindex&masterUrl= As the above page says better than I can, "It is possible to pass on extra attribute 'masterUrl' or other attributes like 'compression' (or any other parameter which is specified in the tag) to do a one time replication from a master. This obviates the need for hardcoding the master in the slave." HTH, Upayavira On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili" wrote: > Hi Upayavira, > this is a good start for solving my problem, can you please tell how does > such a replication URL look like? > Thanks, > Tommaso > > 2010/12/1 Upayavira > > > Hi Tommaso, > > > > I believe you can tell each server to act as a master (which means it > > can have its indexes pulled from it). > > > > You can then include the master hostname in the URL that triggers a > > replication process. Thus, if you triggered replication from outside > > solr, you'd have control over which master you pull from. > > > > Does this answer your question? > > > > Upayavira > > > > > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler" > > wrote: > > > Hi Tommaso, > > > > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: > > > > > > > Hi all, > > > > > > > > in a replication environment if the host where the master is running > > > > goes > > > > down for some reason, is there a way to communicate to the slaves to > > > > point > > > > to a different (backup) master without manually changing > > > > configuration (and > > > > restarting the slaves or their cores)? > > > > > > > > Basically I'd like to be able to change the replication master > > > > dinamically > > > > inside the slaves. > > > > > > > > Do you have any idea of how this could be achieved? > > > > > > One common approach is to use VIP (virtual IP) support provided by > > > load balancers. > > > > > > Your slaves are configured to use a VIP to talk to the master, so that > > > it's easy to dynamically change which master they use, via updates to > > > the load balancer config. > > > > > > -- Ken > > > > > > -- > > > Ken Krugler > > > +1 530-210-6378 > > > http://bixolabs.com > > > e l a s t i c w e b m i n i n g > > > > > > > > > > > > > > > > > > > > >
Re: Dinamically change master
Hi Upayavira, this is a good start for solving my problem, can you please tell how does such a replication URL look like? Thanks, Tommaso 2010/12/1 Upayavira > Hi Tommaso, > > I believe you can tell each server to act as a master (which means it > can have its indexes pulled from it). > > You can then include the master hostname in the URL that triggers a > replication process. Thus, if you triggered replication from outside > solr, you'd have control over which master you pull from. > > Does this answer your question? > > Upayavira > > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler" > wrote: > > Hi Tommaso, > > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: > > > > > Hi all, > > > > > > in a replication environment if the host where the master is running > > > goes > > > down for some reason, is there a way to communicate to the slaves to > > > point > > > to a different (backup) master without manually changing > > > configuration (and > > > restarting the slaves or their cores)? > > > > > > Basically I'd like to be able to change the replication master > > > dinamically > > > inside the slaves. > > > > > > Do you have any idea of how this could be achieved? > > > > One common approach is to use VIP (virtual IP) support provided by > > load balancers. > > > > Your slaves are configured to use a VIP to talk to the master, so that > > it's easy to dynamically change which master they use, via updates to > > the load balancer config. > > > > -- Ken > > > > -- > > Ken Krugler > > +1 530-210-6378 > > http://bixolabs.com > > e l a s t i c w e b m i n i n g > > > > > > > > > > > > >
Re: Dinamically change master
Hi Tommaso, I believe you can tell each server to act as a master (which means it can have its indexes pulled from it). You can then include the master hostname in the URL that triggers a replication process. Thus, if you triggered replication from outside solr, you'd have control over which master you pull from. Does this answer your question? Upayavira On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler" wrote: > Hi Tommaso, > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: > > > Hi all, > > > > in a replication environment if the host where the master is running > > goes > > down for some reason, is there a way to communicate to the slaves to > > point > > to a different (backup) master without manually changing > > configuration (and > > restarting the slaves or their cores)? > > > > Basically I'd like to be able to change the replication master > > dinamically > > inside the slaves. > > > > Do you have any idea of how this could be achieved? > > One common approach is to use VIP (virtual IP) support provided by > load balancers. > > Your slaves are configured to use a VIP to talk to the master, so that > it's easy to dynamically change which master they use, via updates to > the load balancer config. > > -- Ken > > -- > Ken Krugler > +1 530-210-6378 > http://bixolabs.com > e l a s t i c w e b m i n i n g > > > > > >
Re: Dinamically change master
Hi, Thanks Jacob and Ken for your replies. I am not able to change project architecture to add Lucandra even if it looks like a nice solution. Going the VIP way can definitely an option even if I'd be more keen to solve that "inside" Solr. I am thinking to try and play with Collection Distribution scripts or doing some custom stuff. But, please, if you have any other advice let me know. Thanks again. Tommaso 2010/11/30 Ken Krugler > Hi Tommaso, > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: > > Hi all, >> >> in a replication environment if the host where the master is running goes >> down for some reason, is there a way to communicate to the slaves to point >> to a different (backup) master without manually changing configuration >> (and >> restarting the slaves or their cores)? >> >> Basically I'd like to be able to change the replication master dinamically >> inside the slaves. >> >> Do you have any idea of how this could be achieved? >> > > One common approach is to use VIP (virtual IP) support provided by load > balancers. > > Your slaves are configured to use a VIP to talk to the master, so that it's > easy to dynamically change which master they use, via updates to the load > balancer config. > > -- Ken > > -- > Ken Krugler > +1 530-210-6378 > http://bixolabs.com > e l a s t i c w e b m i n i n g > > > > > >
Re: Dinamically change master
Hi Tommaso, On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote: Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? One common approach is to use VIP (virtual IP) support provided by load balancers. Your slaves are configured to use a VIP to talk to the master, so that it's easy to dynamically change which master they use, via updates to the load balancer config. -- Ken -- Ken Krugler +1 530-210-6378 http://bixolabs.com e l a s t i c w e b m i n i n g
Re: Dinamically change master
Your best bet might be to look into Lucandra: https://github.com/tjake/Lucandra On Tue, Nov 30, 2010 at 10:41 AM, Tommaso Teofili wrote: > Hi all, > > in a replication environment if the host where the master is running goes > down for some reason, is there a way to communicate to the slaves to point > to a different (backup) master without manually changing configuration (and > restarting the slaves or their cores)? > > Basically I'd like to be able to change the replication master dinamically > inside the slaves. > > Do you have any idea of how this could be achieved? > > Thanks in advance for any help. > Regards, > Tommaso > -- Jacob Elder @jelder (646) 535-3379
Dinamically change master
Hi all, in a replication environment if the host where the master is running goes down for some reason, is there a way to communicate to the slaves to point to a different (backup) master without manually changing configuration (and restarting the slaves or their cores)? Basically I'd like to be able to change the replication master dinamically inside the slaves. Do you have any idea of how this could be achieved? Thanks in advance for any help. Regards, Tommaso
