HA and load balancing Qauestion
Hello there, Howdy. I'd like to know if I can configure Multiple Solr instances working with a single read-only index partition for failover/HA and load balancing purposes. Or is there any other way that Solr has built-in features to handle the same. Any ideas/thoughts are greatly appreciated. -- Thanks, Venkatesh "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away." - Antoine de Saint-Exupéry
Re: HA and load balancing Qauestion
On 3/8/07, Venkatesh Seetharam <[EMAIL PROTECTED]> wrote: Howdy. I'd like to know if I can configure Multiple Solr instances working with a single read-only index partition for failover/HA and load balancing purposes. Or is there any other way that Solr has built-in features to handle the same. On the front-end, HTTP is easily load-balanced via software or hardware loadbalancers. To distribute a single index to multiple solr searchers, see http://wiki.apache.org/solr/CollectionDistribution You don't have to do it that way though... if you have another mechanism to get the index to the searchers, that could work too. -Yonik
Re: HA and load balancing Qauestion
Thanks for the reply Yonik. I'm not using HTTP and using a wrapper to wrap Solr for searching. I'm using RPC to talk to multiple servers. Can I point 2 Solr instances to the same index partition, having the same path in SolrConfig? Is this safe or I need to make 2 copies of the same index partition and point the Solr instances to these copies? Since my index partition lives on a shared NetApp mount, I'd like to use the same index partition for multiple Solr instances. Thanks for any help, Venkatesh On 3/8/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 3/8/07, Venkatesh Seetharam <[EMAIL PROTECTED]> wrote: > Howdy. I'd like to know if I can configure Multiple Solr instances working > with a single read-only index partition for failover/HA and load balancing > purposes. Or is there any other way that Solr has built-in features to > handle the same. On the front-end, HTTP is easily load-balanced via software or hardware loadbalancers. To distribute a single index to multiple solr searchers, see http://wiki.apache.org/solr/CollectionDistribution You don't have to do it that way though... if you have another mechanism to get the index to the searchers, that could work too. -Yonik
Re: HA and load balancing Qauestion
On 3/8/07, Venkatesh Seetharam <[EMAIL PROTECTED]> wrote: Can I point 2 Solr instances to the same index partition, having the same path in SolrConfig? Yes, that should work fine. -Yonik
Re: HA and load balancing Qauestion
: > Can I point 2 Solr instances to the same index partition, having the same : > path in SolrConfig? : : Yes, that should work fine. you might run into some weirdness if you send updates/delets to both instances .. basically you'll want to configure all but one instace as a "slave" and anytime you do a commit on the master you'll want to trigger a commit on all of the slaves so that they reopen the index. (just like using the snapshot scripts, except you don't need to snapshoot, snappull, or snapinstall) -Hoss
Re: HA and load balancing Qauestion
Thanks Yonik and Chris for your confirmation. Chris, these are read-only index partitions. I perform updates/deletions on a master index which will be snapshotted at some fixed intervals. I'll look into the Collection Distribution of Solr. Sounds very powerful. I'm struck with Solr requiring an index directory under dataDir configured in SolrConfig. Why does it not take a complete path_to_index configured under dataDir but append "index"? Is there anyway I can workaround this? org.apache.solr.core.SolrCore: this.index_path = dataDir + "/" + "index"; Thanks, Venkatesh On 3/8/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : > Can I point 2 Solr instances to the same index partition, having the same : > path in SolrConfig? : : Yes, that should work fine. you might run into some weirdness if you send updates/delets to both instances .. basically you'll want to configure all but one instace as a "slave" and anytime you do a commit on the master you'll want to trigger a commit on all of the slaves so that they reopen the index. (just like using the snapshot scripts, except you don't need to snapshoot, snappull, or snapinstall) -Hoss
Re: HA and load balancing Qauestion
: I'm struck with Solr requiring an index directory under dataDir configured : in SolrConfig. Why does it not take a complete path_to_index configured : under dataDir but append "index"? Is there anyway I can workaround this? i think at one time we were assuming there might be other types of data you'd want Solr to store besides the index... the assumption is that you tell Solr where you want it to keep all of it's data, and after that you shouldn't care what lives in that directory. if i remember correctly, the dataDir is alwo where the solr snapshots and temp dirs get put ... if Solr let you configure the indexdir directly, we'd need you to also configure those locations seperately -- except that they have to be on the same physical disk for hardlinks to work, so it's relaly just a lot simpler if you tell SOlr the dataDir and let it take care of everything else. (except in your case where you *want* to take care of it ... Solr wasn't really designed for that case) -Hoss
Re: HA and load balancing Qauestion
Thanks Hoss for the clarification. I think I can make a copy of the index for searching and rename 'em. I think I can work around this one but good to know the bigger picture. Venkatesh On 3/8/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : I'm struck with Solr requiring an index directory under dataDir configured : in SolrConfig. Why does it not take a complete path_to_index configured : under dataDir but append "index"? Is there anyway I can workaround this? i think at one time we were assuming there might be other types of data you'd want Solr to store besides the index... the assumption is that you tell Solr where you want it to keep all of it's data, and after that you shouldn't care what lives in that directory. if i remember correctly, the dataDir is alwo where the solr snapshots and temp dirs get put ... if Solr let you configure the indexdir directly, we'd need you to also configure those locations seperately -- except that they have to be on the same physical disk for hardlinks to work, so it's relaly just a lot simpler if you tell SOlr the dataDir and let it take care of everything else. (except in your case where you *want* to take care of it ... Solr wasn't really designed for that case) -Hoss
