bq: So inorder for me move the shards to its own instances, I will have to take a down time and move the newly created shards & replicas to its own instances.
No, this is not true. The easiest way to move things around is use the collections API ADDREPLICA command after splitting. Let's call this particular shard S1 on machine M1, and the results of the SPLITSHARD command S1.1 and S1.2 Further, let's say that your goal is to move _one_ of the subshards from machine M1 to M2 So the sequence is: 1> issue SPLITSHARD and wait for it to complete. This requires no downtime and after the split the old shard becomes inactive and the two new subshards are servicing all requests. I'd probably stop indexing during this operation just to be on the safe side, although that's not necessary. So now you have both S1.1 and S1.2 running on M1 2> Use the ADDREPLICA command to add a replica of S1.2 to M2. Again, no downtime required. Wait until the new replica is "active", at which point it's fully operational. So now we have S1.1 and S1.2 running on M1 and S1.2.2 running on M2. 3> Use the DELETEREPLICA command to remove S1.2 from M1. Now you have S1.1 running on M1 and S1.2.1 running on M2. No downtime during any of this. 4> You should be able to delete S1 now from M1 just to tidy up. 5> Repeat for the other shards. Best, Erick On Sun, Apr 17, 2016 at 3:09 PM, Jay Potharaju <jspothar...@gmail.com> wrote: > Erik thanks for the reply. In my current prod setup I anticipate the number > of documents to grow almost 5 times by the end of the year and therefore > planning on how to scale when required. We have high query volume and > growing dataset, that is why would like to scale by sharding & replication. > > In my dev sandbox, I have 2 replicas & 2 shards created using compositeId > as my routing option. If I split the shard, it will create 2 new shards on > each the solr instances including replicas and my request will start going > to the new shards. > So inorder for me move the shards to its own instances, I will have to > take a down time and move the newly created shards & replicas to its own > instances. Is that a correct interpretation of the how shard splitting > would work > > I was hoping that solr will automagically split the existing shard & > create replicas on the new instances rather than the existing nodes. That > is why I said the current shard splitting will not work for me. > Thanks > > On Sat, Apr 16, 2016 at 8:08 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > >> Why don't you think splitting the shards will do what you need? >> Admittedly it will have to be applied to each shard and will >> double the number of shards you have, that's the current >> limitation. At the end, though, you will have 4 shards when >> you used to have 2 and you can move them around to whatever >> hardware you can scrape up. >> >> This assumes you're using the default compositeId routing >> scheme and not implicit routing. If you are using compositeId >> there is no provision to add another shard. >> >> As far as SOLR-5025 is concerned, nobody's working on that >> that I know of. >> >> I have to ask though whether you've tuned your existing >> machines. How many docs are on each? Why do you think >> you need more shards? Query speed? OOMs? Java heaps >> getting too big? >> >> Best, >> Erick >> >> On Fri, Apr 15, 2016 at 10:50 PM, Jay Potharaju <jspothar...@gmail.com> >> wrote: >> > I found ticket https://issues.apache.org/jira/browse/SOLR-5025 which >> talks >> > about sharding in solrcloud. Are there any plans to address this issue in >> > near future? >> > Can any of the users on the forum comment how they are handling this >> > scenario in production? >> > Thanks >> > >> > On Fri, Apr 15, 2016 at 4:28 PM, Jay Potharaju <jspothar...@gmail.com> >> > wrote: >> > >> >> Hi, >> >> I have an existing collection which has 2 shards, one on each node in >> the >> >> cloud. Now I want to split the existing collection into 3 shards >> because of >> >> increase in volume of data. And create this new shard on a new node in >> the >> >> solrCloud. >> >> >> >> I read about splitting a shard & creating a shard, but not sure it will >> >> work. >> >> >> >> Any suggestions how are others handling this scenario in production. >> >> -- >> >> Thanks >> >> Jay >> >> >> >> >> > >> > >> > >> > -- >> > Thanks >> > Jay Potharaju >> > > > > -- > Thanks > Jay Potharaju
