Hours sounds too long indeed. We recently had a client with several thousand collections, but restart wasn't taking hours...
Otis Solr & ElasticSearch Support http://sematext.com/ On Mar 20, 2014 5:49 PM, "Erick Erickson" <erickerick...@gmail.com> wrote: > How many total replicas are we talking here? > As in how many shards and, for each shard, > how many replicas? I'm not asking for a long list > here, just if you have a bazillion replicas in aggregate. > > Hours is surprising. > > Best, > Erick > > On Thu, Mar 20, 2014 at 2:17 PM, Chris W <chris1980....@gmail.com> wrote: > > Thanks, Shalin. Making clusterstate.json on a collection basis sounds > > awesome. > > > > I am not having problems with #2 . #3 is a major time hog in my > > environment. I have over 300 +collections and restarting the entire > cluster > > takes in the order of hours. (2-3 hour). Can you explain more about the > > leaderVoteWait setting? > > > > > > > > > > On Thu, Mar 20, 2014 at 1:28 PM, Shalin Shekhar Mangar < > > shalinman...@gmail.com> wrote: > > > >> There are no arbitrary limits on the number of collections but yes > >> there are practical limits. For example, the cluster state can become > >> a bottleneck. There is a lot of work happening on finding and > >> addressing these problems. See > >> https://issues.apache.org/jira/browse/SOLR-5381 > >> > >> Boot up time is because of: > >> 1) Core discovery, schema/config parsing etc > >> 2) Transaction log replay on startup > >> 3) Wait time for enough replicas to become available before leader > >> election happens > >> > >> You can't do much about 1 right now I think. For #2, you can keep your > >> transaction logs smaller by a hard commit before shutdown. For #3 > >> there is a leaderVoteWait settings but I'd rather not touch that > >> unless it becomes a problem. > >> > >> On Fri, Mar 21, 2014 at 1:39 AM, Chris W <chris1980....@gmail.com> > wrote: > >> > Hi there > >> > > >> > Is there a limit on the # of collections solrcloud can support? Can > >> > zk/solrcloud handle 1000s of collections? > >> > > >> > Also i see that the bootup time of solrcloud increases with increase > in # > >> > of cores. I do not have any expensive warm up queries. How do i > speedup > >> > solr startup? > >> > > >> > -- > >> > Best > >> > -- > >> > C > >> > >> > >> > >> -- > >> Regards, > >> Shalin Shekhar Mangar. > >> > > > > > > > > -- > > Best > > -- > > C >
