oops, the actual url is -http://64.251.14.47:1981/solr/
Also, another issue that needs to be raised is the creation of cores from the "core admin" section of the gui, doesnt really work well, it creates files but then they do not work (again i am using 4.4) On Wed, Oct 16, 2013 at 4:12 PM, Chris <christu...@gmail.com> wrote: > Hi, > > Please find the clusterstate.json as below: > > I have created a dev environment on one of my servers so that you can see > the issue live - http://64.251.14.47:1984/solr/ > > Also, There seems to be something wrong in zookeeper, when we try to add > documents using solrj, it works fine as long as load of insert is not much, > but once we start doing many inserts, then it throws a lot of errors... > > I am doing something like - > > CloudSolrServer solrCoreCloud = new CloudSolrServer(cloudURL); > solrCoreCloud.setDefaultCollection("Image"); > UpdateResponse up = solrCoreCloud.addBean(resultItem); > UpdateResponse upr = solrCoreCloud.commit(); > > > > clusterstate.json --- > > { > "collection1":{ > "shards":{ > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{"core_node4":{ > "state":"active", > "core":"collection1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{"core_node5":{ > "state":"active", > "core":"collection1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{ > "core_node2":{ > "state":"active", > "core":"collection1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr"}, > "core_node6":{ > "state":"active", > "core":"collection1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}, > "shard5":{ > "range":"4ccc0000-7fffffff", > "state":"active", > "replicas":{"core_node3":{ > "state":"active", > "core":"collection1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "Web":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{"core_node2":{ > "state":"active", > "core":"Web_shard1_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{"core_node3":{ > "state":"active", > "core":"Web_shard2_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{"core_node4":{ > "state":"active", > "core":"Web_shard3_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{"core_node5":{ > "state":"active", > "core":"Web_shard4_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}, > "shard5":{ > "range":null, > "state":"active", > "replicas":{"core_node1":{ > "state":"active", > "core":"Web_shard5_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "Image":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{"core_node1":{ > "state":"active", > "core":"Image_shard1_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{"core_node2":{ > "state":"active", > "core":"Image_shard2_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{"core_node3":{ > "state":"active", > "core":"Image_shard3_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{"core_node5":{ > "state":"active", > "core":"Image_shard4_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr";, > "leader":"true"}}}, > "shard5":{ > "range":null, > "state":"active", > "replicas":{"core_node4":{ > "state":"active", > "core":"Image_shard5_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "image":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{ > "core_node1":{ > "state":"active", > "core":"image_shard1_replica2", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr"}, > "core_node3":{ > "state":"active", > "core":"image_shard1_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr";, > "leader":"true"}}}, > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{ > "core_node9":{ > "state":"active", > "core":"image_shard2_replica2", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr"}, > "core_node5":{ > "state":"active", > "core":"image_shard2_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{ > "core_node4":{ > "state":"active", > "core":"image_shard3_replica2", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr"}, > "core_node7":{ > "state":"active", > "core":"image_shard3_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{ > "core_node2":{ > "state":"active", > "core":"image_shard4_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr"}, > "core_node6":{ > "state":"active", > "core":"image_shard4_replica2", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}, > "shard5":{ > "range":"4ccc0000-7fffffff", > "state":"active", > "replicas":{ > "core_node8":{ > "state":"active", > "core":"image_shard5_replica2", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr"}, > "core_node10":{ > "state":"active", > "core":"image_shard5_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "Reference":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{"core_node1":{ > "state":"active", > "core":"Reference_shard1_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{"core_node2":{ > "state":"active", > "core":"Reference_shard3_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{"core_node5":{ > "state":"active", > "core":"Reference_shard4_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr";, > "leader":"true"}}}, > "shard5":{ > "range":"4ccc0000-7fffffff", > "state":"active", > "replicas":{"core_node4":{ > "state":"active", > "core":"Reference_shard5_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard2":{ > "range":null, > "state":"active", > "replicas":{"core_node3":{ > "state":"active", > "core":"Reference_shard2_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "web":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{ > "core_node4":{ > "state":"active", > "core":"web_shard1_replica2", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr"}, > "core_node2":{ > "state":"active", > "core":"web_shard1_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}, > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{ > "core_node6":{ > "state":"active", > "core":"web_shard2_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr"}, > "core_node7":{ > "state":"active", > "core":"web_shard2_replica2", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{ > "core_node9":{ > "state":"active", > "core":"web_shard3_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr"}, > "core_node1":{ > "state":"active", > "core":"web_shard3_replica2", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}, > "shard4":{ > "range":"19990000-4ccbffff", > "state":"active", > "replicas":{ > "core_node3":{ > "state":"active", > "core":"web_shard4_replica2", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr"}, > "core_node5":{ > "state":"active", > "core":"web_shard4_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard5":{ > "range":"4ccc0000-7fffffff", > "state":"active", > "replicas":{ > "core_node8":{ > "state":"active", > "core":"web_shard5_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr"}, > "core_node10":{ > "state":"active", > "core":"web_shard5_replica2", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}}, > "router":"compositeId"}, > "News":{ > "shards":{ > "shard1":{ > "range":"80000000-b332ffff", > "state":"active", > "replicas":{"core_node1":{ > "state":"active", > "core":"News_shard1_replica1", > "node_name":"64.251.14.47:1984_solr", > "base_url":"http://64.251.14.47:1984/solr";, > "leader":"true"}}}, > "shard2":{ > "range":"b3330000-e665ffff", > "state":"active", > "replicas":{"core_node3":{ > "state":"active", > "core":"News_shard2_replica1", > "node_name":"64.251.14.47:1983_solr", > "base_url":"http://64.251.14.47:1983/solr";, > "leader":"true"}}}, > "shard3":{ > "range":"e6660000-1998ffff", > "state":"active", > "replicas":{"core_node2":{ > "state":"active", > "core":"News_shard3_replica1", > "node_name":"64.251.14.47:1982_solr", > "base_url":"http://64.251.14.47:1982/solr";, > "leader":"true"}}}, > "shard5":{ > "range":"4ccc0000-7fffffff", > "state":"active", > "replicas":{"core_node5":{ > "state":"active", > "core":"News_shard5_replica1", > "node_name":"64.251.14.47:1985_solr", > "base_url":"http://64.251.14.47:1985/solr";, > "leader":"true"}}}, > "shard4":{ > "range":null, > "state":"active", > "replicas":{"core_node4":{ > "state":"active", > "core":"News_shard4_replica1", > "node_name":"64.251.14.47:1981_solr", > "base_url":"http://64.251.14.47:1981/solr";, > "leader":"true"}}}}, > "router":"compositeId"}} > > > > > On Wed, Oct 16, 2013 at 2:19 PM, <primoz.sk...@policija.si> wrote: > >> Yap, you are right - I only created extra replicas with cores API. For a >> new shard I had to use "split shard" command. >> >> My apologies. >> >> Primož >> >> >> >> From: Shalin Shekhar Mangar <shalinman...@gmail.com> >> To: solr-user@lucene.apache.org >> Date: 16.10.2013 10:45 >> Subject: Re: Regarding Solr Cloud issue... >> >> >> >> If the initial collection was created with a numShards parameter (and >> hence >> compositeId router then there was no way to create a new logical shard. >> You >> can add replicas with the core admin API but only to shards that already >> exist. A new logical shard can only be created by splitting an existing >> one. >> >> The "createshard" API also has the same limitation -- it cannot create a >> shard for a collection with compositeId router. It is supposed to be used >> for collections with custom sharding (i.e. "implicit" router). In such >> collections, there is no concept of a hash range and routing is done >> explicitly by the user using the "shards" parameter in the request or by >> sending the request to the target core/node directly. >> >> So, in summary, attempting to add a new logical shard to a collection with >> compositeId router via CoreAdmin APIs is wrong, unsupported and should be >> disallowed. Adding replicas to existing logical shards is okay though. >> >> >> On Wed, Oct 16, 2013 at 12:56 PM, <primoz.sk...@policija.si> wrote: >> >> > If I am not mistaken the only way to create a new shard from a >> collection >> > in 4.4.0 was to use cores API. That worked fine for me until I used >> > *other* cores API commands. Those usually produced null ranges. >> > >> > In 4.5.0 this is fixed with newly added commands "createshard" etc. to >> the >> > collections API, right? >> > >> > Primoz >> > >> > >> > >> > From: Shalin Shekhar Mangar <shalinman...@gmail.com> >> > To: solr-user@lucene.apache.org >> > Date: 16.10.2013 09:06 >> > Subject: Re: Regarding Solr Cloud issue... >> > >> > >> > >> > Chris, can you post your complete clusterstate.json? Do all shards have >> a >> > null range? Also, did you issue any core admin CREATE commands apart >> from >> > the create collection api. >> > >> > Primoz, I was able to reproduce this but by doing an illegal operation. >> > Suppose I create a collection with numShards=5 and then I issue a core >> > admin create command such as: >> > >> > >> >> http://localhost:8983/solr/admin/cores?action=CREATE&name=xyz&collection=mycollection51&shard=shard6 >> >> > >> > >> > Then a "shard6" is added to the collection with a null range. This is a >> > bug >> > because we should never allow such a core admin create to succeed >> anyway. >> > I'll open an issue. >> > >> > >> > >> > On Wed, Oct 16, 2013 at 11:49 AM, <primoz.sk...@policija.si> wrote: >> > >> > > I sometimes also do get null ranges when doing colletions/cores API >> > > actions CREATE or/and UNLOAD, etc... In 4.4.0 that was not easily >> fixed >> > > because zkCli had problems with "putfile" command, but in 4.5.0 it >> works >> > > OK. All you have to do is "download" clusterstate.json from ZK ("get >> > > /clusterstate.json"), fix ranges to appropriate values and upload the >> > file >> > > back to ZK with zkCli. >> > > >> > > But why those null ranges happen at all is beyond me :) >> > > >> > > Primoz >> > > >> > > >> > > >> > > From: Shalin Shekhar Mangar <shalinman...@gmail.com> >> > > To: solr-user@lucene.apache.org >> > > Date: 16.10.2013 07:37 >> > > Subject: Re: Regarding Solr Cloud issue... >> > > >> > > >> > > >> > > I'm sorry I am not able to reproduce this issue. >> > > >> > > I started 5 solr-4.4 instances. >> > > I copied example directory into example1, example2, example3 and >> > example4 >> > > cd example; java -Dbootstrap_confdir=./solr/collection1/conf >> > > -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar >> > > cd example1; java -Djetty.port=7574 -DzkHost=localhost:9983 -jar >> > start.jar >> > > cd example2; java -Djetty.port=7575 -DzkHost=localhost:9983 -jar >> > start.jar >> > > cd example3; java -Djetty.port=7576 -DzkHost=localhost:9983 -jar >> > start.jar >> > > cd example4; java -Djetty.port=7577 -DzkHost=localhost:9983 -jar >> > start.jar >> > > >> > > After that I invoked: >> > > >> > > >> > >> > >> >> http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection51&numShards=5&replicationFactor=1 >> >> > >> > > >> > > >> > > I can see all shards having non-null ranges in clusterstate. >> > > >> > > >> > > On Tue, Oct 15, 2013 at 8:47 PM, Chris <christu...@gmail.com> wrote: >> > > >> > > > Hi Shalin,. >> > > > >> > > > Thank you for your quick reply. I appreciate all the help. >> > > > >> > > > I started the solr cloud servers first...with 5 nodes. >> > > > >> > > > then i issued a command like below to create the shards - >> > > > >> > > > >> > > > >> > > >> > > >> > >> > >> >> http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=5&replicationFactor=1 >> >> > >> > > >> > > > < >> > > > >> > > >> > > >> > >> > >> >> http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=4 >> >> > >> > > >> > > > > >> > > > >> > > > Please advice. >> > > > >> > > > Regards, >> > > > Chris >> > > > >> > > > >> > > > On Tue, Oct 15, 2013 at 8:07 PM, Shalin Shekhar Mangar < >> > > > shalinman...@gmail.com> wrote: >> > > > >> > > > > How did you create these shards? Can you tell us how to reproduce >> > the >> > > > > issue? >> > > > > >> > > > > Any shard in a collection with compositeId router should never >> have >> > > null >> > > > > ranges. >> > > > > >> > > > > >> > > > > On Tue, Oct 15, 2013 at 7:07 PM, Chris <christu...@gmail.com> >> wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > I am using solr 4.4 as cloud. while creating shards i see that >> the >> > > last >> > > > > > shard has range of "null". i am not sure if this is a bug. >> > > > > > >> > > > > > I am stuck with having null value for the range in >> > clusterstate.json >> > > > > > (attached below) >> > > > > > >> > > > > > "shard5":{ "range":null, "state":"active", >> > > "replicas":{"core_node1":{ >> > > > > > "state":"active", "core":"Web_shard5_replica1", >> > > > > > "node_name":"domain-name.com:1981_solr", "base_url":" >> > > > > > http://domain-name.com:1981/solr";, "leader":"true"}}}}, >> > > > > > "router":"compositeId"}, >> > > > > > >> > > > > > I tried to use zookeeper cli to change this, but it was not able >> > to. >> > > I >> > > > > > tried to locate this file, but didn't find it anywhere. >> > > > > > >> > > > > > Can you please let me know how do i change the range from null >> to >> > > > > something >> > > > > > meaningful? i have the range that i need, so if i can find the >> > file, >> > > > > maybe >> > > > > > i can change it manually. >> > > > > > >> > > > > > My next question is - can we have a catch all for ranges, i mean >> > if >> > > > > things >> > > > > > don't match any other range then insert in this shard..is this >> > > > possible? >> > > > > > >> > > > > > Kindly advice. >> > > > > > Chris >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Regards, >> > > > > Shalin Shekhar Mangar. >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > Regards, >> > > Shalin Shekhar Mangar. >> > > >> > > >> > >> > >> > -- >> > Regards, >> > Shalin Shekhar Mangar. >> > >> > >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> >
