I've solved this problem myself. If you use core discovery, you must specify the "numShards" parameter in core.properties. or else solr won't be allocate range for each shards and then documents won't be distributed properly.
Using core discovery to set up solr cloud in tomcat is much easier and clean than coreAdmin described in the wiki: http://wiki.apache.org/solr/SolrCloudTomcat. It costs me some time to move from jetty to tomcat, but I think our IT team will like this way. :) On 6 October 2013 23:53, Liu Bo <diabl...@gmail.com> wrote: > Hi all > > I've sent out this mail before, but I only subscribed to lucene-user but > not solr-user at that time. Sorry for repeating if any and your help will > be much of my appreciation. > > I'm trying out the tutorial about solrcloud, and then I manage to write my > own plugin to import data from our set of databases, I use SolrWriter from > DataImporter package and the docs could be distributed commit to shards. > > Every thing works fine using jetty from the solr example, but when I move > to tomcat, solrcloud seems not been configured right. As the documents are > just committed to the shard where update requested goes to. > > The cause probably is the range is null for shards in clusterstate.json. > The router is "implicit" instead of "compositeId" as well. > > Is there anything missed or configured wrong in the following steps? How > could I fix it. Your help will be much of my appreciation. > > PS, solr cloud tomcat wiki page isn't up to 4.4 with core discovery, I'm > trying out after reading SoclrCloud, SolrCloudJboss, and CoreAdmin wiki > pages. > > Here's what I've done and some useful logs: > > 1. start three zookeeper server. > 2. upload configuration files to zookeeper, the collection name is > "content_collection" > 3. start three tomcat instants on three server with core discovery > > a) core file: > name=content > loadOnStartup=true > transient=false > shard=shard1 (differrent on servers) > collection=content_collection > b) solr.xml > > <solr> > > <solrcloud> > > <str name="host">${host:}</str> > > <str name="hostContext">${hostContext:solr}</str> > > <int name="hostPort">8080</int> > > <int name="zkClientTimeout">${zkClientTimeout:15000}</int> > > <str name="zkHost">10.199.46.176:2181,10.199.46.165:2181, > 10.199.46.158:2181</str> > > <bool name="genericCoreNodeNames">${genericCoreNodeNames:true}</bool> > > </solrcloud> > > > <shardHandlerFactory name="shardHandlerFactory" > > class="HttpShardHandlerFactory"> > > <int name="socketTimeout">${socketTimeout:0}</int> > > <int name="connTimeout">${connTimeout:0}</int> > > </shardHandlerFactory> > > </solr> > > 4. In the solr.log, I see the three shards are recognized, and the > solrcloud can see the content_collection has three shards as well. > 5. write documents to content_collection using my update request, the > documents only commits to the shard the request goes to, in the log I can > see the DistributedUpdateProcessorFactory is in the processorChain and > disribute commit is triggered: > > INFO - 2013-09-30 16:31:43.205; > com.microstrategy.alert.search.solr.plugin.index.handler.IndexRequestHandler; > updata request processor factories: > > INFO - 2013-09-30 16:31:43.206; > com.microstrategy.alert.search.solr.plugin.index.handler.IndexRequestHandler; > org.apache.solr.update.processor.LogUpdateProcessorFactory@4ae7b77 > > INFO - 2013-09-30 16:31:43.207; > com.microstrategy.alert.search.solr.plugin.index.handler.IndexRequestHandler; > org.apache.solr.update.processor.*DistributedUpdateProcessorFactory* > @5b2bc407 > > INFO - 2013-09-30 16:31:43.207; > com.microstrategy.alert.search.solr.plugin.index.handler.IndexRequestHandler; > org.apache.solr.update.processor.RunUpdateProcessorFactory@1652d654 > > INFO - 2013-09-30 16:31:43.283; org.apache.solr.core.SolrDeletionPolicy; > SolrDeletionPolicy.onInit: commits: num=1 > > > commit{dir=/home/bold/work/tomcat/solr/content/data/index,segFN=segments_1,generation=1} > > INFO - 2013-09-30 16:31:43.284; org.apache.solr.core.SolrDeletionPolicy; > newest commit generation = 1 > > INFO - 2013-09-30 16:31:43.440; *org.apache.solr.update.SolrCmdDistributor; > Distrib commit to*:[StdNode: http://10.199.46.176:8080/solr/content/, > StdNode: http://10.199.46.165:8080/solr/content/] > params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false > > but the documents won't go to other shards, the other shards only has a > request with not documents: > > INFO - 2013-09-30 16:31:43.841; > org.apache.solr.update.DirectUpdateHandler2; start > commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} > > INFO - 2013-09-30 16:31:43.855; org.apache.solr.core.SolrDeletionPolicy; > SolrDeletionPolicy.onInit: commits: num=1 > > > commit{dir=/home/bold/work/tomcat/solr/content/data/index,segFN=segments_1,generation=1} > > INFO - 2013-09-30 16:31:43.855; org.apache.solr.core.SolrDeletionPolicy; > newest commit generation = 1 > > INFO - 2013-09-30 16:31:43.856; > org.apache.solr.update.DirectUpdateHandler2; No uncommitted changes. > Skipping IW.commit. > > INFO - 2013-09-30 16:31:43.865; org.apache.solr.search.SolrIndexSearcher; > Opening Searcher@3c74c144 main > > INFO - 2013-09-30 16:31:43.869; org.apache.solr.core.QuerySenderListener; > QuerySenderListener sending requests to > Searcher@3c74c144main{StandardDirectoryReader(segments_1:1:nrt)} > > INFO - 2013-09-30 16:31:43.869; org.apache.solr.core.QuerySenderListener; > QuerySenderListener done. > > INFO - 2013-09-30 16:31:43.869; org.apache.solr.core.SolrCore; [content] > Registered new searcher > Searcher@3c74c144main{StandardDirectoryReader(segments_1:1:nrt)} > > INFO - 2013-09-30 16:31:43.870; > org.apache.solr.update.DirectUpdateHandler2; end_commit_flush > > INFO - 2013-09-30 16:31:43.870; > org.apache.solr.update.processor.LogUpdateProcessor; [content] webapp=/solr > path=/update > params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} > {commit=} 0 42 > > 6) later I found the range is null in clusterstate.json which might have > caused the document isn't committed distributively > > {"content_collection":{ > > "shards":{ > > "shard1":{ > > * "range":null,* > > "state":"active", > > "replicas":{"core_node1":{ > > "state":"active", > > "core":"content", > > "node_name":"10.199.46.176:8080_solr", > > "base_url":"http://10.199.46.176:8080/solr";, > > "leader":"true"}}}, > > "shard3":{ > > * "range":null,* > > "state":"active", > > "replicas":{"core_node2":{ > > "state":"active", > > "core":"content", > > "node_name":"10.199.46.202:8080_solr", > > "base_url":"http://10.199.46.202:8080/solr";, > > "leader":"true"}}}, > > "shard2":{ > > * "range":null,* > > "state":"active", > > "replicas":{"core_node3":{ > > "state":"active", > > "core":"content", > > "node_name":"10.199.46.165:8080_solr", > > "base_url":"http://10.199.46.165:8080/solr";, > > "leader":"true"}}}}, > > *"router":"implicit"*}} > > > > -- > All the best > > Liu Bo > > -- All the best Liu Bo
