I restarted one of the Solr server of the SolrCloud and then domain creation worked. Not sure why this happened, whether anybody has seen it, and when this problem will happen again.
Thanks, Ray On Wednesday, February 5, 2014 11:19 AM, Ray Cheng <rch...@rocketmail.com> wrote: Some more information that may help developers find out the cause. > > >INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired on >path: /overseer/collection-queue-work state: SyncConnected type >NodeChildrenChanged >... >(there were 393 of these "Watcher fired on path" lines in total) >... > > >Thanks, >Ray > > >INFO 2014-02-04 14:47:08,923 CollectionsHandler.java (line 295) Creating >Collection : numShards=1&name=abc.edu.xyz&replicationFactor=2&action=CREATE > INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired >on path: /overseer/collection-queue-work state: SyncConnected type >NodeChildrenChanged >INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired on >path: /overseer/collection-queue-work state: SyncConnected type >NodeChildrenChanged >... >(there were 393 of these "Watcher fired on path" lines in total) >... > INFO 2014-02-04 14:47:08,955 DistributedQueue.java (line 211) Watcher fired >on path: /overseer/collection-queue-work state: SyncConnected type >NodeChildrenChanged > INFO 2014-02-04 14:48:08,944 DistributedQueue.java (line 211) Watcher fired >on path: /overseer/collection-queue-work/qnr-0000000134 state: SyncConnected >type NodeDeleted >ERROR 2014-02-04 14:48:08,945 SolrException.java (line 108) >org.apache.solr.common.SolrException: createcollection the collection time >out:60s > at >org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:204) > at >org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:185) > at >org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:319) > > > >On Tuesday, February 4, 2014 7:26 PM, Ray Cheng <rch...@rocketmail.com> wrote: > >Hi, >> >>Our SolrCloud with Solr 4.6.0 has two replicas and uses three external >>ZooKeeper servers. We have created 20 collections successfully in the last >>two weeks. Then, creating new collections started failing since yesterday. >>New collections are created with curl like this one: >> >>curl >>'http://localhost:8983/solr/admin/collections?action=CREATE&name=abcd.xyz&numShards=1&replicationFactor=2&collection.configName=oursolrconfig' >> >>telnet to port 8983 from one to the other Solr host was okay. >>The largest ZooKeeper node is about 17K bytes. >>curl http://localhost:8983/solr/admin/ping printed OK on either SolrCloud >>host. >> >>The load on these two SolrCloud host is very low. So, these two systems are >>not busy at all. >>After Updated/inserted a document in one SolrCloud server, the document >>appeared on the other SolrCloud server correctly. >>Thus far only creating new collections failed. (I haven't tried to delete >>collections.) >> >>The create collection printed the following lines on the shell that I ran the >>above curl command. Solr on the other Linux didn't print anything during this >>time frame. If I run the same curl on the second SolrCloud host, it printed >>the same error message and the first SolrCloud host didn't print anything. >> >>Any idea on the cause of this problem or the fix? Thanks in advance. >> >>$ curl >>'http://localhost:8983/solr/admin/collections?action=CREATE&name=abcd.xyz&numShards=1&replicationFactor=2&collection.configName=oursolrconfig' >><?xml version="1.0" encoding="UTF-8"?> >><response> >><lst name="responseHeader"><int name="status">500</int><int >>name="QTime">60016</int></lst><lst name="error"><str >>name="msg">createcollection the collection time out:60s</str><str >>name="trace">org.apache.solr.common.SolrException: createcollection the >>collection time out:60s >>at >>org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:204) >>at >>org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:185) >>at >>org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:319) >>at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:134) >>at >>org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >>at >>org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662) >>at >>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) >>at >>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) >>at >>org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) >>at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) >>at >>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) >>at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) >>at >>org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) >>at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) >>at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) >>at >>org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) >>at >>org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) >>at >>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) >>at >>org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) >>at >>org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) >>at >>org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) >>at org.eclipse.jetty.server.Server.handle(Server.java:368) >>at >>org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) >>at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) >>at >>org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) >>at >>org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) >>at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) >>at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >>at >>org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) >>at >>org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) >>at >>org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) >>at >>org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) >>at java.lang.Thread.run(Thread.java:744) >></str><int name="code">500</int></lst> >></response> >> >> >>Ray >> >> > >
