Re: Using the collections API to create a new collection
Thanks. I think I found the final problem that I was facing in this ticket: https://issues.apache.org/jira/browse/SOLR-5306 On Sun, Mar 15, 2015 at 11:53 AM, Erick Erickson wrote: > Yes, "configs" is the same as configsets, I tend to use them > interchangeably. > > You're still missing the point. Once the files are up in Zookeeper, that's > where > they live. They do NOT then live on the nodes hosting the replicas. So, > assuming that when you write > bq: Our ZK configuration data is unde /dev-local-solr/configs > you mean that's the directory you specified with the upconfig command, it's > not totally irrelevant to your replicas. When you create your > collection, you give it the name of one of your config sets that you've > uploaded > to ZK. It doesn't matter in the least _how_ they go there. > > Now whenever one of the replicas for that collection starts up, it contact > ZK and reads the config files and starts up. The replica does _not_ > copy the files locally. > > HTH, > Erick > > On Sun, Mar 15, 2015 at 6:16 AM, Mike Thomsen > wrote: > > I tried that with upconfig, and it created it under /configs. Our ZK > > configuration data is under /dev-local-solr/configs. Not sure how to > > specify that. Also, is "configs" the same thing as "configsets" for the > > version of solr that I'm using? > > > > Thanks, > > > > Mike > > > > On Sat, Mar 14, 2015 at 6:38 PM, Anshum Gupta > > wrote: > > > >> Hi Mike, > >> > >> Here's what you want to do: > >> 1. Create or use an existing config set. > >> 2. Upload it to ZooKeeper ( > >> https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities > ) > >> 3. Use the config name when you create the collection. This would link > the > >> config set in zk with your collection. > >> > >> I think it would make a lot of sense for you to go through the getting > >> started with SolrCloud section in the Solr Reference Guide for 4.5. > >> > >> On Sat, Mar 14, 2015 at 12:02 PM, Mike Thomsen > >> wrote: > >> > >> > I looked in the tree view and I have only a node called "configs." > >> Nothing > >> > called "configsets." That's a serious problem, right? So if I'm > reading > >> > this correctly, I should be able to create a configset based on an > >> existing > >> > collection and load it into zookeeper once I find the right location > to > >> put > >> > it on our system. Sound right? > >> > > >> > https://cwiki.apache.org/confluence/display/solr/Config+Sets > >> > > >> > Thanks, > >> > > >> > Mike > >> > > >> > On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson < > erickerick...@gmail.com > >> > > >> > wrote: > >> > > >> > > I bet you did not push the configuration to Zookeeper before > creating > >> > > the collection. > >> > > If you look in your admin UI, the Cloud link and the "tree" version, > >> > > you'll find > >> > > a "configsets" directory that'll show you what you _have_ put in ZK, > >> and > >> > > I'll > >> > > bet you find nothing like a config set (containing schema.xml etc) > >> named > >> > > what > >> > > you specified for someExistingCollection. It's not the _collection_ > >> > > that should be > >> > > existing, it should be the configset. > >> > > > >> > > It's often a bit confusing because if the configName is not > specified, > >> > > the default > >> > > is to look for a config set of the same name as the collection being > >> > > created. > >> > > > >> > > Best, > >> > > Erick > >> > > > >> > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen < > mikerthom...@gmail.com > >> > > >> > > wrote: > >> > > > We're running SolrCloud 4.5.0. It's just a standard version of > >> > SolrCloud > >> > > > deployed in Tomcat, not something like the Cloudera distribution > (I > >> > note > >> > > > that because I can't seem to find solrctl and other things > referenced > >> > in > >> > > > the Cloudera tutorials). > >> > > > > >> > > > I'm trying to create a new Solr collection like this: > >> > > > > >> > > > > >> > > > >> > > >> > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > >> > > > > >> > > > Then I found this error message in the logs: > >> > > > > >> > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config > >> does > >> > > not > >> > > > exist in ZooKeeper:newCollection2 > >> > > > at > >> > > > > >> > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > >> > > > at > >> > > > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > >> > > > at > >> > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > >> > > > at > >> > > > > >> > > > >> > > >> > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > >> > > > at > >> > > > > >> > > > >> > > >> > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > >> > > > at > >> > > > > >> > > > >> > > >> > org.apache.solr.handler.RequestHandlerBas
Re: Using the collections API to create a new collection
Yes, "configs" is the same as configsets, I tend to use them interchangeably. You're still missing the point. Once the files are up in Zookeeper, that's where they live. They do NOT then live on the nodes hosting the replicas. So, assuming that when you write bq: Our ZK configuration data is unde /dev-local-solr/configs you mean that's the directory you specified with the upconfig command, it's not totally irrelevant to your replicas. When you create your collection, you give it the name of one of your config sets that you've uploaded to ZK. It doesn't matter in the least _how_ they go there. Now whenever one of the replicas for that collection starts up, it contact ZK and reads the config files and starts up. The replica does _not_ copy the files locally. HTH, Erick On Sun, Mar 15, 2015 at 6:16 AM, Mike Thomsen wrote: > I tried that with upconfig, and it created it under /configs. Our ZK > configuration data is under /dev-local-solr/configs. Not sure how to > specify that. Also, is "configs" the same thing as "configsets" for the > version of solr that I'm using? > > Thanks, > > Mike > > On Sat, Mar 14, 2015 at 6:38 PM, Anshum Gupta > wrote: > >> Hi Mike, >> >> Here's what you want to do: >> 1. Create or use an existing config set. >> 2. Upload it to ZooKeeper ( >> https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) >> 3. Use the config name when you create the collection. This would link the >> config set in zk with your collection. >> >> I think it would make a lot of sense for you to go through the getting >> started with SolrCloud section in the Solr Reference Guide for 4.5. >> >> On Sat, Mar 14, 2015 at 12:02 PM, Mike Thomsen >> wrote: >> >> > I looked in the tree view and I have only a node called "configs." >> Nothing >> > called "configsets." That's a serious problem, right? So if I'm reading >> > this correctly, I should be able to create a configset based on an >> existing >> > collection and load it into zookeeper once I find the right location to >> put >> > it on our system. Sound right? >> > >> > https://cwiki.apache.org/confluence/display/solr/Config+Sets >> > >> > Thanks, >> > >> > Mike >> > >> > On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson > > >> > wrote: >> > >> > > I bet you did not push the configuration to Zookeeper before creating >> > > the collection. >> > > If you look in your admin UI, the Cloud link and the "tree" version, >> > > you'll find >> > > a "configsets" directory that'll show you what you _have_ put in ZK, >> and >> > > I'll >> > > bet you find nothing like a config set (containing schema.xml etc) >> named >> > > what >> > > you specified for someExistingCollection. It's not the _collection_ >> > > that should be >> > > existing, it should be the configset. >> > > >> > > It's often a bit confusing because if the configName is not specified, >> > > the default >> > > is to look for a config set of the same name as the collection being >> > > created. >> > > >> > > Best, >> > > Erick >> > > >> > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > > >> > > wrote: >> > > > We're running SolrCloud 4.5.0. It's just a standard version of >> > SolrCloud >> > > > deployed in Tomcat, not something like the Cloudera distribution (I >> > note >> > > > that because I can't seem to find solrctl and other things referenced >> > in >> > > > the Cloudera tutorials). >> > > > >> > > > I'm trying to create a new Solr collection like this: >> > > > >> > > > >> > > >> > >> /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection >> > > > >> > > > Then I found this error message in the logs: >> > > > >> > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config >> does >> > > not >> > > > exist in ZooKeeper:newCollection2 >> > > > at >> > > > >> > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) >> > > > at >> > > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) >> > > > at >> > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) >> > > > at >> > > > >> > > >> > >> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) >> > > > at >> > > > >> > > >> > >> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) >> > > > at >> > > > >> > > >> > >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >> > > > at >> > > > >> > > >> > >> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) >> > > > at >> > > > >> > > >> > >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) >> > > > at >> > > > >> > > >> > >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) >> > > > at >> > > > >> > > >> > >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
Re: Using the collections API to create a new collection
I tried that with upconfig, and it created it under /configs. Our ZK configuration data is under /dev-local-solr/configs. Not sure how to specify that. Also, is "configs" the same thing as "configsets" for the version of solr that I'm using? Thanks, Mike On Sat, Mar 14, 2015 at 6:38 PM, Anshum Gupta wrote: > Hi Mike, > > Here's what you want to do: > 1. Create or use an existing config set. > 2. Upload it to ZooKeeper ( > https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) > 3. Use the config name when you create the collection. This would link the > config set in zk with your collection. > > I think it would make a lot of sense for you to go through the getting > started with SolrCloud section in the Solr Reference Guide for 4.5. > > On Sat, Mar 14, 2015 at 12:02 PM, Mike Thomsen > wrote: > > > I looked in the tree view and I have only a node called "configs." > Nothing > > called "configsets." That's a serious problem, right? So if I'm reading > > this correctly, I should be able to create a configset based on an > existing > > collection and load it into zookeeper once I find the right location to > put > > it on our system. Sound right? > > > > https://cwiki.apache.org/confluence/display/solr/Config+Sets > > > > Thanks, > > > > Mike > > > > On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson > > > wrote: > > > > > I bet you did not push the configuration to Zookeeper before creating > > > the collection. > > > If you look in your admin UI, the Cloud link and the "tree" version, > > > you'll find > > > a "configsets" directory that'll show you what you _have_ put in ZK, > and > > > I'll > > > bet you find nothing like a config set (containing schema.xml etc) > named > > > what > > > you specified for someExistingCollection. It's not the _collection_ > > > that should be > > > existing, it should be the configset. > > > > > > It's often a bit confusing because if the configName is not specified, > > > the default > > > is to look for a config set of the same name as the collection being > > > created. > > > > > > Best, > > > Erick > > > > > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > > > > wrote: > > > > We're running SolrCloud 4.5.0. It's just a standard version of > > SolrCloud > > > > deployed in Tomcat, not something like the Cloudera distribution (I > > note > > > > that because I can't seem to find solrctl and other things referenced > > in > > > > the Cloudera tutorials). > > > > > > > > I'm trying to create a new Solr collection like this: > > > > > > > > > > > > > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > > > > > > > Then I found this error message in the logs: > > > > > > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config > does > > > not > > > > exist in ZooKeeper:newCollection2 > > > > at > > > > > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > > > > at > > > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > > > > at > > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > > > > at > > > > > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > > > > at > > > > > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > > > at > > > > > > > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > > > at > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > > > > at > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > > > > at > > > > > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > > > > at > > > > > > > > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > > > at > > > > > > > > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > > > at > > > > > > > > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > > > > at > > > > > > > > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > > > > at > > > > > > > > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > > > > at > > > > > > > > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > > > > at > > > > > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > > > > at > > > > > > > > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > > > > at > > > > > > > > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:4
Re: Using the collections API to create a new collection
Hi Mike, Here's what you want to do: 1. Create or use an existing config set. 2. Upload it to ZooKeeper ( https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) 3. Use the config name when you create the collection. This would link the config set in zk with your collection. I think it would make a lot of sense for you to go through the getting started with SolrCloud section in the Solr Reference Guide for 4.5. On Sat, Mar 14, 2015 at 12:02 PM, Mike Thomsen wrote: > I looked in the tree view and I have only a node called "configs." Nothing > called "configsets." That's a serious problem, right? So if I'm reading > this correctly, I should be able to create a configset based on an existing > collection and load it into zookeeper once I find the right location to put > it on our system. Sound right? > > https://cwiki.apache.org/confluence/display/solr/Config+Sets > > Thanks, > > Mike > > On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson > wrote: > > > I bet you did not push the configuration to Zookeeper before creating > > the collection. > > If you look in your admin UI, the Cloud link and the "tree" version, > > you'll find > > a "configsets" directory that'll show you what you _have_ put in ZK, and > > I'll > > bet you find nothing like a config set (containing schema.xml etc) named > > what > > you specified for someExistingCollection. It's not the _collection_ > > that should be > > existing, it should be the configset. > > > > It's often a bit confusing because if the configName is not specified, > > the default > > is to look for a config set of the same name as the collection being > > created. > > > > Best, > > Erick > > > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > > wrote: > > > We're running SolrCloud 4.5.0. It's just a standard version of > SolrCloud > > > deployed in Tomcat, not something like the Cloudera distribution (I > note > > > that because I can't seem to find solrctl and other things referenced > in > > > the Cloudera tutorials). > > > > > > I'm trying to create a new Solr collection like this: > > > > > > > > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > > > > > Then I found this error message in the logs: > > > > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does > > not > > > exist in ZooKeeper:newCollection2 > > > at > > > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > > > at > > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > > > at > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > > > at > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > > > at > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > > at > > > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > > > at > > > > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > > at > > > > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > > at > > > > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > > > at > > > > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > > > at > > > > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > > > at > > > > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > > > at > > > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > > > at > > > > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > > > at > > > > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > > > at > > > > > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > > > at > > > > > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > > > at > > > > > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > > Source) > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > > > Source) > > > at > > > > > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(T
Re: Using the collections API to create a new collection
I added this to my solr.xml and restarted, but it didn't do anything even
though the path is valid and /opt/configsets contains a folder called base
with a conf folder and valid schema and solrconfig...
${configSetBaseDir:/opt/configsets}
Any ideas? Is there a way to force an update into zookeeper? Or should I
just purge the zookeeper data?
On Sat, Mar 14, 2015 at 3:02 PM, Mike Thomsen
wrote:
> I looked in the tree view and I have only a node called "configs." Nothing
> called "configsets." That's a serious problem, right? So if I'm reading
> this correctly, I should be able to create a configset based on an existing
> collection and load it into zookeeper once I find the right location to put
> it on our system. Sound right?
>
> https://cwiki.apache.org/confluence/display/solr/Config+Sets
>
> Thanks,
>
> Mike
>
> On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson
> wrote:
>
>> I bet you did not push the configuration to Zookeeper before creating
>> the collection.
>> If you look in your admin UI, the Cloud link and the "tree" version,
>> you'll find
>> a "configsets" directory that'll show you what you _have_ put in ZK, and
>> I'll
>> bet you find nothing like a config set (containing schema.xml etc) named
>> what
>> you specified for someExistingCollection. It's not the _collection_
>> that should be
>> existing, it should be the configset.
>>
>> It's often a bit confusing because if the configName is not specified,
>> the default
>> is to look for a config set of the same name as the collection being
>> created.
>>
>> Best,
>> Erick
>>
>> On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen
>> wrote:
>> > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud
>> > deployed in Tomcat, not something like the Cloudera distribution (I note
>> > that because I can't seem to find solrctl and other things referenced in
>> > the Cloudera tutorials).
>> >
>> > I'm trying to create a new Solr collection like this:
>> >
>> >
>> /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection
>> >
>> > Then I found this error message in the logs:
>> >
>> > org.apache.solr.common.cloud.ZooKeeperException: Specified config does
>> not
>> > exist in ZooKeeper:newCollection2
>> > at
>> > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742)
>> > at
>> > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244)
>> > at
>> org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>> > at
>> >
>> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465)
>> > at
>> >
>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146)
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>> > at
>> >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
>> > at
>> >
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
>> > at
>> >
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
>> > at
>> >
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>> > at
>> >
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
>> > at
>> >
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>> > at
>> >
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
>> > at
>> >
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
>> > at
>> >
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
>> > at
>> >
>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
>> Source)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> > Source)
>> > at
>> >
>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>> > at java.lang.Thread.run(Unknown Source)
>> >
>> > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log
>> > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCor
Re: Using the collections API to create a new collection
I looked in the tree view and I have only a node called "configs." Nothing called "configsets." That's a serious problem, right? So if I'm reading this correctly, I should be able to create a configset based on an existing collection and load it into zookeeper once I find the right location to put it on our system. Sound right? https://cwiki.apache.org/confluence/display/solr/Config+Sets Thanks, Mike On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson wrote: > I bet you did not push the configuration to Zookeeper before creating > the collection. > If you look in your admin UI, the Cloud link and the "tree" version, > you'll find > a "configsets" directory that'll show you what you _have_ put in ZK, and > I'll > bet you find nothing like a config set (containing schema.xml etc) named > what > you specified for someExistingCollection. It's not the _collection_ > that should be > existing, it should be the configset. > > It's often a bit confusing because if the configName is not specified, > the default > is to look for a config set of the same name as the collection being > created. > > Best, > Erick > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > wrote: > > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud > > deployed in Tomcat, not something like the Cloudera distribution (I note > > that because I can't seem to find solrctl and other things referenced in > > the Cloudera tutorials). > > > > I'm trying to create a new Solr collection like this: > > > > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > > > Then I found this error message in the logs: > > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does > not > > exist in ZooKeeper:newCollection2 > > at > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > > at > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > > at > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > > at > > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > > at > > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > > at > > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > > Source) > > at > > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > > at java.lang.Thread.run(Unknown Source) > > > > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log > > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > > 'newCollection2_shard1_replica1': Unable to create core: > > newCollection2_shard1_replica1 > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > >
Re: Using the collections API to create a new collection
I bet you did not push the configuration to Zookeeper before creating the collection. If you look in your admin UI, the Cloud link and the "tree" version, you'll find a "configsets" directory that'll show you what you _have_ put in ZK, and I'll bet you find nothing like a config set (containing schema.xml etc) named what you specified for someExistingCollection. It's not the _collection_ that should be existing, it should be the configset. It's often a bit confusing because if the configName is not specified, the default is to look for a config set of the same name as the collection being created. Best, Erick On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen wrote: > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud > deployed in Tomcat, not something like the Cloudera distribution (I note > that because I can't seem to find solrctl and other things referenced in > the Cloudera tutorials). > > I'm trying to create a new Solr collection like this: > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > Then I found this error message in the logs: > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does not > exist in ZooKeeper:newCollection2 > at > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > at > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Unknown Source) > > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > 'newCollection2_shard1_replica1': Unable to create core: > newCollection2_shard1_replica1 > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > at
Using the collections API to create a new collection
We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud deployed in Tomcat, not something like the Cloudera distribution (I note that because I can't seem to find solrctl and other things referenced in the Cloudera tutorials). I'm trying to create a new Solr collection like this: /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection Then I found this error message in the logs: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper:newCollection2 at org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) at org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Unknown Source) Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore 'newCollection2_shard1_replica1': Unable to create core: newCollection2_shard1_replica1 at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskT
Re: Using the Collections API
What version of Solr? I think there was a bug a couple versions back (perhaps introduced in 4.1 if I remember right) that made it so creates were not spread correctly. - Mark
Re: Using the Collections API
On 5/17/2013 4:03 AM, Jared Rodriguez wrote: > So it sounds like you want the collection created with a master and a > replica and you want one to be on each node? If so, I believe that you can > get that effect by specifying maxShardsPerNode=1 as part of your url line. > This will tell solr to create the master and replica that you desire but > to not put them on the same node. Jared, The default value of maxShardsPerNode is 1. Even if you make it higher, the collections API won't put replicas for the same shard on the same server, as long as the number of available hosts is a multiple of your replication factor. I would consider it a bug if the API were to ever put more than one replica for the same shard on the same host, regardless of the number of available hosts. I do not know whether the API has such a bug or not. A override option to allow replicas on the same host might make sense for proof-of-concept setups, but you'd never want to use it in production. Thanks, Shawn
Re: Using the Collections API
Hi Jared, yes that is what I want to achieve: Creating a master and a replica and I want them to be separate nodes. I just realized that I posted the wrong URL, I was already using the parameter maxShardsPerNode=1. But just to be sure, I also tried it with your URL and I get the same result. Both replicas will be created on app02. It does not matter where I create the collection (meaning app02 or app03) via the collections api. And as I said, both nodes appear live in zookeeper [zk: localhost:4181(CONNECTED) 1] ls /live_nodes [app02:9985_solr, app03:9985_solr] Any clue? Regards Alexander Am 2013-05-17 12:03, schrieb Jared Rodriguez: Hi Alexander, So it sounds like you want the collection created with a master and a replica and you want one to be on each node? If so, I believe that you can get that effect by specifying maxShardsPerNode=1 as part of your url line. This will tell solr to create the master and replica that you desire but to not put them on the same node. Your url would look like: http://app02:9985/solr/admin/collections?action=CREATE&name=storage&numShards=1&replicationFactor=2&collection.configName=storage-conf&maxShardsPerNode=1 The SolrCloud wiki does a good job of explaining the params and how they function. http://wiki.apache.org/solr/SolrCloud Jared On Fri, May 17, 2013 at 4:57 AM, A.Eibner wrote: Hi, sorry for the delay. I have two live nodes (also zookeeper knows these two [app02:9985_solrl,app03:9985_**solr]) But when I want to create a collection via: http://app02:9985/solr/admin/**collections?action=CREATE&** name=storage&numShards=1&**replicationFactor=2&** collection.configName=storage-**conf<http://app02:9985/solr/admin/collections?action=CREATE&name=storage&numShards=1&replicationFactor=2&collection.configName=storage-conf> Both replicas will be created on app02. Any clues ? Should I post anything else? Regards Alexander Am 2013-05-15 14:48, schrieb Mark Miller: Yeah, I use both on an empty Solr - what is the error? - Mark On May 15, 2013, at 6:53 AM, A.Eibner wrote: Hi, I just wanted to ask, if anyone is using the collections API to create collections, or if not how they use the coreAPI to create a collection with replication ? Because I run into errors when creating a collection on an empty solr. Kind regards Alexander
Re: Using the Collections API
Hi Alexander, So it sounds like you want the collection created with a master and a replica and you want one to be on each node? If so, I believe that you can get that effect by specifying maxShardsPerNode=1 as part of your url line. This will tell solr to create the master and replica that you desire but to not put them on the same node. Your url would look like: http://app02:9985/solr/admin/collections?action=CREATE&name=storage&numShards=1&replicationFactor=2&collection.configName=storage-conf&maxShardsPerNode=1 The SolrCloud wiki does a good job of explaining the params and how they function. http://wiki.apache.org/solr/SolrCloud Jared On Fri, May 17, 2013 at 4:57 AM, A.Eibner wrote: > Hi, sorry for the delay. > > I have two live nodes (also zookeeper knows these two > [app02:9985_solrl,app03:9985_**solr]) > > But when I want to create a collection via: > > http://app02:9985/solr/admin/**collections?action=CREATE&** > name=storage&numShards=1&**replicationFactor=2&** > collection.configName=storage-**conf<http://app02:9985/solr/admin/collections?action=CREATE&name=storage&numShards=1&replicationFactor=2&collection.configName=storage-conf> > > Both replicas will be created on app02. > > Any clues ? > Should I post anything else? > > Regards > Alexander > > Am 2013-05-15 14:48, schrieb Mark Miller: > > Yeah, I use both on an empty Solr - what is the error? >> >> - Mark >> >> On May 15, 2013, at 6:53 AM, A.Eibner wrote: >> >> Hi, >>> >>> I just wanted to ask, if anyone is using the collections API to create >>> collections, >>> or if not how they use the coreAPI to create a collection with >>> replication ? >>> >>> Because I run into errors when creating a collection on an empty solr. >>> >>> Kind regards >>> Alexander >>> >> >> >> > -- Jared Rodriguez
Re: Using the Collections API
Hi, sorry for the delay. I have two live nodes (also zookeeper knows these two [app02:9985_solrl,app03:9985_solr]) But when I want to create a collection via: http://app02:9985/solr/admin/collections?action=CREATE&name=storage&numShards=1&replicationFactor=2&collection.configName=storage-conf Both replicas will be created on app02. Any clues ? Should I post anything else? Regards Alexander Am 2013-05-15 14:48, schrieb Mark Miller: Yeah, I use both on an empty Solr - what is the error? - Mark On May 15, 2013, at 6:53 AM, A.Eibner wrote: Hi, I just wanted to ask, if anyone is using the collections API to create collections, or if not how they use the coreAPI to create a collection with replication ? Because I run into errors when creating a collection on an empty solr. Kind regards Alexander
Re: Using the Collections API
Hi Mark, Yes, I am using reload. Here is the jira that I filed. https://issues.apache.org/jira/browse/SOLR-4805 Please let me know if there is any additional data that you need. On Wed, May 15, 2013 at 12:53 PM, Mark Miller wrote: > > On May 15, 2013, at 12:26 PM, Jared Rodriguez > wrote: > > > the cores in the collection stay offline even if there are no > > material changes. > > I've used reload - if you are having trouble with it, please post more > details or file a JIRA issue. > > - Mark -- Jared Rodriguez
Re: Using the Collections API
On May 15, 2013, at 12:26 PM, Jared Rodriguez wrote: > the cores in the collection stay offline even if there are no > material changes. I've used reload - if you are having trouble with it, please post more details or file a JIRA issue. - Mark
Re: Using the Collections API
I have used both and they seem to work well for basic operations - create, delete, etc. Although newer operations like reload do not function as they should - the cores in the collection stay offline even if there are no material changes. On Wed, May 15, 2013 at 6:53 AM, A.Eibner wrote: > Hi, > > I just wanted to ask, if anyone is using the collections API to create > collections, > or if not how they use the coreAPI to create a collection with replication > ? > > Because I run into errors when creating a collection on an empty solr. > > Kind regards > Alexander > -- Jared Rodriguez
Re: Using the Collections API
On 5/15/2013 4:53 AM, A.Eibner wrote: > I just wanted to ask, if anyone is using the collections API to create > collections, > or if not how they use the coreAPI to create a collection with > replication ? For my little SolrCloud install using 4.2.1, I have used the collections API exclusively. It has worked perfectly. I updated SOLR-4734. Thanks, Shawn
Re: Using the Collections API
Yeah, I use both on an empty Solr - what is the error? - Mark On May 15, 2013, at 6:53 AM, A.Eibner wrote: > Hi, > > I just wanted to ask, if anyone is using the collections API to create > collections, > or if not how they use the coreAPI to create a collection with replication ? > > Because I run into errors when creating a collection on an empty solr. > > Kind regards > Alexander
Using the Collections API
Hi, I just wanted to ask, if anyone is using the collections API to create collections, or if not how they use the coreAPI to create a collection with replication ? Because I run into errors when creating a collection on an empty solr. Kind regards Alexander
