Re: To understand SolrCloud configurations
Hi Erick, This question came to my mind after sometime seen your reply, Why Solr configurations are kept on ZooKeeper ? As far as I know, ZooKeeper is generic system can be used for any cross node configurations, not only with Solr, Solr configurations are Solr Specific, How ZooKeeper know/read Solr specific configurations. Does ZK has auto detect ZooKeeper plugins ? Can you help me to understand this ? thanks, Shanaka On 12 December 2014 at 14:23, Erick Erickson erickerick...@gmail.com wrote: bq: 1. Like to know what I have done is correct ? Looks fine to me. bq: 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) I think you're a bit confused here. The configuration stuff is NOT under collection1/conf as far as SolrCloud is concerned, it's in Zookeeper in /configs/default, take a look at the admincloud page, click the /configs entry and I think you'll see a defaults node. As far as SolrCloud is concerned, that's where your configs live. The fact that they exist in solr-home/collection1/conf on your local machine is totally irrelevant. Tomorrow, you could issue an upconfig and use something like .-confdir mytotallynewdirectory/conf -confname default and SolrCloud would happily overwrite your configs in the Zookeeper default node with the new ones, _and_ distribute them to all the Solr nodes when they were restarted. So where your configs should live is in some kind of version control.. HTH, Erick On Thu, Dec 11, 2014 at 6:19 PM, E S J esj.f...@gmail.com wrote: Hello Team, I would like to get clarified where to place schema.xml on SolrCloud set-up. My Solr cloud set-up , 3 nodes, 3 shards and 3 replications, 3 ZooKeeper What I have done is, 1. Taken a solr.war from solr default download ( solr-4.10.2/example/webapps/solr.war - 4.10.2) and placed tomcat-home/webapps/ folder. 2. Taken Solr home from solr default download ( solr-4.10.2/example/solr/) and placed on solr.home (Copied Collection folder as well along with solr.xml) 3. Started 3 solr nodes and zookeepr instances ( after correct configuration) 4. Register solr configurations of ZooKeeper using, zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default 5. Create 3 Shard's and 3 Replicas : http://solr1.internal:7003/solr/admin/collections?action=CREATEname=c-insreplicationFactor=3numShards=3collection.configName=defaultmaxShardsPerNode=3wt=jsonindent=2 After that I can see following folder structure in Solr node1's solr-home directory ( Can see similar structure on my other 2 solr nodes) -rw-r--r-- solr.xml drwxrwxr-x c-ins_shard1_replica1 drwxrwxr-x c-ins_shard2_replica1 drwxrwxr-x c-ins_shard3_replica1 drwxr-xr-x collection1 I've done some xml docuemnt indexing and it's working fine, Zoo-keepers are also working fine, My Questions are, 1. Like to know what I have done is correct ? 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) Appreciate your time on this. Thanks - Elike
Re: To understand SolrCloud configurations
Zookeeper knows nothing at all about Solr, it's fully generic. The code for SolrCloud on each of the Solr instances _does_ know about Zookeeper, and where to expect certain information, specifically where the configurations are stored on Zookeeper. So on startup, the Solr instance queries Zookeeper for, in this case, the config set which is just a path on Zookeeper. For each collection on the Solr node it pulls the appropriate configs down, loads and goes. So the path is something like /configs/whatevertheconfignameis. HTH, Erick On Thu, Dec 18, 2014 at 3:53 PM, E S J esj.f...@gmail.com wrote: Hi Erick, This question came to my mind after sometime seen your reply, Why Solr configurations are kept on ZooKeeper ? As far as I know, ZooKeeper is generic system can be used for any cross node configurations, not only with Solr, Solr configurations are Solr Specific, How ZooKeeper know/read Solr specific configurations. Does ZK has auto detect ZooKeeper plugins ? Can you help me to understand this ? thanks, Shanaka On 12 December 2014 at 14:23, Erick Erickson erickerick...@gmail.com wrote: bq: 1. Like to know what I have done is correct ? Looks fine to me. bq: 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) I think you're a bit confused here. The configuration stuff is NOT under collection1/conf as far as SolrCloud is concerned, it's in Zookeeper in /configs/default, take a look at the admincloud page, click the /configs entry and I think you'll see a defaults node. As far as SolrCloud is concerned, that's where your configs live. The fact that they exist in solr-home/collection1/conf on your local machine is totally irrelevant. Tomorrow, you could issue an upconfig and use something like .-confdir mytotallynewdirectory/conf -confname default and SolrCloud would happily overwrite your configs in the Zookeeper default node with the new ones, _and_ distribute them to all the Solr nodes when they were restarted. So where your configs should live is in some kind of version control.. HTH, Erick On Thu, Dec 11, 2014 at 6:19 PM, E S J esj.f...@gmail.com wrote: Hello Team, I would like to get clarified where to place schema.xml on SolrCloud set-up. My Solr cloud set-up , 3 nodes, 3 shards and 3 replications, 3 ZooKeeper What I have done is, 1. Taken a solr.war from solr default download ( solr-4.10.2/example/webapps/solr.war - 4.10.2) and placed tomcat-home/webapps/ folder. 2. Taken Solr home from solr default download ( solr-4.10.2/example/solr/) and placed on solr.home (Copied Collection folder as well along with solr.xml) 3. Started 3 solr nodes and zookeepr instances ( after correct configuration) 4. Register solr configurations of ZooKeeper using, zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default 5. Create 3 Shard's and 3 Replicas : http://solr1.internal:7003/solr/admin/collections?action=CREATEname=c-insreplicationFactor=3numShards=3collection.configName=defaultmaxShardsPerNode=3wt=jsonindent=2 After that I can see following folder structure in Solr node1's solr-home directory ( Can see similar structure on my other 2 solr nodes) -rw-r--r-- solr.xml drwxrwxr-x c-ins_shard1_replica1 drwxrwxr-x c-ins_shard2_replica1 drwxrwxr-x c-ins_shard3_replica1 drwxr-xr-x collection1 I've done some xml docuemnt indexing and it's working fine, Zoo-keepers are also working fine, My Questions are, 1. Like to know what I have done is correct ? 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) Appreciate your time on this. Thanks - Elike
Re: To understand SolrCloud configurations
Thanks Lot and It's clear to me Eric. On 19 December 2014 at 11:08, Erick Erickson erickerick...@gmail.com wrote: Zookeeper knows nothing at all about Solr, it's fully generic. The code for SolrCloud on each of the Solr instances _does_ know about Zookeeper, and where to expect certain information, specifically where the configurations are stored on Zookeeper. So on startup, the Solr instance queries Zookeeper for, in this case, the config set which is just a path on Zookeeper. For each collection on the Solr node it pulls the appropriate configs down, loads and goes. So the path is something like /configs/whatevertheconfignameis. HTH, Erick On Thu, Dec 18, 2014 at 3:53 PM, E S J esj.f...@gmail.com wrote: Hi Erick, This question came to my mind after sometime seen your reply, Why Solr configurations are kept on ZooKeeper ? As far as I know, ZooKeeper is generic system can be used for any cross node configurations, not only with Solr, Solr configurations are Solr Specific, How ZooKeeper know/read Solr specific configurations. Does ZK has auto detect ZooKeeper plugins ? Can you help me to understand this ? thanks, Shanaka On 12 December 2014 at 14:23, Erick Erickson erickerick...@gmail.com wrote: bq: 1. Like to know what I have done is correct ? Looks fine to me. bq: 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) I think you're a bit confused here. The configuration stuff is NOT under collection1/conf as far as SolrCloud is concerned, it's in Zookeeper in /configs/default, take a look at the admincloud page, click the /configs entry and I think you'll see a defaults node. As far as SolrCloud is concerned, that's where your configs live. The fact that they exist in solr-home/collection1/conf on your local machine is totally irrelevant. Tomorrow, you could issue an upconfig and use something like .-confdir mytotallynewdirectory/conf -confname default and SolrCloud would happily overwrite your configs in the Zookeeper default node with the new ones, _and_ distribute them to all the Solr nodes when they were restarted. So where your configs should live is in some kind of version control.. HTH, Erick On Thu, Dec 11, 2014 at 6:19 PM, E S J esj.f...@gmail.com wrote: Hello Team, I would like to get clarified where to place schema.xml on SolrCloud set-up. My Solr cloud set-up , 3 nodes, 3 shards and 3 replications, 3 ZooKeeper What I have done is, 1. Taken a solr.war from solr default download ( solr-4.10.2/example/webapps/solr.war - 4.10.2) and placed tomcat-home/webapps/ folder. 2. Taken Solr home from solr default download ( solr-4.10.2/example/solr/) and placed on solr.home (Copied Collection folder as well along with solr.xml) 3. Started 3 solr nodes and zookeepr instances ( after correct configuration) 4. Register solr configurations of ZooKeeper using, zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default 5. Create 3 Shard's and 3 Replicas : http://solr1.internal:7003/solr/admin/collections?action=CREATEname=c-insreplicationFactor=3numShards=3collection.configName=defaultmaxShardsPerNode=3wt=jsonindent=2 After that I can see following folder structure in Solr node1's solr-home directory ( Can see similar structure on my other 2 solr nodes) -rw-r--r-- solr.xml drwxrwxr-x c-ins_shard1_replica1 drwxrwxr-x c-ins_shard2_replica1 drwxrwxr-x c-ins_shard3_replica1 drwxr-xr-x collection1 I've done some xml docuemnt indexing and it's working fine, Zoo-keepers are also working fine, My Questions are, 1. Like to know what I have done is correct ? 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) Appreciate your time on this. Thanks - Elike
Re: To understand SolrCloud configurations
HI Shawn, Thanks, You have answered my question to a certain extend, But I wanted to Isolate Solr Cloud from application and do some load testing by setting up Jmeter Script. I could hit Solr instances, but it will not simulate how Application (Client) will deal with Solr Cloud. Any suggestions for a better way of achieving this? I want to do this, Solr keeps failing when we index large data and can't find anything in the logs as well, I wanted to identify where it's failing, I thought of using the above approach to make sure Solr and zookeeper setup is correct , Also, I would like to know a better way to debug Zookeeper and Solr, What I have done so far is, 1. Make sure Zookeeper picking a Solr leader when the existing sun goes down. 2. Setup is working when one (lead) ZooKeeper is down etc ... 3. Access Server runtime and see the data Thanks, Shanaka On 15 December 2014 at 19:22, Shawn Heisey apa...@elyograg.org wrote: On 12/14/2014 12:41 PM, E S J wrote: This question is related to the same configurations I've posted. How should I manually test indexing via Zookeeper, I mean not directly accessing solr nodes like, curl http://solr1.internal:7083/solr/c-ins/update?commit=true -H Content-Type: text/xml -d @mem.xml I have a solr client which uses CloudSolrServer to send request to SolrCloud, But my intention is to isolate my SolrCloud and send index search requests to make sure Solr Cloud setup is working fine. Later I can do Solrclient integration testing. How should I send index requests manually ( like curl) to index data to solrcloud such a way CloudSolrServer use ZooKeeper to LB/Pick Solr instance ? If you have either single-shard collections or multi-shard collections with automatic routing, SolrCloud is designed so that you can send any kind of request to any machine in the entire cloud, and it will be sent where it needs to go. If the collection uses manual (implicit) routing, then queries can go anywhere, but updates must be directed to the correct shard. If you are not using CloudSolrServer, then you must either set up a load balancer in front of SolrCloud, or your application will need to know where your Solr servers are. Curl cannot talk to zookeeper, because zookeeper does not speak HTTP. CloudSolrServer allows your application to specify only the zookeeper hosts, it doesn't need to know where the Solr servers are. This is because it includes a full zookeeper client. There is an API in Solr at /solr/zookeeper that can, with appropriate parameters, return various pieces of information from zookeeper in JSON format. This is the place where the admin UI gathers the information necessary to create the various options on the Cloud tab. Once your application has that information, it can use it to find out the Solr URLs to use. If this doesn't answer your question, please clarify it. Thanks, Shawn
RE: To understand SolrCloud configurations
+1 Date: Mon, 15 Dec 2014 21:44:44 +1300 Subject: Re: To understand SolrCloud configurations From: esj.f...@gmail.com To: solr-user@lucene.apache.org HI Shawn, Thanks, You have answered my question to a certain extend, But I wanted to Isolate Solr Cloud from application and do some load testing by setting up Jmeter Script. I could hit Solr instances, but it will not simulate how Application (Client) will deal with Solr Cloud. Any suggestions for a better way of achieving this? I want to do this, Solr keeps failing when we index large data and can't find anything in the logs as well, I wanted to identify where it's failing, I thought of using the above approach to make sure Solr and zookeeper setup is correct , Also, I would like to know a better way to debug Zookeeper and Solr, What I have done so far is, 1. Make sure Zookeeper picking a Solr leader when the existing sun goes down. 2. Setup is working when one (lead) ZooKeeper is down etc ... 3. Access Server runtime and see the data Thanks, Shanaka On 15 December 2014 at 19:22, Shawn Heisey apa...@elyograg.org wrote: On 12/14/2014 12:41 PM, E S J wrote: This question is related to the same configurations I've posted. How should I manually test indexing via Zookeeper, I mean not directly accessing solr nodes like, curl http://solr1.internal:7083/solr/c-ins/update?commit=true -H Content-Type: text/xml -d @mem.xml I have a solr client which uses CloudSolrServer to send request to SolrCloud, But my intention is to isolate my SolrCloud and send index search requests to make sure Solr Cloud setup is working fine. Later I can do Solrclient integration testing. How should I send index requests manually ( like curl) to index data to solrcloud such a way CloudSolrServer use ZooKeeper to LB/Pick Solr instance ? If you have either single-shard collections or multi-shard collections with automatic routing, SolrCloud is designed so that you can send any kind of request to any machine in the entire cloud, and it will be sent where it needs to go. If the collection uses manual (implicit) routing, then queries can go anywhere, but updates must be directed to the correct shard. If you are not using CloudSolrServer, then you must either set up a load balancer in front of SolrCloud, or your application will need to know where your Solr servers are. Curl cannot talk to zookeeper, because zookeeper does not speak HTTP. CloudSolrServer allows your application to specify only the zookeeper hosts, it doesn't need to know where the Solr servers are. This is because it includes a full zookeeper client. There is an API in Solr at /solr/zookeeper that can, with appropriate parameters, return various pieces of information from zookeeper in JSON format. This is the place where the admin UI gathers the information necessary to create the various options on the Cloud tab. Once your application has that information, it can use it to find out the Solr URLs to use. If this doesn't answer your question, please clarify it. Thanks, Shawn
Re: To understand SolrCloud configurations
Hi All, This question is related to the same configurations I've posted. How should I manually test indexing via Zookeeper, I mean not directly accessing solr nodes like, curl http://solr1.internal:7083/solr/c-ins/update?commit=true -H Content-Type: text/xml -d @mem.xml I have a solr client which uses CloudSolrServer to send request to SolrCloud, But my intention is to isolate my SolrCloud and send index search requests to make sure Solr Cloud setup is working fine. Later I can do Solrclient integration testing. How should I send index requests manually ( like curl) to index data to solrcloud such a way CloudSolrServer use ZooKeeper to LB/Pick Solr instance ? Thanks, Shanaka On 12 December 2014 at 16:09, E S J esj.f...@gmail.com wrote: Thanks, I thought only option is default or schemaless because , When we run bin/solr -e cloud you will get prompt like , To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] 3 Ok, let's start up 3 Solr nodes for your example SolrCloud cluster. Please enter the port for node1 [8983] 8983 Please enter the port for node2 [7574] 7574 Please enter the port for node3 [8984] 8984 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node1 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node2 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node3 Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -d node1 -p 8983 Waiting to see Solr listening on port 8983 [\] Started Solr server on port 8983 (pid=29712). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -d node2 -p 7574 -z localhost:9983 Waiting to see Solr listening on port 7574 [\] Started Solr server on port 7574 (pid=29935). Happy searching! Starting node3 on port 8984 using command: solr start -cloud -d node3 -p 8984 -z localhost:9983 Waiting to see Solr listening on port 8984 [\] Started Solr server on port 8984 (pid=30559). Happy searching! Now let's create a new collection for indexing documents in your 3-node cluster. Please provide a name for your new collection: [gettingstarted] gettingstarted How many shards would you like to split gettingstarted into? [2] 3 3 How many replicas per shard would you like to create? [2] 3 3 *Please choose a configuration for the gettingstarted collection, available options are: default or schemaless [default]* On 12 December 2014 at 16:03, Shawn Heisey apa...@elyograg.org wrote: On 12/11/2014 6:31 PM, E S J wrote: Thanks Eric, I understand your explanation. Quick question, Are configurations sits under /configs/defaults because -configname specified as default when I execute the following command? Can I specify -configname as /c-ins/ zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default Also I noticed that available options for -configname is default or schemaless, that is why I specified as default. The confname can be anything you want it to be. You should not include any slash characters in it, though ... make it c-ins, not /c-ins/. Where do you see information telling you it can be default or schemaless? That sounds completely wrong to me, so I'd like to know what needs to be fixed. Here's part of what zkcli itself says if you run it with no options: -n,--confname arg for upconfig, linkconfig: name of the config set Thanks, Shawn
Re: To understand SolrCloud configurations
On 12/14/2014 12:41 PM, E S J wrote: This question is related to the same configurations I've posted. How should I manually test indexing via Zookeeper, I mean not directly accessing solr nodes like, curl http://solr1.internal:7083/solr/c-ins/update?commit=true -H Content-Type: text/xml -d @mem.xml I have a solr client which uses CloudSolrServer to send request to SolrCloud, But my intention is to isolate my SolrCloud and send index search requests to make sure Solr Cloud setup is working fine. Later I can do Solrclient integration testing. How should I send index requests manually ( like curl) to index data to solrcloud such a way CloudSolrServer use ZooKeeper to LB/Pick Solr instance ? If you have either single-shard collections or multi-shard collections with automatic routing, SolrCloud is designed so that you can send any kind of request to any machine in the entire cloud, and it will be sent where it needs to go. If the collection uses manual (implicit) routing, then queries can go anywhere, but updates must be directed to the correct shard. If you are not using CloudSolrServer, then you must either set up a load balancer in front of SolrCloud, or your application will need to know where your Solr servers are. Curl cannot talk to zookeeper, because zookeeper does not speak HTTP. CloudSolrServer allows your application to specify only the zookeeper hosts, it doesn't need to know where the Solr servers are. This is because it includes a full zookeeper client. There is an API in Solr at /solr/zookeeper that can, with appropriate parameters, return various pieces of information from zookeeper in JSON format. This is the place where the admin UI gathers the information necessary to create the various options on the Cloud tab. Once your application has that information, it can use it to find out the Solr URLs to use. If this doesn't answer your question, please clarify it. Thanks, Shawn
Re: To understand SolrCloud configurations
bq: 1. Like to know what I have done is correct ? Looks fine to me. bq: 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) I think you're a bit confused here. The configuration stuff is NOT under collection1/conf as far as SolrCloud is concerned, it's in Zookeeper in /configs/default, take a look at the admincloud page, click the /configs entry and I think you'll see a defaults node. As far as SolrCloud is concerned, that's where your configs live. The fact that they exist in solr-home/collection1/conf on your local machine is totally irrelevant. Tomorrow, you could issue an upconfig and use something like .-confdir mytotallynewdirectory/conf -confname default and SolrCloud would happily overwrite your configs in the Zookeeper default node with the new ones, _and_ distribute them to all the Solr nodes when they were restarted. So where your configs should live is in some kind of version control.. HTH, Erick On Thu, Dec 11, 2014 at 6:19 PM, E S J esj.f...@gmail.com wrote: Hello Team, I would like to get clarified where to place schema.xml on SolrCloud set-up. My Solr cloud set-up , 3 nodes, 3 shards and 3 replications, 3 ZooKeeper What I have done is, 1. Taken a solr.war from solr default download ( solr-4.10.2/example/webapps/solr.war - 4.10.2) and placed tomcat-home/webapps/ folder. 2. Taken Solr home from solr default download ( solr-4.10.2/example/solr/) and placed on solr.home (Copied Collection folder as well along with solr.xml) 3. Started 3 solr nodes and zookeepr instances ( after correct configuration) 4. Register solr configurations of ZooKeeper using, zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default 5. Create 3 Shard's and 3 Replicas : http://solr1.internal:7003/solr/admin/collections?action=CREATEname=c-insreplicationFactor=3numShards=3collection.configName=defaultmaxShardsPerNode=3wt=jsonindent=2 After that I can see following folder structure in Solr node1's solr-home directory ( Can see similar structure on my other 2 solr nodes) -rw-r--r-- solr.xml drwxrwxr-x c-ins_shard1_replica1 drwxrwxr-x c-ins_shard2_replica1 drwxrwxr-x c-ins_shard3_replica1 drwxr-xr-x collection1 I've done some xml docuemnt indexing and it's working fine, Zoo-keepers are also working fine, My Questions are, 1. Like to know what I have done is correct ? 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) Appreciate your time on this. Thanks - Elike
Re: To understand SolrCloud configurations
Thanks Eric, I understand your explanation. Quick question, Are configurations sits under /configs/defaults because -configname specified as default when I execute the following command? Can I specify -configname as /c-ins/ zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default Also I noticed that available options for -configname is default or schemaless, that is why I specified as default. Thanks, Elike On 12 December 2014 at 14:23, Erick Erickson erickerick...@gmail.com wrote: bq: 1. Like to know what I have done is correct ? Looks fine to me. bq: 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) I think you're a bit confused here. The configuration stuff is NOT under collection1/conf as far as SolrCloud is concerned, it's in Zookeeper in /configs/default, take a look at the admincloud page, click the /configs entry and I think you'll see a defaults node. As far as SolrCloud is concerned, that's where your configs live. The fact that they exist in solr-home/collection1/conf on your local machine is totally irrelevant. Tomorrow, you could issue an upconfig and use something like .-confdir mytotallynewdirectory/conf -confname default and SolrCloud would happily overwrite your configs in the Zookeeper default node with the new ones, _and_ distribute them to all the Solr nodes when they were restarted. So where your configs should live is in some kind of version control.. HTH, Erick On Thu, Dec 11, 2014 at 6:19 PM, E S J esj.f...@gmail.com wrote: Hello Team, I would like to get clarified where to place schema.xml on SolrCloud set-up. My Solr cloud set-up , 3 nodes, 3 shards and 3 replications, 3 ZooKeeper What I have done is, 1. Taken a solr.war from solr default download ( solr-4.10.2/example/webapps/solr.war - 4.10.2) and placed tomcat-home/webapps/ folder. 2. Taken Solr home from solr default download ( solr-4.10.2/example/solr/) and placed on solr.home (Copied Collection folder as well along with solr.xml) 3. Started 3 solr nodes and zookeepr instances ( after correct configuration) 4. Register solr configurations of ZooKeeper using, zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default 5. Create 3 Shard's and 3 Replicas : http://solr1.internal:7003/solr/admin/collections?action=CREATEname=c-insreplicationFactor=3numShards=3collection.configName=defaultmaxShardsPerNode=3wt=jsonindent=2 After that I can see following folder structure in Solr node1's solr-home directory ( Can see similar structure on my other 2 solr nodes) -rw-r--r-- solr.xml drwxrwxr-x c-ins_shard1_replica1 drwxrwxr-x c-ins_shard2_replica1 drwxrwxr-x c-ins_shard3_replica1 drwxr-xr-x collection1 I've done some xml docuemnt indexing and it's working fine, Zoo-keepers are also working fine, My Questions are, 1. Like to know what I have done is correct ? 2. Where to place the schema.xml's and other configurations. Because for the moment it's are under collection1/conf folder and collection1 is not an active collection for me. ( i'm using only c-ins core) Appreciate your time on this. Thanks - Elike
Re: To understand SolrCloud configurations
On 12/11/2014 6:31 PM, E S J wrote: Thanks Eric, I understand your explanation. Quick question, Are configurations sits under /configs/defaults because -configname specified as default when I execute the following command? Can I specify -configname as /c-ins/ zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default Also I noticed that available options for -configname is default or schemaless, that is why I specified as default. The confname can be anything you want it to be. You should not include any slash characters in it, though ... make it c-ins, not /c-ins/. Where do you see information telling you it can be default or schemaless? That sounds completely wrong to me, so I'd like to know what needs to be fixed. Here's part of what zkcli itself says if you run it with no options: -n,--confname arg for upconfig, linkconfig: name of the config set Thanks, Shawn
Re: To understand SolrCloud configurations
Thanks, I thought only option is default or schemaless because , When we run bin/solr -e cloud you will get prompt like , To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] 3 Ok, let's start up 3 Solr nodes for your example SolrCloud cluster. Please enter the port for node1 [8983] 8983 Please enter the port for node2 [7574] 7574 Please enter the port for node3 [8984] 8984 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node1 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node2 Cloning /home/j2ee/solr-4.10.2/example into /home/j2ee/solr-4.10.2/node3 Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -d node1 -p 8983 Waiting to see Solr listening on port 8983 [\] Started Solr server on port 8983 (pid=29712). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -d node2 -p 7574 -z localhost:9983 Waiting to see Solr listening on port 7574 [\] Started Solr server on port 7574 (pid=29935). Happy searching! Starting node3 on port 8984 using command: solr start -cloud -d node3 -p 8984 -z localhost:9983 Waiting to see Solr listening on port 8984 [\] Started Solr server on port 8984 (pid=30559). Happy searching! Now let's create a new collection for indexing documents in your 3-node cluster. Please provide a name for your new collection: [gettingstarted] gettingstarted How many shards would you like to split gettingstarted into? [2] 3 3 How many replicas per shard would you like to create? [2] 3 3 *Please choose a configuration for the gettingstarted collection, available options are: default or schemaless [default]* On 12 December 2014 at 16:03, Shawn Heisey apa...@elyograg.org wrote: On 12/11/2014 6:31 PM, E S J wrote: Thanks Eric, I understand your explanation. Quick question, Are configurations sits under /configs/defaults because -configname specified as default when I execute the following command? Can I specify -configname as /c-ins/ zkcli.sh -zkhost zoo1.internal:2183,zoo2.internal:2183,zoo3.internal:2183 -cmd upconfig -confdir solr-home/collection1/conf -confname default Also I noticed that available options for -configname is default or schemaless, that is why I specified as default. The confname can be anything you want it to be. You should not include any slash characters in it, though ... make it c-ins, not /c-ins/. Where do you see information telling you it can be default or schemaless? That sounds completely wrong to me, so I'd like to know what needs to be fixed. Here's part of what zkcli itself says if you run it with no options: -n,--confname arg for upconfig, linkconfig: name of the config set Thanks, Shawn
Re: To understand SolrCloud configurations
On 12/11/2014 8:09 PM, E S J wrote: Thanks, I thought only option is default or schemaless because , When we run bin/solr -e cloud you will get prompt like , snip *Please choose a configuration for the gettingstarted collection, available options are: default or schemaless [default]* I have almost zero experience with the bin/solr script. It's very very new and has undergone quite a lot of change in the upcoming 5.0 version ... so I'm waiting for the dust to settle before I try to understand it and make suggestions about how to improve it. It doesn't even exist in the Solr versions that I use. The cloud example for the bin/solr script puts everything on one phsyical node. When you do this for real, you are going to want each of those nodes to be physically separate machines ... hopefully the bin/solr script will be able to easily accommodate a production SolrCloud installation. Thanks, Shawn
