Re: [EXTERNAL] - Re: starting SolrCloud nodes

[Shawn Heisey]

On 12/4/2017 7:33 AM, Steve Pruitt wrote:

I edited /etc/default/solr.in.sh to list my ZK hosts and I uncommented 
ZK_CLIENT_TIMEOUT leaving the default value of 15000.


The default is 15 seconds, most of the example configs that Solr 
includes have it increased to 30 seconds.  IMHO, 15 seconds is a 
relative eternity.



I am not sure if I need to set the SOLR_HOST.  This is not a production 
install, but I am running with three ZK machines and three Solr machines in the 
cluster.
The comment states to use it to "exposed to cluster state".  I'm not sure what 
"cluster state" means exactly and not sure if the default localhost is ok.


SOLR_HOST is used to decide what hostname each Solr instance will use 
when it registers itself into ZK, so that the cloud client (or the cloud 
itself) can find all the hosts for load balancing or distributed requests.


If you don't define this, Solr will use Java APIs to figure out the 
primary IP address of the machine, and it will use that when registering 
itself into the SolrCloud clusterstate in ZK.  Sometimes the Java APIs 
will choose the wrong address, especially on multi-homed systems, which 
is why Solr provides a way to manually declare the hostname.


Thanks,
Shawn

RE: [EXTERNAL] - Re: starting SolrCloud nodes

[Steve Pruitt]

Thanks.

I edited /etc/default/solr.in.sh to list my ZK hosts and I uncommented 
ZK_CLIENT_TIMEOUT leaving the default value of 15000.
I am not sure if I need to set the SOLR_HOST.  This is not a production 
install, but I am running with three ZK machines and three Solr machines in the 
cluster.
The comment states to use it to "exposed to cluster state".  I'm not sure what 
"cluster state" means exactly and not sure if the default localhost is ok.

-S

-Original Message-
From: Shawn Heisey [mailto:apa...@elyograg.org] 
Sent: Friday, December 01, 2017 6:34 PM
To: solr-user@lucene.apache.org
Subject: [EXTERNAL] - Re: starting SolrCloud nodes

On 12/1/2017 10:13 AM, Steve Pruitt wrote:
> Thanks to previous help.  I have a ZK ensemble of three nodes running.  I 
> have uploaded the config for my collection and the solr.xml file.
> I have Solr installed on three machines.
>
> I think my next steps are:
>
> Start up each Solr instance:  bin/solr start -c -z 
> "zoo1:2181,zoo2:2181,zoo3:2181"  // I have ZK_Hosts set to my ZK's, but the 
> documentation seems to say I need to provide the list here to prevent 
> embedded ZK from getting used.

The embedded ZK will only get started if you use the -c option and there is no 
ZK_HOST variable and no -z option on the commandline.

If you use both -z and ZK_HOST, then the info I've seen says the -z option will 
take priority.  I haven't looked at the script closely enough to confirm, but 
that would be the most logical way to operate, so it's probably correct.

If ZK_HOST is defined or you use the -z option, you do not need to include the 
-c option when starting Solr.  SolrCloud mode is assumed when ZK info is 
available.  The only time the -c option is *required* is when you want to start 
the embedded zookeeper.  Having the option won't hurt anything, though.

To start a solr *service* in cloud mode, all you need is to add ZK_HOST to 
/etc/default/.in.sh, where  is the service name, which defaults to solr.

> From one of the Solr instances create a collection:
> bin/solr create -c nameofuploadedconfig -s 3 -rf 2 //for example.

Nitpick:  The -c option on the create command is the name of the collection.  
To specify the name of the uploaded config, if it happens to be different from 
the collection name, use the -n option.  You can use the -d option to point at 
a config on disk, and it will be uploaded to a config in zookeeper named after 
the -n option or the collection. The collection name and the config name are 
not required to match.  You can use the same config for multiple collections.

> The documentation I think implies that all of the Solr instances are 
> automatically set with the collection.  There is no further action at this 
> point?

Solr will make an automatic decision as to which nodes will be used to hold the 
collection.  If you use the Collections API directly rather than the 
commandline, you can give Solr an explicit list of nodes to use.  Without the 
explicit list, Solr will spread the collection across the cluster as widely as 
it can.

https://urldefense.proofpoint.com/v2/url?u=https-3A__lucene.apache.org_solr_guide_6-5F6_collections-2Dapi.html-23CollectionsAPI-2Dcreate=DwIDaQ=ZgVRmm3mf2P1-XDAyDsu4A=ksx9qnQFG3QvxkP54EBPEzv1HHDjlk-MFO-7EONGCtY=y01KP5ZxH3grjfa0RoKxcEYEdhqcrmPikTmVLbotY6g=fnzfx6DkM_sSrLrRbVVxXceLNfqCC_w7eiiASQsG9S8=

The "bin/solr create" command, when used on SolrCloud, just makes an HTTP 
request to the Collections API, unless you use the -d option, in which case it 
will upload a config to zookeeper before calling the Collections API.

Thanks,
Shawn

Re: starting SolrCloud nodes

[Shawn Heisey]

On 12/1/2017 10:13 AM, Steve Pruitt wrote:
> Thanks to previous help.  I have a ZK ensemble of three nodes running.  I 
> have uploaded the config for my collection and the solr.xml file.
> I have Solr installed on three machines.
>
> I think my next steps are:
>
> Start up each Solr instance:  bin/solr start -c -z 
> "zoo1:2181,zoo2:2181,zoo3:2181"  // I have ZK_Hosts set to my ZK's, but the 
> documentation seems to say I need to provide the list here to prevent 
> embedded ZK from getting used.

The embedded ZK will only get started if you use the -c option and there
is no ZK_HOST variable and no -z option on the commandline.

If you use both -z and ZK_HOST, then the info I've seen says the -z
option will take priority.  I haven't looked at the script closely
enough to confirm, but that would be the most logical way to operate, so
it's probably correct.

If ZK_HOST is defined or you use the -z option, you do not need to
include the -c option when starting Solr.  SolrCloud mode is assumed
when ZK info is available.  The only time the -c option is *required* is
when you want to start the embedded zookeeper.  Having the option won't
hurt anything, though.

To start a solr *service* in cloud mode, all you need is to add ZK_HOST
to /etc/default/.in.sh, where  is the service name, which
defaults to solr.

> From one of the Solr instances create a collection:
> bin/solr create -c nameofuploadedconfig -s 3 -rf 2 //for example.

Nitpick:  The -c option on the create command is the name of the
collection.  To specify the name of the uploaded config, if it happens
to be different from the collection name, use the -n option.  You can
use the -d option to point at a config on disk, and it will be uploaded
to a config in zookeeper named after the -n option or the collection. 
The collection name and the config name are not required to match.  You
can use the same config for multiple collections.

> The documentation I think implies that all of the Solr instances are 
> automatically set with the collection.  There is no further action at this 
> point?

Solr will make an automatic decision as to which nodes will be used to
hold the collection.  If you use the Collections API directly rather
than the commandline, you can give Solr an explicit list of nodes to
use.  Without the explicit list, Solr will spread the collection across
the cluster as widely as it can.

https://lucene.apache.org/solr/guide/6_6/collections-api.html#CollectionsAPI-create

The "bin/solr create" command, when used on SolrCloud, just makes an
HTTP request to the Collections API, unless you use the -d option, in
which case it will upload a config to zookeeper before calling the
Collections API.

Thanks,
Shawn

Re: starting SolrCloud nodes

[Erick Erickson]

Looks good. If you set ZK_HOST in your solr.in.sh you can forgo
setting it when you start, but that's not necessary.

Best,
Erick

On Fri, Dec 1, 2017 at 9:13 AM, Steve Pruitt  wrote:
> Thanks to previous help.  I have a ZK ensemble of three nodes running.  I 
> have uploaded the config for my collection and the solr.xml file.
> I have Solr installed on three machines.
>
> I think my next steps are:
>
> Start up each Solr instance:  bin/solr start -c -z 
> "zoo1:2181,zoo2:2181,zoo3:2181"  // I have ZK_Hosts set to my ZK's, but the 
> documentation seems to say I need to provide the list here to prevent 
> embedded ZK from getting used.
>
> From one of the Solr instances create a collection:  bin/solr create -c 
> nameofuploadedconfig -s 3 -rf 2 //for example.
>
> The documentation I think implies that all of the Solr instances are 
> automatically set with the collection.  There is no further action at this 
> point?
>
> Thanks.
>
> -S

Re: Starting SolrCloud

[James Muerle]

Hello,

Thanks for reading this, but it has been resolved. I honestly don't know
what was happening, but restarting my shell and running the exact same
commands today instead of yesterday seems to have fixed it.

Best,
James

On Mon, Nov 28, 2016 at 8:07 PM, James Muerle  wrote:

> Hello,
>
> I am very new to Solr, and I'm excited to get it up and running on amazon
> ec2 for some prototypical testing. So, I've installed solr (and java) on
> one ec2 instance, and I've installed zookeeper on another. After starting
> the zookeeper server on the default port of 2181, I run this on the solr
> instance: "opt/solr/bin/solr start -c -z ".us-
> west-2.compute.amazonaws.com/solr"", which seems to complete successfully:
>
> Archiving 1 old GC log files to /opt/solr/server/logs/archived
> Archiving 1 console log files to /opt/solr/server/logs/archived
> Rotating solr logs, keeping a max of 9 generations
> Waiting up to 180 seconds to see Solr running on port 8983 [|]
> Started Solr server on port 8983 (pid=13038). Happy searching!
>
> But then when I run "/opt/solr/bin/solr status", I get this output:
>
> Found 1 Solr nodes:
>
> Solr process 13038 running on port 8983
>
> ERROR: Failed to get system information from http://localhost:8983/solr
> due to: org.apache.http.client.ClientProtocolException: Expected JSON
> response from server but received: 
> 
> 
> Error 500 Server Error
> 
> HTTP ERROR 500
> Problem accessing /solr/admin/info/system. Reason:
> Server ErrorCaused 
> by:org.apache.solr.common.SolrException:
> Error processing the request. CoreContainer is either not initialized or
> shutting down.
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:263)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:254)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1668)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:581)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)
> at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:226)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1160)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:511)
> at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1092)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.
> handle(ContextHandlerCollection.java:213)
> at org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:119)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(
> HttpChannel.java:308)
> at org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:244)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(
> FillInterest.java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> SelectChannelEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> produceAndRun(ExecuteProduceConsume.java:246)
> at org.eclipse.jetty.util.thread.strategy.
> ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:654)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> QueuedThreadPool.java:572)
> at java.lang.Thread.run(Thread.java:745)
> 
>
> 
> 
>
> Typically, this indicates a problem with the Solr server; check the Solr
> server logs for more information.
>
>
> I don't quite understand what things could be causing this problem, so I'm
> really at a loss at the moment. If you need any additional information, I'd
> be glad to provide it.
>
> Thanks for reading!
> James
>

Re: Starting SolrCloud

[Erick Erickson]

You need to find the solr.log file and examine it. What this usually
means is that something's wrong
with, say, your Solr configs. You should see a more informative
message in the Solr log, usually
it's a stack trace.

You say that your start "seems to complete successfully". That implies
that you were prompted for things like
how many Solr instances you wanted to start, a base configset, the
name of your collection and the like. Did
all that occur?


Best,
Erick



On Mon, Nov 28, 2016 at 8:07 PM, James Muerle  wrote:
> Hello,
>
> I am very new to Solr, and I'm excited to get it up and running on amazon
> ec2 for some prototypical testing. So, I've installed solr (and java) on
> one ec2 instance, and I've installed zookeeper on another. After starting
> the zookeeper server on the default port of 2181, I run this on the solr
> instance: "opt/solr/bin/solr start -c -z ".
> us-west-2.compute.amazonaws.com/solr"", which seems to complete
> successfully:
>
> Archiving 1 old GC log files to /opt/solr/server/logs/archived
> Archiving 1 console log files to /opt/solr/server/logs/archived
> Rotating solr logs, keeping a max of 9 generations
> Waiting up to 180 seconds to see Solr running on port 8983 [|]
> Started Solr server on port 8983 (pid=13038). Happy searching!
>
> But then when I run "/opt/solr/bin/solr status", I get this output:
>
> Found 1 Solr nodes:
>
> Solr process 13038 running on port 8983
>
> ERROR: Failed to get system information from http://localhost:8983/solr due
> to: org.apache.http.client.ClientProtocolException: Expected JSON response
> from server but received: 
> 
> 
> Error 500 Server Error
> 
> HTTP ERROR 500
> Problem accessing /solr/admin/info/system. Reason:
> Server ErrorCaused
> by:org.apache.solr.common.SolrException: Error processing the
> request. CoreContainer is either not initialized or shutting down.
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
> at java.lang.Thread.run(Thread.java:745)
> 
>
> 
> 
>
> Typically, this indicates a problem with the Solr server; check the Solr
> server logs for more information.
>
>
> I don't quite understand what things could be causing this problem, so I'm
> really at a loss at the moment. If you need any additional information, I'd
> be glad to provide it.
>
> Thanks for reading!
> James