Re: solrcloud goes down

[Shalin Shekhar Mangar]

You are asking many different questions. Lets take them up one by one:

You say that you use the singleton pattern to create and get the Solr
server connection. Yes, that is fine. The CloudSolrServer is thread safe
and can be shared across requests/threads. When your application exits, you
should shutdown that object.

Once you shutdown ZooKeeper, Solr cannot process any updates. This is by
design. A running and functional Zookeeper is necessary for SolrCloud.

On Fri, Jul 1, 2016 at 2:15 PM, 穆 俊峰  wrote:

> Dear Mr. Experts!
>
>
> I came across a problem when use solrcloud, it goes down. The version is
> 4.9.1.
>
> we got the java core log, it looks like the http connection pool is
> exhausted and most threads are waiting to get a free connection.
>
> I have increased http connection defaults for the SolrJ client, and also
> configed the connection defaults in solr.xml for all shard servers as below.
>
>  class="HttpShardHandlerFactory">
> 6
> 3
> 1
> 500
> 
>
>
>
> the java stack as below
>
> "httpShardExecutor-3-thread-541" prio=10 tid=0x7f7b1c02b000 nid=0x20af
> waiting on condition [0x7f79fd49]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x000605710068> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133)
> at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282)
> at
> org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
> at
> org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177)
> at
> org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170)
> at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:208)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:195)
> at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:422)
> at
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:452)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
> at
> org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:205)
> at
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:162)
> at
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
>
>
>
>
> besides, we use the singleton pattern to create and get the solr server
> connection, I wonder if this pattern is OK. Once I shutdown the zookeeper,
> the application can not do the solr query, the error as "no live
> SolrServers available to handle this request".
>
> so I need to restart our connection to reconnect solrcloud.
>
> as we use the singleton pattern, we do not use the method of "shutdown" to
> release the solrserver connection, will this be a problem? or we need to
> create the connection on demand every time?
>
>
> please help me, look forward to your reply. Thanks a lot!
>
>
> Best Regards!
> Kent Mu
>



-- 
Regards,
Shalin Shekhar Mangar.

Error when highlighting on multiValued fields

[Zheng Lin Edwin Yeo]

Hi,

I'm getting this error when I tried to do highlighting search on a
multiValued field.

  "error":{
"metadata":[
  "error-class","org.apache.solr.common.SolrException",
  
"root-error-class","org.apache.lucene.search.highlight.InvalidTokenOffsetsException"],
"msg":"org.apache.lucene.search.highlight.InvalidTokenOffsetsException:
Token eee1 exceeds length of provided text sized 3",
"trace":"org.apache.solr.common.SolrException:
org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token
eee1 exceeds length of provided text sized 3\r\n\tat
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:614)\r\n\tat
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingOfField(DefaultSolrHighlighter.java:448)\r\n\tat
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:410)\r\n\tat
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:142)\r\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)\r\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\r\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\r\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\r\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\r\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\r\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\r\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\r\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\r\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\r\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\r\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\r\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\r\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\r\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\r\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\r\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\r\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\r\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:518)\r\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\r\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\r\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\r\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\tat
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\r\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\r\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\r\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\r\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\r\n\tat
java.lang.Thread.run(Thread.java:745)\r\nCaused by:
org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token
edm1 exceeds length of provided text sized 3\r\n\tat
org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)\r\n\tat
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:602)\r\n\t...
34 more\r\n",
"code":500}}


Below is my configuration for highlight.

  
  
   none
   10
   json
   true
  text
  id, title, subject, content_type, last_modified, score


  on
   id, title, subject, content, tag
  true
   true
  200
  10
  true

true
signature
true
100
 
  


Below is my setting for the multiValued field in question

   


I got this error when there are 3 or more values in the multiValued field.
The fieldType for the field is "String".

What could be the error?

I'm using Solr 6.1.0

Regards,
Edwin

Re: Json Parse Exception in CloudSolrStream class

[Erick Erickson]

What version of Solr? Are you perhaps running into:
https://issues.apache.org/jira/browse/SOLR-7647?

On Sun, Jul 3, 2016 at 5:54 PM, Susmit Shukla  wrote:
> Hi,
>
> I'm using a string field in sort parameters of a solr query. The query is
> used with /export handler to stream data using CloudSolrStream. When the
> data in field contains a double quote, the cloudSolrStream fails to read
> data and throws this error -
>
> field data = "first (alias) last" 
>
> org.noggit.JSONParser$ParseException: Expected ',' or '}':
> char=F,position=43701
> BEFORE='4DC93D74AEDE28292D27A2EC39F8761E1","field_dv":""F' AFTER='irst
> (alias) last" 
> at org.noggit.JSONParser.err(JSONParser.java:356)
>
> at org.noggit.JSONParser.nextEvent(JSONParser.java:958)
>
> at org.noggit.ObjectBuilder.getObject(ObjectBuilder.java:124)
>
> at org.noggit.ObjectBuilder.getVal(ObjectBuilder.java:57)
>
> at org.noggit.ObjectBuilder.getVal(ObjectBuilder.java:37)
>
> at
> org.apache.solr.client.solrj.io.stream.JSONTupleStream.next(JSONTupleStream.java:85)
>
> at
> org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:148)
>
>
> Should I json escape the string before indexing.. Also I do not need to
> retrieve this field but /export handler makes it mandatory to retrieve all
> the fields that are used for sort criteria.
>
>
> Thanks,
>
> Susmit

Re: 500 error querying solrcloud with >1 shard

[Nuhaa All Bakry]

We’re using Solr 6.

I think it’s because of the Suggester component - it being not supported for 
distributed search. I had to specify shards=shard1 for the query to work. But 
even then if the other shard has nothing (I have only 1 document indexed on 
shard2), it still returns error below. I wonder if that should be a concern?

Thanks for the heads up on the bug. WIll keep that in mind!

--
nuhaa
http://about.me/nuhaa

> On 2 Jul 2016, at 1:16 AM, Erick Erickson  wrote:
> 
> Hmmm, don't see anything outstanding. What version of Solr?
> Historically there have been some bugs when a shard doesn't
> happen to host any docs, which sometimes occurs when you
> are testing and just add a few docs to the collection, is that
> a possibility?
> 
> Best,
> Erick
> 
> On Thu, Jun 30, 2016 at 10:40 PM, Nuhaa All Bakry  
> wrote:
>> ah i see
>> 
>> https://ibin.co/2mWaie7IVxDF.png
>> 
>> Collection A:
>> https://ibin.co/2mWawhCm76cN.png
>> 
>> Collection B:
>> https://ibin.co/2mWb4BmlMom2.png
>> 
>> 
>> 
>> --
>> nuhaa
>> http://about.me/nuhaa
>> 
>>> On 1 Jul 2016, at 12:24 PM, Erick Erickson  wrote:
>>> 
>>> None of the pasted images came through, the mail server is quite aggressive
>>> about stripping them. You'll need to upload them somewhere and provide a
>>> link.
>>> 
>>> Best,
>>> Erick
>>> 
>>> On Thu, Jun 30, 2016 at 7:36 PM, nuhaa  wrote:
>>> 
 hi all,
 
 We have a solrcloud setup with zookeeper, and right now we're testing it
 with indexing and querying.
 
 
 *collection A:*
 
 *collection B:*
 
 
 
 I'm trying to figure out why on collection B, indexing works but querying
 doesn't. I believe by looking at Collection B > Schema > Load Term Info on
 a selected field, if I can see the value histogram, that means they're
 indexed successfully.
 
 On collection A, indexing and querying both work.
 
 This is what returned when querying on collection B:
 
 {
 "responseHeader":{
   "zkConnected":true,
   "status":500,
   "QTime":17,
   "params":{
 "q":"*:*",
 "indent":"on",
 "wt":"json",
 "_":"1467275265841"}},
 "error":{
   "trace":"java.lang.NullPointerException\n\tat
 org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1209)\n\tat
 org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1060)\n\tat
 org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:759)\n\tat
 org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:738)\n\tat
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)\n\tat
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\n\tat
 org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\n\tat
 org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\n\tat
 org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\n\tat
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\n\tat
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\n\tat
 org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\n\tat
 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\n\tat
 org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
 org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
 org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\n\tat
 org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\n\tat
 org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
 org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\n\tat
 org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
 org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
 org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
 org.eclipse.jetty.server.Server.handle(Server.java:518)\n\tat
 org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\n\tat
 org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\n\tat
 org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat
 org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat
 

Json Parse Exception in CloudSolrStream class

[Susmit Shukla]

Hi,

I'm using a string field in sort parameters of a solr query. The query is
used with /export handler to stream data using CloudSolrStream. When the
data in field contains a double quote, the cloudSolrStream fails to read
data and throws this error -

field data = "first (alias) last" 

org.noggit.JSONParser$ParseException: Expected ',' or '}':
char=F,position=43701
BEFORE='4DC93D74AEDE28292D27A2EC39F8761E1","field_dv":""F' AFTER='irst
(alias) last"