Re: Few document replication not happen in solr cloud

2017-08-22 Thread Sanjay Lokhande
   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute
(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call
(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter
(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter
(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle
(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle
(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle
(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle
(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle
(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope
(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope
(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope
(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle
(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle
(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle
(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle
(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle
(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable
(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run
(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run
(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:785)
Caused by: java.util.concurrent.TimeoutException: Idle timeout
expired: 5/5 ms
at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout
(IdleTimeout.java:161)
at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
at java.util.concurrent.Executors$RunnableAdapter.call
(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:267)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor
$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1143)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:618)
... 1 more

    Thanks in advance.

Regards,
Sanjay



From:   "Sanjay Lokhande" <sanjaylokha...@in.ibm.com>
To: solr-user@lucene.apache.org
Date:   08/22/2017 09:00 PM
Subject:Few document replication not happen in solr cloud





Hi guys,
I having 5 node solr cloud setup.  The logs indicates leader
and 2 solr node receiving document add request.  The other 2 nodes logs did
not show the entry to add the document and also these nodes missing the
document.
  - how such issue can be troubleshoot? any pointers?
 - any logging level which can help is this situation?

Regards,
Sanjay






Few document replication not happen in solr cloud

2017-08-22 Thread Sanjay Lokhande


Hi guys,
I having 5 node solr cloud setup.  The logs indicates leader
and 2 solr node receiving document add request.  The other 2 nodes logs did
not show the entry to add the document and also these nodes missing the
document.
  - how such issue can be troubleshoot? any pointers?
 - any logging level which can help is this situation?

Regards,
Sanjay




Re: Solr cloud replica nodes missing some documents

2017-08-21 Thread Sanjay Lokhande


Any pointer guys.



From:   Sanjay Lokhande/India/IBM
To: solr-user@lucene.apache.org
Date:   08/18/2017 02:12 PM
Subject:Solr cloud replica nodes missing some documents


Hello  guys,

  I am having 5 nodes solr cloud setup with single shard. The solr version
is 5.2.1.
  server1 (http://146.XXX.com:4001/solr/contracts_shard1_replica4)is the
leader.
  A document with id '43e14a86cbdd422880cac22d9a15d3c0' was not replicated
3 nodes.
  Log shows that the "{add=[43e14a86cbdd422880cac22d9a15d3c0
(1573510697298427904)]}" request is received only by leader and server5
node.
  The server2, server3 & server4 node did not receive the request and hence
the document is missing in these nodes.

 Search "43e14a86cbdd422880cac22d9a15d3c0 "
  C:\solrIssue\solr_server1.log
INFO  - 2017-07-21 05:54:59.853; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.853; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:59:23.845; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {add=[43e14a86cbdd422880cac22d9a15d3c0
(1573510697298427904)]} 0 26582
  C:\solrIssue\solr_server2\solr.log.1
INFO  - 2017-07-21 05:54:59.595; [contracts shard1 core_node4
contracts_shard1_replica5]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica5] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.595; [contracts shard1 core_node4
contracts_shard1_replica5]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica5] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
  C:\solrIssue\solr_server3.log
INFO  - 2017-07-21 05:54:59.844; [contracts shard1 core_node1
contracts_shard1_replica3]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica3] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.844; [contracts shard1 core_node1
contracts_shard1_replica3]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica3] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
  C:\solrIssue\solr_server4\solr.log.1
INFO  - 2017-07-21 05:54:59.734; [contracts shard1 core_node3
contracts_shard1_replica1]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica1] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.734; [contracts shard1 core_node3
contracts_shard1_replica1]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica1] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http:/

Solr cloud replica nodes missing some documents

2017-08-18 Thread Sanjay Lokhande


Hello  guys,

  I am having 5 nodes solr cloud setup with single shard. The solr version
is 5.2.1.
  server1 (http://146.XXX.com:4001/solr/contracts_shard1_replica4)is the
leader.
  A document with id '43e14a86cbdd422880cac22d9a15d3c0' was not replicated
3 nodes.
  Log shows that the "{add=[43e14a86cbdd422880cac22d9a15d3c0
(1573510697298427904)]}" request is received only by leader and server5
node.
  The server2, server3 & server4 node did not receive the request and hence
the document is missing in these nodes.

 Search "43e14a86cbdd422880cac22d9a15d3c0 "
  C:\solrIssue\solr_server1.log
INFO  - 2017-07-21 05:54:59.853; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.853; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:59:23.845; [contracts shard1 core_node2
contracts_shard1_replica4]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica4] webapp=/solr path=/update params=
{wt=javabin=2} {add=[43e14a86cbdd422880cac22d9a15d3c0
(1573510697298427904)]} 0 26582
  C:\solrIssue\solr_server2\solr.log.1
INFO  - 2017-07-21 05:54:59.595; [contracts shard1 core_node4
contracts_shard1_replica5]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica5] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.595; [contracts shard1 core_node4
contracts_shard1_replica5]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica5] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
  C:\solrIssue\solr_server3.log
INFO  - 2017-07-21 05:54:59.844; [contracts shard1 core_node1
contracts_shard1_replica3]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica3] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.844; [contracts shard1 core_node1
contracts_shard1_replica3]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica3] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
  C:\solrIssue\solr_server4\solr.log.1
INFO  - 2017-07-21 05:54:59.734; [contracts shard1 core_node3
contracts_shard1_replica1]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica1] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad...
INFO  - 2017-07-21 05:54:59.734; [contracts shard1 core_node3
contracts_shard1_replica1]
org.apache.solr.update.processor.LogUpdateProcessor;
[contracts_shard1_replica1] webapp=/solr path=/update params=
{update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2}
 {deleteByQuery=id:(9467353f398448788c261aa347d75b8b
93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022
43e14a86cbdd422880cac22d9a15d3c0