Re: Few document replication not happen in solr cloud
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064) at org.apache.solr.servlet.HttpSolrCall.execute (HttpSolrCall.java:654) at org.apache.solr.servlet.HttpSolrCall.call (HttpSolrCall.java:450) at org.apache.solr.servlet.SolrDispatchFilter.doFilter (SolrDispatchFilter.java:227) at org.apache.solr.servlet.SolrDispatchFilter.doFilter (SolrDispatchFilter.java:196) at org.eclipse.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle (SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle (SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope (SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle (HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run (AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run (QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:785) Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 5/5 ms at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout (IdleTimeout.java:161) at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:267) at java.util.concurrent.ScheduledThreadPoolExecutor $ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor $ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:618) ... 1 more Thanks in advance. Regards, Sanjay From: "Sanjay Lokhande" <sanjaylokha...@in.ibm.com> To: solr-user@lucene.apache.org Date: 08/22/2017 09:00 PM Subject:Few document replication not happen in solr cloud Hi guys, I having 5 node solr cloud setup. The logs indicates leader and 2 solr node receiving document add request. The other 2 nodes logs did not show the entry to add the document and also these nodes missing the document. - how such issue can be troubleshoot? any pointers? - any logging level which can help is this situation? Regards, Sanjay
Few document replication not happen in solr cloud
Hi guys, I having 5 node solr cloud setup. The logs indicates leader and 2 solr node receiving document add request. The other 2 nodes logs did not show the entry to add the document and also these nodes missing the document. - how such issue can be troubleshoot? any pointers? - any logging level which can help is this situation? Regards, Sanjay
Re: Solr cloud replica nodes missing some documents
Any pointer guys. From: Sanjay Lokhande/India/IBM To: solr-user@lucene.apache.org Date: 08/18/2017 02:12 PM Subject:Solr cloud replica nodes missing some documents Hello guys, I am having 5 nodes solr cloud setup with single shard. The solr version is 5.2.1. server1 (http://146.XXX.com:4001/solr/contracts_shard1_replica4)is the leader. A document with id '43e14a86cbdd422880cac22d9a15d3c0' was not replicated 3 nodes. Log shows that the "{add=[43e14a86cbdd422880cac22d9a15d3c0 (1573510697298427904)]}" request is received only by leader and server5 node. The server2, server3 & server4 node did not receive the request and hence the document is missing in these nodes. Search "43e14a86cbdd422880cac22d9a15d3c0 " C:\solrIssue\solr_server1.log INFO - 2017-07-21 05:54:59.853; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.853; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:59:23.845; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {add=[43e14a86cbdd422880cac22d9a15d3c0 (1573510697298427904)]} 0 26582 C:\solrIssue\solr_server2\solr.log.1 INFO - 2017-07-21 05:54:59.595; [contracts shard1 core_node4 contracts_shard1_replica5] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica5] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.595; [contracts shard1 core_node4 contracts_shard1_replica5] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica5] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... C:\solrIssue\solr_server3.log INFO - 2017-07-21 05:54:59.844; [contracts shard1 core_node1 contracts_shard1_replica3] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica3] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.844; [contracts shard1 core_node1 contracts_shard1_replica3] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica3] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... C:\solrIssue\solr_server4\solr.log.1 INFO - 2017-07-21 05:54:59.734; [contracts shard1 core_node3 contracts_shard1_replica1] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica1] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.734; [contracts shard1 core_node3 contracts_shard1_replica1] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica1] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http:/
Solr cloud replica nodes missing some documents
Hello guys, I am having 5 nodes solr cloud setup with single shard. The solr version is 5.2.1. server1 (http://146.XXX.com:4001/solr/contracts_shard1_replica4)is the leader. A document with id '43e14a86cbdd422880cac22d9a15d3c0' was not replicated 3 nodes. Log shows that the "{add=[43e14a86cbdd422880cac22d9a15d3c0 (1573510697298427904)]}" request is received only by leader and server5 node. The server2, server3 & server4 node did not receive the request and hence the document is missing in these nodes. Search "43e14a86cbdd422880cac22d9a15d3c0 " C:\solrIssue\solr_server1.log INFO - 2017-07-21 05:54:59.853; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.853; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:59:23.845; [contracts shard1 core_node2 contracts_shard1_replica4] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica4] webapp=/solr path=/update params= {wt=javabin=2} {add=[43e14a86cbdd422880cac22d9a15d3c0 (1573510697298427904)]} 0 26582 C:\solrIssue\solr_server2\solr.log.1 INFO - 2017-07-21 05:54:59.595; [contracts shard1 core_node4 contracts_shard1_replica5] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica5] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.595; [contracts shard1 core_node4 contracts_shard1_replica5] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica5] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... C:\solrIssue\solr_server3.log INFO - 2017-07-21 05:54:59.844; [contracts shard1 core_node1 contracts_shard1_replica3] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica3] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.844; [contracts shard1 core_node1 contracts_shard1_replica3] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica3] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... C:\solrIssue\solr_server4\solr.log.1 INFO - 2017-07-21 05:54:59.734; [contracts shard1 core_node3 contracts_shard1_replica1] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica1] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0 a0af8cb24ef94d25b9691eee1f7024ca 8ad... INFO - 2017-07-21 05:54:59.734; [contracts shard1 core_node3 contracts_shard1_replica1] org.apache.solr.update.processor.LogUpdateProcessor; [contracts_shard1_replica1] webapp=/solr path=/update params= {update.distrib=FROMLEADER&_version_=-1573510446380482560=http://146.XXX.com:4001/solr/contracts_shard1_replica4/=javabin=2} {deleteByQuery=id:(9467353f398448788c261aa347d75b8b 93332ab7f7ff4141a371713871ab65ad 8568e0eab8364bfc89c876aadfa01022 43e14a86cbdd422880cac22d9a15d3c0