https://bugs.opendaylight.org/show_bug.cgi?id=8022
The dump from http://<host>:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-<member #>-shard-topology-operational,type=DistributedOperDatastore didn’t work for me. So I only added the logs with TRACE on SHARDs. From: Tom Pantelis [mailto:tompante...@gmail.com] Sent: Monday, March 20, 2017 2:41 PM To: Sela, Guy <guy.s...@hpe.com> Cc: controller-dev@lists.opendaylight.org; odl netvirt dev <netvirt-...@lists.opendaylight.org> Subject: Re: [controller-dev] prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 Actually set the log level to trace instead of debug. On Mon, Mar 20, 2017 at 8:29 AM, Tom Pantelis <tompante...@gmail.com<mailto:tompante...@gmail.com>> wrote: Hmm... please enable Shard debug (add log4j.logger.org.opendaylight.controller.cluster.datastore.Shard=debug to etc/org.ops4j.pax.logging.cfg) on all 3 nodes and let it run for like 10 sec and capture the logs and open a bug. Also capture the JMX Shard output from each node, ie: http://<host>:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-<member #>-shard-topology-operational,type=DistributedOperDatastore replacing <member #> appropriately for each node. Please provide the steps that led up to it, eg were nodes restarted etc On Mon, Mar 20, 2017 at 7:50 AM, Sela, Guy <guy.s...@hpe.com<mailto:guy.s...@hpe.com>> wrote: Latest code in a 3-node cluster, and I’m seeing the following errors while OVSs are connecting to the ODL. 2017-03-20 13:48:36,101 | INFO | lt-dispatcher-29 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): The prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 2017-03-20 13:48:36,101 | INFO | lt-dispatcher-29 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=1, success=false, followerId=member-3-shard-topology-operational, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3] 2017-03-20 13:48:36,621 | INFO | lt-dispatcher-51 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): The prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 2017-03-20 13:48:36,621 | INFO | lt-dispatcher-51 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=1, success=false, followerId=member-3-shard-topology-operational, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3] 2017-03-20 13:48:37,140 | INFO | lt-dispatcher-29 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): The prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 2017-03-20 13:48:37,140 | INFO | lt-dispatcher-29 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=1, success=false, followerId=member-3-shard-topology-operational, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3] 2017-03-20 13:48:37,661 | INFO | lt-dispatcher-51 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): The prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 2017-03-20 13:48:37,661 | INFO | lt-dispatcher-51 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=1, success=false, followerId=member-3-shard-topology-operational, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3] 2017-03-20 13:48:38,180 | INFO | lt-dispatcher-68 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): The prevLogIndex 3 was found in the log but the term -1 is not equal to the append entriesprevLogTerm 1 - lastIndex: 5, snapshotIndex: 4 2017-03-20 13:48:38,180 | INFO | lt-dispatcher-68 | Shard | 207 - org.opendaylight.controller.sa<http://org.opendaylight.controller.sa>l-clustering-commons - 1.5.0.SNAPSHOT | member-3-shard-topology-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=1, success=false, followerId=member-3-shard-topology-operational, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3] Thanks, Guy Sela _______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org<mailto:controller-dev@lists.opendaylight.org> https://lists.opendaylight.org/mailman/listinfo/controller-dev
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev