Mike Percy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11142 )
Change subject: KUDU-2463 pt 1: adjust MVCC when replaying no-ops ...................................................................... KUDU-2463 pt 1: adjust MVCC when replaying no-ops Previously, during tablet bootstrap, a tablet replica would only update its MVCC safetime based on write and alter schema messages, as the timestamps in these messages are guaranteed to be serialized with respect to one another, by virtue of being assigned in a single thread (the prepare thread) on the leader replica. >From this, we conclude that timestamps for write and alter schema operations are monotonically increasing in unison with opid. Because no-op and change config operations are not serialized through the prepare thread, they don't necessarily provide the same timestamp monotonicity guarantees. However, in the case of no-ops replicated to assert leadership, we get the same timestamp monotonicity guarantee due to a different mechanism. This patch takes advantage of the fact that our Raft implementation ensures the following sequence of events: 1. replica A becomes leader of Term N 2. leader A assigns a timestamp t1 to its no-op 3. leader A replicates the no-op to replicas B and C, asserting its leadership for Term N 4. leader A prepares a write and assigns it a timestamp t2 > t1 5. leader A replicates the write to replicas B and C, checking that it is leader for the Term N Given the above series of operations, for a given term, the no-op used to assert leadership is always assigned a timestamp that must be lower than any other ops in that term. As such, the timestamps assigned to no-ops can and should be used to bump MVCC safe time. This patch does so at bootstrap time with committed no-ops found in the WAL. This alone isn't enough to always prevent KUDU-2463, but it dramatically reduces the likelihood of it occuring. Change-Id: I26deff32da8c990cb8a2ba220bb81858ddd6d73f Reviewed-on: http://gerrit.cloudera.org:8080/11142 Tested-by: Kudu Jenkins Reviewed-by: Mike Percy <mpe...@apache.org> --- M src/kudu/consensus/consensus.proto M src/kudu/integration-tests/CMakeLists.txt M src/kudu/integration-tests/raft_consensus-itest.cc A src/kudu/integration-tests/timestamp_advancement-itest.cc M src/kudu/tablet/tablet_bootstrap-test.cc M src/kudu/tablet/tablet_bootstrap.cc 6 files changed, 276 insertions(+), 9 deletions(-) Approvals: Kudu Jenkins: Verified Mike Percy: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11142 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I26deff32da8c990cb8a2ba220bb81858ddd6d73f Gerrit-Change-Number: 11142 Gerrit-PatchSet: 11 Gerrit-Owner: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
