Hello Alexey Serbin, Kudu Jenkins, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16952
to look at the new patch set (#13).
Change subject: KUDU-2612: background task to commit transaction
......................................................................
KUDU-2612: background task to commit transaction
This patch introduces background tasks that get run when
KuduTransaction::Commit() is called. The typical workflow is as follows:
1. Commit() is called, resulting in a BeginCommitTransaction() call on
the TxnStatusManager.
2. An update is made to the transaction status table, marking the
transaction's state as COMMIT_IN_PROGRESS.
3. The commit tasks are initiated -- BEGIN_COMMIT ops are sent
asynchronously to every participant of the transaction.
4. Once all responses are received from the participants, a commit
timestamp is determined, and FINALIZE_COMMIT ops are sent
asynchronously to every participant.
5. Once all responses are received from the participants, an update is
made to the transaction status table, marking the transaction's state
as COMMITTED.
There are some nuances here around error handling. Namely, what do we do
if there are errors in sending the above requests? Well, it depends on
the error. Transient errors (i.e. timeouts) are simply retried. More
permanent errors need a bit more thought though:
- If a participant has been deleted, what do we do? This patch makes a
best effort attempt to abort the transaction if so.
- Any other kinds of errors (e.g. illegal state errors from a
participant) aren't expected in normal operation of a cluster. For
this, we stop the commit task and log a warning. Hopefully an operator
can intervene.
Some follow-ups to expect:
- This isn't as robust to failures as an approach that writes an
intermediate state to the TxnStatusManager in between steps 3 and 4. A
follow-up patch will implement that.
- A separate patch will implement aborting transactions.
- I disabled the background tasks in some tests that assume state
changes are entirely controlled by clients. A follow-up change will
address these to account for the state changes more organically.
Change-Id: Ie2258dded3ab3d527cb5d0abdc7d5e7deb4da15e
---
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java
M src/kudu/client/client-test.cc
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/txn_commit-itest.cc
M src/kudu/integration-tests/txn_status_manager-itest.cc
M src/kudu/integration-tests/txn_status_table-itest.cc
M src/kudu/master/txn_manager-test.cc
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/txn_coordinator.h
M src/kudu/transactions/transactions.proto
M src/kudu/transactions/txn_status_entry.cc
M src/kudu/transactions/txn_status_manager-test.cc
M src/kudu/transactions/txn_status_manager.cc
M src/kudu/transactions/txn_status_manager.h
M src/kudu/transactions/txn_system_client.cc
M src/kudu/transactions/txn_system_client.h
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
18 files changed, 1,329 insertions(+), 112 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/16952/13
--
To view, visit http://gerrit.cloudera.org:8080/16952
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie2258dded3ab3d527cb5d0abdc7d5e7deb4da15e
Gerrit-Change-Number: 16952
Gerrit-PatchSet: 13
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
