Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17037
to look at the new patch set (#13).
Change subject: KUDU-2612 tablet servers automatically register txn participants
......................................................................
KUDU-2612 tablet servers automatically register txn participants
With this patch, tablet servers automatically register tablets as
transaction participants and issue appropriate BEGIN_TXN operations
upon receiving write operations targeting tablet replicas which they
host.
Internally, the newly introduced logic is mostly embedded into the
TxnOpDispatcher class. Every TxnOpDispatcher object is allowed to
accumulate up to a certain number of pending write requests in its queue
while awaiting for the completion of the preliminary tasks mentioned
above. Once the TxnOpDispatcher's queue is at capacity, a tablet server
starts responding with the ErrorStatusPB::ERROR_SERVER_TOO_BUSY error
code to incoming write requests in the context of the corresponding
transaction, and Kudu clients automatically retry requests, as expected.
The capacity of the TxnOpDispatcher's queue is controlled by a newly
introduced --tablet_max_pending_txn_write_ops flag. By default, the
flag is set to 2. Buffering a few write requests should help to avoid
needless retries in case if a client packs many operations into one
write request: that's exactly the behavior for Kudu client sessions
using the AUTO_FLUSH_BACKGROUND mode. If such buffering isn't desired
for some reason, set --tablet_max_pending_txn_write_ops=0: in that case
a client will retry the very first operation sent to a tablet server
in the context of a transaction until the tablet server completes the
preliminary tasks mentioned above. The flag has runtime semantics,
so no restart of a tablet server is required upon modification of the
flag's setting.
Each tablet replica maintains a txn_id --> TxnOpDispatcher map,
with an entry's lifecycle as below:
* An entry is added upon receiving write request in the context
of a multi-row transaction.
* An entry is removed upon applying either ParticipantOpPB::ABORT_TXN
or ParticipantOpPB::FINALIZE_COMMIT operation.
* If a write request is received after transaction has been committed
or aborted, the entry is automatically removed once receiving
corresponding error response from any of the following components:
** from TxnStatusManager in at attempts to register a participant
in the context of committed/aborted transaction
** from the replica itself in an attempt to add
ParticipantOpPB::BEGIN_COMMIT operation
In other words, the system automatically gets rid of no-longer-needed
TxnOpDispatcher entries.
This patch also contains several test scenarios to cover the newly
introduced functionality. I also updated other related tests to remove
the artificial registration of corresponding transaction participants,
so those tests now rely on the newly introduced functionality.
Change-Id: Ia383f7afd208c44695c57aab82e3818fa1712ce6
---
M src/kudu/client/batcher.cc
M src/kudu/client/client-test.cc
M src/kudu/integration-tests/CMakeLists.txt
M src/kudu/integration-tests/fuzz-itest.cc
M src/kudu/integration-tests/txn_commit-itest.cc
A src/kudu/integration-tests/txn_write_ops-itest.cc
M src/kudu/tablet/ops/participant_op.cc
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tserver/tablet_service.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
12 files changed, 2,318 insertions(+), 127 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/37/17037/13
--
To view, visit http://gerrit.cloudera.org:8080/17037
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia383f7afd208c44695c57aab82e3818fa1712ce6
Gerrit-Change-Number: 17037
Gerrit-PatchSet: 13
Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
