Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17105
to look at the new patch set (#3).
Change subject: KUDU-2612 add perf scenario to TxnWriteOpsITest
......................................................................
KUDU-2612 add perf scenario to TxnWriteOpsITest
This patch adds WriteOpPerf scenario to TxnWriteOpsITest. The new
scenario is to evaluate --tablet_max_pending_txn_write_ops setting for
tablet servers: it runs for a short time to count number of completed
Write RPCs in context of a transactional session. The scenario focuses
on single-row write operations to pinpoint the latency of processing
txn write operations when performing registration of transaction
participants. Probably, the current locking approach for the
TxnOpDispatcher's queue while submitting the accumulated operations
isn't optimal. Apparently, the exponential back-off timing built into
the client's RPC retry logic is also an important factor: I saw
significant deviation in the number of completed RPCs from run to run.
Below are results averaged for 100 runs of the benchmark scenario with
varying --max_pending_txn_write_ops accordingly and the following
settings fixed:
--prepare_connections_to_tservers=true
--clients=8
--sessions_per_client=1
--benchmark_run_time_ms=50
I used the following script to get the accumulated results for
writes in a transactional context:
for i in {0..99}; do
./bin/txn_write_ops-itest --gtest_filter='*WriteOpPerf' \
--max_pending_txn_write_ops=<X> 2>&1 | grep 'write RPCs' | \
awk '{print $9}'; done | \
awk 'BEGIN {sum=0} {sum += $0} END {print sum}'
RELEASE build:
non-transactional : 581.24 write RPCs
--max_pending_txn_write_ops=0 : 442.13 txn write RPCs
--max_pending_txn_write_ops=2 : 494.33 txn write RPCs
--max_pending_txn_write_ops=5 : 471.90 txn write RPCs
--max_pending_txn_write_ops=10 : 490.22 txn write RPCs
--max_pending_txn_write_ops=20 : 469.21 txn write RPCs
DEBUG build:
non-transactional : 150.24 write RPCs
--max_pending_txn_write_ops=0 : 83.74 txn write RPCs
--max_pending_txn_write_ops=2 : 98.18 txn write RPCs
--max_pending_txn_write_ops=5 : 95.23 txn write RPCs
--max_pending_txn_write_ops=10 : 98.12 txn write RPCs
--max_pending_txn_write_ops=20 : 94.40 txn write RPCs
Change-Id: I0370dbb289a4e1cfc154205ae92e13da510682b4
---
M src/kudu/integration-tests/txn_write_ops-itest.cc
1 file changed, 190 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/17105/3
--
To view, visit http://gerrit.cloudera.org:8080/17105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0370dbb289a4e1cfc154205ae92e13da510682b4
Gerrit-Change-Number: 17105
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
