Hello Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11395
to look at the new patch set (#2).
Change subject: KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
......................................................................
KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
This patch rewrites much of DeltaFileIterator to leverage DeltaPreparer for
delta preparation and service. Seeing as DeltaPreparer was originally built
for DMSIterator, here are the various augmentations that were necessary to
support DeltaFileIterator:
- REINSERT support, which meant more complicated deletion state tracking.
- FilterColumnIdsAndCollectDeltas support, cribbed from DeltaFileIterator.
- A templatized traits system to control which features were enabled. This
also meant templatizing both DeltaPreparer and DeltaFileIterator.
- Early out from the "apply all deltas for a row" loop when the timestamp
is no longer relevant. I modified DMSIterator to take advantage of this,
which should result in a performance improvement.
I tried to centralize as much state tracking in DeltaPreparer, though there
were several aspects of this that were confusing (namely prepared_idx_,
last_added_idx_, and prepared_count_).
The patch's improvements should be most noticeable on wide schemas where the
column-by-column ApplyUpdates() approach yielded a lot of unnecessary delta
decoding. I don't have a suitable microbenchmark to prove this, but I did
run diskrowset-test's TestDeltaApplicationPerformance under perf and the
resulting flame graphs showed the bulk of the iteration time as having moved
from ApplyUpdates() to PrepareBatch().
Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
---
M src/kudu/tablet/delta_key.h
M src/kudu/tablet/delta_store.cc
M src/kudu/tablet/delta_store.h
M src/kudu/tablet/deltafile.cc
M src/kudu/tablet/deltafile.h
M src/kudu/tablet/deltamemstore-test.cc
M src/kudu/tablet/deltamemstore.cc
M src/kudu/tablet/deltamemstore.h
M src/kudu/tablet/mvcc.cc
M src/kudu/tablet/mvcc.h
M src/kudu/tablet/tablet-test-util.h
11 files changed, 417 insertions(+), 393 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/11395/2
--
To view, visit http://gerrit.cloudera.org:8080/11395
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
Gerrit-Change-Number: 11395
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
