Hello Mike Percy, Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/11395
to review the following change.
Change subject: KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
......................................................................
KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
This patch rewrites much of DeltaFileIterator to leverage DeltaPreparer for
delta preparation and service. Seeing as DeltaPreparer was originally built
for DMSIterator, here are the various augmentations that were necessary to
support DeltaFileIterator:
- REINSERT support, which meant more complicated deletion state tracking.
- FilterColumnIdsAndCollectDeltas support, cribbed from DeltaFileIterator.
- A templatized traits system to control which features were enabled. This
also meant templatizing both DeltaPreparer and DeltaFileIterator.
- Early out from the "apply all deltas for a row" loop when the timestamp
is no longer relevant. I modified DMSIterator to take advantage of this,
which should result in a performance improvement.
I tried to centralize as much state tracking in DeltaPreparer, though there
were several aspects of this that were confusing (namely prepared_idx_,
last_added_idx_, and prepared_count_).
The patch's improvements should be most noticeable on wide schemas where the
column-by-column ApplyUpdates() approach yielded a lot of unnecessary delta
I/O and decoding. I don't have a suitable microbenchmark to prove this, but
I did run diskrowset-test's TestDeltaApplicationPerformance under perf and
the resulting flame graphs showed the bulk of the iteration time as having
moved from ApplyUpdates() to PrepareBatch().
Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
---
M src/kudu/tablet/delta_key.h
M src/kudu/tablet/delta_store.cc
M src/kudu/tablet/delta_store.h
M src/kudu/tablet/deltafile.cc
M src/kudu/tablet/deltafile.h
M src/kudu/tablet/deltamemstore-test.cc
M src/kudu/tablet/deltamemstore.cc
M src/kudu/tablet/deltamemstore.h
M src/kudu/tablet/mvcc.cc
M src/kudu/tablet/mvcc.h
M src/kudu/tablet/tablet-test-util.h
11 files changed, 417 insertions(+), 393 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/11395/1
--
To view, visit http://gerrit.cloudera.org:8080/11395
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
Gerrit-Change-Number: 11395
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
