Hello Alexey Serbin, Andrew Wong, Kudu Jenkins, Andrew Wong, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15660
to look at the new patch set (#4).
Change subject: wire_protocol: change columnar serialization of varlen data to
match Arrow
......................................................................
wire_protocol: change columnar serialization of varlen data to match Arrow
This changes the format of variable-length columns serialized on the
wire to match Apache Arrow instead of our internal column format. The
Arrow format consists of an array of n+1 offsets for n rows, such that
the data for cell 'n' spans offset offsets[n]...offsets[n+1].
The obvious advantage here is that clients can zero-copy into Arrow
structures since the format is compatible. The less obvious advantage is
that we are going from 16 bytes (sizeof(Slice)) to 4 bytes
(sizeof(uint32_t offset)) for each serialized string, so this should be
a savings even for non-arrow users of the API.
This patch also adds some more sanity checking of the wire format in the
client API so that it's not the responsibility of the caller to guard
against malicious servers.
Change-Id: Iadf728744feb83f5980e62bea4fd7634a1a52467
---
M src/kudu/client/client-test.cc
M src/kudu/client/columnar_scan_batch.cc
M src/kudu/client/columnar_scan_batch.h
M src/kudu/client/scanner-internal.cc
M src/kudu/client/scanner-internal.h
M src/kudu/common/columnar_serialization.cc
M src/kudu/common/columnar_serialization.h
M src/kudu/common/rowblock.h
M src/kudu/common/wire_protocol-test.cc
M src/kudu/common/wire_protocol.proto
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/tserver/tablet_service.cc
12 files changed, 256 insertions(+), 121 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/15660/4
--
To view, visit http://gerrit.cloudera.org:8080/15660
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iadf728744feb83f5980e62bea4fd7634a1a52467
Gerrit-Change-Number: 15660
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <andrew.w...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
