Hello Andrew Wong, Grant Henke, I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/15660 to review the following change. Change subject: wire_protocol: change columnar serialization of varlen data to match Arrow ...................................................................... wire_protocol: change columnar serialization of varlen data to match Arrow This changes the format of variable-length columns serialized on the wire to match Apache Arrow instead of our internal column format. The Arrow format consists of an array of n+1 offsets for n rows, such that the data for cell 'n' spans offset offsets[n]...offsets[n+1]. The obvious advantage here is that clients can zero-copy into Arrow structures since the format is compatible. The less obvious advantage is that we are going from 16 bytes (sizeof(Slice)) to 4 bytes (sizeof(uint32_t offset)) for each serialized string, so this should be a savings even for non-arrow users of the API. This patch also adds some more sanity checking of the wire format in the client API so that it's not the responsibility of the caller to guard against malicious servers. Change-Id: Iadf728744feb83f5980e62bea4fd7634a1a52467 --- M src/kudu/client/client-test.cc M src/kudu/client/columnar_scan_batch.cc M src/kudu/client/columnar_scan_batch.h M src/kudu/client/scanner-internal.cc M src/kudu/client/scanner-internal.h M src/kudu/common/columnar_serialization.cc M src/kudu/common/columnar_serialization.h M src/kudu/common/rowblock.h M src/kudu/common/wire_protocol-test.cc M src/kudu/common/wire_protocol.proto M src/kudu/tserver/tablet_service.cc 11 files changed, 234 insertions(+), 112 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/15660/1 -- To view, visit http://gerrit.cloudera.org:8080/15660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iadf728744feb83f5980e62bea4fd7634a1a52467 Gerrit-Change-Number: 15660 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Andrew Wong <andrew.w...@cloudera.com> Gerrit-Reviewer: Grant Henke <granthe...@apache.org>