Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/2903
to look at the new patch set (#2).
Change subject: WIP: flatbuffers-based codegen
......................................................................
WIP: flatbuffers-based codegen
This switches to using flatbuffers as a mechanism to pass structured
constants into the precompiled code. The idea here is as follows;
- any "runtime constants" get described in a flatbuffer
- at codegen time, we construct a flatbuffer with all of that information
- we serialize the flat buffer into a 'const char[]' global variable
inside the codegen module, and pass a pointer to it into the
precompiled code
- the precompiled code 'deserializes' it, but since the flatbuffer generated
code is all inlined in headers, clang is smart enough to see that it's all
constant accesses and thus propagate the constants, unroll loops, etc.
The net here is that it's much easier to write the code generation, since almost
no LLVM code has to be written.
Somewhat suprisingly, this seems to reduce the number of instructions
substantially for some projections, and only increases it by one instruction
for a few cases. I ran codegen-test -codegen_print_mc and grepped out the
number of instructions for each test case:
Before patch After patch
----------------------------
8 8
281 254
8 8
8 8
57 57
57 57
222 210
222 210
62 63
62 63
206 187
206 187
281 254
281 254
102 109
53 54
390 344
330 300
107 108
My guess is that this is due to better aliasing analysis when the whole thing
is inside a single function, but who knows.
WIP:
- the test coverage doesn't seem good -- I had a bug where I wasn't copying
to arenas and nothing failed, and also had a bug where I wasn't handling
defaults at all and nothing failed.
- the FindFlatbuffers.cmake file has a lot of copy-paste from the protobuf
one that could probably be edited
Change-Id: I583ef1edad379bb13b34362f121263de28c8334c
---
M CMakeLists.txt
A cmake_modules/FindFlatbuffers.cmake
M src/kudu/codegen/CMakeLists.txt
A src/kudu/codegen/codegen_params.fbs
M src/kudu/codegen/module_builder.cc
M src/kudu/codegen/module_builder.h
M src/kudu/codegen/precompiled.cc
M src/kudu/codegen/row_projector.cc
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
12 files changed, 450 insertions(+), 201 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/03/2903/2
--
To view, visit http://gerrit.cloudera.org:8080/2903
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I583ef1edad379bb13b34362f121263de28c8334c
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
