Todd Lipcon has uploaded a new change for review. http://gerrit.cloudera.org:8080/2903
Change subject: WIP: flatbuffers-based codegen ...................................................................... WIP: flatbuffers-based codegen This switches to using flatbuffers as a mechanism to pass structured constants into the precompiled code. The idea here is as follows; - any "runtime constants" get described in a flatbuffer - at codegen time, we construct a flatbuffer with all of that information - we serialize the flat buffer into a 'const char[]' global variable inside the codegen module, and pass a pointer to it into the precompiled code - the precompiled code 'deserializes' it, but since the flatbuffer generated code is all inlined in headers, clang is smart enough to see that it's all constant accesses and thus propagate the constants, unroll loops, etc. The net here is that it's much easier to write the code generation, since almost no LLVM code has to be written. Somewhat suprisingly, this seems to reduce the number of instructions substantially for some projections, and only increases it by one instruction for a few cases. I ran codegen-test -codegen_print_mc and grepped out the number of instructions for each test case: Before patch After patch ---------------------------- 8 8 281 254 8 8 8 8 57 57 57 57 222 210 222 210 62 63 62 63 206 187 206 187 281 254 281 254 102 109 53 54 390 344 330 300 107 108 My guess is that this is due to better aliasing analysis when the whole thing is inside a single function, but who knows. WIP: - the test coverage doesn't seem good -- I had a bug where I wasn't copying to arenas and nothing failed, and also had a bug where I wasn't handling defaults at all and nothing failed. - the FindFlatbuffers.cmake file has a lot of copy-paste from the protobuf one that could probably be edited Change-Id: I583ef1edad379bb13b34362f121263de28c8334c --- M CMakeLists.txt A cmake_modules/FindFlatbuffers.cmake M src/kudu/codegen/CMakeLists.txt A src/kudu/codegen/codegen_params.fbs M src/kudu/codegen/module_builder.cc M src/kudu/codegen/module_builder.h M src/kudu/codegen/precompiled.cc M src/kudu/codegen/row_projector.cc M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 12 files changed, 450 insertions(+), 201 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/03/2903/1 -- To view, visit http://gerrit.cloudera.org:8080/2903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I583ef1edad379bb13b34362f121263de28c8334c Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <[email protected]>
