Yida Wu has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/19181 )
Change subject: IMPALA-11470: Add Cache For Codegen Functions ...................................................................... IMPALA-11470: Add Cache For Codegen Functions The patch adds supports of the cache for CodeGen functions to improve the performance of sub-second queries. The main idea is to store the codegen functions to a cache, and reuse them when it is appropriate to save the llvm optimization time which could take over hundreds of milliseconds. In this patch, we implement the cache in a fragment-level way, that is to store all the codegen functions of a fragment in a cache entry, if one exactly same fragment comes again, it should be able to find all the codegen functions it needs from the specific cache entry, therefore saving the time. The module bitcode is used as the key to the cache, which will be generated before the module optimization and final compilation. If codegen_cache_debug_mode is set to false, which is by default, we will only store the hash code of the module bitcode as the key to reduce memory consumption. Otherwise we will store the full bitcode string. KrpcDataStreamSenderConfig hash seed is changed to a constant if using codegen cache because it can't hit the cache for the fragment if using a dynamic hash seed. As a limitation, we don't cache the udf functions, because it may rely on the shared library in runtime, if something has been changed to the loading of the shared library, for example recreating a database containing the udf, the cache could become not reusable and result in a crash if reusing it. To disable the cache for udf, we add a logic in the expression analysis, if a fragment contains a function with a non-empty hdfs location, we won't cache any codegen functions of that fragment. Tpch performance test showed no significant difference after adding this logic. The patch also introduces following new flags for start and query options. start option: - codegen_cache_capacity The capacity of the cache, if set to 0, codegen cache is disabled. query option: - disable_codegen_cache Codegen cache will be disabled is set to false. - codegen_cache_debug_mode If enabled, more logs and statistics will be allowed, and the full key content will be stored to the cache, otherwise, the cache will only store the hash code of the key to reduce memory consumption. New impalad metrics: impala.codegen-cache.misses impala.codegen-cache.entries-in-use impala.codegen-cache.entries-in-use-bytes impala.codegen-cache.entries-evicted impala.codegen-cache.hits impala.codegen-cache.entry-sizes New profile Metrics: CodegenCacheLookupTime CodegenCacheSaveTime ModuleBitcodeGenTime NumCachedFunctions TPCH-1 performance evaluation(8 iteration) on AWS m5a.4xlarge: Query Cached(s) NoCache(s) Delta(Avg) NoCodegen(s) Delta(Avg) TPCH-Q1 0.385 1.11 -65.32% 5.35 -92.8% TPCH-Q2 0.56 1.25 -55.2% 0.44 27.27% TPCH-Q3 0.37 0.79 -53.16% 0.43 -13.95% TPCH-Q4 0.355 0.48 -26.04% 0.33 7.58% TPCH-Q5 0.39 1.18 -66.95% 0.39 0% TPCH-Q6 0.24 0.28 -14.29% 0.78 -69.23% TPCH-Q7 0.385 1.27 -69.69% 0.37 4.05% TPCH-Q8 0.575 1.5 -61.67% 0.42 36.9% TPCH-Q9 0.795 1.5 -47.35% 1.04 -23.56% TPCH-Q10 0.595 1.09 -45.41% 0.84 -29.17% TPCH-Q11 0.295 1.01 -70.79% 0.19 55.26% TPCH-Q12 0.275 0.53 -48.11% 0.38 -27.63% TPCH-Q13 1.11 1.2 -7.5% 1.11 0% TPCH-Q14 0.55 0.84 -34.52% 0.42 30.95% TPCH-Q15 0.325 0.76 -57.24% 0.44 -26.14% TPCH-Q16 0.32 0.8 -60% 0.39 -17.95% TPCH-Q17 0.555 0.9 -38.33% 0.88 -36.93% TPCH-Q18 0.535 0.94 -43.09% 0.85 -37.06% TPCH-Q19 0.78 2.48 -68.55% 0.35 122.86% TPCH-Q20 0.335 1.09 -69.27% 0.29 15.52% TPCH-Q21 0.825 1.14 -27.63% 0.89 -7.3% TPCH-Q22 0.255 0.51 -50% 0.26 -1.92% It shows good results compared to current codegen without cache, however, as expected, for short queries, compared to codegen disabled, it is not always faster, there could be room to improve. Tests: Ran core tests. E2e testcase TestCodegenCache is added. Change-Id: If42c78a7f51fd582e5fe331fead494dadf544eb1 --- M be/src/codegen/CMakeLists.txt A be/src/codegen/llvm-codegen-cache.cc A be/src/codegen/llvm-codegen-cache.h M be/src/codegen/llvm-codegen.cc M be/src/codegen/llvm-codegen.h M be/src/exec/exec-node.cc M be/src/exprs/agg-fn.cc M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-state.cc M be/src/runtime/fragment-state.h M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M common/thrift/metrics.json M tests/common/test_result_verifier.py A tests/custom_cluster/test_codegen_cache.py 23 files changed, 824 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/19181/2 -- To view, visit http://gerrit.cloudera.org:8080/19181 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If42c78a7f51fd582e5fe331fead494dadf544eb1 Gerrit-Change-Number: 19181 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
