Yida Wu has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/19181 )
Change subject: IMPALA-11470: Add Cache For Codegen Functions ...................................................................... IMPALA-11470: Add Cache For Codegen Functions The patch adds supports of the cache for CodeGen functions to improve the performance of sub-second queries. The main idea is to store the codegen functions to a cache, and reuse them when it is appropriate to save the llvm optimization time which could take over hundreds of milliseconds. In this patch, we implement the cache in a fragment-level way, that is to store all the codegen functions of a fragment in a cache entry, if one exactly same fragment comes again, it should be able to find all the codegen functions it needs from the specific cache entry, therefore saving the time. The module bitcode is used as the key to the cache, which will be generated before the module optimization and final compilation. If codegen_cache_debug_mode is set to false, which is by default, we will only store the hash code of the module bitcode as the key to reduce memory consumption. Otherwise we will store the full bitcode string. KrpcDataStreamSenderConfig::CodegenHashRow() is changed to pass the hash seed as an argument because it can't hit the cache for the fragment if using a dynamic hash seed within the codegen function. Codegen cache is disabled if the fragment contains a udf, because it could result in a crash in the udf shared library when calling the udf from a cached codegen. The issue happens in the cenario that the udf is recreated after the database has been recreated. One theory is that the change of the udf shared library during the process could lead to the crash. Disable it until there is a better solution. To disable the cache for udf, we add a logic in the expression analysis, if a fragment contains a function with a non-empty hdfs location, we won't cache any codegen functions of that fragment. The patch also introduces following new flags for start and query options. start option: - codegen_cache_capacity The capacity of the cache, if set to 0, codegen cache is disabled. query option: - disable_codegen_cache Codegen cache will be disabled when it is set to true. - codegen_cache_debug_mode If enabled, more logs and statistics will be allowed, and the full key content will be stored to the cache, otherwise, the cache will only store the hash code of the key to reduce memory consumption. New impalad metrics: impala.codegen-cache.misses impala.codegen-cache.entries-in-use impala.codegen-cache.entries-in-use-bytes impala.codegen-cache.entries-evicted impala.codegen-cache.hits impala.codegen-cache.entry-sizes New profile Metrics: CodegenCacheLookupTime CodegenCacheSaveTime ModuleBitcodeGenTime NumCachedFunctions TPCH-1 performance evaluation (8 iteration) on AWS m5a.4xlarge, the result removes the first iteration to show the benefit of the cache: Query Cached(s) NoCache(s) Delta(Avg) NoCodegen(s) Delta(Avg) TPCH-Q1 0.39 1.02 -61.76% 5.59 -93.02% TPCH-Q2 0.56 1.21 -53.72% 0.47 19.15% TPCH-Q3 0.37 0.77 -51.95% 0.43 -13.95% TPCH-Q4 0.36 0.51 -29.41% 0.33 9.09% TPCH-Q5 0.39 1.1 -64.55% 0.39 0% TPCH-Q6 0.24 0.27 -11.11% 0.77 -68.83% TPCH-Q7 0.39 1.2 -67.5% 0.39 0% TPCH-Q8 0.58 1.46 -60.27% 0.45 28.89% TPCH-Q9 0.8 1.38 -42.03% 1 -20% TPCH-Q10 0.6 1.03 -41.75% 0.85 -29.41% TPCH-Q11 0.3 0.93 -67.74% 0.2 50% TPCH-Q12 0.28 0.48 -41.67% 0.38 -26.32% TPCH-Q13 1.11 1.22 -9.02% 1.16 -4.31% TPCH-Q14 0.55 0.78 -29.49% 0.45 22.22% TPCH-Q15 0.33 0.73 -54.79% 0.44 -25% TPCH-Q16 0.32 0.78 -58.97% 0.41 -21.95% TPCH-Q17 0.56 0.84 -33.33% 0.89 -37.08% TPCH-Q18 0.54 0.92 -41.3% 0.89 -39.33% TPCH-Q19 0.35 2.34 -85.04% 0.35 0% TPCH-Q20 0.34 0.98 -65.31% 0.31 9.68% TPCH-Q21 0.83 1.14 -27.19% 0.86 -3.49% TPCH-Q22 0.26 0.52 -50% 0.25 4% >From the result, it shows a pretty good performance compared to codegen without cache (default setting). However, compared to codegen disabled, as expected, for short queries, codegen cache is not always faster, probably because for the codegen cache, it still needs some time to prepare the codegen functions and generate an appropriate module bitcode to be the key, if the time of the preparation is larger than the benefit from the codegen functions, especially for the extremely short queries, the result can be slower than not using the codegen. There could be room to improve in future. Tests: Ran exhaustive tests. Added E2e testcase TestCodegenCache. Added unit testcase LlvmCodeGenCacheTest. Change-Id: If42c78a7f51fd582e5fe331fead494dadf544eb1 --- M be/src/codegen/CMakeLists.txt A be/src/codegen/llvm-codegen-cache-test.cc A be/src/codegen/llvm-codegen-cache.cc A be/src/codegen/llvm-codegen-cache.h M be/src/codegen/llvm-codegen.cc M be/src/codegen/llvm-codegen.h M be/src/exprs/agg-fn.cc M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/fragment-state.h M be/src/runtime/krpc-data-stream-sender-ir.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/krpc-data-stream-sender.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/test-env.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M common/thrift/metrics.json M tests/common/test_result_verifier.py A tests/custom_cluster/test_codegen_cache.py 25 files changed, 1,404 insertions(+), 55 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/19181/10 -- To view, visit http://gerrit.cloudera.org:8080/19181 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If42c78a7f51fd582e5fe331fead494dadf544eb1 Gerrit-Change-Number: 19181 Gerrit-PatchSet: 10 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qfc...@hotmail.com> Gerrit-Reviewer: Yida Wu <wydbaggio...@gmail.com>
