Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11306 )
Change subject: IMPALA-7353: Improve memory estimates for Hbase Scan Nodes ...................................................................... IMPALA-7353: Improve memory estimates for Hbase Scan Nodes Currently for hbase scan nodes we use a constant estimate of 1GB which is generally a gross over-estimation. This patch improves upon those estimates by using huerestics based on how hbase rows are stored and fetched and how the scanners interact with the internal memory pool. Testing: Added/Modified resource requirements planner test. Added a junit test for the estimation logic. Change-Id: I583545c3f5e454854f111871c5fbc4f108ae4bff Reviewed-on: http://gerrit.cloudera.org:8080/11306 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exec/hbase-table-scanner.cc M be/src/runtime/mem-pool.cc M be/src/runtime/mem-pool.h M fe/src/main/java/org/apache/impala/catalog/FeHBaseTable.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test 8 files changed, 472 insertions(+), 266 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11306 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I583545c3f5e454854f111871c5fbc4f108ae4bff Gerrit-Change-Number: 11306 Gerrit-PatchSet: 5 Gerrit-Owner: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
