Hello Andrew Sherman, Lars Volker, Tim Armstrong, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13740 to look at the new patch set (#2). Change subject: [WIP] IMPALA-7486: Prototype for admitting less mem on coord ...................................................................... [WIP] IMPALA-7486: Prototype for admitting less mem on coord Complete: - Calculate the mem needed for the coord fragment + accounting for the runtime filter agg mem required + a safety buffer of 100MB - changes to get the mem_to_admit/limit for coord, accesible through the QuerySchedule - add/remove/modify checks in both reject and admit methods to seperate the coord and executor checks - Update memory accounting in admission controller - use coord's mem limit for the coord queryState - added BE unit tests for most cases - Updated the Admission debug page to show the coord mem values TODO: - add e2e tests: - Simple case (done) - query with num_node = 1 - Any additional test cases not covered by BE unit tests or need an equivalent e2e test - e2e tests are not written with dedicated coord, so all should work fine Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 --- M be/src/runtime/coordinator.cc M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/service/client-request-state.cc M be/src/service/impala-http-handler.cc M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/main/java/org/apache/impala/planner/Planner.java M tests/custom_cluster/test_admission_controller.py M www/admission_controller.tmpl 14 files changed, 516 insertions(+), 93 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/13740/2 -- To view, visit http://gerrit.cloudera.org:8080/13740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2b94e7293b91dec8a18491079c34923eadd94b21 Gerrit-Change-Number: 13740 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>