Bharath Vissapragada has uploaded a new patch set (#6). Change subject: IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention ......................................................................
IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention Holding query_exec_state_map_lock_ and QES::lock_ together in certain paths could potentially block the impalad from registering new queries. The most common occurrence of this is while loading the webpage of a query while the query planning is still in progress. Since we hold the QES::lock_ during planning, it blocks the web page from loading which inturn blocks incoming queries by holding query_exec_state_map_lock_. While the actual fix for this is to not hold the QES::lock_ during metadata load, this patch doesn't do it since it could have some unknown side effects. Instead, - We make sure that QES::lock_ doesn't block query_exec_state_map_lock_ in hot paths. - As an exception, we don't grab QES::lock_ while the query planning is in progress. Also, this patch removes locking inside the GetQueryExecState() method. It is confusing that some callers choose to lock it inside the method and wait on it later. Now, the locking is the responsibility of the callers. Testing: Tested it locally by adding a long sleep in JniFrontend.createExecRequest() and still was able to refresh the web UI and run parallel queries. Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h A tests/custom_cluster/test_query_concurrency.py M www/query_plan.tmpl 7 files changed, 144 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/6707/6 -- To view, visit http://gerrit.cloudera.org:8080/6707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
