Dan Hecht has posted comments on this change. Change subject: IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention ......................................................................
Patch Set 6: (4 comments) http://gerrit.cloudera.org:8080/#/c/6707/6/be/src/service/impala-server.cc File be/src/service/impala-server.cc: Line 192: DEFINE_int64(metadata_loading_pause_injection_ms, 0, "Simulates metadata loading for a " how about prefixing the flag with "stress_", like we do for the one in free-pool.h, to give more of a clue that this is just for stressing the system. PS6, Line 822: // Note: this acquires the exec_state lock *before* the : // query_exec_state_map_ lock. This is the opposite of : // GetQueryExecState(..., true), and therefore looks like a : // candidate for deadlock. The reason this works here is that : // GetQueryExecState cannot find exec_state (under the exec state : // map lock) and take it's lock until RegisterQuery has : // finished. By that point, the exec state map lock will have been : // given up, so the classic deadlock interleaving is not possible. this comment is outdated now. http://gerrit.cloudera.org:8080/#/c/6707/6/be/src/service/impala-server.h File be/src/service/impala-server.h: PS6, Line 85: /// 4. query_exec_state_map_lock_ i think this should be updated. with this change, do we ever hold this lock in conjunction with another lock? http://gerrit.cloudera.org:8080/#/c/6707/6/tests/custom_cluster/test_query_concurrency.py File tests/custom_cluster/test_query_concurrency.py: PS6, Line 55: GetRuntimeProfileStr This doesn't make sense. GetRuntimeProfileStr() doesn't hold the map lock while getting the QES::lock_. I think what's happening is that the call on line 76 to get_in_flight_queries() is hitting the QuerySummaryHandler() handler (please double check that), which then causes the lock cycle. Without the fix, does the test ever get past line 76? I would expect the webserver to block on the QES::lock_ at that point. -- To view, visit http://gerrit.cloudera.org:8080/6707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Henry Robinson <he...@cloudera.com> Gerrit-HasComments: Yes
