[ https://issues.apache.org/jira/browse/IMPALA-12493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824585#comment-17824585 ]
ASF subversion and git services commented on IMPALA-12493: ---------------------------------------------------------- Commit 54642b2549c38625c868f09815b61170e568cdb9 in impala's branch refs/heads/master from Michael Smith [ https://gitbox.apache.org/repos/asf?p=impala.git;h=54642b254 ] IMPALA-12878: Report invalid query if finalized Fixes a case introduced by IMPALA-12493 where a query is closed again in the middle of unregistering - which appears to be possible with cancel_query_and_validate_state - returns "Query not yet running" instead of "Invalid or unknown query handle". IMPALA-12493 moved checks for whether a query is inflight before an atomic CompareAndSwap to mark a QueryDriver as finalized. As part of that change, a query that was Finalized and removed from inflight_queries - but not yet removed from query_driver_map_ - could report "Query not yet running". Updates the check for that error to also verify the query has not been finalized; if it has, we let the next conditional handle that case and return "Invalid or unknown query handle". Also ensures cancelled queries are closed in test_web_pages tests so assertions that num_in_flight_queries == 0 at the start of tests are valid when running the whole suite serially. This doesn't come up in automated tests because the cases that care about that assertion are marked execute_serially, while the cases that failed to close a query were not. Testing: adds a test to test_cancellation that ensures we get the expected "Invalid or unknown query handle" when close() is delayed in calling QueryDriver::Finalize. Change-Id: I3bf910f499147a09352f9dcb755037b0d8616dfd Reviewed-on: http://gerrit.cloudera.org:8080/21110 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > Impala Query cancelled while Analyzing or Compiling partially closes but > query remains on Coordinator > ----------------------------------------------------------------------------------------------------- > > Key: IMPALA-12493 > URL: https://issues.apache.org/jira/browse/IMPALA-12493 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 4.0.0 > Reporter: Ezra Zerihun > Assignee: Michael Smith > Priority: Major > Fix For: Impala 4.4.0 > > > Customer checks Coordinator Web UI and on /queries page they either see: > * query stuck in Running state indefinitely > * query stuck in Waiting to be Closed state indefinitely > Trying to close the query with Cancel button on /queries page or on Impala > > Queries page both have no effect and the query does not close from Web UI > /queries. So the queries remains on /queries page indefinitely, unless we > restart the Impala Daemon. > > Looking at Impala Daemon logs we see below. Query begins Analyzing and takes > a few minutes. While this is processing, the query is closed from the Web UI. > This leaves the query in a partially Closed state, where we see "Invalid or > unknown query handle" repeat indefinitely but the query still is trying to > run on Coordinator. > > {quote}I1010 05:04:25.418613 260373 impala-server.cc:1332] > c64d690d72b8db5c:3e8be7e500000000] Registered query > query_id=c64d690d72b8db5c:3e8be7e500000000 > session_id=a0466ab54306c776:cf2f1660dd18149e > I1010 05:04:25.418879 260373 Frontend.java:1623] > c64d690d72b8db5c:3e8be7e500000000] Analyzing query: create table ... > ... > I1010 05:06:28.007423 263772 status.cc:129] Cancelled from Impala's debug web > interface by client at XXX.XX.XXX.248:54879 > @ 0xe582e3 > @ 0x1450bdf > @ 0x176e5ac > @ 0x1770b62 > @ 0x1786bf0 > @ 0x1789317 > @ 0x7f62459a6ea4 > @ 0x7f62425e4b2c > I1010 05:06:28.008111 263772 impala-server.cc:1416] UnregisterQuery(): > query_id=c64d690d72b8db5c:3e8be7e500000000 > I1010 05:06:28.008137 263772 status.cc:129] Query not yet running > @ 0xe582e3 > @ 0x15741f8 > @ 0x15744ee > @ 0x1aef419 > @ 0x1480aae > @ 0x1450c13 > @ 0x176e5ac > @ 0x1770b62 > @ 0x1786bf0 > @ 0x1789317 > @ 0x7f62459a6ea4 > @ 0x7f62425e4b2c > ... > I1010 05:06:30.887735 260373 AnalysisContext.java:468] > c64d690d72b8db5c:3e8be7e500000000] Analysis took 64 ms > I1010 05:06:30.890645 260373 BaseAuthorizationChecker.java:113] > c64d690d72b8db5c:3e8be7e500000000] Authorization check took 2 ms > I1010 05:06:30.890789 260373 Frontend.java:1666] > c64d690d72b8db5c:3e8be7e500000000] Analysis and authorization finished. > I1010 05:06:31.652444 260373 client-request-state.cc:754] > c64d690d72b8db5c:3e8be7e500000000] DDL exec mode=asynchronous > I1010 05:06:31.700479 260373 impala-server.cc:1372] Query > c64d690d72b8db5c:3e8be7e500000000 has idle timeout of 15m > I1010 05:06:31.700511 260373 impala-server.cc:1378] Query > c64d690d72b8db5c:3e8be7e500000000 has execution time limit of 3h > I1010 05:06:31.701305 260373 impala-server.cc:1566] Invalid or unknown query > handle: c64d690d72b8db5c:3e8be7e500000000. > I1010 05:06:31.703812 260373 impala-server.cc:2446] Connection > a0466ab54306c776:cf2f1660dd18149e from client XXX.XX.XXX.220:59592 to server > beeswax-frontend closed. The connection had 1 associated session(s). > I1010 05:06:31.703851 260373 impala-server.cc:1616] Closing session: > a0466ab54306c776:cf2f1660dd18149e > I1010 05:06:31.703873 260373 impala-server.cc:1416] UnregisterQuery(): > query_id=c64d690d72b8db5c:3e8be7e500000000 > I1010 05:06:31.703892 260373 impala-server.cc:1566] Invalid or unknown query > handle: c64d690d72b8db5c:3e8be7e500000000. > E1010 05:06:31.703907 260373 impala-server.cc:1457] Query de-registration for > query_id=\{0} failed: \{1} > I1010 05:06:31.704043 260373 impala-server.cc:1663] Closed session: > a0466ab54306c776:cf2f1660dd18149e > ... > I1010 05:06:44.944639 263786 status.cc:129] Cancelled from Impala's debug web > interface by client at XXX.XX.XXX.248:54883 > @ 0xe582e3 > @ 0x1450bdf > @ 0x176e5ac > @ 0x1770b62 > @ 0x1786bf0 > @ 0x1789317 > @ 0x7f62459a6ea4 > @ 0x7f62425e4b2c > I1010 05:06:44.945287 263786 impala-server.cc:1416] UnregisterQuery(): > query_id=c64d690d72b8db5c:3e8be7e500000000 > I1010 05:06:44.945313 263786 impala-server.cc:1566] Invalid or unknown query > handle: c64d690d72b8db5c:3e8be7e500000000. > ... > I1010 05:14:15.958953 263810 coordinator.cc:533] > c64d690d72b8db5c:3e8be7e500000000] started execution on 757 backends for > query_id=c64d690d72b8db5c:3e8be7e500000000 > I1010 05:14:15.964103 263811 coordinator.cc:883] Coordinator waiting for > backends to finish, 756 remaining. query_id=c64d690d72b8db5c:3e8be7e500000000 > ... > I1010 05:14:20.445523 1088732 impala-server.cc:1579] Invalid or unknown query > handle: c64d690d72b8db5c:3e8be7e500000000. > I1010 05:14:20.445645 1088732 control-service.cc:179] ReportExecStatus(): > Received report for unknown query ID (probably closed or cancelled): > c64d690d72b8db5c:3e8be7e500000000 remote host=XXX.XX.XXX.9:60548 > ... > I1010 05:14:20.446139 1088704 impala-server.cc:1579] Invalid or unknown query > handle: c64d690d72b8db5c:3e8be7e500000000. > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org