Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/17955 )
Change subject: IMPALA-10967 Load data should handle AWS NLB-type timeout
......................................................................
IMPALA-10967 Load data should handle AWS NLB-type timeout
This patch addresses Impala client hang due to AWS network load balancer
timeout which is fixed at 350s. When some long data loading operations
are executing and the timeout happens, AWS silently drops the connection
and the Impala client enters the hang state.
The fix maintains the current TCLIService protocol between the client
and Impala server and utilizes a separate thread to run the data loading
and metadata refresh operation. Since this thread is waited for in a
wait thread which runs asynchronously, the execution of the entire
operation will not cause a wait on the Impala client. The Impala client
can check the status of the operation via repeated GetOperationStatus()
call.
External behavior change:
1. A new query option 'enable_async_load_data_execution', default to
true, is added. It can be set to false to turn off the patch.
Testing:
1. Added a new test in test_load.py to verify that the asynchronous
execution in BE keeps the session live for hs2, hs2-http and
beeswax three clients;
2. Ran core tests successfully.
Change-Id: I8c2437e9894510204303ec07710cad60102c8821
Reviewed-on: http://gerrit.cloudera.org:8080/17955
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/metadata/test_load.py
7 files changed, 192 insertions(+), 31 deletions(-)
Approvals:
Joe McDonnell: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/17955
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8c2437e9894510204303ec07710cad60102c8821
Gerrit-Change-Number: 17955
Gerrit-PatchSet: 10
Gerrit-Owner: Qifan Chen <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Qifan Chen <[email protected]>
