[
https://issues.apache.org/jira/browse/HIVE-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977175#comment-15977175
]
Vihang Karajgaonkar edited comment on HIVE-16451 at 4/20/17 6:13 PM:
---------------------------------------------------------------------
Thanks for finding this out [~pvary]. Although I didn't quite get how the patch
fixes the race condition. The way I understand the issue is that there is a
Logging thread and the thread executing the HiveStatement. Both these threads
are accessing isLogBeingGenerated, isCancelled, isQueryClosed flags in the same
HiveStatement object. None of these getters and setters are thread safe. I
think there could be more undiscovered race-conditions in this execution path.
was (Author: vihangk1):
Thanks for finding this out [~pvary]. Although I didn't quite get how the patch
fixes the race condition. The way I understand the issue is you have Log thread
and the thread executing the HiveStatement. Both these threads are accessible
isLogBeingGenerated, isCancelled, isQueryClosed flags in HiveStatement object.
None of which are thread safe. I think there could be more undiscovered
race-conditions in this execution path.
> Race condition between HiveStatement.getQueryLog and
> HiveStatement.runAsyncOnServer
> -----------------------------------------------------------------------------------
>
> Key: HIVE-16451
> URL: https://issues.apache.org/jira/browse/HIVE-16451
> Project: Hive
> Issue Type: Bug
> Components: Beeline, JDBC
> Affects Versions: 3.0.0
> Reporter: Peter Vary
> Assignee: Peter Vary
> Attachments: HIVE-16451.02.patch, HIVE-16451.03.patch,
> HIVE-16451.patch
>
>
> During the BeeLineDriver testing I have met the following race condition:
> - Run the query asynchronously through BeeLine
> - Querying the logs in the BeeLine
> In the following code:
> {code:title=HiveStatement.runAsyncOnServer}
> private void runAsyncOnServer(String sql) throws SQLException {
> checkConnection("execute");
> closeClientOperation();
> initFlags();
> [..]
> }
> {code}
> {code:title=HiveStatement.getQueryLog}
> public List<String> getQueryLog(boolean incremental, int fetchSize)
> throws SQLException, ClosedOrCancelledStatementException {
> [..]
> try {
> if (stmtHandle != null) {
> [..]
> } else {
> if (isQueryClosed) {
> throw new ClosedOrCancelledStatementException("Method getQueryLog()
> failed. The " +
> "statement has been closed or cancelled.");
> } else {
> return logs;
> }
> }
> } catch (SQLException e) {
> [..]
> }
> [..]
> }
> {code}
> The runAsyncOnServer {{closeClientOperation}} sets {{isQueryClosed}} flag to
> true:
> {code:title=HiveStatement.closeClientOperation}
> void closeClientOperation() throws SQLException {
> [..]
> isQueryClosed = true;
> isExecuteStatementFailed = false;
> stmtHandle = null;
> }
> {code}
> The {{initFlags}} sets it to false:
> {code}
> private void initFlags() {
> isCancelled = false;
> isQueryClosed = false;
> isLogBeingGenerated = true;
> isExecuteStatementFailed = false;
> isOperationComplete = false;
> }
> {code}
> If the {{getQueryLog}} is called after the {{closeClientOperation}}, but
> before the {{initFlags}}, then we will have a following warning if verbose
> mode is set to true in BeeLine:
> {code}
> Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method
> getQueryLog() failed. The statement has been closed or cancelled.
> (state=,code=0)
> {code}
> This caused this fail:
> https://builds.apache.org/job/PreCommit-HIVE-Build/4691/testReport/org.apache.hadoop.hive.cli/TestBeeLineDriver/testCliDriver_smb_mapjoin_11_/
> {code}
> Error Message
> Client result comparison failed with error code = 1 while executing
> fname=smb_mapjoin_11
> 16a17
> > Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method
> > getQueryLog() failed. The statement has been closed or cancelled.
> > (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
