[ https://issues.apache.org/jira/browse/HIVE-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506844#comment-16506844 ]
Hive QA commented on HIVE-18684: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12926962/HIVE-18684.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14514 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11646/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11646/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11646/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12926962 - PreCommit-HIVE-Build > Race condition in RemoteSparkJobMonitor > --------------------------------------- > > Key: HIVE-18684 > URL: https://issues.apache.org/jira/browse/HIVE-18684 > Project: Hive > Issue Type: Sub-task > Components: Spark > Reporter: Sahil Takiar > Assignee: Sahil Takiar > Priority: Major > Attachments: HIVE-18684.1.patch, HIVE-18684.2.patch, > HIVE-18684.3.patch > > > There is a race condition in {{RemoteSparkJobMonitor}}. Sometimes the info in > {{RemoteSparkJobMonitor#startMonitor.STARTED}} gets printed out, sometimes it > doesn't. This can be easily verified by running a qtest on > {{TestMiniSparkOnYarnCliDriver}} and counting the number of times {{Query > Hive on Spark job}} is printed vs. the number of times {{Finished > successfully in}} gets printed. > The issue is that {{RemoteSparkJobMonitor}} runs every one second, and checks > the state of {{JobHandle}}. Depending on the state, it prints out some > logging info. The content of the logs contain an implicit assumption that > logs in the {{STARTED}} state are printed before the logs in the > {{SUCCEEDED}} state. However, this isn't always the case. The state > transitions are driven by how long the remote Spark job takes to run, and it > it finishes within one second then the logs in the {{STARTED}} state never > printed. > This can be confusing to users, and there is key debugging information that > is printed in the {{STARTED}} state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)