[jira] [Updated] (HIVE-15662) check startTime in SparkTask to make sure startTime is not less than submitTime
[ https://issues.apache.org/jira/browse/HIVE-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HIVE-15662: - Attachment: HIVE-15662.001.patch > check startTime in SparkTask to make sure startTime is not less than > submitTime > --- > > Key: HIVE-15662 > URL: https://issues.apache.org/jira/browse/HIVE-15662 > Project: Hive > Issue Type: Bug >Reporter: zhihai xu >Assignee: zhihai xu >Priority: Minor > Attachments: HIVE-15662.000.patch, HIVE-15662.001.patch > > > Check startTime in SparkTask to make sure startTime is not less than > submitTime. We saw a corner case when the sparkTask is finished in less than > 1 second, the startTime may not be set because RemoteSparkJobMonitor will > sleep for 1 second then check the state, in this case, right after sleep for > one second, the spark job is already completed. one example query with 3 > spark tasks, the second one finished quickly for around 1 second: > {code} > SparkTask1: > "finishTime":1484638391978 > "submitTime":1484638385933 > "startTime": 1484638386973 > SparkTask2: > "finishTime":1484638393019 > "submitTime":1484638391979 > "startTime": 1484638386973 > SparkTask3: > "finishTime":1484638432123 > "submitTime":1484638393020 > "startTime": 1484638394057 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15662) check startTime in SparkTask to make sure startTime is not less than submitTime
[ https://issues.apache.org/jira/browse/HIVE-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HIVE-15662: - Description: Check startTime in SparkTask to make sure startTime is not less than submitTime. We saw a corner case when the sparkTask is finished in less than 1 second, the startTime may not be set because RemoteSparkJobMonitor will sleep for 1 second then check the state, in this case, right after sleep for one second, the spark job is already completed. one example query with 3 spark tasks, the second one finished quickly for around 1 second: {code} SparkTask1: "finishTime":1484638391978 "submitTime":1484638385933 "startTime": 1484638386973 SparkTask2: "finishTime":1484638393019 "submitTime":1484638391979 "startTime": 1484638386973 SparkTask3: "finishTime":1484638432123 "submitTime":1484638393020 "startTime": 1484638394057 {code} was: Check startTime in SparkTask to make sure startTime is not less than submitTime. We saw a corner case when the sparkTask is finished in less than 1 second, the startTime may not be set because RemoteSparkJobMonitor will sleep for 1 second then check the state, in this case, right after sleep for one second, the spark job is already completed. > check startTime in SparkTask to make sure startTime is not less than > submitTime > --- > > Key: HIVE-15662 > URL: https://issues.apache.org/jira/browse/HIVE-15662 > Project: Hive > Issue Type: Bug >Reporter: zhihai xu >Assignee: zhihai xu >Priority: Minor > Attachments: HIVE-15662.000.patch > > > Check startTime in SparkTask to make sure startTime is not less than > submitTime. We saw a corner case when the sparkTask is finished in less than > 1 second, the startTime may not be set because RemoteSparkJobMonitor will > sleep for 1 second then check the state, in this case, right after sleep for > one second, the spark job is already completed. one example query with 3 > spark tasks, the second one finished quickly for around 1 second: > {code} > SparkTask1: > "finishTime":1484638391978 > "submitTime":1484638385933 > "startTime": 1484638386973 > SparkTask2: > "finishTime":1484638393019 > "submitTime":1484638391979 > "startTime": 1484638386973 > SparkTask3: > "finishTime":1484638432123 > "submitTime":1484638393020 > "startTime": 1484638394057 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15662) check startTime in SparkTask to make sure startTime is not less than submitTime
[ https://issues.apache.org/jira/browse/HIVE-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HIVE-15662: - Attachment: HIVE-15662.000.patch > check startTime in SparkTask to make sure startTime is not less than > submitTime > --- > > Key: HIVE-15662 > URL: https://issues.apache.org/jira/browse/HIVE-15662 > Project: Hive > Issue Type: Bug >Reporter: zhihai xu >Assignee: zhihai xu >Priority: Minor > Attachments: HIVE-15662.000.patch > > > Check startTime in SparkTask to make sure startTime is not less than > submitTime. We saw a corner case when the sparkTask is finished in less than > 1 second, the startTime may not be set because RemoteSparkJobMonitor will > sleep for 1 second then check the state, in this case, right after sleep for > one second, the spark job is already completed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15662) check startTime in SparkTask to make sure startTime is not less than submitTime
[ https://issues.apache.org/jira/browse/HIVE-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HIVE-15662: - Status: Patch Available (was: Open) > check startTime in SparkTask to make sure startTime is not less than > submitTime > --- > > Key: HIVE-15662 > URL: https://issues.apache.org/jira/browse/HIVE-15662 > Project: Hive > Issue Type: Bug >Reporter: zhihai xu >Assignee: zhihai xu >Priority: Minor > Attachments: HIVE-15662.000.patch > > > Check startTime in SparkTask to make sure startTime is not less than > submitTime. We saw a corner case when the sparkTask is finished in less than > 1 second, the startTime may not be set because RemoteSparkJobMonitor will > sleep for 1 second then check the state, in this case, right after sleep for > one second, the spark job is already completed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)