[ https://issues.apache.org/jira/browse/SPARK-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376458#comment-16376458 ]
guoxiaolongzte commented on SPARK-23492: ---------------------------------------- I tested with the latest environment, but I did not reproduce your problem, I think there is a problem, please email srowen instead of creating a jira. Thanks. [ { "id" : "app-20180226150516-0002", "name" : "Spark shell", "attempts" : [ { "startTime" : "2018-02-26T07:05:15.006GMT", "endTime" : "1969-12-31T23:59:59.999GMT", "lastUpdated" : "2018-02-26T07:05:19.814GMT", "duration" : 0, "sparkUser" : "root", {color:#FF0000}"completed" : false,{color} "appSparkVersion" : "2.4.0-SNAPSHOT", "startTimeEpoch" : 1519628715006, "endTimeEpoch" : -1, "lastUpdatedEpoch" : 1519628719814 } ] }, { "id" : "app-20180212094839-0008", "name" : "SparkSQL::10.43.183.120", "attempts" : [ { "startTime" : "2018-02-12T01:48:38.165GMT", "endTime" : "1969-12-31T23:59:59.999GMT", "lastUpdated" : "2018-02-26T06:58:00.065GMT", "duration" : 0, "sparkUser" : "root", {color:#FF0000}"completed" : false,{color} "appSparkVersion" : "2.4.0-SNAPSHOT", "startTimeEpoch" : 1518400118165, "endTimeEpoch" : -1, "lastUpdatedEpoch" : 1519628280065 } ] > Application shows up as running in history server even when latest attempt > has completed > ---------------------------------------------------------------------------------------- > > Key: SPARK-23492 > URL: https://issues.apache.org/jira/browse/SPARK-23492 > Project: Spark > Issue Type: Bug > Components: Spark Core, YARN > Affects Versions: 2.2.0 > Reporter: Kevin Kim > Priority: Major > > Hi, the Spark history server API "/application?status=running" returns > applications whose latest attempts have completed (see example output below). > Spark code says that an application is considered "running" when any one of > the attempts are not "completed". > [https://github.com/apache/spark/blob/1cc34f3e58c92dd06545727e9d931008a1082bbf/core/src/main/scala/org/apache/spark/status/api/v1/ApplicationListResource.scala#L44] > In my case, attempt 1 is shown as incomplete, but the attempt has finished > and is stamped with an endTime. Is this a bug in Spark history server? > {code:java} > "attempts" : [ { > "attemptId" : "2", > "startTime" : "2018-02-14T23:59:17.785GMT", > "endTime" : "2018-02-15T08:08:28.927GMT", > "lastUpdated" : "2018-02-15T08:08:28.949GMT", > "duration" : 29351142, > "sparkUser" : {omitted}, > "completed" : true, > "startTimeEpoch" : 1518652757785, > "endTimeEpoch" : 1518682108927, > "lastUpdatedEpoch" : 1518682108949 > }, { > "attemptId" : "1", > "startTime" : "2018-02-14T23:53:02.629GMT", > "endTime" : "2018-02-14T23:59:13.426GMT", > "lastUpdated" : "2018-02-23T05:03:45.434GMT", > "duration" : 370797, > "sparkUser" : {omitted}, > "completed" : false, > "startTimeEpoch" : 1518652382629, > "endTimeEpoch" : 1518652753426, > "lastUpdatedEpoch" : 1519362225434 > } ] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org