[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008552#comment-17008552 ] daile commented on SPARK-29596: --- [~hyukjin.kwon] task detail list use task.taskMetrics info , but task.taskMetrics will only be updated when task finshed , Is it feasible to get task Duration in its running state ? https://github.com/apache/spark/pull/27026 > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008536#comment-17008536 ] Hyukjin Kwon commented on SPARK-29596: -- [~726575...@qq.com] have you made some progresses on this? > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004261#comment-17004261 ] daile commented on SPARK-29596: --- [~hyukjin.kwon] I checked the problem and reproduced in 2.4.4 version and will raise PR soon > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968281#comment-16968281 ] Rakesh Raushan commented on SPARK-29596: Will raise PR in 2-3 days > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967467#comment-16967467 ] Hyukjin Kwon commented on SPARK-29596: -- [~rakson] have you made some progresses on this? > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959438#comment-16959438 ] Shivu Sondur commented on SPARK-29596: -- i am checking this issue > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-29596) Task duration not updating for running tasks
[ https://issues.apache.org/jira/browse/SPARK-29596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959436#comment-16959436 ] Rakesh Raushan commented on SPARK-29596: I will look into this one. > Task duration not updating for running tasks > > > Key: SPARK-29596 > URL: https://issues.apache.org/jira/browse/SPARK-29596 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.4.2 >Reporter: Bharati Jadhav >Priority: Major > Attachments: Screenshot_Spark_live_WebUI.png > > > When looking at the task metrics for running tasks in the task table for the > related stage, the duration column is not updated until the task has > succeeded. The duration values are reported empty or 0 ms until the task has > completed. This is a change in behavior, from earlier versions, when the task > duration was continuously updated while the task was running. The missing > duration values can be observed for both short and long running tasks and for > multiple applications. > > To reproduce this, one can run any code from the spark-shell and observe the > missing duration values for any running task. Only when the task succeeds is > the duration value populated in the UI. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
