[spark] branch master updated: [SPARK-45500][CORE][WEBUI] Show the number of abnormally completed drivers in MasterPage

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 0c1e9a5b19c [SPARK-45500][CORE][WEBUI] Show the number of abnormally 
completed drivers in MasterPage
0c1e9a5b19c is described below

commit 0c1e9a5b19c4cfb77c86f3871998cf2673260b06
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Oct 11 01:46:29 2023 -0700

    [SPARK-45500][CORE][WEBUI] Show the number of abnormally completed drivers 
in MasterPage
    
    ### What changes were proposed in this pull request?
    
    This PR aims to show the number of abnormaly completed drivers in 
MasterPage.
    
    ### Why are the changes needed?
    
    In the `Completed Drivers` table, there are various exit states.
    
    <img width="841" alt="Screenshot 2023-10-11 at 12 01 21 AM" 
src="https://github.com/apache/spark/assets/9700541/ff0b33f5-c546-42e7-870c-8323e2eefded";>
    
    We had better show the abnormally completed drivers in the top of the page.
    
    **BEFORE**
    ```
    Drivers: 0 Running (0 Waiting), 7 Completed
    ```
    
    **AFTER**
    ```
    Drivers: 0 Running (0 Waiting), 7 Completed (1 Killed, 4 Failed, 0 Error)
    ```
    
    <img width="676" alt="Screenshot 2023-10-11 at 12 00 03 AM" 
src="https://github.com/apache/spark/assets/9700541/94deab1f-b9f7-4e5b-8284-aaac4f7520df";>
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, this is a new UI field. However, since this is UI, there will be no 
technical issues.
    
    ### How was this patch tested?
    
    Manual build Spark and check UI.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #43328 from dongjoon-hyun/SPARK-45500.
    
    Authored-by: Dongjoon Hyun <dh...@apple.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala   | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git 
a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala 
b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
index cc4370ad02e..5c1887be5b8 100644
--- a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
@@ -153,7 +153,11 @@ private[ui] class MasterPage(parent: MasterWebUI) extends 
WebUIPage("") {
               <li><strong>Drivers:</strong>
                 {state.activeDrivers.length} Running
                 ({state.activeDrivers.count(_.state == DriverState.SUBMITTED)} 
Waiting),
-                {state.completedDrivers.length} Completed </li>
+                {state.completedDrivers.length} Completed
+                ({state.completedDrivers.count(_.state == DriverState.KILLED)} 
Killed,
+                {state.completedDrivers.count(_.state == DriverState.FAILED)} 
Failed,
+                {state.completedDrivers.count(_.state == DriverState.ERROR)} 
Error)
+              </li>
               <li><strong>Status:</strong> {state.status}</li>
             </ul>
           </div>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org