(spark) branch master updated: [SPARK-46292][CORE][UI] Show a summary of workers in MasterPage

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new e2441c41de4 [SPARK-46292][CORE][UI] Show a summary of workers in 
MasterPage
e2441c41de4 is described below

commit e2441c41de476b09542db60836d7d853d47f6158
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Dec 6 17:49:37 2023 -0800

    [SPARK-46292][CORE][UI] Show a summary of workers in MasterPage
    
    ### What changes were proposed in this pull request?
    
    This PR aims to show a summary of workers in MasterPage.
    
    ### Why are the changes needed?
    
    Although `Alive Workers` is a useful information, it's insufficient to 
analyze the whole cluster status because we don't know how many workers are in 
other status. Especially, this is useful during the recovery process of Spark 
Master HA setting.
    
    In short, this helps the users identify the issues intuitively.
    
    ```
    - Alive Workers: 1
    + Workers: 1 Alive, 1 Dead, 0 Decommissioned, 0 Unknown
    ```
    
    Here is a screenshot.
    
    ![Screenshot 2023-12-06 at 3 13 43 
PM](https://github.com/apache/spark/assets/9700541/f078b6ae-ab22-4721-8c67-661121bb9807)
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Manual test.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #44218 from dongjoon-hyun/SPARK-46292.
    
    Authored-by: Dongjoon Hyun <dh...@apple.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala   | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git 
a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala 
b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
index b2f35984d37..f25e3495d79 100644
--- a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
@@ -144,7 +144,11 @@ private[ui] class MasterPage(parent: MasterWebUI) extends 
WebUIPage("") {
                   </li>
                 }.getOrElse { Seq.empty }
               }
-              <li><strong>Alive Workers:</strong> {aliveWorkers.length}</li>
+              <li><strong>Workers:</strong> {aliveWorkers.length} Alive,
+                {workers.count(_.state == WorkerState.DEAD)} Dead,
+                {workers.count(_.state == WorkerState.DECOMMISSIONED)} 
Decommissioned,
+                {workers.count(_.state == WorkerState.UNKNOWN)} Unknown
+              </li>
               <li><strong>Cores in use:</strong> 
{aliveWorkers.map(_.cores).sum} Total,
                 {aliveWorkers.map(_.coresUsed).sum} Used</li>
               <li><strong>Memory in use:</strong>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org