[GitHub] [spark] SaurabhChawla100 opened a new pull request #27292: [SPARK-30582][Web UI] Spark UI is not showing Aggregated Metrics by Executor in stage page

[GitBox]

SaurabhChawla100 opened a new pull request #27292: [SPARK-30582][Web UI] Spark 
UI is not showing Aggregated Metrics by Executor in stage page
URL: https://github.com/apache/spark/pull/27292
 
 
   ### What changes were proposed in this pull request?
   
   There are scenarios where Spark History Server is located behind the VPC. So 
whenever api calls hit to get the executor Summary(allexecutors). There can be 
delay in getting the response of executor summary and in mean time 
"stage-page-template.html" is loaded and the response of executor Summary is 
not added to the stage-page-template.html.
   
   As the result of which Aggregated Metrics by Executor in stage page is 
showing blank.
   
   This scenario can be easily found in the cases when there is some 
proxy-server which is responsible for sending the request and response to spark 
History server.
   This can be reproduced in Knox/In-house proxy servers which are used to send 
and receive response to Spark History Server.
   
   Alternative scenario to test this case, Open the spark UI in developer mode 
in browser add some breakpoint in stagepage.js, this will add some delay in 
getting the response and now if we check the spark UI for stage Aggregated 
Metrics by Executor in stage page is showing blank.
   
   So In-order to fix this there is a need to add the change in stagepage.js . 
There is a need to add the api call to get the html 
page(stage-page-template.html) first and after that other api calls to get the 
data that needs to attached in the stagepage (like executor Summary, 
stageExecutorSummaryInfoKeys exc)
   
   
   ### Why are the changes needed?
   Since stage page is useful for debugging purpose, This helps in 
understanding how many task ran on the particular executor and information 
related to shuffle read and write on that executor. 
   
   
   ### Does this PR introduce any user-facing change?
   No
   
   
   ### How was this patch tested?
   Manually tested. Testing this in a reproducible way requires a running 
browser or HTML rendering engine that executes the JavaScript.Open the spark UI 
in developer mode in browser add some breakpoint in stagepage.js, this will add 
some delay in getting the response and now if we check the spark UI for stage 
Aggregated Metrics by Executor in stage page is showing blank.
   
   Before fix 
   
   <img width="1529" alt="Screenshot 2020-01-20 at 3 21 55 PM" 
src="https://user-images.githubusercontent.com/34540906/72716739-bcfd3500-3b98-11ea-8dbe-90a135822f92.png";>
   
   After fix 
   
   <img width="1540" alt="Screenshot 2020-01-20 at 3 23 12 PM" 
src="https://user-images.githubusercontent.com/34540906/72716782-d30af580-3b98-11ea-8764-2bde77764604.png";>
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org