This is an automated email from the ASF dual-hosted git repository. tgraves pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push: new 8c077b0 [SPARK-31621][CORE] Fixing Spark Master UI Issue when application is waiting for workers to launch driver 8c077b0 is described below commit 8c077b0a3e27e86ce3dbbf491521cd15b8f9fe79 Author: Akshat Bordia <akshat.bordi...@gmail.com> AuthorDate: Tue May 5 08:58:37 2020 -0500 [SPARK-31621][CORE] Fixing Spark Master UI Issue when application is waiting for workers to launch driver ### What changes were proposed in this pull request? Fixing an issue where Spark Master UI Fails to load if the application is waiting for workers to launch driver. **Root Cause:** This is happening due to the fact that the submitted application is waiting for a worker to be free to run the driver. Due to this resource is set to null in the formatResourcesAddresses method and this is running into null pointer exception. ![image](https://user-images.githubusercontent.com/31816865/80801557-77ee9300-8bca-11ea-92b7-b8df58b68de3.png) **Fix:** Added a null check before forming a resource address and display "None" if the driver isn't launched yet. ### Why are the changes needed? Spark Master UI should load as expected when applications are waiting for workers to run driver. ### Does this PR introduce _any_ user-facing change? The worker column in Spark Master UI will show "None" if the driver hasn't been launched yet. ![image](https://user-images.githubusercontent.com/31816865/80801671-be43f200-8bca-11ea-86c3-381925f82cc7.png) ### How was this patch tested? Tested on a local setup. Launched 2 applications and ensured that Spark Master UI loads fine. ![image](https://user-images.githubusercontent.com/31816865/80801883-5b9f2600-8bcb-11ea-8a1a-cc597aabc4c2.png) Closes #28429 from akshatb1/MasterUIBug. Authored-by: Akshat Bordia <akshat.bordi...@gmail.com> Signed-off-by: Thomas Graves <tgra...@apache.org> (cherry picked from commit c71198ab6c8c9ded6a52eb97859b39dc2119b5fd) Signed-off-by: Thomas Graves <tgra...@apache.org> --- core/src/main/scala/org/apache/spark/deploy/master/DriverInfo.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/src/main/scala/org/apache/spark/deploy/master/DriverInfo.scala b/core/src/main/scala/org/apache/spark/deploy/master/DriverInfo.scala index bf68ba8..252e704 100644 --- a/core/src/main/scala/org/apache/spark/deploy/master/DriverInfo.scala +++ b/core/src/main/scala/org/apache/spark/deploy/master/DriverInfo.scala @@ -37,7 +37,7 @@ private[deploy] class DriverInfo( @transient var worker: Option[WorkerInfo] = None // resources(e.f. gpu/fpga) allocated to this driver // map from resource name to ResourceInformation - private var _resources: Map[String, ResourceInformation] = _ + private var _resources: Map[String, ResourceInformation] = Map.empty init() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org