[spark] branch branch-3.5 updated: [SPARK-44661][CORE][TESTS] `getMapOutputLocation` should not throw NPE

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.5 by this push:
     new 4939889a301 [SPARK-44661][CORE][TESTS] `getMapOutputLocation` should 
not throw NPE
4939889a301 is described below

commit 4939889a301209c678a2f4df978a9f7e6b15edbd
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Thu Aug 3 14:40:59 2023 -0700

    [SPARK-44661][CORE][TESTS] `getMapOutputLocation` should not throw NPE
    
    ### What changes were proposed in this pull request?
    
    This PR aims to add a test coverage for Apache Spark 4.0/3.5/3.4.
    This PR depends on SPARK-44658 (#42323) but is created separately because 
this aims to land `branch-3.4` too.
    
    ### Why are the changes needed?
    
    To prevent a future regression.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Pass the CIs.
    
    Closes #42326 from dongjoon-hyun/SPARK-44661.
    
    Lead-authored-by: Dongjoon Hyun <dh...@apple.com>
    Co-authored-by: Dongjoon Hyun <dongj...@apache.org>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
    (cherry picked from commit 9fbf0b4853c6209675daa0731f8b33a83b2f5cef)
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 .../scala/org/apache/spark/MapOutputTrackerSuite.scala  | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala 
b/core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
index 7ee36137e27..450ff01921a 100644
--- a/core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
@@ -1092,4 +1092,21 @@ class MapOutputTrackerSuite extends SparkFunSuite with 
LocalSparkContext {
     shuffleStatus.removeMapOutput(mapIndex = 1, bmID)
     assert(shuffleStatus.getMapStatus(0).isEmpty)
   }
+
+  test("SPARK-44661: getMapOutputLocation should not throw NPE") {
+    val rpcEnv = createRpcEnv("test")
+    val tracker = newTrackerMaster()
+    try {
+      tracker.trackerEndpoint = 
rpcEnv.setupEndpoint(MapOutputTracker.ENDPOINT_NAME,
+        new MapOutputTrackerMasterEndpoint(rpcEnv, tracker, conf))
+      tracker.registerShuffle(0, 1, 1)
+      tracker.registerMapOutput(0, 0, MapStatus(BlockManagerId("exec-1", 
"hostA", 1000),
+        Array(2L), 0))
+      tracker.removeOutputsOnHost("hostA")
+      assert(tracker.getMapOutputLocation(0, 0) == None)
+    } finally {
+      tracker.stop()
+      rpcEnv.shutdown()
+    }
+  }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org