[
https://issues.apache.org/jira/browse/SPARK-48034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kent Yao resolved SPARK-48034.
------------------------------
Fix Version/s: 3.4.4
3.5.2
4.0.0
Resolution: Fixed
Issue resolved by pull request 46270
[https://github.com/apache/spark/pull/46270]
> NullPointerException in MapStatusesSerDeserBenchmark
> ----------------------------------------------------
>
> Key: SPARK-48034
> URL: https://issues.apache.org/jira/browse/SPARK-48034
> Project: Spark
> Issue Type: Test
> Components: Tests
> Affects Versions: 4.0.0
> Reporter: Kent Yao
> Assignee: Kent Yao
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.4, 3.5.2, 4.0.0
>
>
> {code:java}
> 4/04/28 09:30:31 ERROR MapOutputTrackerMaster: Error communicating with
> MapOutputTracker
> 3197java.lang.NullPointerException: Cannot invoke
> "org.apache.spark.rpc.RpcEndpointRef.askSync(Object, scala.reflect.ClassTag)"
> because the return value of
> "org.apache.spark.MapOutputTracker.trackerEndpoint()" is null
> 3198 at
> org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:541)
> 3199 at
> org.apache.spark.MapOutputTracker.sendTracker(MapOutputTracker.scala:551)
> 3200 at
> org.apache.spark.MapOutputTrackerMaster.stop(MapOutputTracker.scala:1242)
> 3201 at org.apache.spark.SparkEnv.stop(SparkEnv.scala:112)
> 3202 at
> org.apache.spark.SparkContext.$anonfun$stop$25(SparkContext.scala:2354)
> 3203 at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1294)
> 3204 at org.apache.spark.SparkContext.stop(SparkContext.scala:2354)
> 3205 at org.apache.spark.SparkContext.stop(SparkContext.scala:2259)
> 3206 at
> org.apache.spark.MapStatusesSerDeserBenchmark$.afterAll(MapStatusesSerDeserBenchmark.scala:128)
> 3207 at org.apache.spark.benchmark.BenchmarkBase.main(BenchmarkBase.scala:80)
> 3208 at
> org.apache.spark.MapStatusesSerDeserBenchmark.main(MapStatusesSerDeserBenchmark.scala)
> 3209 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 3210 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 3211 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 3212 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> 3213 at
> org.apache.spark.benchmark.Benchmarks$.$anonfun$main$7(Benchmarks.scala:128)
> 3214 at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
> 3215 at org.apache.spark.benchmark.Benchmarks$.main(Benchmarks.scala:91)
> 3216 at org.apache.spark.benchmark.Benchmarks.main(Benchmarks.scala)
> 3217 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 3218 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 3219 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 3220 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> 3221 at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> 3222 at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1009)
> 3223 at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:183)
> 3224 at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:206)
> 3225 at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:80)
> 3226 at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1099)
> 3227 at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1108)
> 3228 at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 322924/04/28 09:30:31 ERROR MapOutputTrackerMaster: Could not tell tracker we
> are stopping.
> 3230org.apache.spark.SparkException: Error communicating with MapOutputTracker
> 3231 at
> org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:545)
> 3232 at
> org.apache.spark.MapOutputTracker.sendTracker(MapOutputTracker.scala:551)
> 3233 at
> org.apache.spark.MapOutputTrackerMaster.stop(MapOutputTracker.scala:1242)
> 3234 at org.apache.spark.SparkEnv.stop(SparkEnv.scala:112)
> 3235 at
> org.apache.spark.SparkContext.$anonfun$stop$25(SparkContext.scala:2354)
> 3236 at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1294)
> 3237 at org.apache.spark.SparkContext.stop(SparkContext.scala:2354)
> 3238 at org.apache.spark.SparkContext.stop(SparkContext.scala:2259)
> 3239 at
> org.apache.spark.MapStatusesSerDeserBenchmark$.afterAll(MapStatusesSerDeserBenchmark.scala:128)
> 3240 at org.apache.spark.benchmark.BenchmarkBase.main(BenchmarkBase.scala:80)
> 3241 at
> org.apache.spark.MapStatusesSerDeserBenchmark.main(MapStatusesSerDeserBenchmark.scala)
> 3242 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 3243 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 3244 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 3245 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> 3246 at
> org.apache.spark.benchmark.Benchmarks$.$anonfun$main$7(Benchmarks.scala:128)
> 3247 at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
> 3248 at org.apache.spark.benchmark.Benchmarks$.main(Benchmarks.scala:91)
> 3249 at org.apache.spark.benchmark.Benchmarks.main(Benchmarks.scala)
> 3250 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 3251 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 3252 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 3253 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> 3254 at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> 3255 at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1009)
> 3256 at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:183)
> 3257 at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:206)
> 3258 at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:80)
> 3259 at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1099)
> 3260 at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1108)
> 3261 at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 3262Caused by: java.lang.NullPointerException: Cannot invoke
> "org.apache.spark.rpc.RpcEndpointRef.askSync(Object, scala.reflect.ClassTag)"
> because the return value of
> "org.apache.spark.MapOutputTracker.trackerEndpoint()" is null
> 3263 at
> org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:541)
> 3264 ... 30 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
