[
https://issues.apache.org/jira/browse/SPARK-42112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42112:
------------------------------------
Assignee: Apache Spark
> Add null check before `ContinuousWriteRDD#compute` method close dataWriter
> --------------------------------------------------------------------------
>
> Key: SPARK-42112
> URL: https://issues.apache.org/jira/browse/SPARK-42112
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Assignee: Apache Spark
> Priority: Minor
>
> Run
>
> mvn clean test -pl sql/core -Dtest=none
> -DwildcardSuites=org.apache.spark.sql.streaming.continuous.ContinuousSuite -am
>
> NPE exists in the test log
>
> {code:java}
> - repeatedly restart
> ...
> 16:07:39.891 ERROR
> org.apache.spark.sql.execution.streaming.continuous.ContinuousWriteRDD:
> Writer for partition 1 is aborting.
> 16:07:39.891 ERROR
> org.apache.spark.sql.execution.streaming.continuous.ContinuousWriteRDD:
> Writer for partition 1 aborted.
> 16:07:39.892 WARN org.apache.spark.util.Utils: Suppressing exception in
> finally: null
> java.lang.NullPointerException
> at
> org.apache.spark.sql.execution.streaming.continuous.ContinuousWriteRDD.$anonfun$compute$7(ContinuousWriteRDD.scala:91)
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1558)
> at
> org.apache.spark.sql.execution.streaming.continuous.ContinuousWriteRDD.compute(ContinuousWriteRDD.scala:91)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
> at
> org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
> at org.apache.spark.scheduler.Task.run(Task.scala:139)
> at
> org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
> at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1502)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750) {code}
>
>
> The test did not fail because Utils.tryWithSafeFinallyAndFailureCallbacks
> function suppressed Exception in finally block
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
