(spark) branch branch-3.5 updated: [SPARK-45878][SQL][TESTS] Fix ConcurrentModificationException in CliSuite

[yao]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.5 by this push:
     new 0b68e1700f6 [SPARK-45878][SQL][TESTS] Fix 
ConcurrentModificationException in CliSuite
0b68e1700f6 is described below

commit 0b68e1700f60ad1a32f066c10a0f76bea893b7ce
Author: Kent Yao <y...@apache.org>
AuthorDate: Fri Nov 10 21:09:43 2023 +0800

    [SPARK-45878][SQL][TESTS] Fix ConcurrentModificationException in CliSuite
    
    ### What changes were proposed in this pull request?
    
    This PR changes the ArrayBuffer for logs to immutable for reading to 
prevent ConcurrentModificationException which hides the actual cause of failure
    
    ### Why are the changes needed?
    
    ```scala
    [info] - SPARK-29022 Commands using SerDe provided in ADD JAR sql *** 
FAILED *** (11 seconds, 105 milliseconds)
    [info]   java.util.ConcurrentModificationException: mutation occurred 
during iteration
    [info]   at 
scala.collection.mutable.MutationTracker$.checkMutations(MutationTracker.scala:43)
    [info]   at 
scala.collection.mutable.CheckedIndexedSeqView$CheckedIterator.hasNext(CheckedIndexedSeqView.scala:47)
    [info]   at 
scala.collection.IterableOnceOps.addString(IterableOnce.scala:1247)
    [info]   at 
scala.collection.IterableOnceOps.addString$(IterableOnce.scala:1241)
    [info]   at scala.collection.AbstractIterable.addString(Iterable.scala:933)
    [info]   at 
org.apache.spark.sql.hive.thriftserver.CliSuite.runCliWithin(CliSuite.scala:205)
    [info]   at 
org.apache.spark.sql.hive.thriftserver.CliSuite.$anonfun$new$20(CliSuite.scala:501)
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    no
    
    ### How was this patch tested?
    
    existing tests
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    no
    
    Closes #43749 from yaooqinn/SPARK-45878.
    
    Authored-by: Kent Yao <y...@apache.org>
    Signed-off-by: Kent Yao <y...@apache.org>
    (cherry picked from commit b347237735094e9092f4100583ed1d6f3eacf1f6)
    Signed-off-by: Kent Yao <y...@apache.org>
---
 .../test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala   | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git 
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
 
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
index 343b32e6227..38dcd1d8b00 100644
--- 
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
+++ 
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
@@ -193,7 +193,7 @@ class CliSuite extends SparkFunSuite {
       ThreadUtils.awaitResult(foundAllExpectedAnswers.future, timeoutForQuery)
       log.info("Found all expected output.")
     } catch { case cause: Throwable =>
-      val message =
+      val message = lock.synchronized {
         s"""
            |=======================
            |CliSuite failure output
@@ -207,6 +207,7 @@ class CliSuite extends SparkFunSuite {
            |End CliSuite failure output
            |===========================
          """.stripMargin
+      }
       logError(message, cause)
       fail(message, cause)
     } finally {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org