[
https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-36636:
------------------------------------
Assignee: Apache Spark
> SparkContextSuite random failure in Scala 2.13
> ----------------------------------------------
>
> Key: SPARK-36636
> URL: https://issues.apache.org/jira/browse/SPARK-36636
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.3.0
> Reporter: Yang Jie
> Assignee: Apache Spark
> Priority: Major
> Attachments: image-2021-09-11-00-29-21-168.png,
> image-2021-09-11-00-30-20-237.png, image-2021-09-11-00-39-43-752.png
>
>
> run
> {code:java}
> build/mvn clean install -Pscala-2.13 -pl core -am{code}
> or
> {code:java}
> build/mvn clean install -Pscala-2.13 -pl core -am -Dtest=none
> -DwildcardSuites=org.apache.spark.SparkContextSuite
> {code}
> Some cases may fail as follows:
>
> {code:java}
> - SPARK-33084: Add jar support Ivy URI -- test param key case sensitive ***
> FAILED ***
> java.lang.IllegalStateException: Cannot call methods on a stopped
> SparkContext.
> This stopped SparkContext was created at:
> org.apache.spark.SparkContextSuite.$anonfun$new$154(SparkContextSuite.scala:1155)
> org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> org.scalatest.Transformer.apply(Transformer.scala:22)
> org.scalatest.Transformer.apply(Transformer.scala:20)
> org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
> org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:190)
> org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
> org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
> org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:62)
> org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
> org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
> org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:62)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
> org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
> scala.collection.immutable.List.foreach(List.scala:333)
> The currently active SparkContext was created at:
> org.apache.spark.SparkContextSuite.$anonfun$new$154(SparkContextSuite.scala:1155)
> org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> org.scalatest.Transformer.apply(Transformer.scala:22)
> org.scalatest.Transformer.apply(Transformer.scala:20)
> org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
> org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:190)
> org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
> org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
> org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:62)
> org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
> org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
> org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:62)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
> org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
> scala.collection.immutable.List.foreach(List.scala:333)
> at org.apache.spark.SparkContext.assertNotStopped(SparkContext.scala:118)
> at org.apache.spark.SparkContext.getSchedulingMode(SparkContext.scala:1887)
> at
> org.apache.spark.SparkContext.postEnvironmentUpdate(SparkContext.scala:2575)
> at org.apache.spark.SparkContext.addJar(SparkContext.scala:2008)
> at org.apache.spark.SparkContext.addJar(SparkContext.scala:1928)
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$154(SparkContextSuite.scala:1156)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> at org.scalatest.Transformer.apply(Transformer.scala:22)
> ...
> - SPARK-33084: Add jar support Ivy URI -- test transitive value case
> insensitive *** FAILED ***
> org.apache.spark.SparkException: Only one SparkContext should be running in
> this JVM (see SPARK-2243).The currently running SparkContext was created at:
> org.apache.spark.SparkContextSuite.$anonfun$new$154(SparkContextSuite.scala:1155)
> org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> org.scalatest.Transformer.apply(Transformer.scala:22)
> org.scalatest.Transformer.apply(Transformer.scala:20)
> org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
> org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:190)
> org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
> org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
> org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:62)
> org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
> org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
> org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:62)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
> org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
> scala.collection.immutable.List.foreach(List.scala:333)
> at
> org.apache.spark.SparkContext$.$anonfun$assertNoOtherContextIsRunning$2(SparkContext.scala:2647)
> at scala.Option.foreach(Option.scala:437)
> at
> org.apache.spark.SparkContext$.assertNoOtherContextIsRunning(SparkContext.scala:2644)
> at
> org.apache.spark.SparkContext$.markPartiallyConstructed(SparkContext.scala:2734)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:95)
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$159(SparkContextSuite.scala:1166)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> at org.scalatest.Transformer.apply(Transformer.scala:22)
> ...
> - SPARK-34346: hadoop configuration priority for spark/hive/hadoop configs
> *** FAILED ***
> org.apache.spark.SparkException: Only one SparkContext should be running in
> this JVM (see SPARK-2243).The currently running SparkContext was created at:
> org.apache.spark.SparkContextSuite.$anonfun$new$154(SparkContextSuite.scala:1155)
> org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> org.scalatest.Transformer.apply(Transformer.scala:22)
> org.scalatest.Transformer.apply(Transformer.scala:20)
> org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
> org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:190)
> org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
> org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
> org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
> org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:62)
> org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
> org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
> org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:62)
> org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
> org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
> scala.collection.immutable.List.foreach(List.scala:333)
> at
> org.apache.spark.SparkContext$.$anonfun$assertNoOtherContextIsRunning$2(SparkContext.scala:2647)
> at scala.Option.foreach(Option.scala:437)
> at
> org.apache.spark.SparkContext$.assertNoOtherContextIsRunning(SparkContext.scala:2644)
> at
> org.apache.spark.SparkContext$.markPartiallyConstructed(SparkContext.scala:2734)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:95)
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$164(SparkContextSuite.scala:1192)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> at org.scalatest.Transformer.apply(Transformer.scala:22)
> ...
> - SPARK-34225: addFile/addJar shouldn't further encode URI if a URI form
> string is passed *** FAILED ***
> java.lang.NullPointerException:
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$166(SparkContextSuite.scala:1245)
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$166$adapted(SparkContextSuite.scala:1211)
> at org.apache.spark.SparkFunSuite.withTempDir(SparkFunSuite.scala:208)
> at
> org.apache.spark.SparkContextSuite.$anonfun$new$165(SparkContextSuite.scala:1211)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> at org.scalatest.Transformer.apply(Transformer.scala:22)
> at org.scalatest.Transformer.apply(Transformer.scala:20)
> ...
> {code}
> but which case fails has a certain randomness
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
