Hi Zoi, the spark version itself was not changed, but the support for Scala 2.11 and java 8 was removed. However, locally I managed to fix the problem by explicitly adding "2.12" in artifactIds of dependencies in the "wayang-tests-integration" pom.xml. I hope that this is also reflected in a successful build in the PR when the action is started.
Best, Juri ________________________________ From: Zoi Kaoudi <[email protected]> Sent: 11 January 2024 14:12 To: [email protected] <[email protected]> Subject: Re: Integration test failure caused by Spark [You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Juri, just to better understand: was this test failing before? Did you change the Spark version? Best -- Zoi Στις Πέμπτη 11 Ιανουαρίου 2024 στις 11:39:19 π.μ. CET, ο χρήστης Juri Petersen <[email protected]> έγραψε: Hello, in the process of finishing the PR https://github.com/apache/incubator-wayang/pull/389 for issue https://github.com/apache/incubator-wayang/issues/28<https://github.com/apache/incubator-wayang/issues/283>3, I encountered failures when running the integration tests in "wayang-tests-integration". When verifying the build, I get the following error message during the execution of tests: [ERROR] org.apache.wayang.tests.WordCountIT.testOnSparkToJava Time elapsed: 0.03 s <<< ERROR! org.apache.wayang.core.api.exception.WayangException: Job execution failed. at org.apache.wayang.tests.WordCountIT.testOnSparkToJava(WordCountIT.java:281) Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.storage.StorageUtils$ at org.apache.wayang.tests.WordCountIT.testOnSparkToJava(WordCountIT.java:281) >From resources like >https://java.tutorialink.com/running-unit-tests-with-spark-3-3-0-on-java-17-fails-with-illegalaccesserror-class-storageutils-cannot-access-class-sun-nio-ch-directbuffer/, > I can see that this is likely due to the configuration of Spark and Java 11. >The mentioned export flags for the "maven-surefire-plugin" didn't solve the >problem. Has anyone experienced a similar error or even this exact error and knows how to configure the build to avoid it? Thanks in advance, Juri Petersen
