[
https://issues.apache.org/jira/browse/SPARK-38340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500487#comment-17500487
]
Yang Jie commented on SPARK-38340:
----------------------------------
cc [~hyukjin.kwon] , do you know how GA tests code that doesn't merged with
hadoop-2 profile?
> Upgrade protobuf-java from 2.5.0 to 3.16.1
> ------------------------------------------
>
> Key: SPARK-38340
> URL: https://issues.apache.org/jira/browse/SPARK-38340
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.3.0
> Reporter: Bjørn Jørgensen
> Priority: Major
>
>
> [CVE-2021-22569|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-22569]
> To do this upgrade I have done
> external/kinesis-asl-assembly/pom.xml change line 65 to
> <version>3.16.1</version>
> pom.xml change line 124 to <protobuf.version>3.16.1</protobuf.version>
> run
> ./dev/test-dependencies.sh --replace-manifest
> witch change
> dev/deps/spark-deps-hadoop-2-hive-2.3 line 235 to
> protobuf-java/3.16.1//protobuf-java-3.16.1.jar
> and
> dev/deps/spark-deps-hadoop-3-hive-2.3 to
> protobuf-java/3.16.1//protobuf-java-3.16.1.jar
> My branch
> [protobuf-java-from-2.5.0-to-3.16.1|https://github.com/bjornjorgensen/spark/tree/protobuf-java-from-2.5.0-to-3.16.1]
> is OK with testes, but when I run
> ./build/mvn -DskipTests clean package && ./build/mvn -e package
>
> I get this error:
> 01:01:41.381 WARN
> org.apache.spark.sql.execution.datasources.orc.OrcColumnarBatchReaderSuite:
> ===== POSSIBLE THREAD LEAK IN SUITE
> o.a.s.sql.execution.datasources.orc.OrcColumnarBatchReaderSuite, threads:
> rpc-boss-3348-1 (daemon=true), shuffle-boss-3351-1 (daemon=true) =====
> Run completed in 1 hour, 7 minutes, 35 seconds.
> Total number of tests run: 11260
> Suites: completed 505, aborted 0
> Tests: succeeded 11259, failed 1, canceled 5, ignored 57, pending 0
> *** 1 TEST FAILED ***
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary for Spark Project Parent POM 3.3.0-SNAPSHOT:
> [INFO]
> [INFO] Spark Project Parent POM ........................... SUCCESS [ 3.396
> s]
> [INFO] Spark Project Tags ................................. SUCCESS [ 7.374
> s]
> [INFO] Spark Project Sketch ............................... SUCCESS [ 9.324
> s]
> [INFO] Spark Project Local DB ............................. SUCCESS [ 4.097
> s]
> [INFO] Spark Project Networking ........................... SUCCESS [ 47.468
> s]
> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 10.478
> s]
> [INFO] Spark Project Unsafe ............................... SUCCESS [ 2.425
> s]
> [INFO] Spark Project Launcher ............................. SUCCESS [ 2.767
> s]
> [INFO] Spark Project Core ................................. SUCCESS [30:56
> min]
> [INFO] Spark Project ML Local Library ..................... SUCCESS [ 29.105
> s]
> [INFO] Spark Project GraphX ............................... SUCCESS [02:09
> min]
> [INFO] Spark Project Streaming ............................ SUCCESS [05:21
> min]
> [INFO] Spark Project Catalyst ............................. SUCCESS [08:15
> min]
> [INFO] Spark Project SQL .................................. FAILURE [ 01:11
> h]
> [INFO] Spark Project ML Library ........................... SKIPPED
> [INFO] Spark Project Tools ................................ SKIPPED
> [INFO] Spark Project Hive ................................. SKIPPED
> [INFO] Spark Project REPL ................................. SKIPPED
> [INFO] Spark Project Assembly ............................. SKIPPED
> [INFO] Kafka 0.10+ Token Provider for Streaming ........... SKIPPED
> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
> [INFO] Kafka 0.10+ Source for Structured Streaming ........ SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
> [INFO] Spark Avro ......................................... SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 02:00 h
> [INFO] Finished at: 2022-02-27T01:01:44+01:00
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.scalatest:scalatest-maven-plugin:2.0.2:test (test) on project
> spark-sql_2.12: There are test failures -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.scalatest:scalatest-maven-plugin:2.0.2:test (test) on project
> spark-sql_2.12: There are test failures
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:215)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> Caused by: org.apache.maven.plugin.MojoFailureException: There are test
> failures
> at org.scalatest.tools.maven.TestMojo.execute (TestMojo.java:109)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:210)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> [ERROR]
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <args> -rf :spark-sql_2.12
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
