Hi Sean: Thanks for your reply, You mentioned that “You need to run ./dev/change-scala-version.sh 2.12 first” I have done that but still have this problem, currently the problem is about tools.jar does not exist in JDK9 but some dependencies on it when using “mvn clean install –DskipTests”.
Best Regards Kelly Zhang/Zhang,Liyun From: Sean Owen [mailto:so...@cloudera.com] Sent: Friday, December 22, 2017 6:56 AM To: Zhang, Liyun <liyun.zh...@intel.com> Cc: dev@spark.apache.org Subject: Re: Anyone know how to bypass tools.jar problem in JDK9 when mvn clean install SPARK code You need to run ./dev/change-scala-version.sh 2.12 first On Thu, Dec 21, 2017 at 4:38 PM Zhang, Liyun <liyun.zh...@intel.com<mailto:liyun.zh...@intel.com>> wrote: Hi all: Now I am using JDK9 to compile Spark by (mvn clean install –DskipTests), but exception was thrown [root@bdpe41 spark_source]# java -version java version "9.0.1" Java(TM) SE Runtime Environment (build 9.0.1+11) Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode) #mvn clean install -Pscala-2.12 -Pyarn -Pparquet-provided -DskipTests -X>log.mvn.clean.installed 2>&1 654189 [INFO] Spark Project Hive ................................. SUCCESS [ 30.708 s] 654190 [INFO] Spark Project REPL ................................. SUCCESS [ 2.795 s] 654191 [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 6.411 s] 654192 [INFO] Spark Project YARN ................................. FAILURE [ 0.047 s] 654193 [INFO] Spark Project Assembly ............................. SKIPPED 654194 [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED 654195 [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED 654196 [INFO] Spark Project Examples ............................. SKIPPED 654197 [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED 654198 [INFO] ------------------------------------------------------------------------ 654199 [INFO] BUILD FAILURE 654200 [INFO] ------------------------------------------------------------------------ 654201 [INFO] Total time: 07:04 min 654202 [INFO] Finished at: 2017-12-21T03:38:40+08:00 654203 [INFO] Final Memory: 85M/280M 654204 [INFO] ------------------------------------------------------------------------ 654205 [ERROR] Failed to execute goal on project spark-yarn_2.12: Could not resolve dependencies for project org.apache.spark:spark-yarn_2.12:jar:2.3.0-SNAPS HOT: Could not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path /home/zly/prj/oss/jdk-9.0.1/../lib/tools.jar -> [Help 1] 654206 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project spark-yarn_2.12: Could not resolve dependencies for project org.apache.spark:spark-yarn_2.12:jar:2.3.0-SNAPSHOT: Could not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path /home/zly/prj/oss/jdk-9.0.1 /../lib/tools.noite There is no tools.jar in JDK9<http://blog.codefx.org/java/dev/how-java-9-and-project-jigsaw-may-break-your-code/#Internal-JARs-Become-Unavailable>. I need to generate spark 2.3-SNAPSHOT in my local mvn repository to build other component(Hive on Spark) . Anyone knows how to bypass this problem? Best Regards Kelly Zhang/Zhang,Liyun
