[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158429#comment-15158429 ]
Zhichao Zhang commented on SPARK-13431: ---------------------------------------- I meet the same problem with maven, how to resolve this issue? *build command as follow:* bq. mvn clean package -DskipTests -Pscala-2.11 -Phadoop-2.6 -Phive -Phive-thriftserver -Pyarn -Dyarn.version=2.6.0-cdh5.4.7 -Dhadoop.version=2.6.0-cdh5.4.7 *build error message:* {quote} [INFO] Excluding org.scala-lang:scala-reflect:jar:2.11.7 from the shaded jar. [INFO] Excluding org.scala-lang:scala-library:jar:2.11.7 from the shaded jar. [INFO] Excluding org.apache.spark:spark-core_2.11:jar:2.0.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 from the shaded jar. [INFO] Excluding org.apache.avro:avro-ipc:jar:1.7.7 from the shaded jar. [INFO] Excluding org.apache.avro:avro:jar:1.7.7 from the shaded jar. [INFO] Excluding org.apache.avro:avro-ipc:jar:tests:1.7.7 from the shaded jar. [INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar. [INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar. [INFO] Excluding com.twitter:chill_2.11:jar:0.5.0 from the shaded jar. [INFO] Excluding com.esotericsoftware.kryo:kryo:jar:2.21 from the shaded jar. [INFO] Excluding com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07 from the shaded jar. [INFO] Excluding com.esotericsoftware.minlog:minlog:jar:1.2 from the shaded jar. [INFO] Excluding org.objenesis:objenesis:jar:1.2 from the shaded jar. [INFO] Excluding com.twitter:chill-java:jar:0.5.0 from the shaded jar. [INFO] Excluding org.apache.xbean:xbean-asm5-shaded:jar:4.4 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-client:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-common:jar:2.2.0 from the shaded jar. [INFO] Excluding commons-cli:commons-cli:jar:1.2 from the shaded jar. [INFO] Excluding org.apache.commons:commons-math:jar:2.1 from the shaded jar. [INFO] Excluding xmlenc:xmlenc:jar:0.52 from the shaded jar. [INFO] Excluding commons-io:commons-io:jar:2.1 from the shaded jar. [INFO] Excluding commons-lang:commons-lang:jar:2.6 from the shaded jar. [INFO] Excluding commons-configuration:commons-configuration:jar:1.6 from the shaded jar. [INFO] Excluding commons-collections:commons-collections:jar:3.2.2 from the shaded jar. [INFO] Excluding commons-digester:commons-digester:jar:1.8 from the shaded jar. [INFO] Excluding commons-beanutils:commons-beanutils:jar:1.7.0 from the shaded jar. [INFO] Excluding commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar. [INFO] Excluding com.google.protobuf:protobuf-java:jar:2.5.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-auth:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.commons:commons-compress:jar:1.4.1 from the shaded jar. [INFO] Excluding org.tukaani:xz:jar:1.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-hdfs:jar:2.2.0 from the shaded jar. [INFO] Excluding org.mortbay.jetty:jetty-util:jar:6.1.26 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 from the shaded jar. [INFO] Excluding com.google.inject:guice:jar:3.0 from the shaded jar. [INFO] Excluding javax.inject:javax.inject:jar:1 from the shaded jar. [INFO] Excluding aopalliance:aopalliance:jar:1.0 from the shaded jar. [INFO] Excluding com.sun.jersey.jersey-test-framework:jersey-test-framework-grizzly2:jar:1.9 from the shaded jar. [INFO] Excluding com.sun.jersey.jersey-test-framework:jersey-test-framework-core:jar:1.9 from the shaded jar. [INFO] Excluding javax.servlet:javax.servlet-api:jar:3.0.1 from the shaded jar. [INFO] Excluding com.sun.jersey:jersey-client:jar:1.9 from the shaded jar. [INFO] Excluding com.sun.jersey:jersey-grizzly2:jar:1.9 from the shaded jar. [INFO] Excluding org.glassfish.grizzly:grizzly-http:jar:2.1.2 from the shaded jar. [INFO] Excluding org.glassfish.grizzly:grizzly-framework:jar:2.1.2 from the shaded jar. [INFO] Excluding org.glassfish.gmbal:gmbal-api-only:jar:3.0.0-b023 from the shaded jar. [INFO] Excluding org.glassfish.external:management-api:jar:3.0.0-b012 from the shaded jar. [INFO] Excluding org.glassfish.grizzly:grizzly-http-server:jar:2.1.2 from the shaded jar. [INFO] Excluding org.glassfish.grizzly:grizzly-rcm:jar:2.1.2 from the shaded jar. [INFO] Excluding org.glassfish.grizzly:grizzly-http-servlet:jar:2.1.2 from the shaded jar. [INFO] Excluding org.glassfish:javax.servlet:jar:3.1 from the shaded jar. [INFO] Excluding com.sun.jersey:jersey-json:jar:1.9 from the shaded jar. [INFO] Excluding org.codehaus.jettison:jettison:jar:1.1 from the shaded jar. [INFO] Excluding com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 from the shaded jar. [INFO] Excluding javax.xml.bind:jaxb-api:jar:2.2.2 from the shaded jar. [INFO] Excluding javax.activation:activation:jar:1.1 from the shaded jar. [INFO] Excluding org.codehaus.jackson:jackson-jaxrs:jar:1.9.13 from the shaded jar. [INFO] Excluding org.codehaus.jackson:jackson-xc:jar:1.9.13 from the shaded jar. [INFO] Excluding com.sun.jersey.contribs:jersey-guice:jar:1.9 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-common:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-annotations:jar:2.2.0 from the shaded jar. [INFO] Excluding org.apache.spark:spark-launcher_2.11:jar:2.0.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.apache.spark:spark-network-common_2.11:jar:2.0.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.apache.spark:spark-network-shuffle_2.11:jar:2.0.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 from the shaded jar. [INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.5.3 from the shaded jar. [INFO] Excluding net.java.dev.jets3t:jets3t:jar:0.7.1 from the shaded jar. [INFO] Excluding commons-httpclient:commons-httpclient:jar:3.1 from the shaded jar. [INFO] Excluding org.apache.curator:curator-recipes:jar:2.4.0 from the shaded jar. [INFO] Excluding org.apache.curator:curator-framework:jar:2.4.0 from the shaded jar. [INFO] Excluding org.apache.curator:curator-client:jar:2.4.0 from the shaded jar. [INFO] Excluding org.apache.zookeeper:zookeeper:jar:3.4.5 from the shaded jar. [INFO] Excluding jline:jline:jar:0.9.94 from the shaded jar. [INFO] Excluding org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016 from the shaded jar. [INFO] Excluding org.apache.commons:commons-lang3:jar:3.3.2 from the shaded jar. [INFO] Excluding org.apache.commons:commons-math3:jar:3.4.1 from the shaded jar. [INFO] Excluding com.google.code.findbugs:jsr305:jar:1.3.9 from the shaded jar. [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.16 from the shaded jar. [INFO] Excluding org.slf4j:jul-to-slf4j:jar:1.7.16 from the shaded jar. [INFO] Excluding org.slf4j:jcl-over-slf4j:jar:1.7.16 from the shaded jar. [INFO] Excluding log4j:log4j:jar:1.2.17 from the shaded jar. [INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.16 from the shaded jar. [INFO] Excluding com.ning:compress-lzf:jar:1.0.3 from the shaded jar. [INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.2 from the shaded jar. [INFO] Excluding net.jpountz.lz4:lz4:jar:1.3.0 from the shaded jar. [INFO] Excluding org.roaringbitmap:RoaringBitmap:jar:0.5.11 from the shaded jar. [INFO] Excluding commons-net:commons-net:jar:2.2 from the shaded jar. [INFO] Excluding org.json4s:json4s-jackson_2.11:jar:3.2.10 from the shaded jar. [INFO] Excluding org.json4s:json4s-core_2.11:jar:3.2.10 from the shaded jar. [INFO] Excluding org.json4s:json4s-ast_2.11:jar:3.2.10 from the shaded jar. [INFO] Excluding org.scala-lang:scalap:jar:2.11.7 from the shaded jar. [INFO] Excluding org.scala-lang:scala-compiler:jar:2.11.7 from the shaded jar. [INFO] Excluding org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 from the shaded jar. [INFO] Excluding com.sun.jersey:jersey-server:jar:1.9 from the shaded jar. [INFO] Excluding asm:asm:jar:3.1 from the shaded jar. [INFO] Excluding com.sun.jersey:jersey-core:jar:1.9 from the shaded jar. [INFO] Excluding org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1 from the shaded jar. [INFO] Excluding io.netty:netty-all:jar:4.0.29.Final from the shaded jar. [INFO] Excluding io.netty:netty:jar:3.8.0.Final from the shaded jar. [INFO] Excluding com.clearspring.analytics:stream:jar:2.7.0 from the shaded jar. [INFO] Excluding io.dropwizard.metrics:metrics-core:jar:3.1.2 from the shaded jar. [INFO] Excluding io.dropwizard.metrics:metrics-jvm:jar:3.1.2 from the shaded jar. [INFO] Excluding io.dropwizard.metrics:metrics-json:jar:3.1.2 from the shaded jar. [INFO] Excluding io.dropwizard.metrics:metrics-graphite:jar:3.1.2 from the shaded jar. [INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.5.3 from the shaded jar. [INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.5.3 from the shaded jar. [INFO] Excluding com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.5.3 from the shaded jar. [INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.6 from the shaded jar. [INFO] Excluding org.apache.ivy:ivy:jar:2.4.0 from the shaded jar. [INFO] Excluding oro:oro:jar:2.0.8 from the shaded jar. [INFO] Excluding net.razorvine:pyrolite:jar:4.9 from the shaded jar. [INFO] Excluding net.sf.py4j:py4j:jar:0.9.1 from the shaded jar. [INFO] Excluding org.apache.spark:spark-unsafe_2.11:jar:2.0.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.codehaus.janino:janino:jar:2.7.8 from the shaded jar. [INFO] Excluding org.codehaus.janino:commons-compiler:jar:2.7.8 from the shaded jar. [INFO] Excluding org.antlr:antlr-runtime:jar:3.5.2 from the shaded jar. [INFO] Excluding commons-codec:commons-codec:jar:1.10 from the shaded jar. [INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar. [INFO] Excluding org.scala-lang.modules:scala-xml_2.11:jar:1.0.2 from the shaded jar. [DEBUG] Processing JAR /home/myubuntu/Works/workspace/spark/sql/catalyst/target/spark-catalyst_2.11-2.0.0-SNAPSHOT.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 52.733 s (Wall Clock) [INFO] Finished at: 2016-02-23T13:26:26+08:00 [INFO] Final Memory: 38M/744M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project spark-catalyst_2.11: Error creating shaded jar: Method code too large! -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.4.3:shade (default) on project spark-catalyst_2.11: Error creating shaded jar: Method code too large! at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185) at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.maven.plugin.MojoExecutionException: Error creating shaded jar: Method code too large! at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:540) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 11 more Caused by: java.lang.RuntimeException: Method code too large! at org.objectweb.asm.MethodWriter.a(Unknown Source) at org.objectweb.asm.ClassWriter.toByteArray(Unknown Source) at org.apache.maven.plugins.shade.DefaultShader.addRemappedClass(DefaultShader.java:453) at org.apache.maven.plugins.shade.DefaultShader.shadeSingleJar(DefaultShader.java:219) at org.apache.maven.plugins.shade.DefaultShader.shadeJars(DefaultShader.java:179) at org.apache.maven.plugins.shade.DefaultShader.shade(DefaultShader.java:104) at org.apache.maven.plugins.shade.mojo.ShadeMojo.execute(ShadeMojo.java:454) ... 13 more {quote} > Maven build fails due to: Method code too large! in Catalyst > ------------------------------------------------------------ > > Key: SPARK-13431 > URL: https://issues.apache.org/jira/browse/SPARK-13431 > Project: Spark > Issue Type: Bug > Components: Build > Affects Versions: 2.0.0 > Reporter: Stavros Kontopoulos > Priority: Blocker > > Cannot build the project when run the normal build commands: > eg. > {code} > build/mvn -Phadoop-2.6 -Dhadoop.version=2.6.0 clean package > ./make-distribution.sh --name test --tgz -Phadoop-2.6 > {code} > Integration builds are also failing: > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.6/229/console > https://ci.typesafe.com/job/mit-docker-test-zk-ref/12/console > It looks like this is the commit that introduced the issue: > https://github.com/apache/spark/commit/7925071280bfa1570435bde3e93492eaf2167d56 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org