[
https://issues.apache.org/jira/browse/HADOOP-14685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104329#comment-16104329
]
Sean Busbey commented on HADOOP-14685:
--------------------------------------
the test-jar artifacts aren't the problem. We probably need to include them, as
that's where the minicluster classes are defined. (and those are why we create
this artifact in the first place).
What we don't need are these things that got flagged on HADOOP-14089:
{code}
testjar/
testjar/ClassWordCount$MapClass.class
testjar/ClassWordCount$Reduce.class
testjar/ClassWordCount.class
testjar/CustomOutputCommitter.class
testjar/ExternalIdentityReducer.class
testjar/ExternalMapperReducer.class
testjar/ExternalWritable.class
testjar/JobKillCommitter$CommitterWithFailCleanup.class
testjar/JobKillCommitter$CommitterWithFailSetup.class
testjar/JobKillCommitter$CommitterWithNoError.class
testjar/JobKillCommitter$MapperFail.class
testjar/JobKillCommitter$MapperPass.class
testjar/JobKillCommitter$MapperPassSleep.class
testjar/JobKillCommitter$ReducerFail.class
testjar/JobKillCommitter$ReducerPass.class
testjar/JobKillCommitter.class
testjar/UserNamePermission$UserNameMapper.class
testjar/UserNamePermission$UserNameReducer.class
testjar/UserNamePermission.class
testshell/
testshell/ExternalMapReduce$MapClass.class
testshell/ExternalMapReduce$Reduce.class
testshell/ExternalMapReduce.class
.options
jdtCompilerAdapter.jar
{code}
I haven't chased down where these come from, but we should filter out just them.
We probably also ought to exclude unneeded properties files that come out of
the test jars too, most importantly the log4j.properties file since that should
be up to downstream.
> Test jars to exclude from hadoop-client-minicluster jar
> -------------------------------------------------------
>
> Key: HADOOP-14685
> URL: https://issues.apache.org/jira/browse/HADOOP-14685
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
> Affects Versions: 3.0.0-beta1
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
>
> This jira is to discuss, what test jars to be included/excluded from
> hadoop-client-minicluster
> Jars included/excluded when building hadoop-client-minicluster
> [INFO] --- maven-shade-plugin:2.4.3:shade (default) @
> hadoop-client-minicluster ---
> [INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-client-runtime:jar:3.0.0-beta1-SNAPSHOT from the
> shaded jar.
> [INFO] Excluding org.apache.htrace:htrace-core4:jar:4.1.0-incubating from the
> shaded jar.
> [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
> [INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded
> jar.
> [INFO] Excluding junit:junit:jar:4.11 from the shaded jar.
> [INFO] Including org.hamcrest:hamcrest-core:jar:1.3 in the shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT from the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT in the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including de.ruedigermoeller:fst:jar:2.50 in the shaded jar.
> [INFO] Including com.cedarsoftware:java-util:jar:1.9.0 in the shaded jar.
> [INFO] Including com.cedarsoftware:json-io:jar:2.5.1 in the shaded jar.
> [INFO] Including org.apache.curator:curator-test:jar:2.12.0 in the shaded jar.
> [INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
> [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721 in
> the shaded jar.
> [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
> [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar.
> [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-hs:jar:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Excluding
> org.apache.hadoop:hadoop-yarn-server-timelineservice:jar:3.0.0-beta1-SNAPSHOT
> from the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT in the
> shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:3.0.0-beta1-SNAPSHOT
> in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-core:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-client:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-json:jar:1.19 in the shaded jar.
> [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar.
> [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-server:jar:1.19 in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-servlet:jar:1.19 in the shaded jar.
> [INFO] Including org.eclipse.jdt:core:jar:3.1.1 in the shaded jar.
> [INFO] Including net.sf.kosmosfs:kfs:jar:0.3 in the shaded jar.
> [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar.
> [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded
> jar.
> [INFO] Including com.jcraft:jsch:jar:0.1.54 in the shaded jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including com.codahale.metrics:metrics-core:jar:3.0.1 in the shaded
> jar.
> [INFO] Including
> org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:3.0.0-beta1-SNAPSHOT in
> the shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721 in the
> shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721 in the
> shaded jar.
> [INFO] Including org.eclipse.jetty:jetty-io:jar:9.3.11.v20160721 in the
> shaded jar.
> [INFO] Including com.google.inject.extensions:guice-servlet:jar:4.0 in the
> shaded jar.
> [INFO] Including com.google.inject:guice:jar:4.0 in the shaded jar.
> [INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
> [INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
> [INFO] Including
> com.sun.jersey.jersey-test-framework:jersey-test-framework-grizzly2:jar:1.19
> in the shaded jar.
> [INFO] Including com.sun.jersey:jersey-grizzly2-servlet:jar:1.19 in the
> shaded jar.
> [INFO] Including org.glassfish.grizzly:grizzly-http-servlet:jar:2.2.21 in the
> shaded jar.
> [INFO] Including org.glassfish.grizzly:grizzly-http-server:jar:2.2.21 in the
> shaded jar.
> [INFO] Including org.glassfish.grizzly:grizzly-rcm:jar:2.2.21 in the shaded
> jar.
> [INFO] Including com.sun.jersey:jersey-grizzly2:jar:1.19 in the shaded jar.
> [INFO] Including org.glassfish.grizzly:grizzly-http:jar:2.2.21 in the shaded
> jar.
> [INFO] Including org.glassfish.grizzly:grizzly-framework:jar:2.2.21 in the
> shaded jar.
> [INFO] Including com.sun.jersey.contribs:jersey-guice:jar:1.19 in the shaded
> jar.
> [INFO] Replacing original artifact with shaded artifact.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
