[
https://issues.apache.org/jira/browse/STORM-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286905#comment-15286905
]
Robert Joseph Evans commented on STORM-1599:
--------------------------------------------
I think that marking them as provided is the outlier. The majority of storm-*
don't mark their dependencies as provided. In general though we should work
towards doing a much better job of managing the system classpath, dependencies,
and making it consistent everywhere. Hopefully we can do that as part of this
JIRA.
Right now the only things that storm "provides" on the system classpath are
storm-core and it's non-shaded dependencies, and storm-rename-hack. Nothing
else is truly provided by storm.
I ran {{find . -iname pom.xml | xargs grep -B 3 provided | grep artifactId |
egrep -v 'storm-core|storm-rename-hack'}} to find everywhere that we list
something as provided that really isn't
{code}
./examples/storm-starter/pom.xml- <artifactId>storm-clojure</artifactId>
./examples/storm-starter/pom.xml- <artifactId>storm-kafka</artifactId>
./examples/storm-starter/pom.xml-
<artifactId>${kafka.artifact.id}</artifactId>
./external/sql/storm-sql-core/pom.xml-
<artifactId>storm-sql-runtime</artifactId>
./external/sql/storm-sql-kafka/pom.xml-
<artifactId>storm-sql-runtime</artifactId>
./external/sql/storm-sql-kafka/pom.xml-
<artifactId>storm-kafka</artifactId>
./external/sql/storm-sql-kafka/pom.xml-
<artifactId>${kafka.artifact.id}</artifactId>
./external/storm-cassandra/pom.xml-
<artifactId>cassandra-driver-core</artifactId>
./external/storm-kafka/pom.xml-
<artifactId>${kafka.artifact.id}</artifactId>
./storm-buildtools/maven-shade-clojure-transformer/pom.xml-
<artifactId>maven-shade-plugin</artifactId>
./storm-buildtools/storm-maven-plugins/pom.xml-
<artifactId>maven-plugin-annotations</artifactId>
{code}
storm-clojure is new in 2.0 and should not be provided. It is not in lib.
storm-kafka and kafka.artifact.id is what this jira is about. storm-sql-* had
some issues where it was on the classpath even though it should not have been,
so they need to be updated accordingly too.
Looks like storm-cassandra should have the cassandra-driver-core fixed as well.
Then all that is left is the build-tools, which are not part of a topology jar
and can be ignored.
As such I will update the title of this JIRA to indicate that we should fix
storm-cassandra and strom-starter's dependency on storm-clojure.
> Kafka dependencies all marked as provided (so storm-starter does not run)
> -------------------------------------------------------------------------
>
> Key: STORM-1599
> URL: https://issues.apache.org/jira/browse/STORM-1599
> Project: Apache Storm
> Issue Type: Bug
> Components: examples, Flux, storm-kafka
> Affects Versions: 0.10.0, 1.0.0, 2.0.0
> Reporter: Robert Joseph Evans
> Assignee: Hugo Louro
>
> When we mark a dependency as provided it indicates the shade and assembly
> plugins to not include this particular dependency in the uber topology jar
> because it will be {provided} on the class path by the system.
> We have been doing this for all of our kafka dependencies incorrectly. This
> means that storm-starter does not have any version of kafka packaged it the
> resulting jar and any example that uses kafka, TridentKafkaWordCount, will
> fail with missing class errors.
> storm-starter/pom.xml has should change its dependency on storm-kafka to be
> compile, and it should delete dependencies on kafka and kafka-clients as
> those should come from storm-kafka as transitive dependencies.
> the main pom.xml should not have kafka-clients marked as provided in the
> dependency management section.
> storm-kafka should remove its provided tag on kafka, and flux examples +
> storm-sql-kafka should remove dependencies on kafka and kafka-clients, and
> storm-kafka should not me marked as provided.
> the flux and sql code I am not as familiar with, but looking at them, and
> running `mvn dependecy:tree` and `mvn dependency:analyze` it looks like
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
