James Xu created STORM-54:
-----------------------------
Summary: New topology config to specify an additional path to
include on the classpath for workers
Key: STORM-54
URL: https://issues.apache.org/jira/browse/STORM-54
Project: Apache Storm (Incubating)
Issue Type: New Feature
Reporter: James Xu
The use case for this is to avoid having to copy dependencies into the cluster
for every deploy. Instead, you deploy them once to the topology-specific path
on every worker machine and Storm will include those jars on the classpath of
the workers.
--------------------
b4hand: I personally like this feature better than
https://github.com/nathanmarz/storm/issues/281 as I think it handles large
third-party dependencies like HBase better than a JAR specific approach. HBase
relies on specific configuration files being in specific locations relative to
its root directory. Likewise, it has native libraries that it needs to link
against. I think it would be simpler to just add the output of hbase classpath
(which contains wildcards) to the storm.yaml config and this should just work.
Whereas the addPlatformJar approach doesn't take into account wildcards, config
files, or native libraries. Also, since storm already allows adding to the
java.libary.path via the storm.yaml, it seems like this approach would be more
similar to the existing functionality rather than having two entirely different
approaches to handling 3rd party resources.
I wouldn't mind taking a stab at this issue myself.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
