Martin Kleppmann created SAMZA-305:
--------------------------------------
Summary: Isolate user library dependencies from framework
dependencies
Key: SAMZA-305
URL: https://issues.apache.org/jira/browse/SAMZA-305
Project: Samza
Issue Type: New Feature
Affects Versions: 0.7.0
Reporter: Martin Kleppmann
In a running container, Samza framework classes and user job classes are
bundled into the same classpath. This can cause conflicts, especially with
transitive dependencies. For example, samza-yarn depends on hadoop-common which
depends on Guava, and the user job may depend on some database client library
which depends on a different version of Guava. The ensuing version conflicts
can be nasty.
The same problem also plagues other frameworks that run user code, for example
MapReduce and Storm. To my knowledge, neither of those have solved the problem.
Tomcat *has* solved the problem by using a separate classloader for every
webapp running in its container.
Storm's ideas for classpath isolation are discussed
[here|https://github.com/xumingming/storm/wiki/Storm-Dependency-Isolation] and
[here|https://www.bountysource.com/issues/87805-keep-storm-dependencies-and-user-dependencies-independent].
It's tracked in STORM-129.
It would be worth investigating this — if classpath isolation requires an
intrusive change to Samza, it might be worth getting the pain out of the way
while Samza is still young.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
