[
https://issues.apache.org/jira/browse/SAMZA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056871#comment-14056871
]
Chris Riccomini commented on SAMZA-305:
---------------------------------------
Storm's wiki and discussion on this issue seems to be identical to what we'd
have to do.
I was concerned about the performance penalty of a class loader switch, but
Krill (on GH discussion) claims it's negligible.
This discussion is also somewhat related to SAMZA-307, since this change would
require us being able to tell whether a jar is a Samza dependency or a
user-code dependency. Being able to split the YARN runtime dependencies from
the code dependencies (as discussed in SAMZA-307) would allow us to tell the
difference.
> Isolate user library dependencies from framework dependencies
> -------------------------------------------------------------
>
> Key: SAMZA-305
> URL: https://issues.apache.org/jira/browse/SAMZA-305
> Project: Samza
> Issue Type: New Feature
> Affects Versions: 0.7.0
> Reporter: Martin Kleppmann
>
> In a running container, Samza framework classes and user job classes are
> bundled into the same classpath. This can cause conflicts, especially with
> transitive dependencies. For example, samza-yarn depends on hadoop-common
> which depends on Guava, and the user job may depend on some database client
> library which depends on a different version of Guava. The ensuing version
> conflicts can be nasty.
> The same problem also plagues other frameworks that run user code, for
> example MapReduce and Storm. To my knowledge, neither of those have solved
> the problem. Tomcat *has* solved the problem by using a separate classloader
> for every webapp running in its container.
> Storm's ideas for classpath isolation are discussed
> [here|https://github.com/xumingming/storm/wiki/Storm-Dependency-Isolation]
> and
> [here|https://www.bountysource.com/issues/87805-keep-storm-dependencies-and-user-dependencies-independent].
> It's tracked in STORM-129.
> It would be worth investigating this — if classpath isolation requires an
> intrusive change to Samza, it might be worth getting the pain out of the way
> while Samza is still young.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
