[
https://issues.apache.org/jira/browse/KAFKA-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418143#comment-16418143
]
Chris Egerton commented on KAFKA-6417:
--------------------------------------
[~cotedm] Alright, took another look at the Connect code base and have some bad
news :( It looks like there are enough different places that the
{{ClassNotFoundException}} can be thrown from at enough different times that
catching it and displaying a useful error message would be harder than I
originally thought. On top of that, it would be pretty difficult to
differentiate between "It looks like this JAR is missing because you specified
an incorrect plugin path" and "This JAR is actually missing" in most cases,
since the exception will have occurred after the initial plugin class was
loaded and outside any code that deals with plugin isolation.
As far as forcing all JAR files to go into their own directories goes, I have a
few more thoughts. If someone tries to start up connect, gets an error about
JAR files found on the plugin path (possibly also with the information that
"All JAR files must be placed in a subdirectory of a plugin path directory" or
something along those lines), and wants to fix the issue, do you think it's
likely that the next step for them would be to then create a subdirectory and
then place all of their uber JARs inside that directory? It may seem like a
reasonable assumption that uber JARs would be loaded independently of each
other, but if we don't have a meaningful way of differentiating them from
non-uber JARs in the Connect framework, we'd just end up loading all of the
plugins in that directory with the same classloader, potentially leading to the
same overlapping dependency issues that the whole {{plugin.path}} configuration
is meant to prevent.
> plugin.path pointing at a plugin directory causes ClassNotFoundException
> ------------------------------------------------------------------------
>
> Key: KAFKA-6417
> URL: https://issues.apache.org/jira/browse/KAFKA-6417
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 1.0.0
> Reporter: Dustin Cote
> Priority: Major
>
> When using the {{plugin.path}} configuration for the Connect workers, the
> user is expected to specify a list containing the following per the docs:
> {quote}
> The list should consist of top level directories that include any combination
> of: a) directories immediately containing jars with plugins and their
> dependencies b) uber-jars with plugins and their dependencies c) directories
> immediately containing the package directory structure of classes of plugins
> and their dependencies
> {quote}
> This means we would expect {{plugin.path=/usr/share/plugins}} for a structure
> like {{/usr/share/plugins/myplugin1}},{{/usr/share/plugins/myplugin2}}, etc.
> However if you specify {{plugin.path=/usr/share/plugins/myplugin1}} the
> resulting behavior is that dependencies for {{myplugin1}} are not properly
> loaded. This causes a {{ClassNotFoundException}} that is not intuitive to
> debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
