[
https://issues.apache.org/jira/browse/KAFKA-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058235#comment-17058235
]
Nigel Liang edited comment on KAFKA-9712 at 3/12/20, 8:46 PM:
--------------------------------------------------------------
If we were to revert back to v0.9.11, we would bring back dependency on guava.
As noted in https://issues.apache.org/jira/browse/KAFKA-3061, this is
undesirable since having it in framework leads to potential version conflicts
downstream for users of framework that want to use their own Guava. OTOH, this
would be the path of lowest risk since no new code would be introduced.
reflections8 claims that they are obsoleted by v0.9.12 of reflections -
[https://github.com/aschoerk/reflections8] . This library will likely not be
maintained in the future, while reflections library will likely be fixed in a
future 0.9.13 release. There is already a PR out for the fix -
[https://github.com/ronmamo/reflections/pull/278]
The try...catch workaround seems like it will be most promising. The proposal
is to catch the exception [here|#L342]] and return empty result on exception.
The `Reflections` object we build and use here scans over either a single URL
at a time expecting to find an Uber jar or an array of URLs representing
contents of the plugin directory. This bug is triggered only when the
`Reflections` object could find no classes in the URLs provided. In which case,
returning empty results is indeed the correct response. Empty result is also
essentially the response that you would get from previous versions of
reflections that do not contain the bug. So, making the change should be pretty
safe in the sense that existing users upgrading to `2.5` would see the same
behavior as before (non-jars or jars containing no classes silently skipped).
Exception handling comes with a little bit of a performance penalty, but this
is in the worker startup path only and it will be only for exceptional
situations. The happy path will be unchanged.
was (Author: nigel.liang):
If we were to revert back to v0.9.11, we would bring back dependency on guava.
As noted in https://issues.apache.org/jira/browse/KAFKA-3061, this is
undesirable since having it in framework leads to potential version conflicts
downstream for users of framework that want to use their own Guava. OTOH, this
would be the path of lowest risk since no new code would be introduced.
reflections8 claims that they are obsoleted by v0.9.12 of reflections -
[https://github.com/aschoerk/reflections8] . This library will likely not be
maintained in the future, while reflections library will likely be fixed in a
future 0.9.13 release. There are already PRs out for the fix -
[https://github.com/ronmamo/reflections/pull/278]
The try...catch workaround seems like it will be most promising. The proposal
is to catch the exception [here|#L342]] and return empty result on exception.
The `Reflections` object we build and use here scans over either a single URL
at a time expecting to find an Uber jar or an array of URLs representing
contents of the plugin directory. This bug is triggered only when the
`Reflections` object could find no classes in the URLs provided. In which case,
returning empty results is indeed the correct response. Empty result is also
essentially the response that you would get from previous versions of
reflections that do not contain the bug. So, making the change should be pretty
safe in the sense that existing users upgrading to `2.5` would see the same
behavior as before (non-jars or jars containing no classes silently skipped).
Exception handling comes with a little bit of a performance penalty, but this
is in the worker startup path only and it will be only for exceptional
situations. The happy path will be unchanged.
> Reflections library 0.9.12 introduced in 2.5 causes regression scanning for
> plugins on plugin_path
> --------------------------------------------------------------------------------------------------
>
> Key: KAFKA-9712
> URL: https://issues.apache.org/jira/browse/KAFKA-9712
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.5.0, 2.6.0
> Reporter: Nigel Liang
> Assignee: Nigel Liang
> Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> Reflections v0.9.12 was introduced in 2.5 branch to remove Guava dependency -
> https://issues.apache.org/jira/browse/KAFKA-3061
> This version, however, contains a
> [bug|https://github.com/ronmamo/reflections/issues/273] that will cause
> regression when scanning for plugins if the `plugins_path` specified does not
> contain valid plugins. The case where we were able to repro this, the
> `plugins_path` was misconfigured to point to `~/.ssh` which contained
> unrelated files but no plugins. Further testing is needed to figure out if it
> will repro for other cases such as empty directory, combination of valid and
> invalid plugins in directory, just some types of plugins and not others in
> directory, etc.
> {code}
> [2020-03-12 18:07:01,045] INFO Loading plugin from: /home/ducker/.ssh
> (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
> [2020-03-12 18:07:01,047] DEBUG Loading plugin urls: []
> (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
> [2020-03-12 18:07:01,062] ERROR Stopping due to error
> (org.apache.kafka.connect.cli.ConnectDistributed)
> org.reflections.ReflectionsException: Scanner SubTypesScanner was not
> configured
> at org.reflections.Store.get(Store.java:39)
> at org.reflections.Store.get(Store.java:61)
> at org.reflections.Store.get(Store.java:46)
> at org.reflections.Store.getAll(Store.java:93)
> at org.reflections.Reflections.getSubTypesOf(Reflections.java:404)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:342)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:327)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:260)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:252)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:221)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:198)
> at
> org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
> at
> org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
> at
> org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
