[ https://issues.apache.org/jira/browse/KAFKA-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849814#comment-17849814 ]
Ganesh Sadanala edited comment on KAFKA-15203 at 5/28/24 1:17 AM: ------------------------------------------------------------------ [~chia7712] There are 2 ways to invoke the Kafka Connect Plugin discovery. 1. The first is the core application flow(Plugins.initLoaders(...) which is used by Worker.startConnector(...)). I see that initLoaders is still using `ReflectionScanner` along with `ServiceLoader` to load plugins. The use case is to completely transition this to `ServiceLoader`. [~gharris1727] Is `ReflectionScanner` still needed here? {code:java} // public PluginScanResult initLoaders(Set<PluginSource> pluginSources, PluginDiscoveryMode discoveryMode) { PluginScanResult empty = new PluginScanResult(Collections.emptyList()); PluginScanResult serviceLoadingScanResult; try { serviceLoadingScanResult = discoveryMode.serviceLoad() ? new ServiceLoaderScanner().discoverPlugins(pluginSources) : empty; } catch (Throwable t) { throw new ConnectException(String.format( "Unable to perform ServiceLoader scanning as requested by %s=%s. It may be possible to fix this issue by reconfiguring %s=%s", WorkerConfig.PLUGIN_DISCOVERY_CONFIG, discoveryMode, WorkerConfig.PLUGIN_DISCOVERY_CONFIG, PluginDiscoveryMode.ONLY_SCAN), t); } PluginScanResult reflectiveScanResult = discoveryMode.reflectivelyScan() ? new ReflectionScanner().discoverPlugins(pluginSources) : empty; PluginScanResult scanResult = new PluginScanResult(Arrays.asList(reflectiveScanResult, serviceLoadingScanResult)); maybeReportHybridDiscoveryIssue(discoveryMode, serviceLoadingScanResult, scanResult); delegatingLoader.installDiscoveredPlugins(scanResult); return scanResult; } {code} 2. The terminal invoking tool/script `connect-plugin-path.sh` which uses the `org.apache.kafka.tools.ConnectPluginPath` to make use of both the Reflections and ServiceLoader mechanism to load the plugins. Since this a manual invoke (which I assume), there is no harm in using the Reflections for loading other plugins which are not required for the connect and/or not having the manifest files. I think this is the reason why [~gharris1727] said about it in the first comment: "The long term plan will be to make the reflections library less necessary via KAFKA-14627, and then the usage of reflections can be limited to the connect-plugin-path.sh script. We have not yet decided on a deprecation schedule and corresponding sunset date on the usage of this library." I am still going through the codebase to see if any modifications are needed. [~chia7712] To conclude, there is no need to remove the actual ReflectionScanner class but I need to remove the ReflectionScanner from the actual application flow above to not affect the script execution. If script execution is not needed and can be removed, ReflectionScanner can be totally removed. These are my conclusions after researching the KPIs and codebase. I am waiting for your replies so that I can remove unnecessary code part above and raise a PR. was (Author: JIRAUSER305566): [~chia7712] There are 2 ways to invoke the Kafka Connect Plugin discovery. 1. The first is the core application flow(Plugins.initLoaders(...) which is used by Worker.startConnector(...)). I see that initLoaders is still using `ReflectionScanner` along with `ServiceLoader` to load plugins. The use case is to completely transition this to `ServiceLoader`. [~gharris1727] Is `ReflectionScanner` still needed here? 2. The terminal invoking tool/script `connect-plugin-path.sh` which uses the `org.apache.kafka.tools.ConnectPluginPath` to make use of both the Reflections and ServiceLoader mechanism to load the plugins. Since this a manual invoke (which I assume), there is no harm in using the Reflections for loading other plugins which are not required for the connect and/or not having the manifest files. I think this is the reason why [~gharris1727] said about it in the first comment: "The long term plan will be to make the reflections library less necessary via KAFKA-14627, and then the usage of reflections can be limited to the connect-plugin-path.sh script. We have not yet decided on a deprecation schedule and corresponding sunset date on the usage of this library." I am still going through the codebase to see if any modifications are needed. > Remove dependency on Reflections > --------------------------------- > > Key: KAFKA-15203 > URL: https://issues.apache.org/jira/browse/KAFKA-15203 > Project: Kafka > Issue Type: Bug > Components: connect > Reporter: Divij Vaidya > Assignee: Ganesh Sadanala > Priority: Major > Labels: newbie > Fix For: 5.0.0 > > > We currently depend on reflections library which is EOL. Quoting from the > GitHub site: > _> Please note: Reflections library is currently NOT under active development > or maintenance_ > > This poses a supply chain risk for our project where the security fixes and > other major bugs in underlying dependency may not be addressed timely. > Hence, we should plan to remove this dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010)