[ https://issues.apache.org/jira/browse/KAFKA-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Burd resolved KAFKA-6139. -------------------------------- Resolution: Information Provided Error was due to having the plugin directory set a level too low (pointing inside the plugin). > error when loading plugins > -------------------------- > > Key: KAFKA-6139 > URL: https://issues.apache.org/jira/browse/KAFKA-6139 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect > Affects Versions: 0.11.0.0 > Reporter: Eugene Burd > > I am trying to run the Big Query connector for KafkaConnect > (https://github.com/wepay/kafka-connect-bigquery). I have configured it > using a docker container, dropped the jar files for the connector in the > container and configured the plugin.path in the kafka connect config to point > to the directory with the jars. > Upon startup, Kafka is scanning the plugin folder and evaluating all the jar > files found. It encounters the connector jar file, tests to see if it > extends a connector (in this case kcbq-connector-1.0.0-SNAPSHOT.jar), but > then it proceeds to create a new instance of the connector. Code snippet > from DelegatingClassLoader.java below. The problem is that the connector > class relies on other jar files that are in the plugin folder, but not in the > path. This causes a class not found exception (below). > I suspect when the plugin connector is actually executed, it is done through > an isolated context and that context's classpath is set to the plugin folder, > so i think the connector would actually work. But scanning for the connector > fails prior to ever getting there. > I understand that you need a version of the connector and hence why this code > is running, but i think the new instance creation for version check needs to > be done in a sandbox to support the classpaths of the plugin connector. > [2017-10-28 06:04:08,961] INFO Loading plugin from: > /usr/share/java/kafka-connect-bigquery/kcbq-connector-1.0.0-SNAPSHOT.jar > (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:176) > Exception in thread "main" java.lang.NoClassDefFoundError: > com/google/cloud/bigquery/BigQuery > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) > at java.lang.Class.getConstructor0(Class.java:3075) > at java.lang.Class.newInstance(Class.java:412) > at > org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:242) > at > org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:223) > at > org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198) > at > org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:190) > at > org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:150) > at > org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47) > at > org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:63) > Caused by: java.lang.ClassNotFoundException: > com.google.cloud.bigquery.BigQuery > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:62) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 11 more > ------------------------- > private <T> Collection<PluginDesc<T>> getPluginDesc( > Reflections reflections, > Class<T> klass, > ClassLoader loader > ) throws InstantiationException, IllegalAccessException { > Set<Class<? extends T>> plugins = reflections.getSubTypesOf(klass); > Collection<PluginDesc<T>> result = new ArrayList<>(); > for (Class<? extends T> plugin : plugins) { > if (PluginUtils.isConcrete(plugin)) { > // Temporary workaround until all the plugins are versioned. > if (Connector.class.isAssignableFrom(plugin)) { > result.add( > new PluginDesc<>( > plugin, > ((Connector) > plugin.newInstance()).version(), > loader > ) > ); > } else { > result.add(new PluginDesc<>(plugin, "undefined", loader)); > } > } > } > return result; > } -- This message was sent by Atlassian JIRA (v6.4.14#64029)