How to add those dependencies? I followed the documentation of Kafka Input Operator, created a simple java maven project, compiled to create .apa package. So not sure how to add the malhar-contrib library in the application package?
Thanks for the help —Suhas From: Thomas Weise <thomas.we...@gmail.com<mailto:thomas.we...@gmail.com>> Reply-To: "users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>" <users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>> Date: Monday, February 29, 2016 at 9:34 PM To: "users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>" <users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>> Subject: Re: Error finding KafkaSinglePortStringInputOperator (NoClassDefFoundError) malhar-contrib and Kafka dependencies are missing. On Mon, Feb 29, 2016 at 9:32 PM, Suhas Gogate <su...@ampool.io<mailto:su...@ampool.io>> wrote: Thanks for the reply Thomas! I was just trying to workaround the problem although I agree that all dependencies should be part application package. Here is info. Sumedhas-MacBook-Pro-2:Downloads sumedhag$ unzip -t AdStream-1.0-SNAPSHOT.apa Archive: AdStream-1.0-SNAPSHOT.apa testing: META-INF/ OK testing: META-INF/MANIFEST.MF OK testing: app/ OK testing: app/AdStream-1.0-SNAPSHOT.jar OK testing: lib/ OK testing: lib/jetty-websocket-8.1.10.v20130312.jar OK testing: lib/jetty-security-8.1.10.v20130312.jar OK testing: lib/jetty-continuation-8.1.10.v20130312.jar OK testing: lib/activemq-client-5.8.0.jar OK testing: lib/jetty-util-8.1.10.v20130312.jar OK testing: lib/slf4j-api-1.7.5.jar OK testing: lib/jersey-client-1.9.jar OK testing: lib/jetty-servlet-8.1.10.v20130312.jar OK testing: lib/malhar-library-3.1.1.jar OK testing: lib/named-regexp-0.2.3.jar OK testing: lib/fastutil-6.6.4.jar OK testing: lib/jetty-http-8.1.10.v20130312.jar OK testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK testing: lib/jetty-io-8.1.10.v20130312.jar OK testing: lib/commons-logging-1.1.1.jar OK testing: lib/commons-beanutils-1.8.3.jar OK testing: lib/commons-lang3-3.1.jar OK testing: lib/jersey-core-1.9.jar OK testing: lib/jms-api-1.1-rev-1.jar OK testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK testing: lib/jetty-server-8.1.10.v20130312.jar OK testing: lib/hawtbuf-1.9.jar OK testing: lib/javax.mail-1.5.0.jar OK testing: lib/activation-1.1.jar OK testing: lib/commons-compiler-2.7.8.jar OK testing: conf/ OK testing: conf/my-app-conf1.xml OK testing: META-INF/properties.xml OK No errors detected in compressed data of AdStream-1.0-SNAPSHOT.apa. From: Thomas Weise <thomas.we...@gmail.com<mailto:thomas.we...@gmail.com>> Reply-To: "users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>" <users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>> Date: Monday, February 29, 2016 at 9:29 PM To: "users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>" <users@apex.incubator.apache.org<mailto:users@apex.incubator.apache.org>> Subject: Re: Error finding KafkaSinglePortStringInputOperator (NoClassDefFoundError) Suhas, Do not copy any operator libraries into the lib folder. These dependencies need to be packaged with the application into the .apa application package. The .apa will also need to contain the Kafka dependencies. Can you please share what .jar files are in your app package (unzip -l yourapp.apa) Thomas On Mon, Feb 29, 2016 at 9:10 PM, Suhas Gogate <su...@ampool.io<mailto:su...@ampool.io>> wrote: I have single node data torrent/Apex installation. Software installed under "/opt/datatorrent/releases/3.2.0”. When I uploaded and launched my application I get error finding the class “KafkaSinglePortStringInputOperator”. I could compile the application after adding the dt-contrib dependency but not sure how dt-contrib jars be available for application when running on single node data torrent installation. Appreciate help! —Suhas PS: I explicitly copied the dt-contrib-3.1.1.jar to /opt/datatorrent/current/lib/dt-contrib-3.1.1.jar and restarted the gateway.. Error launching the application: An error occurred trying to launch the application. Server message: java.lang.NoClassDefFoundError: com/datatorrent/contrib/kafka/KafkaSinglePortStringInputOperator at io.ampool.demo.adtech.Application.populateDAG(Application.java:44) at com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(LogicalPlanConfiguration.java:2108) at com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLauncher.java:407) at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:482) at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2047) at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3450) at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:106) at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1892) at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) Caused by: java.lang.ClassNotFoundException: com.datatorrent.contrib.kafka.KafkaSinglePortStringInputOperator at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:810) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 9 more Fatal error encountered Application.java import com.datatorrent.api.StreamingApplication; import com.datatorrent.api.DAG; import com.datatorrent.api.DAG.Locality; import com.datatorrent.lib.io.ConsoleOutputOperator; import com.datatorrent.contrib.kafka.KafkaSinglePortStringInputOperator; @ApplicationAnnotation(name = "AdStream") public class Application implements StreamingApplication { @Override public void populateDAG(DAG dag, Configuration entries) { KafkaSinglePortStringInputOperator input = dag.addOperator("MessageReader", newKafkaSinglePortStringInputOperator()); ConsoleOutputOperator output = dag.addOperator("Output", new ConsoleOutputOperator()); dag.addStream("MessageData", input.outputPort, output.input); } }
