Can you send me the full build file to further investigate the issue?
On Fri, Mar 11, 2016 at 4:56 PM, Balaji Rajagopalan <
balaji.rajagopa...@olacabs.com> wrote:
> Robert,
> That did not fix it ( using flink and connector same version) . Tried
> with scala version 2.11, so will try to see scala 2.10 makes any
> difference.
>
> balaji
>
> On Fri, Mar 11, 2016 at 8:06 PM, Robert Metzger <rmetz...@apache.org>
> wrote:
>
>> Hi,
>>
>> you have to use the same version for all dependencies from the
>> "org.apache.flink" group.
>>
>> You said these are the versions you are using:
>>
>> flink.version = 0.10.2
>> kafka.verison = 0.8.2
>> flink.kafka.connection.verion=0.9.1
>>
>> For the connector, you also need to use 0.10.2.
>>
>>
>>
>> On Fri, Mar 11, 2016 at 9:56 AM, Balaji Rajagopalan <
>> balaji.rajagopa...@olacabs.com> wrote:
>>
>>> I am tyring to use the flink kafka connector, for this I have specified
>>> the kafka connector dependency and created a fat jar since default flink
>>> installation does not contain kafka connector jars. I have made sure that
>>> flink-streaming-demo-0.1.jar has the
>>> kafka.javaapi.consumer.SimpleConsumer.class but still I see the class not
>>> found exception.
>>>
>>> The code for kafka connector in flink.
>>>
>>> val env = StreamExecutionEnvironment.getExecutionEnvironment
>>> val prop:Properties = new Properties()
>>> prop.setProperty("zookeeper.connect","somezookeer:2181")
>>> prop.setProperty("group.id","some-group")
>>> prop.setProperty("bootstrap.servers","somebroker:9092")
>>>
>>> val stream = env
>>> .addSource(new FlinkKafkaConsumer082[String]("location", new
>>> SimpleStringSchema, prop))
>>>
>>> jar tvf flink-streaming-demo-0.1.jar | grep
>>> kafka.javaapi.consumer.SimpleConsumer
>>>
>>> 5111 Fri Mar 11 14:18:36 UTC 2016
>>> *kafka/javaapi/consumer/SimpleConsumer*.class
>>>
>>> flink.version = 0.10.2
>>> kafka.verison = 0.8.2
>>> flink.kafka.connection.verion=0.9.1
>>>
>>> The command that I use to run the flink program in yarn cluster is
>>> below,
>>>
>>> HADOOP_CONF_DIR=/etc/hadoop/conf /usr/share/flink/bin/flink run -c
>>> com.dataartisans.flink_demo.examples.DriverEventConsumer -m yarn-cluster
>>> -yn 2 /home/balajirajagopalan/flink-streaming-demo-0.1.jar
>>>
>>> java.lang.NoClassDefFoundError: kafka/javaapi/consumer/SimpleConsumer
>>>
>>> at
>>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.getPartitionsForTopic(FlinkKafkaConsumer.java:691)
>>>
>>> at
>>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.<init>(FlinkKafkaConsumer.java:281)
>>>
>>> at
>>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer082.<init>(FlinkKafkaConsumer082.java:49)
>>>
>>> at
>>> com.dataartisans.flink_demo.examples.DriverEventConsumer$.main(DriverEventConsumer.scala:53)
>>>
>>> at
>>> com.dataartisans.flink_demo.examples.DriverEventConsumer.main(DriverEventConsumer.scala)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:497)
>>>
>>> at
>>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:395)
>>>
>>> at org.apache.flink.client.program.Client.runBlocking(Client.java:252)
>>>
>>> at
>>> org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:676)
>>>
>>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:326)
>>>
>>> at
>>> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:978)
>>>
>>> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1028)
>>>
>>> Caused by: java.lang.ClassNotFoundException:
>>> kafka.javaapi.consumer.SimpleConsumer
>>>
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>
>>> ... 16 more
>>>
>>>
>>> Any help appreciated.
>>>
>>>
>>> balaji
>>>
>>
>>
>
