Re: Kafka/Hadoop consumers and producers

2013-11-22 Thread Abhi Basu
I agree with you. We are looking for a simple solution for data from Kafka to Hadoop. I have tried using Camus earlier (Non-Avro) and documentation is lacking to make it work correctly, as we do not need to introduce another component to the solution. In the meantime, can the Kafka Hadoop Consu

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
98961 2013-11-08 10:03 /tmp/kafka/lib/zkclient-20120522.jar On Fri, Nov 8, 2013 at 9:17 AM, Abhi Basu <9000r...@gmail.com> wrote: > Still get the same error: > [root@idh251-0 hadoop-consumer]# ./run-class.sh > kafka.etl.impl.SimpleKafkaETLJob test/test.properties > > :./../../co

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
/kafka/lib/piggybank.jar -rw-r--r-- 3 root hadoop 98961 2013-11-08 09:04 /tmp/kafka/lib/zkclient-20120522.jar On Fri, Nov 8, 2013 at 9:06 AM, Abhi Basu <9000r...@gmail.com> wrote: > Ok, sorry, missed a step, ran the copy_jars.sh and now retrying > > > On Fri, Nov 8

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
Ok, sorry, missed a step, ran the copy_jars.sh and now retrying On Fri, Nov 8, 2013 at 8:55 AM, Abhi Basu <9000r...@gmail.com> wrote: > Hi Neha: > > I was following the directions outlined here - > https://github.com/apache/kafka/tree/0.8/contrib/hadoop-consumer. It

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
: > ClassNotFound means the Hadoop job is not able to find the related jar. > Have you made sure the related jars are registered in the distributed > cache? > > > On Fri, Nov 8, 2013 at 8:40 AM, Abhi Basu <9000r...@gmail.com> wrote: > > > Can anyone help me with this issue? I

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
Can anyone help me with this issue? I feel like I am very close and am probably making some silly config error. Kafka team, please provide more detailed notes on how to make this component work. Thanks. On Fri, Nov 8, 2013 at 5:23 AM, Abhi Basu <9000r...@gmail.com> wrote: > Simpleka

Re: Kafka Hadoop Consumer issues

2013-11-08 Thread Abhi Basu
Simplekafkaetljob class, as mentioned in the post. Thanks Abhi >From Samsung Galaxy S4 On Nov 7, 2013 8:34 PM, "Jun Rao" wrote: > Which class is not found? > > Thanks, > > Jun > > > On Thu, Nov 7, 2013 at 11:56 AM, Abhi Basu <9000r...@gmail.com>

Kafka Hadoop Consumer issues

2013-11-07 Thread Abhi Basu
Node 1. I just want to do an end to end test to see that messages coming into Kafka are being stored in HDFS with the minimal amount of code change required. Thanks, Abhi -- Abhi Basu