Thanks Dibyendu. 1. Spark itself have api jar for kafka, still we require manual offset management (using simple consumer concept) and manual consumer ? 2.Kafka Spark Consumer which is implemented in kafka 0.8.0 ,Can we use it for kafka 0.8.1 ? 3.How to use Kafka Spark Consumer to produce output
*as parquet file on HDFS ?* *Please give your suggestion.* Regards, Rafeeq S *(“What you do is what matters, not what you think or say or plan.” )* On Tue, Aug 5, 2014 at 11:55 AM, Dibyendu Bhattacharya < dibyendu.bhattach...@gmail.com> wrote: > You can try this Kafka Spark Consumer which I recently wrote. This uses > the Low Level Kafka Consumer > > https://github.com/dibbhatt/kafka-spark-consumer > > Dibyendu > > > > > On Tue, Aug 5, 2014 at 12:52 PM, rafeeq s <rafeeq.ec...@gmail.com> wrote: > >> Hi, >> >> I am new to Apache Spark and Trying to Develop spark streaming program >> to *stream data from kafka topics and output as parquet file on HDFS*. >> >> Please share the *sample reference* program to stream data from kafka >> topics and output as parquet file on HDFS. >> >> Thanks in Advance. >> >> Regards, >> >> Rafeeq S >> *(“What you do is what matters, not what you think or say or plan.” )* >> >> >
