Actually, new and improved Kafka source and sink were committed to Flume trunk (but no release yet). The source is slightly better than the github one - we support buffering by number of messages and by latency. I tested both with 0.8.1.1 and with Kafka trunk.
I blogged about them a bit here: http://ingest.tips/2014/09/26/trying-to-decide-between-flume-and-kafka-try-both/ If you happen to use Flume from CDH5.2, you have the Kafka source and sink in your distribution. Gwen On Thu, Oct 23, 2014 at 11:51 PM, Mangtani, Kushal <[email protected]> wrote: > Yes, Flume has support for Kafka 0.8.1. Refer the below link for Flume Kafka > sink,source. I have been using this in my prod and it is relatively stable > https://github.com/thilinamb/flume-ng-kafka-sink > > -Kushal Mangtani > ________________________________ > From: Mongeol Heo [[email protected]] > Sent: Thursday, October 23, 2014 11:38 PM > To: [email protected] > Subject: Kafka version > > Hi, > > My question is that will flume sink and source for apache kafka support > latest version of it? > As I know, even sink/source for kafka 0.7.2 is not released yet. > It will be really great if it does. > Thanks. > > Best regards, > > - Mungeol
