It sounds like you are describing Flume, with SpoolingDirectory source (or exec source running tail) and Kafka channel.
On Wed, Jan 28, 2015 at 10:39 AM, Fernando O. <fot...@gmail.com> wrote: > Hi all, > I'm evaluating using Kafka. > > I liked this thing of Facebook scribe that you log to your own machine and > then there's a separate process that forwards messages to the central > logger. > > With Kafka it seems that I have to embed the publisher in my app, and deal > with any communication problem managing that on the producer side. > > I googled quite a bit trying to find a project that would basically use > daemon that parses a log file and send the lines to the Kafka cluster > (something like a tail file.log but instead of redirecting the output to > the console: send it to kafka) > > Does anyone knows about something like that? > > > Thanks! > Fernando.
