Not sure if this would fir, but have a look at http://flume.apache.org/FlumeDeveloperGuide.html#embedded-agent
thanks ashish On Tue, May 20, 2014 at 7:16 PM, Jay Vyas <[email protected]> wrote: > Hi flume ! > > I'd like to implement a simple flume sink: > > agent.channels.memory-channel.type = memory > agent.channels.memory-channel.capacity = 2000 > agent.channels.memory-channel.transactionCapacity = 100 > agent.sources.tail-source.type = exec > agent.sources.tail-source.command = tail -F /tmp/flume-smoke > agent.sources.tail-source.channels = memory-channel > > agent.sinks.log-sink.channel = memory-channel > agent.sinks.log-sink.type = logger > > # Define a sink that outputs to the DFS > agent.sinks.hdfs-sink.channel = memory-channel > agent.sinks.hdfs-sink.type = hdfs > agent.sinks.hdfs-sink.hdfs.path = myfs:///tmp/flume-test > agent.sinks.hdfs-sink.hdfs.fileType = DataStream > > > # activate the channels/sinks/sources > agent.channels = memory-channel > agent.sources = tail-source > agent.sinks = log-sink hdfs-sink > > > But id like to do this dynamically, i.e. without having to package any > static text files. > > Is there a simple way to > > 1) Create the above sink definition and > 2) To start something like "timeout 60 flume-ng agent --conf ./flume/conf/ > -f ./flume/conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent" > > All in Java, ? > > -- > Jay Vyas > http://jayunit100.blogspot.com > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
