Hi flume ! I'd like to implement a simple flume sink:
agent.channels.memory-channel.type = memory agent.channels.memory-channel.capacity = 2000 agent.channels.memory-channel.transactionCapacity = 100 agent.sources.tail-source.type = exec agent.sources.tail-source.command = tail -F /tmp/flume-smoke agent.sources.tail-source.channels = memory-channel agent.sinks.log-sink.channel = memory-channel agent.sinks.log-sink.type = logger # Define a sink that outputs to the DFS agent.sinks.hdfs-sink.channel = memory-channel agent.sinks.hdfs-sink.type = hdfs agent.sinks.hdfs-sink.hdfs.path = myfs:///tmp/flume-test agent.sinks.hdfs-sink.hdfs.fileType = DataStream # activate the channels/sinks/sources agent.channels = memory-channel agent.sources = tail-source agent.sinks = log-sink hdfs-sink But id like to do this dynamically, i.e. without having to package any static text files. Is there a simple way to 1) Create the above sink definition and 2) To start something like "timeout 60 flume-ng agent --conf ./flume/conf/ -f ./flume/conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent" All in Java, ? -- Jay Vyas http://jayunit100.blogspot.com
