> On 2012-05-01 04:53:44, Mike Percy wrote: > > Hey; On further reflection, this approach just isn't great fit. The use > > case makes sense. However what we need is a decorator framework, not a > > sink-specific solution, and that is a relatively ambitious undertaking at > > this point. > > > > Let's hold off on pursuing sink-specific decorators until a framework is > > put into place. I'm working on a source/sink-agnostic approach in > > FLUME-1157 and will provide updates on that JIRA soon.
Sure Mike, will wait to get the design from the JIRA - Inder ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4826/#review7413 ----------------------------------------------------------- On 2012-04-20 10:33:14, Inder Singh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/4826/ > ----------------------------------------------------------- > > (Updated 2012-04-20 10:33:14) > > > Review request for Flume, Arvind Prabhakar, Mike Percy, and Hari Shreedharan. > > > Summary > ------- > > Overview of Changes > > 1.EventHeaderDecorator -> generic class having processHeader() > 2.SinkEventHeaderDecorator is a EventHeaderDecorator which overrides > timestamp header at this hop. > 3.EventHeaderDecoratorFactory provides EventHeaderDecorator's based on > <sink>.headerDecorator config > 4.AbstractSink -> has a eventHeaderDecortaor > 5.HDFSEventSink -> checks for decorator through config and calls > processHeader() for each event which modifies/add headers based on event > decorator defined through config. > > Sample Configuration for testing > ---------------------------------- > agent1.sinks.log-sink1.hdfs.headerDecorator = sinkDecorator > > > This addresses bug FLUME-1097. > https://issues.apache.org/jira/browse/FLUME-1097 > > > Diffs > ----- > > > flume-ng-core/src/main/java/org/apache/flume/event/EventHeaderDecorator.java > PRE-CREATION > > flume-ng-core/src/main/java/org/apache/flume/event/EventHeaderDecoratorFactory.java > PRE-CREATION > > flume-ng-core/src/main/java/org/apache/flume/event/SinkEventHeaderDecorator.java > PRE-CREATION > flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSink.java 2334059 > > flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 1b61cad > > Diff: https://reviews.apache.org/r/4826/diff > > > Testing > ------- > > Testing Done > ------------- > 1. Unit tests > 2. manual tests done - agent1.sinks.log-sink1.hdfs.filePrefix = flume-data/%D > results in publishing data with timestamp at agent's hop running HDFSEventSink > > > Thanks, > > Inder > >
