Just to close the loop on this, a Flume developer confirmed separately that the transaction in the source is unnecessary, so I filed FLUME-2297 regarding the unnecessary transaction in the developer guide.
On Sun, Jan 12, 2014 at 9:42 AM, Tom Wheeler <[email protected]> wrote: > I'm confused about the proper way to use transactions in a custom source, > as the documentation and examples seem to disagree with one another. I'm > hoping someone could let me know the recommended approach. > > The custom source in the Flume Developer's guide shows the use of a > transaction in the process() method; specifically, it stores the event in > the channel via the ChannelProcessor's processEvent method, just before it > commits the transaction: > > http://flume.apache.org/FlumeDeveloperGuide.html#source > > It looks to me. however, like ChannelProcessor.processEvent method itself > wraps everything in a transaction: > > > http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java > > Given that, does the custom source *really* need its own transaction like > the example shows? I notice other sources that do not use one: > > > http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java > > http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java > > http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java > > Thanks, > > Tom >
