Juhani Connolly created FLUME-1201:
--------------------------------------
Summary: Create a buffer channel, that stores overflow from a
fast, low capacity channel to a slower high capacity channel
Key: FLUME-1201
URL: https://issues.apache.org/jira/browse/FLUME-1201
Project: Flume
Issue Type: New Feature
Components: Channel
Reporter: Juhani Connolly
Assignee: Juhani Connolly
Fix For: v1.2.0
As it stands, users need to make a choice between either slower channels or
faster ones. The current "middle ground" consists of RecoverableFileChannel
which still stores everything to disk. MemoryChannel on the other hand has
limited capacity or a high memory footprint.
I propose to implement a buffer channel, somewhat like the buffer store in
scribed.
It would normally act as a proxy to the primary channel. Should this channel be
unable to receive events(normally because it is at capacity, but perhaps some
future channels may have other failure cases) it would switch states to
buffering, storing new events to the secondary channel.
In buffering, items continue to be read from the primary channel, and it
attempts to "refill" itself from the secondary. Once the secondary is found to
be empty, operation switches back to streaming mode, with items going directly
to the primary.
The main objective of this would thus be to have a high throughput channel as
the primary mode of operation, allowing it to switch over when takes are not
keeping up with puts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
