[
https://issues.apache.org/jira/browse/FLUME-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281801#comment-13281801
]
Mike Percy commented on FLUME-1201:
-----------------------------------
Hey, sorry for not chiming in earlier. I have concerns about trying to make
channels composable for the following reasons:
1. Potential for a complex implementation and configuration
2. It could confuse the durability guarantees of the compound channel, which
today are very clear (either durable or not)
3. It would quadratically increase the testing surface area (take the outer
product of all channels and then make sure they work together)
So from my perspective, a memory channel that spills to disk, while still
potentially suffering from #2, would best be added as a separate channel type
of its own.
Best,
Mike
> Create a buffer channel, that stores overflow from a fast, low capacity
> channel to a slower high capacity channel
> -----------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1201
> URL: https://issues.apache.org/jira/browse/FLUME-1201
> Project: Flume
> Issue Type: New Feature
> Components: Channel
> Reporter: Juhani Connolly
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
> As it stands, users need to make a choice between either slower channels or
> faster ones. The current "middle ground" consists of RecoverableFileChannel
> which still stores everything to disk. MemoryChannel on the other hand has
> limited capacity or a high memory footprint.
> I propose to implement a buffer channel, somewhat like the buffer store in
> scribed.
> It would normally act as a proxy to the primary channel. Should this channel
> be unable to receive events(normally because it is at capacity, but perhaps
> some future channels may have other failure cases) it would switch states to
> buffering, storing new events to the secondary channel.
> In buffering, items continue to be read from the primary channel, and it
> attempts to "refill" itself from the secondary. Once the secondary is found
> to be empty, operation switches back to streaming mode, with items going
> directly to the primary.
> The main objective of this would thus be to have a high throughput channel as
> the primary mode of operation, allowing it to switch over when takes are not
> keeping up with puts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
