[
https://issues.apache.org/jira/browse/FLUME-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401134#comment-13401134
]
Juhani Connolly edited comment on FLUME-1318 at 6/26/12 3:15 AM:
-----------------------------------------------------------------
One potential way to do this is to have sinks not shutdown until the channel
returns backoff(or add a new return value of closed). On the channel end,
channels that expect to be drained could wait till they are before shutting
down, and others that retain state over shutdowns can just shutdown immediately
and return backoff.
This is more of a policy thing though and would require enforcement, which is
somewhat difficult.
was (Author: juhanic):
One potential way to do this is to have sinks not shutdown until the
channel returns backoff(or add a new return value of closed). On the channel
end, channels that expect to be drained could wait till they are before
shutting down, and others that retain state over shutdowns can just shutdown
immediately and return backoff.
This is more of a policy things though and would require enforcement, which is
somewhat difficulty.
> Graceful shutdown and drain for memory channel
> ----------------------------------------------
>
> Key: FLUME-1318
> URL: https://issues.apache.org/jira/browse/FLUME-1318
> Project: Flume
> Issue Type: Improvement
> Components: Channel, Sinks+Sources
> Reporter: Mubarak Seyed
> Labels: newbie
>
> It appears from our test that if i go and shutdown the agent which uses
> memory channel, there is no clean way of throttling incoming requests to
> source, draining events from memory channel and do clean graceful shutdown of
> an agent. We don't have a visibility of how much data(events) in memory
> channel are getting dropped when shutdown happens.
> It would be good if we add a method to throttle incoming requests to source,
> drain events from memory channel and then shutdown sink/channel/source. (We
> can certainly add counters/metrics as part of channel monitoring). I think
> file-channel covers by adding it to WAL.
> HBase RegionServer shutdown handler does the some drain process (it syncs the
> edits in Write-Ahead-Log and notifies the master for regions move/assignment)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
