[
https://issues.apache.org/jira/browse/FLUME-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408217#comment-13408217
]
Patrick Wendell commented on FLUME-1339:
----------------------------------------
I'm happy to take a look at this, but first I want to fully comprehend the
problem being solved and the need for a standalone executable like this rather
than just tuning an agent to provide the desired performance/durability
trade-off.
Hari tells me that this JIRA originated from observed disk performance overhead
of using file-based channel storage with an exec sink on production machines.
The sink was simply tailing production logs - a very common use case.
If disk persistence is required here for durability, then I don't see how
building this into a standalone client would make any difference, except that
it would be a second implementation of disk-based durability to maintain in
parallel with the existing agent code. I just want to be clear what overhead is
being avoided here.
If I understand the problem correctly - an alternative solution would be to
create a new channel which offers weaker durability semantics than the file
storage, but stronger durability than the memory channel. In other words, find
a middle point on this tradeoff curve but include it in Flume. Flume has
pluggable channels exactly for this reason, right?
> Implement an exec source like client API
> ----------------------------------------
>
> Key: FLUME-1339
> URL: https://issues.apache.org/jira/browse/FLUME-1339
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Patrick Wendell
>
> Originally when the exec source was implemented there was no SDK. It would be
> useful to have an API provided in the sdk that works like the exec source,
> thus removing the requirement of having an entire agent on hosts where the
> exec source is used for tailing logs.
> Maybe we can provide a standalone client that implements it too, which users
> can use off the shelf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
