[
https://issues.apache.org/jira/browse/FLUME-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkatesh Sivasubramanian updated FLUME-1819:
---------------------------------------------
Attachment: FLUME-1819.patch.1
FLUME-1819.patch
Hi Hari: Pls. find the Patch for this issue attached. I have also added a unit
test case for testing the same. Pls. review and let me know if there are any
suggestions/feedbacks. Additionally locally tested it with an actual agent as
well.
I will add this to the review board now as well.
There is a known issue with this fix. Even after a timeout has occurred, it
still expects one event to come before it can flush the contents in the buffer.
That is because of the readLine() call we have, it waits till there is data in
the InputStream. Changing this has larger implications, so I left it there.
Pls. do let me know if you have any suggestions around this as well. Thank you!!
Best Regards,
Venkatesh
> ExecSource don't flush the cache if there is no input entries
> -------------------------------------------------------------
>
> Key: FLUME-1819
> URL: https://issues.apache.org/jira/browse/FLUME-1819
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Fengdong Yu
> Assignee: Venkatesh Sivasubramanian
> Fix For: v1.4.0
>
> Attachments: FLUME-1819.patch, FLUME-1819.patch.1
>
>
> ExecSource has a default batchSize: 20, exec source read data from the
> source, then put it into the cache, after the cache is full, push it to the
> channel.
> but if exec source's cache is not full, and there isn't any input for a long
> time, then these entries always kept in the cache, there is no chance to the
> channel until the source's cache is full.
> so, the patch added a new config line: batchTimeout for ExecSource, and
> default is 3 seconds, if batchTimeout exceeded, push all cached data to the
> channel even the cache is not full.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
