[
https://issues.apache.org/jira/browse/FLUME-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696197#comment-16696197
]
Hudson commented on FLUME-3101:
-------------------------------
SUCCESS: Integrated in Jenkins build flume-trunk #743 (See
[https://builds.apache.org/job/flume-trunk/743/])
FLUME-3101 Add maxBatchCount config property to Taildir Source. (szaboferee:
[http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=b252267ed297b849a8c3d900f7263e4abe5101c9])
* (edit) flume-ng-sources/flume-taildir-source/pom.xml
* (edit)
flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSourceConfigurationConstants.java
* (edit)
flume-ng-sources/flume-taildir-source/src/test/java/org/apache/flume/source/taildir/TestTaildirSource.java
* (edit)
flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java
* (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst
> taildir source may endless loop when tail a file
> ------------------------------------------------
>
> Key: FLUME-3101
> URL: https://issues.apache.org/jira/browse/FLUME-3101
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.7.0
> Reporter: hunshenshi
> Assignee: Peter Turcsanyi
> Priority: Major
> Labels: patch, taildirsource
> Fix For: 1.9.0
>
> Attachments: FLUME-3101-0.patch, FLUME-3101-1.patch,
> FLUME-3101-2.patch, FLUME-3101-3.patch
>
>
> If there are many files in the path that need to be tail, and there is a file
> written by *high frequency* (for example, there are file a, file b and file c
> in the path, file a is written at high frequency), *taildir can read the
> batchSize size event from file a everytime*, then taildir will only read data
> from file a, other files will not to be read, because in
> TaildirSource.tailFileProcess will into an endless loop.
> code:
> {code:title=TaildirSource.java|borderStyle=solid}
> private void tailFileProcess(TailFile tf, boolean backoffWithoutNL)
> throws IOException, InterruptedException {
> while (true) {
> // if events.size >= batchSize will not break while,
> // then into endless loop to only read tf
> if (events.size() < batchSize) {
> break;
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
