[ https://issues.apache.org/jira/browse/FLUME-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073318#comment-16073318 ]
hunshenshi commented on FLUME-3101: ----------------------------------- [~fszabo] Can you assign this task to me? Could you also help review the patch in [ReviewRequest_FLUME-3101|https://reviews.apache.org/r/59743/]? thanks > taildir source may endless loop when tail a file > ------------------------------------------------ > > Key: FLUME-3101 > URL: https://issues.apache.org/jira/browse/FLUME-3101 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources > Affects Versions: 1.7.0 > Reporter: hunshenshi > Labels: patch, taildirsource > Fix For: 1.8.0 > > Attachments: FLUME-3101-0.patch, FLUME-3101-1.patch, > FLUME-3101-2.patch, FLUME-3101-3.patch > > > If there are many files in the path that need to be tail, and there is a file > written by *high frequency* (for example, there are file a, file b and file c > in the path, file a is written at high frequency), *taildir can read the > batchSize size event from file a everytime*, then taildir will only read data > from file a, other files will not to be read, because in > TaildirSource.tailFileProcess will into an endless loop. > code: > {code:title=TaildirSource.java|borderStyle=solid} > private void tailFileProcess(TailFile tf, boolean backoffWithoutNL) > throws IOException, InterruptedException { > while (true) { > // if events.size >= batchSize will not break while, > // then into endless loop to only read tf > if (events.size() < batchSize) { > break; > } > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)