[jira] [Created] (FLUME-3101) taildir source may endless loop when tail a file

[hunshenshi (JIRA)]

hunshenshi created FLUME-3101:
---------------------------------

             Summary: taildir source may endless loop when tail a file
                 Key: FLUME-3101
                 URL: https://issues.apache.org/jira/browse/FLUME-3101
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: 1.7.0
            Reporter: hunshenshi


If there are many files in the path that need to be tail, and there is a file 
written by *high frequency* (for example, there are file a, file b and file c 
in the path, file a is written at high frequency), *taildir can read the 
batchSize size event from file a everytime*, then taildir will only read data 
from file a, other files will not to be read, because in 
TaildirSource.tailFileProcess will into an endless loop.

code:
{code:title=Bar.java|borderStyle=solid}
private void tailFileProcess(TailFile tf, boolean backoffWithoutNL)
    throws IOException, InterruptedException {
  while (true) {
    // if events.size >= batchSize will not break while, 
    // then into endless loop to only read tf
    if (events.size() < batchSize) {
      break;
    }
  }
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)