Cao Zheng created FLUME-3342:
--------------------------------
Summary: Error with TaildirSource
Key: FLUME-3342
URL: https://issues.apache.org/jira/browse/FLUME-3342
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.9.0
Environment: All Flume versions
Reporter: Cao Zheng
Hello ,
I don't know if anybody faced a bug with TaildirSource backoffWithoutNL。
when souce log files, sometimes , i got lines break into 2 Incomplete lines,
and
the backoffWithoutNL did not work，because the file is new arrival and markerd
as idel wrongly。
{code:java}
//代码占位符
public TailFile(File file, Map<String, String> headers, long inode, long pos)
throws IOException {
this.raf = new RandomAccessFile(file, "r");
if (pos > 0) {
raf.seek(pos);
lineReadPos = pos;
}
this.path = file.getAbsolutePath();
this.inode = inode;
this.pos = pos;
this.lastUpdated = 0L; // here, the TailFile Construtor init lastUpdated to
0L
this.needTail = true;
this.headers = headers;
this.oldBuffer = new byte[0];
this.bufferPos = NEED_READING;
}
{code}
the TailFile Construtor init lastUpdated to 0L，and lastUpdated only change
when commit() and close()。
{code:java}
//代码占位符
private class idleFileCheckerRunnable implements Runnable {
@Override
public void run() {
try {
long now = System.currentTimeMillis();
for (TailFile tf : reader.getTailFiles().values()) {
// here, the new file can be mark as idel wrongly
if (tf.getLastUpdated() + idleTimeout < now && tf.getRaf() !=
null) {
idleInodes.add(tf.getInode());
}
}
} catch (Throwable t) {
logger.error("Uncaught exception in IdleFileChecker thread", t);
}
}
}
{code}
i change the TailFile Construtor,
this.lastUpdated = System.currentTimeMillis();
so on , there's no break line any more
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
