----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6329/ -----------------------------------------------------------
Review request for Flume, Hari Shreedharan and Patrick Wendell. Description ------- Here is the description in code changes 1. Remove the 'FileChannel.force(false)'. Each commit from Source will invoke this 'force' method. This method is too heavy for amounts of data comes. Each 'force' action will be consume 50-500ms that it confirms data stored into disk. Normally, OS will flush data from kernal buffer to disk asynchronously with ms level latency. It may useless in each commit operation. Certainly, data loss may occurs in server crash not process crash. Server crash is infrequent. 2. Do not pre-allocate disk space. Disk doesn't need the pre-allocation. 3. Use 'RandomAccessFile.write()' to replace 'FileChannel.write()'. Both in my test result and low-level instruction, the former is better than the latter Here I posted three changes, and I would like to use thread-level cached DirectByteBuffer to replace inner-heap ByteBuffer.allocate() (reuse outer-heap memory to reduce time that copying from heap to kernal). I will test this changes in next phase. This addresses bug https://issues.apache.org/jira/browse/FLUME-1423. https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1423 Diffs ----- trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java 1363210 Diff: https://reviews.apache.org/r/6329/diff/ Testing ------- Thanks, Denny Ye
