[
https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348401#comment-14348401
]
Honghai Chen edited comment on KAFKA-1646 at 3/5/15 10:44 AM:
--------------------------------------------------------------
Hey, [~jkreps] just clarify, the 50MB/s you mentioned before is the checksum
calculation on the machine, not copy replica data from other machine, right?
If that's true, seemly we need do 3 changes:
1, when call logManager.shutdown. and os is windows , truncate active
segment.
2, when start, if the os is windows, add one new segment.
3, remove the change "
KAFKA-1646-truncate-off-trailing-zeros-on-broker-restart-if-bro.patch " made
previously since it's unnecessary.
Make sense?
was (Author: waldenchen):
Actually we want to add one more column to file
"recovery-point-offset-checkpoint", currently it only record offset, like below:
0
2
mvlogs 1 100
mvlogs 0 200
Change to below:
0
2
mvlogs 1 100 8000
mvlogs 0 200 16000
8000 is the start position of the data file for message with offset 100 . And
16000 is start position of the data file for message with offset 200.
Take first one as example, when recover the last segment (in function
LogSegment.recover(maxMessageSize: Int) , ONLY recover file to
min(validBytes, 8000) with offset 100 and rebuild index. Make sense ?
[~jkreps]
> Improve consumer read performance for Windows
> ---------------------------------------------
>
> Key: KAFKA-1646
> URL: https://issues.apache.org/jira/browse/KAFKA-1646
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Affects Versions: 0.8.1.1
> Environment: Windows
> Reporter: xueqiang wang
> Assignee: xueqiang wang
> Labels: newbie, patch
> Attachments: Improve consumer read performance for Windows.patch,
> KAFKA-1646-truncate-off-trailing-zeros-on-broker-restart-if-bro.patch,
> KAFKA-1646_20141216_163008.patch
>
>
> This patch is for Window platform only. In Windows platform, if there are
> more than one replicas writing to disk, the segment log files will not be
> consistent in disk and then consumer reading performance will be dropped down
> greatly. This fix allocates more disk spaces when rolling a new segment, and
> then it will improve the consumer reading performance in NTFS file system.
> This patch doesn't affect file allocation of other filesystems, for it only
> adds statements like 'if(Os.iswindow)' or adds methods used on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
