[
https://issues.apache.org/jira/browse/KAFKA-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060260#comment-15060260
]
Flavio Junqueira commented on KAFKA-2997:
-----------------------------------------
[~afirus] I'm not very sure of what you're trying to achieve here. It sounds
like you don't want to count unflushed messages and instead you want to have a
timer thread to trigger the flush. I'm wondering if you really need to have
that thread to achieve your goal or if you can simply run a thread in a tight
loop that will flush, accumulate while flushing, flush, accumulate while
flushing, and so on. What I'm proposing is essentially what we do in the
SyncRequestProcessor of ZooKeeper:
https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/SyncRequestProcessor.java
> Synchronous write to disk
> -------------------------
>
> Key: KAFKA-2997
> URL: https://issues.apache.org/jira/browse/KAFKA-2997
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.9.0.0
> Reporter: Arkadiusz Firus
> Priority: Minor
> Labels: features, patch
>
> Hi All,
> I am currently work on a mechanism which allows to do an efficient
> synchronous writing to the file system. My idea is to gather few write
> requests for one partition and after that call the fsync.
> As I read the code I find out that the best place to do it is to modify:
> kafka.log.Log.append
> method. Currently at the end of the method (line 368) there is a verification
> if the number of unflushed messages is greater than the flush interval
> (configuration parameter).
> I am thinking of extending this condition. I want to add additional boolean
> configuration parameter (sync write or something like this). If this
> parameter is set to true at the end of this method the thread should hang on
> a lock. On the other hand there will be another timer thread (for every
> partition) which will be invoked every 10ms (configuration parameter). During
> invocation the thread will call flush method and after that will be releasing
> all hanged threads.
> I am writing here because I would like to know your opinion about such
> approach. Do you think this one is good or maybe someone have a better (more
> permanent) one. I would also like to know if such approach is according to
> general Kafka architecture.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
