Arkadiusz Firus created KAFKA-2997:
--------------------------------------
Summary: Synchronous write to disk
Key: KAFKA-2997
URL: https://issues.apache.org/jira/browse/KAFKA-2997
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 0.9.0.0
Reporter: Arkadiusz Firus
Priority: Minor
Hi All,
I am currently work on a mechanism which allows to do an efficient synchronous
writing to the file system. My idea is to gather few write requests for one
partition and after that call the fsync.
As I read the code I find out that the best place to do it is to modify:
kafka.log.Log.append
method. Currently at the end of the method (line 368) there is a verification
if the number of unflushed messages is greater than the flush interval
(configuration parameter).
I am thinking of extending this condition. I want to add additional boolean
configuration parameter (sync write or something like this). If this parameter
is set to true at the end of this method the thread should hang on a lock. On
the other hand there will be another timer thread (for every partition) which
will be invoked every 10ms (configuration parameter). During invocation the
thread will call flush method and after that will be releasing all hanged
threads.
I am writing here because I would like to know your opinion about such
approach. Do you think this one is good or maybe someone have a better (more
permanent) one. I would also like to know if such approach is according to
general Kafka architecture.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
