Guozhang Wang created KAFKA-3193:
------------------------------------
Summary: Wait for producer message acks in StreamTask.commit()
Key: KAFKA-3193
URL: https://issues.apache.org/jira/browse/KAFKA-3193
Project: Kafka
Issue Type: Sub-task
Reporter: Guozhang Wang
Today in StreamTask.commit(), we does the following:
1) flush local state storage if it is persistent (e.g. RocksDB).
2) flush produced records in the downstream and changelog Kafka topics.
3) commit consumed offsets if it is dirty already
However in step 2) above:
{code}
producer.flush();
{code}
does not actually block on all the acks, but only force draining all records in
the buffer and send them. This actually result in a major defect of the
commit() call. We'd better add the ability in producer to wait in flushing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
