Myeonghyeon Lee created KAFKA-8996:
--------------------------------------
Summary: Support health check method of TransactionManager and
KafkaProducer
Key: KAFKA-8996
URL: https://issues.apache.org/jira/browse/KAFKA-8996
Project: Kafka
Issue Type: Wish
Components: clients
Affects Versions: 2.3.0
Reporter: Myeonghyeon Lee
Request to add Transactional KafkaProducer health check method to KafkaProducer
and TransactionManager.
{code:java}
producer.initTransactions();
try {
producer.beginTransaction();
for (int i = 0; i < 100; i++) {
producer.send(new ProducerRecord<>("my-topic", Integer.toString(i),
Integer.toString(i)));
// check broker, transaction coordinator, producer is not zombie
boolean canCommit = producer.canCommitTransaction();
if (canCommit) {
producer.commitTransaction();
} else {
producer.abortTransaction();
}
} catch (....)
}
....
{code}
I want to check if Transactional KafkaProducer is in normal state before Kafka
Transaction commitTransaction.
If Kafka Broker is abnormal or if Producer is Zombie, an error will occur when
commit is executed.
If KafkaProducer has a health check method, I can detect problems and take
other actions before committing Kafka Transaction.
Typically, "best efforts 1PC" for consistency with heterogeneous databases.
{code:java}
1. Start Kafka transaction
2. Start DB transaction
3. Insert record to DB SUCCESS
4. Send message to Kafka SUCCESS
5. Commit DB Transaction
6. Commit Kafka Transaction // can be failed broker status or producer is
zombie{code}
I know that "best efforts 1PC" does not guarantee full consistency.
However, if KafkaProducer is provided with a health check method, I have a
chance to check Kafka's status once before a haterogeneous database commit.
Please review.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
