Hello,
I'm using Kafka 0.8.2.1 (in a Scala/Java project) and trying to find out
how to atomically write n messages (message batch) to a single topic
partition. Is there any client API that gives such a guarantee? I
couldn't find a clear answer reading the documentation, API docs (of the
old and new producer) and mailing list archives - sorry if I missed
something obvious.
With the new org.apache.kafka.clients.producer.KafkaProducer, the send
method has only a single ProducerRecord parameter and record
accumulation and batch-sending of records is an implementation detail
(which can be controlled to some extend by the batch.size configuration
setting etc but not by user-defined message batches). So it seems that
the new producer cannot be used for that. Are there any plans to support
that in future versions?
Only the old kafka.producer.Producer allows me to pass a user-defined
KeyedMessage batch to its send method. What are the semantics of this
method when producer.type=sync and all KeyedMessages for a given send
call are targeted at the same topic partition? Are these messages being
written atomically to the partition?
Are there other options to achieve atomic writes of user-defined message
batches (except making the batch a single Kafka message)?
Thanks for any hints!
Regards,
Martin
