Produce requests in 0.7 are fire-and-forget since there is no
acknowledgements being returned by the broker.
0.8 fixes this by the optional use of required.acks.
I dont know if a non-JVM client is relevant for your needs, but librdkafka
implements all of your listed requirements for 0.8:
1. Message delivery reports (success or specific error) per message.
2.A Prodcuer is fully async, requests are queued locally and sent in
batches to the broker. Multiple requests are usually in-flight at any given
time.
2.B Consumer is fully async. The consumer strives to keep at
least queued.min.messages in the local queue of fetched message ready for
the application to consume.
3. A bit vague, but the application may state what offset to start
consuming from.
See this for more information
https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md
/Magnus
2013/12/6 Tom Brown tombrow...@gmail.com
In our environment we use currently use Kafka 0.7.1.
The core features I am looking for in a client are this:
1. Provide confirmation of produce requests (or notification of
disconnection during requests).
2. Uses asynchronous IO so that:
A. Multiple ops can be queued/in-flight at once.
B. Fetch #2 can be in-flight while fetch #1 is being processed (without
requiring an extra thread per consumer)
3. Allow precise control of offsets (like the SimpleConsumer).
I have been working on a version of the Kafka client that meets those
requirements. However, I recall hearing talk of rewriting the client. Did
this happen as part of 0.8.x? If so, how different is it from the 0.7.x
clients, and how well would it support the requirements listed above?
Thanks in advance!
--Tom
