Neha Narkhede created KAFKA-1326:
------------------------------------
Summary: New consumer checklist
Key: KAFKA-1326
URL: https://issues.apache.org/jira/browse/KAFKA-1326
Project: Kafka
Issue Type: New Feature
Components: consumer
Affects Versions: 0.9.0
Reporter: Neha Narkhede
Assignee: Neha Narkhede
We will use this JIRA to track the list of issues to resolve to get a working
new consumer client. The consumer client can work in phases -
1. Add new consumer APIs and configs
2. Refactor Sender. We will need to use some common APIs from Sender.java
3. Add metadata fetch and refresh functionality to the consumer (This will
require https://issues.apache.org/jira/browse/KAFKA-1316 as )
4. Add functionality to support subscribe(TopicPartition...partitions). This
will add SimpleConsumer functionality to the new consumer. This does not
include any group management related work.
5. Add ability to commit offsets to Kafka. This will include adding
functionality to the commit()/commitAsync()/committed() APIs. This still does
not include any group management related work.
6. Add functionality to the offsetsBeforeTime() API.
7. Add consumer co-ordinator election to the server. This will only add a new
module for the consumer co-ordinator, but not necessarily all the logic to do
group management.
At this point, we will have a fully functional standalone consumer and a server
side co-ordinator module. This will be a good time to start adding group
management functionality to the server and consumer.
8. Add failure detection capability to the consumer when group management is
used. This will not include any rebalancing logic, just the ability to detect
failures using session.timeout.ms.
9. Add rebalancing logic to the server and consumer. This will be a tricky and
potentially large change since it will involve implementing the group
management protocol.
10. Add system tests for the new consumer
11. Add metrics
12. Convert mirror maker to use the new consumer.
13. Convert perf test to use the new consumer
14. Performance testing and analysis.
15. Review and fine tune log4j logging
--
This message was sent by Atlassian JIRA
(v6.2#6252)
