[ https://issues.apache.org/jira/browse/STORM-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sriharsha Chintalapani updated STORM-1015: ------------------------------------------ Assignee: Hang Sun > Store Kafka offsets with Kafka's consumer offset management api > --------------------------------------------------------------- > > Key: STORM-1015 > URL: https://issues.apache.org/jira/browse/STORM-1015 > Project: Apache Storm > Issue Type: Improvement > Components: storm-kafka > Affects Versions: 0.11.0 > Reporter: Hang Sun > Assignee: Hang Sun > Priority: Minor > Labels: consumer, kafka, offset > Original Estimate: 72h > Remaining Estimate: 72h > > Current Kafka spout stores the offsets (and some other states) inside ZK with > its proprietary format. This does not work well with other Kafka offset > monitoring tools such as Burrow, KafkaOffsetMonitor etc. In addition, the > performance does not scale well compared with offsets managed by Kafka's > built-in offset management api. I have added a new option for Kafka to store > the same data using Kafka's built-in offset management capability. The change > is completely backward compatible with the current ZK storage option. The > feature can be turned on by a single configuration option. Hope this will > help people who wants to explore the option of using Kafka's built-in offset > management api. > References: > https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit/FetchAPI > -thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)