Enriched message metadata

2015-05-23 Thread Christopher Piggott
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Enriched+Message+Metadata Just curious if this proposal is in the development plan or not ... I'm particularly interested in being able to add custom KVP type metadata to messages. --Chris

Re: Doubts Kafka

2015-02-08 Thread Christopher Piggott
, Christopher Piggott cpigg...@gmail.com wrote: The consumer used Zookeeper to store offsets, in 0.8.2 there's an option to use Kafka itself for that (by setting *offsets.storage = kafka Does it still really live in zookeeper, and kafka is proxying the requests through? They don't live

Re: Doubts Kafka

2015-02-08 Thread Christopher Piggott
per consumer group, topic, partition triplet, so offset fetch requests can be served quickly without requiring a full scan of the compacted offsets topic. With this feature, consumers can checkpoint offsets very often, possibly per message. On Sun, Feb 8, 2015 at 9:39 AM, Christopher Piggott

Re: Doubts Kafka

2015-02-08 Thread Christopher Piggott
The consumer used Zookeeper to store offsets, in 0.8.2 there's an option to use Kafka itself for that (by setting *offsets.storage = kafka Does it still really live in zookeeper, and kafka is proxying the requests through? On Sun, Feb 8, 2015 at 9:25 AM, Gwen Shapira gshap...@cloudera.com

Re: Consumer questions

2015-01-17 Thread Christopher Piggott
://issues.apache.org/jira/browse/KAFKA-1779 4. Yes Manikumar On Sat, Jan 17, 2015 at 2:25 AM, Christopher Piggott cpigg...@gmail.com wrote: Hi, I am following this link: https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example for a consumer design using

Consumer questions

2015-01-16 Thread Christopher Piggott
as will fit into that many byte buffer? Is that right? Thanks. Christopher Piggott Sr. Staff Engineer Golisano Institute for Sustainability Rochester Institute of Technology