Re: Doubts Kafka

This didn't change in 0.8.2, unfortunately. What I typically do with high level consumer is read messages into my own buffer and once I'd done processing them with no errors, I clear my own buffer, commit offsets and read more messages from Kafka. This way, if I have errors I can re-try from my b

Re: Doubts Kafka

Have there been any changes with 0.8.2 in how the marker gets moved when you use the high-level consumer? One problem I have always had was: what if I pull something from the stream, but then I have an error in processing it? I don't really want to move the marker. I would almost like the client

Re: Doubts Kafka

On Sun, Feb 8, 2015 at 6:39 AM, Christopher Piggott wrote: > > The consumer used Zookeeper to store offsets, in 0.8.2 there's an option > to use Kafka itself for that (by setting *offsets.storage = kafka > > Does it still really live in zookeeper, and kafka is proxying the requests > through? > >

Re: Doubts Kafka

Sorry, I should have read the release notes before I asked this question. The answer was in there. Internally the implementation of the offset storage is just a compacted > Kafka topic ( > __consumer_offsets) keyed on the consumer’s group, to

Re: Doubts Kafka

> The consumer used Zookeeper to store offsets, in 0.8.2 there's an option to use Kafka itself for that (by setting *offsets.storage = kafka Does it still really live in zookeeper, and kafka is proxying the requests through? On Sun, Feb 8, 2015 at 9:25 AM, Gwen Shapira wrote: > Hi Eduardo, > >

Re: Doubts Kafka

Hi Eduardo, 1. "Why sometimes the applications prefer to connect to zookeeper instead brokers?" I assume you are talking about the clients and some of our tools? These are parts of an older design and we are actively working on fixing this. The consumer used Zookeeper to store offsets, in 0.8.2 t

Doubts Kafka

Hi Guys, I have some doubts about the Kafka, the first is Why sometimes the applications prefer to connect to zookeeper instead brokers? Connecting to zookeeper could create an overhead, because we are inserting other element between producer and consumer. Another question is about the informat