Leader crash and data loss

2016-06-22 Thread Saeed Ansari
Hi, I searched a lot for my question and I did not find a good answer may someone help me in this group? When leader broker for a partition fails, ZK elects a new leader and this may take seconds. What happens to data published to that broker during election? How Kafka handles messages to failed b

Re: KafkaStream and Kafka consumer group

2016-06-11 Thread Saeed Ansari
Sorry, I mean KafkaProducer! On Sat, Jun 11, 2016 at 4:18 PM, Saeed Ansari wrote: > When I read from stream, I create AKKA actors for processing events. I am > not processing them in the stream, as the result > I do not have KStream to write into it. So I use KafkaConsumer instead. &

Re: KafkaStream and Kafka consumer group

2016-06-11 Thread Saeed Ansari
our user code? > > Why no use KStream.to("topic-name") ? > > > -Matthias > > On 06/10/2016 12:28 AM, Saeed Ansari wrote: > > Thank you Eno, > > Adding more threads extremely increased the throughput of stream. As I > said > > after processing I se

Re: KafkaStream and Kafka consumer group

2016-06-09 Thread Saeed Ansari
in your app: > > props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 4 /* four threads */); > > Thanks > Eno > > > On 9 Jun 2016, at 18:08, Saeed Ansari wrote: > > > > Hi Eno, > > Thank you for the response. Actually I did not know it automatically > >

Re: KafkaStream and Kafka consumer group

2016-06-09 Thread Saeed Ansari
> > Thanks > Eno > > > On 7 Jun 2016, at 20:02, Saeed Ansari wrote: > > > > Hi, > > I have started a small Kafka streaming application. I need to assign > > partitions to consumers in a consumer group. I am not sure where to start > > a

KafkaStream and Kafka consumer group

2016-06-07 Thread Saeed Ansari
Hi, I have started a small Kafka streaming application. I need to assign partitions to consumers in a consumer group. I am not sure where to start and how to structure consumer groups in KafkaStreams. I found that there is a StreamPartitionAssignor that can be added to config, but still I do not k

Serdes class

2016-06-07 Thread Saeed Ansari
Hi everyone, I have started to use KafkaStreams. I am using following dependency: org.apache.kafka kafka-streams 0.10.0.0 In the KafkaStream examples, there is *Serdes* class which does not include in this version. Is not this the latest version? Thank you, Saeed