Built-in replication for broker

Dear developers, In the paper Kafka: A Distributed Messaging System for Log Processing, Jay Kreps, Neha Narkhede, Jun Rao from LinkedIn, at NetDB workshop 2011, you mentioned this: If a broker goes down, any message stored on it not yet consumed becomes unavailable. If the storage system on a

RE: Built-in replication for broker

Thanks for reply! Date: Tue, 16 Dec 2014 18:23:39 -0800 Subject: Re: Built-in replication for broker From: n...@confluent.io To: users@kafka.apache.org Replication is available in Kafka 0.8.0 onwards. On Tue, Dec 16, 2014 at 4:31 PM, Haoming Zhang haoming.zh...@outlook.com wrote

RE: Partition key not working properly

- key 2 - partition 2 - key 3 - partition 3 and nothing in partition 1 On Wed Nov 26 2014 at 02:35:33 Haoming Zhang haoming.zh...@outlook.com wrote: Hi Svante, Thanks for your reply! As you said, my purpose is let all messages with the same key goes to the same partition

Partition key not working properly

Hi all, I'm struggling with how to use the partition key mechanism properly. My logic is set the partition number as 3, then create three partition keys as 0, 1, 2, then use the partition keys to create three KeyedMessage such as KeyedMessage(topic, 0, message), KeyedMessage(topic, 1,

RE: Partition key not working properly

that for example all logs from the same ip goest to the same partition which means that they can be read by the same producer. /svante 2014-11-26 2:42 GMT+01:00 Haoming Zhang haoming.zh...@outlook.com: Hi all, I'm struggling with how to use the partition key mechanism properly. My

Exception in thread main java.lang.ExceptionInInitializerError

Hi all, Basically I used a lot of codes from this project https://github.com/stealthly/scala-kafka , my idea is to sent a key/value pair to Kafka, so that I can design a partition function in the further. I checked the document and seems I should create a ProducerRecord, then I can specify

Partition Key Cannot be Send Out by Producer

Hi all, I'm a beginner of Kafka, currently I'm stuck by how to send out a KeyedMessage by producer. I would like to design a partition function to route the message based on the key, but the producer cannot send the KeyedMessage and I got this exception: java.lang.ClassCastException: [B cannot

RE: Partition Key Cannot be Send Out by Producer

/KafkaProducer.scala for your partKey it should be string and when you converting it into byte array you can use partKey.getBytes(UTF8) -Harsha On Thu, Nov 20, 2014, at 03:57 PM, Haoming Zhang wrote: Hi all, I'm a beginner of Kafka, currently I'm stuck by how to send out a KeyedMessage

RE: Partition Key Cannot be Send Out by Producer

by Producer Date: Thu, 20 Nov 2014 16:59:19 -0800 also the (key: Key, value: Val, topic: Option[String]) value should be a string converted to a byte array. Can you send a example of your key and value data. On Thu, Nov 20, 2014, at 04:53 PM, Haoming Zhang wrote: Hi Harsha, Thanks

RE: Partition Key Cannot be Send Out by Producer

http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Thu, Nov 20, 2014 at 8:15 PM, Haoming Zhang haoming.zh...@outlook.com wrote: Hi Harsha, I just tried to hard code a string message, then convert

RE: Partition Key Cannot be Send Out by Producer

^) /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Thu, Nov 20, 2014 at 8:42 PM, Haoming Zhang haoming.zh