Re: Introducing Dory

There is an option for writing messages to a local log file, but this is intended purely for troubleshooting. Dory's intended purpose is to get messages off the local machine as quickly and efficiently as possible, and let the Kafka brokers handle the persistence. When a client sends a message

Re: Introducing Dory

Does Dory have the option to persist messages to local disk after the Dory ack but before sending to Kafka? Dave > On Jun 11, 2016, at 17:23, Dave Peterson wrote: > > Hello Kafka users, > > Version 1.1.0 of Dory is now available. See > https://github.com/dspeterson/dory

Broker Notification API for Kafka?

I'd like to add a notification API to Kafka so that I can receive the following notifications for each broker in the cluster: 1. An event notification when a broker is promoted from follower to leader, or when a follower catches up with the leader or falls behind the leader. 2. An event

Re: Regex topics in kafka connect?

Barry, Actually, it's not exposed in 0.10 either. https://issues.apache.org/jira/browse/KAFKA-3073 is filed to track this. We know people will want this, it just hasn't made it in quite yet. -Ewen On Fri, Jun 10, 2016 at 4:40 AM, Barry Kaplan wrote: > The kafka connect

Introducing Dory

Hello Kafka users, Version 1.1.0 of Dory is now available. See https://github.com/dspeterson/dory for details. Dory is the successor to Bruce (https://github.com/tagged/bruce), a Kafka producer daemon I created while working at if(we) (http://www.ifwe.co/). The code has seen a number of

storm mbeans

Hi, all I know kafka automatically expose mbeans to jmx, it seems storm doesn’t, i wonder if anyone has the experience to use JConsole to read storm build-in metrics through mbeans, or I will have to write separate metricConsumer to register metrics to mbeans? Is there such source code

Re: KafkaStream and Kafka consumer group

Sorry, I mean KafkaProducer! On Sat, Jun 11, 2016 at 4:18 PM, Saeed Ansari wrote: > When I read from stream, I create AKKA actors for processing events. I am > not processing them in the stream, as the result > I do not have KStream to write into it. So I use

Re: KafkaStream and Kafka consumer group

When I read from stream, I create AKKA actors for processing events. I am not processing them in the stream, as the result I do not have KStream to write into it. So I use KafkaConsumer instead. On Sat, Jun 11, 2016 at 7:13 AM, Matthias J. Sax wrote: > Do you instantiate

Re: Subscription

Hi Sauradipta, You want to send this email to users-subscr...@kafka.apache.org instead. -brian > On Jun 11, 2016, at 2:35 AM, sauradipta dutta > wrote: > > I want to get subscribed to Kafka mailing list

Subscription

I want to get subscribed to Kafka mailing list

Re: KafkaStream and Kafka consumer group

Do you instantiate KafkaProduer in your user code? Why no use KStream.to("topic-name") ? -Matthias On 06/10/2016 12:28 AM, Saeed Ansari wrote: > Thank you Eno, > Adding more threads extremely increased the throughput of stream. As I said > after processing I send the event to another topic.

Re: error: ... protocols are incompatible with those of existing members ??

Actually, this is exactly what Connect is doing. KafkaConnect uses its own "consumer" protocol called "connect" to distribute tasks between the workers. The default group name for this is connect-cluster, but it is possible to override it in the connect-distributed.properties file. SinkTasks

Re: error: ... protocols are incompatible with those of existing members ??

Thanks Dana, But this error is for a *single* process using the kafka-connect library. It does not seem to make any sense that the kafka-connect library would connect to the broker with two different protocols in the same process. Am I misunderstanding something? On Fri, Jun 10, 2016 at 6:55 PM,