Kafka Consumer v1.0 terminates prematurely

2018-03-16 Thread Javed, Haseeb
Hello, I have a test cluster for Kafka consisting of one broker, one producer and one consumer, each on a separate node. Kafka broker and producer are v1.0.0. I am able to use the ProducerPerformance benchmark to write records to the broker. However, when I try to read the records using Consum

Purpose of mute/unmute logic in SocketServer

2017-11-14 Thread Javed, Haseeb
Hello, I was going through the SocketServer code to understand the networking internals of Kafka code base. I see that at some points in the Processor, we mute/unmute certain Kafka channels. These mute/unmute methods seem to be a wrapper to enable/disbale reading new data from respective chann

Re: How is CorrelationId used for matching request and response

2017-10-01 Thread Javed, Haseeb
to be in order. Thanks all for the help. Haseeb From: Javed, Haseeb Sent: Sunday, October 1, 2017 6:39:13 AM To: dev@kafka.apache.org Subject: Re: How is CorrelationId used for matching request and response Thanks all for reaching out. Ted - I am looking

Re: How is CorrelationId used for matching request and response

2017-09-30 Thread Javed, Haseeb
e: > Which release / version are you looking at ? > In trunk branch, I only see one toSend(): > > protected Send toSend(String destination, ResponseHeader header, short > apiVersion) { > > return new NetworkSend(destination, serialize(apiVersion, header)); > >

How is CorrelationId used for matching request and response

2017-09-29 Thread Javed, Haseeb
The Kafka protocol guide mentions that each request and response contains a correlationId which is a user-supplied integer to match requests and corresponding responses. However, when I look at the code in the class AbstractResponse, we have a method defined as following: public Send toSend(St

Understanding the semantics of Selector.poll()

2017-09-11 Thread Javed, Haseeb
Hello, I was looking into Kafka's network layer code have a few questions regarding the Selector class, particularly who the poll() method is implemented. The poll() method goes something like this: /* check ready keys */ long startSelect = time.nanoseconds(); int ready

Handler Threads in SocketServer

2017-08-01 Thread Javed, Haseeb
Hello, I have recently started looking into the Kafka Server code to get a deeper understanding of the internals and I have a few questions about the SocketServer implementation. The documentation for the SocketServer mentions: /** * The threading model is * 1 Acceptor thread that handles