Re: messages intermittently dropped in a single partition configuration

2016-04-28 Thread Bo Xu
h I end up with, is to significantly increase max_in_flight_requests_per_connection, e.g., to 1024. On Thu, Apr 28, 2016 at 10:31 AM, Bo Xu wrote: > PS: The message dropping occurred intermittently, not all at the end. For > example, it is the 10th, 15th, 18th messages that are missing. It

Re: messages intermittently dropped in a single partition configuration

2016-04-28 Thread Bo Xu
PS: The message dropping occurred intermittently, not all at the end. For example, it is the 10th, 15th, 18th messages that are missing. It it were all at the end, it would be understandable because I'm not using flush() to force transmitting. Bo On Thu, Apr 28, 2016 at 10:15 AM, Bo Xu

messages intermittently dropped in a single partition configuration

2016-04-28 Thread Bo Xu
I set up a simple Kafka configuration, with one topic and one partition. I have a Python producer to continuously publish messages to the Kafka server and a Python consumer to receive messages from the server. Each message is about 10K bytes, far smaller than socket.request.max.bytes=104857600. Wha