Dinesh,
I took a look at your logs, first it seems error_logs_kafka_request.log is
also from the consumer side, not the server side.
And the error logs are pointing to an EOF on the server side while reading
the data, one possibility is that your socket buffer size is configured to
be not as larg
Could you check both the server logs and the consumer logs (with and
without the config specified) and see if there are any error entries /
exception logs?
Guozhang
On Wed, Jan 14, 2015 at 1:53 PM, dinesh kumar wrote:
> We don't have any compression on Kafka
>
> On 14 January 2015 at 22:54, Guo
We don't have any compression on Kafka
On 14 January 2015 at 22:54, Guozhang Wang wrote:
> Did you have compression enabled on Kafka?
>
> On Tue, Jan 13, 2015 at 10:33 AM, dinesh kumar
> wrote:
>
> > We are using 0.8.1.1 version of Kafka and *not 0.8.2 *as mentioned above.
> >
> > Thanks,
> > D
Did you have compression enabled on Kafka?
On Tue, Jan 13, 2015 at 10:33 AM, dinesh kumar wrote:
> We are using 0.8.1.1 version of Kafka and *not 0.8.2 *as mentioned above.
>
> Thanks,
> Dinesh
>
> On 13 January 2015 at 23:35, dinesh kumar wrote:
>
> > Hi Guozhang,
> > Sorry for the misinformat
We are using 0.8.1.1 version of Kafka and *not 0.8.2 *as mentioned above.
Thanks,
Dinesh
On 13 January 2015 at 23:35, dinesh kumar wrote:
> Hi Guozhang,
> Sorry for the misinformation. We have file sizes around 50 - 100 MB. So we
> set *fetch.message.max.bytes* conservatively around 188743680.
Hi Guozhang,
Sorry for the misinformation. We have file sizes around 50 - 100 MB. So we
set *fetch.message.max.bytes* conservatively around 188743680. Can you
please explain me the reason for this behavior?
Thanks,
Dinesh
On 13 January 2015 at 21:42, Guozhang Wang wrote:
> Dinesh,
>
> Your fet
Dinesh,
Your fetch.message.max.bytes is 188743680 < 155MB, but you said some
messages can be as large as 180MB. Could you try to set it to be larger
than, say 200MB and see if it helps?
Guozhang
On Tue, Jan 13, 2015 at 4:18 AM, dinesh kumar wrote:
> Hi,
> I am been facing some JAVA high level
Hi,
I am been facing some JAVA high level consumer related issues lately and
would like to understand more on this.
We have 9 bare-metals (48 core, 250 GB, Terabytes of Hard disks) running *Kafka
0.8.2* and 5 independent VM (8 core, 60 GB) running zookeeper.
I have a topic that has key as metadat
