Re: high level consumer memory footprint

2015-06-25 Thread Kris K
Hi, I found that the consumer config param fetch.message.max.bytes is set to 100 MB on the consumer and I think this is what caused the problem. It would really be helpful if anyone can explain how much memory the consumer (running 100 threads) is going to need for consuming 100K 1 MB messages

high level consumer memory footprint

2015-06-23 Thread Kris K
Hi, I was just wondering if there is any difference in the memory footprint of a high level consumer when: 1. the consumer is live and continuously consuming messages with no backlogs 2. when the consumer is down for quite some time and needs to be brought up to clear the backlog. My test case