fetchSizeBytes is equivalent kafka consumer’s fetch.message.max.bytes . This should be at a minimum equal to server’s max.message.bytes or more. In your case it should be at least 10MB. Also you need to increase buffersizebytes as well. -- Harsha
On March 26, 2015 at 8:31:39 AM, François Méthot ([email protected]) wrote: Hi, We have been having issue with reading from our Topic. The topic would stop reading from the queue without warning or error. It would be stuck at a specific offset of a specific partition. After a lot of debugging and testing, we found out that by increasing the SpoutConfig.fetchSizeBytes from the default 1MB to 2MB or more, the problem would go away. It would be able to consume the offset where it used to stop. To what value should fetchSizeBytes be set? Should we have seen a warning/error in the log when the spout can`t iterate through message within the read bytebuffer? In kafka, we have message.max.bytes and replica.fetch.max.bytes set to 10MB, is there a link between those setting and the fetchSizeBytes. Thanks a lot François
