Hi Team, I have below questions/ understandings for KafkaIO.Read in batch mode :
1. I built an understanding on debugging that, KafkaIO converts unbounded stream into bounded read and *buffers all records* till either of criteria matches - max records/ max time to read. If this understanding is correct, then read is memory intensive as KafkaIO has to buffer all read records before passing to down-streams. Is my understanding correct? 2. If #1 is correct, then is there any way we can keep writing records instead of buffering into memory in KafkaIO read operation (in batch mode) ? Thank You, Shrikant Bang
