Re: No more message scenario question

2016-05-30 Thread Tauzell, Dave
Also check out writing a custom partitioner. You could figure out the number of partitions and then use a custom partitioner to write a stop message to each. I would consider looking into not needing this functionality at all if possible. Dave > On May 29, 2016, at 15:18, Tauzell, Dave wro

Re: No more message scenario question

2016-05-29 Thread Tauzell, Dave
If you use keys it might be possible to write a "no more messages" message to each partition. Dave. > On May 29, 2016, at 01:51, David Olsen wrote: > > I understand that the reception of Kafka messages may be out of > order[1][2]. But I have a multiple producers/ consumers scenario where I > wo

No more message scenario question

2016-05-28 Thread David Olsen
I understand that the reception of Kafka messages may be out of order[1][2]. But I have a multiple producers/ consumers scenario where I would need to know when the last record is processed at the producer side, so once consumers process e.g. NoMoreMessage, the entire system can react such as stopp