Hi Ricardo, Kafka Streams should handle that case as well. What streams config are you using, could you share it? There is one parameter that is called “ConsumerConfig.AUTO_OFFSET_RESET_CONFIG” and by default it’s set to “earliest”. Any chance your app has changed it to “latest”?
Thanks Eno > On Aug 12, 2017, at 5:13 PM, Ricardo Costa <rdsco...@gmail.com> wrote: > > Hi, > > I've implemented a forwarding consumer which literally just consumes the > messages from a source topic, logs them and then publishes them to a target > topic. > > I wanted to keep the implementation simple with very little code so I went > with kafka-streams. I have a really simple topology with a source for the > source topic, a sink for the target topic and a logging processor > in-between. > > I'm quite happy with the solution, really simple and elegant, I ran some > basic tests and everything seemed to be working. As I went on to build more > test cases, I found that the stream only does its thing if I push messages > to the source topic *after* creating the stream and waiting until it is > fully initialized. Is this the expected behaviour? I need the stream to be > started at any point in time and forward the messages that were buffered on > the source topic until then. Are kafka-streams not fit for this use case? > Or am I missing something? > > Thanks in advance! > > -- > Ricardo
