Hi, A pretty reasonable thing to do here would be to have a consumer that moved "old" events to another topic.
Kafka has no concept of an expired queue, the only thing it can do once a message is aged out is delete it. The deletion is done in bulk and typically is set to 24h or even higher (LinkedIn use 4 days, the default is 7 days). Thanks Tom Crayford Heroku Kafka On Thu, Jun 23, 2016 at 10:45 AM, Krish <krishnan.k.i...@gmail.com> wrote: > Hi, > I am trying to design a real-time application where message timeout can be > as low as a minute or two (message can get stale real-fast). > > In the rare chance that the consumers lag too far behind in processing > messages from the broker, is there a concept of expired message queue in > Kafka? > > I would like to know if a message has expired and then park it in some > topic till as such time that a service can dequeue, process it and/or > investigate it. > > Thanks. > > Best, > Krish >
