Tom, When you say this: "Deletion can happen at different times on the different replicas of the log, and to different messages. Whilst a consumer will only be reading from the lead broker for any log at any one time, the leader can and will change to handle broker failure."
basically it means that a dead-letter Q like functionality can get multiple messages, and anyone draining the queue needs to take care of duplicates. Right so far? -- κρισhναν On Thu, Jun 23, 2016 at 9:21 PM, Krish <[email protected]> wrote: > Well, we are already using Kafka and would like to get this feature. > How hard can it be to hack it and use a custom kafka!? ;) > > Let me look up the source code (never have checked it) and see what can be > done. > Thanks Tom and Christian, for helping me decide fast. > > > > -- > κρισhναν > > On Thu, Jun 23, 2016 at 9:16 PM, Christian Posta < > [email protected]> wrote: > >> Sounds like something a traditional message broker (ie, ActiveMQ) would >> be able to do with a TTL setting and expiry. Expired messages get moved to >> a DLQ. >> >> On Thu, Jun 23, 2016 at 2:45 AM, Krish <[email protected]> wrote: >> >>> Hi, >>> I am trying to design a real-time application where message timeout can >>> be >>> as low as a minute or two (message can get stale real-fast). >>> >>> In the rare chance that the consumers lag too far behind in processing >>> messages from the broker, is there a concept of expired message queue in >>> Kafka? >>> >>> I would like to know if a message has expired and then park it in some >>> topic till as such time that a service can dequeue, process it and/or >>> investigate it. >>> >>> Thanks. >>> >>> Best, >>> Krish >>> >> >> >> >> -- >> *Christian Posta* >> twitter: @christianposta >> http://www.christianposta.com/blog >> http://fabric8.io >> >> >
