Krish Iyer created KAFKA-3895:
---------------------------------
Summary: Implement a default queue for expired messages
Key: KAFKA-3895
URL: https://issues.apache.org/jira/browse/KAFKA-3895
Project: Kafka
Issue Type: New Feature
Components: core
Reporter: Krish Iyer
Priority: Minor
As discussed in the mailing list, kafka currently does not support a dead
letter queue-like feature where all the expired messages can be sent to.
A high-level design of such a feature can be as follows:
When kafka needs to expire a message from topic 'X', it sends the message to
'X-expired' topic, rather than deleting it. If 'X' is 'X-expired', dequeue and
requeue the message in the expired topic.
Given that this topic can receive duplicate messages (same expired message
multiple times), the consumer of this topic needs to ensure duplicate message
handling.
Open question:
Does the message need a timestamp to figure out when did it move from a normal
queue to a dead letter queue. Does it need a flag to specify whether it had
expired from the X-expired topic itself?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
