[
https://issues.apache.org/jira/browse/KAFKA-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-12460:
----------------------------------
Description: Eventually we will have to come up with some approach to
recover from committed data loss in the raft quorum (something akin to unclean
leader election for normal partitions). For now, we would rather be stricter
and fail fast rather than allowing committed data to be silently lost.
Specifically, we can prevent any attempt to truncate below the high watermark
since this is a clear indication of data loss. The long term thought I have in
mind is to give users an --unsafe flag or something like that which can be
passed at startup in order to knowingly turn off the stricter validation in
order to let the quorum recover from a disaster scenario. This needs some more
thought though. (was: Eventually we will have to come up with some approach to
recover from committed data loss in the raft quorum (something akin to unclean
leader election for normal partitions). For now, we would rather be stricter
and fail fast rather than allowing committed data to be silently lost.
Specifically, we can prevent any attempt to truncate below the high watermark
since this is a clear indication of data loss. The long term thought I have in
mind is to give users an --unsafe flag or something like that which can be
passed at startup in order to knowingly turn of the stricter validation in
order to let the quorum recover from a disaster scenario. This needs some more
thought though.)
> Raft should prevent truncation below high watermark
> ---------------------------------------------------
>
> Key: KAFKA-12460
> URL: https://issues.apache.org/jira/browse/KAFKA-12460
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Major
>
> Eventually we will have to come up with some approach to recover from
> committed data loss in the raft quorum (something akin to unclean leader
> election for normal partitions). For now, we would rather be stricter and
> fail fast rather than allowing committed data to be silently lost.
> Specifically, we can prevent any attempt to truncate below the high watermark
> since this is a clear indication of data loss. The long term thought I have
> in mind is to give users an --unsafe flag or something like that which can be
> passed at startup in order to knowingly turn off the stricter validation in
> order to let the quorum recover from a disaster scenario. This needs some
> more thought though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
