[ https://issues.apache.org/jira/browse/KAFKA-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Gustafson updated KAFKA-12460: ------------------------------------ Description: Eventually we will have to come up with some approach to recover from committed data loss in the raft quorum (something akin to unclean leader election for normal partitions). For now, we would rather be stricter and fail fast rather than allowing committed data to be silently lost. Specifically, we can prevent any attempt to truncate below the high watermark since this is a clear indication of data loss. The long term thought I have in mind is to give users an --unsafe flag or something like that which can be passed at startup in order to knowingly turn of the stricter validation in order to let the quorum recover from a disaster scenario. This needs some more thought though. (was: Eventually we will have to come up with some approach to recover from committed data loss in the raft quorum (something akin to unclean leader election for normal partitions). For now, we would rather be stricter and fail fast rather than allowing committed data to be silently lost. The long term thought I have in mind is to give users an --unsafe flag or something like that which can be passed at startup in order to knowingly turn of the stricter validation in order to let the quorum recover from a disaster scenario. This needs some more thought though.) > Raft should prevent truncation below high watermark > --------------------------------------------------- > > Key: KAFKA-12460 > URL: https://issues.apache.org/jira/browse/KAFKA-12460 > Project: Kafka > Issue Type: Sub-task > Reporter: Jason Gustafson > Assignee: Jason Gustafson > Priority: Major > > Eventually we will have to come up with some approach to recover from > committed data loss in the raft quorum (something akin to unclean leader > election for normal partitions). For now, we would rather be stricter and > fail fast rather than allowing committed data to be silently lost. > Specifically, we can prevent any attempt to truncate below the high watermark > since this is a clear indication of data loss. The long term thought I have > in mind is to give users an --unsafe flag or something like that which can be > passed at startup in order to knowingly turn of the stricter validation in > order to let the quorum recover from a disaster scenario. This needs some > more thought though. -- This message was sent by Atlassian Jira (v8.3.4#803005)