[
https://issues.apache.org/jira/browse/KAFKA-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-10532:
----------------------------------
Description: Today whenever we are closing-dirty a task, we always wipe out
the state stores if we are under EOS. But when the closing task was a RESTORING
active, or a RUNNING standby, we may actually not need to wipe out the stores
since we know that upon resuming, we would still continue restoring the task
before transit to processing ever (assuming the LEO offset would not be
truncated), i.e. when they resumes it does not matter if the same records gets
applied twice during the continued restoration. (was: Today whenever we are
closing-dirty a task, we always wipe out the state stores if we are under EOS.
But when the closing task was a RESTORING active, or a RUNNING standby, we may
actually not need to wipe out the stores since we know that upon resuming, we
would still restore the task before transit to processing (assuming the LEO
offset would not be truncated), i.e. when they resumes it does not matter if
the same records gets applied twice during the continued restoration.)
> Do not wipe state store under EOS when closing-dirty a RESTORING active or
> RUNNING standby task
> -----------------------------------------------------------------------------------------------
>
> Key: KAFKA-10532
> URL: https://issues.apache.org/jira/browse/KAFKA-10532
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
>
> Today whenever we are closing-dirty a task, we always wipe out the state
> stores if we are under EOS. But when the closing task was a RESTORING active,
> or a RUNNING standby, we may actually not need to wipe out the stores since
> we know that upon resuming, we would still continue restoring the task before
> transit to processing ever (assuming the LEO offset would not be truncated),
> i.e. when they resumes it does not matter if the same records gets applied
> twice during the continued restoration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
