[
https://issues.apache.org/jira/browse/KAFKA-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849888#comment-15849888
]
ASF GitHub Bot commented on KAFKA-4724:
---------------------------------------
GitHub user dguy opened a pull request:
https://github.com/apache/kafka/pull/2486
KAFKA-4724: Clean up of state directories can possibly remove stores that
are about to be used by another thread
Delay the cleanup of state directories that are not locked and not owned by
the current thread such that we only remove the directory if its last modified
is < now - cleanupDelayMs.
This also helps to avoid a race between threads on the same instance, where
during rebalance, one thread releases the lock on the state directory, and
before another thread can take the lock, the cleanup runs and removes the data.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dguy/kafka KAFKA-4724
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2486.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2486
----
commit 229471706dec638361777f00de195e7538a54c29
Author: Damian Guy <[email protected]>
Date: 2017-02-02T12:58:07Z
delay cleanup of state directories
----
> Clean up of state directories can possibly remove stores that are about to be
> used by another thread
> ----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4724
> URL: https://issues.apache.org/jira/browse/KAFKA-4724
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Damian Guy
> Assignee: Damian Guy
> Fix For: 0.10.3.0
>
>
> Currently, each {{StreamThread}} cleans up unused state directories based on
> the config {{StreamsConfig.STATE_CLEANUP_DELAY_MS_CONFIG}}.
> In situations where there are multiple threads configured this can lead to a
> race condition where the state directory gets removed by one thread when the
> task has been assigned (but not yet taken a lock) to another thread in the
> process.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
