[
https://issues.apache.org/jira/browse/KAFKA-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399347#comment-16399347
]
Guozhang Wang commented on KAFKA-6647:
--------------------------------------
Hi George,
By looking into your issue, I think the root cause maybe that there is still
some un-closed handle on that file in which Windows 10 would not actually
delete the file and hence future attempt to recreate the file will fail with
ERROR_ACCESS_DENIED. I'm not an expert in Windows but I find the following may
be relevant:
https://stackoverflow.com/questions/31606978/odd-behaviour-when-deleting-files-with-files-delete
To verify if that is indeed the case, could you share your code snippet
including the shutdown hook implementation for us to try to re-produce in
Windows?
BTW regarding to your PR, changing the state directory structure such as
lifting the lock file on level up would not be a backward compatible change,
since if users upgrade their streams library version to the one that includes
this change their code will be broken. So I'd suggest we hold on the proposed
PR and try to investigate further what actually causes
{{AccessDeniedException}}.
> KafkaStreams.cleanUp creates .lock file in directory its trying to clean
> ------------------------------------------------------------------------
>
> Key: KAFKA-6647
> URL: https://issues.apache.org/jira/browse/KAFKA-6647
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 1.0.1
> Environment: windows 10.
> java version "1.8.0_162"
> Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
> org.apache.kafka:kafka-streams:1.0.1
> Kafka commitId : c0518aa65f25317e
> Reporter: George Bloggs
> Priority: Minor
> Labels: streams
>
> When calling kafkaStreams.cleanUp() before starting a stream the
> StateDirectory.cleanRemovedTasks() method contains this check:
> {code:java}
> ... Line 240
> if (lock(id, 0)) {
> long now = time.milliseconds();
> long lastModifiedMs = taskDir.lastModified();
> if (now > lastModifiedMs + cleanupDelayMs) {
> log.info("{} Deleting obsolete state directory {}
> for task {} as {}ms has elapsed (cleanup delay is {}ms)", logPrefix(),
> dirName, id, now - lastModifiedMs, cleanupDelayMs);
> Utils.delete(taskDir);
> }
> }
> {code}
> The check for lock(id,0) will create a .lock file in the directory that
> subsequently is going to be deleted. If the .lock file already exists from a
> previous run the attempt to delete the .lock file fails with
> AccessDeniedException.
> This leaves the .lock file in the taskDir. Calling Utils.delete(taskDir) will
> then attempt to remove the taskDir path calling Files.delete(path).
> The call to files.delete(path) in postVisitDirectory will then fail
> java.nio.file.DirectoryNotEmptyException as the failed attempt to delete the
> .lock file left the directory not empty. (o.a.k.s.p.internals.StateDirectory
> : stream-thread [restartedMain] Failed to lock the state directory due
> to an unexpected exception)
> This seems to then cause issues using streams from a topic to an inMemory
> store.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
