[ https://issues.apache.org/jira/browse/KAFKA-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400690#comment-16400690 ]
Matthias J. Sax commented on KAFKA-6647: ---------------------------------------- As pointed out by [~yuzhih...@gmail.com], the lock file does not prevent to delete the whole directory – it's just a "hint" that a thread is using the directory. Thus, you code works – however, if there is a running thread using the directory, you would delete the threads used files and the thread would die with an IOException. Thus, our `cleanUp` implementation checks if it can delete the directories safely. The question remains, why for your case, the lock files are not release properly... Not sure if this is related [https://stackoverflow.com/questions/34039846/filechannel-trylock-sometimes-throws-accessdeniedexception] Does the exception you get have a more detailed error message, why the operation is denied? > KafkaStreams.cleanUp creates .lock file in directory its trying to clean > ------------------------------------------------------------------------ > > Key: KAFKA-6647 > URL: https://issues.apache.org/jira/browse/KAFKA-6647 > Project: Kafka > Issue Type: Bug > Components: streams > Affects Versions: 1.0.1 > Environment: windows 10. > java version "1.8.0_162" > Java(TM) SE Runtime Environment (build 1.8.0_162-b12) > Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode) > org.apache.kafka:kafka-streams:1.0.1 > Kafka commitId : c0518aa65f25317e > Reporter: George Bloggs > Priority: Minor > Labels: streams > > When calling kafkaStreams.cleanUp() before starting a stream the > StateDirectory.cleanRemovedTasks() method contains this check: > {code:java} > ... Line 240 > if (lock(id, 0)) { > long now = time.milliseconds(); > long lastModifiedMs = taskDir.lastModified(); > if (now > lastModifiedMs + cleanupDelayMs) { > log.info("{} Deleting obsolete state directory {} > for task {} as {}ms has elapsed (cleanup delay is {}ms)", logPrefix(), > dirName, id, now - lastModifiedMs, cleanupDelayMs); > Utils.delete(taskDir); > } > } > {code} > The check for lock(id,0) will create a .lock file in the directory that > subsequently is going to be deleted. If the .lock file already exists from a > previous run the attempt to delete the .lock file fails with > AccessDeniedException. > This leaves the .lock file in the taskDir. Calling Utils.delete(taskDir) will > then attempt to remove the taskDir path calling Files.delete(path). > The call to files.delete(path) in postVisitDirectory will then fail > java.nio.file.DirectoryNotEmptyException as the failed attempt to delete the > .lock file left the directory not empty. (o.a.k.s.p.internals.StateDirectory > : stream-thread [restartedMain] Failed to lock the state directory due > to an unexpected exception) > This seems to then cause issues using streams from a topic to an inMemory > store. -- This message was sent by Atlassian JIRA (v7.6.3#76005)