[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty
[
https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520131#comment-17520131
]
Clive Cox commented on KAFKA-13787:
---
Any update on this?
I see a file like kafka-streams-process-metadata is left behind after calling
cleanup().
the file contains text of the form (for me):
{code:java}
{"processId":"74a390fc-404d-4bc8-b564-f07a590a2b32"} {code}
Does this cause any issues?
> Failed to delete state store directory for it is not empty
> --
>
> Key: KAFKA-13787
> URL: https://issues.apache.org/jira/browse/KAFKA-13787
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 3.0.0
>Reporter: Nico Pommerening
>Priority: Major
> Attachments: bug-log.txt
>
>
> On Kafka Streams shutdown the Cleanup of state directories seems not to work,
> since the lock and metadata file seem not to be deleted.
> Relevant WARN logging:
> 2022-03-31 10:34:41,689 WARN [SpringApplicationShutdownHook]
> org.apache.kafka.streams.processor.internals.StateDirectory: stream-thread
> [SpringApplicationShutdownHook] Failed to delete state store directory of
> /kafka-streams-statestore/555b9965-95e3-4c92-b467-1d283428da5d/test-test-run-kpi
> for it is not empty
>
> Left over files in directory:
> * .lock
> * kafka-streams-process-metadata
>
> I'm not sure what the consequences of a unclean state cleanup are, but I
> would like to get rid of the Warning.
> I attached a bigger log extract and I've already patched the StateDirectory
> implementation which I'll try to contribute.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516068#comment-17516068 ] Matthias J. Sax commented on KAFKA-13787: - `StateDirectory` should have all relevant code. Not sure why it does not delete the `kafka-streams-process-metadata` file – for the `.lock` file it's ok, as we hold the lock all the time and it will be deleted when the directory is deleted. [~vvcephei] might now more about the file in question? > Failed to delete state store directory for it is not empty > -- > > Key: KAFKA-13787 > URL: https://issues.apache.org/jira/browse/KAFKA-13787 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 3.0.0 >Reporter: Nico Pommerening >Priority: Major > Attachments: bug-log.txt > > > On Kafka Streams shutdown the Cleanup of state directories seems not to work, > since the lock and metadata file seem not to be deleted. > Relevant WARN logging: > 2022-03-31 10:34:41,689 WARN [SpringApplicationShutdownHook] > org.apache.kafka.streams.processor.internals.StateDirectory: stream-thread > [SpringApplicationShutdownHook] Failed to delete state store directory of > /kafka-streams-statestore/555b9965-95e3-4c92-b467-1d283428da5d/test-test-run-kpi > for it is not empty > > Left over files in directory: > * .lock > * kafka-streams-process-metadata > > I'm not sure what the consequences of a unclean state cleanup are, but I > would like to get rid of the Warning. > I attached a bigger log extract and I've already patched the StateDirectory > implementation which I'll try to contribute. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515781#comment-17515781 ] Nico Pommerening commented on KAFKA-13787: -- Hi [~mjsax] : I'm experiencing the warnings on MacOs locally and Linux on Jenkins build server. I noticed the issue with deleting stuff under Windows, but I don't think this one relates since when I'm actually deleting the files with the PR, it works as expected. I did wonder, why this issue appears though and tried to find the part of code that was intended to delete these two files, but couldn't identify the clear spot in StateDirectory logics. Maybe you can also point me to the spot that should delete these two files, so I can dig deeper on why it's not working in our setup? > Failed to delete state store directory for it is not empty > -- > > Key: KAFKA-13787 > URL: https://issues.apache.org/jira/browse/KAFKA-13787 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 3.0.0 >Reporter: Nico Pommerening >Priority: Major > Attachments: bug-log.txt > > > On Kafka Streams shutdown the Cleanup of state directories seems not to work, > since the lock and metadata file seem not to be deleted. > Relevant WARN logging: > 2022-03-31 10:34:41,689 WARN [SpringApplicationShutdownHook] > org.apache.kafka.streams.processor.internals.StateDirectory: stream-thread > [SpringApplicationShutdownHook] Failed to delete state store directory of > /kafka-streams-statestore/555b9965-95e3-4c92-b467-1d283428da5d/test-test-run-kpi > for it is not empty > > Left over files in directory: > * .lock > * kafka-streams-process-metadata > > I'm not sure what the consequences of a unclean state cleanup are, but I > would like to get rid of the Warning. > I attached a bigger log extract and I've already patched the StateDirectory > implementation which I'll try to contribute. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515648#comment-17515648 ] Matthias J. Sax commented on KAFKA-13787: - What file system are you using? > Failed to delete state store directory for it is not empty > -- > > Key: KAFKA-13787 > URL: https://issues.apache.org/jira/browse/KAFKA-13787 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 3.0.0 >Reporter: Nico Pommerening >Priority: Major > Attachments: bug-log.txt > > > On Kafka Streams shutdown the Cleanup of state directories seems not to work, > since the lock and metadata file seem not to be deleted. > Relevant WARN logging: > 2022-03-31 10:34:41,689 WARN [SpringApplicationShutdownHook] > org.apache.kafka.streams.processor.internals.StateDirectory: stream-thread > [SpringApplicationShutdownHook] Failed to delete state store directory of > /kafka-streams-statestore/555b9965-95e3-4c92-b467-1d283428da5d/test-test-run-kpi > for it is not empty > > Left over files in directory: > * .lock > * kafka-streams-process-metadata > > I'm not sure what the consequences of a unclean state cleanup are, but I > would like to get rid of the Warning. > I attached a bigger log extract and I've already patched the StateDirectory > implementation which I'll try to contribute. -- This message was sent by Atlassian Jira (v8.20.1#820001)
