[
https://issues.apache.org/jira/browse/KAFKA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944889#comment-16944889
]
Sophie Blee-Goldman commented on KAFKA-8970:
--------------------------------------------
Ah ok, thanks for clarifying! I agree it wouldn't make much sense :)
In that case though I believe they definitely should have different state
directories. The reason being, they shouldn't really be sharing any state. But
they could end up overwriting each other's state and/or checkpoint files,
pretty much corrupting everything.
The reasoning above still sort of applies here: each instance keeps track of
its own task subdirectories and locks/owners, so threads from different apps
could each "own" a subdirectory. And they're named according to the task naming
scheme, like "1_0" or "2_1", so its pretty likely each app would be looking
at/using the same task subdirectories.
> StateDirectory creation fails with Exception
> --------------------------------------------
>
> Key: KAFKA-8970
> URL: https://issues.apache.org/jira/browse/KAFKA-8970
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Nishkam Ravi
> Priority: Major
>
> When two threads try to create KafkaStreams simultaneously, one of them
> succeeds while the other fails with the following exception:
> org.apache.kafka.streams.errors.StreamsException:
> org.apache.kafka.streams.errors.ProcessorStateException: base state directory
> [/tmp/kafka-streams] doesn't exist and couldn't be created
> Quick investigation suggests that this is because the code at/around:
> [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L82]
> is not synchronized and can lead to race conditions.
> Specifying different values for state.dir can be a workaround for this issue
> but a bit cumbersome. Can we just make this synchronized?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
