[
https://issues.apache.org/jira/browse/KAFKA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944838#comment-16944838
]
Sophie Blee-Goldman commented on KAFKA-8970:
--------------------------------------------
Maybe it would be better to explicitly not allow this case, either by literally
failing in the code or just by explaining it's not safe in the docs. (You can
of course still do so if you give each a separate state.dir).
But I'm still curious as to why you might want to do this in the first place?
One thing I could come up with off the top of my head is that IQ currently is
blocked until all tasks have been completely restored by all threads. If one
thread was significantly slower to restore than another, I suppose it could
reduce the availability gap for IQ. That's definitely something we want to fix
more holistically, so people don't need to circumvent the usual way of
operating.
> StateDirectory creation fails with Exception
> --------------------------------------------
>
> Key: KAFKA-8970
> URL: https://issues.apache.org/jira/browse/KAFKA-8970
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Nishkam Ravi
> Priority: Major
>
> When two threads try to create KafkaStreams simultaneously, one of them
> succeeds while the other fails with the following exception:
> org.apache.kafka.streams.errors.StreamsException:
> org.apache.kafka.streams.errors.ProcessorStateException: base state directory
> [/tmp/kafka-streams] doesn't exist and couldn't be created
> Quick investigation suggests that this is because the code at/around:
> [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L82]
> is not synchronized and can lead to race conditions.
> Specifying different values for state.dir can be a workaround for this issue
> but a bit cumbersome. Can we just make this synchronized?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
