bbejeck commented on code in PR #18732:
URL: https://github.com/apache/kafka/pull/18732#discussion_r1933015786
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java:
##########
@@ -295,6 +294,9 @@ private void closeStartupTasks(final Predicate<Task>
predicate) {
// now that we have exclusive ownership of the drained tasks,
close them
for (final Task task : drainedTasks) {
+ // main thread locked the task initially on startup, but has
moved on and will not unlock
+ // so we need to explicitly swap lock ownership here as this
method is called by a StreamThread
+ lockedTasksToOwner.replace(task.id(), Thread.currentThread());
Review Comment:
This is the fix. Once a `StreamThread` receives its assignment, it will
close the startup tasks. But during the closing process, the
`StandbyTask.closeClean()` method will eventually call the
`StatemanagerUtil.closeStateManager` method which needs to lock the state
directory, but locking requires the calling thread be the current owner of the
state directory. Since the main thread grabs the lock on startup but moves on
without releasing it, we need to update ownership explicitly here in order for
the stream thread to close the startup task and begin processing.
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java:
##########
@@ -107,7 +106,7 @@ public StateDirectoryProcessFile() {
private final boolean hasPersistentStores;
private final boolean hasNamedTopologies;
- private final HashMap<TaskId, Thread> lockedTasksToOwner = new HashMap<>();
+ private final ConcurrentMap<TaskId, Thread> lockedTasksToOwner = new
ConcurrentHashMap<>();
Review Comment:
side fix : with different thread accessing this map I think it makes sense
to change it to a `ConcurrentMap`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
