Guozhang Wang created KAFKA-6106:
------------------------------------
Summary: Postpone normal processing of tasks within a thread until
restoration of all tasks have completed
Key: KAFKA-6106
URL: https://issues.apache.org/jira/browse/KAFKA-6106
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 0.11.0.1, 1.0.0
Reporter: Guozhang Wang
Assignee: Guozhang Wang
Let's say a stream thread hosts multiple tasks, A and B. At the very beginning
when A and B are assigned to the thread, the thread state is
{{TASKS_ASSIGNED}}, and the thread start restoring these two tasks during this
state using the restore consumer while using normal consumer for heartbeating.
If task A's restoration has completed earlier than task B, then the thread will
start processing A immediately even when it is still in the {{TASKS_ASSIGNED}}
phase. But processing task A will slow down restoration of task B since it is
single-thread. So the thread's transition to {{RUNNING}} when all of its
assigned tasks have completed restoring and now can be processed will be
delayed.
Note that the streams instance's state will only transit to {{RUNNING}} when
all of its threads have transit to {{RUNNING}}, so the instance's transition
will also be delayed by this scenario.
We'd better to not start processing ready tasks immediately, but instead focus
on restoration during the {{TASKS_ASSIGNED}} state to shorten the overall time
of the instance's state transition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
