Zhu Zhu created FLINK-14870:
-------------------------------
Summary: Introducing non-nullable assumption for task slot sharing
group
Key: FLINK-14870
URL: https://issues.apache.org/jira/browse/FLINK-14870
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: Zhu Zhu
In runtime, a null slot sharing group means the vertex should be run in an
individual slot which is not shared with other vertices.
However, at the moment, there is no way to set the slot sharing group of a
vertex to null in production. This is because {{StreamGraphGenerator}} would
always assign a non-null slot sharing group to an operator({{StreamNode}}), and
that would be the group of the generated {{JobVertex}}. (In the case of
{{DataSet}}, all vertices are in the "default" group.)
Moreover, currently users are already able to force an operator to run in an
individual slot by specify a different slot sharing group key to that operator.
So looks that supporting null slot sharing groups in runtime does not make much
sense. And it's making things more complex, with codes like {{@Nullable}}
annotations for slot sharing group, task required resources calculations, and
single task slot allocation.
To make runtime code base and future development simpler, I'd propose to
introduce the assumption that task slot sharing group should not be null.
The detailed proposal is:
1. In {{StreamingJobGraphGenerator}}, we check to ensure the slot sharing group
key of a {{StreamNode}} is non-null, so that we always set a non-null
{{SlotSharingGroup}} to a JobVertex
2. Remove all the {{@Nullable}} annotations for {{SlotSharingGroup}} and
{{SlotSharingGroupID}} in several classes
3. Always use the slot sharing group resources as the physical slot resources
for a task
4. Remove {{SchedulerImp#allocateSingleSlot}} and the related code paths
cc [~chesnay] [~gjy] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
