zentol opened a new pull request #13464: URL: https://github.com/apache/flink/pull/13464
Adds the `ResourceTracker` one of the components of the new declarative slot manager. The purpose of this tracker is to answer questions such as "How many resources does this job have?" or "How many resources does job still need?". To that end the tracker will be informed by the slot manager whenever the JobMaster has updated its `ResourceRequirements`, a resource was acquired (==TaskExecutor confirmed slot allocation) or a resource was lost (==JobMaster released slot or TaskExecutor failed). The default implementation is the `DefaultResourceTracker` which mostly delegates tracking work to the `JobScopedResourceTracker`. This tracker tracks the requirements and acquired resources for a single job. It heavily relies on 2 new data-structures: 1) `ResourceCounter`: Essentially a wrapper around `Map<ResourceProfile, Integer>, tracking the number of instances of a particular resource profile. This structure is used for tracking the requirements for a job, and acquired resources that exceed the requirements. It ensures that the counts are always positive. 2) `BiDirectionalResourceToRequirementMapping`: This structure tracks which resources are being used for which requirement, and which requirement is being (potentially partially) fulfilled by which resources. Essentially, this structure exists so that we can easily figure out which resources are no longer needed if requirements change, or inversely, which requirements require new slots if we lost one. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
