Jin Xing created FLINK-22674: -------------------------------- Summary: Provide JobID when apply shuffle resource by ShuffleMaster Key: FLINK-22674 URL: https://issues.apache.org/jira/browse/FLINK-22674 Project: Flink Issue Type: Sub-task Components: Runtime / Network Reporter: Jin Xing
In current Flink 'pluggable shuffle service' framework, only PartitionDescriptor and ProducerDescriptor are included as parameters in ShuffleMaster#registerPartitionWithProducer. But when extending a remote shuffle service based on 'pluggable shuffle service', JobID is also needed when apply shuffle resource from remote cluster. It can be used as an identification to link shuffle resource with the corresponding job: # Remote shuffle cluster can isolate or do capacity control on shuffle resource between jobs; # Remote shuffle cluster can use JobID for shuffle data cleanup when job is lost thus to avoid file leak; -- This message was sent by Atlassian Jira (v8.3.4#803005)