[ https://issues.apache.org/jira/browse/FLINK-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhu Zhu updated FLINK-14536: ---------------------------- Description: I'm raising this question because I find {{cpuCores}} in {{ResourceSpec#merge}} are aggregated with {{max()}}, while in {{ResourceProfile#merge}} it is {{sum()}}. This means that when calculating resources of a vertex from within operators, the {{cpuCores}} is the max value. While it is a sum(or subtraction in {{ResourceProfile#subtract}}) when dealing with shared slot bookkeeping and related checks. This is confusing to me, especially when I'm considering how to generate a shared slot resource spec merged from all vertices in it(see FLINK-14314). I'm not pretty sure if we already have a concise definition for it? If there is, we need to respect it and change {{ResourceSpec}} or {{ResourceProfile}} accordingly. If not, we need to decide it first before we can move on with fine grained resources. Need to mention that if we take the {{max()}} way, we need to re-consider how we can conduct a correct {{ResourceProfile#subtract}} regarding {{cpuCores}}, since there is not a clear way to reverse a {{max()}} computation. cc [~trohrmann] [~xintongsong] was: I'm raising this question because I find {{cpuCores}} in {{ResourceSpec#merge}} are aggregated with {{max()}}, while in {{ResourceProfile#merge}} it is {{sum()}}. This means that when calculating resources of a vertex from within operators, the {{cpuCores}} is the max value. While it is a sum(or subtraction in {{ResourceProfile#subtract}}) when dealing with shared slot bookkeeping and related checks. This is confusing to me, especially when I'm considering how to generate a shared slot resource spec merged from all vertices in it(see FLINK-14314). I'm not pretty sure if we already have a concise definition for it? If there is, we need to respect it and change {{ResourceSpec}} or {{ResourceProfile}} accordingly. If not, we need to decide it first before we can move on with fine grained resources. Need to mention that if we take the {{max()}} way, we need to re-consider how we can conduct a correct {{ResourceProfile#subtract}} regarding {{cpuCores}}, since there is not a clear way to reverse a {{max()}} computation. cc [~trohrmann] [~wuzang] > Make clear the way to aggregate specified cpuCores resources > ------------------------------------------------------------ > > Key: FLINK-14536 > URL: https://issues.apache.org/jira/browse/FLINK-14536 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination > Affects Versions: 1.10.0 > Reporter: Zhu Zhu > Priority: Major > Fix For: 1.10.0 > > > I'm raising this question because I find {{cpuCores}} in > {{ResourceSpec#merge}} are aggregated with {{max()}}, while in > {{ResourceProfile#merge}} it is {{sum()}}. > This means that when calculating resources of a vertex from within operators, > the {{cpuCores}} is the max value. While it is a sum(or subtraction in > {{ResourceProfile#subtract}}) when dealing with shared slot bookkeeping and > related checks. > This is confusing to me, especially when I'm considering how to generate a > shared slot resource spec merged from all vertices in it(see FLINK-14314). > I'm not pretty sure if we already have a concise definition for it? > If there is, we need to respect it and change {{ResourceSpec}} or > {{ResourceProfile}} accordingly. > If not, we need to decide it first before we can move on with fine grained > resources. > Need to mention that if we take the {{max()}} way, we need to re-consider how > we can conduct a correct {{ResourceProfile#subtract}} regarding {{cpuCores}}, > since there is not a clear way to reverse a {{max()}} computation. > cc [~trohrmann] [~xintongsong] -- This message was sent by Atlassian Jira (v8.3.4#803005)