[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825377#comment-17825377 ] lincoln lee commented on FLINK-25068: - [~Zhanghao Chen] Thanks for the updates. Closing this with changing the fix version back to 1.18 (as is in https://issues.apache.org/jira/browse/FLINK-32000). > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Assignee: zlzhang0122 >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.20.0 > > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825369#comment-17825369 ] Zhanghao Chen commented on FLINK-25068: --- Hi [~zlzhang0122][~lincoln.86xy], given that [FLINK-32000](https://issues.apache.org/jira/browse/FLINK-32000) has been resolved, we can close this issue now. > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Assignee: zlzhang0122 >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.20.0 > > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17529298#comment-17529298 ] zlzhang0122 commented on FLINK-25068: - [~gaoyunhaii] [~airblader] Can you help to review this pr? thanks! > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Assignee: zlzhang0122 >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.16.0 > > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510417#comment-17510417 ] zlzhang0122 commented on FLINK-25068: - [~chesnay] can you help to review the pr? > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Assignee: zlzhang0122 >Priority: Major > Labels: pull-request-available > Fix For: 1.16.0 > > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500462#comment-17500462 ] zlzhang0122 commented on FLINK-25068: - [~airblader] can you help to review the pr? > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Assignee: zlzhang0122 >Priority: Major > Labels: pull-request-available > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474408#comment-17474408 ] zlzhang0122 commented on FLINK-25068: - Since the FLink-12067 have been merged and the implemention can be more easily. [~trohrmann] Could you assign me the ticket? > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Priority: Major > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450809#comment-17450809 ] zlzhang0122 commented on FLINK-25068: - [~trohrmann] Nice advice!!! +1 for expose maximum parallelism on a per operator basis. I think we can put it in the Job Overview - Job Graph, just under the Parallelism and also in the Job Detail tab. > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Priority: Major > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450510#comment-17450510 ] Till Rohrmann commented on FLINK-25068: --- I think this is a good improvement. We might actually also want to expose on a per operator basis because individual operators can have different max parallelism values. > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Priority: Major > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450242#comment-17450242 ] zlzhang0122 commented on FLINK-25068: - [~Zhanghao Chen] Same as what i thought! We can add the max parallelism just under the job parallelism, name as job max parallelism.etc, and the implemetion is easy since we can add an attribute in JobConfigInfo and ArchivedExecutionConfig. [~trohrmann] [~chesnay] [~sewen] what do you think? If this implemetion has no problem, I can create a pr for this. > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Priority: Major > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25068) Show the maximum parallelism (number of key groups) of a job in Web UI
[ https://issues.apache.org/jira/browse/FLINK-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449837#comment-17449837 ] Zhanghao Chen commented on FLINK-25068: --- +1 for this feature > Show the maximum parallelism (number of key groups) of a job in Web UI > -- > > Key: FLINK-25068 > URL: https://issues.apache.org/jira/browse/FLINK-25068 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.14.0 >Reporter: zlzhang0122 >Priority: Major > > Now, Flink use maximum parallelism as the number of key groups to distribute > the key, the maximum parallelism can be set manually, or flink will set-up > automatically, sometimes the value is useful and we may want to know it, > maybe we can expose in the Web UI. > By doing this, we can easily know the max parallelism we can suggest the user > to scale when they are facing the leak of through-output, and we can know > which subtask will processing the special value and we can find the log soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
