walterddr commented on a change in pull request #9336:
[FLINK-13548][Deployment/YARN]Support priority of the Flink YARN application
URL: https://github.com/apache/flink/pull/9336#discussion_r314567473
##########
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/configuration/YarnConfigOptions.java
##########
@@ -180,6 +180,25 @@
" Flink on YARN on an environment with a restrictive
firewall, this option allows specifying a range of" +
" allowed ports.");
+ /**
+ * A non-negative integer indicating the priority for submitting a
Flink YARN application. It will only take effect
+ * if the Hadoop version >= 2.8.5 and YARN priority scheduling setting
is enabled. Larger integer corresponds with
+ * higher priority. If priority is negative or set to '-1'(default),
Flink will unset yarn priority setting and use
+ * cluster default priority.
+ *
+ * @see <a
href="https://hadoop.apache.org/docs/r2.8.5/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html";>YARN
Capacity Scheduling Doc</a>
+ */
+ public static final ConfigOption<Integer> APPLICATION_PRIORITY =
+ key("yarn.application.priority")
+ .defaultValue(-1)
+ .withDescription(Description.builder()
+ .text("A non-negative integer indicating the
priority for submitting a Flink YARN application. It" +
+ " will only take effect if the Hadoop
version >= 2.8.5 and YARN priority scheduling setting is enabled." +
Review comment:
@tillrohrmann Yes. I saw that the API was there for a while. However,
according the documentation, priority scheduling only appears in YARN
documentation after 2.8.x. In fact I tried the integration test in
`flink-yarn-test` module by compiling against default (2.4.x?) and 2.8.x.
My finding was: only 2.8.x tests has specific logs indicating that priority
setting has taken effect:
```
2019-08-15 19:46:52,182 INFO org.apache.flink.yarn.YarnResourceManager
- Received new container: container_1565923587220_0001_01_000002
- Remaining pending container requests: 1
2019-08-15 19:46:52,184 INFO org.apache.flink.yarn.YarnResourceManager
- Removing container request Capability[<memory:1024,
vCores:1>]**Priority[1]**. Pending container requests 0.
```
similar logs does not exist on 2.4.x.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
