[
https://issues.apache.org/jira/browse/YARN-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149505#comment-16149505
]
Junping Du edited comment on YARN-7138 at 8/31/17 8:12 PM:
-----------------------------------------------------------
Thanks [~leftnoteasy] for reply. I will downgrade this JIRA's priority to
unblock current progressing releases - as Wangda point out, this API is not get
public in our API doc.
While I agree in API definition it is a private one (as all yarn-server APIs)
so far. However, my second thoughts is we make YarnSchduler pluggable since
long time ago to build ecosystem for third parties to enhance/improve
schedulers on their scheduling scenario. Making resource scheduler pluggable
has been a long time strategy for hadoop even before YARN comes out. Other
competitive resource scheduling systems (like K8S, etc.) also has similar
design. I don't know since when CS and FS scheduler is the only option for
hadoop YARN but this break our promise to make scheduler pluggable - yes.
current yarn is still pluggable but if every upgrade can potentially breaking
third-party schedulers (given no guarantee of stable) then what this pluggable
meaning for?
was (Author: djp):
Thanks [~leftnoteasy] for reply. I agree in API definition it is a private one
(as all yarn-server APIs). However, my second thoughts is we make YarnSchduler
pluggable since long time ago to build ecosystem for third parties to
enhance/improve schedulers on their scheduling scenario. Making resource
scheduler pluggable has been a long time strategy for hadoop even before YARN
comes out. Other competitive resource scheduling systems (like K8S, etc.) also
has similar design. I don't know since when CS and FS scheduler is the only
option for hadoop YARN but this break our promise to make scheduler pluggable -
yes. current yarn is still pluggable but if every upgrade breaking third-party
schedulers then what this pluggable meaning for?
On the other side, I also check our history between 2.4 to 2.7.3 (
https://builds.apache.org/job/Hadoop-2.8-JACC/383/) and found out we indeed
break the same API at least once (or multiple times). so I will downgrade this
JIRA's priority to unblock current release and at the same time mark YARN-5521
as incompatible changes.
I believe since Hadoop 3, we should stabilize this API (and mark as public) to
make YARN scheduler real pluggable. I will start discussion thread on our dev
list soon.
> Fix incompatible API change for YarnScheduler involved by YARN-5521
> -------------------------------------------------------------------
>
> Key: YARN-7138
> URL: https://issues.apache.org/jira/browse/YARN-7138
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Reporter: Junping Du
> Priority: Critical
>
> From JACC report for 2.8.2 against 2.7.4, it indicates that we have
> incompatible changes happen in YarnScheduler:
> {noformat}
> hadoop-yarn-server-resourcemanager-2.7.4.jar, YarnScheduler.class
> package org.apache.hadoop.yarn.server.resourcemanager.scheduler
> YarnScheduler.allocate ( ApplicationAttemptId p1, List<ResourceRequest> p2,
> List<ContainerId> p3, List<String> p4, List<String> p5 ) [abstract] :
> Allocation
> {noformat}
> The root cause is YARN-5221. We should change it back or workaround this by
> adding back original API (mark as deprecated if not used any more).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
