[jira] [Commented] (YARN-3005) [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java
[
https://issues.apache.org/jira/browse/YARN-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272400#comment-14272400
]
Akira AJISAKA commented on YARN-3005:
-
Thanks for the patch [~sekikn]. Mostly looks good to me. Minor nit:
{code}
throw new ServiceStateException(E_UNKNOWN_AUTHENTICATION_MECHANISM
- + "\"" + auth + "\"");
+ + "\"" + auth + "\"");
{code}
Would you keep the original style?
> [JDK7] Use switch statement for String instead of if-else statement in
> RegistrySecurity.java
>
>
> Key: YARN-3005
> URL: https://issues.apache.org/jira/browse/YARN-3005
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Akira AJISAKA
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-3005.001.patch
>
>
> Since we have moved to JDK7, we can refactor the below if-else statement for
> String.
> {code}
> // TODO JDK7 SWITCH
> if (REGISTRY_CLIENT_AUTH_KERBEROS.equals(auth)) {
> access = AccessPolicy.sasl;
> } else if (REGISTRY_CLIENT_AUTH_DIGEST.equals(auth)) {
> access = AccessPolicy.digest;
> } else if (REGISTRY_CLIENT_AUTH_ANONYMOUS.equals(auth)) {
> access = AccessPolicy.anon;
> } else {
> throw new ServiceStateException(E_UNKNOWN_AUTHENTICATION_MECHANISM
> + "\"" + auth + "\"");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3005) [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java
[
https://issues.apache.org/jira/browse/YARN-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-3005:
Affects Version/s: 2.7.0
> [JDK7] Use switch statement for String instead of if-else statement in
> RegistrySecurity.java
>
>
> Key: YARN-3005
> URL: https://issues.apache.org/jira/browse/YARN-3005
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Akira AJISAKA
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-3005.001.patch
>
>
> Since we have moved to JDK7, we can refactor the below if-else statement for
> String.
> {code}
> // TODO JDK7 SWITCH
> if (REGISTRY_CLIENT_AUTH_KERBEROS.equals(auth)) {
> access = AccessPolicy.sasl;
> } else if (REGISTRY_CLIENT_AUTH_DIGEST.equals(auth)) {
> access = AccessPolicy.digest;
> } else if (REGISTRY_CLIENT_AUTH_ANONYMOUS.equals(auth)) {
> access = AccessPolicy.anon;
> } else {
> throw new ServiceStateException(E_UNKNOWN_AUTHENTICATION_MECHANISM
> + "\"" + auth + "\"");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272456#comment-14272456
]
Hudson commented on YARN-3014:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #69 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/69/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272455#comment-14272455 ] Hudson commented on YARN-3002: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #69 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/69/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272454#comment-14272454 ] Hudson commented on YARN-2956: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #69 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/69/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-project/src/site/site.xml * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3025) Provide API for retrieving blacklisted nodes
[
https://issues.apache.org/jira/browse/YARN-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272462#comment-14272462
]
Steve Loughran commented on YARN-3025:
--
Bikas,
When our AM restarts, it loses all history of which nodes were unreliable.
At least on restart, give us the list of last-preserved state.
> Provide API for retrieving blacklisted nodes
>
>
> Key: YARN-3025
> URL: https://issues.apache.org/jira/browse/YARN-3025
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Ted Yu
>
> We have the following method which updates blacklist:
> {code}
> public synchronized void updateBlacklist(List blacklistAdditions,
> List blacklistRemovals) {
> {code}
> Upon AM failover, there should be an API which returns the blacklisted nodes
> so that the new AM can make consistent decisions.
> The new API can be:
> {code}
> public synchronized List getBlacklistedNodes()
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3020) n similar addContainerRequest()s produce n*(n+1)/2 containers
[ https://issues.apache.org/jira/browse/YARN-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272463#comment-14272463 ] Steve Loughran commented on YARN-3020: -- getting back more than you asked for is a situation I've encountered; when doing a code review of that bit of Slider, the Samza team commented on it too. This could be a root cause > n similar addContainerRequest()s produce n*(n+1)/2 containers > - > > Key: YARN-3020 > URL: https://issues.apache.org/jira/browse/YARN-3020 > Project: Hadoop YARN > Issue Type: Bug > Components: client >Affects Versions: 2.5.0, 2.6.0, 2.5.1, 2.5.2 >Reporter: Peter D Kirchner > Original Estimate: 24h > Remaining Estimate: 24h > > BUG: If the application master calls addContainerRequest() n times, but with > the same priority, I get up to 1+2+3+...+n containers = n*(n+1)/2 . The most > containers are requested when the interval between calls to > addContainerRequest() exceeds the heartbeat interval of calls to allocate() > (in AMRMClientImpl's run() method). > If the application master calls addContainerRequest() n times, but with a > unique priority each time, I get n containers (as I intended). > Analysis: > There is a logic problem in AMRMClientImpl.java. > Although AMRMClientImpl.java, allocate() does an ask.clear() , on subsequent > calls to addContainerRequest(), addResourceRequest() finds the previous > matching remoteRequest and increments the container count rather than > starting anew, and does an addResourceRequestToAsk() which defeats the > ask.clear(). > From documentation and code comments, it was hard for me to discern the > intended behavior of the API, but the inconsistency reported in this issue > suggests one case or the other is implemented incorrectly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272469#comment-14272469 ] Hudson commented on YARN-3002: -- FAILURE: Integrated in Hadoop-Yarn-trunk #803 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/803/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm * hadoop-yarn-project/CHANGES.txt > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272470#comment-14272470
]
Hudson commented on YARN-3014:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #803 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/803/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272468#comment-14272468 ] Hudson commented on YARN-2956: -- FAILURE: Integrated in Hadoop-Yarn-trunk #803 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/803/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-project/src/site/site.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272520#comment-14272520 ] Hudson commented on YARN-2956: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2001 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2001/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-project/src/site/site.xml * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272521#comment-14272521 ] Hudson commented on YARN-3002: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2001 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2001/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272522#comment-14272522
]
Hudson commented on YARN-3014:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #2001 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2001/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272528#comment-14272528 ] Hudson commented on YARN-2956: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #66 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/66/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-project/src/site/site.xml * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272530#comment-14272530
]
Hudson commented on YARN-3014:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #66 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/66/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272529#comment-14272529 ] Hudson commented on YARN-3002: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #66 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/66/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm * hadoop-yarn-project/CHANGES.txt > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272539#comment-14272539
]
Hudson commented on YARN-3014:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #70 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/70/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272537#comment-14272537 ] Hudson commented on YARN-2956: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #70 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/70/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-project/src/site/site.xml * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272538#comment-14272538 ] Hudson commented on YARN-3002: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #70 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/70/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm * hadoop-yarn-project/CHANGES.txt > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3002) YARN documentation needs updating post-shell rewrite
[ https://issues.apache.org/jira/browse/YARN-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272546#comment-14272546 ] Hudson commented on YARN-3002: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2020 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2020/]) YARN-3002. YARN documentation needs updating post-shell rewrite (aw) (aw: rev ab5cd99177f8e92ef851104c2d25001ac29b0240) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/TimelineServer.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm * hadoop-yarn-project/CHANGES.txt > YARN documentation needs updating post-shell rewrite > > > Key: YARN-3002 > URL: https://issues.apache.org/jira/browse/YARN-3002 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0 > > Attachments: YARN-3002-00.patch, YARN-3002-01.patch > > > After HADOOP-9902, the YARN documentation is out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3014) Replaces labels on a host should update all NM's labels on that host
[
https://issues.apache.org/jira/browse/YARN-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272547#comment-14272547
]
Hudson commented on YARN-3014:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2020 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2020/])
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. Contributed by Wangda Tan (jianhe: rev
a2604062681230442eefae79815db5d77ccc4c2e)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestCommonNodeLabelsManager.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsManager.java
> Replaces labels on a host should update all NM's labels on that host
>
>
> Key: YARN-3014
> URL: https://issues.apache.org/jira/browse/YARN-3014
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.7.0
>
> Attachments: YARN-3014-1.patch, YARN-3014-2.patch, YARN-3014-3.patch
>
>
> Admin can either specify labels on a host (by running {{yarn rmadmin
> -replaceLabelsOnNode host1,label1}}) OR on a single NM (by running {{yarn
> rmadmin -replaceLabelsOnNode host1:port,label1}}).
> If user has specified label=x on a NM (instead of host), and later set the
> label=y on host of the NM. NM's label should update to y as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2956) Some yarn-site index linked pages are difficult to discover because are not in the side bar
[ https://issues.apache.org/jira/browse/YARN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272545#comment-14272545 ] Hudson commented on YARN-2956: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2020 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2020/]) YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki (jianhe: rev 27a489cba63d9473e4cbcaef018b66108fb6c122) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm * hadoop-project/src/site/site.xml * hadoop-yarn-project/CHANGES.txt > Some yarn-site index linked pages are difficult to discover because are not > in the side bar > --- > > Key: YARN-2956 > URL: https://issues.apache.org/jira/browse/YARN-2956 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Remus Rusanu >Assignee: Masatake Iwasaki >Priority: Minor > Labels: documentation > Fix For: 2.7.0 > > Attachments: YARN-2956.1.patch, YARN-2956.2.patch > > > The yarn-site index.apt.vm page is difficult to 'stumble upon' because the > hadoop.apache.org/ sidebar navigation does not link to it. One needs to know > the URL http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ to > land on it. > The links from the index page do not match the links from the side bar, so > some pages are quickly accessible (from sidebar). > I propose the links from the index.apt.vm to match the links from the YARN > side bar subsection (Ideally through one single definition file, but I don't > understand the APT generation process well enough to call out how this can be > achieved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2932) Add entry for preemption setting to queue status screen and startup/refresh logging
[
https://issues.apache.org/jira/browse/YARN-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-2932:
-
Attachment: YARN-2932.v2.txt
Thanks very much, [~leftnoteasy], for your thorough review of this patch and
for your helpful comments.
{quote}
1) Since the QUEUE_PREEMPTION_DISABLED is an option for CS, I suggest to make
it as a member of CapacitySchedulerConfiguration, like
getUserLimitFactor/setUserLimit, etc. This will void some String operations.
{quote}
This is a good idea. I added {{isQueuePreemptable}} and
{{setQueuePreemptable}}. For {{isQueuePreemptable}}, I needed to add a default
value parameter because the default for the queue at a particular level should
be whatever its parent's value is.
{quote}
2) Rename {{context}} in {{AbstractCSQueue}} to name like {{csContext}} since
we have {{rmContext}}
{quote}
Renamed.
{quote}
3) I suggest to add a member var like {{preemptable}} to {{AbstractCSQueue}},
instead of calling:
{code}
+ @Private
+ public boolean isPreemptable() {
+return context.getConfiguration().isPreemptable(getQueuePath());
+ }
{code}
The implementation of {{CSConfiguration.isPreemptable(..)}} seems too complex
to me. {{CSConfiguration}} should only care about value of configuration file,
such logic should put to {{AbstractCSQueue.setupQueueConfigs(...)}}
{quote}
I moved the logic to {{AbstractCSQueue.setupQueueConfigs(...)}}, and you are
right. It is much cleaner that way. Thanks!
{quote}
4) It's better to web UI name (preemptable) and configuration name
(disable_preemption) consistent. I prefer "preemptable" personally.
{quote}
Yes, it is less confusing that way. In this patch, the only things that worry
about the {{disable_preemption}} property are the internals of the
{{CSConfiguration}} methods. The APIs are now all asking whether or not the
queue is preemptable.
{quote}
5) {{testIsPreemptable}} should be a part of {{TestCapacityScheduler}} instead
of putting it to {{TestProportionalCapacityPreemptionPolicy}}.
{quote}
Thanks. I moved the test to {{testIsPreemptable}}. However, since the interface
for changing a queue's preemptability changed, there were also several changes
to {{TestProportionalCapacityPreemptionPolicy}}.
{quote}
6) In {{ProportionalCapacityPreemptionPolicy.cloneQueues}}, preemptable field
should get from Queue instead of getting from configuration.
{quote}
Done.
> Add entry for preemption setting to queue status screen and startup/refresh
> logging
> ---
>
> Key: YARN-2932
> URL: https://issues.apache.org/jira/browse/YARN-2932
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: YARN-2932.v1.txt, YARN-2932.v2.txt
>
>
> YARN-2056 enables the ability to turn preemption on or off on a per-queue
> level. This JIRA will provide the preemption status for each queue in the
> {{HOST:8088/cluster/scheduler}} UI and in the RM log during startup/queue
> refresh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2932) Add entry for preemption setting to queue status screen and startup/refresh logging
[
https://issues.apache.org/jira/browse/YARN-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272684#comment-14272684
]
Hadoop QA commented on YARN-2932:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12691517/YARN-2932.v2.txt
against trunk revision ef3c3a8.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6303//console
This message is automatically generated.
> Add entry for preemption setting to queue status screen and startup/refresh
> logging
> ---
>
> Key: YARN-2932
> URL: https://issues.apache.org/jira/browse/YARN-2932
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: YARN-2932.v1.txt, YARN-2932.v2.txt
>
>
> YARN-2056 enables the ability to turn preemption on or off on a per-queue
> level. This JIRA will provide the preemption status for each queue in the
> {{HOST:8088/cluster/scheduler}} UI and in the RM log during startup/queue
> refresh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3019) Enable RM work-preserving restart by default
[ https://issues.apache.org/jira/browse/YARN-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272792#comment-14272792 ] Junping Du commented on YARN-3019: -- bq. To clarify: this jira is to flip recovery mode to work-preserving recovery from non-work-preserving recovery. The feature itself remains disabled. i.e. "yarn.resourcemanager.recovery.enabled" remains false. updating the description. Can you explain the behavior different with "yarn.resourcemanager.work-preserving-recovery.enabled" to be true while "yarn.resourcemanager.recovery.enabled" to be false? Is there any solid scenario for this configuration combination or only for identifying bug purpose? For latter one, more convenient way could apply on individual cluster rather than trunk. Thoughts? > Enable RM work-preserving restart by default > - > > Key: YARN-3019 > URL: https://issues.apache.org/jira/browse/YARN-3019 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Jian He >Assignee: Jian He > > The proposal is to set > "yarn.resourcemanager.work-preserving-recovery.enabled" to true by default > to flip recovery mode to work-preserving recovery from non-work-preserving > recovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
