[jira] [Comment Edited] (YARN-7118) AHS REST API can return NullPointerException
[
https://issues.apache.org/jira/browse/YARN-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177930#comment-16177930
]
Anu Engineer edited comment on YARN-7118 at 9/23/17 6:16 PM:
-
This patch causes a compilation error on Branch-2. I have reverted this in
branch-2. Please feel free to check this in after it is fixed.
cc: [~djp]
was (Author: anu):
This patch causes a compilation error on Branch-2. I have reverted this in
branch-2. Please feel free to check this in after it is fixed.
> AHS REST API can return NullPointerException
>
>
> Key: YARN-7118
> URL: https://issues.apache.org/jira/browse/YARN-7118
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Billie Rinaldi
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-7118.01.patch
>
>
> ApplicationHistoryService REST Api returns NullPointerException
> {code}
> [prabhu@prabhu2 root]$ curl --negotiate -u: 'http:// IP>:8188/ws/v1/applicationhistory/apps?queue=test'
> {"exception":"NullPointerException","javaClassName":"java.lang.NullPointerException"}
> {code}
> TimelineServer logs shows below.
> {code}
> 2017-08-17 17:54:54,128 WARN webapp.GenericExceptionHandler
> (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.webapp.WebServices.getApps(WebServices.java:191)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices.getApps(AHSWebServices.java:96)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7118) AHS REST API can return NullPointerException
[
https://issues.apache.org/jira/browse/YARN-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177930#comment-16177930
]
Anu Engineer commented on YARN-7118:
This patch causes a compilation error on Branch-2. I have reverted this in
branch-2. Please feel free to check this in after it is fixed.
> AHS REST API can return NullPointerException
>
>
> Key: YARN-7118
> URL: https://issues.apache.org/jira/browse/YARN-7118
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Billie Rinaldi
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-7118.01.patch
>
>
> ApplicationHistoryService REST Api returns NullPointerException
> {code}
> [prabhu@prabhu2 root]$ curl --negotiate -u: 'http:// IP>:8188/ws/v1/applicationhistory/apps?queue=test'
> {"exception":"NullPointerException","javaClassName":"java.lang.NullPointerException"}
> {code}
> TimelineServer logs shows below.
> {code}
> 2017-08-17 17:54:54,128 WARN webapp.GenericExceptionHandler
> (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.webapp.WebServices.getApps(WebServices.java:191)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices.getApps(AHSWebServices.java:96)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-7118) AHS REST API can return NullPointerException
[
https://issues.apache.org/jira/browse/YARN-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer reopened YARN-7118:
> AHS REST API can return NullPointerException
>
>
> Key: YARN-7118
> URL: https://issues.apache.org/jira/browse/YARN-7118
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Prabhu Joseph
>Assignee: Billie Rinaldi
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-7118.01.patch
>
>
> ApplicationHistoryService REST Api returns NullPointerException
> {code}
> [prabhu@prabhu2 root]$ curl --negotiate -u: 'http:// IP>:8188/ws/v1/applicationhistory/apps?queue=test'
> {"exception":"NullPointerException","javaClassName":"java.lang.NullPointerException"}
> {code}
> TimelineServer logs shows below.
> {code}
> 2017-08-17 17:54:54,128 WARN webapp.GenericExceptionHandler
> (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.webapp.WebServices.getApps(WebServices.java:191)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices.getApps(AHSWebServices.java:96)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-7243) Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager
[ https://issues.apache.org/jira/browse/YARN-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeliang Cang updated YARN-7243: --- Comment: was deleted (was: Sorry, there are many other "org.apache.commons.logging.Log" occurrences, will submit a patch again!) > Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager > --- > > Key: YARN-7243 > URL: https://issues.apache.org/jira/browse/YARN-7243 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yeliang Cang >Assignee: Yeliang Cang > Attachments: YARN-7243.001.patch, YARN-7243.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-65) Reduce RM app memory footprint once app has completed
[
https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177798#comment-16177798
]
Hadoop QA commented on YARN-65:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 29s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 444 unchanged - 1 fixed = 447 total (was 445)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 24s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-65 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888706/YARN-65.014.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 7813aac3c740 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / cda3378 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17610/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/17610/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17610/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17610/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
[jira] [Commented] (YARN-65) Reduce RM app memory footprint once app has completed
[
https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1615#comment-1615
]
Manikandan R commented on YARN-65:
--
[~rohithsharma] Thanks for very detailed reviews and trying out better
solutions as well.
1,2:
Yes, MockMemoryRMStateStore#clone can be avoided and use cloned application
submission context inside MockMemoryRMStateStore#loadState itself.
3:
Those two test cases were using old state (retrieved using
MockMemoryRMStateStore#loadState) after RM restarts to reinstate the old state
info and trying to load into the state store again. Now, this old state has to
repopulated with cloned application submission context similar to
MockMemoryRMStateStore#loadState flow and can be used further.
Attached patch for the same.
> Reduce RM app memory footprint once app has completed
> -
>
> Key: YARN-65
> URL: https://issues.apache.org/jira/browse/YARN-65
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 0.23.3
>Reporter: Jason Lowe
>Assignee: Manikandan R
> Attachments: YARN-65.001.patch, YARN-65.002.patch, YARN-65.003.patch,
> YARN-65.004.patch, YARN-65.005.patch, YARN-65.006.patch, YARN-65.007.patch,
> YARN-65.008.patch, YARN-65.009.patch, YARN-65.010.patch, YARN-65.011.patch,
> YARN-65.012.patch, YARN-65.013.patch, YARN-65.014.patch
>
>
> The ResourceManager holds onto a configurable number of completed
> applications (yarn.resource.max-completed-applications, defaults to 1),
> and the memory footprint of these completed applications can be significant.
> For example, the {{submissionContext}} in RMAppImpl contains references to
> protocolbuffer objects and other items that probably aren't necessary to keep
> around once the application has completed. We could significantly reduce the
> memory footprint of the RM by releasing objects that are no longer necessary
> once an application completes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-65) Reduce RM app memory footprint once app has completed
[
https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikandan R updated YARN-65:
-
Attachment: YARN-65.014.patch
> Reduce RM app memory footprint once app has completed
> -
>
> Key: YARN-65
> URL: https://issues.apache.org/jira/browse/YARN-65
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 0.23.3
>Reporter: Jason Lowe
>Assignee: Manikandan R
> Attachments: YARN-65.001.patch, YARN-65.002.patch, YARN-65.003.patch,
> YARN-65.004.patch, YARN-65.005.patch, YARN-65.006.patch, YARN-65.007.patch,
> YARN-65.008.patch, YARN-65.009.patch, YARN-65.010.patch, YARN-65.011.patch,
> YARN-65.012.patch, YARN-65.013.patch, YARN-65.014.patch
>
>
> The ResourceManager holds onto a configurable number of completed
> applications (yarn.resource.max-completed-applications, defaults to 1),
> and the memory footprint of these completed applications can be significant.
> For example, the {{submissionContext}} in RMAppImpl contains references to
> protocolbuffer objects and other items that probably aren't necessary to keep
> around once the application has completed. We could significantly reduce the
> memory footprint of the RM by releasing objects that are no longer necessary
> once an application completes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7246) Fix the default docker binary path
[ https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Kumpf reassigned YARN-7246: - Assignee: Shane Kumpf > Fix the default docker binary path > -- > > Key: YARN-7246 > URL: https://issues.apache.org/jira/browse/YARN-7246 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Shane Kumpf >Assignee: Shane Kumpf >Priority: Blocker > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7246) Fix the default docker binary path
[ https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177723#comment-16177723 ] Shane Kumpf commented on YARN-7246: --- I'll work on this. > Fix the default docker binary path > -- > > Key: YARN-7246 > URL: https://issues.apache.org/jira/browse/YARN-7246 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Shane Kumpf >Priority: Blocker > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7246) Fix the default docker binary path
Shane Kumpf created YARN-7246: - Summary: Fix the default docker binary path Key: YARN-7246 URL: https://issues.apache.org/jira/browse/YARN-7246 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Shane Kumpf Priority: Blocker -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()
[
https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177714#comment-16177714
]
Hadoop QA commented on YARN-6953:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api: The
patch generated 0 new + 7 unchanged - 2 fixed = 7 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6953 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888679/YARN-6953.008.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f3abfb39f2eb 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / cda3378 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17609/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17609/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and
> setMaximumAllocationForMandatoryResources()
> --
>
> Key: YARN-6953
> URL: https://issues.apache.org/jira/browse/YARN-6953
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components:
[jira] [Updated] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()
[
https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikandan R updated YARN-6953:
---
Attachment: YARN-6953.008.patch
> Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and
> setMaximumAllocationForMandatoryResources()
> --
>
> Key: YARN-6953
> URL: https://issues.apache.org/jira/browse/YARN-6953
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
>Priority: Minor
> Labels: newbie
> Attachments: YARN-6953.007.patch, YARN-6953.008.patch,
> YARN-6953-YARN-3926.001.patch, YARN-6953-YARN-3926.002.patch,
> YARN-6953-YARN-3926.003.patch, YARN-6953-YARN-3926.004.patch,
> YARN-6953-YARN-3926.005.patch, YARN-6953-YARN-3926.006.patch,
> YARN-6953-YARN-3926-WIP.patch
>
>
> The {{setMinimumAllocationForMandatoryResources()}} and
> {{setMaximumAllocationForMandatoryResources()}} methods are quite convoluted.
> They'd be much simpler if they just handled CPU and memory manually instead
> of trying to be clever about doing it in a loop. There are also issues, such
> as the log warning always talking about memory or the last element of the
> inner array being a copy of the first element.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()
[
https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177700#comment-16177700
]
Manikandan R commented on YARN-6953:
Fixed checkstyle issues.
> Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and
> setMaximumAllocationForMandatoryResources()
> --
>
> Key: YARN-6953
> URL: https://issues.apache.org/jira/browse/YARN-6953
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
>Priority: Minor
> Labels: newbie
> Attachments: YARN-6953.007.patch, YARN-6953.008.patch,
> YARN-6953-YARN-3926.001.patch, YARN-6953-YARN-3926.002.patch,
> YARN-6953-YARN-3926.003.patch, YARN-6953-YARN-3926.004.patch,
> YARN-6953-YARN-3926.005.patch, YARN-6953-YARN-3926.006.patch,
> YARN-6953-YARN-3926-WIP.patch
>
>
> The {{setMinimumAllocationForMandatoryResources()}} and
> {{setMaximumAllocationForMandatoryResources()}} methods are quite convoluted.
> They'd be much simpler if they just handled CPU and memory manually instead
> of trying to be clever about doing it in a loop. There are also issues, such
> as the log warning always talking about memory or the last element of the
> inner array being a copy of the first element.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6504) Add support for resource profiles in MapReduce
[ https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177636#comment-16177636 ] Daniel Templeton commented on YARN-6504: Ya know, that's a really good point. I've been looking at profiles as an clone of Grid Engine's "job categories", in which case being able to override parts of the profile is important. Considering, though, what an unnatural fit the override capability is in the current implementation, it may be better to treat profiles like EC2 instance types, meaning that they can't be overridden. Given that a user can always just ask for a list of resources explicitly if they know exactly what they want, the case for allowing resource profiles to be selectively overridden is pretty weak. Let me think about it over the weekend, but I'm inclined say that we should get rid of the override capability altogether. > Add support for resource profiles in MapReduce > -- > > Key: YARN-6504 > URL: https://issues.apache.org/jira/browse/YARN-6504 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: YARN-6504-YARN-3926.001.patch, > YARN-6504.YARN-3926.002.patch, YARN-6504.YARN-3926.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7135) Clean up lock-try order in common scheduler code
[
https://issues.apache.org/jira/browse/YARN-7135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177616#comment-16177616
]
Daniel Templeton commented on YARN-7135:
I looked through the source, and it throws Error, but I didn't see any
unchecked exceptions explicitly created. That doesn't mean one can't happen in
an unexpected way. There's a good bit of code in there... In any case, just
because the current choice of RW lock implementation doesn't throw an
exception, that doesn't mean we'll never switch to one that does. Better to be
safe.
> Clean up lock-try order in common scheduler code
>
>
> Key: YARN-7135
> URL: https://issues.apache.org/jira/browse/YARN-7135
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
>Affects Versions: 3.0.0-alpha4
>Reporter: Daniel Templeton
>Assignee: weiyuan
> Labels: newbie
> Attachments: YARN-7135.001.patch, YARN-7135.002.patch,
> YARN-7135.003.patch, YARN-7135.004.patch
>
>
> There are many places that follow the pattern:{code}try {
> lock.lock();
> ...
> } finally {
> lock.unlock();
> }{code}
> There are a couple of reasons that's a bad idea. The correct pattern
> is:{code}lock.lock();
> try {
> ...
> } finally {
> lock.unlock();
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7109) Extend aggregation operation for new ATS design
[ https://issues.apache.org/jira/browse/YARN-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zian Chen reassigned YARN-7109: --- Assignee: Linlin Zhou (was: Zian Chen) > Extend aggregation operation for new ATS design > --- > > Key: YARN-7109 > URL: https://issues.apache.org/jira/browse/YARN-7109 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zian Chen >Assignee: Linlin Zhou > Labels: patch > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7109) Extend aggregation operation for new ATS design
[ https://issues.apache.org/jira/browse/YARN-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177595#comment-16177595 ] Zian Chen commented on YARN-7109: - Sure Linlin, I'm glad you are interested in working on this ticket. Please let me know if you need any help while working on this patch. > Extend aggregation operation for new ATS design > --- > > Key: YARN-7109 > URL: https://issues.apache.org/jira/browse/YARN-7109 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zian Chen >Assignee: Zian Chen > Labels: patch > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
