[jira] [Commented] (YARN-5082) Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit
[
https://issues.apache.org/jira/browse/YARN-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325741#comment-15325741
]
Hudson commented on YARN-5082:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #9949 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9949/])
YARN-5082. Limit ContainerId increase in fair scheduler if the num of (arun
suresh: rev 5279af7cd4afb090da742a96b5786d9dee6224bc)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
> Limit ContainerId increase in fair scheduler if the num of node app reserved
> reached the limit
> ---
>
> Key: YARN-5082
> URL: https://issues.apache.org/jira/browse/YARN-5082
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: sandflee
>Assignee: sandflee
> Fix For: 2.9.0
>
> Attachments: YARN-5082.01.patch, YARN-5082.02.patch,
> YARN-5082.03.patch, YARN-5082.04.patch
>
>
> see many logs like
> {quote}
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_03 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_04 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_05 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_06 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_07 is not reservable
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1942) Many of ConverterUtils methods need to have public interfaces
[
https://issues.apache.org/jira/browse/YARN-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325736#comment-15325736
]
Hadoop QA commented on YARN-1942:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 34 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 12s
{color} | {color:green} branch-2.8 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 39s
{color} | {color:green} branch-2.8 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s {color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m 31s
{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
24s {color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
23s {color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 5s
{color} | {color:green} branch-2.8 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 54s
{color} | {color:green} branch-2.8 passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 47s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 28s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 28s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 34s
{color} | {color:red} root: The patch generated 72 new + 1805 unchanged - 21
fixed = 1877 total (was 1826) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
44s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 12 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 16m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 26s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 27s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 17s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hadoop-yarn-server-common in the patch passed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 21s {color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK
v1.8.0_91. {color} |
|
[jira] [Commented] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[ https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325735#comment-15325735 ] Sreenath Somarajapuram commented on YARN-5161: -- The texts are a bit on the lighter side. Would be better to use a more darker shade. > [YARN-3368] Add Apache Hadoop logo to UI home page > -- > > Key: YARN-5161 > URL: https://issues.apache.org/jira/browse/YARN-5161 > Project: Hadoop YARN > Issue Type: Sub-task > Components: webapp >Reporter: Sunil G >Assignee: Kai Sasaki > Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot > 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch, > YARN-5161-YARN-3368.04.patch, YARN-5161.01.patch, YARN-5161.02.patch, > apache_logo.png, hadoop_logo.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5082) Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit
[
https://issues.apache.org/jira/browse/YARN-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-5082:
--
Fix Version/s: 2.9.0
> Limit ContainerId increase in fair scheduler if the num of node app reserved
> reached the limit
> ---
>
> Key: YARN-5082
> URL: https://issues.apache.org/jira/browse/YARN-5082
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: sandflee
>Assignee: sandflee
> Fix For: 2.9.0
>
> Attachments: YARN-5082.01.patch, YARN-5082.02.patch,
> YARN-5082.03.patch, YARN-5082.04.patch
>
>
> see many logs like
> {quote}
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_03 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_04 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_05 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_06 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_07 is not reservable
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5082) Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit
[
https://issues.apache.org/jira/browse/YARN-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-5082:
--
Summary: Limit ContainerId increase in fair scheduler if the num of node
app reserved reached the limit (was: ContainerId rapidly increased in fair
scheduler if the num of node app reserved reached the limit)
> Limit ContainerId increase in fair scheduler if the num of node app reserved
> reached the limit
> ---
>
> Key: YARN-5082
> URL: https://issues.apache.org/jira/browse/YARN-5082
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: sandflee
>Assignee: sandflee
> Attachments: YARN-5082.01.patch, YARN-5082.02.patch,
> YARN-5082.03.patch, YARN-5082.04.patch
>
>
> see many logs like
> {quote}
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_03 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_04 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_05 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_06 is not reservable
> 16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as
> container container_1463159225729_0002_01_07 is not reservable
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-5124:
--
Attachment: YARN-5124.017.patch
Updating patch to add a remove method (throws unsupportedEx.. can implement if
required) and fixed the remaining javac warning.
The whitespace error is spurious..
> Modify AMRMClient to set the ExecutionType in the ResourceRequest
> -
>
> Key: YARN-5124
> URL: https://issues.apache.org/jira/browse/YARN-5124
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-5124.001.patch, YARN-5124.002.patch,
> YARN-5124.003.patch, YARN-5124.004.patch, YARN-5124.005.patch,
> YARN-5124.006.patch, YARN-5124.008.patch, YARN-5124.009.patch,
> YARN-5124.010.patch, YARN-5124.011.patch, YARN-5124.012.patch,
> YARN-5124.013.patch, YARN-5124.014.patch, YARN-5124.015.patch,
> YARN-5124.016.patch, YARN-5124.017.patch,
> YARN-5124_YARN-5180_combined.007.patch, YARN-5124_YARN-5180_combined.008.patch
>
>
> Currently the {{ContainerRequest}} allows the AM to set the {{ExecutionType}}
> in the AMRMClient, but it is not being set in the actual {{ResourceRequest}}
> that is sent to the RM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5197) RM leaks containers if running container disappears from node update
[ https://issues.apache.org/jira/browse/YARN-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325723#comment-15325723 ] Rohith Sharma K S commented on YARN-5197: - Committed to trunk/branch-2!! Thanks [~jlowe] for your contributions. For branch-2.8,branch-2.7 and branch-2.6, need to rebase the patch, would you rebase the patch please? > RM leaks containers if running container disappears from node update > > > Key: YARN-5197 > URL: https://issues.apache.org/jira/browse/YARN-5197 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.2, 2.6.4 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: YARN-5197.001.patch, YARN-5197.002.patch, > YARN-5197.003.patch > > > Once a node reports a container running in a status update, the corresponding > RMNodeImpl will track the container in its launchedContainers map. If the > node somehow misses sending the completed container status to the RM and the > container simply disappears from subsequent heartbeats, the container will > leak in launchedContainers forever and the container completion event will > not be sent to the scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5197) RM leaks containers if running container disappears from node update
[ https://issues.apache.org/jira/browse/YARN-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325722#comment-15325722 ] Hudson commented on YARN-5197: -- SUCCESS: Integrated in Hadoop-trunk-Commit #9948 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9948/]) YARN-5197. RM leaks containers if running container disappears from node (rohithsharmaks: rev e0f4620cc7db3db4b781e6042ab7dd754af28f18) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java > RM leaks containers if running container disappears from node update > > > Key: YARN-5197 > URL: https://issues.apache.org/jira/browse/YARN-5197 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.7.2, 2.6.4 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: YARN-5197.001.patch, YARN-5197.002.patch, > YARN-5197.003.patch > > > Once a node reports a container running in a status update, the corresponding > RMNodeImpl will track the container in its launchedContainers map. If the > node somehow misses sending the completed container status to the RM and the > container simply disappears from subsequent heartbeats, the container will > leak in launchedContainers forever and the container completion event will > not be sent to the scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5122) "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete
[
https://issues.apache.org/jira/browse/YARN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325717#comment-15325717
]
Hadoop QA commented on YARN-5122:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 19s {color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 45s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMClient |
| | hadoop.yarn.client.cli.TestLogsCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809617/YARN-5122.3.patch |
| JIRA Issue | YARN-5122 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3e2370bd50ab 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/11990/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/11990/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-YARN-Build/11990/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/11990/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
[jira] [Commented] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[ https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325716#comment-15325716 ] Sunil G commented on YARN-5161: --- Thank you very much [~kaisasak]. Path applies cleanly now. Color tone seems closer to logo now. I think I will check with [~Sreenath] and [~leftnoteasy] for an opinion here on UI color tone. [~Sreenath]/[~leftnoteasy] pls share your thoughts on UI links color tone w.r.t apache logo. > [YARN-3368] Add Apache Hadoop logo to UI home page > -- > > Key: YARN-5161 > URL: https://issues.apache.org/jira/browse/YARN-5161 > Project: Hadoop YARN > Issue Type: Sub-task > Components: webapp >Reporter: Sunil G >Assignee: Kai Sasaki > Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot > 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch, > YARN-5161-YARN-3368.04.patch, YARN-5161.01.patch, YARN-5161.02.patch, > apache_logo.png, hadoop_logo.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers
[
https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325713#comment-15325713
]
Sunil G commented on YARN-5215:
---
Hi [~elgoiri]
Thanks for initiating this. This looks useful. I have some comments on this.
- Eventhough we have a minimumAllocation from Scheduler, its better we define a
deadzone around the delta (nodeUtilization - containersUtilization) as it may
help to avoid thrashing.
- externalUtilization is considered as follows
{code}
+ externalUtilization = ResourceUtilization.newInstance(nodeUtilization);
+ externalUtilization.subtractFrom(
+ containersUtilization.getPhysicalMemory(),
+ containersUtilization.getVirtualMemory(),
+ containersUtilization.getCPU());
{code}
Please correct me if I understood wrongly as I think there is a corner case.
Assume a node where 16GB memory is available and only 8Gb is assigned to
NodeManager. And this node has some other process also running. So if 4GB is
used by such external process, I think node's {{getUnallocatedResource}} will
come as {{8GB(NM configured capacity) - 4GB (external process)}}.
{code}
Resources.subtractFrom(unallocatedResource, externalResource);
{code}
I think NodeResourceMonitorImpl seems returning resourceUtilization of whole
node. Its not capping with Node's configured capacity.
- This is a suggestion. As per current design, we are jumping to the possible
unallocated resource in a node fast. Will it be better if we reach to this
aggregated unallocated limit after checking few cycles of Node Utilization?
> Scheduling containers based on external load in the servers
> ---
>
> Key: YARN-5215
> URL: https://issues.apache.org/jira/browse/YARN-5215
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Inigo Goiri
> Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the
> resources. The proposal is to use the utilization information in the node and
> the containers to estimate how much is consumed by external processes and
> schedule based on this estimation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325708#comment-15325708 ] Varun Saxena edited comment on YARN-5239 at 6/11/16 4:37 AM: - I had to tried to fix these earlier. As the report shows, javadoc gets introduced if we write out fully qualified package name and break it across multiple lines. If we do not break it, more than 80 lines checkstyle issues comes in. So unused import gets replaced by that. I guess we will have to live with these checkstyle issues (of unused import). It is basically a bug of checkstyle that it incorrectly points these out as unused imports. was (Author: varun_saxena): I had to tried to fix these earlier. As the report shows, javadoc gets introduces if we write out fully qualified package name and break it across multiple lines. If we do not break it, more than 80 lines checkstyle issues comes in. So unused import gets replaced by that. I guess we will have to live with these checkstyle issues (of unused import). It is basically a bug of checkstyle that it incorrectly points these out as unused imports. > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Labels: yarn-2928-1st-milestone > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5122) "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete
[
https://issues.apache.org/jira/browse/YARN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-5122:
--
Attachment: YARN-5122.3.patch
> "yarn logs" for running containers should print an explicit footer saying
> that the log may be incomplete
>
>
> Key: YARN-5122
> URL: https://issues.apache.org/jira/browse/YARN-5122
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Jian He
> Attachments: YARN-5122.1.patch, YARN-5122.1.patch, YARN-5122.2.patch,
> YARN-5122.3.patch
>
>
> We can have a footer of the sort {quote}[This log file belongs to a running
> container and so may not be complete..]{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325708#comment-15325708 ] Varun Saxena edited comment on YARN-5239 at 6/11/16 4:30 AM: - I had to tried to fix these earlier. As the report shows, javadoc gets introduces if we write out fully qualified package name and break it across multiple lines. If we do not break it, more than 80 lines checkstyle issues comes in. So unused import gets replaced by that. I guess we will have to live with these checkstyle issues (of unused import). It is basically a bug of checkstyle that it incorrectly points these out as unused imports. was (Author: varun_saxena): I had to tried to fix these earlier. As the report shows, javadoc gets introduces if we git fully qualified package name and break it across multiple lines. If we do not break it, more than 80 lines checkstyle issues comes in. So unused import gets replaced by that. I guess we will have to live with these checkstyle issues (of unused import). It is basically a bug of checkstyle that it incorrectly points these out as unused imports. > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Labels: yarn-2928-1st-milestone > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5240) TestSystemMetricsPublisher.testPublishApplicationMetrics fails in trunk
[
https://issues.apache.org/jira/browse/YARN-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-5240:
Component/s: (was: t)
> TestSystemMetricsPublisher.testPublishApplicationMetrics fails in trunk
> ---
>
> Key: YARN-5240
> URL: https://issues.apache.org/jira/browse/YARN-5240
> Project: Hadoop YARN
> Issue Type: Test
> Components: test
>Reporter: Rohith Sharma K S
> Labels: test
>
> In the build
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/11975/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt]
> test case failed.
> {noformat}
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.554 sec <<<
> FAILURE! - in
> org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher
> testPublishApplicationMetrics(org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher)
> Time elapsed: 2.206 sec <<< FAILURE!
> java.lang.AssertionError: expected:<> but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher.testPublishApplicationMetrics(TestSystemMetricsPublisher.java:201)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325708#comment-15325708 ] Varun Saxena commented on YARN-5239: I had to tried to fix these earlier. As the report shows, javadoc gets introduces if we git fully qualified package name and break it across multiple lines. If we do not break it, more than 80 lines checkstyle issues comes in. So unused import gets replaced by that. I guess we will have to live with these checkstyle issues (of unused import). It is basically a bug of checkstyle that it incorrectly points these out as unused imports. > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Labels: yarn-2928-1st-milestone > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5240) TestSystemMetricsPublisher.testPublishApplicationMetrics fails in trunk
Rohith Sharma K S created YARN-5240:
---
Summary: TestSystemMetricsPublisher.testPublishApplicationMetrics
fails in trunk
Key: YARN-5240
URL: https://issues.apache.org/jira/browse/YARN-5240
Project: Hadoop YARN
Issue Type: Test
Components: t, test
Reporter: Rohith Sharma K S
In the build
[link|https://builds.apache.org/job/PreCommit-YARN-Build/11975/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt]
test case failed.
{noformat}
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.554 sec <<<
FAILURE! - in
org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher
testPublishApplicationMetrics(org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher)
Time elapsed: 2.206 sec <<< FAILURE!
java.lang.AssertionError: expected:<> but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher.testPublishApplicationMetrics(TestSystemMetricsPublisher.java:201)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[ https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325706#comment-15325706 ] Varun Saxena commented on YARN-5052: Thanks [~vrushalic] for the review and commit. Thanks [~jrottinghuis] for the reviews. > Update timeline service v2 documentation to capture information about filters > - > > Key: YARN-5052 > URL: https://issues.apache.org/jira/browse/YARN-5052 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Fix For: YARN-2928 > > Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service > v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch, > YARN-5052-YARN-2928.02.patch > > > Since YARN-4447 has gone in, we can update our documentation to capture > information about usage of filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325697#comment-15325697 ] Vrushali C commented on YARN-5070: -- Thanks Joep, based on our offline chat, I will look at updating the patch shortly with some code refactoring. > upgrade HBase version for first merge > - > > Key: YARN-5070 > URL: https://issues.apache.org/jira/browse/YARN-5070 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Vrushali C >Priority: Critical > Labels: yarn-2928-1st-milestone > Attachments: YARN-5070-YARN-2928.01.patch, > YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch, > YARN-5070-YARN-2928.04.patch > > > Currently we set the HBase version for the timeline service storage to 1.0.1. > This is a fairly old version, and there are reasons to upgrade to a newer > version. We should upgrade it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[
https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325689#comment-15325689
]
Hadoop QA commented on YARN-5161:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 8s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:6d3a5f5 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809615/YARN-5161-YARN-3368.04.patch
|
| JIRA Issue | YARN-5161 |
| Optional Tests | asflicense |
| uname | Linux 83727a573467 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3368 / b775df6 |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11989/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [YARN-3368] Add Apache Hadoop logo to UI home page
> --
>
> Key: YARN-5161
> URL: https://issues.apache.org/jira/browse/YARN-5161
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: webapp
>Reporter: Sunil G
>Assignee: Kai Sasaki
> Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot
> 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch,
> YARN-5161-YARN-3368.04.patch, YARN-5161.01.patch, YARN-5161.02.patch,
> apache_logo.png, hadoop_logo.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[
https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325687#comment-15325687
]
Kai Sasaki commented on YARN-5161:
--
[~sunilg]
I updated the patch in `format-patch` format which seems include binary file
({{hadoop_logo.png}} and attached the screen shot after updated. Please check
this when you get a chance.
> [YARN-3368] Add Apache Hadoop logo to UI home page
> --
>
> Key: YARN-5161
> URL: https://issues.apache.org/jira/browse/YARN-5161
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: webapp
>Reporter: Sunil G
>Assignee: Kai Sasaki
> Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot
> 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch,
> YARN-5161-YARN-3368.04.patch, YARN-5161.01.patch, YARN-5161.02.patch,
> apache_logo.png, hadoop_logo.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[ https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated YARN-5161: - Attachment: YARN-5161-YARN-3368.04.patch > [YARN-3368] Add Apache Hadoop logo to UI home page > -- > > Key: YARN-5161 > URL: https://issues.apache.org/jira/browse/YARN-5161 > Project: Hadoop YARN > Issue Type: Sub-task > Components: webapp >Reporter: Sunil G >Assignee: Kai Sasaki > Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot > 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch, > YARN-5161-YARN-3368.04.patch, YARN-5161.01.patch, YARN-5161.02.patch, > apache_logo.png, hadoop_logo.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[ https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated YARN-5161: - Attachment: Screen Shot 2016-06-11 at 12.33.39.png > [YARN-3368] Add Apache Hadoop logo to UI home page > -- > > Key: YARN-5161 > URL: https://issues.apache.org/jira/browse/YARN-5161 > Project: Hadoop YARN > Issue Type: Sub-task > Components: webapp >Reporter: Sunil G >Assignee: Kai Sasaki > Attachments: Screen Shot 2016-05-31 at 21.22.30.png, Screen Shot > 2016-06-11 at 12.33.39.png, YARN-5161-YARN-3368.03.patch, YARN-5161.01.patch, > YARN-5161.02.patch, apache_logo.png, hadoop_logo.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325672#comment-15325672
]
Hadoop QA commented on YARN-5124:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 50s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 57s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 57s {color}
| {color:red} hadoop-yarn-project_hadoop-yarn generated 1 new + 33 unchanged -
0 fixed = 34 total (was 33) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 33s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: The patch generated 15
new + 130 unchanged - 31 fixed = 145 total (was 161) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 7s
{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 36s
{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809607/YARN-5124.016.patch |
| JIRA Issue | YARN-5124 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ce82fd9acb36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| javac |
https://builds.apache.org/job/PreCommit-YARN-Build/11988/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/11988/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace |
[jira] [Commented] (YARN-1942) Many of ConverterUtils methods need to have public interfaces
[
https://issues.apache.org/jira/browse/YARN-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325665#comment-15325665
]
Hadoop QA commented on YARN-1942:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 36 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 39s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 44s
{color} | {color:green} branch-2 passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 23s
{color} | {color:green} branch-2 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
3s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m 46s
{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
25s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
30s {color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 11s
{color} | {color:green} branch-2 passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 51s
{color} | {color:green} branch-2 passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 36s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 22s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 58s
{color} | {color:red} root: The patch generated 110 new + 2989 unchanged - 35
fixed = 3099 total (was 3024) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
29s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 12 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 14m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s
{color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 6s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.8.0_74. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hadoop-yarn-server-common in the patch passed with JDK
v1.8.0_74. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m 1s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:green}+1{color}
[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-5124:
--
Attachment: YARN-5124.016.patch
Fixing whitespace (had included a spurious file), some of the checkstyles (the
rest don't seem relevant) and the javac warning
> Modify AMRMClient to set the ExecutionType in the ResourceRequest
> -
>
> Key: YARN-5124
> URL: https://issues.apache.org/jira/browse/YARN-5124
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-5124.001.patch, YARN-5124.002.patch,
> YARN-5124.003.patch, YARN-5124.004.patch, YARN-5124.005.patch,
> YARN-5124.006.patch, YARN-5124.008.patch, YARN-5124.009.patch,
> YARN-5124.010.patch, YARN-5124.011.patch, YARN-5124.012.patch,
> YARN-5124.013.patch, YARN-5124.014.patch, YARN-5124.015.patch,
> YARN-5124.016.patch, YARN-5124_YARN-5180_combined.007.patch,
> YARN-5124_YARN-5180_combined.008.patch
>
>
> Currently the {{ContainerRequest}} allows the AM to set the {{ExecutionType}}
> in the AMRMClient, but it is not being set in the actual {{ResourceRequest}}
> that is sent to the RM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325642#comment-15325642 ] Joep Rottinghuis commented on YARN-5070: >From my reading this code is equivalent to what we had before the bump to >HBase 1.1.0. We need to decide if we want to handle the getBatchSize and perhaps unify handling of both limits in this patch, or make that a separate jira and have this one focus on just the HBase bump. We do need to make sure that we refer to the above two jiras HBASE-12972 and HBASE-13421 to describe where this backwards incompatibility comes from. That can be done in YARN-5174 > upgrade HBase version for first merge > - > > Key: YARN-5070 > URL: https://issues.apache.org/jira/browse/YARN-5070 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Vrushali C >Priority: Critical > Labels: yarn-2928-1st-milestone > Attachments: YARN-5070-YARN-2928.01.patch, > YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch, > YARN-5070-YARN-2928.04.patch > > > Currently we set the HBase version for the timeline service storage to 1.0.1. > This is a fairly old version, and there are reasons to upgrade to a newer > version. We should upgrade it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325641#comment-15325641 ] Joep Rottinghuis commented on YARN-5239: No unit tests were added 'cause hit patch changes only documentation and imports. Tests ran locally and checkstyle came through clean. > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Labels: yarn-2928-1st-milestone > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325635#comment-15325635
]
Joep Rottinghuis commented on YARN-5070:
Yes I think (eventually) we need to do that, although at the moment we don't
honor those.
I think we're aware of this and will need to add that to the initial call to
nextRaw to ensure we set all these values in the builder.
See these previous comment:
bq. Btw. should we also honor regionScanner.getMaxResultSize()
bq. Will be uploading a patch very shortly today. Although not being used today
in our scan, we still need to check that values are being respected for
MaxResultSize and getMaxResultsPerColumnFamily.
If we're ok do deal with these additional parameters, that we previously
(before this particular patch) also didn't deal with, in a separate patch then
we can possibly continue on with this current approach for the initial merge.
This does beg the question why we defer getMaxResultSize() from the scanner
whereas we hang on to the batchSize through a member initialized in the
constructor.
{code}
@Override
public long getMaxResultSize() {
if (regionScanner == null) {
throw new IllegalStateException(
"RegionScanner.isFilterDone() called when the flow "
+ "scanner's scanner is not a RegionScanner");
}
return regionScanner.getMaxResultSize();
}
{code}
We could arguably get both from the scanner and not have that as an argument to
the constructor. On two calls in the call hierarchy we actually get the value
put into the constructor from the scanner to begin with, in other cases we set
it to -1.
I don't know if that -1 is needed to visit all cell versions or if this limit
refers to rowkey/column qualifier cell counts.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
> YARN-5070-YARN-2928.04.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5239) Eliminate unused imports checkstyle warnings
[
https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325630#comment-15325630
]
Hadoop QA commented on YARN-5239:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s
{color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice:
The patch generated 0 new + 16 unchanged - 7 fixed = 16 total (was 23) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 18s
{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-jdk1.7.0_101
with JDK v1.7.0_101 generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 46s
{color} | {color:green} hadoop-yarn-server-timelineservice in the patch passed
with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 51s
{color} | {color:green} hadoop-yarn-server-timelineservice in the patch passed
with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 33s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809602/YARN-5239-YARN-2928.01.patch
|
[jira] [Comment Edited] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325613#comment-15325613
]
Vrushali C edited comment on YARN-5070 at 6/11/16 1:43 AM:
---
bq. l.120: From the javadoc, it appears that ScannerContext keeps track of the
progress towards the limits. If the progress should be monitored across
multiple invocations of nextRaw(List) . I'm not sure if this will do
that.
Yes, what I believe is that the progress to be tracked is within the context of
the invocation of the "next" call, not across. Although the ScannerContext
class has keepProgress settings in case we want to track progress across RPCs.
But this patch does not do that.
The documentation in the ScannerContext class says
https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java;h=29bffd26753795f33b90f31e9b77a5d1387e5cd7;hb=refs/heads/branch-1.1
{code}
* ScannerContext instances encapsulate limit tracking AND progress towards
those limits during
* invocations of {@link InternalScanner#next(java.util.List)} and
* {@link RegionScanner#next(java.util.List)}.
{code}
For the flow run coprocessor, the nextRaw/next functions call the nextInternal
function which is the one that actually does the iteration. Hence the batch
limit is set up here.
bq. Are we even supposed to create instances of ScannerContext? Am I off? I'm
basically not sure what is the correct way of using the ScannerContext.
An example of how the ScannerContext is being used in the hbase region server
code:
https://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/StoreFlusher.html
bq. Are we certain that batchLimit is the correct one to use in ScannerContext?
batchLimit is the one that tracks the batch size during the scan’s next, hence
we are using that. There are other settings like max results per column family
or max result size which I believe will have corresponding limit settings in
ScannerContext. We are not keeping track of those in FlowScanner.
That said, all this is what I have gathered looking at the code by myself.
Would appreciate a feedback from an hbase person.
was (Author: vrushalic):
bq. l.120: From the javadoc, it appears that ScannerContext keeps track of the
progress towards the limits. If the progress should be monitored across
multiple invocations of nextRaw(List) . I'm not sure if this will do
that.
Yes, I believe the progress to be tracked is within the context of the
invocation of the "next" call, not across.
The documentation in the ScannerContext class says
https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java;h=29bffd26753795f33b90f31e9b77a5d1387e5cd7;hb=refs/heads/branch-1.1
{code}
* ScannerContext instances encapsulate limit tracking AND progress towards
those limits during
* invocations of {@link InternalScanner#next(java.util.List)} and
* {@link RegionScanner#next(java.util.List)}.
{code}
For the flow run coprocessor, the nextRaw/next functions call the nextInternal
function which is the one that actually does the iteration. Hence the batch
limit is set up here.
bq. Are we even supposed to create instances of ScannerContext? Am I off? I'm
basically not sure what is the correct way of using the ScannerContext.
An example of how the ScannerContext is being used in the hbase region server
code:
https://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/StoreFlusher.html
bq. Are we certain that batchLimit is the correct one to use in ScannerContext?
batchLimit is the one that tracks the batch size during the scan’s next, hence
we are using that.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
> YARN-5070-YARN-2928.04.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325624#comment-15325624
]
Hadoop QA commented on YARN-5124:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 50s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 28s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 28s {color}
| {color:red} hadoop-yarn-project_hadoop-yarn generated 2 new + 33 unchanged -
0 fixed = 35 total (was 33) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: The patch generated 17
new + 130 unchanged - 31 fixed = 147 total (was 161) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 19s
{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 43s {color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 55s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.client.cli.TestLogsCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809596/YARN-5124.015.patch |
| JIRA Issue | YARN-5124 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 61027b346800 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| javac |
https://builds.apache.org/job/PreCommit-YARN-Build/11985/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/11985/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace
[jira] [Commented] (YARN-5122) "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete
[
https://issues.apache.org/jira/browse/YARN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325617#comment-15325617
]
Hadoop QA commented on YARN-5122:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 23s {color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 24s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.client.cli.TestLogsCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809601/YARN-5122.2.patch |
| JIRA Issue | YARN-5122 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f4f94d78a25f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/11986/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/11986/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-YARN-Build/11986/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/11986/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11986/console |
| Powered by | Apache Yetus 0.3.0
[jira] [Commented] (YARN-5161) [YARN-3368] Add Apache Hadoop logo to UI home page
[
https://issues.apache.org/jira/browse/YARN-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325615#comment-15325615
]
Kai Sasaki commented on YARN-5161:
--
{quote}
So could you please regenerate patch by using git format-patch command.
{quote}
Sure, I'll rebase and generate a patch with {{format-patch}}.
{quote}
I think we can change the color scheme of UI links.
{quote}
It's difficult to answer for me since I'm not and graphical designer. Current
YARN UI also has same color tone and I don't feel so much of wrong style. But
anyway I can change and I'll attache screen shot of new style.
Also I'll change alternate text as you said. Thanks!
> [YARN-3368] Add Apache Hadoop logo to UI home page
> --
>
> Key: YARN-5161
> URL: https://issues.apache.org/jira/browse/YARN-5161
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: webapp
>Reporter: Sunil G
>Assignee: Kai Sasaki
> Attachments: Screen Shot 2016-05-31 at 21.22.30.png,
> YARN-5161-YARN-3368.03.patch, YARN-5161.01.patch, YARN-5161.02.patch,
> apache_logo.png, hadoop_logo.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325613#comment-15325613
]
Vrushali C commented on YARN-5070:
--
bq. l.120: From the javadoc, it appears that ScannerContext keeps track of the
progress towards the limits. If the progress should be monitored across
multiple invocations of nextRaw(List) . I'm not sure if this will do
that.
Yes, I believe the progress to be tracked is within the context of the
invocation of the "next" call, not across.
The documentation in the ScannerContext class says
https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java;h=29bffd26753795f33b90f31e9b77a5d1387e5cd7;hb=refs/heads/branch-1.1
{code}
* ScannerContext instances encapsulate limit tracking AND progress towards
those limits during
* invocations of {@link InternalScanner#next(java.util.List)} and
* {@link RegionScanner#next(java.util.List)}.
{code}
For the flow run coprocessor, the nextRaw/next functions call the nextInternal
function which is the one that actually does the iteration. Hence the batch
limit is set up here.
bq. Are we even supposed to create instances of ScannerContext? Am I off? I'm
basically not sure what is the correct way of using the ScannerContext.
An example of how the ScannerContext is being used in the hbase region server
code:
https://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/StoreFlusher.html
bq. Are we certain that batchLimit is the correct one to use in ScannerContext?
batchLimit is the one that tracks the batch size during the scan’s next, hence
we are using that.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
> YARN-5070-YARN-2928.04.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5083) YARN CLI for AM logs does not give any error message if entered invalid am value
[
https://issues.apache.org/jira/browse/YARN-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325610#comment-15325610
]
Junping Du commented on YARN-5083:
--
Thanks Jian! The patch above looks good.
However, can we add a unit test for this invalid AM value case? It should be
similar as TestLogsCLI#testInvalidApplicationId().
> YARN CLI for AM logs does not give any error message if entered invalid am
> value
>
>
> Key: YARN-5083
> URL: https://issues.apache.org/jira/browse/YARN-5083
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Jian He
> Attachments: YARN-5083.1.patch, YARN-5083.1.patch
>
>
> Entering invalid value for am in yarn logs CLI does not give any error message
> {code:title= there is no amattempt 30 for the application}
> yarn logs -applicationId -am 30
> impl.TimelineClientImpl: Timeline service address:
> INFO client.RMProxy: Connecting to ResourceManager at
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5239: --- Labels: yarn-2928-1st-milestone (was: ) > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Labels: yarn-2928-1st-milestone > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5239) Eliminate unused imports checkstyle warnings
[ https://issues.apache.org/jira/browse/YARN-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5239: --- Attachment: YARN-5239-YARN-2928.01.patch Attaching patch 01 that eliminates the checkstyle warnings. > Eliminate unused imports checkstyle warnings > > > Key: YARN-5239 > URL: https://issues.apache.org/jira/browse/YARN-5239 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > Attachments: YARN-5239-YARN-2928.01.patch > > > There are ~8 existing checkstyle issues generated due to unused imports. > By fully qualifying the classes in javadoc and manually wrapping the javadoc > under 80 characters we can eliminate these warnings. > This will help with the eventual merge because then we introduce 8 fewer > checkstyle warnings. > The only checkstyle warnings left are now too many arguments, which cannot be > easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5083) YARN CLI for AM logs does not give any error message if entered invalid am value
[
https://issues.apache.org/jira/browse/YARN-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325605#comment-15325605
]
Jian He commented on YARN-5083:
---
The highlighted one in blue is the right one. I'll change the name accordingly
if any review comments.
https://issues.apache.org/jira/secure/attachment/12809274/YARN-5083.1.patch
> YARN CLI for AM logs does not give any error message if entered invalid am
> value
>
>
> Key: YARN-5083
> URL: https://issues.apache.org/jira/browse/YARN-5083
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Jian He
> Attachments: YARN-5083.1.patch, YARN-5083.1.patch
>
>
> Entering invalid value for am in yarn logs CLI does not give any error message
> {code:title= there is no amattempt 30 for the application}
> yarn logs -applicationId -am 30
> impl.TimelineClientImpl: Timeline service address:
> INFO client.RMProxy: Connecting to ResourceManager at
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5083) YARN CLI for AM logs does not give any error message if entered invalid am value
[
https://issues.apache.org/jira/browse/YARN-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325603#comment-15325603
]
Junping Du commented on YARN-5083:
--
Hi [~jianhe], thanks for delivering patch. However, the attached two files are
name the same and we cannot see different from timestamp (it shows yesterday in
my local timezone). Can you point me which one is the latest or rename the
patch in a right order? Thanks!
> YARN CLI for AM logs does not give any error message if entered invalid am
> value
>
>
> Key: YARN-5083
> URL: https://issues.apache.org/jira/browse/YARN-5083
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Jian He
> Attachments: YARN-5083.1.patch, YARN-5083.1.patch
>
>
> Entering invalid value for am in yarn logs CLI does not give any error message
> {code:title= there is no amattempt 30 for the application}
> yarn logs -applicationId -am 30
> impl.TimelineClientImpl: Timeline service address:
> INFO client.RMProxy: Connecting to ResourceManager at
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5239) Eliminate unused imports checkstyle warnings
Joep Rottinghuis created YARN-5239: -- Summary: Eliminate unused imports checkstyle warnings Key: YARN-5239 URL: https://issues.apache.org/jira/browse/YARN-5239 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: YARN-2928 Reporter: Joep Rottinghuis Assignee: Joep Rottinghuis Priority: Trivial There are ~8 existing checkstyle issues generated due to unused imports. By fully qualifying the classes in javadoc and manually wrapping the javadoc under 80 characters we can eliminate these warnings. This will help with the eventual merge because then we introduce 8 fewer checkstyle warnings. The only checkstyle warnings left are now too many arguments, which cannot be easily refactored w/o changing code structure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5122) "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete
[
https://issues.apache.org/jira/browse/YARN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325593#comment-15325593
]
Jian He commented on YARN-5122:
---
no specific reason, reverted that change any way. submit a new patch.
> "yarn logs" for running containers should print an explicit footer saying
> that the log may be incomplete
>
>
> Key: YARN-5122
> URL: https://issues.apache.org/jira/browse/YARN-5122
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Jian He
> Attachments: YARN-5122.1.patch, YARN-5122.1.patch, YARN-5122.2.patch
>
>
> We can have a footer of the sort {quote}[This log file belongs to a running
> container and so may not be complete..]{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5122) "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete
[
https://issues.apache.org/jira/browse/YARN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-5122:
--
Attachment: YARN-5122.2.patch
> "yarn logs" for running containers should print an explicit footer saying
> that the log may be incomplete
>
>
> Key: YARN-5122
> URL: https://issues.apache.org/jira/browse/YARN-5122
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Jian He
> Attachments: YARN-5122.1.patch, YARN-5122.1.patch, YARN-5122.2.patch
>
>
> We can have a footer of the sort {quote}[This log file belongs to a running
> container and so may not be complete..]{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325584#comment-15325584
]
Arun Suresh commented on YARN-5124:
---
Also, ive raised the follow up jira you requested : YARN-5238
> Modify AMRMClient to set the ExecutionType in the ResourceRequest
> -
>
> Key: YARN-5124
> URL: https://issues.apache.org/jira/browse/YARN-5124
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-5124.001.patch, YARN-5124.002.patch,
> YARN-5124.003.patch, YARN-5124.004.patch, YARN-5124.005.patch,
> YARN-5124.006.patch, YARN-5124.008.patch, YARN-5124.009.patch,
> YARN-5124.010.patch, YARN-5124.011.patch, YARN-5124.012.patch,
> YARN-5124.013.patch, YARN-5124.014.patch, YARN-5124.015.patch,
> YARN-5124_YARN-5180_combined.007.patch, YARN-5124_YARN-5180_combined.008.patch
>
>
> Currently the {{ContainerRequest}} allows the AM to set the {{ExecutionType}}
> in the AMRMClient, but it is not being set in the actual {{ResourceRequest}}
> that is sent to the RM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-5124:
--
Attachment: YARN-5124.015.patch
Thanks for the review [~kasha]..
Uploading patch addressing your concerns:
bq. RequestInfoIterator does not implement remove.
Hmmm.. I intentionally left it out, since it is a bit complicated and is not
used by {{AMRMClientImpl}}. I vote we put that in, when its required.
bq. The naming of helper methods getLocationMap etc. can be more explicit. e.g.
getRemoteRequestsForLocation
So, it is not really returning a map of RemoteRequests for the location, it is
actually a sub-map, so a more descriptive name might be hard. Given that I have
docuemnted in a comment on the Key used by the root map, I am hoping we can
just keep this as it is, also, it is a package protected method only to be used
by AMRMClientImpl.
> Modify AMRMClient to set the ExecutionType in the ResourceRequest
> -
>
> Key: YARN-5124
> URL: https://issues.apache.org/jira/browse/YARN-5124
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-5124.001.patch, YARN-5124.002.patch,
> YARN-5124.003.patch, YARN-5124.004.patch, YARN-5124.005.patch,
> YARN-5124.006.patch, YARN-5124.008.patch, YARN-5124.009.patch,
> YARN-5124.010.patch, YARN-5124.011.patch, YARN-5124.012.patch,
> YARN-5124.013.patch, YARN-5124.014.patch, YARN-5124.015.patch,
> YARN-5124_YARN-5180_combined.007.patch, YARN-5124_YARN-5180_combined.008.patch
>
>
> Currently the {{ContainerRequest}} allows the AM to set the {{ExecutionType}}
> in the AMRMClient, but it is not being set in the actual {{ResourceRequest}}
> that is sent to the RM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325566#comment-15325566
]
Hadoop QA commented on YARN-5070:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 40s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
6s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 2s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 51s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 10s
{color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
10s {color} | {color:green} YARN-2928 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 45s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 9s
{color} | {color:green} hadoop-project in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 51s
{color} | {color:green} hadoop-yarn-server-timelineservice in the patch passed
with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 43s
{color} | {color:green}
[jira] [Created] (YARN-5238) Handle enforceExecutionType == false in AMRMClient
Arun Suresh created YARN-5238:
-
Summary: Handle enforceExecutionType == false in AMRMClient
Key: YARN-5238
URL: https://issues.apache.org/jira/browse/YARN-5238
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Arun Suresh
Assignee: Arun Suresh
Currently only *enforceExecutionType == true* is supported. To support *false*,
The {{RemoteRequestTable#addResourceRequest}} used by the AMRMClientImpl should
be modified to something like :
{noformat}
if (!execTypeReq.getEnforceExecutionType()) {
put(priority, resourceName,
execTypeReq.getExecutionType(), capability, resourceRequestInfo);
} else {
for (ExecutionType eType : ExecutionType.values()) {
put(priority, resourceName, eType,
capability, resourceRequestInfo);
}
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5237) Not all logs get aggregated with rolling log aggregation.
[
https://issues.apache.org/jira/browse/YARN-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325561#comment-15325561
]
Hadoop QA commented on YARN-5237:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 16s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 170 unchanged - 1 fixed = 171 total (was 171)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 56s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809587/YARN-5237.1.patch |
| JIRA Issue | YARN-5237 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1c7de60022d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/11984/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/11984/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/11984/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11984/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Not all logs get aggregated with rolling log
[jira] [Commented] (YARN-5237) Not all logs get aggregated with rolling log aggregation.
[
https://issues.apache.org/jira/browse/YARN-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325539#comment-15325539
]
Xuan Gong commented on YARN-5237:
-
Currently, we remove the finished container when we do the log aggregation
which works fine if rolling log aggregation is disabled. Because the log
aggregation only happens once.
But, it would cause the issue if rolling log aggregation is enabled. Because,
if we set includePattern/excludePattern, and we would not consider this during
the application is still running. If the container finishes, and we remove the
finished container, we would lose the chance to do the log aggregation (for
includePattern/excludePattern) when the app finishes.
{code}
this.pendingContainers.drainTo(pendingContainerInThisCycle);
Set finishedContainers =
new HashSet(pendingContainerInThisCycle);
// This container is finished, and all its logs have been uploaded,
// remove it from containerLogAggregators.
if (finishedContainers.contains(container)) {
containerLogAggregators.remove(container);
}
{code}
> Not all logs get aggregated with rolling log aggregation.
> -
>
> Key: YARN-5237
> URL: https://issues.apache.org/jira/browse/YARN-5237
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Attachments: YARN-5237.1.patch
>
>
> Steps to reproduce:
> 1) enable RM recovery
> 2) Run a sleep job
> 3) restart RM
> 4) kill the application
> We can not find that the logs for the first attempt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5237) Not all logs get aggregated with rolling log aggregation.
[ https://issues.apache.org/jira/browse/YARN-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5237: Attachment: YARN-5237.1.patch > Not all logs get aggregated with rolling log aggregation. > - > > Key: YARN-5237 > URL: https://issues.apache.org/jira/browse/YARN-5237 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-5237.1.patch > > > Steps to reproduce: > 1) enable RM recovery > 2) Run a sleep job > 3) restart RM > 4) kill the application > We can not find that the logs for the first attempt -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5237) Not all logs get aggregated with rolling log aggregation.
Xuan Gong created YARN-5237: --- Summary: Not all logs get aggregated with rolling log aggregation. Key: YARN-5237 URL: https://issues.apache.org/jira/browse/YARN-5237 Project: Hadoop YARN Issue Type: Bug Reporter: Xuan Gong Assignee: Xuan Gong Steps to reproduce: 1) enable RM recovery 2) Run a sleep job 3) restart RM 4) kill the application We can not find that the logs for the first attempt -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325530#comment-15325530
]
Sangjin Lee commented on YARN-5174:
---
Oh OK. I'm trying to understand where this is going. :) What might be good is
an actual patch. It would be great if you could add the content in the form of
a patch, and we can go from there.
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325519#comment-15325519
]
Li Lu commented on YARN-5174:
-
bq. So you're envisioning a separate document aimed at framework writers?
Actually Let's try our best to unify our documentations? We can add more
content and create a more detailed documentation for our object models in the
original doc?
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325509#comment-15325509
]
Sangjin Lee commented on YARN-5070:
---
Thanks for the update [~vrushalic]!
(FlowScanner.java)
- l.120: From the javadoc, it appears that {{ScannerContext}} keeps track of
the progress towards the limits. But {{nextRaw(List)}} creates a new
{{ScannerContext}} every time it is invoked. If the progress should be
monitored across multiple invocations of {{nextRaw(List)}}, I'm not sure
if this will do that. Are we even supposed to create instances of
{{ScannerContext}}? Am I off? I'm basically not sure what is the correct way of
using the {{ScannerContext}}. The same comment goes for {{next(List)}}.
- Are we certain that {{batchLimit}} is the correct one to use in
{{ScannerContext}}? I also see {{sizeLimit}} in {{ScannerContext}}.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch,
> YARN-5070-YARN-2928.04.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5070) upgrade HBase version for first merge
[ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-5070: - Attachment: YARN-5070-YARN-2928.04.patch Uploading v4 after rebasing. > upgrade HBase version for first merge > - > > Key: YARN-5070 > URL: https://issues.apache.org/jira/browse/YARN-5070 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Vrushali C >Priority: Critical > Labels: yarn-2928-1st-milestone > Attachments: YARN-5070-YARN-2928.01.patch, > YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch, > YARN-5070-YARN-2928.04.patch > > > Currently we set the HBase version for the timeline service storage to 1.0.1. > This is a fairly old version, and there are reasons to upgrade to a newer > version. We should upgrade it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325458#comment-15325458
]
Sangjin Lee commented on YARN-5174:
---
Oh OK. So you're envisioning a separate document aimed at framework writers?
Where would that document be added? Would it be still somewhere under
{{hadoop-yarn-site}}?
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4308) ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats
[
https://issues.apache.org/jira/browse/YARN-4308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325451#comment-15325451
]
Sunil G commented on YARN-4308:
---
Thanks [~Naganarasimha Garla] for the review and commit. Thanks [~kasha] for
the initial patch and thanks [~djp], [~templedf] for the review.
> ContainersAggregated CPU resource utilization reports negative usage in first
> few heartbeats
>
>
> Key: YARN-4308
> URL: https://issues.apache.org/jira/browse/YARN-4308
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Fix For: 2.9.0
>
> Attachments: 0001-YARN-4308.patch, 0002-YARN-4308.patch,
> 0003-YARN-4308.patch, 0004-YARN-4308.patch, 0005-YARN-4308.patch,
> 0006-YARN-4308.patch, 0007-YARN-4308.patch, 0008-YARN-4308.patch,
> 0009-YARN-4308.patch, 0010-YARN-4308.patch
>
>
> NodeManager reports ContainerAggregated CPU resource utilization as -ve value
> in first few heartbeats cycles. I added a new debug print and received below
> values from heartbeats.
> {noformat}
> INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
> ContainersResource Utilization : CpuTrackerUsagePercent : -1.0
> INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:ContainersResource
> Utilization : CpuTrackerUsagePercent : 198.94598
> {noformat}
> Its better we send 0 as CPU usage rather than sending a negative values in
> heartbeats eventhough its happening in only first few heartbeats.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325431#comment-15325431 ] Vrushali C commented on YARN-5070: -- Uploading patch v3. This addresses the batch limits on a scan on the flow run table. I have also added several unit tests for regular metrics in the flow run table as well as non numeric data. I cleaned up the nextInternal function such that it does not check for ValueConverter types. The code in nextInternal should collect/emit based on tags (not based on type of ValueConverters). > upgrade HBase version for first merge > - > > Key: YARN-5070 > URL: https://issues.apache.org/jira/browse/YARN-5070 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Vrushali C >Priority: Critical > Labels: yarn-2928-1st-milestone > Attachments: YARN-5070-YARN-2928.01.patch, > YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch > > > Currently we set the HBase version for the timeline service storage to 1.0.1. > This is a fairly old version, and there are reasons to upgrade to a newer > version. We should upgrade it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325433#comment-15325433
]
Hadoop QA commented on YARN-5070:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} YARN-5070 does not apply to YARN-2928. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809572/YARN-5070-YARN-2928.03.patch
|
| JIRA Issue | YARN-5070 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11982/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325429#comment-15325429
]
Hadoop QA commented on YARN-5070:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} YARN-5070 does not apply to YARN-2928. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809572/YARN-5070-YARN-2928.03.patch
|
| JIRA Issue | YARN-5070 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11981/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5070) upgrade HBase version for first merge
[
https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325424#comment-15325424
]
Hadoop QA commented on YARN-5070:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} YARN-5070 does not apply to YARN-2928. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809572/YARN-5070-YARN-2928.03.patch
|
| JIRA Issue | YARN-5070 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11980/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> upgrade HBase version for first merge
> -
>
> Key: YARN-5070
> URL: https://issues.apache.org/jira/browse/YARN-5070
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Vrushali C
>Priority: Critical
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-5070-YARN-2928.01.patch,
> YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch
>
>
> Currently we set the HBase version for the timeline service storage to 1.0.1.
> This is a fairly old version, and there are reasons to upgrade to a newer
> version. We should upgrade it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5070) upgrade HBase version for first merge
[ https://issues.apache.org/jira/browse/YARN-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-5070: - Attachment: YARN-5070-YARN-2928.03.patch > upgrade HBase version for first merge > - > > Key: YARN-5070 > URL: https://issues.apache.org/jira/browse/YARN-5070 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Vrushali C >Priority: Critical > Labels: yarn-2928-1st-milestone > Attachments: YARN-5070-YARN-2928.01.patch, > YARN-5070-YARN-2928.02.patch, YARN-5070-YARN-2928.03.patch > > > Currently we set the HBase version for the timeline service storage to 1.0.1. > This is a fairly old version, and there are reasons to upgrade to a newer > version. We should upgrade it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5236) FlowRunCoprocessor brings down HBase RegionServer
[ https://issues.apache.org/jira/browse/YARN-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325401#comment-15325401 ] Vrushali C commented on YARN-5236: -- Hi [~haibochen] The current hbase version that timelineservice V2 depends on is 1.0.1. There is a jira in progress to upgrade to 1.1.3. at YARN-5070 Although we would like to use hbase 1.2.x, phoenix presently does not have a supported release for hbase 1.2.x, so we will upgrade to hbase 1.2.x when the appropriate phoenix release is available. thanks Vrushali > FlowRunCoprocessor brings down HBase RegionServer > - > > Key: YARN-5236 > URL: https://issues.apache.org/jira/browse/YARN-5236 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Haibo Chen > > The FlowRunCoprocessor, when loaded in HBase, will bring down the region > server with exception > java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion() > I am running it with HBase 1.2.1 in pseudo-distributed mode to try out ATS v2 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325396#comment-15325396
]
Li Lu commented on YARN-5174:
-
Thanks [~sjlee0]! At very beginning I was planning to write something for
downstream projects to use ATS v2. That's why I included much contents from
YARN-3150. The main update is mostly on the object model section, and if
possible we can treat that as a starting point to have more details documented?
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5221) Expose UpdateResourceRequest API to allow AM to request for change in container properties
[ https://issues.apache.org/jira/browse/YARN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325388#comment-15325388 ] Arun Suresh commented on YARN-5221: --- [~mding] bq. when this ticket is completed, will you be able to update YARN-4175 for an update example? Sure.. will do Can you / [~leftnoteasy] please take a look at the latest patch, given that Container Resizing is due for 2.8.0, It maybe better for it go with the new API.. Thoughts ? > Expose UpdateResourceRequest API to allow AM to request for change in > container properties > -- > > Key: YARN-5221 > URL: https://issues.apache.org/jira/browse/YARN-5221 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-5221.001.patch, YARN-5221.002.patch > > > YARN-1197 introduced APIs to allow an AM to request for Increase and Decrease > of Container Resources after initial allocation. > YARN-5085 proposes to allow an AM to request for a change of Container > ExecutionType. > This JIRA proposes to unify both of the above into an Update Container API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325358#comment-15325358
]
Sangjin Lee commented on YARN-5174:
---
Thanks [~gtCarrera9] for the great content!
Just so I understand, your proposal is to restructure and expand the existing
{{TimelineServiceV2.html}} to include this content, right? There seems to be a
fair amount of overlap with this doc and the existing doc, and I wanted to make
sure we're on the same page.
The content looks good to me for the most part. If I have to nitpick one,
bq. Each metric item contains metric name (id), value, and what kind of
aggregation operation should be performed in this metric (noop by default).
I am not sure if we need to highlight the aggregation operation bit. First of
all, this is only about the *application* aggregation. Also, user code should
normally post the application-level metrics directly onto the application
entity (e.g. MR AM writing job metrics on the application entity). Should we
simply drop the mention of the aggregation op part?
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5236) FlowRunCoprocessor brings down HBase RegionServer
[ https://issues.apache.org/jira/browse/YARN-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5236: - Description: The FlowRunCoprocessor, when loaded in HBase, will bring down the region server with exception java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion() I am running it with HBase 1.2.1 in pseudo-distributed mode to try out ATS v2 was: The FlowRunCoprocessor, when loaded in HBase, will bring down the region server with exception java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion() I am running it with HBase 1.2.1 in pseudo-distributed mode > FlowRunCoprocessor brings down HBase RegionServer > - > > Key: YARN-5236 > URL: https://issues.apache.org/jira/browse/YARN-5236 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Reporter: Haibo Chen > > The FlowRunCoprocessor, when loaded in HBase, will bring down the region > server with exception > java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion() > I am running it with HBase 1.2.1 in pseudo-distributed mode to try out ATS v2 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5236) FlowRunCoprocessor brings down HBase RegionServer
Haibo Chen created YARN-5236: Summary: FlowRunCoprocessor brings down HBase RegionServer Key: YARN-5236 URL: https://issues.apache.org/jira/browse/YARN-5236 Project: Hadoop YARN Issue Type: Bug Components: timelineserver Reporter: Haibo Chen The FlowRunCoprocessor, when loaded in HBase, will bring down the region server with exception java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion() I am running it with HBase 1.2.1 in pseudo-distributed mode -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5174) several updates/corrections to timeline service documentation
[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Lu updated YARN-5174:
Attachment: PublishingApplicationDatatoYARNTimelineServicev.pdf
I've done a simple write up for application programmers about how to post data
to timeline v2 with our new data model. Some of client related contents are
from our existing documentations, but the data model part is new. This is a
draft and let's decide what else need to be added for next step. Thanks!
> several updates/corrections to timeline service documentation
> -
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: PublishingApplicationDatatoYARNTimelineServicev.pdf
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325309#comment-15325309
]
Hadoop QA commented on YARN-5231:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 50s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 13s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 7s
{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m 59s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 5s
{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809544/YARN-5231.1.patch |
| JIRA Issue | YARN-5231 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e9f0c2b0700b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8a1dcce |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/11979/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/11979/testReport/ |
| modules | C:
[jira] [Updated] (YARN-5137) Make DiskChecker pluggable in NodeManager
[ https://issues.apache.org/jira/browse/YARN-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-5137: --- Summary: Make DiskChecker pluggable in NodeManager (was: Make DiskChecker pluggable) > Make DiskChecker pluggable in NodeManager > - > > Key: YARN-5137 > URL: https://issues.apache.org/jira/browse/YARN-5137 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Ray Chiang >Assignee: Yufei Gu > Labels: supportability > Attachments: YARN-5137.001.patch > > > It would be nice to have the option for a DiskChecker that has more > sophisticated checking capabilities. In order to do this, we would first > need DiskChecker to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4958) The file localization process should allow for wildcards to reduce the application footprint in the state store
[
https://issues.apache.org/jira/browse/YARN-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325298#comment-15325298
]
Sangjin Lee commented on YARN-4958:
---
{quote}
Strictly speaking this should be two separate JIRAs, but I don't think anyone
is that fussy about it. I've seen plenty of patches that touch more than one
project. I've submitted several myself that touched common and HDFS.
{quote}
OK, although it'd be ideal to have two JIRAs (YARN to enable support for
wildcards in container launch context and MAPREDUCE to take advantage of it),
it might be good to move it to MAPREDUCE at least. The majority of the changes
are really in MAPREDUCE. What do you think?
> The file localization process should allow for wildcards to reduce the
> application footprint in the state store
> ---
>
> Key: YARN-4958
> URL: https://issues.apache.org/jira/browse/YARN-4958
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: YARN-4958.001.patch, YARN-4958.002.patch,
> YARN-4958.003.patch
>
>
> When using the -libjars option to add classes to the classpath, every library
> so added is explicitly listed in the {{ContainerLaunchContext}}'s local
> resources even though they're all uploaded to the same directory in HDFS.
> When using tools like Crunch without an uber JAR or when trying to take
> advantage of the shared cache, the number of libraries can be quite large.
> We've seen many cases where we had to turn down the max number of
> applications to prevent ZK from running out of heap because of the size of
> the state store entries.
> Rather than listing all files independently, this JIRA proposes to have the
> NM allow wildcards in the resource localization paths. Specifically, we
> propose to allow a path to have a final component (name) set to "*", which is
> interpreted by the NM as "download the full directory and link to every file
> in it from the job's working directory." This behavior is the same as the
> current behavior when using -libjars, but avoids explicitly listing every
> file.
> This JIRA does not attempt to provide more general purpose wildcards, such as
> "\*.jar" or "file\*", as having multiple entries for a single directory
> presents numerous logistical issues.
> This JIRA also does not attempt to integrate with the shared cache. That
> work will be left to a future JIRA. Specifically, this JIRA only applies
> when a full directory is uploaded. Currently the shared cache does not
> handle directory uploads.
> This JIRA proposes to allow for wildcards both in the internal processing of
> the -libjars switch and in paths added through the {{Job}} and
> {{DistributedCache}} classes.
> The proposed approach is to treat a path, "dir/\*", as "dir" for purposes of
> all file verification and localization. In the final step, the NM will query
> the localized directory to get a list of the files in "dir" such that each
> can be linked from the job's working directory. Since $PWD/\* is always
> included on the classpath, all JAR files in "dir" will be in the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5124) Modify AMRMClient to set the ExecutionType in the ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325276#comment-15325276
]
Karthik Kambatla commented on YARN-5124:
Comments on the latest patch:
# AMRMClient: Javdoc for the new method refers to the other method as "above".
It is okay to duplicate the comment to the extent needed.
# AMRMClientAsync: Instead of throwing unsupported, shouldn't the method be
calling client.getMatchingRequests? If throwing an exception is appropriate,
let us update the exception message? It is likely okay to skip the
implementation of this method in AMRMClientAsyncImpl?
# AMRMClientImpl: Unused import Iterator
# RemoteRequestsTable
## RequestInfoIterator does not implement remove.
## The following two lines could be on the same line.
{code}
class RemoteRequestsTable implements
Iterable{
{code}
## When declaring remoteRequestsTable, can we add info on what each level of
the map is for? e.g. It is not immediately obvious what the String in the map
stands for.
## The naming of helper methods getLocationMap etc. can be more explicit. e.g.
getRemoteRequestsForLocation.
## addResourceRequest: Mind filing a follow up JIRA under YARN-1011 and post
this snippet as a JIRA comment so we can get rid of it from the source.
# AMRMClientImpl
## Same as what Carlo mentioned. While iterating over remoteRequestsTable,
change Object to ResourceRequestInfo?
## Nit; In the following snippet, would prefer req and getExecutionTypeRequest
to be on the same line for readability.
{code}
addResourceRequest(req.getPriority(), node, req
.getExecutionTypeRequest(), req.getCapability(), req, true,
req.getNodeLabelExpression());
{code}
> Modify AMRMClient to set the ExecutionType in the ResourceRequest
> -
>
> Key: YARN-5124
> URL: https://issues.apache.org/jira/browse/YARN-5124
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-5124.001.patch, YARN-5124.002.patch,
> YARN-5124.003.patch, YARN-5124.004.patch, YARN-5124.005.patch,
> YARN-5124.006.patch, YARN-5124.008.patch, YARN-5124.009.patch,
> YARN-5124.010.patch, YARN-5124.011.patch, YARN-5124.012.patch,
> YARN-5124.013.patch, YARN-5124.014.patch,
> YARN-5124_YARN-5180_combined.007.patch, YARN-5124_YARN-5180_combined.008.patch
>
>
> Currently the {{ContainerRequest}} allows the AM to set the {{ExecutionType}}
> in the AMRMClient, but it is not being set in the actual {{ResourceRequest}}
> that is sent to the RM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4958) The file localization process should allow for wildcards to reduce the application footprint in the state store
[
https://issues.apache.org/jira/browse/YARN-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325272#comment-15325272
]
Daniel Templeton commented on YARN-4958:
No worries. Thanks for the review, [~sjlee0].
On point #1, I had the same thought with the same conclusion. Since it's an
issue that already exists for directories, I don't see where this patch really
changes anything. In the majority of cases, this patch will be used via
-libjars, in which case there is no issue. In the remaining cases, the
behavior is clearly documented in the javadoc, so I should think that's OK.
On point #2, I haven't actually tested it, but it should work as one would
expect: each subdir should be linked from the working dir.
#3 is a good point. I think I did, but I don't remember anymore. I'll give
that a go and let you know.
I'll post a patch with fixes for #4 and #5.
Strictly speaking this should be two separate JIRAs, but I don't think anyone
is that fussy about it. I've seen plenty of patches that touch more than one
project. I've submitted several myself that touched common and HDFS.
> The file localization process should allow for wildcards to reduce the
> application footprint in the state store
> ---
>
> Key: YARN-4958
> URL: https://issues.apache.org/jira/browse/YARN-4958
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: YARN-4958.001.patch, YARN-4958.002.patch,
> YARN-4958.003.patch
>
>
> When using the -libjars option to add classes to the classpath, every library
> so added is explicitly listed in the {{ContainerLaunchContext}}'s local
> resources even though they're all uploaded to the same directory in HDFS.
> When using tools like Crunch without an uber JAR or when trying to take
> advantage of the shared cache, the number of libraries can be quite large.
> We've seen many cases where we had to turn down the max number of
> applications to prevent ZK from running out of heap because of the size of
> the state store entries.
> Rather than listing all files independently, this JIRA proposes to have the
> NM allow wildcards in the resource localization paths. Specifically, we
> propose to allow a path to have a final component (name) set to "*", which is
> interpreted by the NM as "download the full directory and link to every file
> in it from the job's working directory." This behavior is the same as the
> current behavior when using -libjars, but avoids explicitly listing every
> file.
> This JIRA does not attempt to provide more general purpose wildcards, such as
> "\*.jar" or "file\*", as having multiple entries for a single directory
> presents numerous logistical issues.
> This JIRA also does not attempt to integrate with the shared cache. That
> work will be left to a future JIRA. Specifically, this JIRA only applies
> when a full directory is uploaded. Currently the shared cache does not
> handle directory uploads.
> This JIRA proposes to allow for wildcards both in the internal processing of
> the -libjars switch and in paths added through the {{Job}} and
> {{DistributedCache}} classes.
> The proposed approach is to treat a path, "dir/\*", as "dir" for purposes of
> all file verification and localization. In the final step, the NM will query
> the localized directory to get a list of the files in "dir" such that each
> can be linked from the job's working directory. Since $PWD/\* is always
> included on the classpath, all JAR files in "dir" will be in the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[
https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325260#comment-15325260
]
Vrushali C commented on YARN-5052:
--
Committed this in. Thanks [~varun_saxena] for the patch and [~jrottinghuis] for
the review!
{code}
commit 64a278f0005af64af039ab0ea7c575728926d695
Author: Vrushali Channapattan
Date: Fri Jun 10 13:42:24 2016 -0700
YARN-5052 Update timeline service v2 documentation to capture information
about filters (Varun Saxena via Vrushali C)
{code}
> Update timeline service v2 documentation to capture information about filters
> -
>
> Key: YARN-5052
> URL: https://issues.apache.org/jira/browse/YARN-5052
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service
> v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch,
> YARN-5052-YARN-2928.02.patch
>
>
> Since YARN-4447 has gone in, we can update our documentation to capture
> information about usage of filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5235) Avoid re-creation of EvenColumnNameConverter in HBaseTimelineWriterImpl#storeEvents
[ https://issues.apache.org/jira/browse/YARN-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5235: --- Description: As per discussion in YARN-5170 [~varun_saxena] noted: bq. In HBaseTimelineWriterImpl#storeEvents, we iterate over all events in a loop and will be creating EventColumnNameConverter object each time. Although its not a very heavy object right now, but can't we just create it once outside the loop ? was: As per discussion in YARN-5052 [~varun_saxena] noted: bq. In HBaseTimelineWriterImpl#storeEvents, we iterate over all events in a loop and will be creating EventColumnNameConverter object each time. Although its not a very heavy object right now, but can't we just create it once outside the loop ? > Avoid re-creation of EvenColumnNameConverter in > HBaseTimelineWriterImpl#storeEvents > --- > > Key: YARN-5235 > URL: https://issues.apache.org/jira/browse/YARN-5235 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis >Priority: Trivial > > As per discussion in YARN-5170 [~varun_saxena] noted: > bq. In HBaseTimelineWriterImpl#storeEvents, we iterate over all events in a > loop and will be creating EventColumnNameConverter object each time. Although > its not a very heavy object right now, but can't we just create it once > outside the loop ? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[ https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated YARN-5052: -- Comment: was deleted (was: I'll take a quick look at it, and respond within a couple of hours.) > Update timeline service v2 documentation to capture information about filters > - > > Key: YARN-5052 > URL: https://issues.apache.org/jira/browse/YARN-5052 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service > v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch, > YARN-5052-YARN-2928.02.patch > > > Since YARN-4447 has gone in, we can update our documentation to capture > information about usage of filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[ https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325246#comment-15325246 ] Vrushali C commented on YARN-5052: -- +1 Committing it in now > Update timeline service v2 documentation to capture information about filters > - > > Key: YARN-5052 > URL: https://issues.apache.org/jira/browse/YARN-5052 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service > v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch, > YARN-5052-YARN-2928.02.patch > > > Since YARN-4447 has gone in, we can update our documentation to capture > information about usage of filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[ https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325247#comment-15325247 ] Sangjin Lee commented on YARN-5052: --- I'll take a quick look at it, and respond within a couple of hours. > Update timeline service v2 documentation to capture information about filters > - > > Key: YARN-5052 > URL: https://issues.apache.org/jira/browse/YARN-5052 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service > v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch, > YARN-5052-YARN-2928.02.patch > > > Since YARN-4447 has gone in, we can update our documentation to capture > information about usage of filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5235) Avoid re-creation of EvenColumnNameConverter in HBaseTimelineWriterImpl#storeEvents
Joep Rottinghuis created YARN-5235: -- Summary: Avoid re-creation of EvenColumnNameConverter in HBaseTimelineWriterImpl#storeEvents Key: YARN-5235 URL: https://issues.apache.org/jira/browse/YARN-5235 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: YARN-2928 Reporter: Joep Rottinghuis Assignee: Joep Rottinghuis Priority: Trivial As per discussion in YARN-5052 [~varun_saxena] noted: bq. In HBaseTimelineWriterImpl#storeEvents, we iterate over all events in a loop and will be creating EventColumnNameConverter object each time. Although its not a very heavy object right now, but can't we just create it once outside the loop ? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5234) ResourceManager REST API missing descriptions for what's returned when using Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5234: --- Component/s: resourcemanager fairscheduler > ResourceManager REST API missing descriptions for what's returned when using > Fair Scheduler > --- > > Key: YARN-5234 > URL: https://issues.apache.org/jira/browse/YARN-5234 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation, fairscheduler, resourcemanager >Reporter: Grant Sohn >Priority: Minor > > Cluster Scheduler API indicates support for Capacity and Fifo. What's > missing is what would be returned if using Fair scheduling. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5230) Document FairScheduler's allowPreemptionFrom flag
[ https://issues.apache.org/jira/browse/YARN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla reassigned YARN-5230: -- Assignee: Karthik Kambatla > Document FairScheduler's allowPreemptionFrom flag > - > > Key: YARN-5230 > URL: https://issues.apache.org/jira/browse/YARN-5230 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation, fairscheduler >Affects Versions: 2.9.0 >Reporter: Grant Sohn >Assignee: Karthik Kambatla >Priority: Minor > > Feature added in https://issues.apache.org/jira/browse/YARN-4462 is not > documented in the Hadoop: Fair Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated YARN-5231:
Issue Type: Sub-task (was: Bug)
Parent: YARN-4904
> obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
> ---
>
> Key: YARN-5231
> URL: https://issues.apache.org/jira/browse/YARN-5231
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 2.9.0
>Reporter: Sumana Sathish
>Assignee: Xuan Gong
>Priority: Blocker
> Attachments: YARN-5231.1.patch
>
>
> Obtaining logs for last 'n' bytes gives the following exception
> {code}
> yarn logs -applicationId application_1465421211793_0004 -containerId
> container_e07_1465421211793_0004_01_01 -logFiles syslog -size -1000
> Exception in thread "main" java.io.IOException: The bytes were skipped are
> different from the caller requested
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readContainerLogsForALogType(AggregatedLogFormat.java:838)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainerLogsForALogType(LogCLIHelpers.java:300)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainersLogsForALogTypeWithoutNodeId(LogCLIHelpers.java:224)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.printContainerLogsForFinishedApplicationWithoutNodeId(LogsCLI.java:447)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:782)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:228)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325239#comment-15325239
]
Xuan Gong commented on YARN-5231:
-
The exception might happen when there might not be that much data available to
read (the other end of the pipe might not have sent that much data yet).
Use org.apache.hadoop.io.IOUtils.skipFully(valueStream, toSkip) instead of
valueStream.skip
> obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
> ---
>
> Key: YARN-5231
> URL: https://issues.apache.org/jira/browse/YARN-5231
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 2.9.0
>Reporter: Sumana Sathish
>Assignee: Xuan Gong
>Priority: Blocker
> Attachments: YARN-5231.1.patch
>
>
> Obtaining logs for last 'n' bytes gives the following exception
> {code}
> yarn logs -applicationId application_1465421211793_0004 -containerId
> container_e07_1465421211793_0004_01_01 -logFiles syslog -size -1000
> Exception in thread "main" java.io.IOException: The bytes were skipped are
> different from the caller requested
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readContainerLogsForALogType(AggregatedLogFormat.java:838)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainerLogsForALogType(LogCLIHelpers.java:300)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainersLogsForALogTypeWithoutNodeId(LogCLIHelpers.java:224)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.printContainerLogsForFinishedApplicationWithoutNodeId(LogsCLI.java:447)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:782)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:228)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5230) allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents
[ https://issues.apache.org/jira/browse/YARN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5230: --- Component/s: fairscheduler > allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents > > > Key: YARN-5230 > URL: https://issues.apache.org/jira/browse/YARN-5230 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation, fairscheduler >Affects Versions: 2.9.0 >Reporter: Grant Sohn >Priority: Minor > > Feature added in https://issues.apache.org/jira/browse/YARN-4462 is not > documented in the Hadoop: Fair Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5230) allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents
[ https://issues.apache.org/jira/browse/YARN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5230: --- Summary: allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents (was: allowPreemptionFrom flag not mentioned in Hadoop: Fair Scheduler documents) > allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents > > > Key: YARN-5230 > URL: https://issues.apache.org/jira/browse/YARN-5230 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation, fairscheduler >Affects Versions: 2.9.0 >Reporter: Grant Sohn >Priority: Minor > > Feature added in https://issues.apache.org/jira/browse/YARN-4462 is not > documented in the Hadoop: Fair Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated YARN-5231:
Attachment: YARN-5231.1.patch
> obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
> ---
>
> Key: YARN-5231
> URL: https://issues.apache.org/jira/browse/YARN-5231
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 2.9.0
>Reporter: Sumana Sathish
>Assignee: Xuan Gong
>Priority: Blocker
> Attachments: YARN-5231.1.patch
>
>
> Obtaining logs for last 'n' bytes gives the following exception
> {code}
> yarn logs -applicationId application_1465421211793_0004 -containerId
> container_e07_1465421211793_0004_01_01 -logFiles syslog -size -1000
> Exception in thread "main" java.io.IOException: The bytes were skipped are
> different from the caller requested
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readContainerLogsForALogType(AggregatedLogFormat.java:838)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainerLogsForALogType(LogCLIHelpers.java:300)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainersLogsForALogTypeWithoutNodeId(LogCLIHelpers.java:224)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.printContainerLogsForFinishedApplicationWithoutNodeId(LogsCLI.java:447)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:782)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:228)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5230) Document FairScheduler's allowPreemptionFrom flag
[ https://issues.apache.org/jira/browse/YARN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-5230: --- Summary: Document FairScheduler's allowPreemptionFrom flag (was: allowPreemptionFrom flag not mentioned in HadoopFair Scheduler documents) > Document FairScheduler's allowPreemptionFrom flag > - > > Key: YARN-5230 > URL: https://issues.apache.org/jira/browse/YARN-5230 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation, fairscheduler >Affects Versions: 2.9.0 >Reporter: Grant Sohn >Priority: Minor > > Feature added in https://issues.apache.org/jira/browse/YARN-4462 is not > documented in the Hadoop: Fair Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5052) Update timeline service v2 documentation to capture information about filters
[ https://issues.apache.org/jira/browse/YARN-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325236#comment-15325236 ] Joep Rottinghuis commented on YARN-5052: Who else needs to look at this before we can commit it? Can we look for any large problems and keep little tinkering and adjustments that might be needed to YARN-5174 > Update timeline service v2 documentation to capture information about filters > - > > Key: YARN-5052 > URL: https://issues.apache.org/jira/browse/YARN-5052 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: Apache Hadoop 3.0.0-SNAPSHOT – The YARN Timeline Service > v.pdf, The YARN Timeline Service v2.02.pdf, YARN-5052-YARN-2928.01.patch, > YARN-5052-YARN-2928.02.patch > > > Since YARN-4447 has gone in, we can update our documentation to capture > information about usage of filters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4848) [YARN-3368] Unify app/container pages and other miscellaneous issues
[ https://issues.apache.org/jira/browse/YARN-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325164#comment-15325164 ] Varun Saxena commented on YARN-4848: Looked at this. Now what we have currently in node pages is a list of apps and containers running on that node(IMO should be there). Please note that NMs' will have state and diagnostics of apps and containers as well which might be useful (say if localization failed or container launch failed). Flow for current NM pages is as under : 1. From list of running apps, we click on each app and go to individual app page(NM view of app). Now the only thing we see there is user, app state and a list of containers belonging to this app(running on that NM). We can click on each one of these containers and go to an individual container page. 2. From the list of containers too, we can click on each one and go to individual container page. This page gives us container state, exit code, diagnostics, resources required by the container and link to logs. Now for 1, I think instead of going to NM view of app we can directly go to Main app page(as proposed). The only downside of this is that we will then have to go to app attempt page to see containers instead of directly seeing containers running. And also we wont be directly able to see containers of this app running on a particular node. But I do not think this is required as users will try to find out details about only containers which are stuck or failing, instead of trying to find out which containers are running on which node. For 2, however we do not have a single container page(RM view of container). We see all containers in a timeline view on app attempt page. And we see a small table for container on the side. Making user jump by clicking container link on NM page to app attempt page will be awkward. I think what we can do is that have a separate Container page which has details from both NM and RM for a container. The table next to Container timeline view can have bare minimum information. With ATSv2 we will have container metrics (CPU / Memory) available to us as well (even as a time series). Maybe in that individual container page we can fetch these metrics as a time series and plot a line graph or some other type of graph. > [YARN-3368] Unify app/container pages and other miscellaneous issues > > > Key: YARN-4848 > URL: https://issues.apache.org/jira/browse/YARN-4848 > Project: Hadoop YARN > Issue Type: Sub-task > Components: webapp >Reporter: Varun Saxena >Assignee: Varun Saxena > > We need to consolidate app and container views. Currently apps and container > views from NM page do not land up on a unified app/container page. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4958) The file localization process should allow for wildcards to reduce the application footprint in the state store
[
https://issues.apache.org/jira/browse/YARN-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325157#comment-15325157
]
Sangjin Lee commented on YARN-4958:
---
Sorry [~templedf] it took me a while to get back to this.
I finally got around to trying out the patch with a pseudo-distributed setup. I
can confirm that the main use cases seem to work correctly, including the case
of a non-jar resource in the staging directory. That said, I do have some high
level comments as well as a couple of minor nits.
(1)
Regarding the decision of determining the public-ness solely based on the
parent directory in the case of the wildcard, I'm wondering whether that would
have any implications. It's probably not going to be common, but it is possible
that the directory is public but there may be files that are not readable by
others. Again, it's hard to imagine why one would do this, but if they did,
would it cause a security issue on localization or a localization failure?
Should we chalk that up to an unsupported setting? To be fair, I can see this
being an issue if a directory was specified (not a wildcard), too. In that
sense, we could say this is a manifestation of an existing issue... Thoughts?
(2)
With {{ContainerExecutor.java}}, what happens if the wildcarded directory has
further nested directories? It appears we're symlinking only at the immediate
child level. I suspect it would work correctly, but wanted to double check.
(3)
Were you able to test it in the local job mode?
(4) {{ClientDistributedCacheManager.java}}
- l.303: change {{System.out.println()}} to a logger logging statement
(5) {{DistributedCache.java}}
- l.295: typo: "it's" -> "its"
Finally, since most of the changes are really in MAPREDUCE, perhaps this JIRA
should be moved to the MAPREDUCE project. What do you think? If we really want
to follow the rules to the letter, we would need to create separate JIRAs for
all projects involved (HADOOP, YARN, and MAPREDUCE). I'd like to hear what you
think. On a related note, you may want to drop the changes to {{Path.java}} if
you can help it.
> The file localization process should allow for wildcards to reduce the
> application footprint in the state store
> ---
>
> Key: YARN-4958
> URL: https://issues.apache.org/jira/browse/YARN-4958
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 2.8.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Critical
> Attachments: YARN-4958.001.patch, YARN-4958.002.patch,
> YARN-4958.003.patch
>
>
> When using the -libjars option to add classes to the classpath, every library
> so added is explicitly listed in the {{ContainerLaunchContext}}'s local
> resources even though they're all uploaded to the same directory in HDFS.
> When using tools like Crunch without an uber JAR or when trying to take
> advantage of the shared cache, the number of libraries can be quite large.
> We've seen many cases where we had to turn down the max number of
> applications to prevent ZK from running out of heap because of the size of
> the state store entries.
> Rather than listing all files independently, this JIRA proposes to have the
> NM allow wildcards in the resource localization paths. Specifically, we
> propose to allow a path to have a final component (name) set to "*", which is
> interpreted by the NM as "download the full directory and link to every file
> in it from the job's working directory." This behavior is the same as the
> current behavior when using -libjars, but avoids explicitly listing every
> file.
> This JIRA does not attempt to provide more general purpose wildcards, such as
> "\*.jar" or "file\*", as having multiple entries for a single directory
> presents numerous logistical issues.
> This JIRA also does not attempt to integrate with the shared cache. That
> work will be left to a future JIRA. Specifically, this JIRA only applies
> when a full directory is uploaded. Currently the shared cache does not
> handle directory uploads.
> This JIRA proposes to allow for wildcards both in the internal processing of
> the -libjars switch and in paths added through the {{Job}} and
> {{DistributedCache}} classes.
> The proposed approach is to treat a path, "dir/\*", as "dir" for purposes of
> all file verification and localization. In the final step, the NM will query
> the localized directory to get a list of the files in "dir" such that each
> can be linked from the job's working directory. Since $PWD/\* is always
> included on the classpath, all JAR files in "dir" will be in the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-5197) RM leaks containers if running container disappears from node update
[
https://issues.apache.org/jira/browse/YARN-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325069#comment-15325069
]
Hadoop QA commented on YARN-5197:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 39s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 1s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 0 new + 150 unchanged - 1 fixed = 150 total (was 151)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 16s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 27s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809463/YARN-5197.003.patch |
| JIRA Issue | YARN-5197 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 414e02c617a6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0bbb4dd |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/11975/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/11975/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-YARN-Build/11975/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/11975/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
[jira] [Updated] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated YARN-5231:
Affects Version/s: 2.9.0
> obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
> ---
>
> Key: YARN-5231
> URL: https://issues.apache.org/jira/browse/YARN-5231
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 2.9.0
>Reporter: Sumana Sathish
>Assignee: Xuan Gong
>Priority: Blocker
>
> Obtaining logs for last 'n' bytes gives the following exception
> {code}
> yarn logs -applicationId application_1465421211793_0004 -containerId
> container_e07_1465421211793_0004_01_01 -logFiles syslog -size -1000
> Exception in thread "main" java.io.IOException: The bytes were skipped are
> different from the caller requested
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readContainerLogsForALogType(AggregatedLogFormat.java:838)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainerLogsForALogType(LogCLIHelpers.java:300)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainersLogsForALogTypeWithoutNodeId(LogCLIHelpers.java:224)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.printContainerLogsForFinishedApplicationWithoutNodeId(LogsCLI.java:447)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:782)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:228)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-5232) Support for specifying a path for ATS plugin jars
[ https://issues.apache.org/jira/browse/YARN-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu resolved YARN-5232. - Resolution: Duplicate JIRA problem, close the duplicated issue. > Support for specifying a path for ATS plugin jars > - > > Key: YARN-5232 > URL: https://issues.apache.org/jira/browse/YARN-5232 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Li Lu >Assignee: Li Lu > > Third-party plugins need to add their jars to ATS. Most of the times, > isolation is not needed. However, there needs to be a way to specify the > path. For now, the jars on that path can be added to default classloader. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4143) Optimize the check for AMContainer allocation needed by blacklisting and ContainerType
[
https://issues.apache.org/jira/browse/YARN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325031#comment-15325031
]
Hadoop QA commented on YARN-4143:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} YARN-4143 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12757137/YARN-4143.001.patch |
| JIRA Issue | YARN-4143 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11978/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Optimize the check for AMContainer allocation needed by blacklisting and
> ContainerType
> --
>
> Key: YARN-4143
> URL: https://issues.apache.org/jira/browse/YARN-4143
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Anubhav Dhoot
>Assignee: Daniel Templeton
> Attachments: YARN-4143.001.patch
>
>
> In YARN-2005 there are checks made to determine if the allocation is for an
> AM container. This happens in every allocate call and should be optimized
> away since it changes only once per SchedulerApplicationAttempt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-4143) Optimize the check for AMContainer allocation needed by blacklisting and ContainerType
[ https://issues.apache.org/jira/browse/YARN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton reassigned YARN-4143: -- Assignee: Daniel Templeton (was: Anubhav Dhoot) > Optimize the check for AMContainer allocation needed by blacklisting and > ContainerType > -- > > Key: YARN-4143 > URL: https://issues.apache.org/jira/browse/YARN-4143 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Anubhav Dhoot >Assignee: Daniel Templeton > Attachments: YARN-4143.001.patch > > > In YARN-2005 there are checks made to determine if the allocation is for an > AM container. This happens in every allocate call and should be optimized > away since it changes only once per SchedulerApplicationAttempt -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5234) ResourceManager REST API missing descriptions for what's returned when using Fair Scheduler
Grant Sohn created YARN-5234: Summary: ResourceManager REST API missing descriptions for what's returned when using Fair Scheduler Key: YARN-5234 URL: https://issues.apache.org/jira/browse/YARN-5234 Project: Hadoop YARN Issue Type: Bug Components: documentation Reporter: Grant Sohn Priority: Minor Cluster Scheduler API indicates support for Capacity and Fifo. What's missing is what would be returned if using Fair scheduling. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5233) Support for specifying a path for ATS plugin jars
Li Lu created YARN-5233: --- Summary: Support for specifying a path for ATS plugin jars Key: YARN-5233 URL: https://issues.apache.org/jira/browse/YARN-5233 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.8.0 Reporter: Li Lu Assignee: Li Lu Third-party plugins need to add their jars to ATS. Most of the times, isolation is not needed. However, there needs to be a way to specify the path. For now, the jars on that path can be added to default classloader. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5232) Support for specifying a path for ATS plugin jars
Li Lu created YARN-5232: --- Summary: Support for specifying a path for ATS plugin jars Key: YARN-5232 URL: https://issues.apache.org/jira/browse/YARN-5232 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.8.0 Reporter: Li Lu Assignee: Li Lu Third-party plugins need to add their jars to ATS. Most of the times, isolation is not needed. However, there needs to be a way to specify the path. For now, the jars on that path can be added to default classloader. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5231) obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
[
https://issues.apache.org/jira/browse/YARN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumana Sathish updated YARN-5231:
-
Summary: obtaining app logs for last 'n' bytes using CLI gives
'java.io.IOException' (was: obtaining yarn logs for last 'n' bytes using CLI
gives 'java.io.IOException')
> obtaining app logs for last 'n' bytes using CLI gives 'java.io.IOException'
> ---
>
> Key: YARN-5231
> URL: https://issues.apache.org/jira/browse/YARN-5231
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Xuan Gong
>Priority: Blocker
>
> Obtaining logs for last 'n' bytes gives the following exception
> {code}
> yarn logs -applicationId application_1465421211793_0004 -containerId
> container_e07_1465421211793_0004_01_01 -logFiles syslog -size -1000
> Exception in thread "main" java.io.IOException: The bytes were skipped are
> different from the caller requested
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readContainerLogsForALogType(AggregatedLogFormat.java:838)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainerLogsForALogType(LogCLIHelpers.java:300)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAContainersLogsForALogTypeWithoutNodeId(LogCLIHelpers.java:224)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.printContainerLogsForFinishedApplicationWithoutNodeId(LogsCLI.java:447)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:782)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:228)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:264)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
