[jira] [Commented] (YARN-6802) Support view leaf queue am resource usage in RM web ui
[
https://issues.apache.org/jira/browse/YARN-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104492#comment-16104492
]
Yufei Gu commented on YARN-6802:
Thank [~daemon] for working on this. The patch looks good to me generally. Some
nits:
# Need to rebase due to HADOOP-11875 checked in today.
# One typo in Java doc of {{getAMMaxResources}}. "thie" -> "this"
> Support view leaf queue am resource usage in RM web ui
> --
>
> Key: YARN-6802
> URL: https://issues.apache.org/jira/browse/YARN-6802
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Affects Versions: 2.7.2
>Reporter: YunFan Zhou
>Assignee: YunFan Zhou
> Fix For: 2.8.0
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> YARN-6802.001.patch, YARN-6802.002.patch
>
>
> RM Web ui should support view leaf queue am resource usage.
> !screenshot-2.png!
> I will upload my patch later.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-65) Reduce RM app memory footprint once app has completed
[
https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104478#comment-16104478
]
Manikandan R commented on YARN-65:
--
Based on earlier suggestions, made changes to free memory by clearing unused
variables explicitly with either null or empty values, particularly
submissioncontext object. As of now, we are clearing up
app.submissionContext.getAMContainerSpec().setTokensConf(null);
inside AddApplicationToSchedulerTransition, RMAppRecoveredTransition &
FinalTransition class in RMAppImpl.java. Created a new method to clear all
unused fields including above mentioned specific field so that it can be called
from above mentioned classes.
Attached initial patch for review.
Couple of junits test cases like
TestContainerAllocation.testLogAggregationContextPassedIntoContainerToken
TestRMWebServicesAppsModification.testGetNewApplicationAndSubmit
is failing because actual value would have been cleared and hence not matching
with the expected value. To confirm whether those fields has been used in
application report, checked the AppBlock.java and don't see any usage over
there? Please correct me if this understanding is wrong. Based on feedback, we
may need to fix the Junit test cases accordingly.
In addition, to understand the memory savings, submitted an simple yarn app to
RM (local setup) in series and tried to visualize memory usage using jconsole
during GC before and after this code changes. I am not seeing any significant
difference as may be because of app is very light and size of each field inside
submissioncontext etc.
> Reduce RM app memory footprint once app has completed
> -
>
> Key: YARN-65
> URL: https://issues.apache.org/jira/browse/YARN-65
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 0.23.3
>Reporter: Jason Lowe
>Assignee: Naganarasimha G R
> Attachments: YARN-65.001.patch
>
>
> The ResourceManager holds onto a configurable number of completed
> applications (yarn.resource.max-completed-applications, defaults to 1),
> and the memory footprint of these completed applications can be significant.
> For example, the {{submissionContext}} in RMAppImpl contains references to
> protocolbuffer objects and other items that probably aren't necessary to keep
> around once the application has completed. We could significantly reduce the
> memory footprint of the RM by releasing objects that are no longer necessary
> once an application completes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-65) Reduce RM app memory footprint once app has completed
[
https://issues.apache.org/jira/browse/YARN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikandan R updated YARN-65:
-
Attachment: YARN-65.001.patch
> Reduce RM app memory footprint once app has completed
> -
>
> Key: YARN-65
> URL: https://issues.apache.org/jira/browse/YARN-65
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 0.23.3
>Reporter: Jason Lowe
>Assignee: Naganarasimha G R
> Attachments: YARN-65.001.patch
>
>
> The ResourceManager holds onto a configurable number of completed
> applications (yarn.resource.max-completed-applications, defaults to 1),
> and the memory footprint of these completed applications can be significant.
> For example, the {{submissionContext}} in RMAppImpl contains references to
> protocolbuffer objects and other items that probably aren't necessary to keep
> around once the application has completed. We could significantly reduce the
> memory footprint of the RM by releasing objects that are no longer necessary
> once an application completes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6898) RM node labels page should display total used resources of each label.
[ https://issues.apache.org/jira/browse/YARN-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6898: -- Description: The RM node labels page only show *Label Name*、*Label Type*、*Num Of Active NMs*、*Total Resource* information of each node label, but there isn't any place for us to see the total used resource of the node label. The total used resource of the node label is very important, because we can use it to check the overall load for this label. We will implement it. Any suggestion? was: The RM node labels page only show *Label Name*、*Label Type*、*Num Of Active NMs*、*Total Resource* information of each node label, but there isn't any place for us to see the total used resource of the node label. The total used resource of the node label is very important, because we can use it to check the overall load for this label. > RM node labels page should display total used resources of each label. > -- > > Key: YARN-6898 > URL: https://issues.apache.org/jira/browse/YARN-6898 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: YunFan Zhou > > The RM node labels page only show *Label Name*、*Label Type*、*Num Of Active > NMs*、*Total Resource* > information of each node label, but there isn't any place for us to see the > total used resource of the node label. > The total used resource of the node label is very important, because we can > use it to check the overall load for this > label. We will implement it. Any suggestion? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6898) RM node labels page should display total used resources of each label.
[ https://issues.apache.org/jira/browse/YARN-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6898: -- Description: The RM node labels page only show *Label Name*、*Label Type*、*Num Of Active NMs*、*Total Resource* information of each node label, but there isn't any place for us to see the total used resource of the node label. The total used resource of the node label is very important, because we can use it to check the overall load for this label. > RM node labels page should display total used resources of each label. > -- > > Key: YARN-6898 > URL: https://issues.apache.org/jira/browse/YARN-6898 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: YunFan Zhou > > The RM node labels page only show *Label Name*、*Label Type*、*Num Of Active > NMs*、*Total Resource* > information of each node label, but there isn't any place for us to see the > total used resource of the node label. > The total used resource of the node label is very important, because we can > use it to check the overall load for this > label. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6898) RM node labels page should display total used resources of each label.
[ https://issues.apache.org/jira/browse/YARN-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6898: -- Summary: RM node labels page should display total used resources of each label. (was: RM node labels page show display total used resources of each label.) > RM node labels page should display total used resources of each label. > -- > > Key: YARN-6898 > URL: https://issues.apache.org/jira/browse/YARN-6898 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: YunFan Zhou > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6898) RM node labels page show display total used resources of each label.
YunFan Zhou created YARN-6898: - Summary: RM node labels page show display total used resources of each label. Key: YARN-6898 URL: https://issues.apache.org/jira/browse/YARN-6898 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: YunFan Zhou -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6593) [API] Introduce Placement Constraint object
[
https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104341#comment-16104341
]
Hadoop QA commented on YARN-6593:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 59s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 24 unchanged - 0 fixed = 25 total (was 24) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api
generated 1 new + 123 unchanged - 0 fixed = 124 total (was 123) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
27s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6593 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879279/YARN-6593.007.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 76744de0f9fc 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 38c6fa5 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16585/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-YARN-Build/16585/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
|
| Test
[jira] [Commented] (YARN-5892) Support user-specific minimum user limit percentage in Capacity Scheduler
[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104304#comment-16104304
]
Sunil G commented on YARN-5892:
---
I will help to commit the same to branch-2.8.
Jenkins seems fine and test case failures are not related.
> Support user-specific minimum user limit percentage in Capacity Scheduler
> -
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
>Reporter: Eric Payne
>Assignee: Eric Payne
> Fix For: 3.0.0-alpha3
>
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
> YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch,
> YARN-5892.012.patch, YARN-5892.013.patch, YARN-5892.014.patch,
> YARN-5892.015.patch, YARN-5892.branch-2.015.patch,
> YARN-5892.branch-2.016.patch, YARN-5892.branch-2.8.016.patch,
> YARN-5892.branch-2.8.017.patch, YARN-5892.branch-2.8.018.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
>
>
> yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent
> 25
>
>
>
> yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent
> 75
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5892) Support user-specific minimum user limit percentage in Capacity Scheduler
[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104294#comment-16104294
]
Eric Payne commented on YARN-5892:
--
Thanks [~sunilg] for all your help on this JIRA. Were you planning on
committing the branch-2.8 patch, or were you expecting me to do that?
> Support user-specific minimum user limit percentage in Capacity Scheduler
> -
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
>Reporter: Eric Payne
>Assignee: Eric Payne
> Fix For: 3.0.0-alpha3
>
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
> YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch,
> YARN-5892.012.patch, YARN-5892.013.patch, YARN-5892.014.patch,
> YARN-5892.015.patch, YARN-5892.branch-2.015.patch,
> YARN-5892.branch-2.016.patch, YARN-5892.branch-2.8.016.patch,
> YARN-5892.branch-2.8.017.patch, YARN-5892.branch-2.8.018.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
>
>
> yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent
> 25
>
>
>
> yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent
> 75
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6698) Backport YARN-5121 to branch-2.7
[ https://issues.apache.org/jira/browse/YARN-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated YARN-6698: -- Fix Version/s: (was: 2.7.4) > Backport YARN-5121 to branch-2.7 > > > Key: YARN-6698 > URL: https://issues.apache.org/jira/browse/YARN-6698 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: YARN-6698-branch-2.7-01.patch, > YARN-6698-branch-2.7-test.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6897) Refactoring RMWebServices by moving some util methods in RMWebAppUtil
Giovanni Matteo Fumarola created YARN-6897:
--
Summary: Refactoring RMWebServices by moving some util methods in
RMWebAppUtil
Key: YARN-6897
URL: https://issues.apache.org/jira/browse/YARN-6897
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Giovanni Matteo Fumarola
Assignee: Giovanni Matteo Fumarola
In YARN-6896 the router needs to use some methods already implemented in
{{RMWebServices}}. This jira continues the work done in YARN-6634.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6593) [API] Introduce Placement Constraint object
[
https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104264#comment-16104264
]
Konstantinos Karanasos commented on YARN-6593:
--
Thanks [~leftnoteasy] -- I addressed your comments in the latest patch.
Note: regarding the #NODE/RACK, I made them package protected in
{{PlacementConstraint}}, but kept them public in {{PlacementConstraints}}
because they are required for applications to define the scope of the
constraints.
> [API] Introduce Placement Constraint object
> ---
>
> Key: YARN-6593
> URL: https://issues.apache.org/jira/browse/YARN-6593
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Konstantinos Karanasos
>Assignee: Konstantinos Karanasos
> Attachments: YARN-6593.001.patch, YARN-6593.002.patch,
> YARN-6593.003.patch, YARN-6593.004.patch, YARN-6593.005.patch,
> YARN-6593.006.patch, YARN-6593.007.patch
>
>
> Just removed Fixed version and moved it to target version as we set fix
> version only after patch is committed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6593) [API] Introduce Placement Constraint object
[ https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Karanasos updated YARN-6593: - Attachment: YARN-6593.007.patch Adding new version to address latest comments from [~leftnoteasy] and fix few remaining checkstyle issues. > [API] Introduce Placement Constraint object > --- > > Key: YARN-6593 > URL: https://issues.apache.org/jira/browse/YARN-6593 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Konstantinos Karanasos >Assignee: Konstantinos Karanasos > Attachments: YARN-6593.001.patch, YARN-6593.002.patch, > YARN-6593.003.patch, YARN-6593.004.patch, YARN-6593.005.patch, > YARN-6593.006.patch, YARN-6593.007.patch > > > Just removed Fixed version and moved it to target version as we set fix > version only after patch is committed. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3477) TimelineClientImpl swallows exceptions
[
https://issues.apache.org/jira/browse/YARN-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104253#comment-16104253
]
Hadoop QA commented on YARN-3477:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-3477 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-3477 |
| GITHUB PR | https://github.com/apache/hadoop/pull/47 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16584/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TimelineClientImpl swallows exceptions
> --
>
> Key: YARN-3477
> URL: https://issues.apache.org/jira/browse/YARN-3477
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 2.6.0, 2.7.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Labels: oct16-easy
> Attachments: YARN-3477-001.patch, YARN-3477-002.patch,
> YARN-3477-trunk.003.patch, YARN-3477-trunk.004.patch,
> YARN-3477-trunk.004.patch
>
>
> If timeline client fails more than the retry count, the original exception is
> not thrown. Instead some runtime exception is raised saying "retries run out"
> # the failing exception should be rethrown, ideally via
> NetUtils.wrapException to include URL of the failing endpoing
> # Otherwise, the raised RTE should (a) state that URL and (b) set the
> original fault as the inner cause
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6864) FSPreemptionThread cleanup for readability
[ https://issues.apache.org/jira/browse/YARN-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104250#comment-16104250 ] Yufei Gu commented on YARN-6864: +1 > FSPreemptionThread cleanup for readability > -- > > Key: YARN-6864 > URL: https://issues.apache.org/jira/browse/YARN-6864 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Daniel Templeton >Priority: Minor > Attachments: YARN-6864.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6893) Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab
[
https://issues.apache.org/jira/browse/YARN-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104160#comment-16104160
]
Hadoop QA commented on YARN-6893:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6893 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879270/YARN-6893.001.patch |
| Optional Tests | asflicense |
| uname | Linux a728cce99fd4 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c6330f2 |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16583/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab
> ---
>
> Key: YARN-6893
> URL: https://issues.apache.org/jira/browse/YARN-6893
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
>Affects Versions: 3.0.0-alpha4
>Reporter: Abdullah Yousufi
>Assignee: Abdullah Yousufi
> Attachments: YARN-6893.001.patch
>
>
> The table in the Flow Activity tab currently shows the aggregate memory and
> number of cpu vcores as N/A.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6893) Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab
[ https://issues.apache.org/jira/browse/YARN-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdullah Yousufi updated YARN-6893: --- Attachment: YARN-6893.001.patch > Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab > --- > > Key: YARN-6893 > URL: https://issues.apache.org/jira/browse/YARN-6893 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.0.0-alpha4 >Reporter: Abdullah Yousufi >Assignee: Abdullah Yousufi > Attachments: YARN-6893.001.patch > > > The table in the Flow Activity tab currently shows the aggregate memory and > number of cpu vcores as N/A. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6896) Federation: routing REST invocations transparently to multiple RMs
Giovanni Matteo Fumarola created YARN-6896: -- Summary: Federation: routing REST invocations transparently to multiple RMs Key: YARN-6896 URL: https://issues.apache.org/jira/browse/YARN-6896 Project: Hadoop YARN Issue Type: Sub-task Reporter: Giovanni Matteo Fumarola Assignee: Giovanni Matteo Fumarola This JIRA tracks the design/implementation of the layer for routing RMWebServicesProtocol requests to the appropriate RM(s) in a federated YARN cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6853) Add MySql Scripts for FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-6853: --- Attachment: YARN-6853.proto.patch > Add MySql Scripts for FederationStateStore > -- > > Key: YARN-6853 > URL: https://issues.apache.org/jira/browse/YARN-6853 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-6853.proto.patch > > > In YARN-3663 we added the SQL scripts for SQLServer. We want to add the MySql > scripts to be able to run Federation with a MySQL servers which will be less > performant but convenient. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6734) Ensure sub-application user is extracted & sent to timeline service
[ https://issues.apache.org/jira/browse/YARN-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104049#comment-16104049 ] Vrushali C commented on YARN-6734: -- I looked over the patch. LGTM over all. Thanks for updating the api calls to be Timeline Context related. +1 > Ensure sub-application user is extracted & sent to timeline service > --- > > Key: YARN-6734 > URL: https://issues.apache.org/jira/browse/YARN-6734 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Rohith Sharma K S > Attachments: YARN-6734-YARN-5355.001.patch, > YARN-6734-YARN-5355.002.patch, YARN-6734-YARN-5355.003.patch, > YARN-6734-YARN-5355.004.patch > > > After a discussion with Tez folks, we have been thinking over introducing a > table to store sub-application information. YARN-6733 > For example, if a Tez session runs for a certain period as User X and runs a > few AMs. These AMs accept DAGs from other users. Tez will execute these dags > with a doAs user. ATSv2 should store this information in a new table perhaps > called as "sub_application" table. > YARN-6733 tracks the code changes needed for table schema creation. > This jira tracks writing to that table, updating the user name fields to > include sub-application user etc. This would mean adding a field to Flow > Context which can store an additional user -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6853) Add MySql Scripts for FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola reassigned YARN-6853: -- Assignee: Giovanni Matteo Fumarola > Add MySql Scripts for FederationStateStore > -- > > Key: YARN-6853 > URL: https://issues.apache.org/jira/browse/YARN-6853 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola > > In YARN-3663 we added the SQL scripts for SQLServer. We want to add the MySql > scripts to be able to run Federation with a MySQL servers which will be less > performant but convenient. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3254) HealthReport should include disk full information
[ https://issues.apache.org/jira/browse/YARN-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103993#comment-16103993 ] Suma Shivaprasad commented on YARN-3254: Thanks [~sunilg] Will update patch to address comments. > HealthReport should include disk full information > - > > Key: YARN-3254 > URL: https://issues.apache.org/jira/browse/YARN-3254 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.6.0 >Reporter: Akira Ajisaka >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: Screen Shot 2015-02-24 at 17.57.39.png, Screen Shot > 2015-02-25 at 14.38.10.png, YARN-3254-001.patch, YARN-3254-002.patch, > YARN-3254-003.patch > > > When a NodeManager's local disk gets almost full, the NodeManager sends a > health report to ResourceManager that "local/log dir is bad" and the message > is displayed on ResourceManager Web UI. It's difficult for users to detect > why the dir is bad. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6594) [API] Introduce SchedulingRequest object
[
https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103986#comment-16103986
]
Arun Suresh edited comment on YARN-6594 at 7/27/17 10:01 PM:
-
Given that YARN-6593 looks close to committing, Think we should move forward
with this.
[~leftnoteasy] / [~jianhe], we should discuss further about the changes to the
AMRMClient in YARN-6619
was (Author: asuresh):
Given that YARN-6593 looks close to committing, Think we should move forward
with this.
[~leftnoteasy] / [~jianhe], this we should discuss further about the changes to
the AMRMClient in YARN-6619
> [API] Introduce SchedulingRequest object
>
>
> Key: YARN-6594
> URL: https://issues.apache.org/jira/browse/YARN-6594
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Konstantinos Karanasos
>Assignee: Konstantinos Karanasos
> Attachments: YARN-6594.001.patch
>
>
> This JIRA introduces a new SchedulingRequest object.
> It will be part of the {{AllocateRequest}} and will be used to define sizing
> (e.g., number of allocations, size of allocations) and placement constraints
> for allocations.
> Applications can use either this new object (when rich placement constraints
> are required) or the existing {{ResourceRequest}} object.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6594) [API] Introduce SchedulingRequest object
[
https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103986#comment-16103986
]
Arun Suresh commented on YARN-6594:
---
Given that YARN-6593 looks close to committing, Think we should move forward
with this.
[~leftnoteasy] / [~jianhe], this we should discuss further about the changes to
the AMRMClient in YARN-6619
> [API] Introduce SchedulingRequest object
>
>
> Key: YARN-6594
> URL: https://issues.apache.org/jira/browse/YARN-6594
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Konstantinos Karanasos
>Assignee: Konstantinos Karanasos
> Attachments: YARN-6594.001.patch
>
>
> This JIRA introduces a new SchedulingRequest object.
> It will be part of the {{AllocateRequest}} and will be used to define sizing
> (e.g., number of allocations, size of allocations) and placement constraints
> for allocations.
> Applications can use either this new object (when rich placement constraints
> are required) or the existing {{ResourceRequest}} object.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6875) New aggregated log file format for YARN log aggregation.
[ https://issues.apache.org/jira/browse/YARN-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103982#comment-16103982 ] Robert Kanter commented on YARN-6875: - [~xgong], have you taken a look at YARN-2942 and subtasks? I tried to do something like this a while ago and we went through a few different designs (I think there are 3 major different approaches, and some minor revisions for each); one of the approaches was very similar to your design, where there's an index file. In the end, we decided to do something completely different (MAPREDUCE-6415) by adding a command to combine log files into HAR files. This was to help with the too-many-small-files problem; though we still kept the T-files, so the goal was slightly different. Anyway, I did write a bunch of code for YARN-2942 and some subtasks before we canned it, so you might want to take a look in case you find something useful in there or the design documents. > New aggregated log file format for YARN log aggregation. > > > Key: YARN-6875 > URL: https://issues.apache.org/jira/browse/YARN-6875 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-6875-NewLogAggregationFormat-design-doc.pdf > > > T-file is the underlying log format for the aggregated logs in YARN. We have > seen several performance issues, especially for very large log files. > We will introduce a new log format which have better performance for large > log files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6593) [API] Introduce Placement Constraint object
[
https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103963#comment-16103963
]
Wangda Tan commented on YARN-6593:
--
Thanks [~kkaranasos] for updating the patch, could you check the reported
Javadocs warning?
In addition to that, some comments for API annotations:
1) Since we're in the early phase of the feature, instead of marking
PlacementConstraint(s) to be public/evolving, I prefer to mark them to
public/unstable.
2) private/evolving is not proper, private should come unstable or empty (Just
like CapacitySchedulerConfiguration) always.
3) Do we really need to mark {{api/resource/package-info.java}} to public? It
could accidentally add {{@public}} classes in the future
4) A couple of {{static final}} fields in PlacementConstraint(s) should be
{{@private}}, for example: {{PlacementConstraints#NODE/RACK}}.
> [API] Introduce Placement Constraint object
> ---
>
> Key: YARN-6593
> URL: https://issues.apache.org/jira/browse/YARN-6593
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Konstantinos Karanasos
>Assignee: Konstantinos Karanasos
> Attachments: YARN-6593.001.patch, YARN-6593.002.patch,
> YARN-6593.003.patch, YARN-6593.004.patch, YARN-6593.005.patch,
> YARN-6593.006.patch
>
>
> Just removed Fixed version and moved it to target version as we set fix
> version only after patch is committed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5412) Create a proxy chain for ResourceManager REST API in the Router
[ https://issues.apache.org/jira/browse/YARN-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103955#comment-16103955 ] Carlo Curino commented on YARN-5412: Thanks [~giovanni.fumarola] for the large code contribution. The patch looked good, and I committed to YARN-2915 branch. > Create a proxy chain for ResourceManager REST API in the Router > --- > > Key: YARN-5412 > URL: https://issues.apache.org/jira/browse/YARN-5412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Fix For: YARN-2915 > > Attachments: YARN-5412-YARN-2915.1.patch, > YARN-5412-YARN-2915.2.patch, YARN-5412-YARN-2915.3.patch, > YARN-5412-YARN-2915.4.patch, YARN-5412-YARN-2915.5.patch, > YARN-5412-YARN-2915.6.patch, YARN-5412-YARN-2915.7.patch, > YARN-5412-YARN-2915.proto.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager REST API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6793) Duplicated reservation in Fair Scheduler preemption
[ https://issues.apache.org/jira/browse/YARN-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu resolved YARN-6793. Resolution: Duplicate > Duplicated reservation in Fair Scheduler preemption > > > Key: YARN-6793 > URL: https://issues.apache.org/jira/browse/YARN-6793 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.8.1, 3.0.0-alpha3 >Reporter: Yufei Gu >Assignee: Yufei Gu >Priority: Critical > > There is a delay between preemption happen and containers are killed. If > resources released from nodes before container killing are not enough for the > resource request preemption asking for, reservation happens again at that > node. > E.g. scheduler reserves in node 1 for app 1 while > preemption. It will take 15s by default to kill containers in node 1 for > fulfill that resource requests. If was released from > node 1 before the killing, scheduler reserves again in > node 1 for app1. The second reservation may never be unreserved. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6130) [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM
[ https://issues.apache.org/jira/browse/YARN-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103863#comment-16103863 ] Jian He commented on YARN-6130: --- looks good to me, +1 > [ATSv2 Security] Generate a delegation token for AM when app collector is > created and pass it to AM via NM and RM > - > > Key: YARN-6130 > URL: https://issues.apache.org/jira/browse/YARN-6130 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-5355-merge-blocker > Attachments: YARN-6130-YARN-5355.01.patch, > YARN-6130-YARN-5355.02.patch, YARN-6130-YARN-5355.03.patch, > YARN-6130-YARN-5355.04.patch, YARN-6130-YARN-5355.05.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5412) Create a proxy chain for ResourceManager REST API in the Router
[
https://issues.apache.org/jira/browse/YARN-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103861#comment-16103861
]
Hadoop QA commented on YARN-5412:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} YARN-2915 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
10s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
3s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
21s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} YARN-2915 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 57s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 4 new + 241 unchanged - 1 fixed = 245 total (was 242) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 42m
47s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
2s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}106m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5412 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879219/YARN-5412-YARN-2915.7.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 359075e4e509 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-2915 / 62f1ce2 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
[jira] [Commented] (YARN-6612) Update fair scheduler policies to be aware of resource types
[
https://issues.apache.org/jira/browse/YARN-6612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103856#comment-16103856
]
Yufei Gu commented on YARN-6612:
Some major thoughts:
# I prefer to put functionality of ResourceWeights sort(e.g.,
{{getResourcesSortedByWeight}}) in class
{{DominantResourceFairnessComparator}}, that case we could exam the performance
of {{compare()}} more easily. That is due to when I was thinking about which
data structure is better for {{ResourceWeights#resources}}, ArrayList or
TreeSet. They seems no difference in {{compare()}}.
# Another performance improvement would be calculating fair share only if
necessary, which is similar to what YARN-6307 does. I guess we can do similar
refactor in this jira.
# I'm not sure we handle minShare usage well, e.g. what if multiple resource
usages are under their minShare, but we probably think more about it and
somehow solve it in followup JIRAs.
Minor ones:
# Why remove the {{FileNotFoundException fe}} in LOG.info() in Class
{{ResourceUtils}}?
# {float weight = 0.0f;} in {{ResourceWeights}} initialization is not necessary.
# We can't use ResourceType here, {{for (int i = 0; i <
ResourceType.values().length; i++)}} in {{ResourceWeights#toString}}. Moreover,
the enum {{ResourceType}} and related code should be removed.
# Several unused imports
# I like the refactor of {{ResourceNotFoundException}}. Do we need this method
{{ResourceNotFoundException(String resource, Throwable cause)}} since it is
never used?
# Method {{getResourceInformation()}} throw an unchecked exception, which isn't
necessary to go into the method signature. This applies to
{{getResourceValue()}} and several other methods in this class. Some of them
you didn't touch. Maybe you follow the convention, but why do we do that at the
first place? Besides that, {{ResourcePBImpl#getResourceInformation}} doesn't
throw {{ResourceNotFoundException}} in method signature, but
{{ResourcePBImpl#setResourceValue}} does. I think we need to solve this
inconsistency.
# I like new message in class {{ResourceNotFoundException}}, can we also
provide possible reasons of RM inconsistent state?
> Update fair scheduler policies to be aware of resource types
>
>
> Key: YARN-6612
> URL: https://issues.apache.org/jira/browse/YARN-6612
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-6612.YARN-3926.001.patch,
> YARN-6612.YARN-3926.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6890) If UI is not secured, we allow user to kill other users' job even yarn cluster is secured.
[
https://issues.apache.org/jira/browse/YARN-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103854#comment-16103854
]
Hadoop QA commented on YARN-6890:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 58s{color} | {color:orange} root: The patch generated 5 new + 127 unchanged
- 0 fixed = 132 total (was 127) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
58s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
45s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6890 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879223/YARN-6890.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 9da7b237bdf4 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 11ece0b |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16582/artifact/patchprocess/diff-checkstyle-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16582/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U:
. |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16582/console |
|
[jira] [Commented] (YARN-6876) Create an abstract log writer for extendability
[
https://issues.apache.org/jira/browse/YARN-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103842#comment-16103842
]
Hadoop QA commented on YARN-6876:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} branch-2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} branch-2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 50s{color}
| {color:red} hadoop-yarn-project_hadoop-yarn-jdk1.8.0_131 with JDK v1.8.0_131
generated 7 new + 129 unchanged - 7 fixed = 136 total (was 136) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 45s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 32 new + 505 unchanged - 10 fixed = 537 total (was 515) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
36s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
30s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_131 with JDK
v1.8.0_131 generated 4 new + 4577 unchanged - 0 fixed = 4581 total (was 4577)
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 29s{color}
| {color:red} hadoop-yarn-api in the patch failed with JDK v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.7.0_131. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 9s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK
v1.7.0_131. {color} |
| {color:red}-1{color} |
[jira] [Commented] (YARN-6788) Improve performance of resource profile branch
[
https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103823#comment-16103823
]
Hadoop QA commented on YARN-6788:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
41s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
40s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
38s{color} | {color:green} YARN-3926 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
19s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
YARN-3926 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
16s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
34s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 0s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 13 new + 126 unchanged - 17 fixed = 139 total (was 143) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
28s{color} | {color:red} hadoop-yarn-project_hadoop-yarn generated 1 new + 6364
unchanged - 0 fixed = 6365 total (was 6364) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
18s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api
generated 1 new + 123 unchanged - 0 fixed = 124 total (was 123) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 20s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hadoop-yarn-common in the
[jira] [Created] (YARN-6895) Preemption reservation may cause regular reservation leaks
Miklos Szegedi created YARN-6895: Summary: Preemption reservation may cause regular reservation leaks Key: YARN-6895 URL: https://issues.apache.org/jira/browse/YARN-6895 Project: Hadoop YARN Issue Type: Bug Components: fairscheduler Affects Versions: 3.0.0-alpha4 Reporter: Miklos Szegedi Assignee: Miklos Szegedi Priority: Blocker We found a limitation in the implementation of YARN-6432. If the container released is smaller than the preemption request, a node reservation is created that is never deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103728#comment-16103728
]
Sunil G commented on YARN-5219:
---
Thanks [~suma.shivaprasad]
I will update patch based on the comments.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219.006.patch,
> YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103724#comment-16103724
]
Varun Saxena commented on YARN-5548:
Committed to trunk, branch-2.
Thanks [~bibinchundatt] for your contribution and [~rohithsharma] for reviews
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103722#comment-16103722
]
Suma Shivaprasad commented on YARN-5219:
3. The sb.setExitOnFailure should be called outside of the environment checks
since it applies to the whole script
if(!environment.isEmpty()) {
+sb.setEnvValidator();
+ }
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219.006.patch,
> YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103717#comment-16103717
]
Hudson commented on YARN-5548:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12058 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12058/])
Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test (varunsaxena:
rev 11ece0bda1f6e5dd9d0f828b7c29acacf6087baa)
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRMMemoryStateStore.java
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6890) If UI is not secured, we allow user to kill other users' job even yarn cluster is secured.
[ https://issues.apache.org/jira/browse/YARN-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-6890: - Attachment: YARN-6890.patch The fix is straight-forward. > If UI is not secured, we allow user to kill other users' job even yarn > cluster is secured. > -- > > Key: YARN-6890 > URL: https://issues.apache.org/jira/browse/YARN-6890 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Junping Du >Priority: Critical > Attachments: YARN-6890.patch > > > Configuring SPNEGO for web browser could be a head ache, so many production > cluster choose to configure a unsecured UI access even for a secured cluster. > In this setup, users (login as some random guy) could watch other users job > which is expected. However, the kill button (added in YARN-3249 which enabled > by default) shouldn't work in this situation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6894) RM Apps API returns only active apps when query parameter queue used
Grant Sohn created YARN-6894: Summary: RM Apps API returns only active apps when query parameter queue used Key: YARN-6894 URL: https://issues.apache.org/jira/browse/YARN-6894 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, restapi Reporter: Grant Sohn Priority: Minor If you run RM's Cluster Applications API with no query parameters, you get a list of apps. If you run RM's Cluster Applications API with any query parameters other than "queue" you get the list of apps with the parameter filters being applied. However, when you use the "queue" query parameter, you only see the applications that are active in the cluster (NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING). This behavior is inconsistent with the API. If there is a sound reason behind this, it should be documented and it seems like there might be as the mapred queue CLI behaves similarly. http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6893) Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab
Abdullah Yousufi created YARN-6893: -- Summary: Fix Missing Memory and CPU Metric in WebUI v2 Flow Activity Tab Key: YARN-6893 URL: https://issues.apache.org/jira/browse/YARN-6893 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Affects Versions: 3.0.0-alpha4 Reporter: Abdullah Yousufi Assignee: Abdullah Yousufi The table in the Flow Activity tab currently shows the aggregate memory and number of cpu vcores as N/A. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6892) Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation
[ https://issues.apache.org/jira/browse/YARN-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6892: -- Attachment: YARN-6892-YARN-3926.001.patch Attaching an initial version of patch. This work is done based on YARN-6788. > Improve API implementation in Resources and DominantResourceCalculator in > align to ResourceInformation > -- > > Key: YARN-6892 > URL: https://issues.apache.org/jira/browse/YARN-6892 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-6892-YARN-3926.001.patch > > > In YARN-3926, apis in Resources and DRC spents significant cpu cycles in most > of its api. For better performance, its better to improve the apis as > resource types order is defined in system level (ResourceUtils class ensures > this post YARN-6788) > This work is preceding to YARN-6788 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6892) Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation
Sunil G created YARN-6892: - Summary: Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation Key: YARN-6892 URL: https://issues.apache.org/jira/browse/YARN-6892 Project: Hadoop YARN Issue Type: Sub-task Reporter: Sunil G Assignee: Sunil G In YARN-3926, apis in Resources and DRC spents significant cpu cycles in most of its api. For better performance, its better to improve the apis as resource types order is defined in system level (ResourceUtils class ensures this post YARN-6788) This work is preceding to YARN-6788 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5412) Create a proxy chain for ResourceManager REST API in the Router
[ https://issues.apache.org/jira/browse/YARN-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103677#comment-16103677 ] Giovanni Matteo Fumarola commented on YARN-5412: Fixed the remaining 2 yetus warnings in V7. The failed tests are not related (YARN-5816 already tracks the effort to fix it). > Create a proxy chain for ResourceManager REST API in the Router > --- > > Key: YARN-5412 > URL: https://issues.apache.org/jira/browse/YARN-5412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5412-YARN-2915.1.patch, > YARN-5412-YARN-2915.2.patch, YARN-5412-YARN-2915.3.patch, > YARN-5412-YARN-2915.4.patch, YARN-5412-YARN-2915.5.patch, > YARN-5412-YARN-2915.6.patch, YARN-5412-YARN-2915.7.patch, > YARN-5412-YARN-2915.proto.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager REST API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103675#comment-16103675
]
Varun Saxena commented on YARN-5548:
Had missed to push the newly added file. So made another commit to push it.
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5412) Create a proxy chain for ResourceManager REST API in the Router
[ https://issues.apache.org/jira/browse/YARN-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-5412: --- Attachment: YARN-5412-YARN-2915.7.patch > Create a proxy chain for ResourceManager REST API in the Router > --- > > Key: YARN-5412 > URL: https://issues.apache.org/jira/browse/YARN-5412 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Subru Krishnan >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-5412-YARN-2915.1.patch, > YARN-5412-YARN-2915.2.patch, YARN-5412-YARN-2915.3.patch, > YARN-5412-YARN-2915.4.patch, YARN-5412-YARN-2915.5.patch, > YARN-5412-YARN-2915.6.patch, YARN-5412-YARN-2915.7.patch, > YARN-5412-YARN-2915.proto.patch > > > As detailed in the proposal in the umbrella JIRA, we are introducing a new > component that routes client request to appropriate ResourceManager(s). This > JIRA tracks the creation of a proxy for ResourceManager REST API in the > Router. This provides a placeholder for: > 1) throttling mis-behaving clients (YARN-1546) > 3) mask the access to multiple RMs (YARN-3659) > We are planning to follow the interceptor pattern like we did in YARN-2884 to > generalize the approach and have only dynamically coupling for Federation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6890) If UI is not secured, we allow user to kill other users' job even yarn cluster is secured.
[ https://issues.apache.org/jira/browse/YARN-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-6890: - Target Version/s: 3.0.0-beta1, 2.8.2 (was: 2.8.2) > If UI is not secured, we allow user to kill other users' job even yarn > cluster is secured. > -- > > Key: YARN-6890 > URL: https://issues.apache.org/jira/browse/YARN-6890 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Junping Du >Priority: Critical > > Configuring SPNEGO for web browser could be a head ache, so many production > cluster choose to configure a unsecured UI access even for a secured cluster. > In this setup, users (login as some random guy) could watch other users job > which is expected. However, the kill button (added in YARN-3249 which enabled > by default) shouldn't work in this situation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6891) Can kill other user's applications via RM UI
[
https://issues.apache.org/jira/browse/YARN-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du resolved YARN-6891.
--
Resolution: Duplicate
> Can kill other user's applications via RM UI
>
>
> Key: YARN-6891
> URL: https://issues.apache.org/jira/browse/YARN-6891
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
>
> In a secured cluster with UI unsecured which has following config
> {code}
> "hadoop.http.authentication.simple.anonymous.allowed" => "true"
> "hadoop.http.authentication.type" => kerberos
> {code}
> UI can be accessed without any security setting.
> Also any user can kill other user's applications via UI
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6891) Can kill other user's applications via RM UI
Sumana Sathish created YARN-6891:
Summary: Can kill other user's applications via RM UI
Key: YARN-6891
URL: https://issues.apache.org/jira/browse/YARN-6891
Project: Hadoop YARN
Issue Type: Bug
Reporter: Sumana Sathish
Assignee: Junping Du
Priority: Critical
In a secured cluster with UI unsecured which has following config
{code}
"hadoop.http.authentication.simple.anonymous.allowed" => "true"
"hadoop.http.authentication.type" => kerberos
{code}
UI can be accessed without any security setting.
Also any user can kill other user's applications via UI
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103611#comment-16103611
]
Hudson commented on YARN-5548:
--
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12057 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12057/])
YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A
(varunsaxena: rev f66fd11e514fb326fd1f37a88b444a5276f0947b)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestWorkPreservingRMRestart.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationCleanup.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestApplicationLifetimeMonitor.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestAMRestart.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestWorkPreservingRMRestartForNodeLabel.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestNodeBlacklistingOnAMFailures.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestContainerResourceUsage.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec {
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103603#comment-16103603
]
Varun Saxena commented on YARN-5548:
Thanks [~bibinchundatt] for the patch. LGTM.
Will commit it shortly
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6890) If UI is not secured, we allow user to kill other users' job even yarn cluster is secured.
Junping Du created YARN-6890: Summary: If UI is not secured, we allow user to kill other users' job even yarn cluster is secured. Key: YARN-6890 URL: https://issues.apache.org/jira/browse/YARN-6890 Project: Hadoop YARN Issue Type: Bug Reporter: Sumana Sathish Assignee: Junping Du Priority: Critical Configuring SPNEGO for web browser could be a head ache, so many production cluster choose to configure a unsecured UI access even for a secured cluster. In this setup, users (login as some random guy) could watch other users job which is expected. However, the kill button (added in YARN-3249 which enabled by default) shouldn't work in this situation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103517#comment-16103517 ] Ravi Prakash commented on YARN-6889: Also, it would be good if you can publish your methodology and results for the tests you did. > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou >Priority: Critical > > Recent large-scale pressure test single timeline server, and I found the > throughput of timeline server is too small. > I setup multiple servers, and each of them setup multiple processes. Each > process was setup up multiple threads, I send different data size with each > thread. > Although I use different pressures and different scenarios. But the timeline > server processing power is basically similar, and > the ability to process messages is about 70 per second. > It can't meet our requirements, we should improve it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved YARN-6889. Resolution: Duplicate > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou >Priority: Critical > > Recent large-scale pressure test single timeline server, and I found the > throughput of timeline server is too small. > I setup multiple servers, and each of them setup multiple processes. Each > process was setup up multiple threads, I send different data size with each > thread. > Although I use different pressures and different scenarios. But the timeline > server processing power is basically similar, and > the ability to process messages is about 70 per second. > It can't meet our requirements, we should improve it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103515#comment-16103515 ] Ravi Prakash commented on YARN-6889: Which version are you using? Are you aware of https://issues.apache.org/jira/browse/YARN-2928 and https://issues.apache.org/jira/browse/YARN-5355 ? I'm closing this issue as Duplicate. Please re-open if you meant something other than the above two. > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou >Priority: Critical > > Recent large-scale pressure test single timeline server, and I found the > throughput of timeline server is too small. > I setup multiple servers, and each of them setup multiple processes. Each > process was setup up multiple threads, I send different data size with each > thread. > Although I use different pressures and different scenarios. But the timeline > server processing power is basically similar, and > the ability to process messages is about 70 per second. > It can't meet our requirements, we should improve it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6788) Improve performance of resource profile branch
[ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6788: -- Attachment: YARN-6788-YARN-3926.015.patch Fixed test cases and improved FixedResource as per suggestion Daniel > Improve performance of resource profile branch > -- > > Key: YARN-6788 > URL: https://issues.apache.org/jira/browse/YARN-6788 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Sunil G >Assignee: Sunil G >Priority: Blocker > Attachments: YARN-6788-YARN-3926.001.patch, > YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch, > YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch, > YARN-6788-YARN-3926.006.patch, YARN-6788-YARN-3926.007.patch, > YARN-6788-YARN-3926.008.patch, YARN-6788-YARN-3926.009.patch, > YARN-6788-YARN-3926.010.patch, YARN-6788-YARN-3926.011.patch, > YARN-6788-YARN-3926.012.patch, YARN-6788-YARN-3926.013.patch, > YARN-6788-YARN-3926.014.patch, YARN-6788-YARN-3926.015.patch > > > Currently we could see a 15% performance delta with this branch. > Few performance improvements to improve the same. > Also this patch will handle > [comments|https://issues.apache.org/jira/browse/YARN-6761?focusedCommentId=16075418=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16075418] > from [~leftnoteasy]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6889: -- Description: Recent large-scale pressure test single timeline server, and I found the throughput of timeline server is too small. I setup multiple servers, and each of them setup multiple processes. Each process was setup up multiple threads, I send different data size with each thread. Although I use different pressures and different scenarios. But the timeline server processing power is basically similar, and the ability to process messages is about 70 per second. It can't meet our requirements, we should improve it. was:Recent large-scale pressure test timeline server, and I found the > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou > > Recent large-scale pressure test single timeline server, and I found the > throughput of timeline server is too small. > I setup multiple servers, and each of them setup multiple processes. Each > process was setup up multiple threads, I send different data size with each > thread. > Although I use different pressures and different scenarios. But the timeline > server processing power is basically similar, and > the ability to process messages is about 70 per second. > It can't meet our requirements, we should improve it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6889: -- Priority: Critical (was: Major) > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou >Priority: Critical > > Recent large-scale pressure test single timeline server, and I found the > throughput of timeline server is too small. > I setup multiple servers, and each of them setup multiple processes. Each > process was setup up multiple threads, I send different data size with each > thread. > Although I use different pressures and different scenarios. But the timeline > server processing power is basically similar, and > the ability to process messages is about 70 per second. > It can't meet our requirements, we should improve it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6889) The throughput of timeline server is too small
[ https://issues.apache.org/jira/browse/YARN-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YunFan Zhou updated YARN-6889: -- Description: Recent large-scale pressure test timeline server, and I found the > The throughput of timeline server is too small > -- > > Key: YARN-6889 > URL: https://issues.apache.org/jira/browse/YARN-6889 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineserver >Reporter: YunFan Zhou > > Recent large-scale pressure test timeline server, and I found the -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6889) The throughput of timeline server is too small
YunFan Zhou created YARN-6889: - Summary: The throughput of timeline server is too small Key: YARN-6889 URL: https://issues.apache.org/jira/browse/YARN-6889 Project: Hadoop YARN Issue Type: Improvement Components: timelineserver Reporter: YunFan Zhou -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103372#comment-16103372
]
Hadoop QA commented on YARN-5548:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 8s{color} | {color:orange} root: The patch generated 3 new + 963 unchanged
- 8 fixed = 966 total (was 971) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 42m
17s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
40s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5548 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879119/YARN-5548.0018.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ab08eb2374f8 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27a1a5f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16578/artifact/patchprocess/diff-checkstyle-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16578/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
U: . |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16578/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT
[jira] [Commented] (YARN-6875) New aggregated log file format for YARN log aggregation.
[ https://issues.apache.org/jira/browse/YARN-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103346#comment-16103346 ] Jason Lowe commented on YARN-6875: -- Thanks for posting the doc! I'm not a big fan of having a separate file, even temporarily, because log aggregation can already be a large portion of the namenode's write load on large clusters. Having that separate file will increase the namenode write load significantly (approximately 2x per log aggregation cycle if I understand it correctly). Note that the separate index file doesn't solve all the race conditions for the reader. For example, this sequence: # Reader checks for an index file which is not there # Writer begins append and creates index file and starts appending # Reader seeks to the end of the log file but does _not_ find the metainfo structure because the writer is in the process of appending more data This could be mitigated by having the reader repeat the attempt to read process from the beginning so it can rediscover the index file, but this requires that the reader is capable of recognizing that it is _not_ looking at a proper metainfo block on that first attempt. The document does not cover this necessary rinse-repeat cycle required on the reader's part, nor how a reader can reliably identify the case where it is not looking at a proper metainfo block because it happened to try to read just as an append operation occurs. I'm wondering if we can eliminate the need for the index file, and thus reduce the write load on the namenode, by having the reader be able to discover the metainfo file even during an append operation. Similar to sync markers in SequenceFile, we could create a unique, UUID-like sync marker that is written out before every metainfo block. The reader would attempt to find the metainfo block normally (i.e.: seek to the last 64 bits of the file, read the 64-bit offset, then seek back that far to check for a metainfo block). If it finds it then great, the reader is ready to read whatever it is looking for. If it does not find a proper metainfo file then it can start scanning backwards through the file looking for a metainfo sync marker. This scan could be accomplished via a number of ways, such as sequentially scanning backwards block at a time in fixed-size blocks or seeking much farther backwards in a larger chunk that is scanned forward in fixed-sized chunks then repeating if the marker is not found. Isn't this a lot slower for the reader when it has to scan for the marker? Yep, it sure is. However I would argue this is probably a rare occurrence in practice for two reasons: # Logs are often written and never read # Appending is a relatively rare and short-lived operation during the lifespan of a log file By having the writer create the index file, we're essentially optimizing for this rare read-during-append case at the expense of making every writer more expensive. Instead the sync marker approach optimizes for the much more common writing case, putting the load on the reader side if it happens to encounter a log file mid-append during a read operation. I would argue that should be a relatively rare occurrence, and thus I'd rather optimize for the more common case. Another alternative to the index file is using xattrs to associate the last good metainfo offset with the file. However that still leads to approximately the same namenode write ops as the separate index file and requires special support on the underlying filesystem. I'm not a fan of using xattrs myself, but I thought I'd mention it in the interest of covering the potential solutions. > New aggregated log file format for YARN log aggregation. > > > Key: YARN-6875 > URL: https://issues.apache.org/jira/browse/YARN-6875 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-6875-NewLogAggregationFormat-design-doc.pdf > > > T-file is the underlying log format for the aggregated logs in YARN. We have > seen several performance issues, especially for very large log files. > We will introduce a new log format which have better performance for large > log files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6842) Implement a new access type for queue
[ https://issues.apache.org/jira/browse/YARN-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103338#comment-16103338 ] YunFan Zhou commented on YARN-6842: --- [~sunilg] [~templedf] [~bibinchundatt] Could you please review my code? > Implement a new access type for queue > - > > Key: YARN-6842 > URL: https://issues.apache.org/jira/browse/YARN-6842 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Affects Versions: 2.8.2 >Reporter: YunFan Zhou >Assignee: YunFan Zhou > Fix For: 2.8.2 > > Attachments: YARN-6842.001.patch, YARN-6842.002.patch, > YARN-6842.003.patch > > > When we want to access applications of a queue, only we can do is become the > administer of the queue at present. > But sometimes we only want authorize someone view applications of a queue > but not modify operation. > In our current mechanism there isn't any way to meet it, so I will implement > a new access type for queue to solve > this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6802) Support view leaf queue am resource usage in RM web ui
[ https://issues.apache.org/jira/browse/YARN-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103335#comment-16103335 ] YunFan Zhou commented on YARN-6802: --- [~yufeigu] Hi, YuFei, could you please review my code? > Support view leaf queue am resource usage in RM web ui > -- > > Key: YARN-6802 > URL: https://issues.apache.org/jira/browse/YARN-6802 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: YunFan Zhou >Assignee: YunFan Zhou > Fix For: 2.8.0 > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > YARN-6802.001.patch, YARN-6802.002.patch > > > RM Web ui should support view leaf queue am resource usage. > !screenshot-2.png! > I will upload my patch later. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6788) Improve performance of resource profile branch
[
https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103250#comment-16103250
]
Hadoop QA commented on YARN-6788:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
36s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
38s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
20s{color} | {color:green} YARN-3926 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
4s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
YARN-3926 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
40s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
27s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 56s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 13 new + 126 unchanged - 17 fixed = 139 total (was 143) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
31s{color} | {color:red} hadoop-yarn-api in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 24s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 58s{color}
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 52s{color}
| {color:red} hadoop-yarn-server-resourcemanager
[jira] [Comment Edited] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103213#comment-16103213
]
Bibin A Chundatt edited comment on YARN-5548 at 7/27/17 1:37 PM:
-
Test failures in not related to patch attached
was (Author: bibinchundatt):
Test failures in no related to patch attached
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103213#comment-16103213
]
Bibin A Chundatt commented on YARN-5548:
Test failures in no related to patch attached
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6861) Reader API for sub application entities
[
https://issues.apache.org/jira/browse/YARN-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103125#comment-16103125
]
Hadoop QA commented on YARN-6861:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-5355 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
55s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} YARN-5355 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 33s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 5 new +
30 unchanged - 0 fixed = 35 total (was 30) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6861 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879160/YARN-6861-YARN-5355.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b0f61ce2229a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5355 / e86c211 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16576/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
|
| whitespace |
[jira] [Commented] (YARN-6762) ChildQueue can override AbstractCSQueue.hasChildQueues to return false always
[
https://issues.apache.org/jira/browse/YARN-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103121#comment-16103121
]
Hadoop QA commented on YARN-6762:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 13s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6762 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875672/YARN-6762.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 01d74eae1184 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27a1a5f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/16575/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16575/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16575/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ChildQueue can override AbstractCSQueue.hasChildQueues to return false always
>
[jira] [Commented] (YARN-6494) add mounting of HDFS Short-Circuit path for docker containers
[
https://issues.apache.org/jira/browse/YARN-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103113#comment-16103113
]
Shane Kumpf commented on YARN-6494:
---
Thanks for the patch [~Jaeboo]! I do have a concern as this automatically bind
mounts the dn socket in every container if short circuit reads are enabled. I
still feel this should be handled via YARN-5534 (or the "auto bind mount list"
approach, if that is pursued). Additionally, in a traditional HDFS cluster, we
only have one DataNode process per server, is it safe to have multiple DNs on a
single server interacting with this socket?
> add mounting of HDFS Short-Circuit path for docker containers
> -
>
> Key: YARN-6494
> URL: https://issues.apache.org/jira/browse/YARN-6494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Jaeboo Jeong
>Assignee: Jaeboo Jeong
> Attachments: YARN-6494.001.patch, YARN-6494.002.patch
>
>
> Currently there is a error message about HDFS short-circuit when docker
> container start.
> {code}
> WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: error
> creating DomainSocket
> java.net.ConnectException: connect(2) error: No such file or directory when
> trying to connect to ‘xxx’
> at org.apache.hadoop.net.unix.DomainSocket.connect0(Native Method)
> at org.apache.hadoop.net.unix.DomainSocket.connect(DomainSocket.java:250)
> at
> org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory.createSocket(DomainSocketFactory.java:164)
> at
> org.apache.hadoop.hdfs.BlockReaderFactory.nextDomainPeer(BlockReaderFactory.java:752)
> ...
> {code}
> if dfs.client.read.shortcircuit is true and dfs.domain.socket.path isn't
> equal “”, we need to mount volume for short-circuit path.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6788) Improve performance of resource profile branch
[
https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-6788:
--
Attachment: YARN-6788-YARN-3926.014.patch
Thanks [~templedf]
Updating new patch based on your comments. There are some comments which i have
some doubts and clarification provided. Kindly check the same.
bq.Is there a reason to have "known" (or "readOnly") in the variable names?
Doesn't seem to add any useful information.
Returning a map and then iterating on its values or keyset by using iterator is
performance bottleneck. As per my profiling analysis, it contributed around 15%
of performance as these methods are used as hotspot (Resources and
ResourceCalculator api). Hence we keep a readonly DS to return as value for all
these callers and work on same to loop or access on single entry based on index.
bq.ResourceUtils.resourceNameToIndex should be RESOURCE_NAME_TO_INDEX
Jus a doubt here. Why we need this to be in CAPS, is its because its final and
static?
bq.In ResourceUtils.initializeResourcesMap(), it seems to me that {{
knownResourceTypes = Collections.unmodifiableMap(resourceInformationMap);}}
should be moved into ResourceUtils.updateReadOnlyResources()
I think we will add one more argument to
ResourceUtils.updateReadOnlyResources/knownResources to take map and then copy
it. But its better to do as early as possible to avoid an extra argument as
possible.
bq.Why doesn't ResourceUtils.getResourceNamesArray() call getResourceTypes()?
As mentioned in first comment, cost of looping on iterator of map.values() is
costlierlieer. Hence its better to operate on simple array.
bq.The for loop to update resource names in ResourceUtils.getResourceTypes() is
a lovely opportunity for a lambda.
Yes. If possible, i would like take it up separately as its not affecting
functionality.
bq.n the for loop to update resource names in ResourceUtils.getResourceTypes()
memory and CPU aren't handled a priori, and there's no guarantee that the set
iteration will match the map values iteration in updateReadOnlyResources()
knownResourceNamesArray, resourceNameToIndex are in sync and in order. This is
because in updateKnownResources, we update Mem and CPU as first two entries and
other resource in following indices. Then resourceNameToIndex is derived from
knownResourceNamesArray itself. All our getter in Resources and
DominantCalcultor uses these DS. getResourceTypes is not used for normal
computation. It just used for initialization.
With this assumption we can assume resource will be in order. Please correct me
if you feel some issues.
bq.In the FixedValueResource() constructor, the resources array is not created
according to the resource type index, so resource lookups will fail.
In {{FixedValueResource.initResourceMap}} we use
{{ResourceUtils.getResourceTypesArray}} to get the index of resources. then we
created all resources in order.
bq.In Resources.addTo() the variable in the foreach should be named lhsValue
instead of declaring a new variable for it.
I have a loop optimization patch ready in this area. And i ll upload in another
jira. I would like that patch separate and on top of this since its loop
optimization for all apis in Resources and DominantResourceCalculator class.
Hence I will take this comment in that one if its fine.
bq.There's now no class that extends Resource other than BaseResource. Maybe
it's time to merge BaseResource into Resource?
I think its too early for that. Resource is basic public class and BaseResource
is internal class to operate on resource compared to ResourcePBImpl. I think we
can revise this once its merged and used well.
bq.In ResourcePBImpl.initResources(), the copy to readOnlyResources should
probably be moved into initResourcesMap()
Actually resources is created in initResourcesMap, but its populated in main
array. So I placed it there.
Regarding lock object in ResourceUtils, we use a double lock model to handle
data corruption during init. I guess this may be a cleaner approach ,how do you
feel?
I will wait for jenkins to see if any test error, if so i ll update another
patch
> Improve performance of resource profile branch
> --
>
> Key: YARN-6788
> URL: https://issues.apache.org/jira/browse/YARN-6788
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Blocker
> Attachments: YARN-6788-YARN-3926.001.patch,
> YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch,
> YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch,
> YARN-6788-YARN-3926.006.patch, YARN-6788-YARN-3926.007.patch,
> YARN-6788-YARN-3926.008.patch, YARN-6788-YARN-3926.009.patch,
> YARN-6788-YARN-3926.010.patch,
[jira] [Commented] (YARN-6861) Reader API for sub application entities
[ https://issues.apache.org/jira/browse/YARN-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103083#comment-16103083 ] Rohith Sharma K S commented on YARN-6861: - Call for initial review. cc :/ [~varun_saxena] [~vrushalic] > Reader API for sub application entities > --- > > Key: YARN-6861 > URL: https://issues.apache.org/jira/browse/YARN-6861 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelinereader >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S > Attachments: YARN-6861-YARN-5355.001.patch > > > YARN-6733 and YARN-6734 writes data into sub application table. There should > be a way to read those entities. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6861) Reader API for sub application entities
[
https://issues.apache.org/jira/browse/YARN-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-6861:
Attachment: YARN-6861-YARN-5355.001.patch
updating v0 patch for REST API to read from sub application table. It is a
vague patch, but has complete flow path from web service to getting result.
Test cases and other modification like adding UID should be done.
New REST API added is {{/entities/$entitytype}}.
> Reader API for sub application entities
> ---
>
> Key: YARN-6861
> URL: https://issues.apache.org/jira/browse/YARN-6861
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
> Attachments: YARN-6861-YARN-5355.001.patch
>
>
> YARN-6733 and YARN-6734 writes data into sub application table. There should
> be a way to read those entities.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6762) ChildQueue can override AbstractCSQueue.hasChildQueues to return false always
[
https://issues.apache.org/jira/browse/YARN-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R reassigned YARN-6762:
---
Assignee: Naganarasimha G R
> ChildQueue can override AbstractCSQueue.hasChildQueues to return false always
> --
>
> Key: YARN-6762
> URL: https://issues.apache.org/jira/browse/YARN-6762
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
>Priority: Trivial
> Attachments: YARN-6762.001.patch
>
>
> As improvement over YARN-6681, we can override to return false for
> {{AbstractCSQueue.hasChildQueues}} in the ChildQueue
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6872) Ensure apps could run given NodeLabels are disabled post RM switchover/restart
[
https://issues.apache.org/jira/browse/YARN-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103054#comment-16103054
]
Hadoop QA commented on YARN-6872:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 122 unchanged - 0 fixed = 123 total (was 122)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 53s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6872 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878988/YARN-6872.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4ada1cf81f0d 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27a1a5f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16574/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/16574/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16574/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16574/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT
[jira] [Commented] (YARN-6734) Ensure sub-application user is extracted & sent to timeline service
[
https://issues.apache.org/jira/browse/YARN-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103007#comment-16103007
]
Hadoop QA commented on YARN-6734:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} YARN-5355 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
57s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} YARN-5355 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} YARN-5355 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new +
12 unchanged - 0 fixed = 13 total (was 12) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
43s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 44s{color}
| {color:red} hadoop-yarn-server-timelineservice-hbase-tests in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6734 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879138/YARN-6734-YARN-5355.004.patch
|
| Optional Tests | asflicense compile javac javadoc
[jira] [Commented] (YARN-6872) Ensure apps could run given NodeLabels are disabled post RM switchover/restart
[ https://issues.apache.org/jira/browse/YARN-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103000#comment-16103000 ] Rohith Sharma K S commented on YARN-6872: - Ahh I see. I submitted the patch for triggering Jenkins > Ensure apps could run given NodeLabels are disabled post RM switchover/restart > -- > > Key: YARN-6872 > URL: https://issues.apache.org/jira/browse/YARN-6872 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-6872.001.patch > > > Post YARN-6031, few apps could be failed during recovery provided they had > some label requirements for AM and labels were disable post RM > restart/switchover. As discussed in YARN-6031, its better to run such apps as > it may be long running apps as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6888) Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created
[ https://issues.apache.org/jira/browse/YARN-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102986#comment-16102986 ] Rohith Sharma K S commented on YARN-6888: - +1 LGTM, will commit it later if no more objections > Refactor AppLevelTimelineCollector such that RM does not have aggregator > threads created > > > Key: YARN-6888 > URL: https://issues.apache.org/jira/browse/YARN-6888 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Vrushali C > Attachments: YARN-6888-YARN-5355.0001.patch > > > Currently both RM and NM use the same AppLevelTimelineCollector class. The NM > requires aggregator threads per application so that it can perform in memory > aggregation for application metrics but the RM does not need this. Since they > share the code, RM has a bunch of "TimelineCollector Aggregation" threads > created (one per running app). > Filing jira to refactor AppLevelTimelineCollector such that RM does not have > aggregator threads created. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6872) Ensure apps could run given NodeLabels are disabled post RM switchover/restart
[ https://issues.apache.org/jira/browse/YARN-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102978#comment-16102978 ] Sunil G commented on YARN-6872: --- Thanks [~rohithsharma] When new container requests comes, its been validated in ApplicationMasterService#allocate and InvalidResourceRequestException is thrown back to AM. Hence scheduler will not have problem. At most what could happen is, AM container may never get scheduled and it may remain in scheduler queue until admin corrects the pblm with labels. (enabled/disable) > Ensure apps could run given NodeLabels are disabled post RM switchover/restart > -- > > Key: YARN-6872 > URL: https://issues.apache.org/jira/browse/YARN-6872 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-6872.001.patch > > > Post YARN-6031, few apps could be failed during recovery provided they had > some label requirements for AM and labels were disable post RM > restart/switchover. As discussed in YARN-6031, its better to run such apps as > it may be long running apps as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102977#comment-16102977
]
Hadoop QA commented on YARN-5548:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 7s{color} | {color:orange} root: The patch generated 3 new + 963 unchanged
- 8 fixed = 966 total (was 971) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 20s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
17s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}131m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5548 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879119/YARN-5548.0018.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b1cda2de7ea1 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27a1a5f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16571/artifact/patchprocess/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/16571/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16571/testReport/ |
| modules | C:
[jira] [Commented] (YARN-6130) [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM
[ https://issues.apache.org/jira/browse/YARN-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102968#comment-16102968 ] Rohith Sharma K S commented on YARN-6130: - cc :/ [~jianhe] would you take a look at last patch i.e *-05.patch please? > [ATSv2 Security] Generate a delegation token for AM when app collector is > created and pass it to AM via NM and RM > - > > Key: YARN-6130 > URL: https://issues.apache.org/jira/browse/YARN-6130 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-5355-merge-blocker > Attachments: YARN-6130-YARN-5355.01.patch, > YARN-6130-YARN-5355.02.patch, YARN-6130-YARN-5355.03.patch, > YARN-6130-YARN-5355.04.patch, YARN-6130-YARN-5355.05.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6734) Ensure sub-application user is extracted & sent to timeline service
[ https://issues.apache.org/jira/browse/YARN-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6734: Attachment: YARN-6734-YARN-5355.004.patch updated the patch with below delta changes # Modified the TimelineWriter#write API. The new API is ((write(TimelineCollectorContext context, TimelineEntities data, UserGroupInformation callerUgi)}} # All the corresponding caller has changed with new API. > Ensure sub-application user is extracted & sent to timeline service > --- > > Key: YARN-6734 > URL: https://issues.apache.org/jira/browse/YARN-6734 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Rohith Sharma K S > Attachments: YARN-6734-YARN-5355.001.patch, > YARN-6734-YARN-5355.002.patch, YARN-6734-YARN-5355.003.patch, > YARN-6734-YARN-5355.004.patch > > > After a discussion with Tez folks, we have been thinking over introducing a > table to store sub-application information. YARN-6733 > For example, if a Tez session runs for a certain period as User X and runs a > few AMs. These AMs accept DAGs from other users. Tez will execute these dags > with a doAs user. ATSv2 should store this information in a new table perhaps > called as "sub_application" table. > YARN-6733 tracks the code changes needed for table schema creation. > This jira tracks writing to that table, updating the user name fields to > include sub-application user etc. This would mean adding a field to Flow > Context which can store an additional user -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102897#comment-16102897
]
Akira Ajisaka commented on YARN-5728:
-
Hi [~rchiang], would you please review this?
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5728.001.patch, YARN-5728.002.patch,
> YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.processWaitTimeAndRetryInfo(RetryInvocationHandler.java:130)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:107)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy85.nodeHeartbeat(Unknown Source)
> at
> org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102892#comment-16102892
]
Hadoop QA commented on YARN-5728:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
25s{color} | {color:green} hadoop-yarn-server-tests in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5728 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879127/YARN-5728.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d5d2afc4fac2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27a1a5f |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16572/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16572/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
>
[jira] [Updated] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5728:
Attachment: YARN-5728.002.patch
002: fix grammar
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5728.001.patch, YARN-5728.002.patch,
> YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.processWaitTimeAndRetryInfo(RetryInvocationHandler.java:130)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:107)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy85.nodeHeartbeat(Unknown Source)
> at
> org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-5728:
---
Assignee: Akira Ajisaka
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5728.001.patch, YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.processWaitTimeAndRetryInfo(RetryInvocationHandler.java:130)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:107)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy85.nodeHeartbeat(Unknown Source)
> at
> org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5728:
Attachment: YARN-5728.001.patch
01 patch: Set hostname to "localhost" for MiniYARNCluster RM if the
InetSocketAddress created by NetUtils.createSocketAddrForHost(hostname, port)
is unresolved.
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
> Attachments: YARN-5728.001.patch, YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.processWaitTimeAndRetryInfo(RetryInvocationHandler.java:130)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:107)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy85.nodeHeartbeat(Unknown Source)
> at
> org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6130) [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM
[ https://issues.apache.org/jira/browse/YARN-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102844#comment-16102844 ] Varun Saxena commented on YARN-6130: bq. token renewer is set to application owner. Is it intended? Since AppCollector runs as part of NM axillary service, renewer should be NM user right? Actually I was initially thinking of providing an ability to renew token from Timeline client too. For tokens for offline collectors for instance. And if we allow for them, the API has to be generic enough to allow renewal for app tokens as well with the facility of automatic renewal. That is why I had kept it as application owner at that time. As we will renew locally i.e. from NM and will have the token cached in collector, it wont matter. However, security design for offline collectors isn't yet finalized. So technically right now, the renewer will always be NM. Will change it in next patch. > [ATSv2 Security] Generate a delegation token for AM when app collector is > created and pass it to AM via NM and RM > - > > Key: YARN-6130 > URL: https://issues.apache.org/jira/browse/YARN-6130 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-5355-merge-blocker > Attachments: YARN-6130-YARN-5355.01.patch, > YARN-6130-YARN-5355.02.patch, YARN-6130-YARN-5355.03.patch, > YARN-6130-YARN-5355.04.patch, YARN-6130-YARN-5355.05.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6888) Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created
[
https://issues.apache.org/jira/browse/YARN-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102829#comment-16102829
]
Hadoop QA commented on YARN-6888:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-5355 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
3s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} YARN-5355 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6888 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879117/YARN-6888-YARN-5355.0001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 798aee679bb3 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5355 / e86c211 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16570/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16570/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Refactor AppLevelTimelineCollector such that RM does not have aggregator
> threads created
>
>
> Key: YARN-6888
> URL: https://issues.apache.org/jira/browse/YARN-6888
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Vrushali C
>Assignee: Vrushali C
> Attachments: YARN-6888-YARN-5355.0001.patch
>
>
>
[jira] [Updated] (YARN-5548) Use MockRMMemoryStateStore to reduce test failures
[
https://issues.apache.org/jira/browse/YARN-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated YARN-5548:
---
Attachment: YARN-5548.0018.patch
Attaching after checkstyle fix. {{TestWorkPreservingRMRestart}} and
{{TestRMRestart}} are already existing issues.
> Use MockRMMemoryStateStore to reduce test failures
> --
>
> Key: YARN-5548
> URL: https://issues.apache.org/jira/browse/YARN-5548
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Labels: oct16-easy, test
> Attachments: YARN-5548.0001.patch, YARN-5548.0002.patch,
> YARN-5548.0003.patch, YARN-5548.0004.patch, YARN-5548.0005.patch,
> YARN-5548.0006.patch, YARN-5548.0007.patch, YARN-5548.0008.patch,
> YARN-5548.0009.patch, YARN-5548.0010.patch, YARN-5548.0011.patch,
> YARN-5548.0012.patch, YARN-5548.0013.patch, YARN-5548.0014.patch,
> YARN-5548.0015.patch, YARN-5548.0016.patch, YARN-5548.0017.patch,
> YARN-5548.0018.patch
>
>
> https://builds.apache.org/job/PreCommit-YARN-Build/12850/testReport/org.apache.hadoop.yarn.server.resourcemanager/TestRMRestart/testFinishedAppRemovalAfterRMRestart/
> {noformat}
> Error Message
> Stacktrace
> java.lang.AssertionError: expected null, but was: application_submission_context { application_id { id: 1 cluster_timestamp:
> 1471885197388 } application_name: "" queue: "default" priority { priority: 0
> } am_container_spec { } cancel_tokens_when_complete: true maxAppAttempts: 2
> resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
> keep_containers_across_application_attempts: false
> attempt_failures_validity_interval: 0 am_container_resource_request {
> priority { priority: 0 } resource_name: "*" capability { memory: 1024
> virtual_cores: 1 } num_containers: 0 relax_locality: true
> node_label_expression: "" execution_type_request { execution_type: GUARANTEED
> enforce_execution_type: false } } } user: "jenkins" start_time: 1471885197417
> application_state: RMAPP_FINISHED finish_time: 1471885197478>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotNull(Assert.java:664)
> at org.junit.Assert.assertNull(Assert.java:646)
> at org.junit.Assert.assertNull(Assert.java:656)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart.testFinishedAppRemovalAfterRMRestart(TestRMRestart.java:1656)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102805#comment-16102805
]
Akira Ajisaka commented on YARN-5728:
-
Ran this test with some log levels raised. Probably this is the root cause.
{noformat}
2017-07-27 06:49:49,715 INFO [Thread-510] retry.RetryInvocationHandler
(RetryInvocationHandler.java:invokeOnce(97)) - Call#-2:
ResourceTracker.nodeHeartbeat([org.apache.hadoop.yarn.server.api.protocolrecords.impl.pb.NodeHeartbeatRequestPBImpl@721ff5c2])
java.net.UnknownHostException: Invalid host name: local host is: (unknown);
destination host is: "649448b35b66":36903; java.net.UnknownHostException; For
more details see: http://wiki.apache.org/hadoop/UnknownHost
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:815)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:758)
at org.apache.hadoop.ipc.Client$Connection.(Client.java:444)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1531)
at org.apache.hadoop.ipc.Client.call(Client.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1346)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy87.nodeHeartbeat(Unknown Source)
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.nodeHeartbeat(ResourceTrackerPBClientImpl.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:410)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:164)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:156)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:347)
at com.sun.proxy.$Proxy88.nodeHeartbeat(Unknown Source)
at
org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: java.net.UnknownHostException
at org.apache.hadoop.ipc.Client$Connection.(Client.java:445)
... 27 more
{noformat}
This is very similar to YARN-6150. I'll attach a patch to fix this.
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
> Attachments: YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
>
[jira] [Updated] (YARN-6888) Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created
[ https://issues.apache.org/jira/browse/YARN-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-6888: - Attachment: YARN-6888-YARN-5355.0001.patch Attaching v001. Created subclass AppLevelTimelineCollectorWithAgg that handles the thread related agg stuff. Creating this class in the NM and keeping the earlier AppLevelTimelineCollector class in RM. > Refactor AppLevelTimelineCollector such that RM does not have aggregator > threads created > > > Key: YARN-6888 > URL: https://issues.apache.org/jira/browse/YARN-6888 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Vrushali C > Attachments: YARN-6888-YARN-5355.0001.patch > > > Currently both RM and NM use the same AppLevelTimelineCollector class. The NM > requires aggregator threads per application so that it can perform in memory > aggregation for application metrics but the RM does not need this. Since they > share the code, RM has a bunch of "TimelineCollector Aggregation" threads > created (one per running app). > Filing jira to refactor AppLevelTimelineCollector such that RM does not have > aggregator threads created. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6888) Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created
Vrushali C created YARN-6888: Summary: Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created Key: YARN-6888 URL: https://issues.apache.org/jira/browse/YARN-6888 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vrushali C Assignee: Vrushali C Currently both RM and NM use the same AppLevelTimelineCollector class. The NM requires aggregator threads per application so that it can perform in memory aggregation for application metrics but the RM does not need this. Since they share the code, RM has a bunch of "TimelineCollector Aggregation" threads created (one per running app). Filing jira to refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5728) TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
[
https://issues.apache.org/jira/browse/YARN-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102772#comment-16102772
]
Akira Ajisaka commented on YARN-5728:
-
I could reproduce this error locally by using Docker environment via
start-build-env.sh. Now I'm investigating the root cause.
> TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout
>
>
> Key: YARN-5728
> URL: https://issues.apache.org/jira/browse/YARN-5728
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
> Attachments: YARN-5728.01.patch
>
>
> TestMiniYARNClusterNodeUtilization.testUpdateNodeUtilization is failing by
> timeout.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/192/testReport/junit/org.apache.hadoop.yarn.server/TestMiniYarnClusterNodeUtilization/testUpdateNodeUtilization/
> {noformat}
> java.lang.Exception: test timed out after 6 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.processWaitTimeAndRetryInfo(RetryInvocationHandler.java:130)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:107)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy85.nodeHeartbeat(Unknown Source)
> at
> org.apache.hadoop.yarn.server.TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization(TestMiniYarnClusterNodeUtilization.java:113)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
