[jira] [Updated] (YARN-6040) Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes
[
https://issues.apache.org/jira/browse/YARN-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-6040:
-
Attachment: YARN-6040.branch-2.009.patch
Attached ver.9 patch for branch-2.
> Introduce api independent PendingAsk to replace usage of ResourceRequest
> within Scheduler classes
> -
>
> Key: YARN-6040
> URL: https://issues.apache.org/jira/browse/YARN-6040
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-6040.001.patch, YARN-6040.002.patch,
> YARN-6040.003.patch, YARN-6040.004.patch, YARN-6040.005.patch,
> YARN-6040.006.patch, YARN-6040.007.patch, YARN-6040.branch-2.007.patch,
> YARN-6040.branch-2.008.patch, YARN-6040.branch-2.009.patch
>
>
> As mentioned by YARN-5906, currently schedulers are using ResourceRequest
> heavily so it will be very hard to adopt the new PowerfulResourceRequest
> (YARN-4902).
> This JIRA is the 2nd step of refactoring, which remove usage of
> ResourceRequest from AppSchedulingInfo / SchedulerApplicationAttempt. Instead
> of returning ResourceRequest, it returns a lightweight and API-independent
> object - {{PendingAsk}}.
> The only remained ResourceRequest API of AppSchedulingInfo will be used by
> web service to get list of ResourceRequests.
> So after this patch, usage of ResourceRequest will be isolated inside
> AppSchedulingInfo, so it will be more flexible to update internal data
> structure and upgrade old ResourceRequest API to new.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake in FileSystemRMStateStore
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965378#comment-15965378 ] Yeliang Cang commented on YARN-6463: [~Naganarasimha], thank you for reviewing the code. Thank you for adding me to the list, I appreciate that! > correct spelling mistake in FileSystemRMStateStore > -- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Yeliang Cang >Priority: Trivial > Labels: newbie > Fix For: 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5331) Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem
[
https://issues.apache.org/jira/browse/YARN-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965318#comment-15965318
]
Hadoop QA commented on YARN-5331:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 9 new + 42 unchanged - 1 fixed = 51 total (was 43) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager
generated 1 new + 880 unchanged - 0 fixed = 881 total (was 880) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 18s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | YARN-5331 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862960/YARN-5331.006.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 94e3fd272d41 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3a91376 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15591/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/15591/artifact/patchprocess/whitespace-eol.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-YARN-Build/15591/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-5331) Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem
[ https://issues.apache.org/jira/browse/YARN-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965276#comment-15965276 ] Sangeetha Abdu Jyothi commented on YARN-5331: - Thanks for the comments, [~seanpo03]! I had missed the negative capacity issue earlier. Thanks for catching that! > Extend RLESparseResourceAllocation with period for supporting recurring > reservations in YARN ReservationSystem > -- > > Key: YARN-5331 > URL: https://issues.apache.org/jira/browse/YARN-5331 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Subru Krishnan >Assignee: Sangeetha Abdu Jyothi > Labels: oct16-medium > Attachments: YARN-5331.001.patch, YARN-5331.002.patch, > YARN-5331.003.patch, YARN-5331.004.patch, YARN-5331.005.patch, > YARN-5331.006.patch > > > YARN-5326 proposes adding native support for recurring reservations in the > YARN ReservationSystem. This JIRA is a sub-task to add a > PeriodicRLESparseResourceAllocation. Please refer to the design doc in the > parent JIRA for details. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5331) Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem
[ https://issues.apache.org/jira/browse/YARN-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangeetha Abdu Jyothi updated YARN-5331: Attachment: YARN-5331.006.patch > Extend RLESparseResourceAllocation with period for supporting recurring > reservations in YARN ReservationSystem > -- > > Key: YARN-5331 > URL: https://issues.apache.org/jira/browse/YARN-5331 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Subru Krishnan >Assignee: Sangeetha Abdu Jyothi > Labels: oct16-medium > Attachments: YARN-5331.001.patch, YARN-5331.002.patch, > YARN-5331.003.patch, YARN-5331.004.patch, YARN-5331.005.patch, > YARN-5331.006.patch > > > YARN-5326 proposes adding native support for recurring reservations in the > YARN ReservationSystem. This JIRA is a sub-task to add a > PeriodicRLESparseResourceAllocation. Please refer to the design doc in the > parent JIRA for details. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5331) Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem
[ https://issues.apache.org/jira/browse/YARN-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangeetha Abdu Jyothi updated YARN-5331: Attachment: YARN-5331.006.patch > Extend RLESparseResourceAllocation with period for supporting recurring > reservations in YARN ReservationSystem > -- > > Key: YARN-5331 > URL: https://issues.apache.org/jira/browse/YARN-5331 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Subru Krishnan >Assignee: Sangeetha Abdu Jyothi > Labels: oct16-medium > Attachments: YARN-5331.001.patch, YARN-5331.002.patch, > YARN-5331.003.patch, YARN-5331.004.patch, YARN-5331.005.patch > > > YARN-5326 proposes adding native support for recurring reservations in the > YARN ReservationSystem. This JIRA is a sub-task to add a > PeriodicRLESparseResourceAllocation. Please refer to the design doc in the > parent JIRA for details. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5331) Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem
[ https://issues.apache.org/jira/browse/YARN-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangeetha Abdu Jyothi updated YARN-5331: Attachment: (was: YARN-5331.006.patch) > Extend RLESparseResourceAllocation with period for supporting recurring > reservations in YARN ReservationSystem > -- > > Key: YARN-5331 > URL: https://issues.apache.org/jira/browse/YARN-5331 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Subru Krishnan >Assignee: Sangeetha Abdu Jyothi > Labels: oct16-medium > Attachments: YARN-5331.001.patch, YARN-5331.002.patch, > YARN-5331.003.patch, YARN-5331.004.patch, YARN-5331.005.patch > > > YARN-5326 proposes adding native support for recurring reservations in the > YARN ReservationSystem. This JIRA is a sub-task to add a > PeriodicRLESparseResourceAllocation. Please refer to the design doc in the > parent JIRA for details. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6467) CSQueueMetrics needs to update the current metrics for default partition only
[ https://issues.apache.org/jira/browse/YARN-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-6467: --- Assignee: Naganarasimha G R > CSQueueMetrics needs to update the current metrics for default partition only > - > > Key: YARN-6467 > URL: https://issues.apache.org/jira/browse/YARN-6467 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.8.0, 2.7.3, 3.0.0-alpha2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > > As a followup to YARN-6195, we need to update existing metrics to only > default Partition. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3666) Federation Intercepting and propagating AM-RM communications
[ https://issues.apache.org/jira/browse/YARN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-3666: --- Attachment: YARN-3666-YARN-2915.v2.patch fix yetus issues in v2. Won't fix for findbug warning: amRegistrationResponse in the catch can be set to non-null by a concurrent thread. > Federation Intercepting and propagating AM-RM communications > > > Key: YARN-3666 > URL: https://issues.apache.org/jira/browse/YARN-3666 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Botong Huang > Attachments: YARN-3666-YARN-2915.v1.patch, > YARN-3666-YARN-2915.v2.patch > > > In order, to support transparent "spanning" of jobs across sub-clusters, all > AM-RM communications are proxied (via YARN-2884). > This JIRA tracks federation-specific mechanisms that decide how to > "split/broadcast" requests to the RMs and "merge" answers to > the AM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3666) Federation Intercepting and propagating AM-RM communications
[
https://issues.apache.org/jira/browse/YARN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965096#comment-15965096
]
Hadoop QA commented on YARN-3666:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
26s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 20 new + 31 unchanged - 0 fixed = 51 total (was 31)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager
generated 8 new + 231 unchanged - 0 fixed = 239 total (was 231) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
22s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| | Redundant nullcheck of
org.apache.hadoop.yarn.server.nodemanager.amrmproxy.FederationInterceptor.amRegistrationResponse
which is known to be null in
org.apache.hadoop.yarn.server.nodemanager.amrmproxy.FederationInterceptor.registerApplicationMaster(RegisterApplicationMasterRequest)
Redundant null check at FederationInterceptor.java:is known to be null in
org.apache.hadoop.yarn.server.nodemanager.amrmproxy.FederationInterceptor.registerApplicationMaster(RegisterApplicationMasterRequest)
Redundant null check at FederationInterceptor.java:[line 219] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | YARN-3666 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862928/YARN-3666-YARN-2915.v1.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux cea35644fb64 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (YARN-6439) Fix ReservationSystem creation of default ReservationQueue
[ https://issues.apache.org/jira/browse/YARN-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965055#comment-15965055 ] Hudson commented on YARN-6439: -- ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #11571 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11571/]) YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (wangda: rev 4d4ad0ebb76198c90c5317f19ce85a6696132c8f) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/TestSchedulerPlanFollowerBase.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/PlanQueue.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerDynamicBehavior.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/TestCapacitySchedulerPlanFollower.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/TestFairSchedulerPlanFollower.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerQueueManager.java > Fix ReservationSystem creation of default ReservationQueue > -- > > Key: YARN-6439 > URL: https://issues.apache.org/jira/browse/YARN-6439 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-6439.v0.patch, YARN-6439.v1.patch, > YARN-6439.v2.patch > > > The ReservationSystem does not create the default ReservationQueue till the > PlanFollower runs the first time. This is problematic for SLS runs, and might > show up in some other corner case. Inlining creation of the queue as part of > PlanQueue creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3666) Federation Intercepting and propagating AM-RM communications
[ https://issues.apache.org/jira/browse/YARN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-3666: --- Attachment: YARN-3666-YARN-2915.v1.patch v1 patch: Federation Interceptor part1, which forward everything to home RM, no secondary subclusters yet. > Federation Intercepting and propagating AM-RM communications > > > Key: YARN-3666 > URL: https://issues.apache.org/jira/browse/YARN-3666 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Botong Huang > Attachments: YARN-3666-YARN-2915.v1.patch > > > In order, to support transparent "spanning" of jobs across sub-clusters, all > AM-RM communications are proxied (via YARN-2884). > This JIRA tracks federation-specific mechanisms that decide how to > "split/broadcast" requests to the RMs and "merge" answers to > the AM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6466) Provide shaded framework jar for containers
[ https://issues.apache.org/jira/browse/YARN-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965019#comment-15965019 ] Andrew Wang commented on YARN-6466: --- Are we planning to get this in for 3.0.0-alpha3 in approximately 1 month? > Provide shaded framework jar for containers > --- > > Key: YARN-6466 > URL: https://issues.apache.org/jira/browse/YARN-6466 > Project: Hadoop YARN > Issue Type: New Feature > Components: build, yarn >Affects Versions: 3.0.0-alpha1 >Reporter: Sean Busbey > > We should build on the existing shading work to provide a jar with all of the > bits needed within a YARN application's container to talk to the resource > manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6466) Provide shaded framework jar for containers
[ https://issues.apache.org/jira/browse/YARN-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated YARN-6466: -- Affects Version/s: 3.0.0-alpha1 Target Version/s: 3.0.0-alpha3 > Provide shaded framework jar for containers > --- > > Key: YARN-6466 > URL: https://issues.apache.org/jira/browse/YARN-6466 > Project: Hadoop YARN > Issue Type: New Feature > Components: build, yarn >Affects Versions: 3.0.0-alpha1 >Reporter: Sean Busbey > > We should build on the existing shading work to provide a jar with all of the > bits needed within a YARN application's container to talk to the resource > manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6443) Prune number of SchedulerRequestKeys to examine in the Scheduler inner loop to improve performance
[ https://issues.apache.org/jira/browse/YARN-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964939#comment-15964939 ] Wangda Tan commented on YARN-6443: -- [~asuresh], The proposal to allow reordering of different allocation id with the same priority looks like a better/cleaner problem definition. bq. The patch you posted seemed to be doing something similar, but I guess that will work if there are multiple outstanding reqs againsts the same SchedulerKey. What we requires is efficient matching across schedulerKeys I think we should move to an approach which can look up multiple nodes for one or more resource requests, otherwise it will be hard for locality scheduling in a large cluster. Setting up missed opportunities, etc. can only alleviate the problem but cannot solve root cause of the issue. Looking forward to your patch so we can have more discussions based on that. > Prune number of SchedulerRequestKeys to examine in the Scheduler inner loop > to improve performance > -- > > Key: YARN-6443 > URL: https://issues.apache.org/jira/browse/YARN-6443 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler, fairscheduler >Reporter: Arun Suresh >Assignee: Hitesh Sharma > > Currently the Schedulers examine an applications pending Requests in Priority > order. This JIRA proposes to introduce a flag (either via the > ApplicationMasterService::registerApplication() or via some Scheduler > configuration) to favor an ordering that is baised to the node that is > currently heartbeating by relaxing the priority constraint. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6469) Extending Synthetic Load Generator for recurring reservation
Carlo Curino created YARN-6469: -- Summary: Extending Synthetic Load Generator for recurring reservation Key: YARN-6469 URL: https://issues.apache.org/jira/browse/YARN-6469 Project: Hadoop YARN Issue Type: Sub-task Reporter: Carlo Curino -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3663) Federation State and Policy Store (DBMS implementation)
[
https://issues.apache.org/jira/browse/YARN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964869#comment-15964869
]
Hadoop QA commented on YARN-3663:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
48s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
22s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
23s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
34s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
7s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 5s{color} | {color:orange} root: The patch generated 8 new + 206 unchanged
- 0 fixed = 214 total (was 206) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
7s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 39s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 21s{color}
| {color:red} hadoop-yarn-server-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
|
| |
org.apache.hadoop.yarn.server.federation.store.impl.SQLFederationStateStore.getApplicationsHomeSubCluster(GetApplicationsHomeSubClusterRequest)
may fail to clean up java.sql.ResultSet Obligation to clean up resource
[jira] [Updated] (YARN-6468) Add Max AM Resource to FairScheduler WebUI
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Description: Max AM Resource indicates maximum resources AM can use in a queue. > Add Max AM Resource to FairScheduler WebUI > -- > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.9.0, 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > > Max AM Resource indicates maximum resources AM can use in a queue. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6468) Add Max AM Resource and AM Resource Usage to FairScheduler WebUI
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Summary: Add Max AM Resource and AM Resource Usage to FairScheduler WebUI (was: Add Max AM Resource to FairScheduler WebUI) > Add Max AM Resource and AM Resource Usage to FairScheduler WebUI > > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.9.0, 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > > Max AM Resource indicates maximum resources AM can use in a queue. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3663) Federation State and Policy Store (DBMS implementation)
[
https://issues.apache.org/jira/browse/YARN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964832#comment-15964832
]
Carlo Curino commented on YARN-3663:
[~giovanni.fumarola], the current patch doesn't seem to contain the definition
of store procedures except for the HSQLDB tests.
I think we need to include at least the SQLServer version you are currently
running in production and open a JIRA under the Federation V2 umbrella
YARN-5597 for a MySQL or DB-agnostic translation of the store
procedures---which will be less performant but convenient.
I know you were hesitating due to the SQLServer JDBC dependency. I think this
is not an issue anymore since MS open-sourced the JDBC driver for SQLServer:
*https://blogs.msdn.microsoft.com/jdbcteam/2016/11/17/open-source-jdbc-maven/*
Please check whether things compile and run well using that dependency (pom
entry below).
{code}
com.microsoft.sqlserver
mssql-jdbc
6.1.0.jre7
{code}
> Federation State and Policy Store (DBMS implementation)
> ---
>
> Key: YARN-3663
> URL: https://issues.apache.org/jira/browse/YARN-3663
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Giovanni Matteo Fumarola
>Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-3663-YARN-2915.v1.patch
>
>
> This JIRA tracks a SQL-based implementation of the Federation State and
> Policy Store, which implements YARN-3662 APIs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6468) Add Max AM Resource to FairScheduler WebUI
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu reassigned YARN-6468: -- Assignee: Yufei Gu > Add Max AM Resource to FairScheduler WebUI > -- > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.9.0, 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6468) Add Max AM Resource to FairScheduler WebUI
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Affects Version/s: 2.9.0 3.0.0-alpha2 > Add Max AM Resource to FairScheduler WebUI > -- > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.9.0, 3.0.0-alpha2 >Reporter: Yufei Gu > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6468) Add
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Component/s: fairscheduler > Add > > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Reporter: Yufei Gu > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6468) Add Max AM Resource to FairScheduler WebUI
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Summary: Add Max AM Resource to FairScheduler WebUI (was: Add ) > Add Max AM Resource to FairScheduler WebUI > -- > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Reporter: Yufei Gu > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6468) Add
[ https://issues.apache.org/jira/browse/YARN-6468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6468: --- Issue Type: Improvement (was: Bug) > Add > > > Key: YARN-6468 > URL: https://issues.apache.org/jira/browse/YARN-6468 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Reporter: Yufei Gu > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6468) Add
Yufei Gu created YARN-6468: -- Summary: Add Key: YARN-6468 URL: https://issues.apache.org/jira/browse/YARN-6468 Project: Hadoop YARN Issue Type: Bug Reporter: Yufei Gu -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6456) Isolation of Docker containers In LinuxContainerExecutor
[
https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964813#comment-15964813
]
Miklos Szegedi commented on YARN-6456:
--
Thank you for the reply [~jlowe]. What I am thinking about is mounting only
{{nm-local-dir/usercache/user/filecache}} and not the whole user folder. This
would give the user some protection against malicious/faulty applications.
Ideally
{{nm-local-dir/usercache/user/appcache/application_1491598755372_0011/filecache}}
is mounted only together with the current container dir, so that the container
directories are not added but as you also said this might not be possible due
to shuffle, other applications, etc. Maybe the container directories could be
outside the application directory. That might be an overkill.
> Isolation of Docker containers In LinuxContainerExecutor
>
>
> Key: YARN-6456
> URL: https://issues.apache.org/jira/browse/YARN-6456
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Miklos Szegedi
>
> One reason to use Docker containers is to be able to isolate different
> workloads, even, if they run as the same user.
> I have noticed some issues in the current design:
> 1. DockerLinuxContainerRuntime mounts containerLocalDirs
> {{nm-local-dir/usercache/user/appcache/application_1491598755372_0011/}} and
> userLocalDirs {{nm-local-dir/usercache/user/}}, so that a container can see
> and modify the files of another container. I think the application file cache
> directory should be enough for the container to run in most of the cases.
> 2. The whole cgroups directory is mounted. Would the container directory be
> enough?
> 3. There is no way to enforce exclusive use of Docker for all containers.
> There should be an option that it is not the user but the admin that requires
> to use Docker.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6466) Provide shaded framework jar for containers
[ https://issues.apache.org/jira/browse/YARN-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964732#comment-15964732 ] Sean Busbey commented on YARN-6466: --- Yeah, my thinking is that we probably don't need much of a footprint within the container in the first place and just relocating the third party libs might be simpler. > Provide shaded framework jar for containers > --- > > Key: YARN-6466 > URL: https://issues.apache.org/jira/browse/YARN-6466 > Project: Hadoop YARN > Issue Type: New Feature > Components: build, yarn >Reporter: Sean Busbey > > We should build on the existing shading work to provide a jar with all of the > bits needed within a YARN application's container to talk to the resource > manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964707#comment-15964707 ] Wangda Tan commented on YARN-6460: -- [~seneque], just added you to contributor, you can assign this JIRA to you. Could you add more details to the description of the JIRA? > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460.001.patch, > YARN-6460-branch-2.000.patch, YARN-6460-branch-2.001.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned YARN-6460: Assignee: Julien Vaudour > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Assignee: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460.001.patch, > YARN-6460-branch-2.000.patch, YARN-6460-branch-2.001.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3663) Federation State and Policy Store (DBMS implementation)
[ https://issues.apache.org/jira/browse/YARN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-3663: --- Attachment: (was: YARN-3663-YARN-2915.v1.patch) > Federation State and Policy Store (DBMS implementation) > --- > > Key: YARN-3663 > URL: https://issues.apache.org/jira/browse/YARN-3663 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-3663-YARN-2915.v1.patch > > > This JIRA tracks a SQL-based implementation of the Federation State and > Policy Store, which implements YARN-3662 APIs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3663) Federation State and Policy Store (DBMS implementation)
[ https://issues.apache.org/jira/browse/YARN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated YARN-3663: --- Attachment: YARN-3663-YARN-2915.v1.patch > Federation State and Policy Store (DBMS implementation) > --- > > Key: YARN-3663 > URL: https://issues.apache.org/jira/browse/YARN-3663 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-3663-YARN-2915.v1.patch > > > This JIRA tracks a SQL-based implementation of the Federation State and > Policy Store, which implements YARN-3662 APIs. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake in FileSystemRMStateStore
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964704#comment-15964704 ] Naganarasimha G R commented on YARN-6463: - Congrats on your first contribution [~Cyl], I have committed the patch to trunk and branch-2 and as well added you to the contributor list. > correct spelling mistake in FileSystemRMStateStore > -- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Yeliang Cang >Priority: Trivial > Labels: newbie > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6463) correct spelling mistake in FileSystemRMStateStore
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-6463: --- Assignee: Yeliang Cang (was: Naganarasimha G R) > correct spelling mistake in FileSystemRMStateStore > -- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Yeliang Cang >Priority: Trivial > Labels: newbie > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6467) CSQueueMetrics needs to update the current metrics for default partition only
Naganarasimha G R created YARN-6467: --- Summary: CSQueueMetrics needs to update the current metrics for default partition only Key: YARN-6467 URL: https://issues.apache.org/jira/browse/YARN-6467 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler Affects Versions: 3.0.0-alpha2, 2.7.3, 2.8.0 Reporter: Naganarasimha G R As a followup to YARN-6195, we need to update existing metrics to only default Partition. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake in FileSystemRMStateStore
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964687#comment-15964687 ] Hudson commented on YARN-6463: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11570 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11570/]) YARN-6463. Correct spelling mistake in FileSystemRMStateStore. (naganarasimha_gr: rev 7d873c465b815816e9955cf40f4e141dd2e7d82b) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java > correct spelling mistake in FileSystemRMStateStore > -- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Naganarasimha G R >Priority: Trivial > Labels: newbie > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6463) correct spelling mistake in FileSystemRMStateStore
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-6463: Summary: correct spelling mistake in FileSystemRMStateStore (was: correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin") > correct spelling mistake in FileSystemRMStateStore > -- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Naganarasimha G R >Priority: Trivial > Labels: newbie > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-6463: --- Assignee: Naganarasimha G R (was: Yeliang Cang) > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Naganarasimha G R >Priority: Trivial > Labels: newbie > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6466) Provide shaded framework jar for containers
[ https://issues.apache.org/jira/browse/YARN-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964646#comment-15964646 ] Sangjin Lee commented on YARN-6466: --- Hi [~busbey], thanks for raising this. How would this compare with the ongoing work at HADOOP-13070? Is your thinking that shading should be the/a mechanism for isolation for containers instead of the classloader isolation? Note that the classloader isolation already exists and folks have been using it to accomplish isolation. I think we may need to sort out what our story on this is for 3.0 going forward. > Provide shaded framework jar for containers > --- > > Key: YARN-6466 > URL: https://issues.apache.org/jira/browse/YARN-6466 > Project: Hadoop YARN > Issue Type: New Feature > Components: build, yarn >Reporter: Sean Busbey > > We should build on the existing shading work to provide a jar with all of the > bits needed within a YARN application's container to talk to the resource > manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-6463: --- Assignee: Yeliang Cang > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Assignee: Yeliang Cang >Priority: Trivial > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964623#comment-15964623 ] Naganarasimha G R commented on YARN-6463: - [~Cyl], agree simple fix and test failure not related hence committing this shortly. > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Priority: Trivial > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX
[ https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964500#comment-15964500 ] Benson Qiu edited comment on YARN-6195 at 4/11/17 3:17 PM: --- Thanks so much [~jlowe] for all of your help throughout this process, and thanks [~Naganarasimha], [~leftnoteasy], and [~sunilg] for taking the time to review my patch as well! I really appreciate it! was (Author: benson.qiu): Thanks so much [~jlowe] for all of your help throughout this process, and thanks Naga, Wangda, and Sunil for taking the time to review my patch as well! I really appreciate it! > Export UsedCapacity and AbsoluteUsedCapacity to JMX > --- > > Key: YARN-6195 > URL: https://issues.apache.org/jira/browse/YARN-6195 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler, metrics, yarn >Affects Versions: 3.0.0-alpha3 >Reporter: Benson Qiu >Assignee: Benson Qiu > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-6195.001.patch, YARN-6195.002.patch, > YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch > > > `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX
[ https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964500#comment-15964500 ] Benson Qiu commented on YARN-6195: -- Thanks so much [~jlowe] for all of your help throughout this process, and thanks Naga, Wangda, and Sunil for taking the time to review my patch as well! I really appreciate it! > Export UsedCapacity and AbsoluteUsedCapacity to JMX > --- > > Key: YARN-6195 > URL: https://issues.apache.org/jira/browse/YARN-6195 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler, metrics, yarn >Affects Versions: 3.0.0-alpha3 >Reporter: Benson Qiu >Assignee: Benson Qiu > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-6195.001.patch, YARN-6195.002.patch, > YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch > > > `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6466) Provide shaded framework jar for containers
[ https://issues.apache.org/jira/browse/YARN-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964490#comment-15964490 ] Sean Busbey commented on YARN-6466: --- reading through the DistributedShell example application, it looks like this should also plan to work with the timeline service apis. > Provide shaded framework jar for containers > --- > > Key: YARN-6466 > URL: https://issues.apache.org/jira/browse/YARN-6466 > Project: Hadoop YARN > Issue Type: New Feature > Components: build, yarn >Reporter: Sean Busbey > > We should build on the existing shading work to provide a jar with all of the > bits needed within a YARN application's container to talk to the resource > manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6466) Provide shaded framework jar for containers
Sean Busbey created YARN-6466: - Summary: Provide shaded framework jar for containers Key: YARN-6466 URL: https://issues.apache.org/jira/browse/YARN-6466 Project: Hadoop YARN Issue Type: New Feature Components: build, yarn Reporter: Sean Busbey We should build on the existing shading work to provide a jar with all of the bits needed within a YARN application's container to talk to the resource manager and node manager. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6461) TestRMAdminCLI has very low test timeouts
[ https://issues.apache.org/jira/browse/YARN-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964459#comment-15964459 ] Hudson commented on YARN-6461: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11568 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11568/]) YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by (jlowe: rev 90d97372ed11aaa95d338db8583f578da159834a) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java > TestRMAdminCLI has very low test timeouts > - > > Key: YARN-6461 > URL: https://issues.apache.org/jira/browse/YARN-6461 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Eric Badger > Fix For: 2.9.0, 2.8.1, 3.0.0-alpha3 > > Attachments: YARN-6461.001.patch > > > TestRMAdminCLI has only 500 millisecond timeouts on many of the unit tests. > If the test machine or VM is loaded/slow then the tests can report a false > positive. > I'm not sure these tests need explicit timeouts. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6461) TestRMAdminCLI has very low test timeouts
[ https://issues.apache.org/jira/browse/YARN-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964424#comment-15964424 ] Jason Lowe commented on YARN-6461: -- +1 lgtm. Committing this. > TestRMAdminCLI has very low test timeouts > - > > Key: YARN-6461 > URL: https://issues.apache.org/jira/browse/YARN-6461 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Eric Badger > Attachments: YARN-6461.001.patch > > > TestRMAdminCLI has only 500 millisecond timeouts on many of the unit tests. > If the test machine or VM is loaded/slow then the tests can report a false > positive. > I'm not sure these tests need explicit timeouts. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX
[ https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964415#comment-15964415 ] Hudson commented on YARN-6195: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11567 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11567/]) YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. (jlowe: rev 0e065f2edead41f6f894c9d70def389af1c0f239) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ReservationQueue.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueMetrics.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMetricsAnnotations.java * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableGaugeFloat.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMetricsRegistry.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsRegistry.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableMetricsFactory.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java > Export UsedCapacity and AbsoluteUsedCapacity to JMX > --- > > Key: YARN-6195 > URL: https://issues.apache.org/jira/browse/YARN-6195 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler, metrics, yarn >Affects Versions: 3.0.0-alpha3 >Reporter: Benson Qiu >Assignee: Benson Qiu > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-6195.001.patch, YARN-6195.002.patch, > YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch > > > `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6195) Export UsedCapacity and AbsoluteUsedCapacity to JMX
[ https://issues.apache.org/jira/browse/YARN-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964375#comment-15964375 ] Jason Lowe commented on YARN-6195: -- I do not think a separate HADOOP JIRA is necessary here. Committing this. > Export UsedCapacity and AbsoluteUsedCapacity to JMX > --- > > Key: YARN-6195 > URL: https://issues.apache.org/jira/browse/YARN-6195 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler, metrics, yarn >Affects Versions: 3.0.0-alpha3 >Reporter: Benson Qiu >Assignee: Benson Qiu > Attachments: YARN-6195.001.patch, YARN-6195.002.patch, > YARN-6195.003.patch, YARN-6195.004.patch, YARN-6195.005.patch > > > `usedCapacity` and `absoluteUsedCapacity` are currently not available as JMX. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6460) Accelerated time in SLS
[
https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964346#comment-15964346
]
Hadoop QA commented on YARN-6460:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 18
new + 136 unchanged - 0 fixed = 154 total (was 136) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 2m
48s{color} | {color:green} The patch generated 0 new + 74 unchanged - 1 fixed =
74 total (was 75) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
11s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} hadoop-tools_hadoop-sls generated 2 new + 20 unchanged
- 0 fixed = 22 total (was 20) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
1s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | YARN-6460 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862844/YARN-6460.001.patch |
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs compile
javac javadoc mvninstall findbugs checkstyle |
| uname | Linux f902d2906664 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2fd568f |
| Default Java | 1.8.0_121 |
| shellcheck | v0.4.6 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15587/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-sls.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-YARN-Build/15587/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15587/testReport/ |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
[jira] [Updated] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Vaudour updated YARN-6460: - Attachment: YARN-6460.001.patch > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460.001.patch, > YARN-6460-branch-2.000.patch, YARN-6460-branch-2.001.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Vaudour updated YARN-6460: - Attachment: (was: YARN-6460.001.patch) > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460.001.patch, > YARN-6460-branch-2.000.patch, YARN-6460-branch-2.001.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6460) Accelerated time in SLS
[
https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964301#comment-15964301
]
Hadoop QA commented on YARN-6460:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} branch-2 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} branch-2 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} branch-2 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} branch-2 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 17
new + 135 unchanged - 0 fixed = 152 total (was 135) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
7s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
8s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
11s{color} | {color:red} hadoop-tools_hadoop-sls-jdk1.8.0_121 with JDK
v1.8.0_121 generated 2 new + 23 unchanged - 0 fixed = 25 total (was 23) {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
55s{color} | {color:green} hadoop-sls in the patch passed with JDK v1.7.0_121.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:b59b8b7 |
| JIRA Issue | YARN-6460 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862842/YARN-6460-branch-2.001.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs compile
javac javadoc mvninstall findbugs
[jira] [Updated] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Vaudour updated YARN-6460: - Attachment: YARN-6460-branch-2.001.patch YARN-6460.001.patch I forgot the shell script in my previous patch > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460.001.patch, > YARN-6460-branch-2.000.patch, YARN-6460-branch-2.001.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6462) Add yarn command to list all queues
[
https://issues.apache.org/jira/browse/YARN-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964197#comment-15964197
]
Hadoop QA commented on YARN-6462:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 3 new +
143 unchanged - 0 fixed = 146 total (was 143) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 34m 42s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.yarn.client.api.impl.TestOpportunisticContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | YARN-6462 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862809/YARN-6462_2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 536a2f62ae37 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / aabf08d |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15585/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15585/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15585/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15585/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add yarn command to list all queues
>
[jira] [Updated] (YARN-6465) FairScheduler preempt should consider task running time
[ https://issues.apache.org/jira/browse/YARN-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated YARN-6465: Summary: FairScheduler preempt should consider task running time (was: FairScheduler preempt should consider task progress) > FairScheduler preempt should consider task running time > --- > > Key: YARN-6465 > URL: https://issues.apache.org/jira/browse/YARN-6465 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler preemption >Reporter: darion yaphet > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6462) Add yarn command to list all queues
[
https://issues.apache.org/jira/browse/YARN-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated YARN-6462:
--
Attachment: YARN-6462_2.patch
update a patch,and run yarn shell:
{code}yarn queue -list
2017-04-11 17:08:25,034 INFO client.RMProxal/10.139*.*8032
Queue list :
root.default
root.dev
root.fol
root.fol.user1
root.fol.user2
{code}
> Add yarn command to list all queues
> ---
>
> Key: YARN-6462
> URL: https://issues.apache.org/jira/browse/YARN-6462
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Reporter: Shen Yinjie
> Fix For: 3.0.0-alpha3
>
> Attachments: YARN-6462_1.patch, YARN-6462_2.patch
>
>
> we need a yarn command to list all queues ,and there is this kind of command
> for applications and nodemangers already...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6465) FairScheduler preempt should consider task progress
darion yaphet created YARN-6465: --- Summary: FairScheduler preempt should consider task progress Key: YARN-6465 URL: https://issues.apache.org/jira/browse/YARN-6465 Project: Hadoop YARN Issue Type: Improvement Components: scheduler preemption Reporter: darion yaphet -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-6462) Add yarn command to list all queues
[
https://issues.apache.org/jira/browse/YARN-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shen Yinjie updated YARN-6462:
--
Comment: was deleted
(was: I upload a patch ,so that we can run yarn-client shell:
{code}yarn queue -list
017-04-10 17:56:46,444 INFO client.RMProxy: Connecting to ResourceManager at
shendev.novalocal/10.139.*.*:8032
Queue list:
[root.default, root.devtest, root.test, root.test.dev_1, root.test.dev_2,
root.test.dev_4]{code}
)
> Add yarn command to list all queues
> ---
>
> Key: YARN-6462
> URL: https://issues.apache.org/jira/browse/YARN-6462
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Reporter: Shen Yinjie
> Fix For: 3.0.0-alpha3
>
> Attachments: YARN-6462_1.patch
>
>
> we need a yarn command to list all queues ,and there is this kind of command
> for applications and nodemangers already...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6460) Accelerated time in SLS
[ https://issues.apache.org/jira/browse/YARN-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964011#comment-15964011 ] Julien Vaudour commented on YARN-6460: -- you guess well > Accelerated time in SLS > --- > > Key: YARN-6460 > URL: https://issues.apache.org/jira/browse/YARN-6460 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Julien Vaudour >Priority: Minor > Attachments: YARN-6460.000.patch, YARN-6460-branch-2.000.patch > > > Be able to accelerate time in sls -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5892) Capacity Scheduler: Support user-specific minimum user limit percent
[
https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963994#comment-15963994
]
Sunil G commented on YARN-5892:
---
Thanks [~eepayne].
Few comments from end:
# {{getNumActive/AllWeightedUsers}} is iterating on all users now. It could be
optimized by considering only weighted users.
# Given we have only one user in cluster, and we gave UL < 1 for that user.
This will be increase the defined UserLimit for that user, because
{{getNumActive/AllWeightedUsers}} will give a count < 1 in this case. Correct?
# I think user_weight=0 is tough to go in with existing nomenclatures or
usages. If AM container is huge, still we may give resource for that user.
> Capacity Scheduler: Support user-specific minimum user limit percent
>
>
> Key: YARN-5892
> URL: https://issues.apache.org/jira/browse/YARN-5892
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: Active users highlighted.jpg, YARN-5892.001.patch,
> YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch,
> YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
> YARN-5892.008.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}}
> property is per queue. A cluster admin should be able to set the minimum user
> limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled
> (YARN-4945 / YARN-2113), some users can be deemed as more important than
> other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user
> {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed
> 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like
> this:
> {code}
>
>
> yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent
> 25
>
>
>
> yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent
> 75
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6464) Rename the API Application to Service
[ https://issues.apache.org/jira/browse/YARN-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He reassigned YARN-6464: - Assignee: Jian He > Rename the API Application to Service > - > > Key: YARN-6464 > URL: https://issues.apache.org/jira/browse/YARN-6464 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > > Given that we consider service as first-class citizen in YARN, we may call > the abstraction as Service, instead of Application ? > The native-service UI also now has a differentiation of service vs > application. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6405) Improve configuring services through REST API
[ https://issues.apache.org/jira/browse/YARN-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963971#comment-15963971 ] Jian He commented on YARN-6405: --- addressed the comments in latest patch, also opened YARN-6464 for renaming application to service > Improve configuring services through REST API > - > > Key: YARN-6405 > URL: https://issues.apache.org/jira/browse/YARN-6405 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-6405.yarn-native-services.01.patch, > YARN-6405.yarn-native-services.02.patch, > YARN-6405.yarn-native-services.03.patch, > YARN-6405.yarn-native-services.04.patch, > YARN-6405.yarn-native-services.05.patch > > > YARN-4793 defined various ways that user can config services through the > configuration section of the REST API. But, some semantics are not yet > supported in the back-end server (AM). YARN-6255 has done some work, this > jira is to complete this task - support configuring services through REST API -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6464) Rename the API Application to Service
[ https://issues.apache.org/jira/browse/YARN-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963969#comment-15963969 ] Jian He commented on YARN-6464: --- [~billie.rinaldi] and I agreed on this. [~gsaha], [~vinodkv] your opinion ? > Rename the API Application to Service > - > > Key: YARN-6464 > URL: https://issues.apache.org/jira/browse/YARN-6464 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > > Given that we consider service as first-class citizen in YARN, we may call > the abstraction as Service, instead of Application ? > The native-service UI also now has a differentiation of service vs > application. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6464) Rename the API Application to Service
Jian He created YARN-6464: - Summary: Rename the API Application to Service Key: YARN-6464 URL: https://issues.apache.org/jira/browse/YARN-6464 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jian He Given that we consider service as first-class citizen in YARN, we may call the abstraction as Service, instead of Application ? The native-service UI also now has a differentiation of service vs application. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeliang Cang updated YARN-6463: --- Attachment: YARN-6463.002.patch > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Priority: Trivial > Attachments: YARN-6463.001.patch, YARN-6463.002.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeliang Cang updated YARN-6463: --- Component/s: (was: resourcemanager) > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Priority: Trivial > Attachments: YARN-6463.001.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[ https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963962#comment-15963962 ] Yeliang Cang commented on YARN-6463: This is a minor spelling change, and no need to add UT. Also the failed unit test seems unrelated. Upload the patch for another try! > correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin" > --- > > Key: YARN-6463 > URL: https://issues.apache.org/jira/browse/YARN-6463 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0-alpha2 >Reporter: Yeliang Cang >Priority: Trivial > Attachments: YARN-6463.001.patch > > > I have found some spelling mistakes in FileSystemRMStateStore.java > "makeUnradableByAdmin" should be "makeUnreadableByAdmin" instead. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6463) correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
[
https://issues.apache.org/jira/browse/YARN-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963885#comment-15963885
]
Hadoop QA commented on YARN-6463:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 11s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | YARN-6463 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12862781/YARN-6463.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 6e77cb35d135 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7999318a |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15581/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15581/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15581/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> correct spelling mistake, "makeUnreadableByAdmin" vs "makeUnradableByAdmin"
>
