[jira] [Commented] (YARN-9925) CapacitySchedulerQueueManager allows unsupported Queue hierarchy
[
https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014047#comment-17014047
]
Prabhu Joseph commented on YARN-9925:
-
YARN-9879 provides support for multiple leaf queues with the same name. Hence
will close this as a duplicate of YARN-9879 if there is no objections.
> CapacitySchedulerQueueManager allows unsupported Queue hierarchy
>
>
> Key: YARN-9925
> URL: https://issues.apache.org/jira/browse/YARN-9925
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9925-001.patch, YARN-9925-002.patch,
> YARN-9925-003.patch, YARN-9925-004.patch, YARN-9925-005.patch
>
>
> CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When
> creating a queue with same name as an existing parent queue name - it has to
> fail with below.
> {code:java}
> Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after
> refresh, which is not allowed.Caused by: java.io.IOException: A is moved
> from:root.A to:root.B.A after refresh, which is not allowed. at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473)
> ... 70 more
> {code}
> In Some cases, the error is not thrown while creating the queue but thrown at
> submission of job "Failed to submit application_1571677375269_0002 to YARN :
> Application application_1571677375269_0002 submitted by user : systest to
> non-leaf queue : B"
> Below scenarios are allowed but it should not
> {code:java}
> It allows root.A.A1.B when root.B.B1 already exists.
>
> 1. Add root.A
> 2. Add root.A.A1
> 3. Add root.B
> 4. Add root.B.B1
> 5. Allows Add of root.A.A1.B
> It allows two root queues:
>
> 1. Add root.A
> 2. Add root.B
> 3. Add root.A.A1
> 4. Allows Add of root.A.A1.root
>
> {code}
> Below scenario is handled properly:
> {code:java}
> It does not allow root.B.A when root.A.A1 already exists.
>
> 1. Add root.A
> 2. Add root.B
> 3. Add root.A.A1
> 4. Does not Allow Add of root.B.A
> {code}
> This error handling has to be consistent in all scenarios.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8148) Update decimal values for queue capacities shown on queue status CLI
[
https://issues.apache.org/jira/browse/YARN-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014014#comment-17014014
]
Prabhu Joseph commented on YARN-8148:
-
[~snemeth] Testcase failure is not related and tracked by YARN-6272.
> Update decimal values for queue capacities shown on queue status CLI
>
>
> Key: YARN-8148
> URL: https://issues.apache.org/jira/browse/YARN-8148
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-8148-002.patch, YARN-8148-002.patch,
> YARN-8148-branch-3.1.001.patch, YARN-8148-branch-3.2.001.patch,
> YARN-8148-branch-3.2.001.patch, YARN-8148.1.patch
>
>
> Capacities are shown with two decimal values in RM UI as part of YARN-6182.
> The queue status cli are still showing one decimal value.
> {code}
> [root@bigdata3 yarn]# yarn queue -status default
> Queue Information :
> Queue Name : default
> State : RUNNING
> Capacity : 69.9%
> Current Capacity : .0%
> Maximum Capacity : 70.0%
> Default Node Label expression :
> Accessible Node Labels : *
> Preemption : enabled
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10070) NPE if no rule is defined and application-tag-based-placement is enabled
[
https://issues.apache.org/jira/browse/YARN-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013847#comment-17013847
]
Kinga Marton edited comment on YARN-10070 at 1/12/20 6:44 PM:
--
Thank you [~prabhujoseph] for checking it.
I think I found the bug in the code part you have shared:
* the logic it should be followed is: we should check if the submitting user
(hive) has access to submit application in the queue of the user from the
application tag (tezq). If no, the placement will be done using the submitting
user (hive) instead of the one from the application tag (ambart-qa), so it
should be placed to the default queue.
* The bug is in the following code:
{code:java}
String queue = placementManager
.placeApplication(context, usernameUsedForPlacement).getQueue();
{code}
- here {{usernameUsedForPlacement}} should be replaced with
{{userNameFromApptag}} in order to get the queue the application would be
placed.
{code:java}
UserGroupInformation callerUGI = UserGroupInformation
.createRemoteUser(userNameFromAppTag);
{code}
- here {{userNameFromAppTag}} should be replaced with {{user}}, so we can
check than if the user(hive) has access to submit applications to the queue
where the application would be placed using the username from the apptag(tezq).
was (Author: kmarton):
Thank you [~prabhujoseph] for checking it.
I think I found the bug in the code part you have shared:
* the logic it should be followed is: we should check if the submitting use
(hive) has access to submit application in the queue of the user from the
application tag (tezq). If no, the placement will be done using the submitting
user (hive) instead of the one from the application tag (ambart-qa), so it
should be placed to the default queue.
* The bug is in the following code:
{code:java}
String queue = placementManager
.placeApplication(context, usernameUsedForPlacement).getQueue();
{code}
- here {{usernameUsedForPlacement}} should be replaces with
{{userNameFromApptag}} in order to get the queue the application would be
placed.
{code:java}
UserGroupInformation callerUGI = UserGroupInformation
.createRemoteUser(userNameFromAppTag);
{code}
- here {{userNameFromAppTag}} should be replaced with {{user}}, so we can check
than if the user(hive) has access to submit applications to the queue where the
application would be placed using the username from the apptag(tezq).
> NPE if no rule is defined and application-tag-based-placement is enabled
>
>
> Key: YARN-10070
> URL: https://issues.apache.org/jira/browse/YARN-10070
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
>
> If there is no rule defined for a user NPE is thrown by the following line.
> {code:java}
> String queue = placementManager
> .placeApplication(context, usernameUsedForPlacement).getQueue();{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10070) NPE if no rule is defined and application-tag-based-placement is enabled
[
https://issues.apache.org/jira/browse/YARN-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013847#comment-17013847
]
Kinga Marton commented on YARN-10070:
-
Thank you [~prabhujoseph] for checking it.
I think I found the bug in the code part you have shared:
* the logic it should be followed is: we should check if the submitting use
(hive) has access to submit application in the queue of the user from the
application tag (tezq). If no, the placement will be done using the submitting
user (hive) instead of the one from the application tag (ambart-qa), so it
should be placed to the default queue.
* The bug is in the following code:
{code:java}
String queue = placementManager
.placeApplication(context, usernameUsedForPlacement).getQueue();
{code}
- here {{usernameUsedForPlacement}} should be replaces with
{{userNameFromApptag}} in order to get the queue the application would be
placed.
{code:java}
UserGroupInformation callerUGI = UserGroupInformation
.createRemoteUser(userNameFromAppTag);
{code}
- here {{userNameFromAppTag}} should be replaced with {{user}}, so we can check
than if the user(hive) has access to submit applications to the queue where the
application would be placed using the username from the apptag(tezq).
> NPE if no rule is defined and application-tag-based-placement is enabled
>
>
> Key: YARN-10070
> URL: https://issues.apache.org/jira/browse/YARN-10070
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
>
> If there is no rule defined for a user NPE is thrown by the following line.
> {code:java}
> String queue = placementManager
> .placeApplication(context, usernameUsedForPlacement).getQueue();{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9912) Capacity scheduler: support u:user2:%secondary_group queue mapping
[
https://issues.apache.org/jira/browse/YARN-9912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013845#comment-17013845
]
Hadoop QA commented on YARN-9912:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 29 unchanged - 1 fixed = 30 total (was 30) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m
35s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-9912 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12990664/YARN-9912.005.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d202a6c9a34f 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d842dff |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25374/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25374/testReport/ |
| Max. process+thread count | 828 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
[jira] [Updated] (YARN-9912) Capacity scheduler: support u:user2:%secondary_group queue mapping
[ https://issues.apache.org/jira/browse/YARN-9912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-9912: --- Attachment: YARN-9912.005.patch > Capacity scheduler: support u:user2:%secondary_group queue mapping > -- > > Key: YARN-9912 > URL: https://issues.apache.org/jira/browse/YARN-9912 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler, capacityscheduler >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9912-004.patch, YARN-9912.001.patch, > YARN-9912.002.patch, YARN-9912.003.patch, YARN-9912.005.patch > > > Similar to u:user2:%primary_group mapping, add support for > u:user2:%secondary_group queue mapping as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9912) Capacity scheduler: support u:user2:%secondary_group queue mapping
[ https://issues.apache.org/jira/browse/YARN-9912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013804#comment-17013804 ] Manikandan R commented on YARN-9912: Rebasing.. > Capacity scheduler: support u:user2:%secondary_group queue mapping > -- > > Key: YARN-9912 > URL: https://issues.apache.org/jira/browse/YARN-9912 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler, capacityscheduler >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9912-004.patch, YARN-9912.001.patch, > YARN-9912.002.patch, YARN-9912.003.patch, YARN-9912.005.patch > > > Similar to u:user2:%primary_group mapping, add support for > u:user2:%secondary_group queue mapping as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9866) u:user2:%primary_group is not working as expected
[ https://issues.apache.org/jira/browse/YARN-9866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013759#comment-17013759 ] Hudson commented on YARN-9866: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17855 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17855/]) YARN-9866. u:user2:%primary_group is not working as expected. (snemeth: rev d842dfffa53c8b565f3d65af44ccd7e1cc706733) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerQueueMappingFactory.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/UserGroupMappingPlacementRule.java > u:user2:%primary_group is not working as expected > - > > Key: YARN-9866 > URL: https://issues.apache.org/jira/browse/YARN-9866 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9866.001.patch, YARN-9866.002.patch, > YARN-9866.003.patch, YARN-9866.004.patch, YARN-9866.005.patch > > > Please refer #1 in > https://issues.apache.org/jira/browse/YARN-9841?focusedCommentId=16937024=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937024 > for more details -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8148) Update decimal values for queue capacities shown on queue status CLI
[
https://issues.apache.org/jira/browse/YARN-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013756#comment-17013756
]
Szilard Nemeth commented on YARN-8148:
--
Hi [~prabhujoseph],
Could you please check the UT failure?
thanks.
> Update decimal values for queue capacities shown on queue status CLI
>
>
> Key: YARN-8148
> URL: https://issues.apache.org/jira/browse/YARN-8148
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-8148-002.patch, YARN-8148-002.patch,
> YARN-8148-branch-3.1.001.patch, YARN-8148-branch-3.2.001.patch,
> YARN-8148-branch-3.2.001.patch, YARN-8148.1.patch
>
>
> Capacities are shown with two decimal values in RM UI as part of YARN-6182.
> The queue status cli are still showing one decimal value.
> {code}
> [root@bigdata3 yarn]# yarn queue -status default
> Queue Information :
> Queue Name : default
> State : RUNNING
> Capacity : 69.9%
> Current Capacity : .0%
> Maximum Capacity : 70.0%
> Default Node Label expression :
> Accessible Node Labels : *
> Preemption : enabled
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9912) Capacity scheduler: support u:user2:%secondary_group queue mapping
[ https://issues.apache.org/jira/browse/YARN-9912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013753#comment-17013753 ] Szilard Nemeth commented on YARN-9912: -- Hi [~maniraj...@gmail.com], I just committed YARN-9866. Could you please rebase this patch to trunk? thanks. > Capacity scheduler: support u:user2:%secondary_group queue mapping > -- > > Key: YARN-9912 > URL: https://issues.apache.org/jira/browse/YARN-9912 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler, capacityscheduler >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9912-004.patch, YARN-9912.001.patch, > YARN-9912.002.patch, YARN-9912.003.patch > > > Similar to u:user2:%primary_group mapping, add support for > u:user2:%secondary_group queue mapping as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10067) Add dry-run feature to FS-CS converter tool
[ https://issues.apache.org/jira/browse/YARN-10067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013751#comment-17013751 ] Hudson commented on YARN-10067: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17854 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17854/]) YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by (snemeth: rev 24e6a9e43a210cdecaa8e87926eef09c869988f9) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigArgumentHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigRuleHandler.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverterBuilder.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/DryRunResultHolder.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverter.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverterMain.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConversionOptions.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigRuleHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSQueueConverter.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigArgumentHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverter.java > Add dry-run feature to FS-CS converter tool > --- > > Key: YARN-10067 > URL: https://issues.apache.org/jira/browse/YARN-10067 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10067-001.patch, YARN-10067-002.patch, > YARN-10067-003.patch, YARN-10067-004.patch, YARN-10067-005.patch, > YARN-10067-006.patch, YARN-10067-007.patch, YARN-10067-007.patch > > > Add a "d" / "-dry-run" switch to the tool. The purpose of this would be to > inform the user whether a conversion is possible and if it is, are there any > warnings. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9866) u:user2:%primary_group is not working as expected
[ https://issues.apache.org/jira/browse/YARN-9866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013752#comment-17013752 ] Szilard Nemeth commented on YARN-9866: -- Hi [~maniraj...@gmail.com]! Latest patch looks good to me, +1. Committed to trunk. Thanks [~pbacsko] for the review! > u:user2:%primary_group is not working as expected > - > > Key: YARN-9866 > URL: https://issues.apache.org/jira/browse/YARN-9866 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9866.001.patch, YARN-9866.002.patch, > YARN-9866.003.patch, YARN-9866.004.patch, YARN-9866.005.patch > > > Please refer #1 in > https://issues.apache.org/jira/browse/YARN-9841?focusedCommentId=16937024=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937024 > for more details -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10026) Pull out common code pieces from ATS v1.5 and v2
[ https://issues.apache.org/jira/browse/YARN-10026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10026: -- Fix Version/s: 3.2.2 > Pull out common code pieces from ATS v1.5 and v2 > > > Key: YARN-10026 > URL: https://issues.apache.org/jira/browse/YARN-10026 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2, yarn >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0, 3.2.2 > > Attachments: YARN-10026.001.patch, YARN-10026.002.patch, > YARN-10026.003.patch, YARN-10026.branch-3.2.001.patch > > > ATSv1.5 and ATSv2 has lots of common code that can be pulled to an abstract > service / package. The logic is the same, and the code is _almost_ the same. > As far as I see, the only ATS specific thing in that AppInfo is constructed > from an ApplicationReport, which information is extracted from the > TimelineReader client, > Later the appInfo object's user and appState fields are used, but I see no > other dependency on the timeline part. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10026) Pull out common code pieces from ATS v1.5 and v2
[ https://issues.apache.org/jira/browse/YARN-10026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013748#comment-17013748 ] Szilard Nemeth commented on YARN-10026: --- Thanks [~adam.antal]. Committed branch-3.2 patch as well. Resolving this jira. > Pull out common code pieces from ATS v1.5 and v2 > > > Key: YARN-10026 > URL: https://issues.apache.org/jira/browse/YARN-10026 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2, yarn >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10026.001.patch, YARN-10026.002.patch, > YARN-10026.003.patch, YARN-10026.branch-3.2.001.patch > > > ATSv1.5 and ATSv2 has lots of common code that can be pulled to an abstract > service / package. The logic is the same, and the code is _almost_ the same. > As far as I see, the only ATS specific thing in that AppInfo is constructed > from an ApplicationReport, which information is extracted from the > TimelineReader client, > Later the appInfo object's user and appState fields are used, but I see no > other dependency on the timeline part. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10067) Add dry-run feature to FS-CS converter tool
[ https://issues.apache.org/jira/browse/YARN-10067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013745#comment-17013745 ] Szilard Nemeth commented on YARN-10067: --- Thanks [~pbacsko]! Latest patch looks good to me, committed to trunk. Resolving the jira as I don't think we need any backports. > Add dry-run feature to FS-CS converter tool > --- > > Key: YARN-10067 > URL: https://issues.apache.org/jira/browse/YARN-10067 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10067-001.patch, YARN-10067-002.patch, > YARN-10067-003.patch, YARN-10067-004.patch, YARN-10067-005.patch, > YARN-10067-006.patch, YARN-10067-007.patch, YARN-10067-007.patch > > > Add a "d" / "-dry-run" switch to the tool. The purpose of this would be to > inform the user whether a conversion is possible and if it is, are there any > warnings. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
