[jira] [Commented] (YARN-9606) Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient
[
https://issues.apache.org/jira/browse/YARN-9606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081525#comment-17081525
]
Brahma Reddy Battula commented on YARN-9606:
[~BilwaST] thanks for uploading the patch. Latest patch lgtm. [~prabhujoseph]
do you any comments on this as reviewed before YARN-10120.?
> Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient
> --
>
> Key: YARN-9606
> URL: https://issues.apache.org/jira/browse/YARN-9606
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: YARN-9606-001.patch, YARN-9606-002.patch,
> YARN-9606.003.patch
>
>
> Yarn logs fails for running containers
>
>
> {quote}
>
>
>
> Unable to fetch log files list
> Exception in thread "main" java.io.IOException:
> com.sun.jersey.api.client.ClientHandlerException:
> javax.net.ssl.SSLHandshakeException: Error while authenticating with
> endpoint:
> [https://vm2:65321/ws/v1/node/containers/container_e05_1559802125016_0001_01_08/logs]
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.getContainerLogFiles(LogsCLI.java:543)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.getMatchedContainerLogFiles(LogsCLI.java:1338)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.getMatchedOptionForRunningApp(LogsCLI.java:1514)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchContainerLogs(LogsCLI.java:1052)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:367)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:152)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:399)
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9998) Code cleanup in LeveldbConfigurationStore
[ https://issues.apache.org/jira/browse/YARN-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080796#comment-17080796 ] Brahma Reddy Battula edited comment on YARN-9998 at 4/11/20, 6:09 PM: -- [~snemeth] could you please review the branch-3.2 patch and close the jira..? I am planning for 3.3.0 release shortly and this Jira shouldn't open as this merged to 3.3.0 also. was (Author: brahmareddy): Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker. > Code cleanup in LeveldbConfigurationStore > - > > Key: YARN-9998 > URL: https://issues.apache.org/jira/browse/YARN-9998 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Benjamin Teke >Priority: Minor > Fix For: 3.3.0 > > Attachments: YARN-9998.001.patch, YARN-9998.002.patch > > > Many things can be improved: > * Field compactionTimer could be a local variable > * Field versiondb should be camelcase > * initDatabase is a very long method: Initialize db / versionDb should be in > separate methods, split this method into smaller chunks > * Remove TODOs > * Remove duplicated code block in > LeveldbConfigurationStore.CompactionTimerTask > * Any other cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10002) Code cleanup and improvements in ConfigurationStoreBaseTest
[ https://issues.apache.org/jira/browse/YARN-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080809#comment-17080809 ] Brahma Reddy Battula edited comment on YARN-10002 at 4/11/20, 6:08 PM: --- [~snemeth] could you please review the branch-3.2 patch and close the jira..? I am planning for 3.3.0 release shortly and this Jira shouldn't open as this merged to 3.3.0 also. was (Author: brahmareddy): Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker. > Code cleanup and improvements in ConfigurationStoreBaseTest > --- > > Key: YARN-10002 > URL: https://issues.apache.org/jira/browse/YARN-10002 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Benjamin Teke >Priority: Minor > Fix For: 3.3.0 > > Attachments: YARN-10002.001.patch, YARN-10002.002.patch, > YARN-10002.003.patch, YARN-10002.004.patch, YARN-10002.005.patch, > YARN-10002.006.patch, YARN-10002.branch-3.2.001.patch > > > * Some protected fields could be package-private > * Could add a helper method that prepares a simple LogMutation with 1, 2 or 3 > updates (Key + value) as this pattern is used extensively in subclasses -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9354) Resources should be created with ResourceTypesTestHelper instead of TestUtils
[ https://issues.apache.org/jira/browse/YARN-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080800#comment-17080800 ] Brahma Reddy Battula edited comment on YARN-9354 at 4/11/20, 6:07 PM: -- [~snemeth] could you please review the branch-3.2 patch and close the jira..? I am planning for 3.3.0 release shortly and this Jira shouldn't open as this merged to 3.3.0 also. was (Author: brahmareddy): Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker. > Resources should be created with ResourceTypesTestHelper instead of TestUtils > - > > Key: YARN-9354 > URL: https://issues.apache.org/jira/browse/YARN-9354 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Andras Gyori >Priority: Trivial > Labels: newbie, newbie++ > Fix For: 3.3.0 > > Attachments: YARN-9354.001.patch, YARN-9354.002.patch, > YARN-9354.003.patch, YARN-9354.004.patch, YARN-9354.branch-3.2.001.patch, > YARN-9354.branch-3.2.002.patch, YARN-9354.branch-3.2.003.patch > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestUtils#createResource > has not identical, but very similar implementation to > org.apache.hadoop.yarn.resourcetypes.ResourceTypesTestHelper#newResource. > Since these 2 methods are doing the same essentially and > ResourceTypesTestHelper is newer and used more, TestUtils#createResource > should be replaced with ResourceTypesTestHelper#newResource with all > occurrence. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-10154: Attachment: YARN-10154.003.patch > CS Dynamic Queues cannot be configured with absolute resources > -- > > Key: YARN-10154 > URL: https://issues.apache.org/jira/browse/YARN-10154 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Sunil G >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10154.001.patch, YARN-10154.002.patch, > YARN-10154.003.patch > > > In CS, ManagedParent Queue and its template cannot take absolute resource > value like > [memory=8192,vcores=8] > Thsi Jira is to track and improve the configuration reading module of > DynamicQueue to support absolute resource values. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081378#comment-17081378 ] Manikandan R commented on YARN-10154: - Sorry for the delay. Attaching .003.patch covering suggested above test cases. > CS Dynamic Queues cannot be configured with absolute resources > -- > > Key: YARN-10154 > URL: https://issues.apache.org/jira/browse/YARN-10154 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Sunil G >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10154.001.patch, YARN-10154.002.patch > > > In CS, ManagedParent Queue and its template cannot take absolute resource > value like > [memory=8192,vcores=8] > Thsi Jira is to track and improve the configuration reading module of > DynamicQueue to support absolute resource values. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10232) InvalidStateTransitionException: Invalid event: LAUNCH_FAILED at RUNNING
[
https://issues.apache.org/jira/browse/YARN-10232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YCozy updated YARN-10232:
-
Description:
We were testing YARN under network partition and found the following ERROR in
RM's log.
{code:java}
2020-04-11 13:10:39,739 ERROR
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
App attempt: appattempt_6_0001_02 can't handle this event at current state
org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event:
LAUNCH_FAILED at RUNNING
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:916)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:121)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1097)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1078)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:222)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:138)
at java.lang.Thread.run(Thread.java:748)
{code}
After analyzing the logs, we have recovered the triggering process of this bug:
* We have a cluster with one RM and one NM.
* A client tries to start a YARN service.
* RM send a request to NM to start the containers:
NM's log:
{code:java}
2020-04-11 14:23:44,030 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
2020-04-11 14:23:44,229 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Start request for container_6_0001_02_01 by user appattempt_6_0001_02
{code}
* NM starts the containers successfully:
NM's log:
{code:java}
2020-04-11 14:23:44,347 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl:
Application application_6_0001 transitioned from INITING to RUNNING
2020-04-11 14:23:44,357 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl:
Container container_6_0001_02_01 transitioned from NEW to LOCALIZING
{code}
* However, due to network partition, NM failed to send back the RPC response.
* After a while, the application is running happily:
RM's log:
{code:java}
2020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_6_0001_02 State change from ALLOCATED to RUNNING on event =
REGISTERED
020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_6_0001 State change from ACCEPTED to RUNNING on event =
ATTEMPT_REGISTERED{code}
* Then, since RM didn't receive the RPC response for startContainers, it
retries. The network partition has already stopped, so NM will receive the new
startContainers RPC:
NM's log:
{code:java}
2020-04-11 14:23:54,392 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
{code}
* But since the attempt is actually running, this launch request does not
succeed:
NM's log:
{code:java}
2020-04-11 14:23:54,401 ERROR
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Unauthorized request to start container.
Attempt to relaunch the same container with id container_6_0001_02_01.
{code}
RM's log:
{code:java}
2020-04-11 14:23:54,428 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Error
launching appattempt_6_0001_02. Got exception:
org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start
container.
Attempt to relaunch the same container with id container_6_0001_02_01.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateExceptionImpl(SerializedExceptionPBImpl.java:171)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:182)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at
[jira] [Updated] (YARN-10232) InvalidStateTransitionException: Invalid event: LAUNCH_FAILED at RUNNING
[
https://issues.apache.org/jira/browse/YARN-10232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YCozy updated YARN-10232:
-
Description:
We were testing YARN under network partition and found the following ERROR in
RM's log.
{code:java}
2020-04-11 13:10:39,739 ERROR
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
App attempt: appattempt_6_0001_02 can't handle this event at current state
org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event:
LAUNCH_FAILED at RUNNING
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:916)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:121)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1097)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1078)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:222)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:138)
at java.lang.Thread.run(Thread.java:748)
{code}
After analyzing the logs, we have recovered the triggering process of this bug:
* We have a cluster with one RM and one NM.
* A client tries to start a YARN service.
* RM send a request to NM to start the containers:
NM's log:
{code:java}
2020-04-11 14:23:44,030 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
2020-04-11 14:23:44,229 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Start request for container_6_0001_02_01 by user appattempt_6_0001_02
{code}
* NM starts the containers successfully:
NM's log:
{code:java}
2020-04-11 14:23:44,347 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl:
Application application_6_0001 transitioned from INITING to RUNNING
2020-04-11 14:23:44,357 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl:
Container container_6_0001_02_01 transitioned from NEW to LOCALIZING
{code}
* However, due to network partition, NM failed to send back the RPC response.
* After a while, the application is running happily:
RM's log:
{code:java}
2020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_6_0001_02 State change from ALLOCATED to RUNNING on event =
REGISTERED
020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_6_0001 State change from ACCEPTED to RUNNING on event =
ATTEMPT_REGISTERED{code}
* Then, since RM didn't receive the RPC response for startContainers, it
retries:
NM's log:
{code:java}
2020-04-11 14:23:54,392 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
{code}
* But since the attempt is actually running, this launch request does not
succeed:
NM's log:
{code:java}
2020-04-11 14:23:54,401 ERROR
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Unauthorized request to start container.
Attempt to relaunch the same container with id container_6_0001_02_01.
{code}
RM's log:
{code:java}
2020-04-11 14:23:54,428 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Error
launching appattempt_6_0001_02. Got exception:
org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start
container.
Attempt to relaunch the same container with id container_6_0001_02_01.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateExceptionImpl(SerializedExceptionPBImpl.java:171)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:182)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:137)
at
[jira] [Created] (YARN-10232) InvalidStateTransitionException: Invalid event: LAUNCH_FAILED at RUNNING
YCozy created YARN-10232:
Summary: InvalidStateTransitionException: Invalid event:
LAUNCH_FAILED at RUNNING
Key: YARN-10232
URL: https://issues.apache.org/jira/browse/YARN-10232
Project: Hadoop YARN
Issue Type: Bug
Reporter: YCozy
We were testing YARN under network partition and found the following ERROR in
RM's log.
{code:java}
2020-04-11 13:10:39,739 ERROR
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
App attempt: appattempt_6_0001_02 can't handle this event at current state
org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event:
LAUNCH_FAILED at RUNNING
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:916)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:121)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1097)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher.handle(ResourceManager.java:1078)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:222)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:138)
at java.lang.Thread.run(Thread.java:748)
{code}
After analyzing the logs, we have recovered the triggering process of this bug:
* We have a cluster with one RM and one NM.
* A client tries to start a YARN service.
* RM send a request to NM to start the containers:
NM's log:
{code:java}
2020-04-11 14:23:44,030 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
2020-04-11 14:23:44,229 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Start request for container_6_0001_02_01 by user appattempt_6_0001_02
{code}
* NM starts the containers successfully:
NM's log:
{code:java}
2020-04-11 14:23:44,347 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl:
Application application_6_0001 transitioned from INITING to RUNNING
2020-04-11 14:23:44,357 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl:
Container container_6_0001_02_01 transitioned from NEW to LOCALIZING
{code}
* However, due to network partition, NM failed to send back the RPC response.
* After a while, the application is running happily:
RM's log:
{code:java}
2020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_6_0001_02 State change from ALLOCATED to RUNNING on event =
REGISTERED
020-04-11 14:23:50,359 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_6_0001 State change from ACCEPTED to RUNNING on event =
ATTEMPT_REGISTERED{code}
* Then, since RM didn't receive the RPC response for startContainers, it
retries:
NM's log:
{code:java}
2020-04-11 14:23:54,392 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth
successful for appattempt_6_0001_02 (auth:SIMPLE)
{code}
* But since the attempt is actually running, this launch request does not
succeed:
NM's log:
{code:java}
2020-04-11 14:23:54,401 ERROR
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Unauthorized request to start container.
Attempt to relaunch the same container with id container_6_0001_02_01.
{code}
RM's log:
{code:java}
2020-04-11 14:23:54,428 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Error
launching appattempt_6_0001_02. Got exception:
org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start
container.
Attempt to relaunch the same container with id container_6_0001_02_01.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateExceptionImpl(SerializedExceptionPBImpl.java:171)
at
org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:182)
at
[jira] [Comment Edited] (YARN-10004) Javadoc of YarnConfigurationStore#initialize is not straightforward
[ https://issues.apache.org/jira/browse/YARN-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081308#comment-17081308 ] Siddharth Ahuja edited comment on YARN-10004 at 4/11/20, 2:09 PM: -- Hey [~snemeth], it would be great if you could kindly clarify what you would like to be fixed in regards to the javadoc for the initialize method inside YarnConfigurationStore class and I will fix it. Thanks in advance for your assistance! was (Author: sahuja): Hey [~snemeth], it would be great if you could kindly clarify what you would like to be fixed in regards to the javadoc for the initialize method inside YarnConfigurationStore class. Thanks in advance for your assistance! > Javadoc of YarnConfigurationStore#initialize is not straightforward > --- > > Key: YARN-10004 > URL: https://issues.apache.org/jira/browse/YARN-10004 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10004) Javadoc of YarnConfigurationStore#initialize is not straightforward
[ https://issues.apache.org/jira/browse/YARN-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081308#comment-17081308 ] Siddharth Ahuja commented on YARN-10004: Hey [~snemeth], it would be great if you could kindly clarify what you would like to be fixed in regards to the javadoc for the initialize method inside YarnConfigurationStore class. Thanks in advance for your assistance! > Javadoc of YarnConfigurationStore#initialize is not straightforward > --- > > Key: YARN-10004 > URL: https://issues.apache.org/jira/browse/YARN-10004 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9831) NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService allocate flow
[ https://issues.apache.org/jira/browse/YARN-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081250#comment-17081250 ] Bilwa S T commented on YARN-9831: - Hi [~maniraj...@gmail.com] As nodeSet is ConcurrentHashSet there wont be any inconsistent data. It will take care of synchronization. > NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService > allocate flow > > > Key: YARN-9831 > URL: https://issues.apache.org/jira/browse/YARN-9831 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin Chundatt >Assignee: Bilwa S T >Priority: Critical > Attachments: YARN-9831.001.patch, YARN-9831.002.patch > > > Currently attempt's NMToken cannot be generated independently. > Each attempts allocate flow blocks each other. We should improve the same -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[ https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081220#comment-17081220 ] Siddharth Ahuja commented on YARN-9996: --- Hi [~snemeth], I uploaded a patch 3 hours ago and I still don't see any builds being triggered in Jenkins. Not sure if Jenkins is also taking an Easter break :) > Code cleanup in QueueAdminConfigurationMutationACLPolicy > > > Key: YARN-9996 > URL: https://issues.apache.org/jira/browse/YARN-9996 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Major > Attachments: YARN-9996.001.patch > > > Method 'isMutationAllowed' contains many uses of substring and lastIndexOf. > These could be extracted and simplified. > Also, some logging could be added as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[ https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081166#comment-17081166 ] Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 9:06 AM: - # Instead of manually extracting out the last child of the queue in the for-loop and then doing the same activity in the while-loop for the trimmed queue, my code makes this neater (and simpler) by pulling in the logic outside of the while-loop into the while-loop itself. This code optimization makes a saving of a call each to lastIndexOf() and substring(). # Further, I have made private methods for calls containing lastIndexOf() and substring() for the different scenarios to provide better context of what these calls are about. This abstracts away the need for the person reviewing the code to have to understand the internal logic of how and what needs to be done to extract out the queue names. # Added logging wherever necessary to provide more context. # No new JUnits added as the changes do not introduce/modify existing use-cases, they only optimize them i.e. changes are cosmetic at the end of the day. Class - org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestConfigurationMutationACLPolicies already contains test methods relevant for QueueAdminConfigurationMutationACLPolicy. Re-ran the existing test methods to ensure changes are good. was (Author: sahuja): # Instead of manually extracting out the last child of the queue in the for-loop and then doing the same activity in the while-loop for the trimmed queue, my code makes this neater my pulling in the logic outside of the while-loop into the while-loop itself. This code optimization makes a saving of a call each to lastIndexOf() and substring(). # Further, I have made private methods for calls containing lastIndexOf() and substring() for the different scenarios to provide better context of what these calls are about. This abstracts away the need for the person reviewing the code to have to understand the internal logic of how and what needs to be done to extract out the queue names. # Added logging wherever necessary to provide more context. # No new JUnits added as the changes do not introduce/modify existing use-cases, they only optimize them i.e. changes are cosmetic at the end of the day. Class - org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestConfigurationMutationACLPolicies already contains test methods relevant for QueueAdminConfigurationMutationACLPolicy. Re-ran the existing test methods to ensure changes are good. > Code cleanup in QueueAdminConfigurationMutationACLPolicy > > > Key: YARN-9996 > URL: https://issues.apache.org/jira/browse/YARN-9996 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Major > Attachments: YARN-9996.001.patch > > > Method 'isMutationAllowed' contains many uses of substring and lastIndexOf. > These could be extracted and simplified. > Also, some logging could be added as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 9:06 AM:
-
Further details:
The overall aim of the code (as I understand) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
+*After* (un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
+*After* (un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9279) Remove the old hamlet package
[
https://issues.apache.org/jira/browse/YARN-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081206#comment-17081206
]
Hadoop QA commented on YARN-9279:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
38m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
38s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
30s{color} | {color:green} root generated 0 new + 1845 unchanged - 25 fixed =
1845 total (was 1870) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
30s{color} | {color:green} root: The patch generated 0 new + 0 unchanged - 552
fixed = 0 total (was 552) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 16m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 5s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
10s{color} | {color:green} root generated 0 new + 983 unchanged - 4183 fixed =
983 total (was 5166) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}595m 32s{color}
| {color:red} root in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m
13s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}767m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.TestSLSRunner |
| | hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.hdfs.server.datanode.TestBPOfferService |
| | hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage |
| | hadoop.hdfs.server.datanode.TestDataNodeLifeline |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| |
[jira] [Updated] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[ https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Ahuja updated YARN-9996: -- Attachment: (was: YARN-9996.001.patch) > Code cleanup in QueueAdminConfigurationMutationACLPolicy > > > Key: YARN-9996 > URL: https://issues.apache.org/jira/browse/YARN-9996 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Major > Attachments: YARN-9996.001.patch > > > Method 'isMutationAllowed' contains many uses of substring and lastIndexOf. > These could be extracted and simplified. > Also, some logging could be added as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[ https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Ahuja updated YARN-9996: -- Attachment: YARN-9996.001.patch > Code cleanup in QueueAdminConfigurationMutationACLPolicy > > > Key: YARN-9996 > URL: https://issues.apache.org/jira/browse/YARN-9996 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Siddharth Ahuja >Priority: Major > Attachments: YARN-9996.001.patch > > > Method 'isMutationAllowed' contains many uses of substring and lastIndexOf. > These could be extracted and simplified. > Also, some logging could be added as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 6:15 AM:
-
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
+*After* (un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
*After* (un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 6:12 AM:
-
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
*After* (un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
*After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 6:12 AM:
-
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
*After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 6:12 AM:
-
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
{code}
After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja commented on YARN-9996:
---
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before *(un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
+*After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9996) Code cleanup in QueueAdminConfigurationMutationACLPolicy
[
https://issues.apache.org/jira/browse/YARN-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081178#comment-17081178
]
Siddharth Ahuja edited comment on YARN-9996 at 4/11/20, 6:11 AM:
-
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before* (un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
was (Author: sahuja):
Further details:
The overall aim of the code (as understood) is to get the last child in the
queue hierarchy that has queue information available to check for admin user
access. Until this queue information is available, keep going up in the queue
hierarchy one child at a time.
+*Before *(un-important bits extracted away):+
{code}
for each queue
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
while we do not have queue info
{
get the queue minus its last child (move up in the hierarchy)
if the trimmed queue has a child, get the last child.
Try getting the queue info for this child.
}
...
}
+*After*(un-important bits extracted away):+
{code}
for each queue
{
while we do not have queue info
{
if the queue has a child, get the last child.
Try getting the queue info for this child.
if the queue has a child, trim the queue and get the queue minus its
last child (move up in the hierarchy).
}
...
}
{code}
> Code cleanup in QueueAdminConfigurationMutationACLPolicy
>
>
> Key: YARN-9996
> URL: https://issues.apache.org/jira/browse/YARN-9996
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Siddharth Ahuja
>Priority: Major
> Attachments: YARN-9996.001.patch
>
>
> Method 'isMutationAllowed' contains many uses of substring and lastIndexOf.
> These could be extracted and simplified.
> Also, some logging could be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
