[jira] [Commented] (YARN-8291) RMRegistryOperationService don't have limit on AsyncPurge threads

2018-05-18 Thread Prabhu Joseph (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480424#comment-16480424
 ] 

Prabhu Joseph commented on YARN-8291:
-

The trunk code is having this issue.

> RMRegistryOperationService don't have limit on AsyncPurge threads
> -
>
> Key: YARN-8291
> URL: https://issues.apache.org/jira/browse/YARN-8291
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn
>Affects Versions: 3.0.0
>Reporter: Prabhu Joseph
>Priority: Major
>
> When there are more than 1+ containers finished - 
> RMRegistryOperationService will create 1+ threads for performing 
> AsyncPurge which can slowdown the ResourceManager process. There should be a 
> limit on the number of threads.
> {code}
> "RegistryAdminService 554485" #824351 prio=5 os_prio=0 tid=0x7fe4b2bc9800 
> nid=0xf8ed in Object.wait() [0x7fe31a5e4000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1386)
> - locked <0x0007902ec7d8> (a 
> org.apache.zookeeper.ClientCnxn$Packet)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:158)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)
> at 
> org.apache.hadoop.registry.client.impl.zk.CuratorService.zkStat(CuratorService.java:455)
> at 
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.stat(RegistryOperationsService.java:137)
> at 
> org.apache.hadoop.registry.client.binding.RegistryUtils.statChildren(RegistryUtils.java:210)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService.purge(RegistryAdminService.java:450)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService.purge(RegistryAdminService.java:520)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService$AsyncPurge.call(RegistryAdminService.java:570)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService$AsyncPurge.call(RegistryAdminService.java:543)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-8291) RMRegistryOperationService don't have limit on AsyncPurge threads

2018-05-18 Thread Feng Yuan (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480334#comment-16480334
 ] 

Feng Yuan commented on YARN-8291:
-

Hi, are you sure version 2.7.3?

> RMRegistryOperationService don't have limit on AsyncPurge threads
> -
>
> Key: YARN-8291
> URL: https://issues.apache.org/jira/browse/YARN-8291
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn
>Affects Versions: 2.7.3
>Reporter: Prabhu Joseph
>Priority: Major
>
> When there are more than 1+ containers finished - 
> RMRegistryOperationService will create 1+ threads for performing 
> AsyncPurge which can slowdown the ResourceManager process. There should be a 
> limit on the number of threads.
> {code}
> "RegistryAdminService 554485" #824351 prio=5 os_prio=0 tid=0x7fe4b2bc9800 
> nid=0xf8ed in Object.wait() [0x7fe31a5e4000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1386)
> - locked <0x0007902ec7d8> (a 
> org.apache.zookeeper.ClientCnxn$Packet)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:158)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)
> at 
> org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)
> at 
> org.apache.hadoop.registry.client.impl.zk.CuratorService.zkStat(CuratorService.java:455)
> at 
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.stat(RegistryOperationsService.java:137)
> at 
> org.apache.hadoop.registry.client.binding.RegistryUtils.statChildren(RegistryUtils.java:210)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService.purge(RegistryAdminService.java:450)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService.purge(RegistryAdminService.java:520)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService$AsyncPurge.call(RegistryAdminService.java:570)
> at 
> org.apache.hadoop.registry.server.services.RegistryAdminService$AsyncPurge.call(RegistryAdminService.java:543)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org