[ 
https://issues.apache.org/jira/browse/HDFS-12163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108359#comment-16108359
 ] 

Weiwei Yang commented on HDFS-12163:
------------------------------------

Run 1 DN in mini ozone cluster, there were *402* threads once the cluster is 
up, checked the thread dump, there are *280* RPC handler threads, with 
following trace

{code}
IPC Server handler 0 on 51641@2330 - priority:5 - threadId:0x45 - nativeId:NA - 
state:WAITING
stackTrace:
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:287)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2661)
{code}

this is because
 {{dfs.storage.service.handler.count}} = {{10}}
 {{ozone.ksm.handler.count.key}} = {{200}}

we are setting a pretty big value for KSM handlers, this is not necessary for a 
mini cluster, we can overwrite this config in mini cluster to reduce the number 
of threads. I will check the leak issue a bit later.

> Ozone: MiniOzoneCluster uses 400+ threads
> -----------------------------------------
>
>                 Key: HDFS-12163
>                 URL: https://issues.apache.org/jira/browse/HDFS-12163
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone, test
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Weiwei Yang
>         Attachments: TestOzoneThreadCount20170719.patch
>
>
> Checked the number of active threads used in MiniOzoneCluster with various 
> settings:
> - Local handlers
> - Distributed handlers
> - Ratis-Netty
> - Ratis-gRPC
> The results are similar for all the settings.  It uses 400+ threads for an 
> 1-datanode MiniOzoneCluster.
> Moreover, there is a thread leak -- a number of the threads do not shutdown 
> after the test is finished.  Therefore, when tests run consecutively, the 
> later tests use more threads.
> Will post the details in comments.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to