[jira] [Created] (TS-1006) memory management, cut down memory waste ?

2011-10-29 Thread Zhao Yongming (Created) (JIRA) 
memory management, cut down memory waste ?
--

 Key: TS-1006
 URL: https://issues.apache.org/jira/browse/TS-1006
 Project: Traffic Server
  Issue Type: Improvement
  Components: Core
Affects Versions: 3.1.1
Reporter: Zhao Yongming
Assignee: Zhao Yongming
 Fix For: 3.1.3


when we review the memory usage in the production, there is something abnormal, 
ie, looks like TS take much memory than index data + common system waste, and 
here is some memory dump result by set proxy.config.dump_mem_info_frequency

1, the one on a not so busy forwarding system:
physics memory: 32G
RAM cache: 22G
DISK: 6140 GB
average_object_size 64000
{code}
 allocated  |in-use  | type size  |   free list name
|||--
  671088640 |   37748736 |2097152 | 
memory/ioBufAllocator[14]
 2248146944 | 2135949312 |1048576 | 
memory/ioBufAllocator[13]
 1711276032 | 1705508864 | 524288 | 
memory/ioBufAllocator[12]
 1669332992 | 1667760128 | 262144 | 
memory/ioBufAllocator[11]
 2214592512 | 221184 | 131072 | 
memory/ioBufAllocator[10]
 2325741568 | 2323775488 |  65536 | memory/ioBufAllocator[9]
 2091909120 | 2089123840 |  32768 | memory/ioBufAllocator[8]
 1956642816 | 1956478976 |  16384 | memory/ioBufAllocator[7]
 2094530560 | 2094071808 |   8192 | memory/ioBufAllocator[6]
  356515840 |  355540992 |   4096 | memory/ioBufAllocator[5]
1048576 |  14336 |   2048 | memory/ioBufAllocator[4]
 131072 |  0 |   1024 | memory/ioBufAllocator[3]
  65536 |  0 |512 | memory/ioBufAllocator[2]
  32768 |  0 |256 | memory/ioBufAllocator[1]
  16384 |  0 |128 | memory/ioBufAllocator[0]
  0 |  0 |576 | 
memory/ICPRequestCont_allocator
  0 |  0 |112 | 
memory/ICPPeerReadContAllocator
  0 |  0 |432 | 
memory/PeerReadDataAllocator
  0 |  0 | 32 | 
memory/MIMEFieldSDKHandle
  0 |  0 |240 | memory/INKVConnAllocator
  0 |  0 | 96 | memory/INKContAllocator
   4096 |  0 | 32 | memory/apiHookAllocator
  0 |  0 |288 | memory/FetchSMAllocator
  0 |  0 | 80 | 
memory/prefetchLockHandlerAllocator
  0 |  0 |176 | 
memory/PrefetchBlasterAllocator
  0 |  0 | 80 | 
memory/prefetchUrlBlaster
  0 |  0 | 96 | memory/blasterUrlList
  0 |  0 | 96 | 
memory/prefetchUrlEntryAllocator
  0 |  0 |128 | 
memory/socksProxyAllocator
  0 |  0 |144 | memory/ObjectReloadCont
3258368 | 576016 |592 | 
memory/httpClientSessionAllocator
 825344 | 139568 |208 | 
memory/httpServerSessionAllocator
   22597632 |1284848 |   9808 | memory/httpSMAllocator
  0 |  0 | 32 | 
memory/CacheLookupHttpConfigAllocator
  0 |  0 |   9856 | 
memory/httpUpdateSMAllocator
  0 |  0 |128 | memory/RemapPluginsAlloc
  0 |  0 | 48 | 
memory/CongestRequestParamAllocator
  0 |  0 |128 | 
memory/CongestionDBContAllocator
5767168 | 704512 |   2048 | memory/hdrStrHeap
   18350080 |1153024 |   2048 | memory/hdrHeap
  53248 |   2912 |208 | 
memory/httpCacheAltAllocator
  0 |  0 |112 | 
memory/OneWayTunnelAllocator
 157696 |   1232 |   1232 | 
memory/hostDBContAllocator
 102240 |  17040 |  17040 | memory/dnsBufAllocator
 323584 |  0 |   1264 | memory/dnsEntryAllocator
  0 |  0 | 16 | 
memory/DNSRequestDataAllocator
  0 |  0 |   1072 | memory/SRVAllocator
  0 |  0 | 48 | 
memory/ClusterVConnectionCache::Entry
  0 |  0

[jira] [Commented] (TS-1006) memory management, cut down memory waste ?

2011-10-29 Thread Conan Wang (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139397#comment-13139397
 ] 

Conan Wang commented on TS-1006:


my reverse box: 3.0.1 on centos 5.4 64bit

physics memory: 4G
RAM cache: 1.8G
DISK: 407 GB (raw device)
average_object_size: 8000

{code}
 allocated  |in-use  | type size  |   free list name
|||--
   67108864 |  0 |2097152 | 
memory/ioBufAllocator[14]
 1744830464 |   76546048 |1048576 | 
memory/ioBufAllocator[13]
   33554432 |4194304 | 524288 | 
memory/ioBufAllocator[12]
   33554432 |6553600 | 262144 | 
memory/ioBufAllocator[11]
   37748736 |8912896 | 131072 | 
memory/ioBufAllocator[10]
   69206016 |   31850496 |  65536 | memory/ioBufAllocator[9]
  529530880 |  365658112 |  32768 | memory/ioBufAllocator[8]
  414187520 |  411942912 |  16384 | memory/ioBufAllocator[7]
  898629632 |  898547712 |   8192 | memory/ioBufAllocator[6]
   83361792 |   83161088 |   4096 | memory/ioBufAllocator[5]
 262144 |  0 |   2048 | memory/ioBufAllocator[4]
 131072 |  0 |   1024 | memory/ioBufAllocator[3]
  65536 |  0 |512 | memory/ioBufAllocator[2]
  32768 |  0 |256 | memory/ioBufAllocator[1]
  0 |  0 |128 | memory/ioBufAllocator[0]
  0 |  0 |576 | 
memory/ICPRequestCont_allocator
  0 |  0 |112 | 
memory/ICPPeerReadContAllocator
  0 |  0 |432 | 
memory/PeerReadDataAllocator
   4096 |160 | 32 | 
memory/MIMEFieldSDKHandle
  0 |  0 |240 | memory/INKVConnAllocator
  12288 | 96 | 96 | memory/INKContAllocator
   4096 | 32 | 32 | memory/apiHookAllocator
  0 |  0 |288 | memory/FetchSMAllocator
  0 |  0 | 80 | 
memory/prefetchLockHandlerAllocator
  0 |  0 |176 | 
memory/PrefetchBlasterAllocator
  0 |  0 | 80 | 
memory/prefetchUrlBlaster
  0 |  0 | 96 | memory/blasterUrlList
  0 |  0 | 96 | 
memory/prefetchUrlEntryAllocator
  0 |  0 |128 | 
memory/socksProxyAllocator
  0 |  0 |144 | memory/ObjectReloadCont
1060864 |   2960 |592 | 
memory/httpClientSessionAllocator
  53248 |416 |208 | 
memory/httpServerSessionAllocator
   17575936 |  19616 |   9808 | memory/httpSMAllocator
  0 |  0 | 32 | 
memory/CacheLookupHttpConfigAllocator
  0 |  0 |   9856 | 
memory/httpUpdateSMAllocator
  0 |  0 |128 | memory/RemapPluginsAlloc
  0 |  0 | 48 | 
memory/CongestRequestParamAllocator
  0 |  0 |128 | 
memory/CongestionDBContAllocator
7340032 | 264192 |   2048 | memory/hdrStrHeap
8650752 | 274432 |   2048 | memory/hdrHeap
#
  26624 |  0 |208 | 
memory/httpCacheAltAllocator
  0 |  0 |112 | 
memory/OneWayTunnelAllocator
 315392 |   1232 |   1232 | 
memory/hostDBContAllocator
  68160 |  17040 |  17040 | memory/dnsBufAllocator
 161792 |  0 |   1264 | memory/dnsEntryAllocator
  0 |  0 | 16 | 
memory/DNSRequestDataAllocator
  0 |  0 |   1072 | memory/SRVAllocator
  0 |  0 | 48 | 
memory/ClusterVConnectionCache::Entry
  0 |  0 |560 | 
memory/cacheContAllocator
  0 |  0 |112 | 
memory/inControlAllocator
  0 |  0 |112 | 
memory/outControlAllocator
  0 |  0 | 32 | memory/byteBankAllocator
  0 |  0 |576 | 
memory/clusterVCAllocator
  0 |