[ https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539172#comment-13539172 ]
jaekyung oh commented on TS-1006: --------------------------------- i applied the patches onto ats 3.2.0 and i've got these errors. OS : OpenSuse 11.4 RAM : 16G config : CONFIG proxy.config.cache.ram_cache.size INT 8000000000 CONFIG proxy.config.cache.ram_cache_cutoff INT 6000000000 traffic.out [Dec 21 19:05:00.070] Server {0x2ba0c51d1700} STATUS: The logfile /usr/local/var/log/trafficserver/squid.log was rolled to /usr/local/var/log/trafficserver/squid.log_PM0010-Z001-R001.20121221.18h59m55s-20121221.19h05m00s.old. /usr/local/bin/traffic_server - STACK TRACE: /usr/local/bin/traffic_server[0x64a68f] /usr/local/bin/traffic_server(_ZN7CacheVC10handleReadEiP5Event+0x51a)[0x64f28a] /usr/local/bin/traffic_server(_ZN5Cache9open_readEP12ContinuationP7INK_MD5P7HTTPHdrP21CacheLookupHttpConfig13CacheFragTypePci+0x754)[0x67ef54] /usr/local/bin/traffic_server(_ZN14CacheProcessor9open_readEP12ContinuationP3URLP7HTTPHdrP21CacheLookupHttpConfigl13CacheFragType+0x130)[0x6540f0] /usr/local/bin/traffic_server(_ZN11HttpCacheSM9open_readEP3URLP7HTTPHdrP21CacheLookupHttpConfigl+0x74)[0x521c14] /usr/local/bin/traffic_server(_ZN6HttpSM24do_cache_lookup_and_readEv+0xfd)[0x538c1d] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0xb20)[0x54ffd0] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0xb92)[0x550042] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x9a)[0x55091a] /usr/local/bin/traffic_server(TSHttpTxnReenable+0x40d)[0x4bec1d] /usr/local/libexec/trafficserver/purge.so(+0x1e56)[0x2ba0c5a3fe56] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0xfb)[0x54927b] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM32state_read_client_request_headerEiPv+0x84f)[0x54723f] /usr/local/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xe8)[0x54d208] /usr/local/bin/traffic_server[0x6b6f4a] /usr/local/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x1fe)[0x6aff0e] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8e)[0x6dfd4e] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x4f0)[0x6e06c0] /usr/local/bin/traffic_server(main+0xdee)[0x4cc2ce] /lib64/libc.so.6(__libc_start_main+0xfd)[0x2ba0b976cbfd] /usr/local/bin/traffic_server[0x48a369] FATAL: Failed to mmap 4198400 bytes, Cannot allocate memory /usr/local/bin/traffic_server - STACK TRACE: /usr/local/lib/libtsutil.so.3(ink_fatal+0x88)[0x2ba0b73b7c68] /usr/local/lib/libtsutil.so.3(ink_freelist_new+0xc09)[0x2ba0b73bc1f9] /usr/local/bin/traffic_server[0x64a68f] /usr/local/bin/traffic_server(_ZN7CacheVC10handleReadEiP5Event+0x51a)[0x64f28a] /usr/local/bin/traffic_server(_ZN5Cache9open_readEP12ContinuationP7INK_MD5P7HTTPHdrP21CacheLookupHttpConfig13CacheFragTypePci+0x754)[0x67ef54] /usr/local/bin/traffic_server(_ZN14CacheProcessor9open_readEP12ContinuationP3URLP7HTTPHdrP21CacheLookupHttpConfigl13CacheFragType+0x130)[0x6540f0] /usr/local/bin/traffic_server(_ZN11HttpCacheSM9open_readEP3URLP7HTTPHdrP21CacheLookupHttpConfigl+0x74)[0x521c14] /usr/local/bin/traffic_server(_ZN6HttpSM24do_cache_lookup_and_readEv+0xfd)[0x538c1d] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0xb20)[0x54ffd0] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0xb92)[0x550042] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x33a)[0x547fda] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x353)[0x5494d3] /usr/local/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x9a)[0x55091a] /usr/local/bin/traffic_server(TSHttpTxnReenable+0x40d)[0x4bec1d] /usr/local/libexec/trafficserver/purge.so(+0x1e56)[0x2ba0c5a3fe56] /usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0xfb)[0x54927b] /usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x538)[0x54f9e8] /usr/local/bin/traffic_server(_ZN6HttpSM32state_read_client_request_headerEiPv+0x84f)[0x54723f] /usr/local/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xe8)[0x54d208] /usr/local/bin/traffic_server[0x6b6f4a] /usr/local/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x1fe)[0x6aff0e] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8e)[0x6dfd4e] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x4f0)[0x6e06c0] /usr/local/bin/traffic_server(main+0xdee)[0x4cc2ce] /lib64/libc.so.6(__libc_start_main+0xfd)[0x2ba0b976cbfd] /usr/local/bin/traffic_server[0x48a369] [Dec 21 19:05:29.767] Manager {0x7f730822e720} FATAL: [LocalManager::pollMgmtProcessServer] Error in read (errno: 104) [Dec 21 19:05:29.767] Manager {0x7f730822e720} FATAL: (last system error 104: Connection reset by peer) [Dec 21 19:05:29.767] Manager {0x7f730822e720} NOTE: [LocalManager::mgmtShutdown] Executing shutdown request. [Dec 21 19:05:29.767] Manager {0x7f730822e720} NOTE: [LocalManager::processShutdown] Executing process shutdown request. [Dec 21 19:05:29.767] Manager {0x7f730822e720} ERROR: [LocalManager::sendMgmtMsgToProcesses] Error writing message [Dec 21 19:05:29.767] Manager {0x7f730822e720} ERROR: (last system error 32: Broken pipe) [E. Mgmt] log ==> [TrafficManager] using root directory '/usr/local' [Dec 21 19:05:29.799] {0x7f3b1cf41720} STATUS: opened /usr/local/var/log/trafficserver/manager.log [Dec 21 19:05:29.799] {0x7f3b1cf41720} NOTE: updated diags config [Dec 21 19:05:29.799] Manager {0x7f3b1cf41720} NOTE: [appendDefaultDomain] Unable to determine default domain name. Nodes will be know by their unqualified host name [Dec 21 19:05:29.800] Manager {0x7f3b1cf41720} NOTE: [ClusterCom::ClusterCom] Node running on OS: 'Linux' Release: '2.6.37.1-1.2-desktop' [Dec 21 19:05:29.801] Manager {0x7f3b1cf41720} NOTE: [LocalManager::listenForProxy] Listening on port: 80 [Dec 21 19:05:29.801] Manager {0x7f3b1cf41720} NOTE: [LocalManager::listenForProxy] Listening on port: 443 [Dec 21 19:05:29.801] Manager {0x7f3b1cf41720} NOTE: [TrafficManager] Setup complete [Dec 21 19:05:30.816] Manager {0x7f3b1cf41720} NOTE: [LocalManager::startProxy] Launching ts process [TrafficServer] using root directory '/usr/local' [Dec 21 19:05:30.825] Manager {0x7f3b1cf41720} NOTE: [LocalManager::pollMgmtProcessServer] New process connecting fd '14' [Dec 21 19:05:30.825] Manager {0x7f3b1cf41720} NOTE: [Alarms::signalAlarm] Server Process born [Dec 21 19:05:31.843] {0x2b9e0643ce20} STATUS: opened /usr/local/var/log/trafficserver/diags.log [Dec 21 19:05:31.843] {0x2b9e0643ce20} NOTE: updated diags config [Dec 21 19:05:31.848] Server {0x2b9e0643ce20} NOTE: cache clustering disabled [Dec 21 19:05:31.861] Server {0x2b9e0643ce20} NOTE: cache clustering disabled [Dec 21 19:05:31.881] Server {0x2b9e0643ce20} NOTE: logging initialized[15], logging_mode = 3 [Dec 21 19:05:31.890] Server {0x2b9e0643ce20} NOTE: loading plugin '/usr/local/libexec/trafficserver/purge.so' [Dec 21 19:05:31.891] Server {0x2b9e0643ce20} NOTE: loading plugin '/usr/local/libexec/trafficserver/remap_bgcache.so' [Dec 21 19:05:31.893] Server {0x2b9e0643ce20} NOTE: traffic server running NOTE: Traffic Server received Sig 11: Segmentation fault /usr/local/bin/traffic_server - STACK TRACE: /lib64/libpthread.so.0(+0xf2d0)[0x2b9e03f6a2d0] /usr/local/bin/traffic_server(_Z15dir_clear_rangellP3Vol+0x6a)[0x65ac5a] /usr/local/bin/traffic_server(_ZN3Vol24handle_recover_from_dataEiPv+0x4de)[0x65188e] /usr/local/bin/traffic_server(_ZN19AIOCallbackInternal11io_completeEiPv+0x35)[0x656415] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8e)[0x6dfd4e] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x5ab)[0x6e077b] /usr/local/bin/traffic_server[0x6df9b2] /lib64/libpthread.so.0(+0x6a3f)[0x2b9e03f61a3f] /lib64/libc.so.6(clone+0x6d)[0x2b9e0619f67d] [Dec 21 19:06:06.183] Manager {0x7f3b1cf41720} FATAL: [LocalManager::pollMgmtProcessServer] Error in read (errno: 104) [Dec 21 19:06:06.183] Manager {0x7f3b1cf41720} FATAL: (last system error 104: Connection reset by peer) [Dec 21 19:06:06.183] Manager {0x7f3b1cf41720} NOTE: [LocalManager::mgmtShutdown] Executing shutdown request. [Dec 21 19:06:06.183] Manager {0x7f3b1cf41720} NOTE: [LocalManager::processShutdown] Executing process shutdown request. . . . . (repeat restarting by "dir_clear_rangellP3Vol" error) any hint? should i not apply the patches on ats 3.2.0? > memory management, cut down memory waste ? > ------------------------------------------ > > Key: TS-1006 > URL: https://issues.apache.org/jira/browse/TS-1006 > Project: Traffic Server > Issue Type: Improvement > Components: Core > Affects Versions: 3.1.1 > Reporter: Zhao Yongming > Assignee: Bin Chen > Fix For: 3.3.2 > > Attachments: 0001-Allocator-optimize-InkFreeList-memory-pool.patch, > 0002-Allocator-make-InkFreeList-memory-pool-configurable.patch, > Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods > > > when we review the memory usage in the production, there is something > abnormal, ie, looks like TS take much memory than index data + common system > waste, and here is some memory dump result by set > "proxy.config.dump_mem_info_frequency" > 1, the one on a not so busy forwarding system: > physics memory: 32G > RAM cache: 22G > DISK: 6140 GB > average_object_size 64000 > {code} > allocated | in-use | type size | free list name > --------------------|--------------------|------------|---------------------------------- > 671088640 | 37748736 | 2097152 | > memory/ioBufAllocator[14] > 2248146944 | 2135949312 | 1048576 | > memory/ioBufAllocator[13] > 1711276032 | 1705508864 | 524288 | > memory/ioBufAllocator[12] > 1669332992 | 1667760128 | 262144 | > memory/ioBufAllocator[11] > 2214592512 | 2211840000 | 131072 | > memory/ioBufAllocator[10] > 2325741568 | 2323775488 | 65536 | > memory/ioBufAllocator[9] > 2091909120 | 2089123840 | 32768 | > memory/ioBufAllocator[8] > 1956642816 | 1956478976 | 16384 | > memory/ioBufAllocator[7] > 2094530560 | 2094071808 | 8192 | > memory/ioBufAllocator[6] > 356515840 | 355540992 | 4096 | > memory/ioBufAllocator[5] > 1048576 | 14336 | 2048 | > memory/ioBufAllocator[4] > 131072 | 0 | 1024 | > memory/ioBufAllocator[3] > 65536 | 0 | 512 | > memory/ioBufAllocator[2] > 32768 | 0 | 256 | > memory/ioBufAllocator[1] > 16384 | 0 | 128 | > memory/ioBufAllocator[0] > 0 | 0 | 576 | > memory/ICPRequestCont_allocator > 0 | 0 | 112 | > memory/ICPPeerReadContAllocator > 0 | 0 | 432 | > memory/PeerReadDataAllocator > 0 | 0 | 32 | > memory/MIMEFieldSDKHandle > 0 | 0 | 240 | > memory/INKVConnAllocator > 0 | 0 | 96 | > memory/INKContAllocator > 4096 | 0 | 32 | > memory/apiHookAllocator > 0 | 0 | 288 | > memory/FetchSMAllocator > 0 | 0 | 80 | > memory/prefetchLockHandlerAllocator > 0 | 0 | 176 | > memory/PrefetchBlasterAllocator > 0 | 0 | 80 | > memory/prefetchUrlBlaster > 0 | 0 | 96 | memory/blasterUrlList > 0 | 0 | 96 | > memory/prefetchUrlEntryAllocator > 0 | 0 | 128 | > memory/socksProxyAllocator > 0 | 0 | 144 | > memory/ObjectReloadCont > 3258368 | 576016 | 592 | > memory/httpClientSessionAllocator > 825344 | 139568 | 208 | > memory/httpServerSessionAllocator > 22597632 | 1284848 | 9808 | memory/httpSMAllocator > 0 | 0 | 32 | > memory/CacheLookupHttpConfigAllocator > 0 | 0 | 9856 | > memory/httpUpdateSMAllocator > 0 | 0 | 128 | > memory/RemapPluginsAlloc > 0 | 0 | 48 | > memory/CongestRequestParamAllocator > 0 | 0 | 128 | > memory/CongestionDBContAllocator > 5767168 | 704512 | 2048 | memory/hdrStrHeap > 18350080 | 1153024 | 2048 | memory/hdrHeap > 53248 | 2912 | 208 | > memory/httpCacheAltAllocator > 0 | 0 | 112 | > memory/OneWayTunnelAllocator > 157696 | 1232 | 1232 | > memory/hostDBContAllocator > 102240 | 17040 | 17040 | memory/dnsBufAllocator > 323584 | 0 | 1264 | > memory/dnsEntryAllocator > 0 | 0 | 16 | > memory/DNSRequestDataAllocator > 0 | 0 | 1072 | memory/SRVAllocator > 0 | 0 | 48 | > memory/ClusterVConnectionCache::Entry > 0 | 0 | 560 | > memory/cacheContAllocator > 0 | 0 | 112 | > memory/inControlAllocator > 0 | 0 | 112 | > memory/outControlAllocator > 0 | 0 | 32 | > memory/byteBankAllocator > 0 | 0 | 576 | > memory/clusterVCAllocator > 0 | 0 | 48 | memory/evacuationKey > 6144 | 0 | 48 | memory/cacheRemoveCont > 270336 | 262560 | 96 | memory/evacuationBlock > 4997120 | 3968416 | 976 | > memory/cacheVConnection > 798720 | 522080 | 160 | memory/openDirEntry > 0 | 0 | 64 | > memory/RamCacheLRUEntry > 56426496 | 56426304 | 96 | > memory/RamCacheCLFUSEntry > 9584640 | 6168000 | 960 | memory/netVCAllocator > 0 | 0 | 128 | > memory/udpReadContAllocator > 0 | 0 | 128 | > memory/udpWorkContinuationAllocator > 0 | 0 | 160 | > memory/udpPacketAllocator > 0 | 0 | 304 | memory/socksAllocator > 139264 | 68544 | 1088 | > memory/sslNetVCAllocator > 0 | 0 | 128 | > memory/UDPIOEventAllocator > 671744 | 115520 | 64 | > memory/ioBlockAllocator > 28305408 | 28301520 | 48 | memory/ioDataAllocator > 2273280 | 406320 | 240 | memory/ioAllocator > 1904640 | 1489920 | 80 | memory/mutexAllocator > 1105920 | 188544 | 96 | memory/eventAllocator > 2359296 | 129024 | 1024 | memory/ArenaBlock > {code} > this box will crash every 2days, so the memory waste may no that high > 2, our production reverse system: > physics memory: 16G > RAM cache: 8G > DISK: 1516 GB > average_object_size 16384 > and it run for a much long time: > {code} > allocated | in-use | type size | free list name > --------------------|--------------------|------------|---------------------------------- > 805306368 | 0 | 2097152 | > memory/ioBufAllocator[14] > 738197504 | 8388608 | 1048576 | > memory/ioBufAllocator[13] > 1258291200 | 46661632 | 524288 | > memory/ioBufAllocator[12] > 1300234240 | 183762944 | 262144 | > memory/ioBufAllocator[11] > 1170210816 | 466223104 | 131072 | > memory/ioBufAllocator[10] > 1790967808 | 1223426048 | 65536 | > memory/ioBufAllocator[9] > 2970615808 | 2601418752 | 32768 | > memory/ioBufAllocator[8] > 2067791872 | 2044608512 | 16384 | > memory/ioBufAllocator[7] > 1169424384 | 1169121280 | 8192 | > memory/ioBufAllocator[6] > 711458816 | 710463488 | 4096 | > memory/ioBufAllocator[5] > 1572864 | 0 | 2048 | > memory/ioBufAllocator[4] > 131072 | 0 | 1024 | > memory/ioBufAllocator[3] > 65536 | 0 | 512 | > memory/ioBufAllocator[2] > 32768 | 0 | 256 | > memory/ioBufAllocator[1] > 16384 | 0 | 128 | > memory/ioBufAllocator[0] > 0 | 0 | 576 | > memory/ICPRequestCont_allocator > 0 | 0 | 112 | > memory/ICPPeerReadContAllocator > 0 | 0 | 432 | > memory/PeerReadDataAllocator > 0 | 0 | 32 | > memory/MIMEFieldSDKHandle > 0 | 0 | 240 | > memory/INKVConnAllocator > 0 | 0 | 96 | > memory/INKContAllocator > 4096 | 0 | 32 | > memory/apiHookAllocator > 0 | 0 | 288 | > memory/FetchSMAllocator > 0 | 0 | 80 | > memory/prefetchLockHandlerAllocator > 0 | 0 | 176 | > memory/PrefetchBlasterAllocator > 0 | 0 | 80 | > memory/prefetchUrlBlaster > 0 | 0 | 96 | memory/blasterUrlList > 0 | 0 | 96 | > memory/prefetchUrlEntryAllocator > 0 | 0 | 128 | > memory/socksProxyAllocator > 0 | 0 | 144 | > memory/ObjectReloadCont > 1136640 | 125504 | 592 | > memory/httpClientSessionAllocator > 372736 | 27248 | 208 | > memory/httpServerSessionAllocator > 11317248 | 39296 | 9824 | memory/httpSMAllocator > 0 | 0 | 32 | > memory/CacheLookupHttpConfigAllocator > 0 | 0 | 9888 | > memory/httpUpdateSMAllocator > 0 | 0 | 128 | > memory/RemapPluginsAlloc > 0 | 0 | 512 | memory/HCSMAllocator > 0 | 0 | 48 | > memory/VCEntryAllocator > 0 | 0 | 96 | > memory/HCEntryAllocator > 0 | 0 | 64 | > memory/HCHandlerAllocator > 0 | 0 | 48 | > memory/CongestRequestParamAllocator > 0 | 0 | 128 | > memory/CongestionDBContAllocator > 6029312 | 643072 | 2048 | memory/hdrStrHeap > 7077888 | 657408 | 2048 | memory/hdrHeap > 26624 | 208 | 208 | > memory/httpCacheAltAllocator > 0 | 0 | 112 | > memory/OneWayTunnelAllocator > 630784 | 1232 | 1232 | > memory/hostDBContAllocator > 238560 | 17040 | 17040 | memory/dnsBufAllocator > 161792 | 0 | 1264 | > memory/dnsEntryAllocator > 0 | 0 | 16 | > memory/DNSRequestDataAllocator > 0 | 0 | 1072 | memory/SRVAllocator > 0 | 0 | 48 | > memory/ClusterVConnectionCache::Entry > 0 | 0 | 560 | > memory/cacheContAllocator > 0 | 0 | 112 | > memory/inControlAllocator > 0 | 0 | 112 | > memory/outControlAllocator > 0 | 0 | 32 | > memory/byteBankAllocator > 0 | 0 | 576 | > memory/clusterVCAllocator > 0 | 0 | 48 | memory/evacuationKey > 6144 | 0 | 48 | memory/cacheRemoveCont > 17006592 | 14972928 | 96 | memory/evacuationBlock > 1777664 | 759872 | 992 | > memory/cacheVConnection > 307200 | 111520 | 160 | memory/openDirEntry > 0 | 0 | 64 | > memory/RamCacheLRUEntry > 104275968 | 104274048 | 96 | > memory/RamCacheCLFUSEntry > 3440640 | 1819200 | 960 | memory/netVCAllocator > 0 | 0 | 128 | > memory/udpReadContAllocator > 0 | 0 | 128 | > memory/udpWorkContinuationAllocator > 0 | 0 | 160 | > memory/udpPacketAllocator > 0 | 0 | 304 | memory/socksAllocator > 0 | 0 | 1088 | > memory/sslNetVCAllocator > 0 | 0 | 128 | > memory/UDPIOEventAllocator > 237568 | 22528 | 64 | > memory/ioBlockAllocator > 26087424 | 26081904 | 48 | memory/ioDataAllocator > 890880 | 84240 | 240 | memory/ioAllocator > 1525760 | 1403440 | 80 | memory/mutexAllocator > 565248 | 129696 | 96 | memory/eventAllocator > 1179648 | 4096 | 1024 | memory/ArenaBlock > {code} > our team is working on the memory free issue, trying to improve the memory > management. and this a big project, the more input|comment the better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira