[jira] [Commented] (TS-1699) document proxy.config.http.doc_in_cache_skip_dns
[ https://issues.apache.org/jira/browse/TS-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573161#comment-13573161 ] James Peach commented on TS-1699: - Alternatively, we could call the OS_DNS_HOOK even when we skip the DNS request. This would let us keep both the API contract and the performance win. Not sure how hard this would be. > document proxy.config.http.doc_in_cache_skip_dns > > > Key: TS-1699 > URL: https://issues.apache.org/jira/browse/TS-1699 > Project: Traffic Server > Issue Type: Bug > Components: Documentation >Reporter: James Peach > > Authentication plugins typically attache to the OS_DNS_HOOK, but if they do > then they need to set proxy.config.http.doc_in_cache_skip_dns=0 so that the > DNS hook gets called on every request. > We should document this parameter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Created] (TS-1684) Reduce the usage of global allocation/free lists - switch to using local thread allocation/free lists
在 2013年2月7日星期四，Bryan Call 写道： > I was briefly looking at the changes today. I will take a look more at > them when I get some time. > > There needs to be a decision if we are going to use ProxyAllocator or > per-thread allocator in ink_freelist. Adding the ability to specify the > size of the ProxyAllocator would be trivial to add. One way to determine > what one to use is to benchmark them both and see what one is faster. > > Have you done any allocation benchmarks? Does the pre-thread > implementation in the ink_freelist use atomics and how is it getting the > thread id? (haven't looked at the code much for it yet) > > I'm outside now, please search mailing list with 'reclaimable freelist' keyword, I have given some articles links about the usage and design of reclaimable freelist in google doc and apache wiki. > -Bryan > > > > On Feb 3, 2013, at 7:28 PM, Yunkai Zhang > > wrote: > > > Hi Bryan, Leif, weijin: > > > > 1) There is a disadvantage of using ProxyAllocator to manage memory: It's > > hard to configure the size of each ProxyAllocator pool, unsuitable size > may > > cause memory wast or bad performance. > > > > 2) The reclaimable freelist we pushed recently is implemented as > > thread-local pool, feel free to try it. > > > > > > On Mon, Feb 4, 2013 at 10:08 AM, weijin (JIRA) > > > > wrote: > > > >> > >>[ > >> > https://issues.apache.org/jira/browse/TS-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569953#comment-13569953 > ] > >> > >> weijin commented on TS-1684: > >> > >> > >> good idea, the call ink_freelist_new/ink_freelist_free can be > >> significantly reduced by making the ProxyAllocator as cache of global > >> freelist. > >> > >>> Reduce the usage of global allocation/free lists - switch to using > local > >> thread allocation/free lists > >>> > >> > - > >>> > >>>Key: TS-1684 > >>>URL: https://issues.apache.org/jira/browse/TS-1684 > >>>Project: Traffic Server > >>> Issue Type: Improvement > >>> Components: Core > >>> Reporter: Bryan Call > >>> > >>> When running benchmarks ink_freelist_new() normally shows up as one of > >> if not the number one function in the code using the most CPU. > Currently > >> ATS uses global free lists (via ClassAllocator<>, Allocator, and > >> SparceClassAllocator<>) for memory allocation for some of its memory > >> allocation. > >>> Here is a list of how frequently the type of allocations are used and > >> the "name" given to the allocator. This is a benchmark for a small > object > >> in cache fetched 100k times. > >>> 40 ink_freelist_new: hdrHeap > >>> 30 ink_freelist_new: hdrStrHeap > >>> 203541 ink_freelist_new: ioBlockAllocator > >>> 199616 proxy allocator thread_alloc: eventAllocator > >>> 103554 ink_freelist_new: ioDataAllocator > >>> 103554 ink_freelist_new: ioBufAllocator[5] > >>> 100100 ink_freelist_new: ioAllocator > >>> 10 proxy allocator thread_alloc: hdrHeap > >>> 10 proxy allocator thread_alloc: cacheVConnection > >>> 10 ink_freelist_new: httpSMAllocator > >>> 10 ink_freelist_new: ArenaBlock > >>> 18507 ink_freelist_new: mutexAllocator > >>> 4772 ink_freelist_new: eventAllocator > >>>162 ink_freelist_new: cacheVConnection > >>>102 ink_freelist_new: netVCAllocator > >>>100 proxy allocator init thread_alloc: httpClientSessionAllocator > >>>100 ink_freelist_new: httpClientSessionAllocator > >>> 1 proxy allocator thread_alloc: RamCacheCLFUSEntry > >>> 1 ink_freelist_new: RamCacheCLFUSEntry > >>> 1 ink_freelist_new: hostDBContAllocator > >> > >> -- > >> This message is automatically generated by JIRA. > >> If you think it was sent incorrectly, please contact your JIRA > >> administrators > >> For more information on JIRA, see: > http://www.atlassian.com/software/jira > >> > > > > > > > > -- > > Yunkai Zhang > > Work at Taobao > > -- Yunkai Zhang Work at Taobao
[jira] [Created] (TS-1699) document proxy.config.http.doc_in_cache_skip_dns
James Peach created TS-1699: --- Summary: document proxy.config.http.doc_in_cache_skip_dns Key: TS-1699 URL: https://issues.apache.org/jira/browse/TS-1699 Project: Traffic Server Issue Type: Bug Components: Documentation Reporter: James Peach Authentication plugins typically attache to the OS_DNS_HOOK, but if they do then they need to set proxy.config.http.doc_in_cache_skip_dns=0 so that the DNS hook gets called on every request. We should document this parameter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (TS-1684) Reduce the usage of global allocation/free lists - switch to using local thread allocation/free lists
I was briefly looking at the changes today. I will take a look more at them when I get some time. There needs to be a decision if we are going to use ProxyAllocator or per-thread allocator in ink_freelist. Adding the ability to specify the size of the ProxyAllocator would be trivial to add. One way to determine what one to use is to benchmark them both and see what one is faster. Have you done any allocation benchmarks? Does the pre-thread implementation in the ink_freelist use atomics and how is it getting the thread id? (haven't looked at the code much for it yet) -Bryan On Feb 3, 2013, at 7:28 PM, Yunkai Zhang wrote: > Hi Bryan, Leif, weijin: > > 1) There is a disadvantage of using ProxyAllocator to manage memory: It's > hard to configure the size of each ProxyAllocator pool, unsuitable size may > cause memory wast or bad performance. > > 2) The reclaimable freelist we pushed recently is implemented as > thread-local pool, feel free to try it. > > > On Mon, Feb 4, 2013 at 10:08 AM, weijin (JIRA) wrote: > >> >>[ >> https://issues.apache.org/jira/browse/TS-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569953#comment-13569953] >> >> weijin commented on TS-1684: >> >> >> good idea, the call ink_freelist_new/ink_freelist_free can be >> significantly reduced by making the ProxyAllocator as cache of global >> freelist. >> >>> Reduce the usage of global allocation/free lists - switch to using local >> thread allocation/free lists >>> >> - >>> >>>Key: TS-1684 >>>URL: https://issues.apache.org/jira/browse/TS-1684 >>>Project: Traffic Server >>> Issue Type: Improvement >>> Components: Core >>> Reporter: Bryan Call >>> >>> When running benchmarks ink_freelist_new() normally shows up as one of >> if not the number one function in the code using the most CPU. Currently >> ATS uses global free lists (via ClassAllocator<>, Allocator, and >> SparceClassAllocator<>) for memory allocation for some of its memory >> allocation. >>> Here is a list of how frequently the type of allocations are used and >> the "name" given to the allocator. This is a benchmark for a small object >> in cache fetched 100k times. >>> 40 ink_freelist_new: hdrHeap >>> 30 ink_freelist_new: hdrStrHeap >>> 203541 ink_freelist_new: ioBlockAllocator >>> 199616 proxy allocator thread_alloc: eventAllocator >>> 103554 ink_freelist_new: ioDataAllocator >>> 103554 ink_freelist_new: ioBufAllocator[5] >>> 100100 ink_freelist_new: ioAllocator >>> 10 proxy allocator thread_alloc: hdrHeap >>> 10 proxy allocator thread_alloc: cacheVConnection >>> 10 ink_freelist_new: httpSMAllocator >>> 10 ink_freelist_new: ArenaBlock >>> 18507 ink_freelist_new: mutexAllocator >>> 4772 ink_freelist_new: eventAllocator >>>162 ink_freelist_new: cacheVConnection >>>102 ink_freelist_new: netVCAllocator >>>100 proxy allocator init thread_alloc: httpClientSessionAllocator >>>100 ink_freelist_new: httpClientSessionAllocator >>> 1 proxy allocator thread_alloc: RamCacheCLFUSEntry >>> 1 ink_freelist_new: RamCacheCLFUSEntry >>> 1 ink_freelist_new: hostDBContAllocator >> >> -- >> This message is automatically generated by JIRA. >> If you think it was sent incorrectly, please contact your JIRA >> administrators >> For more information on JIRA, see: http://www.atlassian.com/software/jira >> > > > > -- > Yunkai Zhang > Work at Taobao
[jira] [Commented] (TS-1382) jtest is not 64bit nice?
[
https://issues.apache.org/jira/browse/TS-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572570#comment-13572570
]
James Peach commented on TS-1382:
-
Is there any more work to be done here?
> jtest is not 64bit nice?
>
>
> Key: TS-1382
> URL: https://issues.apache.org/jira/browse/TS-1382
> Project: Traffic Server
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Zhao Yongming
> Fix For: 3.3.1
>
>
> {code}
> [root@test223 ~]# jtest -S ts.cn -P 115.238.23.222
> gethostbyname: Success
> con newops 1byte lat bytes/per svrs new opstotal time
> err
> 100 200.0 0 0 0/0 000 15224.1
> 0
> 100 221 1056.61516 15755542/157555 633 633 632 262114874.6
>0
> 100 528 2605.11314 41197350/411973 1559 1559 1558 667285705.6
>0
> 100 831 4131.11113 65919865/659198 2482 2482 2481 105312201
> 6.60
> 100 1117 5571.11012 87058878/870588 3342 3342 3341 139266339
> 7.60
> 100 1268 6325.8 911 98363651/983636 3787 3787 3787 -14692543
> 8.60
> 100 1389 6937.1 810 106898027/1068980 4162 4162 4161 33783077
> 9.60
> 100 1505 7519.1 710 114697045/1146970 4496 4496 4496 73836162
> 10.60
> 100 1663 8308.2 6 9 -44317582/-443175 4982 4982 4981 116560233
> 11.60
> 100 1770 8846.0 6 9 -1355348/-13553 5304 5304 5304 -23180093
> 12.60
> 100 1818 9086.4 6 9 30846878/308468 5436 5436 5437 31424752 13.6
>0
> 100 1799 8999.7 6 9 48851217/488512 5393 5393 5393 65393728 14.6
>0
> 100 1851 9258.2 6 9 69758095/697580 5560 5560 5560 100471603
> 15.60
> 100 1813 9063.9 6 9 80264043/802640 5453 5453 5452 118504186
> 16.60
> 100 1824 9117.8 6 9 94174620/941746 5484 5484 5484 -29646681
> 17.60
> 100 1836 9185.0 6 9 104651080/1046510 5530 5530 5530 22200325
> 18.60
> 100 1860 9303.6 6 9 111025678/1110256 5592 5592 5591 62519520
> 19.60
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-1006) memory management, cut down memory waste ?
[
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572367#comment-13572367
]
ASF subversion and git services commented on TS-1006:
-
Commit 616203849743e32d446f09a9089da29116982947 from Zhao Yongming
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=6162038 ]
TS-1006: Remove unused InkFreeList->offset
I have simplified ink_freelist_init API by removing offset parameter,
but I forgot to remove unused InkFreeList->offset when rebasing code.
So now, the code using an uninitialized value, it would be a bug. Let's
clean it completely.
Signed-off-by: Yunkai Zhang
Signed-off-by: Zhao Yongming
> memory management, cut down memory waste ?
> --
>
> Key: TS-1006
> URL: https://issues.apache.org/jira/browse/TS-1006
> Project: Traffic Server
> Issue Type: Improvement
> Components: Core
>Affects Versions: 3.1.1
>Reporter: Zhao Yongming
>Assignee: Bin Chen
> Fix For: 3.3.2
>
> Attachments:
> 0001-TS-1006-Add-an-enable-reclaimable-freelist-option.patch,
> 0002-TS-1006-Add-a-new-wrapper-ink_atomic_decrement.patch,
> 0003-TS-1006-Introduce-a-reclaimable-InkFreeList.patch,
> 0004-TS-1006-Make-InkFreeList-memory-pool-configurable.patch,
> Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods
>
>
> when we review the memory usage in the production, there is something
> abnormal, ie, looks like TS take much memory than index data + common system
> waste, and here is some memory dump result by set
> "proxy.config.dump_mem_info_frequency"
> 1, the one on a not so busy forwarding system:
> physics memory: 32G
> RAM cache: 22G
> DISK: 6140 GB
> average_object_size 64000
> {code}
> allocated |in-use | type size | free list name
> |||--
> 671088640 | 37748736 |2097152 |
> memory/ioBufAllocator[14]
> 2248146944 | 2135949312 |1048576 |
> memory/ioBufAllocator[13]
> 1711276032 | 1705508864 | 524288 |
> memory/ioBufAllocator[12]
> 1669332992 | 1667760128 | 262144 |
> memory/ioBufAllocator[11]
> 2214592512 | 221184 | 131072 |
> memory/ioBufAllocator[10]
> 2325741568 | 2323775488 | 65536 |
> memory/ioBufAllocator[9]
> 2091909120 | 2089123840 | 32768 |
> memory/ioBufAllocator[8]
> 1956642816 | 1956478976 | 16384 |
> memory/ioBufAllocator[7]
> 2094530560 | 2094071808 | 8192 |
> memory/ioBufAllocator[6]
> 356515840 | 355540992 | 4096 |
> memory/ioBufAllocator[5]
> 1048576 | 14336 | 2048 |
> memory/ioBufAllocator[4]
> 131072 | 0 | 1024 |
> memory/ioBufAllocator[3]
> 65536 | 0 |512 |
> memory/ioBufAllocator[2]
> 32768 | 0 |256 |
> memory/ioBufAllocator[1]
> 16384 | 0 |128 |
> memory/ioBufAllocator[0]
> 0 | 0 |576 |
> memory/ICPRequestCont_allocator
> 0 | 0 |112 |
> memory/ICPPeerReadContAllocator
> 0 | 0 |432 |
> memory/PeerReadDataAllocator
> 0 | 0 | 32 |
> memory/MIMEFieldSDKHandle
> 0 | 0 |240 |
> memory/INKVConnAllocator
> 0 | 0 | 96 |
> memory/INKContAllocator
>4096 | 0 | 32 |
> memory/apiHookAllocator
> 0 | 0 |288 |
> memory/FetchSMAllocator
> 0 | 0 | 80 |
> memory/prefetchLockHandlerAllocator
> 0 | 0 |176 |
> memory/PrefetchBlasterAllocator
> 0 | 0 | 80 |
> memory/prefetchUrlBlaster
> 0 | 0 | 96 | memory/blasterUrlList
> 0 | 0 | 96 |
> memory/prefetchUrlEntryAllocator
> 0 | 0 |128 |
> memory/socksProxyAllocator
> 0 | 0 |144 |
> memory/ObjectReloadCont
> 3258368 | 576016 |592 |
> memory/httpClientSessionAllocator
> 825344 | 139568 |208 |
> memory/httpServerSessionAllocator
>22597632 |1284848 | 9808 | memory/httpSM
[jira] [Commented] (TS-1626) remove WUTS proxy code
[ https://issues.apache.org/jira/browse/TS-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572344#comment-13572344 ] Uri Shachar commented on TS-1626: - I've dug around code from ~2006 -- haven't been able to find any useful info on this (It does seem to be related to the log_spider_codes option). Maybe John Plevyak knows what it's all about? > remove WUTS proxy code > -- > > Key: TS-1626 > URL: https://issues.apache.org/jira/browse/TS-1626 > Project: Traffic Server > Issue Type: Improvement > Components: Core >Reporter: James Peach > > There is a bunch of code doing something with "WUTS proxy status codes". What > is a WUTS and why do we care? If this is cruft, let's get rid of it! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-1698) thread/pthread mis-match on Solaris 10
[
https://issues.apache.org/jira/browse/TS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572297#comment-13572297
]
Igor Galić commented on TS-1698:
These comments out of order seem.
> thread/pthread mis-match on Solaris 10
> --
>
> Key: TS-1698
> URL: https://issues.apache.org/jira/browse/TS-1698
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
>Reporter: Igor Galić
> Attachments: ink_pthead.patch
>
>
> {noformat}
> CXX UnixEventProcessor.o
> UnixEventProcessor.cc: In member function 'virtual int
> EventProcessor::start(int)':
> UnixEventProcessor.cc:186:7: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> UnixEventProcessor.cc:188:9: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> cc1plus: all warnings being treated as errors
> gmake[2]: *** [UnixEventProcessor.o] Error 1
> gmake[2]: Leaving directory
> `/export/home/buildbot/slave5/tserver-solaris-trunk-debug/build/iocore/eventsystem'
> gmake[1]: *** [all-recursive] Error 1
> {noformat}
> This seems to be caused by our naïve code to distinguish between different
> platforms:
> {code}
> #include "P_EventSystem.h" /* MAGIC_EDITING_TAG */
> #include
> #if TS_USE_HWLOC
> #if HAVE_HWLOC_H
> #include
> #endif
> #if HAVE_SCHED_H
> #include
> #if !defined(solaris) && !defined(freebsd)
> typedef cpu_set_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #endif
> #if HAVE_SYS_PARAM_H
> #include
> #endif
> #if HAVE_SYS_CPUSET_H
> #include
> typedef cpuset_t ink_cpuset_t;
> #define PTR_FMT "p"
> #endif
> #if HAVE_PTHREAD_NP_H
> #include
> #endif
> #if HAVE_SYS_PSET_H
> #include
> typedef psetid_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #if HAVE_SYS_TYPES_H
> #include
> #endif
> #endif
> #include "ink_defs.h"
> #if TS_USE_HWLOC
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-1698) thread/pthread mis-match on Solaris 10
[
https://issues.apache.org/jira/browse/TS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572296#comment-13572296
]
Igor Galić commented on TS-1698:
I think we should move {{PTR_FMT}} to the source: {{ink_thread.h}}.
> thread/pthread mis-match on Solaris 10
> --
>
> Key: TS-1698
> URL: https://issues.apache.org/jira/browse/TS-1698
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
>Reporter: Igor Galić
> Attachments: ink_pthead.patch
>
>
> {noformat}
> CXX UnixEventProcessor.o
> UnixEventProcessor.cc: In member function 'virtual int
> EventProcessor::start(int)':
> UnixEventProcessor.cc:186:7: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> UnixEventProcessor.cc:188:9: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> cc1plus: all warnings being treated as errors
> gmake[2]: *** [UnixEventProcessor.o] Error 1
> gmake[2]: Leaving directory
> `/export/home/buildbot/slave5/tserver-solaris-trunk-debug/build/iocore/eventsystem'
> gmake[1]: *** [all-recursive] Error 1
> {noformat}
> This seems to be caused by our naïve code to distinguish between different
> platforms:
> {code}
> #include "P_EventSystem.h" /* MAGIC_EDITING_TAG */
> #include
> #if TS_USE_HWLOC
> #if HAVE_HWLOC_H
> #include
> #endif
> #if HAVE_SCHED_H
> #include
> #if !defined(solaris) && !defined(freebsd)
> typedef cpu_set_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #endif
> #if HAVE_SYS_PARAM_H
> #include
> #endif
> #if HAVE_SYS_CPUSET_H
> #include
> typedef cpuset_t ink_cpuset_t;
> #define PTR_FMT "p"
> #endif
> #if HAVE_PTHREAD_NP_H
> #include
> #endif
> #if HAVE_SYS_PSET_H
> #include
> typedef psetid_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #if HAVE_SYS_TYPES_H
> #include
> #endif
> #endif
> #include "ink_defs.h"
> #if TS_USE_HWLOC
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1698) thread/pthread mis-match on Solaris 10
[
https://issues.apache.org/jira/browse/TS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Galić updated TS-1698:
---
Attachment: ink_pthead.patch
This is a preliminary cleanup: Step one FreeBSD supports these functions. If
there's a platform out there that does not, this is a saner way to catch it.
> thread/pthread mis-match on Solaris 10
> --
>
> Key: TS-1698
> URL: https://issues.apache.org/jira/browse/TS-1698
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
>Reporter: Igor Galić
> Attachments: ink_pthead.patch
>
>
> {noformat}
> CXX UnixEventProcessor.o
> UnixEventProcessor.cc: In member function 'virtual int
> EventProcessor::start(int)':
> UnixEventProcessor.cc:186:7: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> UnixEventProcessor.cc:188:9: error: format '%lu' expects argument of type
> 'long unsigned int', but argument 8 has type 'ink_thread {aka unsigned int}'
> [-Werror=format]
> cc1plus: all warnings being treated as errors
> gmake[2]: *** [UnixEventProcessor.o] Error 1
> gmake[2]: Leaving directory
> `/export/home/buildbot/slave5/tserver-solaris-trunk-debug/build/iocore/eventsystem'
> gmake[1]: *** [all-recursive] Error 1
> {noformat}
> This seems to be caused by our naïve code to distinguish between different
> platforms:
> {code}
> #include "P_EventSystem.h" /* MAGIC_EDITING_TAG */
> #include
> #if TS_USE_HWLOC
> #if HAVE_HWLOC_H
> #include
> #endif
> #if HAVE_SCHED_H
> #include
> #if !defined(solaris) && !defined(freebsd)
> typedef cpu_set_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #endif
> #if HAVE_SYS_PARAM_H
> #include
> #endif
> #if HAVE_SYS_CPUSET_H
> #include
> typedef cpuset_t ink_cpuset_t;
> #define PTR_FMT "p"
> #endif
> #if HAVE_PTHREAD_NP_H
> #include
> #endif
> #if HAVE_SYS_PSET_H
> #include
> typedef psetid_t ink_cpuset_t;
> #define PTR_FMT PRIuPTR
> #endif
> #if HAVE_SYS_TYPES_H
> #include
> #endif
> #endif
> #include "ink_defs.h"
> #if TS_USE_HWLOC
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-1006) memory management, cut down memory waste ?
[
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572288#comment-13572288
]
ASF subversion and git services commented on TS-1006:
-
Commit 61d0d2c11de9c7c8fe7822438c82deb2848ad107 from Zhao Yongming
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=61d0d2c ]
TS-1006: Fix double-linked list initialization in reclaimable_freelist_new
This bug broke the double-linked list which would prevent self-thread
"stealing" memory from other threads. It was found by Weijin when he
reviewed the code, thans a lot!
Signed-off-by: Yunkai Zhang
Signed-off-by: Zhao Yongming
> memory management, cut down memory waste ?
> --
>
> Key: TS-1006
> URL: https://issues.apache.org/jira/browse/TS-1006
> Project: Traffic Server
> Issue Type: Improvement
> Components: Core
>Affects Versions: 3.1.1
>Reporter: Zhao Yongming
>Assignee: Bin Chen
> Fix For: 3.3.2
>
> Attachments:
> 0001-TS-1006-Add-an-enable-reclaimable-freelist-option.patch,
> 0002-TS-1006-Add-a-new-wrapper-ink_atomic_decrement.patch,
> 0003-TS-1006-Introduce-a-reclaimable-InkFreeList.patch,
> 0004-TS-1006-Make-InkFreeList-memory-pool-configurable.patch,
> Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods
>
>
> when we review the memory usage in the production, there is something
> abnormal, ie, looks like TS take much memory than index data + common system
> waste, and here is some memory dump result by set
> "proxy.config.dump_mem_info_frequency"
> 1, the one on a not so busy forwarding system:
> physics memory: 32G
> RAM cache: 22G
> DISK: 6140 GB
> average_object_size 64000
> {code}
> allocated |in-use | type size | free list name
> |||--
> 671088640 | 37748736 |2097152 |
> memory/ioBufAllocator[14]
> 2248146944 | 2135949312 |1048576 |
> memory/ioBufAllocator[13]
> 1711276032 | 1705508864 | 524288 |
> memory/ioBufAllocator[12]
> 1669332992 | 1667760128 | 262144 |
> memory/ioBufAllocator[11]
> 2214592512 | 221184 | 131072 |
> memory/ioBufAllocator[10]
> 2325741568 | 2323775488 | 65536 |
> memory/ioBufAllocator[9]
> 2091909120 | 2089123840 | 32768 |
> memory/ioBufAllocator[8]
> 1956642816 | 1956478976 | 16384 |
> memory/ioBufAllocator[7]
> 2094530560 | 2094071808 | 8192 |
> memory/ioBufAllocator[6]
> 356515840 | 355540992 | 4096 |
> memory/ioBufAllocator[5]
> 1048576 | 14336 | 2048 |
> memory/ioBufAllocator[4]
> 131072 | 0 | 1024 |
> memory/ioBufAllocator[3]
> 65536 | 0 |512 |
> memory/ioBufAllocator[2]
> 32768 | 0 |256 |
> memory/ioBufAllocator[1]
> 16384 | 0 |128 |
> memory/ioBufAllocator[0]
> 0 | 0 |576 |
> memory/ICPRequestCont_allocator
> 0 | 0 |112 |
> memory/ICPPeerReadContAllocator
> 0 | 0 |432 |
> memory/PeerReadDataAllocator
> 0 | 0 | 32 |
> memory/MIMEFieldSDKHandle
> 0 | 0 |240 |
> memory/INKVConnAllocator
> 0 | 0 | 96 |
> memory/INKContAllocator
>4096 | 0 | 32 |
> memory/apiHookAllocator
> 0 | 0 |288 |
> memory/FetchSMAllocator
> 0 | 0 | 80 |
> memory/prefetchLockHandlerAllocator
> 0 | 0 |176 |
> memory/PrefetchBlasterAllocator
> 0 | 0 | 80 |
> memory/prefetchUrlBlaster
> 0 | 0 | 96 | memory/blasterUrlList
> 0 | 0 | 96 |
> memory/prefetchUrlEntryAllocator
> 0 | 0 |128 |
> memory/socksProxyAllocator
> 0 | 0 |144 |
> memory/ObjectReloadCont
> 3258368 | 576016 |592 |
> memory/httpClientSessionAllocator
> 825344 | 139568 |208 |
> memory/httpServerSessionAllocator
>22597632 |1284848 | 9808 | memory/httpSMAllocator
> 0
[jira] [Commented] (TS-1695) test_certlookup fails on FreeBSD
[
https://issues.apache.org/jira/browse/TS-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572278#comment-13572278
]
Igor Galić commented on TS-1695:
Some further digging: If I --disable-freelist {{test_certlookup}} still
segfaults:
{noformat}
igalic@daemonix ...che.trafficserver.81460/obj/iocore/net %
LD_LIBRARY_PATH="/tmp/org.apache.trafficserver.81460/obj/lib/ts/.libs:/usr/local/lib:$LD_LIBRARY_PATH"
gdb .libs/test_certlookup ./test_certlookup.core
GNU gdb (GDB) 7.5.1 [GDB v7.5.1 for FreeBSD]
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd10.0".
For bug reporting instructions, please see:
[jira] [Commented] (TS-1653) Crash report: HostDBContinuation::probeEvent MutexTryLock
[
https://issues.apache.org/jira/browse/TS-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572268#comment-13572268
]
weijin commented on TS-1653:
committed bc7a84441ca1d6c13d1117b9515c879c70cf9d3b
> Crash report: HostDBContinuation::probeEvent MutexTryLock
> -
>
> Key: TS-1653
> URL: https://issues.apache.org/jira/browse/TS-1653
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
>Reporter: Zhao Yongming
>Assignee: Alan M. Carroll
>
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 8080:fd=7'.
> Program terminated with signal 11, Segmentation fault.
> #0 Mutex_trylock (this=0x2b994a800ce0, am=0x0, t=0x2b9948fea010) at
> ../iocore/eventsystem/I_Lock.h:170
> 170 if (m->thread_holding != t) {
> Missing separate debuginfos, use: debuginfo-install
> expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.47.el6_2.9.x86_64
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
> openssl-1.0.0-20.el6_2.4.x86_64 pcre-7.8-3.1.el6.x86_64
> tcl-8.5.7-6.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64
> zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0 Mutex_trylock (this=0x2b994a800ce0, am=0x0, t=0x2b9948fea010) at
> ../iocore/eventsystem/I_Lock.h:170
> #1 MutexTryLock::MutexTryLock (this=0x2b994a800ce0, am=0x0,
> t=0x2b9948fea010) at ../iocore/eventsystem/I_Lock.h:385
> #2 0x005b38dc in HostDBContinuation::probeEvent
> (this=0x2b997329df10, event=, e=0x2b99b8050030) at
> HostDB.cc:1762
> #3 0x0065a1b4 in handleEvent (this=0x2b9948fea010, e=0x2b99b8050030,
> calling_code=2) at I_Continuation.h:146
> #4 EThread::process_event (this=0x2b9948fea010, e=0x2b99b8050030,
> calling_code=2) at UnixEThread.cc:142
> #5 0x0065ad83 in EThread::execute (this=0x2b9948fea010) at
> UnixEThread.cc:221
> #6 0x006594d2 in spawn_thread_internal (a=0x2fb8350) at Thread.cc:88
> #7 0x003e878077f1 in start_thread () from /lib64/libpthread.so.0
> #8 0x003e86ce5ccd in clone () from /lib64/libc.so.6
> (gdb) f 2
> #2 0x005b38dc in HostDBContinuation::probeEvent
> (this=0x2b997329df10, event=, e=0x2b99b8050030) at
> HostDB.cc:1762
> 1762MUTEX_TRY_LOCK_FOR(lock, action.mutex, t, action.continuation);
> (gdb) p action.mutex
> $1 = {m_ptr = 0x0}
> (gdb) p t
> $2 =
> (gdb) p action
> $3 = {_vptr.Action = 0x6601d0, continuation = 0x0, mutex = {m_ptr = 0x0},
> cancelled = 1}
> (gdb)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-1653) Crash report: HostDBContinuation::probeEvent MutexTryLock
[
https://issues.apache.org/jira/browse/TS-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572266#comment-13572266
]
ASF subversion and git services commented on TS-1653:
-
Commit bc7a84441ca1d6c13d1117b9515c879c70cf9d3b from weijin
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=bc7a844 ]
TS-1653 we should not retry dns lookup after timeout happens
> Crash report: HostDBContinuation::probeEvent MutexTryLock
> -
>
> Key: TS-1653
> URL: https://issues.apache.org/jira/browse/TS-1653
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
>Reporter: Zhao Yongming
>Assignee: Alan M. Carroll
>
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 8080:fd=7'.
> Program terminated with signal 11, Segmentation fault.
> #0 Mutex_trylock (this=0x2b994a800ce0, am=0x0, t=0x2b9948fea010) at
> ../iocore/eventsystem/I_Lock.h:170
> 170 if (m->thread_holding != t) {
> Missing separate debuginfos, use: debuginfo-install
> expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.47.el6_2.9.x86_64
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
> openssl-1.0.0-20.el6_2.4.x86_64 pcre-7.8-3.1.el6.x86_64
> tcl-8.5.7-6.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64
> zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0 Mutex_trylock (this=0x2b994a800ce0, am=0x0, t=0x2b9948fea010) at
> ../iocore/eventsystem/I_Lock.h:170
> #1 MutexTryLock::MutexTryLock (this=0x2b994a800ce0, am=0x0,
> t=0x2b9948fea010) at ../iocore/eventsystem/I_Lock.h:385
> #2 0x005b38dc in HostDBContinuation::probeEvent
> (this=0x2b997329df10, event=, e=0x2b99b8050030) at
> HostDB.cc:1762
> #3 0x0065a1b4 in handleEvent (this=0x2b9948fea010, e=0x2b99b8050030,
> calling_code=2) at I_Continuation.h:146
> #4 EThread::process_event (this=0x2b9948fea010, e=0x2b99b8050030,
> calling_code=2) at UnixEThread.cc:142
> #5 0x0065ad83 in EThread::execute (this=0x2b9948fea010) at
> UnixEThread.cc:221
> #6 0x006594d2 in spawn_thread_internal (a=0x2fb8350) at Thread.cc:88
> #7 0x003e878077f1 in start_thread () from /lib64/libpthread.so.0
> #8 0x003e86ce5ccd in clone () from /lib64/libc.so.6
> (gdb) f 2
> #2 0x005b38dc in HostDBContinuation::probeEvent
> (this=0x2b997329df10, event=, e=0x2b99b8050030) at
> HostDB.cc:1762
> 1762MUTEX_TRY_LOCK_FOR(lock, action.mutex, t, action.continuation);
> (gdb) p action.mutex
> $1 = {m_ptr = 0x0}
> (gdb) p t
> $2 =
> (gdb) p action
> $3 = {_vptr.Action = 0x6601d0, continuation = 0x0, mutex = {m_ptr = 0x0},
> cancelled = 1}
> (gdb)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
