[jira] [Commented] (TS-1633) add new stat sync type: TS_STAT_SYNC_MAX
[ https://issues.apache.org/jira/browse/TS-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539005#comment-13539005 ] Yakov Kopel commented on TS-1633: - On second thought, there is a problem with this patch. The max updating is done only when the sync function is activated, so it not really max. Maybe the right way is rewrite the stats handling and moving to atomic variables, so we don't need to use thread summarize (sync) function. using atomic variable for stats that is shared between all the net thread can be good idea. It can fix another issue that stat can be handled only in net-threads and not in another threads. add new stat sync type: TS_STAT_SYNC_MAX Key: TS-1633 URL: https://issues.apache.org/jira/browse/TS-1633 Project: Traffic Server Issue Type: Bug Components: Stats Reporter: Yakov Kopel Attachments: stat_max_ver_1.diff Adding a new stat sync type - max. This is a good type of stat to find peaks in the stats. It can be very useful with the stat clear api (after the fix - TS-1631). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (TS-1528) ats_memalign: couldn't allocate -548249600 bytes in Vol::init()
[
https://issues.apache.org/jira/browse/TS-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao Yongming reopened TS-1528:
---
Assignee: Zhao Yongming (was: James Peach)
from what I can get, all the information in codes, the MAX_ in
P_CacheVol.h, does information us that we do not have a 4G limit here, we
should fix the 4G limit, or fix the MAX_xxx or make a work around, or at least
tell the user what is the root cause hare, not just crashing with some strange
message.
ats_memalign: couldn't allocate -548249600 bytes in Vol::init()
---
Key: TS-1528
URL: https://issues.apache.org/jira/browse/TS-1528
Project: Traffic Server
Issue Type: Bug
Affects Versions: 3.2.0
Environment: Debian testing (wheezy) on i686
Reporter: Jack Bates
Assignee: Zhao Yongming
Fix For: 3.3.3
I consistently get the following error whenever I try to start Traffic Server
(release 3.2.0). Yesterday I built Traffic Server from Git HEAD (34a2ba) to
check if it behaves any differently, but I consistently reproduce this same
error whenever I try to start it, too
Here's my configuration, which is pretty minimal:
http://nottheoilrig.com/trafficserver/201210120/
What details can I provide to help debug this? James Peach suggested
attaching some kind of dump of the volume header:
http://mail-archives.apache.org/mod_mbox/trafficserver-users/201210.mbox/%3C9ED91AE2-2F52-4BDB-9088-E14D40642C34%40apache.org%3E
{code}
administrator@debian$ TS_ROOT=/home/administrator/trafficserver
trafficserver/traffic_server
[TrafficServer] using root directory '/home/administrator/trafficserver'
FATAL: ats_memalign: couldn't allocate -548249600 bytes at alignment 4096 -
insufficient memory
trafficserver/traffic_server - STACK TRACE:
trafficserver/libtsutil.so.3(+0x1075b)[0xb76d075b]
trafficserver/libtsutil.so.3(ats_memalign+0xa1)[0xb76d34c1]
trafficserver/traffic_server(_ZN3Vol4initEPcxxb+0x282)[0x827bd52]
trafficserver/traffic_server(_ZN5Cache4openEbb+0x5d8)[0x827dc48]
trafficserver/traffic_server(_ZN14CacheProcessor15diskInitializedEv+0x323)[0x827e0d3]
trafficserver/traffic_server(_ZN9CacheDisk9openStartEiPv+0x483)[0x828c9c3]
trafficserver/traffic_server(_ZN19AIOCallbackInternal11io_completeEiPv+0x25)[0x8280a75]
trafficserver/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8b)[0x830343b]
trafficserver/traffic_server(_ZN7EThread7executeEv+0x723)[0x8304003]
trafficserver/traffic_server(main+0x178d)[0x80c572d]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7039e46]
trafficserver/traffic_server[0x80cabdd]
administrator@debian:~$
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (TS-1577) Crash report: RangeTransform::change_response_header at Transform.cc:995
[
https://issues.apache.org/jira/browse/TS-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
weijin reassigned TS-1577:
--
Assignee: weijin (was: Zhao Yongming)
Crash report: RangeTransform::change_response_header at Transform.cc:995
Key: TS-1577
URL: https://issues.apache.org/jira/browse/TS-1577
Project: Traffic Server
Issue Type: Bug
Components: HTTP
Affects Versions: 3.3.0
Environment: git master version
Reporter: Zhao Yongming
Assignee: weijin
Priority: Critical
Fix For: 3.3.1
Attachments: ts-1574.diff
This may or may not relate to TS-1574, I'd like track this issue another
thread here.
{code}
Core was generated by `/usr/bin/traffic_server -M --httpport 8080:fd=12'.
Program terminated with signal 6, Aborted.
#0 0x003e86c32885 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.47.el6_2.9.x86_64
keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64
libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64
libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
openssl-1.0.0-20.el6_2.4.x86_64 pcre-7.8-3.1.el6.x86_64
tcl-8.5.7-6.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64
zlib-1.2.3-27.el6.x86_64
(gdb) bt
#0 0x003e86c32885 in raise () from /lib64/libc.so.6
#1 0x003e86c34065 in abort () from /lib64/libc.so.6
#2 0x0035c8013f19 in ink_die_die_die (retval=30342) at ink_error.cc:43
#3 0x0035c8014194 in ink_fatal_va(int, const char *, typedef
__va_list_tag __va_list_tag *) (return_code=1,
message_format=value optimized out, ap=0x2b0b8fcc3ba0) at
ink_error.cc:65
#4 0x0035c80142c8 in ink_fatal (return_code=30342, message_format=0x7693
Address 0x7693 out of bounds) at ink_error.cc:73
#5 0x0035c8012c3f in _ink_assert (expression=0x0, file=0x6 Address 0x6
out of bounds, line=-1) at ink_assert.cc:38
#6 0x004d671c in RangeTransform::change_response_header
(this=0x2b0be4500bf0) at Transform.cc:995
#7 0x004da4cd in RangeTransform::handle_event (this=0x2b0be4500bf0,
event=value optimized out, edata=value optimized out)
at Transform.cc:791
#8 0x00654dd4 in handleEvent (this=0x2b0b8e4ad010, e=0x34aff40,
calling_code=1) at I_Continuation.h:146
#9 EThread::process_event (this=0x2b0b8e4ad010, e=0x34aff40, calling_code=1)
at UnixEThread.cc:142
#10 0x0065593b in EThread::execute (this=0x2b0b8e4ad010) at
UnixEThread.cc:193
#11 0x006540d2 in spawn_thread_internal (a=0x2c79a50) at Thread.cc:88
#12 0x003e878077f1 in start_thread () from /lib64/libpthread.so.0
#13 0x003e86ce5ccd in clone () from /lib64/libc.so.6
(gdb) f 6
#6 0x004d671c in RangeTransform::change_response_header
(this=0x2b0be4500bf0) at Transform.cc:995
995
ink_release_assert(m_transform_resp-field_find(MIME_FIELD_CONTENT_RANGE,
MIME_LEN_CONTENT_RANGE) == NULL);
(gdb) p this
$1 = (RangeTransform * const) 0x2b0be4500bf0
(gdb) p *this
$2 = {INKVConnInternal = {INKContInternal = {DummyVConnection =
{VConnection = {Continuation = {force_VFPT_to_top = {
_vptr.force_VFPT_to_top = 0x667970}, handler = (int
(Continuation::*)(Continuation *, int,
void *)) 0x4da200 RangeTransform::handle_event(int, void*), mutex =
{m_ptr = 0x2b0bf8216110}, link = {SLinkContinuation = {next = 0x0},
prev = 0x0}}, lerrno = 0}, No data fields}, mdata = 0x0,
m_event_func = 0, m_event_count = 0, m_closed = 0, m_deletable = 0,
m_deleted = 0, m_free_magic = INKCONT_INTERN_MAGIC_ALIVE}, m_read_vio =
{_cont = 0x0, nbytes = 0, ndone = 0, op = 0, buffer = {mbuf = 0x0,
entry = 0x0}, vc_server = 0x0, mutex = {m_ptr = 0x0}}, m_write_vio =
{_cont = 0x2b0f9168f020, nbytes = 20480, ndone = 0, op = 2, buffer = {
mbuf = 0x2b0c0407fdd0, entry = 0x2b0c0407fe10}, vc_server =
0x2b0be4500bf0, mutex = {m_ptr = 0x2b0bf8216110}},
m_output_vc = 0x2b0be459f048}, m_output_buf = 0x33b7ca0, m_output_reader
= 0x33b7cb8, m_transform_resp = 0x2b0f9168dc88,
m_output_vio = 0x2b0be459f0c8, m_unsatisfiable_range = false,
m_range_content_length = 0, m_num_chars_for_cl = 1, m_num_range_fields = -1,
m_current_range = 0,
m_content_type = 0x2b1216ea6ac0 application/octet-stream\r\nContent-Range:
bytes 0-20479/119091\r\nContent-Length: 20480\r\nConnection:
keep-alive\r\nDate: Sun, 18 Nov 2012 13:30:05 GMT\r\nServer:
apache\r\n\r\n\343k\031P, m_content_type_len = 24, m_ranges =
0x2b0be45bda90,
m_output_cl = 20480, m_done = 0}
(gdb) p m_ranges
$3 = (RangeRecord *) 0x2b0be45bda90
(gdb) p *m_ranges
$4 = {_start = 20480, _end = 40959, _done_byte = -1}
(gdb) p
[jira] [Commented] (TS-1006) memory management, cut down memory waste ?
[
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539187#comment-13539187
]
Yunkai Zhang commented on TS-1006:
--
[~genext]
It seems that mmap() failed to allocate memory. Was OS memory used up at that
time?
This patches was based on master branch, you should also apply it carefully.
I haven't test these on OpenSuse.
And, you can enable debug info by modify debug_filter's value(do you notice
it?):
##
#
# Configuration for InkFreeList memory allocator
#
##
# Dump debug information according bit mask of debug_filter, if a bit is set
# in the mask, then debug information of the corresponding action are dumped:
# bit 0: reclaim memory in ink_freelist_new
# bit 1: reclaim memory in ink_freelist_free
# bit 2: fetch memory from thread cache
CONFIG proxy.config.allocator.debug_filter INT 3
# The value of enable_reclaim should be 0 or 1. We can disable all reclaiming
# strategy by setting it to be 0.
CONFIG proxy.config.allocator.enable_reclaim INT 1
# The value of reclaim_factor should be in [0, 1], allocator use it to
# calculate average value of idle memory in InkFreeList, which will determine
# when to reclaim memory. The larger the value, the faster the reclaiming.
# This value is effective only when enable_reclaim is 1.
CONFIG proxy.config.allocator.reclaim_factor FLOAT 0.20
# Allocator will reclaim memory only when it continuously satisfy the reclaim
# condition for max_overage times. This value is effective only when
# enable_reclaim is 1.
CONFIG proxy.config.allocator.max_overage INT 100
memory management, cut down memory waste ?
--
Key: TS-1006
URL: https://issues.apache.org/jira/browse/TS-1006
Project: Traffic Server
Issue Type: Improvement
Components: Core
Affects Versions: 3.1.1
Reporter: Zhao Yongming
Assignee: Bin Chen
Fix For: 3.3.2
Attachments: 0001-Allocator-optimize-InkFreeList-memory-pool.patch,
0002-Allocator-make-InkFreeList-memory-pool-configurable.patch,
Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods
when we review the memory usage in the production, there is something
abnormal, ie, looks like TS take much memory than index data + common system
waste, and here is some memory dump result by set
proxy.config.dump_mem_info_frequency
1, the one on a not so busy forwarding system:
physics memory: 32G
RAM cache: 22G
DISK: 6140 GB
average_object_size 64000
{code}
allocated |in-use | type size | free list name
|||--
671088640 | 37748736 |2097152 |
memory/ioBufAllocator[14]
2248146944 | 2135949312 |1048576 |
memory/ioBufAllocator[13]
1711276032 | 1705508864 | 524288 |
memory/ioBufAllocator[12]
1669332992 | 1667760128 | 262144 |
memory/ioBufAllocator[11]
2214592512 | 221184 | 131072 |
memory/ioBufAllocator[10]
2325741568 | 2323775488 | 65536 |
memory/ioBufAllocator[9]
2091909120 | 2089123840 | 32768 |
memory/ioBufAllocator[8]
1956642816 | 1956478976 | 16384 |
memory/ioBufAllocator[7]
2094530560 | 2094071808 | 8192 |
memory/ioBufAllocator[6]
356515840 | 355540992 | 4096 |
memory/ioBufAllocator[5]
1048576 | 14336 | 2048 |
memory/ioBufAllocator[4]
131072 | 0 | 1024 |
memory/ioBufAllocator[3]
65536 | 0 |512 |
memory/ioBufAllocator[2]
32768 | 0 |256 |
memory/ioBufAllocator[1]
16384 | 0 |128 |
memory/ioBufAllocator[0]
0 | 0 |576 |
memory/ICPRequestCont_allocator
0 | 0 |112 |
memory/ICPPeerReadContAllocator
0 | 0 |432 |
memory/PeerReadDataAllocator
0 | 0 | 32 |
memory/MIMEFieldSDKHandle
0 | 0 |240 |
memory/INKVConnAllocator
0 | 0 | 96 |
memory/INKContAllocator
4096 | 0 | 32 |
memory/apiHookAllocator
0 | 0 |288 |
memory/FetchSMAllocator
0
[jira] [Commented] (TS-1006) memory management, cut down memory waste ?
[
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539212#comment-13539212
]
Yunkai Zhang commented on TS-1006:
--
[~genext]
By reading your log:
[2b59f1d13700:40][ink_queue.cc:00674][-] all:4015MB t:566 f:22 m:85 avg:84.6
malloc:544 csize:64 tsize:4096 cbsize:266240
The total memory used by InkFreeList allocator is 4015MB (including the
memory used by ram_cache).
FATAL: Failed to mmap 16781312 bytes, Cannot allocate memory
The patch try to mmap an aligned memory (align 16781312 to 2^n * PAGE_SIZE).
Suppose your OS's PAGE_SIZE is 4K, the aligned size should be: 2^13 * 4K =
2^25 = 32MB.
It seems that your OS can't malloc a continuous blocks of 32MB at that time?
What is the PAGE_SIZE on your OS?
memory management, cut down memory waste ?
--
Key: TS-1006
URL: https://issues.apache.org/jira/browse/TS-1006
Project: Traffic Server
Issue Type: Improvement
Components: Core
Affects Versions: 3.1.1
Reporter: Zhao Yongming
Assignee: Bin Chen
Fix For: 3.3.2
Attachments: 0001-Allocator-optimize-InkFreeList-memory-pool.patch,
0002-Allocator-make-InkFreeList-memory-pool-configurable.patch,
Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods
when we review the memory usage in the production, there is something
abnormal, ie, looks like TS take much memory than index data + common system
waste, and here is some memory dump result by set
proxy.config.dump_mem_info_frequency
1, the one on a not so busy forwarding system:
physics memory: 32G
RAM cache: 22G
DISK: 6140 GB
average_object_size 64000
{code}
allocated |in-use | type size | free list name
|||--
671088640 | 37748736 |2097152 |
memory/ioBufAllocator[14]
2248146944 | 2135949312 |1048576 |
memory/ioBufAllocator[13]
1711276032 | 1705508864 | 524288 |
memory/ioBufAllocator[12]
1669332992 | 1667760128 | 262144 |
memory/ioBufAllocator[11]
2214592512 | 221184 | 131072 |
memory/ioBufAllocator[10]
2325741568 | 2323775488 | 65536 |
memory/ioBufAllocator[9]
2091909120 | 2089123840 | 32768 |
memory/ioBufAllocator[8]
1956642816 | 1956478976 | 16384 |
memory/ioBufAllocator[7]
2094530560 | 2094071808 | 8192 |
memory/ioBufAllocator[6]
356515840 | 355540992 | 4096 |
memory/ioBufAllocator[5]
1048576 | 14336 | 2048 |
memory/ioBufAllocator[4]
131072 | 0 | 1024 |
memory/ioBufAllocator[3]
65536 | 0 |512 |
memory/ioBufAllocator[2]
32768 | 0 |256 |
memory/ioBufAllocator[1]
16384 | 0 |128 |
memory/ioBufAllocator[0]
0 | 0 |576 |
memory/ICPRequestCont_allocator
0 | 0 |112 |
memory/ICPPeerReadContAllocator
0 | 0 |432 |
memory/PeerReadDataAllocator
0 | 0 | 32 |
memory/MIMEFieldSDKHandle
0 | 0 |240 |
memory/INKVConnAllocator
0 | 0 | 96 |
memory/INKContAllocator
4096 | 0 | 32 |
memory/apiHookAllocator
0 | 0 |288 |
memory/FetchSMAllocator
0 | 0 | 80 |
memory/prefetchLockHandlerAllocator
0 | 0 |176 |
memory/PrefetchBlasterAllocator
0 | 0 | 80 |
memory/prefetchUrlBlaster
0 | 0 | 96 | memory/blasterUrlList
0 | 0 | 96 |
memory/prefetchUrlEntryAllocator
0 | 0 |128 |
memory/socksProxyAllocator
0 | 0 |144 |
memory/ObjectReloadCont
3258368 | 576016 |592 |
memory/httpClientSessionAllocator
825344 | 139568 |208 |
memory/httpServerSessionAllocator
22597632 |1284848 | 9808 | memory/httpSMAllocator
0 | 0 | 32 |
memory/CacheLookupHttpConfigAllocator
0 |
