[jira] [Commented] (TS-2723) Add new features for ts_lua.
[ https://issues.apache.org/jira/browse/TS-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986478#comment-13986478 ] portl4t commented on TS-2723: - Kit Chan, can I provide a patch which base on the current code ? Add new features for ts_lua. Key: TS-2723 URL: https://issues.apache.org/jira/browse/TS-2723 Project: Traffic Server Issue Type: Bug Components: Lua, Plugins Reporter: portl4t Assignee: Kit Chan Fix For: 5.0.0 After TS-2555, Kit Chan and me will integrate new features from https://github.com/portl4t/ts-lua into plugins/experimental/ts_lua. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2737) rfc5861 is not an intuitive name
[ https://issues.apache.org/jira/browse/TS-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986737#comment-13986737 ] Phil Sorber commented on TS-2737: - Originally swr and stale_while_revalidate were taken. So now that both of those are gone I suppose we can re-use the name. rfc5861 is not an intuitive name Key: TS-2737 URL: https://issues.apache.org/jira/browse/TS-2737 Project: Traffic Server Issue Type: Bug Components: Plugins Reporter: Igor Galić Assignee: Phil Sorber Fix For: 5.0.0 Can we please rename this plugin to make it a little clearer to users what it does? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2749) rfc5861 errors
[ https://issues.apache.org/jira/browse/TS-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986739#comment-13986739 ] Phil Sorber commented on TS-2749: - Is it possible to recreate this bug and get a proper stack trace? This plugin also has a lot of debug output (or at least it did). Can you run it with debug output? Thanks. rfc5861 errors Key: TS-2749 URL: https://issues.apache.org/jira/browse/TS-2749 Project: Traffic Server Issue Type: Bug Components: Cache, Plugins Reporter: Faysal Banna Assignee: Phil Sorber Fix For: sometime Good Day. I been playing around with Plugins and found this one to cause some errors which illustrated below. FATAL: InkAPI.cc:2669: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS` /usr/local/bin/traffic_server - STACK TRACE: /usr/local/lib/libtsutil.so.4(+0x1ecf7)[0x2aafcbbddcf7] /usr/local/lib/libtsutil.so.4(+0x1dc3f)[0x2aafcbbdcc3f] /usr/local/bin/traffic_server(TSMimeHdrFieldFind+0x2f)[0x4ae1bf] /usr/local/libexec/trafficserver/rfc5861.so(+0x301a)[0x2aafd6c8001a] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x150)[0x6adda0] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x6f3)[0x6ae833] /usr/local/bin/traffic_server[0x6ad6da] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e)[0x2aafccff6f6e] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2aafcdd289cd] [E. Mgmt] log == [TrafficManager] using root directory '/usr/local' which after disabling rfc5861 plugin these errors didn't occur anymore CoreDumps and stack traces are available for debugging if needed much regards -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2749) rfc5861 errors
[ https://issues.apache.org/jira/browse/TS-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986771#comment-13986771 ] Leif Hedstrom commented on TS-2749: --- Also, I wonder if this is another dupe of the WKS / header issues ? We have so many bugs filed on that, all of which seem very similar. rfc5861 errors Key: TS-2749 URL: https://issues.apache.org/jira/browse/TS-2749 Project: Traffic Server Issue Type: Bug Components: Cache, Plugins Reporter: Faysal Banna Assignee: Phil Sorber Fix For: sometime Good Day. I been playing around with Plugins and found this one to cause some errors which illustrated below. FATAL: InkAPI.cc:2669: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS` /usr/local/bin/traffic_server - STACK TRACE: /usr/local/lib/libtsutil.so.4(+0x1ecf7)[0x2aafcbbddcf7] /usr/local/lib/libtsutil.so.4(+0x1dc3f)[0x2aafcbbdcc3f] /usr/local/bin/traffic_server(TSMimeHdrFieldFind+0x2f)[0x4ae1bf] /usr/local/libexec/trafficserver/rfc5861.so(+0x301a)[0x2aafd6c8001a] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x150)[0x6adda0] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x6f3)[0x6ae833] /usr/local/bin/traffic_server[0x6ad6da] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e)[0x2aafccff6f6e] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2aafcdd289cd] [E. Mgmt] log == [TrafficManager] using root directory '/usr/local' which after disabling rfc5861 plugin these errors didn't occur anymore CoreDumps and stack traces are available for debugging if needed much regards -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2749) rfc5861 errors
[ https://issues.apache.org/jira/browse/TS-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986780#comment-13986780 ] Phil Sorber commented on TS-2749: - [~amc] looked at this for [~degreane] and said the bufp passed in was NULL. So maybe there is a code path where that is not allocated properly. But it's hard to tell without knowing where the call even is. rfc5861 errors Key: TS-2749 URL: https://issues.apache.org/jira/browse/TS-2749 Project: Traffic Server Issue Type: Bug Components: Cache, Plugins Reporter: Faysal Banna Assignee: Phil Sorber Fix For: sometime Good Day. I been playing around with Plugins and found this one to cause some errors which illustrated below. FATAL: InkAPI.cc:2669: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS` /usr/local/bin/traffic_server - STACK TRACE: /usr/local/lib/libtsutil.so.4(+0x1ecf7)[0x2aafcbbddcf7] /usr/local/lib/libtsutil.so.4(+0x1dc3f)[0x2aafcbbdcc3f] /usr/local/bin/traffic_server(TSMimeHdrFieldFind+0x2f)[0x4ae1bf] /usr/local/libexec/trafficserver/rfc5861.so(+0x301a)[0x2aafd6c8001a] /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x150)[0x6adda0] /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x6f3)[0x6ae833] /usr/local/bin/traffic_server[0x6ad6da] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e)[0x2aafccff6f6e] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2aafcdd289cd] [E. Mgmt] log == [TrafficManager] using root directory '/usr/local' which after disabling rfc5861 plugin these errors didn't occur anymore CoreDumps and stack traces are available for debugging if needed much regards -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Closed] (TS-2745) TSNetAcceptNamedProtocol should not unregister on error
[
https://issues.apache.org/jira/browse/TS-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Peach closed TS-2745.
---
Resolution: Invalid
On further investigation, this code is correct. {{ssl_unregister_protocol}}
will only unregister if the protocol name and the continuation pointer match.
TSNetAcceptNamedProtocol should not unregister on error
---
Key: TS-2745
URL: https://issues.apache.org/jira/browse/TS-2745
Project: Traffic Server
Issue Type: Bug
Components: SPDY, TS API
Reporter: James Peach
{{TSNetAcceptNamedProtocol}} unregisters the names protocol if there's a
registration error. The most likely error is that the names protocol is
already registered, so this would cause it to get unregistered, which seems
bad.
I'm not sure how to deal with protocols that are registered by the core;
currently a plugin could not override that registration. Maybe that's OK.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (TS-2772) Clean up mgmt/prepare code
Brian Geffon created TS-2772: Summary: Clean up mgmt/prepare code Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon reassigned TS-2772: Assignee: Brian Geffon Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (TS-2771) Improvements to current SPDY implementation
[ https://issues.apache.org/jira/browse/TS-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Peach resolved TS-2771. - Resolution: Duplicate Assignee: James Peach Resolving as a duplicate of TS-2759 and TS-2768 Improvements to current SPDY implementation --- Key: TS-2771 URL: https://issues.apache.org/jira/browse/TS-2771 Project: Traffic Server Issue Type: Improvement Components: SPDY Reporter: Sudheer Vinukonda Assignee: James Peach 1. Current SPDY implementation in ATS core (originally developed as a plugin) uses TS plugin API. 2. Also, the implementation uses a lot of std::string and other STL constructs in the core request processing, which is very inefficient. Opening this ticket to improve the implementation by fixing the above issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986802#comment-13986802 ] ASF subversion and git services commented on TS-2772: - Commit ead727223d228b3a09d3c47acc819b3a55f3a8e5 in trafficserver's branch refs/heads/master from [~briang] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=ead7272 ] TS-2772: Clean up mgmt/preparse code that's unused Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2767) ATS Memory Leak related to SPDY
[
https://issues.apache.org/jira/browse/TS-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986806#comment-13986806
]
Sudheer Vinukonda commented on TS-2767:
---
Just to clarify - the reason the std::string objects are not being released
(before the fix) is due to the usage of freelist in ATS. The destructor of
SpdyRequest doesn't do anything (other than put back the object in the
freelist), so, the destructor of string objects never get called.
ATS Memory Leak related to SPDY
---
Key: TS-2767
URL: https://issues.apache.org/jira/browse/TS-2767
Project: Traffic Server
Issue Type: Bug
Components: SPDY
Reporter: Sudheer Vinukonda
Labels: SPDY_ATS, yahoo
Fix For: 5.0.0
Attachments: ts2767.diff
During production testing of SPDY, we noticed ATS's memory quickly grows to
about 20g RES and 31g VIRT and eventually becomes unresponsive. dstat shows a
lot of disk/paging activity.
Running Valgrind on a single request (using spdycat) shows the below possible
leaks.
{code}
==23000== 103 bytes in 1 blocks are possibly lost in loss record 3,133 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD19A: std::string::_Rep::_M_clone(std::allocatorchar
const, unsigned long) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD5EB: std::string::reserve(unsigned long) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CDABE: std::string::append(std::string const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D28DE: spdy_fetcher_launch(SpdyRequest*, TSFetchMethod)
(basic_string.h:2165)
==23000==by 0x5D379D: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (SpdyCallbacks.cc:294)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==
==23000== 128 bytes in 1 blocks are possibly lost in loss record 3,256 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x5D3C8E: std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string
::_M_insert_aux(__gnu_cxx::__normal_iteratorstd::pairstd::string,
std::string*, std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string ,
std::pairstd::string, std::string const) (new_allocator.h:89)
==23000==by 0x5D32CF: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_vector.h:741)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==by 0x6DC779: SSLNetVConnection::net_read_io(NetHandler*,
EThread*) (SSLNetVConnection.cc:294)
==23000==by 0x6E39F1: NetHandler::mainNetEvent(int, Event*)
(UnixNet.cc:384)
==23000==by 0x71059E: EThread::process_event(Event*, int)
(I_Continuation.h:146)
==23000==by 0x710DCA: EThread::execute() (UnixEThread.cc:269)
==23000==
==23000== 173 bytes in 5 blocks are possibly lost in loss record 3,303 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCDE4: ??? (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCF32: std::basic_stringchar, std::char_traitschar,
std::allocatorchar ::basic_string(char const*, std::allocatorchar
const) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D32A2: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_pair.h:101)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
[jira] [Commented] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986804#comment-13986804 ] ASF subversion and git services commented on TS-2772: - Commit 0df8e8600983f43a7d4bf0bcfe0cb44ade32ae56 in trafficserver's branch refs/heads/master from [~briang] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=0df8e86 ] TS-2772: Clean up mgmt/preparse code that's unused Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon updated TS-2772: - Fix Version/s: 5.0.0 Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2771) Improvements to current SPDY implementation
[ https://issues.apache.org/jira/browse/TS-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986808#comment-13986808 ] Sudheer Vinukonda commented on TS-2771: --- ah, thanks, James - didn't see the other tickets. Improvements to current SPDY implementation --- Key: TS-2771 URL: https://issues.apache.org/jira/browse/TS-2771 Project: Traffic Server Issue Type: Improvement Components: SPDY Reporter: Sudheer Vinukonda Assignee: James Peach 1. Current SPDY implementation in ATS core (originally developed as a plugin) uses TS plugin API. 2. Also, the implementation uses a lot of std::string and other STL constructs in the core request processing, which is very inefficient. Opening this ticket to improve the implementation by fixing the above issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon resolved TS-2772. -- Resolution: Fixed Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (TS-2773) rewrite SSL certificate configuration
James Peach created TS-2773:
---
Summary: rewrite SSL certificate configuration
Key: TS-2773
URL: https://issues.apache.org/jira/browse/TS-2773
Project: Traffic Server
Issue Type: New Feature
Components: Configuration, SSL
Reporter: James Peach
Currently, SSL certificate configuration is split across {{records.config}} and
{{ssl-multicert.config}}. This leads to awkward situations where you can't
enable client certificate validation for a particular server certificate, and
you can't add a SSL key passphrase dialog globally.
I'd like to unify the SSL configuration by pushing all the configuration
parameters down to {{records.config}} and allowing {{ssl-multicert.config}} to
override those settings. This would be logically similar to how overridable
configurations work for the TS API.
I plan to retain backwards compatibility with 4.x {{ssl-multicert.config}}
syntax. You would still need {{ssl-multicert.config}} to be able to configure
multiple SSL certificates.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (TS-2773) rewrite SSL certificate configuration
[
https://issues.apache.org/jira/browse/TS-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986820#comment-13986820
]
James Peach commented on TS-2773:
-
[~rwbarber2], [~bcall], [~sunwei]
rewrite SSL certificate configuration
-
Key: TS-2773
URL: https://issues.apache.org/jira/browse/TS-2773
Project: Traffic Server
Issue Type: New Feature
Components: Configuration, SSL
Reporter: James Peach
Currently, SSL certificate configuration is split across {{records.config}}
and {{ssl-multicert.config}}. This leads to awkward situations where you
can't enable client certificate validation for a particular server
certificate, and you can't add a SSL key passphrase dialog globally.
I'd like to unify the SSL configuration by pushing all the configuration
parameters down to {{records.config}} and allowing {{ssl-multicert.config}}
to override those settings. This would be logically similar to how
overridable configurations work for the TS API.
I plan to retain backwards compatibility with 4.x {{ssl-multicert.config}}
syntax. You would still need {{ssl-multicert.config}} to be able to configure
multiple SSL certificates.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986836#comment-13986836 ] ASF subversion and git services commented on TS-2772: - Commit 71ffb57f66129d022bdb2131e398df9c6259d5d1 in trafficserver's branch refs/heads/master from [~jpe...@apache.org] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=71ffb57 ] TS-2772: remove preparse from traffic_sac Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2767) ATS Memory Leak related to SPDY
[
https://issues.apache.org/jira/browse/TS-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986894#comment-13986894
]
ASF subversion and git services commented on TS-2767:
-
Commit 966f4a55a343283231dbbf7d8fe2ff75858182e2 in trafficserver's branch
refs/heads/master from [~sudheerv]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=966f4a5 ]
TS-2767: ATS Memory Leak related to SPDY
ATS Memory Leak related to SPDY
---
Key: TS-2767
URL: https://issues.apache.org/jira/browse/TS-2767
Project: Traffic Server
Issue Type: Bug
Components: SPDY
Reporter: Sudheer Vinukonda
Labels: SPDY_ATS, yahoo
Fix For: 5.0.0
Attachments: ts2767.diff
During production testing of SPDY, we noticed ATS's memory quickly grows to
about 20g RES and 31g VIRT and eventually becomes unresponsive. dstat shows a
lot of disk/paging activity.
Running Valgrind on a single request (using spdycat) shows the below possible
leaks.
{code}
==23000== 103 bytes in 1 blocks are possibly lost in loss record 3,133 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD19A: std::string::_Rep::_M_clone(std::allocatorchar
const, unsigned long) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD5EB: std::string::reserve(unsigned long) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CDABE: std::string::append(std::string const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D28DE: spdy_fetcher_launch(SpdyRequest*, TSFetchMethod)
(basic_string.h:2165)
==23000==by 0x5D379D: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (SpdyCallbacks.cc:294)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==
==23000== 128 bytes in 1 blocks are possibly lost in loss record 3,256 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x5D3C8E: std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string
::_M_insert_aux(__gnu_cxx::__normal_iteratorstd::pairstd::string,
std::string*, std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string ,
std::pairstd::string, std::string const) (new_allocator.h:89)
==23000==by 0x5D32CF: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_vector.h:741)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==by 0x6DC779: SSLNetVConnection::net_read_io(NetHandler*,
EThread*) (SSLNetVConnection.cc:294)
==23000==by 0x6E39F1: NetHandler::mainNetEvent(int, Event*)
(UnixNet.cc:384)
==23000==by 0x71059E: EThread::process_event(Event*, int)
(I_Continuation.h:146)
==23000==by 0x710DCA: EThread::execute() (UnixEThread.cc:269)
==23000==
==23000== 173 bytes in 5 blocks are possibly lost in loss record 3,303 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCDE4: ??? (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCF32: std::basic_stringchar, std::char_traitschar,
std::allocatorchar ::basic_string(char const*, std::allocatorchar
const) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D32A2: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_pair.h:101)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57:
[jira] [Commented] (TS-2767) ATS Memory Leak related to SPDY
[
https://issues.apache.org/jira/browse/TS-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986897#comment-13986897
]
ASF subversion and git services commented on TS-2767:
-
Commit b3a98049645c6f9a238eac7a9c319be2ff435383 in trafficserver's branch
refs/heads/master from [~bcall]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=b3a9804 ]
added TS-2767 to changes
ATS Memory Leak related to SPDY
---
Key: TS-2767
URL: https://issues.apache.org/jira/browse/TS-2767
Project: Traffic Server
Issue Type: Bug
Components: SPDY
Reporter: Sudheer Vinukonda
Labels: SPDY_ATS, yahoo
Fix For: 5.0.0
Attachments: ts2767.diff
During production testing of SPDY, we noticed ATS's memory quickly grows to
about 20g RES and 31g VIRT and eventually becomes unresponsive. dstat shows a
lot of disk/paging activity.
Running Valgrind on a single request (using spdycat) shows the below possible
leaks.
{code}
==23000== 103 bytes in 1 blocks are possibly lost in loss record 3,133 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD19A: std::string::_Rep::_M_clone(std::allocatorchar
const, unsigned long) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD5EB: std::string::reserve(unsigned long) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CDABE: std::string::append(std::string const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D28DE: spdy_fetcher_launch(SpdyRequest*, TSFetchMethod)
(basic_string.h:2165)
==23000==by 0x5D379D: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (SpdyCallbacks.cc:294)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==
==23000== 128 bytes in 1 blocks are possibly lost in loss record 3,256 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x5D3C8E: std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string
::_M_insert_aux(__gnu_cxx::__normal_iteratorstd::pairstd::string,
std::string*, std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string ,
std::pairstd::string, std::string const) (new_allocator.h:89)
==23000==by 0x5D32CF: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_vector.h:741)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==by 0x6DC779: SSLNetVConnection::net_read_io(NetHandler*,
EThread*) (SSLNetVConnection.cc:294)
==23000==by 0x6E39F1: NetHandler::mainNetEvent(int, Event*)
(UnixNet.cc:384)
==23000==by 0x71059E: EThread::process_event(Event*, int)
(I_Continuation.h:146)
==23000==by 0x710DCA: EThread::execute() (UnixEThread.cc:269)
==23000==
==23000== 173 bytes in 5 blocks are possibly lost in loss record 3,303 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCDE4: ??? (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCF32: std::basic_stringchar, std::char_traitschar,
std::allocatorchar ::basic_string(char const*, std::allocatorchar
const) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D32A2: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_pair.h:101)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57:
[jira] [Commented] (TS-2772) Clean up mgmt/prepare code
[ https://issues.apache.org/jira/browse/TS-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986916#comment-13986916 ] ASF subversion and git services commented on TS-2772: - Commit 3d11d5ba19709fdbdd1c44080c6bc1dfcf4d2260 in trafficserver's branch refs/heads/master from [~zwoop] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=3d11d5b ] TS-2772 Eliminate the preparse Makefile Clean up mgmt/prepare code -- Key: TS-2772 URL: https://issues.apache.org/jira/browse/TS-2772 Project: Traffic Server Issue Type: Improvement Components: Manager Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 mgmt/preparse isn't used anywhere, I'm going to blow it away. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[ https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2636: -- Attachment: ts2636.diff updated the patch file to reflect the new tree for logging (proxy/logging) Enhance ATS custom logging to support WIPE filter action Key: TS-2636 URL: https://issues.apache.org/jira/browse/TS-2636 Project: Traffic Server Issue Type: New Feature Components: Logging Reporter: Sudheer Vinukonda Assignee: Yunkai Zhang Labels: Review Fix For: 5.0.0 Attachments: ts2636.diff Currently, ATS custom logging supports LogFilters with actions Accept, Reject. This feature request is to add a new filter action WIPE. WIPE can be used in hiding sensitive parameters (e.g. userid, password) within the query part of a URL. Attached is a draft version of the patch. Kindly review and comment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[ https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2636: -- Attachment: (was: ts2636.diff) Enhance ATS custom logging to support WIPE filter action Key: TS-2636 URL: https://issues.apache.org/jira/browse/TS-2636 Project: Traffic Server Issue Type: New Feature Components: Logging Reporter: Sudheer Vinukonda Assignee: Yunkai Zhang Labels: Review Fix For: 5.0.0 Attachments: ts2636.diff Currently, ATS custom logging supports LogFilters with actions Accept, Reject. This feature request is to add a new filter action WIPE. WIPE can be used in hiding sensitive parameters (e.g. userid, password) within the query part of a URL. Attached is a draft version of the patch. Kindly review and comment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[ https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2636: -- Attachment: (was: ts2636(1).diff) Enhance ATS custom logging to support WIPE filter action Key: TS-2636 URL: https://issues.apache.org/jira/browse/TS-2636 Project: Traffic Server Issue Type: New Feature Components: Logging Reporter: Sudheer Vinukonda Assignee: Yunkai Zhang Labels: Review Fix For: 5.0.0 Attachments: ts2636.diff Currently, ATS custom logging supports LogFilters with actions Accept, Reject. This feature request is to add a new filter action WIPE. WIPE can be used in hiding sensitive parameters (e.g. userid, password) within the query part of a URL. Attached is a draft version of the patch. Kindly review and comment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2736) Use more file descriptors by default
[ https://issues.apache.org/jira/browse/TS-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986976#comment-13986976 ] ASF subversion and git services commented on TS-2736: - Commit 22bb4923e8d06fc1adcfefafafcb7c97f5d485fc in trafficserver's branch refs/heads/master from [~psudaemon] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=22bb492 ] TS-2736: Add static_cast Use more file descriptors by default Key: TS-2736 URL: https://issues.apache.org/jira/browse/TS-2736 Project: Traffic Server Issue Type: Improvement Reporter: Phil Sorber Assignee: Phil Sorber Fix For: 5.0.0 Currently our connection throttling setting limits the total number of FD's that ATS can use. In addition it has pro-active code that stops taking connections as that limit is approached. So we can theoretically use more FD's than we request and still have that setting be effective. Additionally epoll can use a lot of FD's if we use it for things like AIO and timers. Modern Linux systems have many millions of FD's available. I think we should use more FD's by default but have a setting that we can lower it back down. I propose the default be 90%. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986979#comment-13986979 ] ASF subversion and git services commented on TS-2766: - Commit 1630af7f76643c86751398838fbdf8a1363ce292 in trafficserver's branch refs/heads/master from [~briang] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=1630af7 ] TS-2766: HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986980#comment-13986980 ] ASF subversion and git services commented on TS-2766: - Commit eaf5795e149185f4834c2beff63c9738e85e833c in trafficserver's branch refs/heads/master from [~briang] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=eaf5795 ] TS-2766: HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986985#comment-13986985 ] Brian Geffon commented on TS-2766: -- [~psudaemon], please use 1630af7f76643c86751398838fbdf8a1363ce292 for backport HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon updated TS-2766: - Backport to Version: 4.2.2 HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon reopened TS-2766: -- Opening for backport. HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Closed] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Geffon closed TS-2766. Resolution: Fixed HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[
https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987031#comment-13987031
]
Bryan Call commented on TS-2636:
[~sudheerv]
These changes need to be made to the patch:
1. Indentation is 2 spaces not 4 spaces - there is a mix of indentation
2. Remove spaces at the end of lines
3. Make sure semicolons don't have a space before them - currently: return
cond_satisfied ;
4. Function return types need to be on a separate line - currently: void
LogAccessHttp::set_client_req_url(char *buf, int len) // STR
5. Action name should be WIPE_FIELD_VALUE - currently: const char
*LogFilter::ACTION_NAME[] = { REJECT, ACCEPT, WIPE };
6. No spaces before the parentheses in function calls - currently: wipeField
(field_value, val[i]);
Coding Style:
https://cwiki.apache.org/confluence/display/TS/Coding+Style
Enhance ATS custom logging to support WIPE filter action
Key: TS-2636
URL: https://issues.apache.org/jira/browse/TS-2636
Project: Traffic Server
Issue Type: New Feature
Components: Logging
Reporter: Sudheer Vinukonda
Assignee: Yunkai Zhang
Labels: Review
Fix For: 5.0.0
Attachments: ts2636.diff
Currently, ATS custom logging supports LogFilters with actions Accept,
Reject. This feature request is to add a new filter action WIPE. WIPE can
be used in hiding sensitive parameters (e.g. userid, password) within the
query part of a URL. Attached is a draft version of the patch. Kindly review
and comment.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (TS-2774) TS::AdminClient.pm's get_stat API broken in 5.0
Sudheer Vinukonda created TS-2774: - Summary: TS::AdminClient.pm's get_stat API broken in 5.0 Key: TS-2774 URL: https://issues.apache.org/jira/browse/TS-2774 Project: Traffic Server Issue Type: Bug Components: Management API Reporter: Sudheer Vinukonda With the changes in TS-2637, the order in which the fields in the response are sent is changed. This broke the get_stat() API in TS::AdminClient which expects the response fields in a specific order. Will open a separate jira to fix if there are any other clients also impacted with this API change, but, in the meantime, a few production monitoring tools that we use rely heavily on get_stat(), so fixing this issue first with this ticket. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2774) TS::AdminClient.pm's get_stat API broken in 5.0
[ https://issues.apache.org/jira/browse/TS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2774: -- Labels: SPDY_ATS yahoo (was: ) TS::AdminClient.pm's get_stat API broken in 5.0 --- Key: TS-2774 URL: https://issues.apache.org/jira/browse/TS-2774 Project: Traffic Server Issue Type: Bug Components: Management API Reporter: Sudheer Vinukonda Labels: SPDY_ATS, yahoo With the changes in TS-2637, the order in which the fields in the response are sent is changed. This broke the get_stat() API in TS::AdminClient which expects the response fields in a specific order. Will open a separate jira to fix if there are any other clients also impacted with this API change, but, in the meantime, a few production monitoring tools that we use rely heavily on get_stat(), so fixing this issue first with this ticket. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2774) TS::AdminClient.pm's get_stat API broken in 5.0
[ https://issues.apache.org/jira/browse/TS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2774: -- Attachment: ts2774.diff patch to fix get_stat() client API to match the new order of the Management API send_record_get_reply(). TS::AdminClient.pm's get_stat API broken in 5.0 --- Key: TS-2774 URL: https://issues.apache.org/jira/browse/TS-2774 Project: Traffic Server Issue Type: Bug Components: Management API Reporter: Sudheer Vinukonda Labels: SPDY_ATS, yahoo Attachments: ts2774.diff With the changes in TS-2637, the order in which the fields in the response are sent is changed. This broke the get_stat() API in TS::AdminClient which expects the response fields in a specific order. Will open a separate jira to fix if there are any other clients also impacted with this API change, but, in the meantime, a few production monitoring tools that we use rely heavily on get_stat(), so fixing this issue first with this ticket. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[ https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2636: -- Attachment: ts2636.diff Enhance ATS custom logging to support WIPE filter action Key: TS-2636 URL: https://issues.apache.org/jira/browse/TS-2636 Project: Traffic Server Issue Type: New Feature Components: Logging Reporter: Sudheer Vinukonda Assignee: Yunkai Zhang Labels: Review Fix For: 5.0.0 Attachments: ts2636.diff Currently, ATS custom logging supports LogFilters with actions Accept, Reject. This feature request is to add a new filter action WIPE. WIPE can be used in hiding sensitive parameters (e.g. userid, password) within the query part of a URL. Attached is a draft version of the patch. Kindly review and comment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2636) Enhance ATS custom logging to support WIPE filter action
[ https://issues.apache.org/jira/browse/TS-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2636: -- Attachment: (was: ts2636.diff) Enhance ATS custom logging to support WIPE filter action Key: TS-2636 URL: https://issues.apache.org/jira/browse/TS-2636 Project: Traffic Server Issue Type: New Feature Components: Logging Reporter: Sudheer Vinukonda Assignee: Yunkai Zhang Labels: Review Fix For: 5.0.0 Attachments: ts2636.diff Currently, ATS custom logging supports LogFilters with actions Accept, Reject. This feature request is to add a new filter action WIPE. WIPE can be used in hiding sensitive parameters (e.g. userid, password) within the query part of a URL. Attached is a draft version of the patch. Kindly review and comment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2766) HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size
[ https://issues.apache.org/jira/browse/TS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987228#comment-13987228 ] ASF subversion and git services commented on TS-2766: - Commit 72782383e817e65ddc6448d7eb05207202db4a76 in trafficserver's branch refs/heads/master from [~zwoop] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=7278238 ] TS-2766 Fix indentation / coding style HdrHeap::coalesce_str_heaps doesn't properly calculate new heap size Key: TS-2766 URL: https://issues.apache.org/jira/browse/TS-2766 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 4.2.1, 5.0.0 Reporter: Brian Geffon Assignee: Brian Geffon Fix For: 5.0.0 HdrHeap::coalesce_str_heaps doesn't properly calculate the new heap size for a few reasons: 1) It doesn't current walk the HdrHeap chain (m_next). 2) It doesn't account for HdrHeap objects that point to the same string in the HdrStrHeap. The easiest fix for this is to completely walk all of the HdrHeap objects in the chain and sum up the size of every string; obviously this approach means that strings that were previously aliased will now become independent copies on coalesce; however, the alternative solution that allows continued aliasing would be pretty messy. I'm proposing we just properly determine the size but allow the user to minimizing coalescing by making the read only heap sizes configurable and the StrHdrHeap and HdrHeap default sizes configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2767) ATS Memory Leak related to SPDY
[
https://issues.apache.org/jira/browse/TS-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987269#comment-13987269
]
Yunkai Zhang commented on TS-2767:
--
Why not use {{string.clear()}} ? I bet it should be more efficient than
{{string.swap()}}
{code}
void
SpdyRequest::clear()
{
...
url.clear();
...
}
{code}
ATS Memory Leak related to SPDY
---
Key: TS-2767
URL: https://issues.apache.org/jira/browse/TS-2767
Project: Traffic Server
Issue Type: Bug
Components: SPDY
Reporter: Sudheer Vinukonda
Labels: SPDY_ATS, yahoo
Fix For: 5.0.0
Attachments: ts2767.diff
During production testing of SPDY, we noticed ATS's memory quickly grows to
about 20g RES and 31g VIRT and eventually becomes unresponsive. dstat shows a
lot of disk/paging activity.
Running Valgrind on a single request (using spdycat) shows the below possible
leaks.
{code}
==23000== 103 bytes in 1 blocks are possibly lost in loss record 3,133 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD19A: std::string::_Rep::_M_clone(std::allocatorchar
const, unsigned long) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD5EB: std::string::reserve(unsigned long) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CDABE: std::string::append(std::string const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D28DE: spdy_fetcher_launch(SpdyRequest*, TSFetchMethod)
(basic_string.h:2165)
==23000==by 0x5D379D: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (SpdyCallbacks.cc:294)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==
==23000== 128 bytes in 1 blocks are possibly lost in loss record 3,256 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x5D3C8E: std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string
::_M_insert_aux(__gnu_cxx::__normal_iteratorstd::pairstd::string,
std::string*, std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string ,
std::pairstd::string, std::string const) (new_allocator.h:89)
==23000==by 0x5D32CF: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_vector.h:741)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==by 0x6DC779: SSLNetVConnection::net_read_io(NetHandler*,
EThread*) (SSLNetVConnection.cc:294)
==23000==by 0x6E39F1: NetHandler::mainNetEvent(int, Event*)
(UnixNet.cc:384)
==23000==by 0x71059E: EThread::process_event(Event*, int)
(I_Continuation.h:146)
==23000==by 0x710DCA: EThread::execute() (UnixEThread.cc:269)
==23000==
==23000== 173 bytes in 5 blocks are possibly lost in loss record 3,303 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCDE4: ??? (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCF32: std::basic_stringchar, std::char_traitschar,
std::allocatorchar ::basic_string(char const*, std::allocatorchar
const) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D32A2: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_pair.h:101)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6:
[jira] [Commented] (TS-2767) ATS Memory Leak related to SPDY
[
https://issues.apache.org/jira/browse/TS-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987334#comment-13987334
]
Sudheer Vinukonda commented on TS-2767:
---
coz, std::string.clear() does not free the memory - it will logically set the
string to an empty string, but, not release the memory. You may write a simple
test program to confirm this behavior. The only way you could guarantee freeing
the memory from std::string object is by swapping it out with an empty string
object. In any case, as noted by Leif above, STL constructs use heap memory and
are not efficient. Their use should be avoided during call processing for
better performance.
ATS Memory Leak related to SPDY
---
Key: TS-2767
URL: https://issues.apache.org/jira/browse/TS-2767
Project: Traffic Server
Issue Type: Bug
Components: SPDY
Reporter: Sudheer Vinukonda
Labels: SPDY_ATS, yahoo
Fix For: 5.0.0
Attachments: ts2767.diff
During production testing of SPDY, we noticed ATS's memory quickly grows to
about 20g RES and 31g VIRT and eventually becomes unresponsive. dstat shows a
lot of disk/paging activity.
Running Valgrind on a single request (using spdycat) shows the below possible
leaks.
{code}
==23000== 103 bytes in 1 blocks are possibly lost in loss record 3,133 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD19A: std::string::_Rep::_M_clone(std::allocatorchar
const, unsigned long) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CD5EB: std::string::reserve(unsigned long) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CDABE: std::string::append(std::string const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D28DE: spdy_fetcher_launch(SpdyRequest*, TSFetchMethod)
(basic_string.h:2165)
==23000==by 0x5D379D: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (SpdyCallbacks.cc:294)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==
==23000== 128 bytes in 1 blocks are possibly lost in loss record 3,256 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x5D3C8E: std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string
::_M_insert_aux(__gnu_cxx::__normal_iteratorstd::pairstd::string,
std::string*, std::vectorstd::pairstd::string, std::string,
std::allocatorstd::pairstd::string, std::string ,
std::pairstd::string, std::string const) (new_allocator.h:89)
==23000==by 0x5D32CF: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_vector.h:741)
==23000==by 0x715E9F: spdylay_session_on_syn_stream_received
(spdylay_session.c:1782)
==23000==by 0x716BA6: spdylay_session_mem_recv (spdylay_session.c:2246)
==23000==by 0x716DD8: spdylay_session_recv (spdylay_session.c:2828)
==23000==by 0x5D0E57: spdy_default_handler(tsapi_cont*, TSEvent, void*)
(SpdySM.cc:263)
==23000==by 0x6EC1A6: UnixNetVConnection::readSignalAndUpdate(int)
(I_Continuation.h:146)
==23000==by 0x6DC779: SSLNetVConnection::net_read_io(NetHandler*,
EThread*) (SSLNetVConnection.cc:294)
==23000==by 0x6E39F1: NetHandler::mainNetEvent(int, Event*)
(UnixNet.cc:384)
==23000==by 0x71059E: EThread::process_event(Event*, int)
(I_Continuation.h:146)
==23000==by 0x710DCA: EThread::execute() (UnixEThread.cc:269)
==23000==
==23000== 173 bytes in 5 blocks are possibly lost in loss record 3,303 of
3,823
==23000==at 0x4C285BC: operator new(unsigned long)
(vg_replace_malloc.c:298)
==23000==by 0x72CC3C8: std::string::_Rep::_S_create(unsigned long,
unsigned long, std::allocatorchar const) (in
/usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCDE4: ??? (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x72CCF32: std::basic_stringchar, std::char_traitschar,
std::allocatorchar ::basic_string(char const*, std::allocatorchar
const) (in /usr/lib64/libstdc++.so.6.0.13)
==23000==by 0x5D32A2: spdy_on_ctrl_recv_callback(spdylay_session*,
spdylay_frame_type, spdylay_frame*, void*) (stl_pair.h:101)
==23000==by 0x715E9F:
[jira] [Updated] (TS-2723) Add new features for ts_lua.
[ https://issues.apache.org/jira/browse/TS-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] portl4t updated TS-2723: Attachment: 0001-TS-2723-add-new-features-for-ts_lua.patch add new features Add new features for ts_lua. Key: TS-2723 URL: https://issues.apache.org/jira/browse/TS-2723 Project: Traffic Server Issue Type: Bug Components: Lua, Plugins Reporter: portl4t Assignee: Kit Chan Fix For: 5.0.0 Attachments: 0001-TS-2723-add-new-features-for-ts_lua.patch After TS-2555, Kit Chan and me will integrate new features from https://github.com/portl4t/ts-lua into plugins/experimental/ts_lua. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2774) TS::AdminClient.pm's get_stat API broken in 5.0
[ https://issues.apache.org/jira/browse/TS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2774: -- Description: With the changes in TS-2637, the order in which the fields in the response are sent, has been changed. This broke the get_stat() perl API in TS::AdminClient, which expects the response fields in a specific order. Will open a separate jira to fix any other clients that might have also been impacted with this API change, but, in the meantime, fixing the get_stat() perl API first, since some production monitoring tools that we use are impacted by it. (was: With the changes in TS-2637, the order in which the fields in the response are sent is changed. This broke the get_stat() API in TS::AdminClient which expects the response fields in a specific order. Will open a separate jira to fix if there are any other clients also impacted with this API change, but, in the meantime, a few production monitoring tools that we use rely heavily on get_stat(), so fixing this issue first with this ticket.) TS::AdminClient.pm's get_stat API broken in 5.0 --- Key: TS-2774 URL: https://issues.apache.org/jira/browse/TS-2774 Project: Traffic Server Issue Type: Bug Components: Management API Reporter: Sudheer Vinukonda Labels: SPDY_ATS, yahoo Attachments: ts2774.diff With the changes in TS-2637, the order in which the fields in the response are sent, has been changed. This broke the get_stat() perl API in TS::AdminClient, which expects the response fields in a specific order. Will open a separate jira to fix any other clients that might have also been impacted with this API change, but, in the meantime, fixing the get_stat() perl API first, since some production monitoring tools that we use are impacted by it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (TS-2774) TS::AdminClient.pm's get_stat API broken in 5.0
[ https://issues.apache.org/jira/browse/TS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheer Vinukonda updated TS-2774: -- Description: With the changes in TS-2637, the order in which the fields in the response are sent, has been changed. This broke the get_stat() perl API in TS::AdminClient, which expects the response fields in a specific order. Will open a separate jira to fix any other clients that might have also been impacted by this API change, but, in the meantime, fixing the get_stat() perl API first, since some production monitoring tools that we use are impacted by it. (was: With the changes in TS-2637, the order in which the fields in the response are sent, has been changed. This broke the get_stat() perl API in TS::AdminClient, which expects the response fields in a specific order. Will open a separate jira to fix any other clients that might have also been impacted with this API change, but, in the meantime, fixing the get_stat() perl API first, since some production monitoring tools that we use are impacted by it.) TS::AdminClient.pm's get_stat API broken in 5.0 --- Key: TS-2774 URL: https://issues.apache.org/jira/browse/TS-2774 Project: Traffic Server Issue Type: Bug Components: Management API Reporter: Sudheer Vinukonda Labels: SPDY_ATS, yahoo Attachments: ts2774.diff With the changes in TS-2637, the order in which the fields in the response are sent, has been changed. This broke the get_stat() perl API in TS::AdminClient, which expects the response fields in a specific order. Will open a separate jira to fix any other clients that might have also been impacted by this API change, but, in the meantime, fixing the get_stat() perl API first, since some production monitoring tools that we use are impacted by it. -- This message was sent by Atlassian JIRA (v6.2#6252)
