[jira] [Assigned] (TS-3603) Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled
[ https://issues.apache.org/jira/browse/TS-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3603: -- Assignee: Susan Hinrichs Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled --- Key: TS-3603 URL: https://issues.apache.org/jira/browse/TS-3603 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Susan Hinrichs Attachments: TS-3603.diff This was found while tracking down TS-3597. The assert stack is in a comment on that bug. When you don't have a dedicated assert thread, the mutex is not locked before going into the do_io_read to process the accept event. In the dedicated thread case, you end up exercising UnixNetVConnection::acceptEvent which does grab the mutex. May be a relatively harmless error. Since this is a newly created VC, there should be no race conditions on it. But violating locking assumptions seem like a really bad idea. Especially since grabbing a lock on a supposedly uncontended object should be cheap. A 5.3.x patch is attached to this bug which solves the problem on my build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3604) Transparent mode does not work when accept_threads set to 0
[ https://issues.apache.org/jira/browse/TS-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3604: --- Assignee: Alan M. Carroll Transparent mode does not work when accept_threads set to 0 --- Key: TS-3604 URL: https://issues.apache.org/jira/browse/TS-3604 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Alan M. Carroll If you have transparency enabled on your port and you disable the dedicated accept_threads, the TCP connection does not complete for HTTP and HTTPS traffic. Enabling the accept_thread causes traffic to flow again. Traffic via a remap rule also works (once you apply the fix to TS-3603). Looking at the packet capture, the client sends SYN's but they are never responded to. It appears that the listen doesn't really get set up in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3597) TLS can fail accept / handshake since commit 2a8bb593fd
[ https://issues.apache.org/jira/browse/TS-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542796#comment-14542796 ] Susan Hinrichs commented on TS-3597: Ok, I spoke too soon about being able to reproduce the problem. In my dev environment, I get no TCP handshake completion if I turn off the accept_thread. In reverse proxy mode, I get an assert in UnixNetVConnection::set_enabled called from do_io_read because the lock is not held. It is possible this is what you are seeing in production, since this isn't a release assert and you are seeing timing issues since no lock is held during the accept processing. Or this is something completely different and unique to my environment. For the record, the commit identified has nothing to do with the TCP accept processing. For your reading pleasure, here is my stack. Will dig more tomorrow. {code} #0 0x00351e4328a5 in raise () from /lib64/libc.so.6 #1 0x00351e434085 in abort () from /lib64/libc.so.6 #2 0x77dd9c51 in ink_die_die_die () at ink_error.cc:43 #3 0x77dd9d08 in ink_fatal_va(const char *, typedef __va_list_tag __va_list_tag *) (fmt=0x77deaa58 %s:%d: failed assert `%s`, ap=0x7fffdca0) at ink_error.cc:65 #4 0x77dd9dd9 in ink_fatal ( message_format=0x77deaa58 %s:%d: failed assert `%s`) at ink_error.cc:73 #5 0x77dd7876 in _ink_assert ( expression=0x83a988 vio-mutex-thread_holding == this_ethread() thread, file=0x83a6be UnixNetVConnection.cc, line=859) at ink_assert.cc:37 #6 0x0078c4bd in UnixNetVConnection::set_enabled (this=0x3c0ed20, vio=0x3c0ee40) at UnixNetVConnection.cc:859 #7 0x0078bbb4 in UnixNetVConnection::reenable (this=0x3c0ed20, vio=0x3c0ee40) at UnixNetVConnection.cc:753 #8 0x0050d229 in VIO::reenable (this=0x3c0ee40) at ../iocore/eventsystem/P_VIO.h:112 #9 0x0078b25c in UnixNetVConnection::do_io_read (this=0x3c0ed20, c=0x24a1180, nbytes=4096, buf=0x3357620) at UnixNetVConnection.cc:598 #10 0x005594bd in ProtocolProbeSessionAccept::mainEvent ( this=0x24a92c0, event=202, data=0x3c0ed20) at ProtocolProbeSessionAccept.cc:148 #11 0x0050d1d6 in Continuation::handleEvent (this=0x24a92c0, event=202, data=0x3c0ed20) at ../iocore/eventsystem/I_Continuation.h:145 #12 0x007863e8 in NetAccept::acceptFastEvent (this=0x2480960, event=5, ep=0x1ee5160) at UnixNetAccept.cc:465 #13 0x0050d1d6 in Continuation::handleEvent (this=0x2480960, event=5, data=0x1ee5160) at ../iocore/eventsystem/I_Continuation.h:145 #14 0x007abcb2 in EThread::process_event (this=0x1bb, e=0x1ee5160, calling_code=5) at UnixEThread.cc:128 ---Type return to continue, or q return to quit--- #15 0x007ac2d3 in EThread::execute (this=0x1bb) at UnixEThread.cc:252 #16 0x0054097e in main (argv=0x7fffe398) at Main.cc:1840 {code} TLS can fail accept / handshake since commit 2a8bb593fd --- Key: TS-3597 URL: https://issues.apache.org/jira/browse/TS-3597 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Critical Fix For: 6.0.0 At least under certain conditions (slightly unclear,but possible a race with multiple NUMA nodes), we fail to accept / TLS handshake. I've tracked this down to the commit from 2a8bb593fdd7ca9125efad76e27f3f17f5bca794. The commit prior to this does not expose the problem. [~gancho] also discovered that this problem is only triggered when accept thread is off (0). Also from [~gancho], when this reproduces, a command like e.g. this will fail the handshake completely (no ciphers): {code} openssl s_client -connect 10.1.2.3:443 -tls1 -servername some.host.com {code} Also, since this only happens with accept thread off (0), which implies accept on every ET_NET thread, maybe there's some sort of race condition going on here? That's just a wild speculation though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3597) TLS can fail accept / handshake since commit 2a8bb593fd
[ https://issues.apache.org/jira/browse/TS-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544361#comment-14544361 ] Susan Hinrichs commented on TS-3597: I'm seeing an issue in the no accept thread case where the value of sslHandshakeHookState is not HANDSHAKE_HOOKS_PRE in the sni callback, even though this appears to be the first time through the callback for that vc. It looks like sometimes the VC pointer is reused (reallocated) without having its values returned to the initial value. This means that the correct cert is not selected. Instead the default certificate is used. Not clear this is the error case that you are seeing, but certainly it is a bad indicator. Must take a break for now. Will press on later this evening. TLS can fail accept / handshake since commit 2a8bb593fd --- Key: TS-3597 URL: https://issues.apache.org/jira/browse/TS-3597 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Critical Fix For: 6.0.0 At least under certain conditions (slightly unclear,but possible a race with multiple NUMA nodes), we fail to accept / TLS handshake. I've tracked this down to the commit from 2a8bb593fdd7ca9125efad76e27f3f17f5bca794. The commit prior to this does not expose the problem. [~gancho] also discovered that this problem is only triggered when accept thread is off (0). Also from [~gancho], when this reproduces, a command like e.g. this will fail the handshake completely (no ciphers): {code} openssl s_client -connect 10.1.2.3:443 -tls1 -servername some.host.com {code} Also, since this only happens with accept thread off (0), which implies accept on every ET_NET thread, maybe there's some sort of race condition going on here? That's just a wild speculation though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3603) Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled
[ https://issues.apache.org/jira/browse/TS-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3603. Resolution: Fixed Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled --- Key: TS-3603 URL: https://issues.apache.org/jira/browse/TS-3603 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Susan Hinrichs Attachments: TS-3603.diff This was found while tracking down TS-3597. The assert stack is in a comment on that bug. When you don't have a dedicated assert thread, the mutex is not locked before going into the do_io_read to process the accept event. In the dedicated thread case, you end up exercising UnixNetVConnection::acceptEvent which does grab the mutex. May be a relatively harmless error. Since this is a newly created VC, there should be no race conditions on it. But violating locking assumptions seem like a really bad idea. Especially since grabbing a lock on a supposedly uncontended object should be cheap. A 5.3.x patch is attached to this bug which solves the problem on my build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3604) Transparent mode does not work when accept_threads set to 0
[ https://issues.apache.org/jira/browse/TS-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545733#comment-14545733 ] Susan Hinrichs commented on TS-3604: Alan found the issue. We are not passing the transparent option to the do_listen in the code paths used by the combined accept and regular threads. Propagating that information. Transparent mode does not work when accept_threads set to 0 --- Key: TS-3604 URL: https://issues.apache.org/jira/browse/TS-3604 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Alan M. Carroll If you have transparency enabled on your port and you disable the dedicated accept_threads, the TCP connection does not complete for HTTP and HTTPS traffic. Enabling the accept_thread causes traffic to flow again. Traffic via a remap rule also works (once you apply the fix to TS-3603). Looking at the packet capture, the client sends SYN's but they are never responded to. It appears that the listen doesn't really get set up in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3603) Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled
[ https://issues.apache.org/jira/browse/TS-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3603: --- Backport to Version: 5.3.1 Debug Assert occurs in UnixNetVConnection::set_enabled when accept_threads are disabled --- Key: TS-3603 URL: https://issues.apache.org/jira/browse/TS-3603 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Susan Hinrichs Attachments: TS-3603.diff This was found while tracking down TS-3597. The assert stack is in a comment on that bug. When you don't have a dedicated assert thread, the mutex is not locked before going into the do_io_read to process the accept event. In the dedicated thread case, you end up exercising UnixNetVConnection::acceptEvent which does grab the mutex. May be a relatively harmless error. Since this is a newly created VC, there should be no race conditions on it. But violating locking assumptions seem like a really bad idea. Especially since grabbing a lock on a supposedly uncontended object should be cheap. A 5.3.x patch is attached to this bug which solves the problem on my build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3597) TLS can fail accept / handshake since commit 2a8bb593fd
[ https://issues.apache.org/jira/browse/TS-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3597. Resolution: Fixed Sorry I accidentally committed this fix with the fix for TS-3603 in commit ef467a2be79fc962ae0ec042ef9f6e871d3a775f Both should be applied together in any case. TLS can fail accept / handshake since commit 2a8bb593fd --- Key: TS-3597 URL: https://issues.apache.org/jira/browse/TS-3597 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Critical Fix For: 6.0.0 Attachments: TS-3597.diff At least under certain conditions (slightly unclear,but possible a race with multiple NUMA nodes), we fail to accept / TLS handshake. I've tracked this down to the commit from 2a8bb593fdd7ca9125efad76e27f3f17f5bca794. The commit prior to this does not expose the problem. [~gancho] also discovered that this problem is only triggered when accept thread is off (0). Also from [~gancho], when this reproduces, a command like e.g. this will fail the handshake completely (no ciphers): {code} openssl s_client -connect 10.1.2.3:443 -tls1 -servername some.host.com {code} Also, since this only happens with accept thread off (0), which implies accept on every ET_NET thread, maybe there's some sort of race condition going on here? That's just a wild speculation though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3597) TLS can fail accept / handshake since commit 2a8bb593fd
[ https://issues.apache.org/jira/browse/TS-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3597: --- Attachment: TS-3597.diff Found the problem. We were assuming that the SSLNetVConnection object was initialized when it was allocated. In the case of the accept thread this was the case because the global allocated was used. But without the accept thread, the thread allocator is used and THREAD_ALLOC was called instead of THREAD_ALLOC_INIT. If the object came off the free list, the sslHandshakeHookState variable was not in the initial state so the certificate selection did not occur as designed. TS-3597.diff includes the fix for this and the related missing lock (tracked as a separate bug). Will work on getting both fixes pushed. TLS can fail accept / handshake since commit 2a8bb593fd --- Key: TS-3597 URL: https://issues.apache.org/jira/browse/TS-3597 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Critical Fix For: 6.0.0 Attachments: TS-3597.diff At least under certain conditions (slightly unclear,but possible a race with multiple NUMA nodes), we fail to accept / TLS handshake. I've tracked this down to the commit from 2a8bb593fdd7ca9125efad76e27f3f17f5bca794. The commit prior to this does not expose the problem. [~gancho] also discovered that this problem is only triggered when accept thread is off (0). Also from [~gancho], when this reproduces, a command like e.g. this will fail the handshake completely (no ciphers): {code} openssl s_client -connect 10.1.2.3:443 -tls1 -servername some.host.com {code} Also, since this only happens with accept thread off (0), which implies accept on every ET_NET thread, maybe there's some sort of race condition going on here? That's just a wild speculation though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TS-3604) Transparent mode does not work when accept_threads set to 0
[ https://issues.apache.org/jira/browse/TS-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs closed TS-3604. -- Resolution: Fixed Transparent mode does not work when accept_threads set to 0 --- Key: TS-3604 URL: https://issues.apache.org/jira/browse/TS-3604 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Susan Hinrichs Assignee: Alan M. Carroll If you have transparency enabled on your port and you disable the dedicated accept_threads, the TCP connection does not complete for HTTP and HTTPS traffic. Enabling the accept_thread causes traffic to flow again. Traffic via a remap rule also works (once you apply the fix to TS-3603). Looking at the packet capture, the client sends SYN's but they are never responded to. It appears that the listen doesn't really get set up in this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3596) TSHttpTxnPluginTagGet() returns fetchSM over H2
[ https://issues.apache.org/jira/browse/TS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545757#comment-14545757 ] Susan Hinrichs edited comment on TS-3596 at 5/15/15 4:35 PM: - I don't think there is one. Eric Schwartz and Francious Pesce are working on this issue for internal core access to this information. Probably better revealed as a TSHttpSsnPluginTagGet() if such a thing exists. was (Author: shinrich): I don't think there is one. [~Eric Schwartz e...@yahoo-inc.com] and [~Francios Pesce fpe...@yahoo-inc.com] are working on this issue for internal access. Probably better revealed as a TSHttpSsnPluginTagGet() if such a thing exists. TSHttpTxnPluginTagGet() returns fetchSM over H2 - Key: TS-3596 URL: https://issues.apache.org/jira/browse/TS-3596 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Scott Beardsley Fix For: 6.0.0 This should probably return something else, right? Maybe HTTP2 instead? We would like a way to identify H2 requests from SPDY and/or H1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3606) Add configuration option to allow server context per thread
[ https://issues.apache.org/jira/browse/TS-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548126#comment-14548126 ] Susan Hinrichs commented on TS-3606: Depends on the solution. It will likely result in increased memory usage. Probably fine for small ssl_multicert.configs, but perhaps not good for really large sets of certificates. Add configuration option to allow server context per thread --- Key: TS-3606 URL: https://issues.apache.org/jira/browse/TS-3606 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Bryan Call Assignee: Bryan Call Fix For: 6.0.0 Attachments: TS-3606-hack-for-53.diff This was recommended by John Foley (OpenSSL developer) to reduce lock contention. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3554) ATS memory leak reloading ssl_multicert.config with many ssl cert configs
[ https://issues.apache.org/jira/browse/TS-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548188#comment-14548188 ] Susan Hinrichs commented on TS-3554: Agreed on ref-counting the parent session_cache object. That should make accesses in the callbacks safe. If we do the ref-counting, I don't see the need for rw-locks. If you have a reference to the old session cache, you continue to use that to fetch out the reconstituted SSL_SESSION object. When you are done, you drop the reference, and the old session cache goes away, taking the serialized forms of the old sessions with it. I assume the openssl framework takes care of freeing up the SSL_SESSION object when it is done with it. We could try to be clever and only reinitialize if the session table characteristics change. But if the certs change, perhaps that isn't safe either. Easiest to just always reset I suppose. ATS memory leak reloading ssl_multicert.config with many ssl cert configs - Key: TS-3554 URL: https://issues.apache.org/jira/browse/TS-3554 Project: Traffic Server Issue Type: Bug Components: Configuration, Core, SSL Reporter: Steven Feltner Assignee: Susan Hinrichs Fix For: 6.0.0 Attachments: limit_session_cache_alloc.diff, ts-3554-53-2.diff, ts-3554-53.diff ATS will consume all available memory on a server with 128GB of RAM. @shinrich suspects it may be due to CertLookup table not being freed on a config reload. Our current process: - New cert comes in - ssl_multicert.config and remap.config updated - traffic_line -x This reload could occur as often as every 3 mins with 5000+ certs configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3578) Rearrange Client Session processing to give access to socket at SSN_CLOSE for all protocols
[ https://issues.apache.org/jira/browse/TS-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547320#comment-14547320 ] Susan Hinrichs commented on TS-3578: Incorporated comments from James. Rearrange Client Session processing to give access to socket at SSN_CLOSE for all protocols --- Key: TS-3578 URL: https://issues.apache.org/jira/browse/TS-3578 Project: Traffic Server Issue Type: Improvement Components: HTTP, HTTP/2, SPDY Reporter: Susan Hinrichs Assignee: Susan Hinrichs Labels: review Fix For: 6.0.0 I wanted to use the tcpinfo plugin to look at the kernel measured RTT. Unfortunately, there was really only visibility for HTTP/1.x. Not HTTP/2 or SPDY. In the H2 and SPDY cases, the underlying NetVC is a PluginVC and does not have access to the underlying socket. With HTTP/2, the SSN_CLOSE hook would trigger, but by the time the SSN_CLOSE hook would go off, the netVC had already been closed. I propose making the following changes. 1. Make SpdyClientSession a subclass of ProxyClientSession, so SSN_CLOSE_HOOK can be triggered there too. 2. Rearrange the hook calling and net vc close so the SSN_CLOSE hook is called before the net vc is closed. I've made both changes on my dev build, and in my simple tests, the tcpinfo plugin is recording times for traffic on top of HTTP/1.1, SPDY, and HTTP/2. Since this involves rearranging some of the bowels of the protocol processing, I'll set up a pull request for broader review. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3612) Restructure Proxy Client Sessions to support transaction oriented Sessions execute transaction hooks and connection oriented Sessions execute session hooks
Susan Hinrichs created TS-3612: -- Summary: Restructure Proxy Client Sessions to support transaction oriented Sessions execute transaction hooks and connection oriented Sessions execute session hooks Key: TS-3612 URL: https://issues.apache.org/jira/browse/TS-3612 Project: Traffic Server Issue Type: Improvement Components: HTTP, HTTP/2, SPDY Reporter: Susan Hinrichs In the current code, transaction and session hooks don't have access to H2 and SPDY session data. This was partially addressed by TS-3578. There was a discussion on the mailing list, and the consensus was that session hooks should be invoked on session-oriented sessions (H2, SPDY, and native HTTP/1.x) and transaction hooks should be invoked on transaction-oriented sessions. http://dev.trafficserver.apache.narkive.com/OX9XK0xn/spdy-h2-and-session-hooks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3612) Restructure Proxy Client Sessions to support transaction oriented Sessions execute transaction hooks and connection oriented Sessions execute session hooks
[ https://issues.apache.org/jira/browse/TS-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3612: --- Fix Version/s: 6.0.0 Restructure Proxy Client Sessions to support transaction oriented Sessions execute transaction hooks and connection oriented Sessions execute session hooks --- Key: TS-3612 URL: https://issues.apache.org/jira/browse/TS-3612 Project: Traffic Server Issue Type: Improvement Components: HTTP, HTTP/2, SPDY Reporter: Susan Hinrichs Fix For: 6.0.0 In the current code, transaction and session hooks don't have access to H2 and SPDY session data. This was partially addressed by TS-3578. There was a discussion on the mailing list, and the consensus was that session hooks should be invoked on session-oriented sessions (H2, SPDY, and native HTTP/1.x) and transaction hooks should be invoked on transaction-oriented sessions. http://dev.trafficserver.apache.narkive.com/OX9XK0xn/spdy-h2-and-session-hooks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3578) Rearrange Client Session processing to give access to socket at SSN_CLOSE for all protocols
[ https://issues.apache.org/jira/browse/TS-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547312#comment-14547312 ] Susan Hinrichs commented on TS-3578: From the mailing list, there appears to be consensus on option 1 http://dev.trafficserver.apache.narkive.com/OX9XK0xn/spdy-h2-and-session-hooks. I will open a new bug to track the general solution. Rearrange Client Session processing to give access to socket at SSN_CLOSE for all protocols --- Key: TS-3578 URL: https://issues.apache.org/jira/browse/TS-3578 Project: Traffic Server Issue Type: Improvement Components: HTTP, HTTP/2, SPDY Reporter: Susan Hinrichs Assignee: Susan Hinrichs Labels: review Fix For: 6.0.0 I wanted to use the tcpinfo plugin to look at the kernel measured RTT. Unfortunately, there was really only visibility for HTTP/1.x. Not HTTP/2 or SPDY. In the H2 and SPDY cases, the underlying NetVC is a PluginVC and does not have access to the underlying socket. With HTTP/2, the SSN_CLOSE hook would trigger, but by the time the SSN_CLOSE hook would go off, the netVC had already been closed. I propose making the following changes. 1. Make SpdyClientSession a subclass of ProxyClientSession, so SSN_CLOSE_HOOK can be triggered there too. 2. Rearrange the hook calling and net vc close so the SSN_CLOSE hook is called before the net vc is closed. I've made both changes on my dev build, and in my simple tests, the tcpinfo plugin is recording times for traffic on top of HTTP/1.1, SPDY, and HTTP/2. Since this involves rearranging some of the bowels of the protocol processing, I'll set up a pull request for broader review. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3606) Add configuration option to allow server context per thread
[ https://issues.apache.org/jira/browse/TS-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3606: --- Attachment: TS-3606-hack-for-53.diff TS-3606-hack-for-53.diff contains diffs against the 5.3.x branch that loads the ssl_multicert file for each thread. Likely not the right long term solution. And this hack does not handle reloads of ssl_multicert.config, but it should be functional enough to run some tests to give you an idea of whether moving contexts to per thread is worth the effort in reduced lock contention. If so, then we can figure out the right solution. Add configuration option to allow server context per thread --- Key: TS-3606 URL: https://issues.apache.org/jira/browse/TS-3606 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Bryan Call Attachments: TS-3606-hack-for-53.diff This was recommended by John Foley (OpenSSL developer) to reduce lock contention. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3596) TSHttpTxnPluginTagGet() returns fetchSM over H2
[ https://issues.apache.org/jira/browse/TS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545757#comment-14545757 ] Susan Hinrichs commented on TS-3596: I don't think there is one. [~Eric Schwartz e...@yahoo-inc.com] and [~Francios Pesce fpe...@yahoo-inc.com] are working on this issue for internal access. Probably better revealed as a TSHttpSsnPluginTagGet() if such a thing exists. TSHttpTxnPluginTagGet() returns fetchSM over H2 - Key: TS-3596 URL: https://issues.apache.org/jira/browse/TS-3596 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Scott Beardsley Fix For: 6.0.0 This should probably return something else, right? Maybe HTTP2 instead? We would like a way to identify H2 requests from SPDY and/or H1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3521) Several web sites don't load when HTTP/2 is enabled in ATS
Susan Hinrichs created TS-3521: -- Summary: Several web sites don't load when HTTP/2 is enabled in ATS Key: TS-3521 URL: https://issues.apache.org/jira/browse/TS-3521 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Susan Hinrichs I'm running in transparent forward proxy mode. If I don't have HTTP2 enabled, I can load the following sites from my Firefox 36.04 client running on Win 7 http://busey.com (which redirects to URLs under https://www.busey.com) https://www.google.com When I enable http2, these sites don't load. My Firefox tab stays empty (or shows the previous site). If I turn on debug http2, I see the following from ATS For busey {code} [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49851 transport type = 4 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] session born, netvc 0x7fffe8018840 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session closed [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session destroy {code} For https://www.google.com {code} [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49852 transport type = 4 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] session born, netvc 0x7fffe8018840 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session closed [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session destroy {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3521) Several web sites don't load when HTTP/2 is enabled in ATS
[ https://issues.apache.org/jira/browse/TS-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494423#comment-14494423 ] Susan Hinrichs commented on TS-3521: Doing a more complete debug capture staring with a completely new HTTP/2 connection attempting to contact https://www.google.com {code} [Apr 14 12:13:11.289] Server {0x740ad700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49866 transport type = 4 [Apr 14 12:13:11.289] Server {0x740ad700} DEBUG: (http2_cs) [118] session born, netvc 0x7fffe8018840 [Apr 14 12:13:11.290] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] received connection preface [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_start_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] receiving frame header [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] frame header length=12, type=4, flags=0x0, streamid=0 [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_complete_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] completed frame read, 34 bytes available [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] Received SETTINGS frame. [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] setting param=4 value=131072 [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] setting param=5 value=16384 [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_start_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] receiving frame header [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] frame header length=4, type=8, flags=0x0, streamid=0 [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_complete_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] completed frame read, 13 bytes available [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] Received WINDOW_UPDATE frame. [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_start_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] receiving frame header [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] frame header length=0, type=4, flags=0x1, streamid=0 [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_complete_frame_read, VC_EVENT_READ_READY] [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] completed frame read, 0 bytes available [Apr 14 12:13:11.310] Server {0x740ad700} DEBUG: (http2_cs) [118] Received SETTINGS frame. [Apr 14 12:13:28.553] Server {0x73fac700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe801a580] accepted connection from 192.168.126.130:49877 transport type = 4 [Apr 14 12:13:28.553] Server {0x73fac700} DEBUG: (http2_cs) [129] session born, netvc 0x7fffe801a580 [Apr 14 12:13:28.553] Server {0x73fac700} DEBUG: (http2_cs) [129] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:13:28.556] Server {0x73fac700} DEBUG: (http2_cs) [129] session closed [Apr 14 12:13:28.556] Server {0x73fac700} DEBUG: (http2_cs) [129] session destroy [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_start_frame_read, VC_EVENT_READ_READY] [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] receiving frame header [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] frame header length=8, type=6, flags=0x0, streamid=0 [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_complete_frame_read, VC_EVENT_READ_READY] [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] completed frame read, 8 bytes available [Apr 14 12:14:09.504] Server {0x740ad700} DEBUG: (http2_cs) [118] Received PING frame. [Apr 14 12:15:07.512] Server {0x740ad700} DEBUG: (http2_cs) [118] [Http2ClientSession::state_start_frame_read, VC_EVENT_READ_READY] [Apr 14 12:15:07.512] Server {0x740ad700} DEBUG: (http2_cs) [118] receiving frame header [Apr 14 12:15:07.512] Server {0x740ad700} DEBUG: (http2_cs) [118] frame header length=8,
[jira] [Commented] (TS-3521) Several web sites don't load when HTTP/2 is enabled in ATS
[ https://issues.apache.org/jira/browse/TS-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498277#comment-14498277 ] Susan Hinrichs commented on TS-3521: Things seemed to have healed themselves. I think my browsers updated. For a while earlier today and later yesterday, none of my standard test sites were working in firefox or chrome. They were fetching via nghttp. On chrome, I was seeing the err_spdy_inadequate_transport_security error code. Several web sites don't load when HTTP/2 is enabled in ATS -- Key: TS-3521 URL: https://issues.apache.org/jira/browse/TS-3521 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Susan Hinrichs Fix For: 6.0.0 I'm running in transparent forward proxy mode. If I don't have HTTP2 enabled, I can load the following sites from my Firefox 36.04 client running on Win 7 http://busey.com (which redirects to URLs under https://www.busey.com) https://www.google.com When I enable http2, these sites don't load. My Firefox tab stays empty (or shows the previous site). If I turn on debug http2, I see the following from ATS For busey {code} [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49851 transport type = 4 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] session born, netvc 0x7fffe8018840 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session closed [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session destroy {code} For https://www.google.com {code} [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49852 transport type = 4 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] session born, netvc 0x7fffe8018840 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session closed [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session destroy {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3521) Several web sites don't load when HTTP/2 is enabled in ATS
[ https://issues.apache.org/jira/browse/TS-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498273#comment-14498273 ] Susan Hinrichs commented on TS-3521: Possibly related. Several web sites don't load when HTTP/2 is enabled in ATS -- Key: TS-3521 URL: https://issues.apache.org/jira/browse/TS-3521 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Susan Hinrichs Fix For: 6.0.0 I'm running in transparent forward proxy mode. If I don't have HTTP2 enabled, I can load the following sites from my Firefox 36.04 client running on Win 7 http://busey.com (which redirects to URLs under https://www.busey.com) https://www.google.com When I enable http2, these sites don't load. My Firefox tab stays empty (or shows the previous site). If I turn on debug http2, I see the following from ATS For busey {code} [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49851 transport type = 4 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] session born, netvc 0x7fffe8018840 [Apr 14 12:05:46.544] Server {0x73fac700} DEBUG: (http2_cs) [102] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session closed [Apr 14 12:05:46.549] Server {0x73fac700} DEBUG: (http2_cs) [102] session destroy {code} For https://www.google.com {code} [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_seq) [HttpSessionAccept2:mainEvent 0x7fffe8018840] accepted connection from 192.168.126.130:49852 transport type = 4 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] session born, netvc 0x7fffe8018840 [Apr 14 12:06:49.201] Server {0x73eab700} DEBUG: (http2_cs) [103] [Http2ClientSession::state_read_connection_preface, VC_EVENT_READ_READY] [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session closed [Apr 14 12:06:49.214] Server {0x73eab700} DEBUG: (http2_cs) [103] session destroy {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TS-3528) Create a global for the ticket_key_name from ssl_multicert.config
[ https://issues.apache.org/jira/browse/TS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3528: -- Assignee: Susan Hinrichs Create a global for the ticket_key_name from ssl_multicert.config - Key: TS-3528 URL: https://issues.apache.org/jira/browse/TS-3528 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 The current implementation requires that you specify the ticket_key_name for each entry in the ssl_multicert.config file. In a simple case, you only want to specify one key for all the domains that the ATS device supports. To make that simple case easy, we will add a default ticket_key_name in the records.config. This is related to TS-3371 which proposes adding a default ssl_ticket_enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3528) Create a global for the ticket_key_name from ssl_multicert.config
Susan Hinrichs created TS-3528: -- Summary: Create a global for the ticket_key_name from ssl_multicert.config Key: TS-3528 URL: https://issues.apache.org/jira/browse/TS-3528 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs The current implementation requires that you specify the ticket_key_name for each entry in the ssl_multicert.config file. In a simple case, you only want to specify one key for all the domains that the ATS device supports. To make that simple case easy, we will add a default ticket_key_name in the records.config. This is related to TS-3371 which proposes adding a default ssl_ticket_enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3527) Add plugin hooks for SSL Session state access
[ https://issues.apache.org/jira/browse/TS-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3527: --- Fix Version/s: 6.0.0 Add plugin hooks for SSL Session state access - Key: TS-3527 URL: https://issues.apache.org/jira/browse/TS-3527 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 Add plugin hook functionality to allow hook into the SSL session state lifecycle. This enables session state communication between ATS deployments and additional statistical analysis. One API proposal is at http://network-geographics.com/ats/docs/ssl-session-api.en.html [~briang] suggested merging this with the existing TS_SSL_SNI_HOOK. Will have to look in more detail with the openssl implementations to see if we can hook into to a single point that has access to the functionality needed by both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TS-3527) Add plugin hooks for SSL Session state access
[ https://issues.apache.org/jira/browse/TS-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3527: -- Assignee: Susan Hinrichs Add plugin hooks for SSL Session state access - Key: TS-3527 URL: https://issues.apache.org/jira/browse/TS-3527 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 Add plugin hook functionality to allow hook into the SSL session state lifecycle. This enables session state communication between ATS deployments and additional statistical analysis. One API proposal is at http://network-geographics.com/ats/docs/ssl-session-api.en.html [~briang] suggested merging this with the existing TS_SSL_SNI_HOOK. Will have to look in more detail with the openssl implementations to see if we can hook into to a single point that has access to the functionality needed by both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3527) Add plugin hooks for SSL Session state access
Susan Hinrichs created TS-3527: -- Summary: Add plugin hooks for SSL Session state access Key: TS-3527 URL: https://issues.apache.org/jira/browse/TS-3527 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs Add plugin hook functionality to allow hook into the SSL session state lifecycle. This enables session state communication between ATS deployments and additional statistical analysis. One API proposal is at http://network-geographics.com/ats/docs/ssl-session-api.en.html [~briang] suggested merging this with the existing TS_SSL_SNI_HOOK. Will have to look in more detail with the openssl implementations to see if we can hook into to a single point that has access to the functionality needed by both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3528) Create a global for the ticket_key_name from ssl_multicert.config
[ https://issues.apache.org/jira/browse/TS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3528: --- Fix Version/s: 6.0.0 Create a global for the ticket_key_name from ssl_multicert.config - Key: TS-3528 URL: https://issues.apache.org/jira/browse/TS-3528 Project: Traffic Server Issue Type: Improvement Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 The current implementation requires that you specify the ticket_key_name for each entry in the ssl_multicert.config file. In a simple case, you only want to specify one key for all the domains that the ATS device supports. To make that simple case easy, we will add a default ticket_key_name in the records.config. This is related to TS-3371 which proposes adding a default ssl_ticket_enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3529) Add a config option to allow ATS to start even if some certificates are bad
[ https://issues.apache.org/jira/browse/TS-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499859#comment-14499859 ] Susan Hinrichs commented on TS-3529: Issue was identified by [~reveller] Add a config option to allow ATS to start even if some certificates are bad --- Key: TS-3529 URL: https://issues.apache.org/jira/browse/TS-3529 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs In fixing TS-3329, we changed the ATS start up behavior to fail to start if any of the entries of ssl_multicert.config fails to load. This changes the functionality of ATS in from 5.2 to 5.3. For many/most use cases, this is a desirable change. However, for some use cases, you want to serve and start up even if some of the entries fail to load. We will add a records config entry proxy.config.ssl.server.multicert.exit_on_load_fail It will default to 1 on 5.x. May want to change the default to 0 when we move to 6.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3529) Add a config option to allow ATS to start even if some certificates are bad
Susan Hinrichs created TS-3529: -- Summary: Add a config option to allow ATS to start even if some certificates are bad Key: TS-3529 URL: https://issues.apache.org/jira/browse/TS-3529 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs In fixing TS-3329, we changed the ATS start up behavior to fail to start if any of the entries of ssl_multicert.config fails to load. This changes the functionality of ATS in from 5.2 to 5.3. For many/most use cases, this is a desirable change. However, for some use cases, you want to serve and start up even if some of the entries fail to load. We will add a records config entry proxy.config.ssl.server.multicert.exit_on_load_fail It will default to 1 on 5.x. May want to change the default to 0 when we move to 6.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3382) Complaints of CRYPTO_set_id_callback while compiling against openssl 1.1
[ https://issues.apache.org/jira/browse/TS-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491764#comment-14491764 ] Susan Hinrichs commented on TS-3382: Ok. I was confused. We can set up the callback function to use CRYPTO_THREADID_set_numeric to assign the pthread id. The callback seems to be used by openssl in computing various random values. SSLDiagnostic() was calling CRYPTO_thread_id() to fetch a thread ID value to print in the diagnostic value. You could call CRYPTO_THREADID_get_numeric() to get the previously registered thread ID, but we don't have a CRYPTO_THREADID object at this point. Just calling pthread_self() directly here too seems to be fine. If the move the lowest supported openssl version to something 1.0.0 or better, then we can leave this to be only version, since the CRYPTO_THREADID version of the APIs appeared in openssl 1.0.0 Complaints of CRYPTO_set_id_callback while compiling against openssl 1.1 - Key: TS-3382 URL: https://issues.apache.org/jira/browse/TS-3382 Project: Traffic Server Issue Type: Bug Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 CRYPTO_set_id_callback has been deprecated since openssl 1.0.0. Should update with the replacing call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3522) Seg Fault due to inactivity_cop after lost continutation from write_signal_and_update
[ https://issues.apache.org/jira/browse/TS-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501072#comment-14501072 ] Susan Hinrichs edited comment on TS-3522 at 4/18/15 3:49 AM: - After looking at a few more cores, we have a theory on the scenario and a patch for 5.2 attached here as inactivity_crash.diff. Here is our scenario. Client connection comes in. A transaction is successfully completed. The client has keep alive set. So the VC and client session is saved, but the read and write structures are not cleared. Another request comes in. A read is set up immediately, but things stall before the write (server/cache response) is set up. There are many posts in this workload, so stalling out during a longish post is feasible. When the inactivity timer triggers, the write state for the first transaction is still laying around pointing to the now deleted first state machine. The patch clears the write structure as the VC and client session are saved aside. was (Author: shinrich): After looking at a few more cores, we have a theory on the scenario and a patch attached here as inactivity_crash.diff. Here is our scenario. Client connection comes in. A transaction is successfully completed. The client has keep alive set. So the VC and client session is saved, but the read and write structures are not cleared. Another request comes in. A read is set up immediately, but things stall before the write (server/cache response) is set up. There are many posts in this workload, so stalling out during a longish post is feasible. When the inactivity timer triggers, the write state for the first transaction is still laying around pointing to the now deleted first state machine. The patch clears the write structure as the VC and client session are saved aside. Seg Fault due to inactivity_cop after lost continutation from write_signal_and_update - Key: TS-3522 URL: https://issues.apache.org/jira/browse/TS-3522 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Steven Feltner Assignee: Alan M. Carroll Fix For: 6.0.0 Attachments: inactivity_crash.diff (gdb) bt full #0 0x006ec51e in handleEvent (event=105, vc=0x2b1c900461e0) at ../../iocore/eventsystem/I_Continuation.h:146 No locals. #1 write_signal_and_update (event=105, vc=0x2b1c900461e0) at UnixNetVConnection.cc:154 No locals. #2 0x006ec837 in UnixNetVConnection::mainEvent (this=0x2b1c900461e0, event=value optimized out, e=value optimized out) at UnixNetVConnection.cc:1089 wlock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} signal_event = 105 next_activity_timeout_at = 0 t = 0x0 hlock = {m = {m_ptr = 0x1430c30}, lock_acquired = true} rlock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} signal_timeout = 0x2b1c6b9ddc30 reader_cont = 0x0 writer_cont = 0x2b1d28051d48 signal_timeout_at = 0x2b1c900463f8 #3 0x006e5061 in handleEvent (this=0x14519d0, event=value optimized out, e=0x15792d0) at ../../iocore/eventsystem/I_Continuation.h:146 No locals. #4 InactivityCop::check_inactivity (this=0x14519d0, event=value optimized out, e=0x15792d0) at UnixNet.cc:80 vc = 0x2b1c900461e0 lock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} now = 1428965697221995775 nh = 0x2b1c695bea30 __func__ = check_inactivity #5 0x0070f628 in handleEvent (this=0x2b1c695bb010, e=0x15792d0, calling_code=2) at I_Continuation.h:146 No locals. #6 EThread::process_event (this=0x2b1c695bb010, e=0x15792d0, calling_code=2) at UnixEThread.cc:144 c_temp = 0x14519d0 lock = {m = {m_ptr = 0x1430c30}, lock_acquired = true} #7 0x007101c1 in EThread::execute (this=0x2b1c695bb010) at UnixEThread.cc:223 done_one = true e = value optimized out NegativeQueue = {DLLEvent, Event::Link_link = {head = 0x1579330}, tail = 0x1579330} next_time = 1428963217761407178 #8 0x0070ea52 in spawn_thread_internal (a=0x144a330) at Thread.cc:88 p = 0x144a330 #9 0x00383e8079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #10 0x00383e0e88fd in clone () from /lib64/libc.so.6 No symbol table info available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3522) Seg Fault due to inactivity_cop after lost continutation from write_signal_and_update
[ https://issues.apache.org/jira/browse/TS-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3522: --- Attachment: inactivity_crash.diff After looking at a few more cores, we have a theory on the scenario and a patch attached here as inactivity_crash.diff. Here is our scenario. Client connection comes in. A transaction is successfully completed. The client has keep alive set. So the VC and client session is saved, but the read and write structures are not cleared. Another request comes in. A read is set up immediately, but things stall before the write (server/cache response) is set up. There are many posts in this workload, so stalling out during a longish post is feasible. When the inactivity timer triggers, the write state for the first transaction is still laying around pointing to the now deleted first state machine. The patch clears the write structure as the VC and client session are saved aside. Seg Fault due to inactivity_cop after lost continutation from write_signal_and_update - Key: TS-3522 URL: https://issues.apache.org/jira/browse/TS-3522 Project: Traffic Server Issue Type: Bug Components: Network Reporter: Steven Feltner Assignee: Alan M. Carroll Fix For: 6.0.0 Attachments: inactivity_crash.diff (gdb) bt full #0 0x006ec51e in handleEvent (event=105, vc=0x2b1c900461e0) at ../../iocore/eventsystem/I_Continuation.h:146 No locals. #1 write_signal_and_update (event=105, vc=0x2b1c900461e0) at UnixNetVConnection.cc:154 No locals. #2 0x006ec837 in UnixNetVConnection::mainEvent (this=0x2b1c900461e0, event=value optimized out, e=value optimized out) at UnixNetVConnection.cc:1089 wlock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} signal_event = 105 next_activity_timeout_at = 0 t = 0x0 hlock = {m = {m_ptr = 0x1430c30}, lock_acquired = true} rlock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} signal_timeout = 0x2b1c6b9ddc30 reader_cont = 0x0 writer_cont = 0x2b1d28051d48 signal_timeout_at = 0x2b1c900463f8 #3 0x006e5061 in handleEvent (this=0x14519d0, event=value optimized out, e=0x15792d0) at ../../iocore/eventsystem/I_Continuation.h:146 No locals. #4 InactivityCop::check_inactivity (this=0x14519d0, event=value optimized out, e=0x15792d0) at UnixNet.cc:80 vc = 0x2b1c900461e0 lock = {m = {m_ptr = 0x2b1c90117dd0}, lock_acquired = true} now = 1428965697221995775 nh = 0x2b1c695bea30 __func__ = check_inactivity #5 0x0070f628 in handleEvent (this=0x2b1c695bb010, e=0x15792d0, calling_code=2) at I_Continuation.h:146 No locals. #6 EThread::process_event (this=0x2b1c695bb010, e=0x15792d0, calling_code=2) at UnixEThread.cc:144 c_temp = 0x14519d0 lock = {m = {m_ptr = 0x1430c30}, lock_acquired = true} #7 0x007101c1 in EThread::execute (this=0x2b1c695bb010) at UnixEThread.cc:223 done_one = true e = value optimized out NegativeQueue = {DLLEvent, Event::Link_link = {head = 0x1579330}, tail = 0x1579330} next_time = 1428963217761407178 #8 0x0070ea52 in spawn_thread_internal (a=0x144a330) at Thread.cc:88 p = 0x144a330 #9 0x00383e8079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #10 0x00383e0e88fd in clone () from /lib64/libc.so.6 No symbol table info available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3667) SSL Handhake read does not correctly handle EOF and error cases
[ https://issues.apache.org/jira/browse/TS-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3667. Resolution: Fixed SSL Handhake read does not correctly handle EOF and error cases --- Key: TS-3667 URL: https://issues.apache.org/jira/browse/TS-3667 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Attachments: ts-3667.diff Reported by [~esproul] and postwait. The return value of SSLNetVConnection::read_raw_data() is being ignored. So EOF and errors are not terminated, but rather spin until the inactivity timeout is reached. EAGAIN is not being descheduled until more data is available. This results in higher CPU utilization and hitting the SSL_error() function much more than it needs to be hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3667) SSL Handhake read does not correctly handle EOF and error cases
[ https://issues.apache.org/jira/browse/TS-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3667: --- Attachment: ts-3667.diff I tried the ts-3667.diff patch against master on my dev machine. I've triggered the EOF case, But not the -1 return cases. I'll work on that some more, but I wanted to make the patch available to those seeing the problem sooner. SSL Handhake read does not correctly handle EOF and error cases --- Key: TS-3667 URL: https://issues.apache.org/jira/browse/TS-3667 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Attachments: ts-3667.diff Reported by [~esproul] and postwait. The return value of SSLNetVConnection::read_raw_data() is being ignored. So EOF and errors are not terminated, but rather spin until the inactivity timeout is reached. EAGAIN is not being descheduled until more data is available. This results in higher CPU utilization and hitting the SSL_error() function much more than it needs to be hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TS-3667) SSL Handhake read does not correctly handle EOF and error cases
[ https://issues.apache.org/jira/browse/TS-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3667: -- Assignee: Susan Hinrichs SSL Handhake read does not correctly handle EOF and error cases --- Key: TS-3667 URL: https://issues.apache.org/jira/browse/TS-3667 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Reported by [~esproul] and postwait. The return value of SSLNetVConnection::read_raw_data() is being ignored. So EOF and errors are not terminated, but rather spin until the inactivity timeout is reached. EAGAIN is not being descheduled until more data is available. This results in higher CPU utilization and hitting the SSL_error() function much more than it needs to be hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3667) SSL Handhake read does not correctly handle EOF and error cases
Susan Hinrichs created TS-3667: -- Summary: SSL Handhake read does not correctly handle EOF and error cases Key: TS-3667 URL: https://issues.apache.org/jira/browse/TS-3667 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Reported by [~esproul] and postwait. The return value of SSLNetVConnection::read_raw_data() is being ignored. So EOF and errors are not terminated, but rather spin until the inactivity timeout is reached. EAGAIN is not being descheduled until more data is available. This results in higher CPU utilization and hitting the SSL_error() function much more than it needs to be hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3382) Complaints of CRYPTO_set_id_callback while compiling against openssl 1.1
[ https://issues.apache.org/jira/browse/TS-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3382. Resolution: Fixed Complaints of CRYPTO_set_id_callback while compiling against openssl 1.1 - Key: TS-3382 URL: https://issues.apache.org/jira/browse/TS-3382 Project: Traffic Server Issue Type: Bug Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 CRYPTO_set_id_callback has been deprecated since openssl 1.0.0. Should update with the replacing call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3665: --- Attachment: ts-3665-2.diff ts-3665-2.diff contains an updated patch of the cache read vc cleanup suggested by [~sudheerv]. It also updates the debug asserts to correctly test for the non null cache write vc's floating around in the follow redirect case. The cache read vc is not due to the ts-3661 fix. The cache write vc debug asserts are due to the ts-3661 fix. The fixes should definitely be pushed to master. The fix for ts-3661 should definitely be backported to 5.3.1. I think cleaning up the debug asserts should also be backported (the fix on this bug), but it is not such a big thing. I leave it for [~psudaemon] to decide. Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs Fix For: 6.0.0 Attachments: ts-3665-2.diff, ts-3665.diff This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3640) Drupal auth fails with dda6814f07ee59c over SPDY
[ https://issues.apache.org/jira/browse/TS-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3640: --- Attachment: ts-3640.diff ts-3640.diff contains a patch that disables the close for write at the end of the post/redirect sequence. It isn't clear to me when it would be beneficial to keep the client connection open for reads after completing the post if the connection is marked close. I have not been able to exercise this code in my dev environment. [~zwoop] could you give this patch a try? If it doesn't help, I'll make a better effort in reproductin. Drupal auth fails with dda6814f07ee59c over SPDY Key: TS-3640 URL: https://issues.apache.org/jira/browse/TS-3640 Project: Traffic Server Issue Type: Bug Components: SPDY Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Blocker Fix For: 6.0.0 Attachments: ts-3640.diff With the patch from dda6814f07ee59c, when Drupal authenticates a user, it sends back a 302 redirect to that user's page. This seems to stall the SPDY session entirely (it stops dead in its track at this point). Backing out dda6814f07ee59c makes it work again. I've emailed some potentially sensitive traces directly to [~shinrich] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3378) SpdyRequest used after free()
[ https://issues.apache.org/jira/browse/TS-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3378. Resolution: Fixed This change at least narrows the crash window. @zwoop will keep an eye on this on docs.trafficserver.apache.org to see if it eliminates are reduces this crash. SpdyRequest used after free() - Key: TS-3378 URL: https://issues.apache.org/jira/browse/TS-3378 Project: Traffic Server Issue Type: Bug Components: SPDY Reporter: Leif Hedstrom Assignee: Susan Hinrichs Fix For: 6.0.0 I see this on our docs.ts machine: {code} ==1310==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110004fc974 at pc 0x7c2162 bp 0x7fff97c95010 sp 0x7fff97c95008 READ of size 1 at 0x6110004fc974 thread T0 ([ET_NET 0]) #0 0x7c2161 in spdy_process_fetch /usr/local/src/trafficserver/proxy/spdy/SpdyClientSession.cc:339 #1 0x7c2161 in SpdyClientSession::state_session_readwrite(int, void*) /usr/local/src/trafficserver/proxy/spdy/SpdyClientSession.cc:253 #2 0x4f1308 in Continuation::handleEvent(int, void*) ../iocore/eventsystem/I_Continuation.h:146 #3 0x4f1308 in FetchSM::InvokePluginExt(int) /usr/local/src/trafficserver/proxy/FetchSM.cc:250 #4 0x4f455a in FetchSM::fetch_handler(int, void*) /usr/local/src/trafficserver/proxy/FetchSM.cc:516 #5 0x59f737 in Continuation::handleEvent(int, void*) ../iocore/eventsystem/I_Continuation.h:146 #6 0x59f737 in PluginVC::process_write_side(bool) /usr/local/src/trafficserver/proxy/PluginVC.cc:519 #7 0x5aa2fd in PluginVC::main_handler(int, void*) /usr/local/src/trafficserver/proxy/PluginVC.cc:210 #8 0xc6aabe in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:146 #9 0xc6aabe in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:144 #10 0xc6d0d9 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:238 #11 0x498481 in main /usr/local/src/trafficserver/proxy/Main.cc:1759 #12 0x2b01d58c0af4 in __libc_start_main (/lib64/libc.so.6+0x21af4) #13 0x4ab124 (/opt/ats/bin/traffic_server+0x4ab124) 0x6110004fc974 is located 52 bytes inside of 224-byte region [0x6110004fc940,0x6110004fca20) freed by thread T0 ([ET_NET 0]) here: #0 0x2b01d1d2e1c7 in __interceptor_free ../../.././libsanitizer/asan/asan_malloc_linux.cc:62 #1 0x7c8433 in ClassAllocatorSpdyRequest::free(SpdyRequest*) ../../lib/ts/Allocator.h:138 #2 0x7c8433 in SpdyClientSession::cleanup_request(int) /usr/local/src/trafficserver/proxy/spdy/SpdyClientSession.h:146 #3 0x7c8433 in spdy_prepare_status_response_and_clean_request(SpdyClientSession*, int, char const*) /usr/local/src/trafficserver/proxy/spdy/SpdyCa llbacks.cc:85 #4 0x7c1094 in spdy_process_fetch /usr/local/src/trafficserver/proxy/spdy/SpdyClientSession.cc:354 #5 0x7c1094 in SpdyClientSession::state_session_readwrite(int, void*) /usr/local/src/trafficserver/proxy/spdy/SpdyClientSession.cc:253 #6 0x4f1c95 in Continuation::handleEvent(int, void*) ../iocore/eventsystem/I_Continuation.h:146 #7 0x4f1c95 in FetchSM::InvokePluginExt(int) /usr/local/src/trafficserver/proxy/FetchSM.cc:259 #8 0x4f2eaa in FetchSM::process_fetch_read(int) /usr/local/src/trafficserver/proxy/FetchSM.cc:465 #9 0x4f4542 in FetchSM::fetch_handler(int, void*) /usr/local/src/trafficserver/proxy/FetchSM.cc:514 #10 0x59e077 in Continuation::handleEvent(int, void*) ../iocore/eventsystem/I_Continuation.h:146 #11 0x59e077 in PluginVC::process_read_side(bool) /usr/local/src/trafficserver/proxy/PluginVC.cc:640 #12 0x5aab79 in PluginVC::main_handler(int, void*) /usr/local/src/trafficserver/proxy/PluginVC.cc:206 #13 0xc6aabe in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:146 #14 0xc6aabe in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:144 #15 0xc6d0d9 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:238 #16 0x498481 in main /usr/local/src/trafficserver/proxy/Main.cc:1759 #17 0x2b01d58c0af4 in __libc_start_main (/lib64/libc.so.6+0x21af4) previously allocated by thread T5 ([ET_NET 4]) here: #0 0x2b01d1d2e93b in __interceptor_posix_memalign ../../.././libsanitizer/asan/asan_malloc_linux.cc:130 #1 0x2b01d2c18309 in ats_memalign /usr/local/src/trafficserver/lib/ts/ink_memory.cc:96 #2 0x7c89ba in ClassAllocatorSpdyRequest::alloc() ../../lib/ts/Allocator.h:124 #3 0x7c89ba in spdy_on_ctrl_recv_callback(spdylay_session*, spdylay_frame_type, spdylay_frame*, void*)
[jira] [Assigned] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3665: -- Assignee: Susan Hinrichs Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 Attachments: ts-3665-2.diff, ts-3665.diff This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3656) Activating follow redirection in send server response hook does not work for post
Susan Hinrichs created TS-3656: -- Summary: Activating follow redirection in send server response hook does not work for post Key: TS-3656 URL: https://issues.apache.org/jira/browse/TS-3656 Project: Traffic Server Issue Type: Bug Components: HTTP Reporter: Susan Hinrichs If you have a plugin on the TS_HTTP_SEND_RESPONSE_HDR_HOOK, calls TSHttpTxnFollowRedirect(txn, 1), redirecting a POST request will fail. In the not so bad case, the POST request will be redirected to the new location, but the POST data will be lost. In the more bad case, ATS will crash. The issue is that the post_redirect buffers are freed early on. One could delay the post_redirect deallocation until later in the transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3476) Add a log tag for APLN/NPN negotiated protocol
[ https://issues.apache.org/jira/browse/TS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3476: --- Assignee: Eric Schwartz (was: Susan Hinrichs) Add a log tag for APLN/NPN negotiated protocol -- Key: TS-3476 URL: https://issues.apache.org/jira/browse/TS-3476 Project: Traffic Server Issue Type: New Feature Components: Logging, SSL Reporter: Leif Hedstrom Assignee: Eric Schwartz Fix For: 6.0.0 It seems crucial to be able to log which protocol handler was negotiated with ALPN (and perhaps NPN as long as we support it). This could simple be the string that was negotiated by the client/server in the TLS handshake. For example, with HTTP/2, it would be h2 (or h2-14 with some browsers). A suggested log tag name would be %alpn which seems clear enough to me and easy to remember :). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3680) Refine options for handling post and follow redirect
Susan Hinrichs created TS-3680: -- Summary: Refine options for handling post and follow redirect Key: TS-3680 URL: https://issues.apache.org/jira/browse/TS-3680 Project: Traffic Server Issue Type: Improvement Components: HTTP Reporter: Susan Hinrichs In the current code base (5.3.x), the follow redirect feature applies to POST methods as well as GET/HEAD methods. For POST redirects, the code saves aside a copy (reference counted) of the post data, so it can resend it later if the original server sends a redirect and the follow redirect feature is enabled. This logic has been in there at least through the 5.x code probably earlier. It has been pointed out that in some cases, replaying a POST in a redirect scenario may not be safe. The POST may have already caused a change before the redirect occurred. In other cases, following the redirect on a post may be just fine. If the origin server makes the redirect decision before anything is done on the post request, replaying the post request should be just fine. We should step back and determine if we want to reconsider how we handle follow redirects for POST methods. I see the following options. * Keep the current support and bug fix it as necessary (e.g. TS-3656) * Remove the post redirect support. * Add another config control to enable follow redirect only for GET/HEAD vs enabled following redirect for all methods, vs do not follow redirect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3656) Activating follow redirection in send server response hook does not work for post
[ https://issues.apache.org/jira/browse/TS-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579403#comment-14579403 ] Susan Hinrichs commented on TS-3656: I filed bug TS-3680 to hold the discussion on whether we should be following POST redirects or not. I think this is a broader issue than this issue which is just a bug fix on an existing feature. Activating follow redirection in send server response hook does not work for post - Key: TS-3656 URL: https://issues.apache.org/jira/browse/TS-3656 Project: Traffic Server Issue Type: Bug Components: HTTP Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 If you have a plugin on the TS_HTTP_SEND_RESPONSE_HDR_HOOK, calls TSHttpTxnFollowRedirect(txn, 1), redirecting a POST request will fail. In the not so bad case, the POST request will be redirected to the new location, but the POST data will be lost. In the more bad case, ATS will crash. The issue is that the post_redirect buffers are freed early on. One could delay the post_redirect deallocation until later in the transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573048#comment-14573048 ] Susan Hinrichs commented on TS-3665: Backed out the changes for TS-3585 and it made no difference. Not surprising, since the only functional difference in that patch affects if the cache is not enabled. In our case, the cache is enabled. It correctly directs the flow to do a cache lookup on the redirected target. Will spend a bit of time to determine what is opening the cache write vc. Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs Fix For: 6.0.0 This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3665: --- Attachment: ts-3665.diff With ts-3665.diff, I added a close_read on the cache_sm in the REDIRECT_READ case. I converted the assert in the do_cache_prepare_action to a test against NULL and cache close_write. This enables both of my URLS to be fetched without asserts. Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs Fix For: 6.0.0 Attachments: ts-3665.diff This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572947#comment-14572947 ] Susan Hinrichs commented on TS-3665: So in the follow redirect case, we should ignore any 30x response that happens to already be in the cache. Although we would have to look it up in the cache (as part of normal operating procedure) to know it was a 30x. So we do need to attempt to read from the cache for the initial request anyway. After that, it seems reasonable to not attempt to write to the cache until the final response that is sent back to the client. Again, we may want to be reading from the cache on the intermediates, in case this is the final location and has already been cached. Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572832#comment-14572832 ] Susan Hinrichs commented on TS-3665: BTW, in all if these scenarios we are running with follow redirect enabled. I have redirect retry set to 10. Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TS-3656) Activating follow redirection in send server response hook does not work for post
[ https://issues.apache.org/jira/browse/TS-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3656: -- Assignee: Susan Hinrichs Activating follow redirection in send server response hook does not work for post - Key: TS-3656 URL: https://issues.apache.org/jira/browse/TS-3656 Project: Traffic Server Issue Type: Bug Components: HTTP Reporter: Susan Hinrichs Assignee: Susan Hinrichs If you have a plugin on the TS_HTTP_SEND_RESPONSE_HDR_HOOK, calls TSHttpTxnFollowRedirect(txn, 1), redirecting a POST request will fail. In the not so bad case, the POST request will be redirected to the new location, but the POST data will be lost. In the more bad case, ATS will crash. The issue is that the post_redirect buffers are freed early on. One could delay the post_redirect deallocation until later in the transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3641) Drupal Auth does not seem to work with HTTP/2
[ https://issues.apache.org/jira/browse/TS-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578867#comment-14578867 ] Susan Hinrichs commented on TS-3641: [~zwoop] Does the fix for TS-3640 fix the HTTP/2 case as well? Drupal Auth does not seem to work with HTTP/2 - Key: TS-3641 URL: https://issues.apache.org/jira/browse/TS-3641 Project: Traffic Server Issue Type: Bug Components: HTTP/2 Reporter: Leif Hedstrom Fix For: 6.0.0 Using latest chrome, when authenticating to a Drupal site behind ATS, it fails to authenticate. It silently seems to just ignore the auth, and moves along unauthenticated. It's possible this is similar to TS-3640, but the fix from that Jira does not resolve the HTTP/2 issues. In fact, this problem exists all the way back to 5.3.0, so the fix here would also be a back port for 5.3.1 (or 5.3.2). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3640) Drupal auth fails with dda6814f07ee59c over SPDY
[ https://issues.apache.org/jira/browse/TS-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3640. Resolution: Fixed Drupal auth fails with dda6814f07ee59c over SPDY Key: TS-3640 URL: https://issues.apache.org/jira/browse/TS-3640 Project: Traffic Server Issue Type: Bug Components: SPDY Reporter: Leif Hedstrom Assignee: Susan Hinrichs Priority: Blocker Fix For: 6.0.0 Attachments: ts-3640.diff With the patch from dda6814f07ee59c, when Drupal authenticates a user, it sends back a 302 redirect to that user's page. This seems to stall the SPDY session entirely (it stops dead in its track at this point). Backing out dda6814f07ee59c makes it work again. I've emailed some potentially sensitive traces directly to [~shinrich] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3665) Redirect logic causing debug asserts and leaking cache_vc's
[ https://issues.apache.org/jira/browse/TS-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3665. Resolution: Fixed Redirect logic causing debug asserts and leaking cache_vc's --- Key: TS-3665 URL: https://issues.apache.org/jira/browse/TS-3665 Project: Traffic Server Issue Type: Bug Components: Cache Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 Attachments: ts-3665-2.diff, ts-3665.diff This is related to TS-3140 and TS-3661. I spent this morning reviewing the issue addressed by TS-3140 after the fixes for TS-3661 were put in place. TS-3140 addresses the issue when the 301 is in cache, but I'm seeing asserts for both 301's in cache and 301's not in cache. My first assert was line 109 in HttpCacheSM.cc line 109, ink_assert(cache_read_vc == NULL). I added a cache_sm.close_read() to the HttpTransact::SM_ACTION_REDIRECT_READ: case of HttpSM::handle_api_return. While only debug assert, if we ignore it we will reassign cache_read_vc without freeing the previous. I addressed this by adding cache_sm.close_read() to the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. My second assert is in HttpSM::do_cache_prepare_action (line 4446 of HttpSM.cc). Before the changes for TS-3661, it was expressing itself in SM_ACTION_CACHE_ISSUE_WRITE case of HttpSM::cache_write_state(). In this case, do_cache_prepare_action will open a new cache_write_vc overwriting the original and losing the cache_vc memory. The original fix to TS-3140 addressed this by adding a cache_sm.close_write in the SM_ACTION_REDIRECT_READ case of HttpSM::handle_api_return. But this caused problems of TS-3661 causing the originally selected cache key to be lost, but if you pass through this logic, I assume that the original cache write vc will be lost anyway. [~sudheerv] and [~zwoop] does this situation not happen in your redirect use cases? I'm afraid that I'm not following how the original cache key is preserved in the second cache open only if the first cache write open is not cleaned up. My test URLs are: curl -v --proxy localhost:80 http://whos.amung.us/cwidget/4s62rme9/007071fecc4e.png and curl -v --proxy localhost:80 http://docs.trafficserver.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3714) TS seems to stall between ua_begin and ua_first_read on some transactions resulting in high response times.
[ https://issues.apache.org/jira/browse/TS-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597807#comment-14597807 ] Susan Hinrichs commented on TS-3714: Good observation. If the mbuffer argument was NULL, do you think this would not be an issue then? TS seems to stall between ua_begin and ua_first_read on some transactions resulting in high response times. --- Key: TS-3714 URL: https://issues.apache.org/jira/browse/TS-3714 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.3.0 Reporter: Sudheer Vinukonda An example slow log showing very high *ua_first_read*. {code} ERROR: [8624075] Slow Request: client_ip: xx.xx.xx.xxx url: http://xx status: 200 unique id: bytes: 86 fd: 0 client state: 0 serv er state: 9 ua_begin: 0.000 ua_first_read: 42.224 ua_read_header_done: 42.224 cache_open_rea d_begin: 42.224 cache_open_read_end: 42.224 dns_lookup_begin: 42.224 dns_lookup_end: 42.224 server_connect: 42.224 server_first_read: 42.229 server_read_header_done: 42.229 server_clos e: 42.229 ua_begin_write: 42.229 ua_close: 42.229 sm_finish: 42.229 {code} Initially, I suspected that it might be caused by browser's connecting early before sending any bytes to TS. However, this seems to be happening too frequently and with unrealistically high delay between *ua_begin* and *ua_first_read*. I suspect it's caused due to the code that disables the read temporarily before calling *TXN_START* hook and re-enables it after the API call out. The read disable is done via a 0-byte *do_io_read* on the client vc, but, the problem is that a valid *mbuf* is still passed. Based on what I am seeing, it's possible this results in actually enabling the *read_vio* all the way uptil *ssl_read_from_net* for instance (if there's a race condition and there were bytes already from the client resulting in an epoll read event), which would finally disable the read since, the *ntodo* (nbytes) is 0. However, this may result in the epoll event being lost until a new i/o happens from the client. I'm trying out further experiments to confirm the theory. In most cases, the read buffer already has some bytes by the time the http session and http sm is created, which makes it just work. But, if there's a slight delay in the receiving bytes after making a connection, the epoll mechanism should read it, but, due to the way the temporary read disable is being done, the event may be lost. Some history on this issue - This issue has been a problem for a long time and affects both http and https requests. When this issue was first reported, our router operations team eventually closed it indicating that disabling *accept* threads resolved it ([~yzlai] also reported similar observations and conclusions). It's possible that the race condition window may be slightly reduced by disabling accept threads, but, to the overall performance and through put, accept threads are very important. I now notice that the issue still exists (regardless of whether or not accept threads are enabled/disabled) and am testing further to confirm the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3710) ASAN crash in TLS with 6.0.0
[ https://issues.apache.org/jira/browse/TS-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597648#comment-14597648 ] Susan Hinrichs commented on TS-3710: Does do_io_close() not cancel outstanding reads? ASAN crash in TLS with 6.0.0 Key: TS-3710 URL: https://issues.apache.org/jira/browse/TS-3710 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Fix For: 6.0.0 {code} ==18563==ERROR: AddressSanitizer: heap-use-after-free on address 0x606541a8 at pc 0xb9e0c2 bp 0x2ae1c3cac8d0 sp 0x2ae1c3cac8c8 READ of size 8 at 0x606541a8 thread T4 ([ET_NET 3]) #0 0xb9e0c1 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #1 0xb9e0c1 in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:139 #2 0xb9e0c1 in UnixNetVConnection::mainEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1108 #3 0xb7c47f in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #4 0xb7c47f in InactivityCop::check_inactivity(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:109 #5 0xc215ce in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #6 0xc215ce in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #7 0xc237c7 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:207 #8 0xc201e8 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85 #9 0x2ae1bc976df4 in start_thread (/lib64/libpthread.so.0+0x7df4) #10 0x2ae1be1df1ac in __clone (/lib64/libc.so.6+0xf61ac) 0x606541a8 is located 8 bytes inside of 56-byte region [0x606541a0,0x606541d8) freed by thread T4 ([ET_NET 3]) here: #0 0x2ae1ba573117 in operator delete(void*) ../../.././libsanitizer/asan/asan_new_delete.cc:81 #1 0xb5d53e in SSLNextProtocolTrampoline::ioCompletionEvent(int, void*) /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:89 #2 0xbb185f in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #3 0xbb185f in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:139 #4 0xbb185f in read_signal_done /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:200 #5 0xbb185f in UnixNetVConnection::readSignalDone(int, NetHandler*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:950 #6 0xb55a4d in SSLNetVConnection::net_read_io(NetHandler*, EThread*) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:480 #7 0xb7754c in NetHandler::mainNetEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:551 #8 0xc24459 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #9 0xc24459 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #10 0xc24459 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252 #11 0xc201e8 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85 #12 0x2ae1bc976df4 in start_thread (/lib64/libpthread.so.0+0x7df4) previously allocated by thread T4 ([ET_NET 3]) here: #0 0x2ae1ba572c9f in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cc:50 #1 0xb5c2bb in SSLNextProtocolAccept::mainEvent(int, void*) /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:134 #2 0xb87109 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #3 0xb87109 in NetAccept::acceptFastEvent(int, void*) /usr/local/src/trafficserver/iocore/net/UnixNetAccept.cc:466 #4 0xc24459 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #5 0xc24459 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #6 0xc24459 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252 #7 0xc201e8 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85 #8 0x2ae1bc976df4 in start_thread (/lib64/libpthread.so.0+0x7df4) Thread T4 ([ET_NET 3]) created by T0 ([ET_NET 0]) here: #0 0x2ae1ba54186a in __interceptor_pthread_create ../../.././libsanitizer/asan/asan_interceptors.cc:183 #1 0xc20e75 in ink_thread_create ../../lib/ts/ink_thread.h:150 #2 0xc20e75 in Thread::start(char const*, unsigned long, void*
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597653#comment-14597653 ] Susan Hinrichs commented on TS-3136: [~briang] and [~jacksontj] any comments on your experience with DHE? Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598099#comment-14598099 ] Susan Hinrichs commented on TS-3486: We are seeing a similar core in production. On one set of boxes it drops the core at a high frequency. I'm suspecting that the LAZY_BUF_ALLOC is not working quite right. I'm building an image with that disabled. It attempts to delay allocating buffer space until it is needed, but I don't think checks like buf.writer() check the lazy correctly. There are two spots where the LAZY_BUF_ALLOC is used. One spot immediately allocates a buffer. The other one in HttpServerSession appears to be the source of the buffer for our core. It does not allocate a buffer. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at
[jira] [Updated] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3486: --- Attachment: no_lazy.diff no_lazy.diff contains the 5.3.x patch to disable the LAZY_BUF_ALLOC feature that I suspect might be causing the most recent crash. I'm running this patch. [~esproul] and [~bettydreamit] give it a try if you have some time. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: no_lazy.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6919 #27 0x005b3f5f in
[jira] [Updated] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3486: --- Attachment: ts-3266-2.diff ts-3266-2.diff contains a patch which completes the job. It adjust the locks on the read.vio as it moves from being active to living in the session pool. The first ts-3266.diff patch adds logic to adjust the locks moving from pool to active. I've been running with this path in production for 5 hours without a crash. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: no_lazy.diff, ts-3266-2.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in
[jira] [Updated] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3486: --- Attachment: (was: no_lazy.diff) Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts-3266-2.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6919 #27 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #28 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #29 0x005b980b in
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594125#comment-14594125 ] Susan Hinrichs commented on TS-3136: And because you cannot have too much fun playing with data, here is the negotiated cipher table subtotaled by type of cipher |Cipher Group|% 6/19 list|% 6/18 list|% 5.x default| |non-PFS block cipher |6.89| 6.86|0.48| |GCM|35.69 |35.64 |35.79| |PFS CBC|57.42| 57.5| 40.56| |RC4|0| 0| 23.18| Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594073#comment-14594073 ] Susan Hinrichs edited comment on TS-3136 at 6/19/15 11:15 PM: -- I spent today running experiments with a variety of cipher_suite strings. Based on feedback from my previous suggestion and these experiments, my latest suggested default cipher_suite list is below (which I referred to as the 6/19 list in the comment above). {code} ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} I think it is a good trade off of security, performance, availability, and reliability for a good out-of-the-box experience. My final experiment involved three boxes in the same pod. One running with the list above (6/19 list). One running the list suggested yesterday (6/18 list). One running the 5.x default. There was a little bit of CPU difference. The experiment ran for 100 wall clock minutes. The CPU time for each scenario was |Scenario|CPU Time| |6/19 list|130 minutes| |6/18 list|152 minutes| |5.x default|180 minutes| The summary of negotiated protocols |Cipher |% list 6/19| % list 6/18|% 5.x list| |ECDHE-RSA-AES256-GCM-SHA384|0.01 |4.79| 0.02| |ECDHE-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDHE-RSA-AES256-SHA384|0 |30.43| 0| |ECDHE-ECDSA-AES256-SHA384 |0| 0| 0| |ECDHE-RSA-AES256-SHA |0| 26.92| 0| |ECDHE-ECDSA-AES256-SHA |0| 0| 0| |AES256-GCM-SHA384 |0.32| 0.31| 0| |AES256-SHA256 |0 |0.16 |0| |AES256-SHA |0 |5.07| 0| |ECDHE-RSA-AES128-GCM-SHA256|35.68 |30.85 |35.77| |ECDHE-ECDSA-AES128-GCM-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA256|0 |0 |31.71| |ECDHE-ECDSA-AES128-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA |57.42 |0.15| 8.85| |ECDHE-ECDSA-AES128-SHA |0 |0 |0| |ECDHE-RSA-DES-CBC3-SHA |0 |0 |0| |ECDHE-ECDSA-DES-CBC3-SHA |0 |0 |0| |AES128-GCM-SHA256 |0 |0 |0.42| |AES128-SHA256 |0 |0 |0| |DES-CBC3-SHA |0.79 |0.79 |0| |ECDHE-RSA-RC4-SHA |0| 0 |16.65| |ECDHE-ECDSA-RC4-SHA|0 |0 |0| |RC4-SHA|0 |0 |6.53| |RC4-MD5|0 |0 |0| was (Author: shinrich): I spent today running experiments with a variety of cipher_suite strings. Based on feedback from my previous suggestion and these experiments, my latest suggested default cipher_suite list is below (which I referred to as the 6/19 list in the comment above). {code} ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} I think it is a good trade off of security, performance, availability, and reliability for a good out-of-the-box experience. My final experiment involved three boxes in the same pod. One running with the list above (6/19 list). One running the list suggested yesterday (6/18 list). One running the 5.x default. There was a little bit of CPU difference. The experiment ran for 100 wall clock minutes. The CPU time for each scenario was |Scenario|CPU Time| |6/19 list|130 minutes| |6/18 list|152 minutes| |5.x default|180 minutes| The summary of negotiated protocols |Cipher |% list 6/19| % list 6/18|% 5.x list| |ECDHE-RSA-AES256-GCM-SHA384|0.01 |4.79| 0.02| |ECDHE-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDHE-RSA-AES256-SHA384|0 |30.43| 0| |ECDHE-ECDSA-AES256-SHA384 |0| 0| 0| |ECDHE-RSA-AES256-SHA |0| 26.92| 0| |ECDHE-ECDSA-AES256-SHA |0| 0| 0| |ECDH-RSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA|0 |0 |0| |ECDH-ECDSA-AES256-SHA |0 |0 |0| |AES256-GCM-SHA384
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594017#comment-14594017 ] Susan Hinrichs edited comment on TS-3136 at 6/19/15 9:54 PM: - I ran an experiment to estimate the impact of DHE on our traffic set. I set up 2048 bit dhparams file and inserted the DHE params ciphers right in front of the non PFS ciphers. The following cipher percentages changed |Cipher|6/19 list w/o DHE %|6/19 list with DHE %| |DHE-RSA-AES128-SHA|0|4.12| |AES128-SHA|5.78|0| |AES256-SHA|0|1.28| These don't all add up to equal exchanges. The other ciphers had small shifts one way or the other. Even with DHE there are still a small percentage of CBC ciphers that sneak through. I did these test in series, so these aren't the end-all be-all numbers. I just wanted to get some idea on the scale of the impact. So broadly speaking by introducing DHE most of the non-PFS ciphers get shifted over to DHE. However, I would still argue that we should not include DHE in the default cipher list. Most of the major sites do not offer DHE. We've had a major ATS deployer experience an increase in SSL errors that went away when DHE was removed. If you don't install a good DHParam, the DHE protocol can be hacked. Therefore, for a default stance, I think an ATS deployment will operate more securely and with less stability risk if DHE is not included in the cipher_suites list. was (Author: shinrich): I ran an experiment to estimate the impact of DHE on our traffic set. I set up 2048 bit dhparams file and inserted the DHE params ciphers right in front of the non PFS ciphers. The following cipher percentages changed |_.Cipher|_.6/19 list w/o DHE %|_.6/19 list with DHE %| |DHE-RSA-AES128-SHA|0|4.12| |AES128-SHA|5.78|0| |AES256-SHA|0|1.28| These don't all add up to equal exchanges. The other ciphers had small shifts one way or the other. Even with DHE there are still a small percentage of CBC ciphers that sneak through. I did these test in series, so these aren't the end-all be-all numbers. I just wanted to get some idea on the scale of the impact. So broadly speaking by introducing DHE most of the non-PFS ciphers get shifted over to DHE. However, I would still argue that we should not include DHE in the default cipher list. Most of the major sites do not offer DHE. We've had a major ATS deployer experience an increase in SSL errors that went away when DHE was removed. If you don't install a good DHParam, the DHE protocol can be hacked. Therefore, for a default stance, I think an ATS deployment will operate more securely and with less stability risk if DHE is not included in the cipher_suites list. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594017#comment-14594017 ] Susan Hinrichs commented on TS-3136: I ran an experiment to estimate the impact of DHE on our traffic set. I set up 2048 bit dhparams file and inserted the DHE params ciphers right in front of the non PFS ciphers. The following cipher percentages changed |_.Cipher|_.6/19 list w/o DHE %|_.6/19 list with DHE %| |DHE-RSA-AES128-SHA|0|4.12| |AES128-SHA|5.78|0| |AES256-SHA|0|1.28| These don't all add up to equal exchanges. The other ciphers had small shifts one way or the other. Even with DHE there are still a small percentage of CBC ciphers that sneak through. I did these test in series, so these aren't the end-all be-all numbers. I just wanted to get some idea on the scale of the impact. So broadly speaking by introducing DHE most of the non-PFS ciphers get shifted over to DHE. However, I would still argue that we should not include DHE in the default cipher list. Most of the major sites do not offer DHE. We've had a major ATS deployer experience an increase in SSL errors that went away when DHE was removed. If you don't install a good DHParam, the DHE protocol can be hacked. Therefore, for a default stance, I think an ATS deployment will operate more securely and with less stability risk if DHE is not included in the cipher_suites list. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594017#comment-14594017 ] Susan Hinrichs edited comment on TS-3136 at 6/19/15 9:55 PM: - I ran an experiment to estimate the impact of DHE on our traffic set. I set up 2048 bit dhparams file and inserted the DHE params ciphers right in front of the non PFS ciphers. The following cipher percentages changed |Cipher|6/19 list w/o DHE %|6/19 list with DHE %| |DHE-RSA-AES128-SHA|0|4.12| |AES128-SHA|5.78|0| |AES256-SHA|0|1.28| These don't all add up to equal exchanges. The other ciphers had small shifts one way or the other. Even with DHE there are still a small percentage of non PFS ciphers that sneak through. I did these test in series, so these aren't the end-all be-all numbers. I just wanted to get some idea on the scale of the impact. So broadly speaking by introducing DHE most of the non-PFS ciphers get shifted over to DHE. However, I would still argue that we should not include DHE in the default cipher list. Most of the major sites do not offer DHE. We've had a major ATS deployer experience an increase in SSL errors that went away when DHE was removed. If you don't install a good DHParam, the DHE protocol can be hacked. Therefore, for a default stance, I think an ATS deployment will operate more securely and with less stability risk if DHE is not included in the cipher_suites list. was (Author: shinrich): I ran an experiment to estimate the impact of DHE on our traffic set. I set up 2048 bit dhparams file and inserted the DHE params ciphers right in front of the non PFS ciphers. The following cipher percentages changed |Cipher|6/19 list w/o DHE %|6/19 list with DHE %| |DHE-RSA-AES128-SHA|0|4.12| |AES128-SHA|5.78|0| |AES256-SHA|0|1.28| These don't all add up to equal exchanges. The other ciphers had small shifts one way or the other. Even with DHE there are still a small percentage of CBC ciphers that sneak through. I did these test in series, so these aren't the end-all be-all numbers. I just wanted to get some idea on the scale of the impact. So broadly speaking by introducing DHE most of the non-PFS ciphers get shifted over to DHE. However, I would still argue that we should not include DHE in the default cipher list. Most of the major sites do not offer DHE. We've had a major ATS deployer experience an increase in SSL errors that went away when DHE was removed. If you don't install a good DHParam, the DHE protocol can be hacked. Therefore, for a default stance, I think an ATS deployment will operate more securely and with less stability risk if DHE is not included in the cipher_suites list. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594073#comment-14594073 ] Susan Hinrichs commented on TS-3136: I spent today running experiments with a variety of cipher_suite strings. Based on feedback from my previous suggestion and these experiments, my latest suggested default cipher_suite list is below (which I referred to as the 6/19 list in the comment above). {code} ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} I think it is a good trade off of security, availability, and reliability for a good out-of-the-box experience. My final experiment involved three boxes in the same pod. One running with the list above (6/19 list). One running the list suggested yesterday (6/18 list). One running the 5.x default. There was a little bit of CPU difference. The experiment ran for 100 wall clock minutes. The CPU time for each scenario was |Scenario|CPU Time| |6/19 list|130 minutes| |6/18 list|152 minutes| |5.x default|180 minutes| The summary of negotiated protocols |Cipher |% list 6/19| % list 6/18|% 5.x list| |ECDHE-RSA-AES256-GCM-SHA384|0.01 |4.79| 0.02| |ECDHE-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDHE-RSA-AES256-SHA384|0 |30.43| 0| |ECDHE-ECDSA-AES256-SHA384 |0| 0| 0| |ECDHE-RSA-AES256-SHA |0| 26.92| 0| |ECDHE-ECDSA-AES256-SHA |0| 0| 0| |ECDH-RSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA|0 |0 |0| |ECDH-ECDSA-AES256-SHA |0 |0 |0| |AES256-GCM-SHA384 |0.32| 0.31| 0| |AES256-SHA256 |0 |0.16 |0| |AES256-SHA |0 |5.07| 0| |ECDHE-RSA-AES128-GCM-SHA256|35.68 |30.85 |35.77| |ECDHE-ECDSA-AES128-GCM-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA256|0 |0 |31.71| |ECDHE-ECDSA-AES128-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA |57.42 |0.15| 8.85| |ECDHE-ECDSA-AES128-SHA |0 |0 |0| |ECDHE-RSA-DES-CBC3-SHA |0 |0 |0| |ECDHE-ECDSA-DES-CBC3-SHA |0 |0 |0| |ECDH-RSA-AES128-GCM-SHA256 |0 |0 |0| |ECDH-ECDSA-AES128-GCM-SHA256 |0 |0 |0| |ECDH-RSA-AES128-SHA256 |0 |0 |0| |ECDH-ECDSA-AES128-SHA256 |0 |0 |0| |ECDH-RSA-AES128-SHA|0 |0 |0| |ECDH-ECDSA-AES128-SHA |0 |0 |0| |AES128-GCM-SHA256 |0 |0 |0.42| |AES128-SHA256 |0 |0 |0| |DES-CBC3-SHA |0.79 |0.79 |0| |ECDHE-RSA-RC4-SHA |0| 0 |16.65| |ECDHE-ECDSA-RC4-SHA|0 |0 |0| |ECDH-RSA-RC4-SHA |0 |0 |0| |ECDH-ECDSA-RC4-SHA |0 |0 |0| |RC4-SHA|0 |0 |6.53| |RC4-MD5|0 |0 |0| Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code}
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594077#comment-14594077 ] Susan Hinrichs commented on TS-3136: For reference, here is the 5.x default cipher suite list {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:RC4-MD5:AES128-SHA:AES256-SHA:DES-CBC3-SHA!SRP:!DSS:!PSK:!aNULL:!eNULL:!SSLv2 {code} Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
Susan Hinrichs created TS-3711: -- Summary: Allow DHE ciphers in the ciphersuite list to be negotiable Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the could that will create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3711: --- Description: As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). was: As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the could that will create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd to have a two step process for activating DHE- ciphers (unlike any other cipher family). Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3711: --- Fix Version/s: 6.0.0 Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Fix For: 6.0.0 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the could that will create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3711: -- Assignee: Susan Hinrichs Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the could that will create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595053#comment-14595053 ] Susan Hinrichs commented on TS-3136: Agreed. Independent of the DHE in default cipher list issue, enabling DHE ciphers should be a single step process. I filed TS-3711 to track that issue. It is a minor issue to reenable. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595053#comment-14595053 ] Susan Hinrichs edited comment on TS-3136 at 6/21/15 2:08 PM: - Agreed. Independent of the DHE in default cipher list issue, enabling DHE ciphers should be a single step process is something we should do in 6.0. I filed TS-3711 to track that issue. It is a minor issue to reenable. was (Author: shinrich): Agreed. Independent of the DHE in default cipher list issue, enabling DHE ciphers should be a single step process. I filed TS-3711 to track that issue. It is a minor issue to reenable. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594073#comment-14594073 ] Susan Hinrichs edited comment on TS-3136 at 6/19/15 11:02 PM: -- I spent today running experiments with a variety of cipher_suite strings. Based on feedback from my previous suggestion and these experiments, my latest suggested default cipher_suite list is below (which I referred to as the 6/19 list in the comment above). {code} ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} I think it is a good trade off of security, performance, availability, and reliability for a good out-of-the-box experience. My final experiment involved three boxes in the same pod. One running with the list above (6/19 list). One running the list suggested yesterday (6/18 list). One running the 5.x default. There was a little bit of CPU difference. The experiment ran for 100 wall clock minutes. The CPU time for each scenario was |Scenario|CPU Time| |6/19 list|130 minutes| |6/18 list|152 minutes| |5.x default|180 minutes| The summary of negotiated protocols |Cipher |% list 6/19| % list 6/18|% 5.x list| |ECDHE-RSA-AES256-GCM-SHA384|0.01 |4.79| 0.02| |ECDHE-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDHE-RSA-AES256-SHA384|0 |30.43| 0| |ECDHE-ECDSA-AES256-SHA384 |0| 0| 0| |ECDHE-RSA-AES256-SHA |0| 26.92| 0| |ECDHE-ECDSA-AES256-SHA |0| 0| 0| |ECDH-RSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-GCM-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA384 |0 |0 |0| |ECDH-ECDSA-AES256-SHA384 |0 |0 |0| |ECDH-RSA-AES256-SHA|0 |0 |0| |ECDH-ECDSA-AES256-SHA |0 |0 |0| |AES256-GCM-SHA384 |0.32| 0.31| 0| |AES256-SHA256 |0 |0.16 |0| |AES256-SHA |0 |5.07| 0| |ECDHE-RSA-AES128-GCM-SHA256|35.68 |30.85 |35.77| |ECDHE-ECDSA-AES128-GCM-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA256|0 |0 |31.71| |ECDHE-ECDSA-AES128-SHA256 |0 |0 |0| |ECDHE-RSA-AES128-SHA |57.42 |0.15| 8.85| |ECDHE-ECDSA-AES128-SHA |0 |0 |0| |ECDHE-RSA-DES-CBC3-SHA |0 |0 |0| |ECDHE-ECDSA-DES-CBC3-SHA |0 |0 |0| |ECDH-RSA-AES128-GCM-SHA256 |0 |0 |0| |ECDH-ECDSA-AES128-GCM-SHA256 |0 |0 |0| |ECDH-RSA-AES128-SHA256 |0 |0 |0| |ECDH-ECDSA-AES128-SHA256 |0 |0 |0| |ECDH-RSA-AES128-SHA|0 |0 |0| |ECDH-ECDSA-AES128-SHA |0 |0 |0| |AES128-GCM-SHA256 |0 |0 |0.42| |AES128-SHA256 |0 |0 |0| |DES-CBC3-SHA |0.79 |0.79 |0| |ECDHE-RSA-RC4-SHA |0| 0 |16.65| |ECDHE-ECDSA-RC4-SHA|0 |0 |0| |ECDH-RSA-RC4-SHA |0 |0 |0| |ECDH-ECDSA-RC4-SHA |0 |0 |0| |RC4-SHA|0 |0 |6.53| |RC4-MD5|0 |0 |0| was (Author: shinrich): I spent today running experiments with a variety of cipher_suite strings. Based on feedback from my previous suggestion and these experiments, my latest suggested default cipher_suite list is below (which I referred to as the 6/19 list in the comment above). {code} ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} I think it is a good trade off of security, availability, and reliability for a good out-of-the-box experience. My final experiment involved three boxes in the same pod. One running with the list above (6/19 list). One running the list suggested yesterday (6/18 list). One running the 5.x default. There was a little bit of CPU difference. The experiment ran for 100 wall clock minutes. The CPU time for each scenario was |Scenario|CPU Time| |6/19 list|130 minutes| |6/18 list|152 minutes| |5.x default|180 minutes| The summary of negotiated protocols |Cipher
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594586#comment-14594586 ] Susan Hinrichs commented on TS-3136: As I recall, with the dhparams enabled, their servers suffered a significant increase in the ssl_error_ssl stat. I don't think they even had DHE in the their ciphersuite list. But there were several issues we addressed to work things out. [~briang], can you remind us of your DHE issue? Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594654#comment-14594654 ] Susan Hinrichs commented on TS-3136: [~zwoop] that is correct. We changed things to the current state. If there is no dhparams file specific, ATS will not load one for you. So the DHE- protocosl will not be selected during negotiation. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601065#comment-14601065 ] Susan Hinrichs commented on TS-3711: For this issue, I'm going to reenable the use of the fixed 2048 bit DH param if no file is specified. This will enable the negotiation of the DHE- cipher is present in the ciphersuites list. For older clients (like java 7 clients), the DHE- negotiation will fail because they cannot handle DH params greater than 1024 bits. This is not an ideal solution because very ATS server by default will be using the dh param. TS-3624 outlines a superior solution that creates a new DH param on every deployment after a fixed interval. But this is a lower implementation risk, so we do this first. Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.0 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3710) ASAN crash in TLS with 6.0.0
[ https://issues.apache.org/jira/browse/TS-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3710: --- Attachment: ts-3710.diff Give ts-3710.diff a try. It should address the immediate use after free reported. It could be that the SSL session timeout check is getting called during the npn as well as the initial handhake. I'll have to look back in the openssl code. You might be able to make this happen more often by adjusting down the SSL session timeout. Setting proxy.config.ssl.session_cache.timeout (in seconds). ASAN crash in TLS with 6.0.0 Key: TS-3710 URL: https://issues.apache.org/jira/browse/TS-3710 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Leif Hedstrom Priority: Critical Fix For: 6.0.0 Attachments: ts-3710.diff {code} ==9570==ERROR: AddressSanitizer: heap-use-after-free on address 0x60649f48 at pc 0xb9f969 bp 0x2b8dbc348920 sp 0x2b8dbc348918 READ of size 8 at 0x60649f48 thread T8 ([ET_NET 7]) #0 0xb9f968 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #1 0xb9f968 in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:142 #2 0xb9f968 in UnixNetVConnection::mainEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1115 #3 0xb7daf7 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #4 0xb7daf7 in InactivityCop::check_inactivity(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:102 #5 0xc21ffe in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #6 0xc21ffe in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #7 0xc241f7 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:207 #8 0xc20c18 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85 #9 0x2b8db3ff6df4 in start_thread (/lib64/libpthread.so.0+0x7df4) #10 0x2b8db585f1ac in __clone (/lib64/libc.so.6+0xf61ac) 0x60649f48 is located 8 bytes inside of 56-byte region [0x60649f40,0x60649f78) freed by thread T8 ([ET_NET 7]) here: #0 0x2b8db1bf3117 in operator delete(void*) ../../.././libsanitizer/asan/asan_new_delete.cc:81 #1 0xb5b20e in SSLNextProtocolTrampoline::ioCompletionEvent(int, void*) /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:89 #2 0xbb2eef in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #3 0xbb2eef in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:142 #4 0xbb2eef in read_signal_done /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:203 #5 0xbb2eef in UnixNetVConnection::readSignalDone(int, NetHandler*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:957 #6 0xb55d6d in SSLNetVConnection::net_read_io(NetHandler*, EThread*) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:480 #7 0xb748fc in NetHandler::mainNetEvent(int, Event*) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:516 #8 0xc24e89 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #9 0xc24e89 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #10 0xc24e89 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252 #11 0xc20c18 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85 #12 0x2b8db3ff6df4 in start_thread (/lib64/libpthread.so.0+0x7df4) previously allocated by thread T8 ([ET_NET 7]) here: #0 0x2b8db1bf2c9f in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cc:50 #1 0xb59f8b in SSLNextProtocolAccept::mainEvent(int, void*) /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:134 #2 0xb888e9 in Continuation::handleEvent(int, void*) ../../iocore/eventsystem/I_Continuation.h:145 #3 0xb888e9 in NetAccept::acceptFastEvent(int, void*) /usr/local/src/trafficserver/iocore/net/UnixNetAccept.cc:466 #4 0xc24e89 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145 #5 0xc24e89 in EThread::process_event(Event*, int) /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128 #6 0xc24e89 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252 #7 0xc20c18 in spawn_thread_internal
[jira] [Updated] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3486: --- Attachment: ts-3266-complete.diff ts-3266-complete.diff replaces the other ts-3266*.diff files on this bug and on ts-3266. I ran this overnight with a dev build on a production box. I switched over to a release build this morning and it has been running for 5 hours on that build. I think the underlying issue is due to shared server sessions moving between threads. If you are opening a new server session or you are doing thread-local server session sharing, the IO events for the client session and the server session will always be on the same thread, so we can be a bit lax in our concurrency control for events on the same state machine. Once you introduce global server session sharing, the server session network events may be handled on different threads than user agent events. In particular, this is problematic when bringing a session out of the server session pool. Thread 1 may be re-activating the server session for a new State Machine while Thread 2 is handing a server initiated network event. In this latest patch, we deal with this potential read condition by holding both the pool lock and the state machine lock while canceling the old pool read and setting up the new state machine read. As long as the read.vio is updated atomically, then the parallel server network event will be protected by a consistent read.vio lock. Eventually we may want to look at a means to migrate a server session to another thread (or reactivate it in another thread), as a means to simplify our locking model. But I think this solution solves the immediate problem. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts-3266-2.diff, ts-3266-complete.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP)
[jira] [Updated] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3711: --- Fix Version/s: (was: 6.0.0) 6.0.1 Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.1 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3711: --- Backport to Version: 6.0.0 Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.1 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3136: --- Backport to Version: 6.0.0 Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.1 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3136. Resolution: Fixed Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.1 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3136: --- Fix Version/s: (was: 6.0.0) 6.0.1 Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.1 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TS-3711) Allow DHE ciphers in the ciphersuite list to be negotiable
[ https://issues.apache.org/jira/browse/TS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs resolved TS-3711. Resolution: Fixed Allow DHE ciphers in the ciphersuite list to be negotiable -- Key: TS-3711 URL: https://issues.apache.org/jira/browse/TS-3711 Project: Traffic Server Issue Type: Bug Components: SSL Reporter: Susan Hinrichs Assignee: Susan Hinrichs Fix For: 6.0.1 As it stands, adding a DHE- cipher to the cipher suite list is not sufficient to allow a DHE protocol to be negotiated. One must also add a dhparams file. We should re-introduce the logic to automatically create DHParams if none is specified. We currently have logic in the that could create a fixed 2048 bit DHParams, but it is not currently enabled. The disabling was tracked in TS-3437. Now that we are at a major release, we should reactivate this logic, since it seems odd and not user-friendly to have a two step process for activating DHE- ciphers (unlike any other cipher family). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598795#comment-14598795 ] Susan Hinrichs edited comment on TS-3486 at 6/24/15 11:43 AM: -- ts-3266-2.diff contains a patch which completes the job. It adjust the locks on the read.vio as it moves from being active to living in the session pool. The first ts-3266.diff patch adds logic to adjust the locks moving from pool to active. I've been running with this patch in production for 5 hours without a crash. was (Author: shinrich): ts-3266-2.diff contains a patch which completes the job. It adjust the locks on the read.vio as it moves from being active to living in the session pool. The first ts-3266.diff patch adds logic to adjust the locks moving from pool to active. I've been running with this path in production for 5 hours without a crash. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts-3266-2.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599395#comment-14599395 ] Susan Hinrichs commented on TS-3136: Talking with more people, some clients (namely Java 7 clients https://wiki.mozilla.org/Security/Server_Side_TLS#DHE_and_Java) do not support DH parameters greater than 1024 bits. In ATS, the default parameters are 2048 bits (1024 bit parameters are vulnerable to attack). It is possible that your increase in error count was due to older clients trying to negotiate DHE but failing once the too large parameters were presented. I think this is another argument against enabling DHE protocols by default. To usefully support DHE, you need to understand your client base. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599671#comment-14599671 ] Susan Hinrichs edited comment on TS-3136 at 6/24/15 4:24 PM: - [~bcall] noted that no clients will actually negotiate CHACHA since it is so low in the list. since CHACHA is not yet widely deployed we are a bit hesitant to put it on top at this point. So we are removing CHACHA from the default list. Individual deployments may want to add it depending on their client and server sets. was (Author: shinrich): @bcall noted that no clients will actually negotiate CHACHA since it is so low in the list. since CHACHA is not yet widely deployed we are a bit hesitant to put it on top at this point. So we are removing CHACHA from the default list. Individual deployments may want to add it depending on their client and server sets. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3716) Add TLS-version control on origin server to config
[ https://issues.apache.org/jira/browse/TS-3716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3716: --- Assignee: Dave Thompson Add TLS-version control on origin server to config -- Key: TS-3716 URL: https://issues.apache.org/jira/browse/TS-3716 Project: Traffic Server Issue Type: Improvement Reporter: Dave Thompson Assignee: Dave Thompson Currently, (unlike between client and ATS) ATS does not support TLS version configuration control between ATS and origin server.Currently ATS is hardcoded to use SSLv23_client_method() for all outgoing client connections, which locks it into old SSL, and also becomes a problem for adding TLS-extensions on the client-hello, e.g. session ticket cache re-use (TS-3559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599671#comment-14599671 ] Susan Hinrichs commented on TS-3136: @bcall noted that no clients will actually negotiate CHACHA since it is so low in the list. since CHACHA is not yet widely deployed we are a bit hesitant to put it on top at this point. So we are removing CHACHA from the default list. Individual deployments may want to add it depending on their client and server sets. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TS-3699) TLS 64GB transfer fails with AES GCM cipher
[ https://issues.apache.org/jira/browse/TS-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs updated TS-3699: --- Assignee: Dave Thompson TLS 64GB transfer fails with AES GCM cipher --- Key: TS-3699 URL: https://issues.apache.org/jira/browse/TS-3699 Project: Traffic Server Issue Type: Bug Reporter: Dave Thompson Assignee: Dave Thompson Running ATS 5.0.1, over a TLS connection using cipher suite AES128-GCM-SHA256 will fail every time just before hitting 64GB. Switching cipher to the same CBC cipher (AES128-SHA), and data transfers can go beyond the 64GB limit. It appears we are hitting the GCM design limit of 2^39-256 bits (64GB). TLS should be able to renegotiate keys which resets the GCM counter, and in fact I have successfully tested this with ATS 4.0.2. Work around is to use the CBC variant (AES128-SHA), though it would be good to know what changed between 5.0.1 and 4.0.2 to stop cipher run out initiated renegotiation. FWIW proxy.config.ssl.allow_client_renegotiation, does not appear to come into play here. Looking at the code, this appears to be written to prevent client initiated renegotiation (prevent renegotiation attack circa 2009). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593477#comment-14593477 ] Susan Hinrichs commented on TS-3136: [~jeaglesham] and [~ivanr] thanks for your comments. Looking back at the performance numbers that [~davet] did last year, the AES128 vs AES256 performance numbers are consistent with yours Ivan. I'm really surprised that SHA256 vs SHA would have such a big performance impact. Since SHA has been broken so long, I just had a knee jerk reaction against it. But I think you make a good argument that SHA is good enough in that case. You do bring up a good point about the dhparams. We do provide a means to set your own, but I think the default is the 1024 bit one, which is no good these days. [~bcall] what do you think about setting a 2048 bit DHParam by default? I think [~i.galic] filed a bug on dh params a while back. I'll review the current state of things. Since this string is supposed to be reasonable for at least the coming year, adding ChaCha seems quite reasonable. I think doing the additional test that you suggest is a good idea. I'll see if ops will give me two or three machines so I can compare my proposed string from yesterday, an updated string based on your comments, and potentially the 5.x default cipher string. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593493#comment-14593493 ] Susan Hinrichs commented on TS-3486: Bummer. It looks like we still have a freed netvc lingering in the event processing queue. [~bettydreamit] and [~esproul] is the crash frequency lower? [~esproul] do you have some trace information on your crash? Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6919 #27 0x005b3f5f in
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593518#comment-14593518 ] Susan Hinrichs commented on TS-3136: TS-3624 is the bug Igor filed suggesting that we autogenerate DH parameters periodically. Don't think we want to do that for 6.0 at this point. The way it currently stands, unless you specify a dhparams file, no dhparams is registered, so the DHE ciphers won't be negotiated (which explains why my measurements had no DHE ciphers). At one point in 5.2/5.3 we registered a 2048 bit DH param from RFC 5114, but that messed things up for [~briang] so the change was backed out and the param is only set if the user specifies it via a file. In my opinion for 6.0, we should either: * Remove DHE from the list of default ciphers * Re-introduce an auto-generated DH param so the DHE ciphers might get negotiated. From a lazy developer's perspective I'd vote for just removing DHE from the list. It looks like ECDHE is pretty prevalent. Is there big value for keeping DHE in the defaults? Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593573#comment-14593573 ] Susan Hinrichs commented on TS-3136: It looks like we have around 5% hitting non-PFS protocols. I'll add an experiment to set a dh params file (to actually activate the DHE protocols) and see how much of that 5% we can convert to DHE protocols. Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 from https://cipherli.st/ {quote} [~jacksontj] had responded with: {quote} [~i.galic] That cipher quite is geared towards security, but doesn't support quite a few older clients. I'd recommend we use the suite from mozilla (https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Server_Configurations) which is a good mix of security and compatibility: {code} ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA {code} {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593620#comment-14593620 ] Susan Hinrichs commented on TS-3486: [~esproul] a line number in read_from_net would be helpful. Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Phil Sorber Labels: crash Fix For: sometime Attachments: ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6919 #27 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #28 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at
[jira] [Commented] (TS-3136) Change default TLS cipher suites
[ https://issues.apache.org/jira/browse/TS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592485#comment-14592485 ] Susan Hinrichs commented on TS-3136: Ran some tests on a production box in Y! Based on those results, I suggest the following cipher string. ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA The upshot is that we remove RC4, add new ciphers, and rearrange the list to give preference to cipher attributes in the following order: PFS, then GCM, then stronger SHA, then stronger AES. 3DES is at the end to scoop up the remainders. We tested in the Y! environment which tends to have a wide variety of clients. Removing RC4 did not seem to significantly impact handshake success rate. CBC algorithms are also concerning, but if we care about out-of-the-box experience it looks like the CBC algorithms need to stick around for a while longer. Here are details of the test With Y! original cipher string 0.0102% ssl_error_ssl The number of DES-CBC3-SHA sessions was negligible (45). The Y! initial configuration has one RC4 algorithm listed kind of early, so the RC4 percentage was around 30% as [~davet] noted in an earlier comment. With proposed default cipher string running for an hour 0.009% ssl_error_ssl The percentage of DES-CBC3-SHA sessions grew to 0.9% of sessions. In my experiment, it was impossible to isolate the CPU impact of this change. To test a new cipher without updating all the machines in the production pod, I remove the test box from the SSL session sharing communication. The test box experienced around a 30% increase in CPU utilization, but I think that can be mostly attributed to increased session negotiation since it did not know about the sessions negotiated by other machines in the pod. We did one experiment with the RC4 ciphers added after DES-CBC3 as another measure of how many clients are only willing to do RC4. After about an hour, 2 RC4 sessions were started. 510932 = Total Successful Handshakes Percentage of various cipher's negotiated # Start with PFS/GCM ciphers. Give slight preference to AES256 over AES128, and prefer stronger SHA 0% ECDHE-ECDSA-AES256-GCM-SHA384: 4.2% ECDHE-RSA-AES256-GCM-SHA384: 0% ECDHE-ECDSA-AES128-GCM-SHA256: 30.6% ECDHE-RSA-AES128-GCM-SHA256: # DHE still gives of PFS but at increased computation cost 0% DHE-RSA-AES256-GCM-SHA384: 0% DHE-DSS-AES256-GCM-SHA384: 0% DHE-RSA-AES128-GCM-SHA256: 0% DHE-DSS-AES128-GCM-SHA256: # CBC versions of the PFS ciphers 0% ECDHE-ECDSA-AES256-SHA384: 30.6% ECDHE-RSA-AES256-SHA384: 0% ECDHE-ECDSA-AES256-SHA: 27.7% ECDHE-RSA-AES256-SHA: 0% ECDHE-ECDSA-AES128-SHA256: 0% ECDHE-RSA-AES128-SHA256: 0% ECDHE-ECDSA-AES128-SHA: 0.14% ECDHE-RSA-AES128-SHA: 0% DHE-RSA-AES256-SHA256: 0% DHE-DSS-AES256-SHA256: 0% DHE-RSA-AES128-SHA256: 0% DHE-DSS-AES128-SHA256: 0% DHE-RSA-AES256-SHA: 0% DHE-DSS-AES256-SHA: 0% DHE-RSA-AES128-SHA: 0% DHE-DSS-AES128-SHA: # No PFS, GCM 0.3% AES256-GCM-SHA384: 0% AES128-GCM-SHA256: # No PFS, CBC 0.2% AES256-SHA256: 0% AES128-SHA256: 4.8% AES256-SHA: 0.5% AES128-SHA: # 3DES as a last resort 0.9% DES-CBC3-SHA Change default TLS cipher suites Key: TS-3136 URL: https://issues.apache.org/jira/browse/TS-3136 Project: Traffic Server Issue Type: Improvement Components: Security, SSL Reporter: Leif Hedstrom Assignee: Susan Hinrichs Labels: compatibility Fix For: 6.0.0 In TS-3135 [~i.galic] suggested: {quote} also, recommendations for a safer ciphersuite: SSLCipherSuite
[jira] [Assigned] (TS-3486) Segfault in do_io_write with plugin (??)
[ https://issues.apache.org/jira/browse/TS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Susan Hinrichs reassigned TS-3486: -- Assignee: Susan Hinrichs (was: Phil Sorber) Segfault in do_io_write with plugin (??) Key: TS-3486 URL: https://issues.apache.org/jira/browse/TS-3486 Project: Traffic Server Issue Type: Bug Affects Versions: 5.2.0, 5.3.0 Reporter: Qiang Li Assignee: Susan Hinrichs Labels: crash Fix For: sometime Attachments: ts-3266-2.diff, ts-3266-complete.diff, ts3486-ptrace.txt.gz {code} (gdb) bt #0 0x005bdb8b in HttpServerSession::do_io_write (this=value optimized out, c=0x2aaadccc4bf0, nbytes=576, buf=0x2aaafc2ffee8, owner=false) at HttpServerSession.cc:104 #1 0x005acc1d in HttpSM::setup_server_send_request (this=0x2aaadccc4bf0) at HttpSM.cc:5686 #2 0x005b3f85 in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1520 #3 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #4 0x005b980b in HttpSM::state_api_callback (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1275 #5 0x004d7a1b in TSHttpTxnReenable (txnp=0x2aaadccc4bf0, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5614 #6 0x2ba118441c89 in cachefun (contp=value optimized out, event=value optimized out, edata=0x2aaadccc4bf0) at main.cpp:1876 #7 0x005b4466 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=value optimized out, data=value optimized out) at HttpSM.cc:1381 #8 0x005b627d in HttpSM::do_http_server_open (this=0x2aaadccc4bf0, raw=value optimized out) at HttpSM.cc:4639 #9 0x005baa04 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7021 #10 0x005b25a3 in HttpSM::state_cache_open_write (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2442 #11 0x005b5b28 in HttpSM::main_handler (this=0x2aaadccc4bf0, event=1108, data=0x2aab1c3b6800) at HttpSM.cc:2554 #12 0x0059338a in handleEvent (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at ../../iocore/eventsystem/I_Continuation.h:145 #13 HttpCacheSM::state_cache_open_write (this=0x2aaadccc6618, event=value optimized out, data=0x2aab1c3b6800) at HttpCacheSM.cc:167 #14 0x00697223 in handleEvent (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:145 #15 CacheVC::callcont (this=0x2aab1c3b6800, event=value optimized out) at ../../iocore/cache/P_CacheInternal.h:662 #16 0x00715940 in Cache::open_write (this=value optimized out, cont=value optimized out, key=0x2ba0ff762d70, info=value optimized out, apin_in_cache=46914401429576, type=CACHE_FRAG_TYPE_HTTP, hostname=0x2aaadd281078 www.mifangba.comhttpapi.phpwww.mifangba.comhttp://www.mifangba.com/api.php?op=countid=4modelid=12;, host_len=16) at CacheWrite.cc:1788 #17 0x006e5765 in open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at P_CacheInternal.h:1093 #18 CacheProcessor::open_write (this=value optimized out, cont=0x2aaadccc6618, expected_size=value optimized out, url=0x2aaadccc5310, cluster_cache_local=value optimized out, request=value optimized out, old_info=0x0, pin_in_cache=0, type=CACHE_FRAG_TYPE_HTTP) at Cache.cc:3622 #19 0x005936f0 in HttpCacheSM::open_write (this=0x2aaadccc6618, url=value optimized out, request=value optimized out, old_info=value optimized out, pin_in_cache=value optimized out, retry=value optimized out, allow_multiple=false) at HttpCacheSM.cc:298 #20 0x005a022e in HttpSM::do_cache_prepare_action (this=0x2aaadccc4bf0, c_sm=0x2aaadccc6618, object_read_info=0x0, retry=true, allow_multiple=false) at HttpSM.cc:4511 #21 0x005babd9 in do_cache_prepare_write (this=0x2aaadccc4bf0) at HttpSM.cc:4436 #22 HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:7098 #23 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #24 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=0, data=0x0) at HttpSM.cc:1455 #25 0x005ba712 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6876 #26 0x005ba702 in HttpSM::set_next_state (this=0x2aaadccc4bf0) at HttpSM.cc:6919 #27 0x005b3f5f in HttpSM::handle_api_return (this=0x2aaadccc4bf0) at HttpSM.cc:1517 #28 0x005b45f8 in HttpSM::state_api_callout (this=0x2aaadccc4bf0, event=6, data=0x0) at HttpSM.cc:1455 #29