[ https://issues.apache.org/jira/browse/TS-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086391#comment-13086391 ]
William Bardwell commented on TS-880: ------------------------------------- With 'performance_2nd_req_another_version.patch' I get similar crashes to with my original patches, so I think 'try3' is the best option so far. [Aug 17 15:32:31.704] Server {4134497168} NOTE: cache enabled FATAL: UnixNetVConnection.cc:968: failed assert `thread == this_ethread()` /usr/local/akamai/ts7/proxy/traffic_server - STACK TRACE: FATAL: UnixNetVConnection.cc:968: failed assert `thread == this_ethread()` /usr/local/akamai/ts7/proxy/traffic_server - STACK TRACE: /usr/local/akamai/ts7/proxy/traffic_server(ink_fatal+0x1a)[0x844621a] /usr/local/akamai/ts7/proxy/traffic_server(ink_fatal+0x1a)[0x844621a] /usr/local/akamai/ts7/proxy/traffic_server(_ink_assert+0x6a)[0x844593a] /usr/local/akamai/ts7/proxy/traffic_server(_ink_assert+0x6a)[0x844593a] /usr/local/akamai/ts7/proxy/traffic_server(_ZN18UnixNetVConnection9mainEventEiP5Event+0x5b)[0x835c97b] /usr/local/akamai/ts7/proxy/traffic_server(_ZN18UnixNetVConnection9mainEventEiP5Event+0x5b)[0x835c97b] /usr/local/akamai/ts7/proxy/traffic_server(_ZN13InactivityCop16check_inactivityEiP5Event+0xd1)[0x83518b1] /usr/local/akamai/ts7/proxy/traffic_server(_ZN13InactivityCop16check_inactivityEiP5Event+0xd1)[0x83518b1] /usr/local/akamai/ts7/proxy/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x3c3)[0x8385233] /usr/local/akamai/ts7/proxy/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x3c3)[0x8385233] /usr/local/akamai/ts7/proxy/traffic_server(_ZN7EThread7executeEv+0x4cf)[0x8385d4f] /usr/local/akamai/ts7/proxy/traffic_server(_ZN7EThread7executeEv+0x4cf)[0x8385d4f] /usr/local/akamai/ts7/proxy/traffic_server[0x8384c53] /usr/local/akamai/ts7/proxy/traffic_server[0x8384c53] /lib/tls/i686/cmov/libpthread.so.0[0xf77264fb] /lib/tls/i686/cmov/libpthread.so.0[0xf77264fb] /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xf741de5e] /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xf741de5e] Program received signal SIGABRT, Aborted. [Switching to Thread 0xf66f6b90 (LWP 9967)] 0xffffe425 in __kernel_vsyscall () (gdb) where #0 0xffffe425 in __kernel_vsyscall () #1 0xf7372085 in raise () from /lib/tls/i686/cmov/libc.so.6 #2 0xf7373a01 in abort () from /lib/tls/i686/cmov/libc.so.6 #3 0x084461da in ink_fatal_va (return_code=1, message_format=0xf66f5f54 "UnixNetVConnection.cc:968: failed assert `thread == this_ethread()`", ap=0xf66f5f48 "") at ink_error.cc:43 #4 0x0844621a in ink_fatal (return_code=1, message_format=0xf66f5f54 "UnixNetVConnection.cc:968: failed assert `thread == this_ethread()`") at ink_error.cc:73 #5 0x0844593a in _ink_assert (a=0x84abe39 "thread == this_ethread()", f=0x84abd9d "UnixNetVConnection.cc", l=968) at ink_assert.cc:44 #6 0x0835c97b in UnixNetVConnection::mainEvent (this=0x8a9edb0, event=1, e=0x8a829a0) at UnixNetVConnection.cc:968 #7 0x083518b1 in InactivityCop::check_inactivity (this=0xf5400898, event=2, e=0x8a829a0) at ../../iocore/eventsystem/I_Continuation.h:146 #8 0x08385233 in EThread::process_event (this=0xf67f8008, e=0x8a829a0, calling_code=2) at I_Continuation.h:146 #9 0x08385d4f in EThread::execute (this=0xf67f8008) at UnixEThread.cc:226 #10 0x08384c53 in spawn_thread_internal (a=0x8a5c320) at Thread.cc:88 #11 0xf77264fb in start_thread () from /lib/tls/i686/cmov/libpthread.so.0 #12 0xf741de5e in clone () from /lib/tls/i686/cmov/libc.so.6 > Major performance problem with second request on same keep-alive connection > --------------------------------------------------------------------------- > > Key: TS-880 > URL: https://issues.apache.org/jira/browse/TS-880 > Project: Traffic Server > Issue Type: Bug > Components: HTTP > Affects Versions: 3.0.0, 2.1.4 > Environment: 32-bit TS on linux, using epoll. With > proxy.config.http.keep_alive_enabled_out = 1 > Reporter: William Bardwell > Assignee: William Bardwell > Fix For: 3.1.1 > > Attachments: performance.try3.diff, performance_2nd_req.diff, > performance_2nd_req_another_version.patch, performance_2nd_req_try2.diff > > > If I load the same URL through TS twice in a row to a server with keep-alives > turned on I get really slow performance on the second request. > E.g. (loading 212M over loopback with uncachable content, but results are > similar with cachable content): > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left Speed > 100 212M 100 212M 0 0 140M 0 0:00:01 0:00:01 --:--:-- > 142M* > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left Speed > 100 212M 100 212M 0 0 6397k 0 0:00:34 0:00:34 --:--:-- > 6400k* > If I turn off proxy.config.http.keep_alive_enabled_out the problem goes away, > it can also be partially solved by making proxy.config.io.max_buffer_size > really big (but it needs to be bigger for larger content, and that supports > the comment below, and proves that it isn't the origin server's fault.) > When I turn on debug messages it looks like the IO loop for the second > request only wakes up every 10ms (when an epoll_wait times out) and then it > reads and writes, for the first request it goes much faster without those > pauses. My theory is that the issue is related to the the fact that the > outgoing connection fd gets added to the epoll fd for the thread handling the > first request and the first user agent fd is added to the same epoll fd, and > it stays on that epoll fd. But the new user agent request is on a different > epoll fd which I assume means is tied to a different thread. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira