[
https://issues.apache.org/jira/browse/TS-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789261#comment-13789261
]
David Carlin edited comment on TS-2267 at 10/8/13 3:10 PM:
-----------------------------------------------------------
I built master (commit 7ba121c9ac87a90e874c726ddd134a45d10ebfe3) and applied
Bryan's patch - I still see the same problem. Bryan's patch adds this debug
message, and it occurs constantly:
{noformat}
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aeea8028360 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600a9d20 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600479a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600464a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee6003c720 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60058e00 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60057ba0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee6005baa0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee681df510 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60054f00 inactivity timeout not set, setting a default of 5 minutes
{noformat}
was (Author: dcarlin):
I built master (commit 7ba121c9ac87a90e874c726ddd134a45d10ebfe3) and still see
the same problem. Bryan's patch adds this debug message, and it occurs
constantly:
{noformat}
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aeea8028360 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600a9d20 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600479a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.830] Server {0x2aee50201700} DEBUG: (inactivity_cop) vc:
0x2aee600464a0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee6003c720 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60058e00 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60057ba0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee6005baa0 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee681df510 inactivity timeout not set, setting a default of 5 minutes
[Oct 8 14:57:03.845] Server {0x2aee4be01700} DEBUG: (inactivity_cop) vc:
0x2aee60054f00 inactivity timeout not set, setting a default of 5 minutes
{noformat}
> Leaking file descriptors - no inactivity timeout set
> ----------------------------------------------------
>
> Key: TS-2267
> URL: https://issues.apache.org/jira/browse/TS-2267
> Project: Traffic Server
> Issue Type: Bug
> Components: Core, Network
> Affects Versions: 4.0.1
> Environment: RHEL 6.4
> Reporter: David Carlin
>
> We noticed a problem on two of our *forward proxy* hosts where ATS would run
> out of file descriptors. Even after a host was removed from a VIP for hours,
> ATS would report:
> proxy.process.http.current_server_connections = 0
> proxy.process.http.current_active_client_connections = 43559
> # ss -s
> Total: 43622 (kernel 43688)
> TCP: 43632 (estab 2, closed 39553, orphaned 0, synrecv 0, timewait 87/0),
> ports 55
> Note that ATS reports 43559 active connections, but there are only 2
> established connections on the host. On this particular host,
> proxy.config.net.connections_throttle was set to 50000 so we first noticed
> the problem because ATS started throttling and never recovered - the host
> kept flapping as the connection count stayed near the throttling limit.
> Bryan added some debugging to inactivity cop to create a log entry every time
> there was a connection with no inactivity timeout set - this helped confirm
> the issue. He then made a patch to set a fixed 5 min timeout if there is no
> inactivity timeout specified - this helped stabilize the issue.
> The only zero timeout in records.config was
> "proxy.config.http.transaction_active_timeout_out = 0" which is the default.
> Changing this to something else did not alleviate the problem - we could
> still see in the debug logs that connections had no inactivity timeout set.
> We downgraded from 4.0.1 to 3.2.0 and the problem still occured in 3.2.0
> Thanks to Bryan Call for troubleshooting/patching!
--
This message was sent by Atlassian JIRA
(v6.1#6144)
