[jira] [Assigned] (TS-1871) No Error on Startup if auto_conf Port is Unavailable

[James Peach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Peach reassigned TS-1871:
---

Assignee: James Peach

This bit me recently, since it's possible to get ATS itself to bind to this 
port (eg. the proxy_pac port). Way too easy to shoot a toe off here.

> No Error on Startup if auto_conf Port is Unavailable
> 
>
> Key: TS-1871
> URL: https://issues.apache.org/jira/browse/TS-1871
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Management
>Reporter: Clinton Wolfe
>Assignee: James Peach
> Fix For: 3.3.6
>
>
> If another process is already listening on the auto_conf port (8083 by 
> default),  traffic_manager silently fails to bind to it.  
> This can break heartbeats, because heartbeats are sent to the process on 8083 
> (which will probably not give the expected response as 
> http://127.0.0.1:8083/synthetic.txt).  With heartbeats broken, traffic_cop 
> constantly re-starts traffic_server - which was the obvious symptom, in my 
> case.
> Observed on ts 3.2.4, stock config, centos 5.9
> discussed in #traffic-server on freenode IRC, ortho_stice (clintoncwolfe) and 
> amc, 2013-05-01 and 2013-05-02

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (TS-1966) Make ProxyAllocator slot size configurable

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom reassigned TS-1966:
-

Assignee: Leif Hedstrom

> Make ProxyAllocator slot size configurable
> --
>
> Key: TS-1966
> URL: https://issues.apache.org/jira/browse/TS-1966
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Reporter: Leif Hedstrom
>Assignee: Leif Hedstrom
>  Labels: NUMA
> Fix For: 3.3.5
>
>
> Right now, the max number of slots for all Proxy Allocators is fixed, I 
> believe at 500. Above this, a proxy allocator will fall back to the global 
> allocator.
> This should be configurable, via records.config if possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1966) Make ProxyAllocator slot size configurable

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698287#comment-13698287
 ] 

Leif Hedstrom commented on TS-1966:
---

So, this turns out to be pretty damn hairy, at least without significant 
performance implications...

James had a good idea, putting this setting into the command line arguments to 
traffic_server. This can then be modified from the defaults through 
records.config via proxy.config.proxy_binary_opts .

> Make ProxyAllocator slot size configurable
> --
>
> Key: TS-1966
> URL: https://issues.apache.org/jira/browse/TS-1966
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Reporter: Leif Hedstrom
>  Labels: NUMA
> Fix For: 3.3.5
>
>
> Right now, the max number of slots for all Proxy Allocators is fixed, I 
> believe at 500. Above this, a proxy allocator will fall back to the global 
> allocator.
> This should be configurable, via records.config if possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-954) when use raw disks, some blocks is lost when caculate disk usable blocks

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-954:
-

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> when use raw disks, some blocks is lost when caculate disk usable blocks
> 
>
> Key: TS-954
> URL: https://issues.apache.org/jira/browse/TS-954
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache
>Affects Versions: 3.1.0
> Environment: all when use raw disks
>Reporter: weijin
>Assignee: John Plevyak
> Fix For: 3.3.6
>
> Attachments: calcu_blocks.dff
>
>
> when use raw disks, some blocks may be lost because the skip variable is in 
> bytes not in blocks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1509) Remove TS_PARSE_OK constant

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1509:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> Remove TS_PARSE_OK constant
> ---
>
> Key: TS-1509
> URL: https://issues.apache.org/jira/browse/TS-1509
> Project: Traffic Server
>  Issue Type: Bug
>  Components: TS API
>Reporter: James Peach
>Assignee: James Peach
>Priority: Minor
> Fix For: 3.3.6
>
>
> There's TS_PARSE_DONE and TS_PARSE_OK. Which one should a developer handle 
> and what's the difference between?
> Well TS_PARSE_OK is never returned from TSHttpParseResp() so we should just 
> remove it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1201) Crash report: hostdb multicache, double free

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1201:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> Crash report: hostdb multicache, double free
> 
>
> Key: TS-1201
> URL: https://issues.apache.org/jira/browse/TS-1201
> Project: Traffic Server
>  Issue Type: Bug
>  Components: DNS
>Reporter: Zhao Yongming
>  Labels: A, crash
> Fix For: 3.3.6
>
>
> {code}
> *** glibc detected *** /usr/bin/traffic_server: corrupted double-linked list: 
> 0x1fe10ef0 ***
> === Backtrace: =
> /lib64/libc.so.6[0x3db2072555]   
> /lib64/libc.so.6(cfree+0x4b)[0x3db20728bb]
> /usr/bin/traffic_server(MultiCacheSync::mcEvent(int, Event*)+0xa4)[0x5dae04]
> /usr/bin/traffic_server(EThread::process_event(Event*, int)+0x22f)[0x691c8f]
> /usr/bin/traffic_server(EThread::execute()+0x6a1)[0x692681]
> /usr/bin/traffic_server[0x69115e]
> /lib64/libpthread.so.0[0x3db280673d]
> /lib64/libc.so.6(clone+0x6d)[0x3db20d44bd]
> === Memory map: 
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-966) cache.config dest_domain= dest_hostname= dest_ip= do not match anything

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-966:
-

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> cache.config dest_domain= dest_hostname= dest_ip= do not match anything
> ---
>
> Key: TS-966
> URL: https://issues.apache.org/jira/browse/TS-966
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache
>Affects Versions: 3.1.0, 3.0.1
>Reporter: Igor Galić
>  Labels: A
> Fix For: 3.3.6
>
>
> Caching policies are not applied when using these options to match targets.
> It is also not very clear *what* dest_domain= vs dest_hostname= can match.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1527) Coredump: FATAL: HttpTunnel.cc:532: failed assert `active == false`

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1527:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> Coredump: FATAL: HttpTunnel.cc:532: failed assert `active == false`
> ---
>
> Key: TS-1527
> URL: https://issues.apache.org/jira/browse/TS-1527
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 3.2.0
> Environment: RHEL 6.2: Linux 
> 2.6.32-220.23.1.el6.YAHOO.20120713.x86_64 #1 SMP x86_64 x86_64 x86_64 
> GNU/Linux
>Reporter: Abhishek Nayani
>  Labels: A
> Fix For: 3.3.6
>
> Attachments: sample.cc
>
>
> I get the following stact trace:
> {noformat}
> FATAL: HttpTunnel.cc:532: failed assert `active == false`
> /home/y/bin/traffic_server - STACK TRACE: 
> /home/y/lib64/libtsutil.so.3(ink_fatal+0x88)[0x2b024cb8b868]
> /home/y/lib64/libtsutil.so.3(_ink_assert+0x1f)[0x2b024cb89edf]
> /home/y/bin/traffic_server(HttpTunnel::deallocate_buffers()+0x924)[0x56adc4]
> /home/y/bin/traffic_server(HttpSM::kill_this()+0x6df)[0x52b47f]
> /home/y/bin/traffic_server(HttpSM::main_handler(int, void*)+0x198)[0x52b9e8]
> /home/y/bin/traffic_server(EThread::process_event(Event*, int)+0xb4)[0x696ed4]
> /home/y/bin/traffic_server(EThread::execute()+0x633)[0x6979d3]
> /home/y/bin/traffic_server[0x695ea2]
> /lib64/libpthread.so.0[0x3387207851]
> /lib64/libc.so.6(clone+0x6d)[0x3386ee76dd]
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1508) Evacuation stats blank

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1508:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> Evacuation stats blank
> --
>
> Key: TS-1508
> URL: https://issues.apache.org/jira/browse/TS-1508
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Stats
>Affects Versions: 3.2.0
> Environment: CentOS 6
>Reporter: Jon Cowie
>Assignee: Phil Sorber
> Fix For: 3.3.6
>
> Attachments: graph.gif, TS-1508.diff
>
>
> With a full disk cache, neither neither proxy.process.cache.evacuate.* or 
> proxy.process.cache.gc_bytes_evacuated are showing any signs of activity. 
> One possibly related symptom is that proxy.node.cache.bytes_free_mb flatlined 
> at 48MB free out of 880GB after dropping steadily at the rate I was 
> expecting, and hasn't moved since.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1883) SSL origin connections do not support connection timeouts

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1883:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> SSL origin connections do not support connection timeouts
> -
>
> Key: TS-1883
> URL: https://issues.apache.org/jira/browse/TS-1883
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core, SSL
>Reporter: James Peach
> Fix For: 3.3.6
>
>
> In {{proxy/http/HttpSM.cc}}, we can see that origin connections do not 
> support timeouts if the scheme is HTTPS:
> {code}
> void
> HttpSM::do_http_server_open(bool raw)
> {
> ...
>   if (t_state.scheme == URL_WKSIDX_HTTPS) {
> DebugSM("http", "calling sslNetProcessor.connect_re");
> connect_action_handle = sslNetProcessor.connect_re(this,// state 
> machine
>
> &t_state.current.server->addr.sa,// addr + port
>&opt);
>   } else {
> ...
>   // Setup the timeouts
>   // Set the inactivity timeout to the connect timeout so that we
>   //   we fail this server if it doesn't start sending the response
>   //   header
>   MgmtInt connect_timeout;
>   if (t_state.method == HTTP_WKSIDX_POST || t_state.method == 
> HTTP_WKSIDX_PUT) {
> connect_timeout = t_state.txn_conf->post_connect_attempts_timeout;
>   } else if (t_state.current.server == &t_state.parent_info) {
> connect_timeout = t_state.http_config_param->parent_connect_timeout;
>   } else {
> if (t_state.pCongestionEntry != NULL)
>   connect_timeout = t_state.pCongestionEntry->connect_timeout();
> else
>   connect_timeout = t_state.txn_conf->connect_attempts_timeout;
>   }
>   DebugSM("http", "calling netProcessor.connect_s");
>   connect_action_handle = netProcessor.connect_s(this,  // state 
> machine
>  
> &t_state.current.server->addr.sa,// addr + port
>  connect_timeout, &opt);
> ...
>   }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1486) drop solaris studio support

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1486:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> drop solaris studio support
> ---
>
> Key: TS-1486
> URL: https://issues.apache.org/jira/browse/TS-1486
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core, Portability
>Reporter: James Peach
>Assignee: Igor Galić
> Fix For: 3.3.6
>
>
> We should drop Solaris Studio support for the 3.4 release.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1871) No Error on Startup if auto_conf Port is Unavailable

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1871:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> No Error on Startup if auto_conf Port is Unavailable
> 
>
> Key: TS-1871
> URL: https://issues.apache.org/jira/browse/TS-1871
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Management
>Reporter: Clinton Wolfe
> Fix For: 3.3.6
>
>
> If another process is already listening on the auto_conf port (8083 by 
> default),  traffic_manager silently fails to bind to it.  
> This can break heartbeats, because heartbeats are sent to the process on 8083 
> (which will probably not give the expected response as 
> http://127.0.0.1:8083/synthetic.txt).  With heartbeats broken, traffic_cop 
> constantly re-starts traffic_server - which was the obvious symptom, in my 
> case.
> Observed on ts 3.2.4, stock config, centos 5.9
> discussed in #traffic-server on freenode IRC, ortho_stice (clintoncwolfe) and 
> amc, 2013-05-01 and 2013-05-02

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1696) ESI build fails on fBSD

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1696:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> ESI build fails on fBSD
> ---
>
> Key: TS-1696
> URL: https://issues.apache.org/jira/browse/TS-1696
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Plugins
>Reporter: Igor Galić
> Fix For: 3.3.6
>
>
> For reference: 
> http://ci.apache.org/builders/tserver-fbsd-trunk/builds/1672/steps/compile_3/logs/stdio
> {noformat}
> gmake[4]: Entering directory 
> `/usr/home/buildslave27/slave27/tserver-fbsd-trunk/build/plugins/experimental/esi'
>   CXX  docnode_test.o
> cc1plus: warnings being treated as errors
> test/docnode_test.cc: In function 'void checkNodeList2(const 
> EsiLib::DocNodeList&)':
> test/docnode_test.cc:85: warning: comparison between signed and unsigned 
> integer expressions
> test/docnode_test.cc:114: warning: comparison between signed and unsigned 
> integer expressions
> gmake[4]: *** [docnode_test.o] Error 1
> {noformat}
> It might make sense to convert {{_len}} to {{size_t}} - but there are some 
> clever uses of negative {{len}} like here:
> {code}
>   inline std::string unescape(const char *str, int len = -1) {
>   
>   
> std::string retval("");
> if (str) {
>   for (int i = 0; (((len == -1) && (str[i] != '\0')) || (i < len)); ++i) {
> if (str[i] != '\\') {
>   retval += str[i];
> }
>   }
> }
> return retval;
>   }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1006) memory management, cut down memory waste ?

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1006:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> memory management, cut down memory waste ?
> --
>
> Key: TS-1006
> URL: https://issues.apache.org/jira/browse/TS-1006
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: 3.1.1
>Reporter: Zhao Yongming
>Assignee: Bin Chen
> Fix For: 3.3.6
>
> Attachments: 
> 0001-TS-1006-Add-an-enable-reclaimable-freelist-option.patch, 
> 0002-TS-1006-Add-a-new-wrapper-ink_atomic_decrement.patch, 
> 0003-TS-1006-Introduce-a-reclaimable-InkFreeList.patch, 
> 0004-TS-1006-Make-InkFreeList-memory-pool-configurable.patch, 
> Memory-Usage-After-Introduced-New-Allocator.png, memusage.ods, memusage.ods
>
>
> when we review the memory usage in the production, there is something 
> abnormal, ie, looks like TS take much memory than index data + common system 
> waste, and here is some memory dump result by set 
> "proxy.config.dump_mem_info_frequency"
> 1, the one on a not so busy forwarding system:
> physics memory: 32G
> RAM cache: 22G
> DISK: 6140 GB
> average_object_size 64000
> {code}
>  allocated  |in-use  | type size  |   free list name
> |||--
>   671088640 |   37748736 |2097152 | 
> memory/ioBufAllocator[14]
>  2248146944 | 2135949312 |1048576 | 
> memory/ioBufAllocator[13]
>  1711276032 | 1705508864 | 524288 | 
> memory/ioBufAllocator[12]
>  1669332992 | 1667760128 | 262144 | 
> memory/ioBufAllocator[11]
>  2214592512 | 221184 | 131072 | 
> memory/ioBufAllocator[10]
>  2325741568 | 2323775488 |  65536 | 
> memory/ioBufAllocator[9]
>  2091909120 | 2089123840 |  32768 | 
> memory/ioBufAllocator[8]
>  1956642816 | 1956478976 |  16384 | 
> memory/ioBufAllocator[7]
>  2094530560 | 2094071808 |   8192 | 
> memory/ioBufAllocator[6]
>   356515840 |  355540992 |   4096 | 
> memory/ioBufAllocator[5]
> 1048576 |  14336 |   2048 | 
> memory/ioBufAllocator[4]
>  131072 |  0 |   1024 | 
> memory/ioBufAllocator[3]
>   65536 |  0 |512 | 
> memory/ioBufAllocator[2]
>   32768 |  0 |256 | 
> memory/ioBufAllocator[1]
>   16384 |  0 |128 | 
> memory/ioBufAllocator[0]
>   0 |  0 |576 | 
> memory/ICPRequestCont_allocator
>   0 |  0 |112 | 
> memory/ICPPeerReadContAllocator
>   0 |  0 |432 | 
> memory/PeerReadDataAllocator
>   0 |  0 | 32 | 
> memory/MIMEFieldSDKHandle
>   0 |  0 |240 | 
> memory/INKVConnAllocator
>   0 |  0 | 96 | 
> memory/INKContAllocator
>4096 |  0 | 32 | 
> memory/apiHookAllocator
>   0 |  0 |288 | 
> memory/FetchSMAllocator
>   0 |  0 | 80 | 
> memory/prefetchLockHandlerAllocator
>   0 |  0 |176 | 
> memory/PrefetchBlasterAllocator
>   0 |  0 | 80 | 
> memory/prefetchUrlBlaster
>   0 |  0 | 96 | memory/blasterUrlList
>   0 |  0 | 96 | 
> memory/prefetchUrlEntryAllocator
>   0 |  0 |128 | 
> memory/socksProxyAllocator
>   0 |  0 |144 | 
> memory/ObjectReloadCont
> 3258368 | 576016 |592 | 
> memory/httpClientSessionAllocator
>  825344 | 139568 |208 | 
> memory/httpServerSessionAllocator
>22597632 |1284848 |   9808 | memory/httpSMAllocator
>   0 |  0 | 32 | 
> memory/CacheLookupHttpConfigAllocator
>   0 |  0 |   9856 | 
> memory/httpUpdateSMAllocator
>   0 |  0 |128 | 
> memory/RemapPluginsAlloc
>   0 |  0 | 48 | 
> memory/CongestRequestParamAllocator
>  

[jira] [Updated] (TS-1337) Extend TS API to support TSHttpConnect with outbound transparency

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1337:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> Extend TS API to support TSHttpConnect with outbound transparency
> -
>
> Key: TS-1337
> URL: https://issues.apache.org/jira/browse/TS-1337
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: TS API
>Reporter: James Peach
>Assignee: Alan M. Carroll
>Priority: Minor
> Fix For: 3.3.6
>
> Attachments: ASF.LICENSE.NOT.GRANTED--api_transparency.diff, 
> ASF.LICENSE.NOT.GRANTED--ts-port-descriptor.patch
>
>
> This is required for protocol plugins to use this capability.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1975) LocalManager may cause manager crash

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1975:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

Moving out to v3.3.6 for now, which means unless someone moves it back to 
v3.3.5, this will not go into v3.4.0.

> LocalManager may cause manager crash
> 
>
> Key: TS-1975
> URL: https://issues.apache.org/jira/browse/TS-1975
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Management
>Affects Versions: 3.3.4
>Reporter: Zhao Yongming
>Assignee: portl4t
> Fix For: 3.3.6
>
>
> when something wrong with the LocalManager, with 
> [LocalManager::pollMgmtProcessServer] Error in read (errno: 104), then you 
> will get manager and server restart.
> {code}
> Jun 17 17:40:06 cache163 traffic_manager[25654]: {0x7f528b4297e0} FATAL: 
> [LocalManager::pollMgmtProcessServer] Error in read (errno: 104)
> Jun 17 17:40:06 cache163 traffic_manager[25654]: {0x7f528b4297e0} FATAL:  
> (last system error 104: Connection reset by peer)
> Jun 17 17:40:06 cache163 traffic_manager[25654]: {0x7f528b4297e0} ERROR: 
> [LocalManager::sendMgmtMsgToProcesses] Error writing message
> Jun 17 17:40:06 cache163 traffic_manager[25654]: {0x7f528b4297e0} ERROR:  
> (last system error 32: Broken pipe)
> Jun 17 17:40:07 cache163 traffic_cop[25652]: cop received child status signal 
> [25654 2816]
> Jun 17 17:40:07 cache163 traffic_cop[25652]: traffic_manager not running, 
> making sure traffic_server is dead
> Jun 17 17:40:07 cache163 traffic_cop[25652]: spawning traffic_manager
> Jun 17 17:40:07 cache163 traffic_manager[10118]: NOTE: --- Manager Starting 
> ---
> Jun 17 17:40:07 cache163 traffic_manager[10118]: NOTE: Manager Version: 
> Apache Traffic Server - traffic_manager - 3.2.0 - (build # 51516 on Jun 15 
> 2013 at 16:01:06)
> Jun 17 17:40:07 cache163 traffic_manager[10118]: NOTE: 
> RLIMIT_NOFILE(7):cur(16),max(16)
> Jun 17 17:40:07 cache163 traffic_manager[10118]: {0x7f26fc24a7e0} STATUS: 
> opened /var/log/trafficserver/manager.log
> Jun 17 17:40:09 cache163 traffic_server[10131]: NOTE: --- Server Starting ---
> Jun 17 17:40:09 cache163 traffic_server[10131]: NOTE: Server Version: Apache 
> Traffic Server - traffic_server - 3.2.0 - (build # 51516 on Jun 15 2013 at 
> 16:01:31)
> Jun 17 17:40:09 cache163 traffic_server[10131]: {0x2b167ded2280} STATUS: 
> opened /var/log/trafficserver/diags.log
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

[Tommy Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698224#comment-13698224
 ] 

Tommy Lee commented on TS-1956:
---

Leif, thanks for the efforts.

In some conditions the traffic_server restarts after some days with one strange 
crash. No problem with this, as we know that traffic_cop and traffic_manager 
takes care of that. But with this crash, ATS get stucked.

I can reproduce easily here simulating traffic server crash by killing the 
process.

Do you need more debug infos ? Please let me know and I will fill the ticket.

Thanks again.

> Under Heavy Load TS 3.3.4-dev can't (re)start
> -
>
> Key: TS-1956
> URL: https://issues.apache.org/jira/browse/TS-1956
> Project: Traffic Server
>  Issue Type: Bug
>Reporter: Tommy Lee
>Priority: Blocker
> Fix For: 3.3.6
>
> Attachments: backtrace.log
>
>
> Hi,
>  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
> 3.3.2-dev without problems.
>  But today, we tried to upgrade to version 3.3.4-dev without lucky.
>  We've noticed that, if TS restarts, it enters in this Segfault Loop.
>  Below are traffic.out logs with debug .*
>  I'll try to debug with GDB too, but I cannot stop this server for too long, 
> because of our operations.
>   Thanks in advance.
> 
> {code}
>  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fd9e0 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fdd00 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.81.5:46089 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.101.4:41361 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.156.38:59164 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
> 10.202.81.5:46089 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.35.9:51533 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.201.20:10964 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
> 10.200.156.38:59164 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.148.2:44203 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fe020 on port 3128 as inbound transparent
> [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
> 10.202.101.4:41361 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
> 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
> Segmentation fault
> /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> NOTE: Traffic Server received Sig 11: Segmentation fault
> [Jun 14 11:54:14.564] Ser

[jira] [Commented] (TS-1327) Crash report: thread_alloc (this=0x1162060, t=0x0), in HttpSM::do_http_server_open

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698216#comment-13698216
 ] 

Leif Hedstrom commented on TS-1327:
---

Weijin: Can you take this and propose a fix? Or should we move out to 3.5.0?

> Crash report: thread_alloc (this=0x1162060, t=0x0), in 
> HttpSM::do_http_server_open
> --
>
> Key: TS-1327
> URL: https://issues.apache.org/jira/browse/TS-1327
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HTTP, Network
>Affects Versions: 3.3.0
> Environment: tree 3.2.x x86_64 cluster type=1, with some patches in 
> cluster
>Reporter: Zhao Yongming
>Assignee: weijin
> Fix For: 3.3.5
>
>
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 
> 8080:fd=12,80:fd=13'.
> Program terminated with signal 11, Segmentation fault.
> #0  thread_alloc (this=0x1162060, t=0x0) at 
> ../../iocore/eventsystem/I_ProxyAllocator.h:50
> 50if (l.freelist) {
> Missing separate debuginfos, use: debuginfo-install 
> expat-2.0.1-9.1.el6.x86_64 glibc-2.12-1.47.el6.x86_64 
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6.x86_64 
> libattr-2.4.44-7.el6.x86_64 libcap-2.16-5.5.el6.x86_64 
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 
> ncurses-libs-5.7-3.20090208.el6.x86_64 openssl-1.0.0-20.el6.x86_64 
> pcre-7.8-3.1.el6.x86_64 readline-6.0-3.el6.x86_64 tcl-8.5.7-6.el6.x86_64 
> xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0  thread_alloc (this=0x1162060, t=0x0) at 
> ../../iocore/eventsystem/I_ProxyAllocator.h:50
> #1  UnixNetProcessor::allocateThread (this=0x1162060, t=0x0) at 
> UnixNetProcessor.cc:474
> #2  0x00645c91 in UnixNetProcessor::connect_re_internal 
> (this=0x1162060, cont=, target=0x2aba94506d88, 
> opt=0x2aba4dc8f980) at UnixNetProcessor.cc:200
> #3  0x00525587 in connect_re (this=0x2aba945066f0, raw= optimized out>) at ../../iocore/net/P_UnixNetProcessor.h:89
> #4  HttpSM::do_http_server_open (this=0x2aba945066f0, raw= out>) at HttpSM.cc:4333
> #5  0x005291f8 in HttpSM::set_next_state (this=0x2aba945066f0) at 
> HttpSM.cc:6591
> #6  0x0052a128 in HttpSM::state_send_server_request_header 
> (this=0x2aba945066f0, event=104, data=0x2aba90010da8) at HttpSM.cc:1932
> #7  0x005202b8 in HttpSM::main_handler (this=0x2aba945066f0, 
> event=104, data=0x2aba90010da8) at HttpSM.cc:2463
> #8  0x00648651 in handleEvent (event=, 
> nh=0x2aba4d4901e8, vc=0x2aba90010ca0)
> at ../../iocore/eventsystem/I_Continuation.h:146
> #9  read_signal_and_update (event=, nh=0x2aba4d4901e8, 
> vc=0x2aba90010ca0) at UnixNetVConnection.cc:138
> #10 read_signal_done (event=, nh=0x2aba4d4901e8, 
> vc=0x2aba90010ca0) at UnixNetVConnection.cc:168
> #11 0x0064abf5 in read_from_net (nh=0x2aba4d4901e8, 
> vc=0x2aba90010ca0, thread=) at UnixNetVConnection.cc:291
> #12 0x00643c6a in NetHandler::mainNetEvent (this=0x2aba4d4901e8, 
> event=, e=)
> at UnixNet.cc:372
> #13 0x006698a4 in handleEvent (this=0x2aba4d48d010, e=0x1b0fdc0, 
> calling_code=5) at I_Continuation.h:146
> #14 EThread::process_event (this=0x2aba4d48d010, e=0x1b0fdc0, calling_code=5) 
> at UnixEThread.cc:142
> #15 0x0066a233 in EThread::execute (this=0x2aba4d48d010) at 
> UnixEThread.cc:264
> #16 0x00668b72 in spawn_thread_internal (a=0x1ac9640) at Thread.cc:88
> #17 0x0032d38077f1 in start_thread () from /lib64/libpthread.so.0
> #18 0x0032d34e570d in clone () from /lib64/libc.so.6
> (gdb) 
> (gdb) f 2
> #2  0x00645c91 in UnixNetProcessor::connect_re_internal 
> (this=0x1162060, cont=, target=0x2aba94506d88, 
> opt=0x2aba4dc8f980) at UnixNetProcessor.cc:200
> 200   UnixNetVConnection *vc = allocateThread(t);
> (gdb) l
> 195   sockaddr const* target,
> 196   NetVCOptions * opt
> 197 ) {
> 198   ProxyMutex *mutex = cont->mutex;
> 199   EThread *t = mutex->thread_holding;
> 200   UnixNetVConnection *vc = allocateThread(t);
> 201
> 202   if (opt)
> 203 vc->options = *opt;
> 204   else
> (gdb) p *t
> Cannot access memory at address 0x0
> (gdb) 
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1493) TShtrime() returning negative value

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1493:
--

Fix Version/s: (was: 3.3.5)
   3.5.0
   Labels:   (was: A)

Bianca: I'm unfortunately going to have to move this out to v3.5.0 for now. Any 
chance you guys can work on this, and propose a fix? If so, please move the bug 
back to v3.3.5 and I promise to review it.

> TShtrime() returning negative value
> ---
>
> Key: TS-1493
> URL: https://issues.apache.org/jira/browse/TS-1493
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Reporter: bianca cooper
> Fix For: 3.5.0
>
>
> calling TShrtime function after TS_EVENT_HTTP_TXN_CLOSE (and before reenable 
> the transaction) sometimes returns negative number 
> calling the same function in other places in code always returns positive 
> number. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1219) Crash report: ink_freelist_new causing core dumps

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1219:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

> Crash report: ink_freelist_new causing core dumps
> -
>
> Key: TS-1219
> URL: https://issues.apache.org/jira/browse/TS-1219
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 3.0.4, 3.0.2
>Reporter: Manjesh Nilange
>  Labels: A, crash
> Fix For: 3.3.6
>
>
> Our TS based production boxes are seeing a couple of crashes per day with 
> stack traces ending in
> #3  0x004c85ea in signal_handler (sig=11) at signals.cc:225
> #4  
> #5  0x003312414160 in ink_freelist_new (f=0x3312629ce0) at 
> ink_queue.cc:326
> #6  0x00331240d47a in alloc_void (this=0x7fff80173cd8, size=511, 
> alignment=1) at Allocator.h:208
> #7  blk_alloc (this=0x7fff80173cd8, size=511, alignment=1) at Arena.cc:45
> #8  Arena::alloc (this=0x7fff80173cd8, size=511, alignment=1) at Arena.cc:118
> #9  0x00569b93 in str_alloc (arena=, 
> url=0x31be6a8 "*"..., 
> len_in=, len_out=0x7fff80173d20) at 
> ../../lib/ts/Arena.h:123
> #10 LogUtils::escapify_url (arena=, 
> url=0x31be6a8 "*"..., 
> len_in=, len_out=0x7fff80173d20) at LogUtils.cc:392
> #11 0x005482e9 in LogAccessHttp::init (this=0x7fff80173cc0) at 
> LogAccessHttp.cc:98
> #12 0x0054b436 in Log::access (lad=0x7fff80173cc0) at Log.cc:1055
> #13 0x004f3085 in HttpSM::update_stats (this=0x2ae5480651e0) at 
> HttpSM.cc:6044
> #14 0x004f8918 in HttpSM::kill_this (this=0x2ae5480651e0) at 
> HttpSM.cc:6005
> #15 0x004f8c08 in HttpSM::main_handler (this=0x2ae5480651e0, 
> event=2301, data=0x2ae548066ec8)
> at HttpSM.cc:2452
> The URL was valid, I just "anonymized" it. This is our environment
> $ uname -a
> Linux xxx.prod 2.6.32-131.4.1.el6.x86_64 #1 SMP Fri Jun 10 10:54:26 EDT 2011 
> x86_64 x86_64 x86_64 GNU/Linux
> $ file /usr/bin/traffic_server 
> /usr/bin/traffic_server: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), 
> dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped
> There doesn't seem to be more useful information in the frame:
> (gdb) f 5
> #5  0x003312414160 in ink_freelist_new (f=0x3312629ce0) at 
> ink_queue.cc:326
> 326  FREELIST_VERSION(item) + 1);
> (gdb) list
> 321 fastalloc_mem_in_use += f->chunk_size * f->type_size;
> 322   #endif
> 323   
> 324   } else {
> 325 SET_FREELIST_POINTER_VERSION(next, 
> *ADDRESS_OF_NEXT(TO_PTR(FREELIST_POINTER(item)), f->offset),
> 326  FREELIST_VERSION(item) + 1);
> 327   #if !defined(INK_USE_MUTEX_FOR_FREELISTS)
> 328 result = ink_atomic_cas64((int64_t *) & f->head.data, item.data, 
> next.data);
> 329   #else
> 330 f->head.data = next.data;
> (gdb) p next
> $2 = 
> (gdb) p f
> $3 = (InkFreeList *) 0x3312629ce0
> (gdb) p *f 
> $4 = {head = {data = -8941773651046140890}, name = 0x331241e92f "ArenaBlock", 
> type_size = 1024, chunk_size = 128, 
>   count = 519, allocated = 1536, offset = 0, alignment = 8, allocated_base = 
> 0, count_base = 0}
> (gdb) p item
> $5 = {data = -8953314125091277786}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1219) Crash report: ink_freelist_new causing core dumps

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698215#comment-13698215
 ] 

Leif Hedstrom commented on TS-1219:
---

Any updates on this? Still an issue?

> Crash report: ink_freelist_new causing core dumps
> -
>
> Key: TS-1219
> URL: https://issues.apache.org/jira/browse/TS-1219
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 3.0.4, 3.0.2
>Reporter: Manjesh Nilange
>  Labels: A, crash
> Fix For: 3.3.6
>
>
> Our TS based production boxes are seeing a couple of crashes per day with 
> stack traces ending in
> #3  0x004c85ea in signal_handler (sig=11) at signals.cc:225
> #4  
> #5  0x003312414160 in ink_freelist_new (f=0x3312629ce0) at 
> ink_queue.cc:326
> #6  0x00331240d47a in alloc_void (this=0x7fff80173cd8, size=511, 
> alignment=1) at Allocator.h:208
> #7  blk_alloc (this=0x7fff80173cd8, size=511, alignment=1) at Arena.cc:45
> #8  Arena::alloc (this=0x7fff80173cd8, size=511, alignment=1) at Arena.cc:118
> #9  0x00569b93 in str_alloc (arena=, 
> url=0x31be6a8 "*"..., 
> len_in=, len_out=0x7fff80173d20) at 
> ../../lib/ts/Arena.h:123
> #10 LogUtils::escapify_url (arena=, 
> url=0x31be6a8 "*"..., 
> len_in=, len_out=0x7fff80173d20) at LogUtils.cc:392
> #11 0x005482e9 in LogAccessHttp::init (this=0x7fff80173cc0) at 
> LogAccessHttp.cc:98
> #12 0x0054b436 in Log::access (lad=0x7fff80173cc0) at Log.cc:1055
> #13 0x004f3085 in HttpSM::update_stats (this=0x2ae5480651e0) at 
> HttpSM.cc:6044
> #14 0x004f8918 in HttpSM::kill_this (this=0x2ae5480651e0) at 
> HttpSM.cc:6005
> #15 0x004f8c08 in HttpSM::main_handler (this=0x2ae5480651e0, 
> event=2301, data=0x2ae548066ec8)
> at HttpSM.cc:2452
> The URL was valid, I just "anonymized" it. This is our environment
> $ uname -a
> Linux xxx.prod 2.6.32-131.4.1.el6.x86_64 #1 SMP Fri Jun 10 10:54:26 EDT 2011 
> x86_64 x86_64 x86_64 GNU/Linux
> $ file /usr/bin/traffic_server 
> /usr/bin/traffic_server: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), 
> dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped
> There doesn't seem to be more useful information in the frame:
> (gdb) f 5
> #5  0x003312414160 in ink_freelist_new (f=0x3312629ce0) at 
> ink_queue.cc:326
> 326  FREELIST_VERSION(item) + 1);
> (gdb) list
> 321 fastalloc_mem_in_use += f->chunk_size * f->type_size;
> 322   #endif
> 323   
> 324   } else {
> 325 SET_FREELIST_POINTER_VERSION(next, 
> *ADDRESS_OF_NEXT(TO_PTR(FREELIST_POINTER(item)), f->offset),
> 326  FREELIST_VERSION(item) + 1);
> 327   #if !defined(INK_USE_MUTEX_FOR_FREELISTS)
> 328 result = ink_atomic_cas64((int64_t *) & f->head.data, item.data, 
> next.data);
> 329   #else
> 330 f->head.data = next.data;
> (gdb) p next
> $2 = 
> (gdb) p f
> $3 = (InkFreeList *) 0x3312629ce0
> (gdb) p *f 
> $4 = {head = {data = -8941773651046140890}, name = 0x331241e92f "ArenaBlock", 
> type_size = 1024, chunk_size = 128, 
>   count = 519, allocated = 1536, offset = 0, alignment = 8, allocated_base = 
> 0, count_base = 0}
> (gdb) p item
> $5 = {data = -8953314125091277786}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1050) Cached objects become inaccessible when new volumes are added to the cache.

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1050:
--

Fix Version/s: (was: 3.3.5)
   sometime

> Cached objects become inaccessible when new volumes are added to the cache.
> ---
>
> Key: TS-1050
> URL: https://issues.apache.org/jira/browse/TS-1050
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache
>Affects Versions: 3.1.1
>Reporter: B Wyatt
> Fix For: sometime
>
>
> During the investigation of TS-949 it came to light that even a consistent 
> hashing solution fails to maintain access for all objects in the cache when a 
> new volume is added.  This is because a new volume will necessarily assume 
> ownership for some portion of the object/hash space.  As a side effect of 
> this ownership change, new searches for previously cached objects may return 
> the new owner instead of the previous owner (which retained the cached copy). 
>  According to the volume itself (and thus several aggregate statistics like 
> cache space usage), the objects are still valid and will remain as effective 
> dead weight until evicted during the normal cache operation.
> See comments on TS-949 for initial discussion of possible solutions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1997) Deprecate TSHttpTxnCachedUrlSet()

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1997:
--

Fix Version/s: 3.3.5
 Assignee: Leif Hedstrom
   Labels: A  (was: )

> Deprecate TSHttpTxnCachedUrlSet()
> -
>
> Key: TS-1997
> URL: https://issues.apache.org/jira/browse/TS-1997
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: TS API
>Reporter: Leif Hedstrom
>Assignee: Leif Hedstrom
>  Labels: A
> Fix For: 3.3.5
>
>
> This is effectively replaced with TSCacheUrlSet(), which will hopefully be 
> replaced with the outcome of TS-1118.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1997) Deprecate TSHttpTxnCachedUrlSet()

[Leif Hedstrom (JIRA)]

Leif Hedstrom created TS-1997:
-

 Summary: Deprecate TSHttpTxnCachedUrlSet()
 Key: TS-1997
 URL: https://issues.apache.org/jira/browse/TS-1997
 Project: Traffic Server
  Issue Type: Improvement
  Components: TS API
Reporter: Leif Hedstrom


This is effectively replaced with TSCacheUrlSet(), which will hopefully be 
replaced with the outcome of TS-1118.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1439) Replace functionality of TSHttpTxnNewCacheLookupDo

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1439:
--

Fix Version/s: (was: 3.3.5)
   3.5.0

> Replace functionality of TSHttpTxnNewCacheLookupDo
> --
>
> Key: TS-1439
> URL: https://issues.apache.org/jira/browse/TS-1439
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: TS API
>Reporter: Leif Hedstrom
>Assignee: Phil Sorber
> Fix For: 3.5.0
>
>
> We have a set of APIs (in ts/experimtental.h) to do a "second" CacheSM / 
> cache lookup from a plugin API. This includes the API 
> TSHttpTxnNewCacheLookupDo(). We now also have a plugin that depends on this 
> behavior, mostly as a side effect (it wants the second cache lookup, but with 
> the same cache key).
> I think the implementation of this is rather unfortunate, having a second 
> CacheSM where you provide a second cache URL (to parse / MD5 etc.) to perform 
> this second lookup. It would be noticeably cleaner to be able to reuse the 
> CacheSM, and move the HttpSM back in the state chain, to retry a cache lookup 
> again. This would allow a plugin to retry cache lookups any number of times 
> for example, and also lets us unify on a single cache key API.
> The latter is important for some of the changes I would like to do around the 
> cacheURL. Right now, we store (and set) the cache URL, which means that a 
> plugin who wishes to change the cache key has to create an artificial URL, 
> which gets parsed, and then turned into an MD5. We should eliminate this, and 
> provide Set/Get APIs to manipulate the (one and only) CacheSM's MD5 cache-key 
> directly from the APIs. This *could* be via a URL, but could be done on 
> whatever the plugin would prefer (e.g. an MD5 of a combination of the URL and 
> the request headers).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1439) Replace functionality of TSHttpTxnNewCacheLookupDo

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1439:
--

Summary: Replace functionality of TSHttpTxnNewCacheLookupDo  (was: 
Deprecate TSHttpTxnNewCacheLookupDo)

> Replace functionality of TSHttpTxnNewCacheLookupDo
> --
>
> Key: TS-1439
> URL: https://issues.apache.org/jira/browse/TS-1439
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: TS API
>Reporter: Leif Hedstrom
>Assignee: Phil Sorber
> Fix For: 3.3.5
>
>
> We have a set of APIs (in ts/experimtental.h) to do a "second" CacheSM / 
> cache lookup from a plugin API. This includes the API 
> TSHttpTxnNewCacheLookupDo(). We now also have a plugin that depends on this 
> behavior, mostly as a side effect (it wants the second cache lookup, but with 
> the same cache key).
> I think the implementation of this is rather unfortunate, having a second 
> CacheSM where you provide a second cache URL (to parse / MD5 etc.) to perform 
> this second lookup. It would be noticeably cleaner to be able to reuse the 
> CacheSM, and move the HttpSM back in the state chain, to retry a cache lookup 
> again. This would allow a plugin to retry cache lookups any number of times 
> for example, and also lets us unify on a single cache key API.
> The latter is important for some of the changes I would like to do around the 
> cacheURL. Right now, we store (and set) the cache URL, which means that a 
> plugin who wishes to change the cache key has to create an artificial URL, 
> which gets parsed, and then turned into an MD5. We should eliminate this, and 
> provide Set/Get APIs to manipulate the (one and only) CacheSM's MD5 cache-key 
> directly from the APIs. This *could* be via a URL, but could be done on 
> whatever the plugin would prefer (e.g. an MD5 of a combination of the URL and 
> the request headers).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1996) Deprecate TSHttpTxnNewCacheLookupDo

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1996:
--

Fix Version/s: 3.3.5
 Assignee: Leif Hedstrom
   Labels: A  (was: )

> Deprecate TSHttpTxnNewCacheLookupDo
> ---
>
> Key: TS-1996
> URL: https://issues.apache.org/jira/browse/TS-1996
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: TS API
>Reporter: Leif Hedstrom
>Assignee: Leif Hedstrom
>  Labels: A
> Fix For: 3.3.5
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1996) Deprecate TSHttpTxnNewCacheLookupDo

[Leif Hedstrom (JIRA)]

Leif Hedstrom created TS-1996:
-

 Summary: Deprecate TSHttpTxnNewCacheLookupDo
 Key: TS-1996
 URL: https://issues.apache.org/jira/browse/TS-1996
 Project: Traffic Server
  Issue Type: Improvement
  Components: TS API
Reporter: Leif Hedstrom




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1995) After TS-1740, traffic_shell and traffic_line starts to show zero stats in some fields.

[Tommy Lee (JIRA)]

Tommy Lee created TS-1995:
-

 Summary: After TS-1740, traffic_shell and traffic_line starts to 
show zero stats in some fields.
 Key: TS-1995
 URL: https://issues.apache.org/jira/browse/TS-1995
 Project: Traffic Server
  Issue Type: Bug
  Components: Stats
Reporter: Tommy Lee


In 3.3.1-dev traffic_shell and traffic_line shows this output:

% show:proxy-stats

Document Hit Rate  2.868852 %*
Ram cache Hit Rate --- 0.00 %*
Bandwidth Saving - 0.330866 %*
Cache Percent Free --- 70.415193 %
Open Server Connections -- 17
Open Client Connections -- 11
Open Cache Connections --- 1
Client Throughput  0.048656 MBit/Sec
Transaction Per Second --- 0.299951


But after apply TS-1740 patch the output is always this:

% show:proxy-stats

Document Hit Rate  0.00 %*
Ram cache Hit Rate --- 0.00 %*
Bandwidth Saving - 0.00 %*
Cache Percent Free --- 0.00 %
Open Server Connections -- 43
Open Client Connections -- 19
Open Cache Connections --- 0
Client Throughput  9123.543945 MBit/Sec
Transaction Per Second --- 0.00



I'm trying to exclude only this patch with 3.3.2-dev but without luck.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1409) Add webdav methods to ip allow/quick filter

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1409:
--

Fix Version/s: (was: 3.3.5)
   3.5.0

> Add webdav methods to ip allow/quick filter
> ---
>
> Key: TS-1409
> URL: https://issues.apache.org/jira/browse/TS-1409
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Security
>Affects Versions: 3.2.0
>Reporter: Bryan Call
>Assignee: Bryan Call
> Fix For: 3.5.0
>
>
> I know of PROPFIND and REPORT should be added.  There might need to be more 
> added.
> HTTP Extensions for Distributed Authoring -- WEBDAV
> http://www.ietf.org/rfc/rfc2518.txt

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1409) Add webdav methods to ip allow/quick filter

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698203#comment-13698203
 ] 

Leif Hedstrom commented on TS-1409:
---

Since there are no takers, moving this out to v3.5.0.


> Add webdav methods to ip allow/quick filter
> ---
>
> Key: TS-1409
> URL: https://issues.apache.org/jira/browse/TS-1409
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Security
>Affects Versions: 3.2.0
>Reporter: Bryan Call
>Assignee: Bryan Call
> Fix For: 3.3.5
>
>
> I know of PROPFIND and REPORT should be added.  There might need to be more 
> added.
> HTTP Extensions for Distributed Authoring -- WEBDAV
> http://www.ietf.org/rfc/rfc2518.txt

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1573) does rolling options work for ATS?

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698169#comment-13698169
 ] 

Leif Hedstrom commented on TS-1573:
---

Is this still an issue? If so, please reopen. I've tested this in a number of 
ways, and it seems to rotate my logs fine. 

I remember vaguely there were some cases at Yahoo where log rotation stopped 
doing its magic, but there were never any consistent results or ways to 
reproduce.

> does rolling options work for ATS?
> --
>
> Key: TS-1573
> URL: https://issues.apache.org/jira/browse/TS-1573
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Logging
>Affects Versions: 3.2.0
> Environment: RHEL6.3
>Reporter: Aidan McGurn
> Fix For: 3.3.5
>
>
> I have configured my rolling options as follows:
> CONFIG proxy.config.log.rolling_enabled INT 1 << enabled
> CONFIG proxy.config.log.rolling_interval_sec INT 300   << min 5 mins
> CONFIG proxy.config.log.rolling_offset_hr INT 16  << 16 = 4pm today  (also 
> left this running from yesterday)
> Above should have rolled @4pm and every 5 min's thereafter….
> I tried this and left it overnight without any logs appearing..
> looked under:
> .../tools/trafficserver/var/log/trafficserver/
> we have 2 logs from traffic server:
> 1. normal debug which is standard out which is redirected to file which the 
> system uses for all standard out
> 2. ATS's errors in error.log
> i.e. in this setup do we expect at most the error.log to roll?
> Also for #1, is this not working becuause of the way we redirect std::out,
> (TS launched like this:
> master process -> calls perl script execs -> ATM -> ATS  - std:out redirected 
> to central log file)
> could someone confirm or have an example of rolling working on 3.2.0 so i can 
> work around the above limitations we may have set for ourselves?
> thanks,
> /aidan

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Closed] (TS-1573) does rolling options work for ATS?

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom closed TS-1573.
-

Resolution: Cannot Reproduce

> does rolling options work for ATS?
> --
>
> Key: TS-1573
> URL: https://issues.apache.org/jira/browse/TS-1573
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Logging
>Affects Versions: 3.2.0
> Environment: RHEL6.3
>Reporter: Aidan McGurn
> Fix For: 3.3.5
>
>
> I have configured my rolling options as follows:
> CONFIG proxy.config.log.rolling_enabled INT 1 << enabled
> CONFIG proxy.config.log.rolling_interval_sec INT 300   << min 5 mins
> CONFIG proxy.config.log.rolling_offset_hr INT 16  << 16 = 4pm today  (also 
> left this running from yesterday)
> Above should have rolled @4pm and every 5 min's thereafter….
> I tried this and left it overnight without any logs appearing..
> looked under:
> .../tools/trafficserver/var/log/trafficserver/
> we have 2 logs from traffic server:
> 1. normal debug which is standard out which is redirected to file which the 
> system uses for all standard out
> 2. ATS's errors in error.log
> i.e. in this setup do we expect at most the error.log to roll?
> Also for #1, is this not working becuause of the way we redirect std::out,
> (TS launched like this:
> master process -> calls perl script execs -> ATM -> ATS  - std:out redirected 
> to central log file)
> could someone confirm or have an example of rolling working on 3.2.0 so i can 
> work around the above limitations we may have set for ourselves?
> thanks,
> /aidan

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1316) ATS connection refused once cache direntries exhausted

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1316:
--

Fix Version/s: (was: 3.3.5)
   3.5.0

> ATS connection refused once cache direntries exhausted
> --
>
> Key: TS-1316
> URL: https://issues.apache.org/jira/browse/TS-1316
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache
>Affects Versions: 3.2.0
> Environment: RHEL 6.2 x86_64
>Reporter: David Carlin
>  Labels: A
> Fix For: 3.5.0
>
>
> We had a pair of ATS 3.2.0 boxes that stopped passing traffic simultaneously. 
>  Here are the traffic.out msgs we saw on both boxes:
> [Jun 22 05:53:27.637] Server {0x2b6b6d9da700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 36, purging...
> [Jun 22 05:56:05.542] Server {0x2b6b6d2d3700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 85, purging...
> [Jun 22 05:56:07.434] Server {0x2b6b6d4d5700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 71, purging...
> [Jun 22 05:58:24.743] Server {0x2b6b6d8d9700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 33, purging...
> Those messages went on for a couple minutes, then traffic apparently ceased - 
> our monitoring system saw connection refused for port 80 on ATS from then on. 
> The connection refused state went on for many hours until ATS was restarted.  
> There were no traffic_cop msgs in /var/log/messages indicating that the 
> heartbeat failed.
> Here are the relevant ATS settings/stats:
> proxy.process.cache.bytes_total = 190690320384
> proxy.process.cache.direntries.total = 5817752
> proxy.config.cache.min_average_object_size = 32768
> We previously came up with proxy.config.cache.min_average_object_size by 
> waiting for the cache to fill and dividing proxy.process.cache.bytes_used by 
> proxy.process.cache.direntries.used - which equals about 34KB.
> We're assuming ATS ran out of direntries and it didn't handle this situation 
> gracefully.  As a possible workaround, I'm going to lower 
> proxy.config.cache.min_average_object_size to 24KB.
> Thanks to Bryan Call for helping me troubleshoot this!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Closed] (TS-1862) build error with --enable-linux-native-aio

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom closed TS-1862.
-

Resolution: Duplicate

> build error with --enable-linux-native-aio
> --
>
> Key: TS-1862
> URL: https://issues.apache.org/jira/browse/TS-1862
> Project: Traffic Server
>  Issue Type: Bug
>Reporter: bettydramit
> Fix For: 3.3.5
>
>
> make[2]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/net'
> Making all in aio
> make[2]: Entering directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/aio'
>   CXXAIO.o
>   CXXInline.o
> cc1plus: warnings being treated as errors
> AIO.cc:546: error: unused parameter 'event'
> AIO.cc:600: error: unused parameter 'fromAPI'
> AIO.cc:610: error: unused parameter 'fromAPI'
> AIO.cc:620: error: unused parameter 'fromAPI'
> AIO.cc:647: error: unused parameter 'fromAPI'
> make[2]: *** [AIO.o] Error 1
> make[2]: *** Waiting for unfinished jobs
> make[2]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/aio'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore'
> make: *** [all-recursive] Error 1
> error: Bad exit status from /var/tmp/rpm-tmp.Yz6dqL (%build)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1862) build error with --enable-linux-native-aio

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698165#comment-13698165
 ] 

Leif Hedstrom commented on TS-1862:
---

I believe this was fixed with TS-1941. Please reopen if this is still an issue.

> build error with --enable-linux-native-aio
> --
>
> Key: TS-1862
> URL: https://issues.apache.org/jira/browse/TS-1862
> Project: Traffic Server
>  Issue Type: Bug
>Reporter: bettydramit
> Fix For: 3.3.5
>
>
> make[2]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/net'
> Making all in aio
> make[2]: Entering directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/aio'
>   CXXAIO.o
>   CXXInline.o
> cc1plus: warnings being treated as errors
> AIO.cc:546: error: unused parameter 'event'
> AIO.cc:600: error: unused parameter 'fromAPI'
> AIO.cc:610: error: unused parameter 'fromAPI'
> AIO.cc:620: error: unused parameter 'fromAPI'
> AIO.cc:647: error: unused parameter 'fromAPI'
> make[2]: *** [AIO.o] Error 1
> make[2]: *** Waiting for unfinished jobs
> make[2]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore/aio'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/data/rpmbuild/BUILD/ts-3.3.3/iocore'
> make: *** [all-recursive] Error 1
> error: Bad exit status from /var/tmp/rpm-tmp.Yz6dqL (%build)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (TS-1316) ATS connection refused once cache direntries exhausted

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698166#comment-13698166
 ] 

Leif Hedstrom commented on TS-1316:
---

So, as serious as this is, the code to deal with this situation is, ehm, less 
than perfect :). I'm going to move this out for now, the short story is, don't 
run out of Directory entries.

> ATS connection refused once cache direntries exhausted
> --
>
> Key: TS-1316
> URL: https://issues.apache.org/jira/browse/TS-1316
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache
>Affects Versions: 3.2.0
> Environment: RHEL 6.2 x86_64
>Reporter: David Carlin
>  Labels: A
> Fix For: 3.3.5
>
>
> We had a pair of ATS 3.2.0 boxes that stopped passing traffic simultaneously. 
>  Here are the traffic.out msgs we saw on both boxes:
> [Jun 22 05:53:27.637] Server {0x2b6b6d9da700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 36, purging...
> [Jun 22 05:56:05.542] Server {0x2b6b6d2d3700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 85, purging...
> [Jun 22 05:56:07.434] Server {0x2b6b6d4d5700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 71, purging...
> [Jun 22 05:58:24.743] Server {0x2b6b6d8d9700} WARNING: cache directory 
> overflow on '/dev/dm-4' segment 33, purging...
> Those messages went on for a couple minutes, then traffic apparently ceased - 
> our monitoring system saw connection refused for port 80 on ATS from then on. 
> The connection refused state went on for many hours until ATS was restarted.  
> There were no traffic_cop msgs in /var/log/messages indicating that the 
> heartbeat failed.
> Here are the relevant ATS settings/stats:
> proxy.process.cache.bytes_total = 190690320384
> proxy.process.cache.direntries.total = 5817752
> proxy.config.cache.min_average_object_size = 32768
> We previously came up with proxy.config.cache.min_average_object_size by 
> waiting for the cache to fill and dividing proxy.process.cache.bytes_used by 
> proxy.process.cache.direntries.used - which equals about 34KB.
> We're assuming ATS ran out of direntries and it didn't handle this situation 
> gracefully.  As a possible workaround, I'm going to lower 
> proxy.config.cache.min_average_object_size to 24KB.
> Thanks to Bryan Call for helping me troubleshoot this!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Deleted] (TS-1982) Allow for @method=ALL in remap.config

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1982:
--

Comment: was deleted

(was: I think we should get this in for v3.4.)

> Allow for @method=ALL  in remap.config
> --
>
> Key: TS-1982
> URL: https://issues.apache.org/jira/browse/TS-1982
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Leif Hedstrom
> Fix For: sometime
>
>
> This makes it more consistent with the configurations for ip_allow.config.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1982) Allow for @method=ALL in remap.config

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1982:
--

Fix Version/s: (was: 3.3.5)
   sometime

> Allow for @method=ALL  in remap.config
> --
>
> Key: TS-1982
> URL: https://issues.apache.org/jira/browse/TS-1982
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Leif Hedstrom
> Fix For: sometime
>
>
> This makes it more consistent with the configurations for ip_allow.config.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1577) Crash report: RangeTransform::change_response_header at Transform.cc:995

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1577:
--

Fix Version/s: (was: 3.3.5)
   3.3.2

> Crash report: RangeTransform::change_response_header at Transform.cc:995
> 
>
> Key: TS-1577
> URL: https://issues.apache.org/jira/browse/TS-1577
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HTTP
>Affects Versions: 3.3.0
> Environment: git master version
>Reporter: Zhao Yongming
>Assignee: weijin
>Priority: Critical
> Fix For: 3.3.2
>
> Attachments: ts-1574.diff
>
>
> This may or may not relate to TS-1574, I'd like track this issue another 
> thread here.
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 8080:fd=12'.
> Program terminated with signal 6, Aborted.
> #0  0x003e86c32885 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install 
> expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.47.el6_2.9.x86_64 
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64 
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 
> openssl-1.0.0-20.el6_2.4.x86_64 pcre-7.8-3.1.el6.x86_64 
> tcl-8.5.7-6.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 
> zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0  0x003e86c32885 in raise () from /lib64/libc.so.6
> #1  0x003e86c34065 in abort () from /lib64/libc.so.6
> #2  0x0035c8013f19 in ink_die_die_die (retval=30342) at ink_error.cc:43
> #3  0x0035c8014194 in ink_fatal_va(int, const char *, typedef 
> __va_list_tag __va_list_tag *) (return_code=1, 
> message_format=, ap=0x2b0b8fcc3ba0) at 
> ink_error.cc:65
> #4  0x0035c80142c8 in ink_fatal (return_code=30342, message_format=0x7693 
> ) at ink_error.cc:73
> #5  0x0035c8012c3f in _ink_assert (expression=0x0, file=0x6  out of bounds>, line=-1) at ink_assert.cc:38
> #6  0x004d671c in RangeTransform::change_response_header 
> (this=0x2b0be4500bf0) at Transform.cc:995
> #7  0x004da4cd in RangeTransform::handle_event (this=0x2b0be4500bf0, 
> event=, edata=)
> at Transform.cc:791
> #8  0x00654dd4 in handleEvent (this=0x2b0b8e4ad010, e=0x34aff40, 
> calling_code=1) at I_Continuation.h:146
> #9  EThread::process_event (this=0x2b0b8e4ad010, e=0x34aff40, calling_code=1) 
> at UnixEThread.cc:142
> #10 0x0065593b in EThread::execute (this=0x2b0b8e4ad010) at 
> UnixEThread.cc:193
> #11 0x006540d2 in spawn_thread_internal (a=0x2c79a50) at Thread.cc:88
> #12 0x003e878077f1 in start_thread () from /lib64/libpthread.so.0
> #13 0x003e86ce5ccd in clone () from /lib64/libc.so.6
> (gdb) f 6
> #6  0x004d671c in RangeTransform::change_response_header 
> (this=0x2b0be4500bf0) at Transform.cc:995
> 995 
> ink_release_assert(m_transform_resp->field_find(MIME_FIELD_CONTENT_RANGE, 
> MIME_LEN_CONTENT_RANGE) == NULL);
> (gdb) p this
> $1 = (RangeTransform * const) 0x2b0be4500bf0
> (gdb) p *this
> $2 = { = { = { = 
> { = { = { = {
>   _vptr.force_VFPT_to_top = 0x667970}, handler = (int 
> (Continuation::*)(Continuation *, int, 
> void *)) 0x4da200 , mutex = 
> {m_ptr = 0x2b0bf8216110}, link = {> = {next = 0x0}, 
>   prev = 0x0}}, lerrno = 0}, }, mdata = 0x0, 
> m_event_func = 0, m_event_count = 0, m_closed = 0, m_deletable = 0, 
>   m_deleted = 0, m_free_magic = INKCONT_INTERN_MAGIC_ALIVE}, m_read_vio = 
> {_cont = 0x0, nbytes = 0, ndone = 0, op = 0, buffer = {mbuf = 0x0, 
> entry = 0x0}, vc_server = 0x0, mutex = {m_ptr = 0x0}}, m_write_vio = 
> {_cont = 0x2b0f9168f020, nbytes = 20480, ndone = 0, op = 2, buffer = {
> mbuf = 0x2b0c0407fdd0, entry = 0x2b0c0407fe10}, vc_server = 
> 0x2b0be4500bf0, mutex = {m_ptr = 0x2b0bf8216110}}, 
> m_output_vc = 0x2b0be459f048}, m_output_buf = 0x33b7ca0, m_output_reader 
> = 0x33b7cb8, m_transform_resp = 0x2b0f9168dc88, 
>   m_output_vio = 0x2b0be459f0c8, m_unsatisfiable_range = false, 
> m_range_content_length = 0, m_num_chars_for_cl = 1, m_num_range_fields = -1, 
>   m_current_range = 0, 
>   m_content_type = 0x2b1216ea6ac0 "application/octet-stream\r\nContent-Range: 
> bytes 0-20479/119091\r\nContent-Length: 20480\r\nConnection: 
> keep-alive\r\nDate: Sun, 18 Nov 2012 13:30:05 GMT\r\nServer: 
> apache\r\n\r\n\343k\031P", m_content_type_len = 24, m_ranges = 
> 0x2b0be45bda90, 
>   m_output_cl = 20480, m_done = 0}
> (gdb) p m_ranges
> $3 = (RangeRecord *) 0x2b0be45bda90
> (gdb) p *m_ranges
> $4 = {_start = 20480, _end = 40959, _done_byte = -1}
> (gdb) p *m_transform_resp
> $1 = { = { = {m_heap = 0x2b0f914ec010}, m_mime = 
> 0x2b0f914ec0c8}, m_http = 0x2b0f914ec098, 
>   m_url_cached = { = {m_

[jira] [Resolved] (TS-1577) Crash report: RangeTransform::change_response_header at Transform.cc:995

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom resolved TS-1577.
---

Resolution: Fixed

This seems to have been fixed, so closing.

> Crash report: RangeTransform::change_response_header at Transform.cc:995
> 
>
> Key: TS-1577
> URL: https://issues.apache.org/jira/browse/TS-1577
> Project: Traffic Server
>  Issue Type: Bug
>  Components: HTTP
>Affects Versions: 3.3.0
> Environment: git master version
>Reporter: Zhao Yongming
>Assignee: weijin
>Priority: Critical
> Fix For: 3.3.5
>
> Attachments: ts-1574.diff
>
>
> This may or may not relate to TS-1574, I'd like track this issue another 
> thread here.
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 8080:fd=12'.
> Program terminated with signal 6, Aborted.
> #0  0x003e86c32885 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install 
> expat-2.0.1-11.el6_2.x86_64 glibc-2.12-1.47.el6_2.9.x86_64 
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6_2.1.x86_64 
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 
> openssl-1.0.0-20.el6_2.4.x86_64 pcre-7.8-3.1.el6.x86_64 
> tcl-8.5.7-6.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 
> zlib-1.2.3-27.el6.x86_64
> (gdb) bt
> #0  0x003e86c32885 in raise () from /lib64/libc.so.6
> #1  0x003e86c34065 in abort () from /lib64/libc.so.6
> #2  0x0035c8013f19 in ink_die_die_die (retval=30342) at ink_error.cc:43
> #3  0x0035c8014194 in ink_fatal_va(int, const char *, typedef 
> __va_list_tag __va_list_tag *) (return_code=1, 
> message_format=, ap=0x2b0b8fcc3ba0) at 
> ink_error.cc:65
> #4  0x0035c80142c8 in ink_fatal (return_code=30342, message_format=0x7693 
> ) at ink_error.cc:73
> #5  0x0035c8012c3f in _ink_assert (expression=0x0, file=0x6  out of bounds>, line=-1) at ink_assert.cc:38
> #6  0x004d671c in RangeTransform::change_response_header 
> (this=0x2b0be4500bf0) at Transform.cc:995
> #7  0x004da4cd in RangeTransform::handle_event (this=0x2b0be4500bf0, 
> event=, edata=)
> at Transform.cc:791
> #8  0x00654dd4 in handleEvent (this=0x2b0b8e4ad010, e=0x34aff40, 
> calling_code=1) at I_Continuation.h:146
> #9  EThread::process_event (this=0x2b0b8e4ad010, e=0x34aff40, calling_code=1) 
> at UnixEThread.cc:142
> #10 0x0065593b in EThread::execute (this=0x2b0b8e4ad010) at 
> UnixEThread.cc:193
> #11 0x006540d2 in spawn_thread_internal (a=0x2c79a50) at Thread.cc:88
> #12 0x003e878077f1 in start_thread () from /lib64/libpthread.so.0
> #13 0x003e86ce5ccd in clone () from /lib64/libc.so.6
> (gdb) f 6
> #6  0x004d671c in RangeTransform::change_response_header 
> (this=0x2b0be4500bf0) at Transform.cc:995
> 995 
> ink_release_assert(m_transform_resp->field_find(MIME_FIELD_CONTENT_RANGE, 
> MIME_LEN_CONTENT_RANGE) == NULL);
> (gdb) p this
> $1 = (RangeTransform * const) 0x2b0be4500bf0
> (gdb) p *this
> $2 = { = { = { = 
> { = { = { = {
>   _vptr.force_VFPT_to_top = 0x667970}, handler = (int 
> (Continuation::*)(Continuation *, int, 
> void *)) 0x4da200 , mutex = 
> {m_ptr = 0x2b0bf8216110}, link = {> = {next = 0x0}, 
>   prev = 0x0}}, lerrno = 0}, }, mdata = 0x0, 
> m_event_func = 0, m_event_count = 0, m_closed = 0, m_deletable = 0, 
>   m_deleted = 0, m_free_magic = INKCONT_INTERN_MAGIC_ALIVE}, m_read_vio = 
> {_cont = 0x0, nbytes = 0, ndone = 0, op = 0, buffer = {mbuf = 0x0, 
> entry = 0x0}, vc_server = 0x0, mutex = {m_ptr = 0x0}}, m_write_vio = 
> {_cont = 0x2b0f9168f020, nbytes = 20480, ndone = 0, op = 2, buffer = {
> mbuf = 0x2b0c0407fdd0, entry = 0x2b0c0407fe10}, vc_server = 
> 0x2b0be4500bf0, mutex = {m_ptr = 0x2b0bf8216110}}, 
> m_output_vc = 0x2b0be459f048}, m_output_buf = 0x33b7ca0, m_output_reader 
> = 0x33b7cb8, m_transform_resp = 0x2b0f9168dc88, 
>   m_output_vio = 0x2b0be459f0c8, m_unsatisfiable_range = false, 
> m_range_content_length = 0, m_num_chars_for_cl = 1, m_num_range_fields = -1, 
>   m_current_range = 0, 
>   m_content_type = 0x2b1216ea6ac0 "application/octet-stream\r\nContent-Range: 
> bytes 0-20479/119091\r\nContent-Length: 20480\r\nConnection: 
> keep-alive\r\nDate: Sun, 18 Nov 2012 13:30:05 GMT\r\nServer: 
> apache\r\n\r\n\343k\031P", m_content_type_len = 24, m_ranges = 
> 0x2b0be45bda90, 
>   m_output_cl = 20480, m_done = 0}
> (gdb) p m_ranges
> $3 = (RangeRecord *) 0x2b0be45bda90
> (gdb) p *m_ranges
> $4 = {_start = 20480, _end = 40959, _done_byte = -1}
> (gdb) p *m_transform_resp
> $1 = { = { = {m_heap = 0x2b0f914ec010}, m_mime = 
> 0x2b0f914ec0c8}, m_http = 0x2b0f914ec098, 
>   m_u

[jira] [Commented] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

[Leif Hedstrom (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698154#comment-13698154
 ] 

Leif Hedstrom commented on TS-1956:
---

James, I think I only changed the autoconf stuff around this, not the actual 
code. I tried to reproduce this, killing the traffic_server under pretty heavy 
load (150,000 requests / second), and I could not get it to behave like this.

Tommy: How are you "restarting" ATS? killing the traffic_server process? or 
trafficserver restart ? I tried both, and neither triggers this problem for me.

> Under Heavy Load TS 3.3.4-dev can't (re)start
> -
>
> Key: TS-1956
> URL: https://issues.apache.org/jira/browse/TS-1956
> Project: Traffic Server
>  Issue Type: Bug
>Reporter: Tommy Lee
>Priority: Blocker
> Fix For: 3.3.5
>
> Attachments: backtrace.log
>
>
> Hi,
>  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
> 3.3.2-dev without problems.
>  But today, we tried to upgrade to version 3.3.4-dev without lucky.
>  We've noticed that, if TS restarts, it enters in this Segfault Loop.
>  Below are traffic.out logs with debug .*
>  I'll try to debug with GDB too, but I cannot stop this server for too long, 
> because of our operations.
>   Thanks in advance.
> 
> {code}
>  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fd9e0 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fdd00 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.81.5:46089 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.101.4:41361 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.156.38:59164 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
> 10.202.81.5:46089 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.35.9:51533 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.201.20:10964 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
> 10.200.156.38:59164 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.148.2:44203 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fe020 on port 3128 as inbound transparent
> [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
> 10.202.101.4:41361 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
> 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
> Segmentation fault
> /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> NOTE: Traffic Server received Sig 11: Segmentation fault
> [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1956:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

> Under Heavy Load TS 3.3.4-dev can't (re)start
> -
>
> Key: TS-1956
> URL: https://issues.apache.org/jira/browse/TS-1956
> Project: Traffic Server
>  Issue Type: Bug
>Reporter: Tommy Lee
>Priority: Blocker
> Fix For: 3.3.6
>
> Attachments: backtrace.log
>
>
> Hi,
>  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
> 3.3.2-dev without problems.
>  But today, we tried to upgrade to version 3.3.4-dev without lucky.
>  We've noticed that, if TS restarts, it enters in this Segfault Loop.
>  Below are traffic.out logs with debug .*
>  I'll try to debug with GDB too, but I cannot stop this server for too long, 
> because of our operations.
>   Thanks in advance.
> 
> {code}
>  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fd9e0 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fdd00 on port 3128 as inbound transparent
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.81.5:46089 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.101.4:41361 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.156.38:59164 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
> 10.202.81.5:46089 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.35.9:51533 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.201.20:10964 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
> 10.200.156.38:59164 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
> NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
> sockopt 0x0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.202.148.2:44203 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
> accept server 0x20fe020 on port 3128 as inbound transparent
> [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
> 10.202.101.4:41361 transport type = 0
> [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
> 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
> Segmentation fault
> /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
> port inbound transparency enabled.
> [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
> Created accept thread #1 for port 3128
> NOTE: Traffic Server received Sig 11: Segmentation fault
> [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
> [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
> 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
>  - STACK TRACE: 
> [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
> Connection accepted [Server]. 10.200.131.24:65434 -> *Not IP address [0]*:0
> [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_se

[jira] [Commented] (TS-1994) Default RAM Cache size is very small

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/TS-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698141#comment-13698141
 ] 

ASF subversion and git services commented on TS-1994:
-

Commit d05f47ea40d563a89c0d3d2120aacfb831b7d928 in branch refs/heads/master 
from [~zwoop]
[ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=d05f47e ]

TS-1994 Increase default RAM cache size by a magnitude


> Default RAM Cache size is very small
> 
>
> Key: TS-1994
> URL: https://issues.apache.org/jira/browse/TS-1994
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Cache
>Reporter: Igor Galić
>Assignee: Leif Hedstrom
> Fix For: 3.3.5
>
>
> The default size for the RAM cache is 1KiB for 1MiB of disk cache.
> For a 32 GiB disk, that's a mere 32 MiB.
> I think it's safe to say that these days we can easily increase that ratio 5 
> or even ten fold.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (TS-1994) Default RAM Cache size is very small

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom resolved TS-1994.
---

Resolution: Fixed

> Default RAM Cache size is very small
> 
>
> Key: TS-1994
> URL: https://issues.apache.org/jira/browse/TS-1994
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Cache
>Reporter: Igor Galić
>Assignee: Leif Hedstrom
> Fix For: 3.3.5
>
>
> The default size for the RAM cache is 1KiB for 1MiB of disk cache.
> For a 32 GiB disk, that's a mere 32 MiB.
> I think it's safe to say that these days we can easily increase that ratio 5 
> or even ten fold.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (TS-1994) Default RAM Cache size is very small

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom reassigned TS-1994:
-

Assignee: Leif Hedstrom

> Default RAM Cache size is very small
> 
>
> Key: TS-1994
> URL: https://issues.apache.org/jira/browse/TS-1994
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Cache
>Reporter: Igor Galić
>Assignee: Leif Hedstrom
> Fix For: 3.3.5
>
>
> The default size for the RAM cache is 1KiB for 1MiB of disk cache.
> For a 32 GiB disk, that's a mere 32 MiB.
> I think it's safe to say that these days we can easily increase that ratio 5 
> or even ten fold.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1994) Default RAM Cache size is very small

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1994:
--

Fix Version/s: 3.3.5

> Default RAM Cache size is very small
> 
>
> Key: TS-1994
> URL: https://issues.apache.org/jira/browse/TS-1994
> Project: Traffic Server
>  Issue Type: Improvement
>  Components: Cache
>Reporter: Igor Galić
> Fix For: 3.3.5
>
>
> The default size for the RAM cache is 1KiB for 1MiB of disk cache.
> For a 32 GiB disk, that's a mere 32 MiB.
> I think it's safe to say that these days we can easily increase that ratio 5 
> or even ten fold.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1994) Default RAM Cache size is very small

[JIRA]

Igor Galić created TS-1994:
--

 Summary: Default RAM Cache size is very small
 Key: TS-1994
 URL: https://issues.apache.org/jira/browse/TS-1994
 Project: Traffic Server
  Issue Type: Improvement
  Components: Cache
Reporter: Igor Galić


The default size for the RAM cache is 1KiB for 1MiB of disk cache.
For a 32 GiB disk, that's a mere 32 MiB.

I think it's safe to say that these days we can easily increase that ratio 5 or 
even ten fold.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1993) ATS looking for chain certificate in the wrong place

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1993:
--

Fix Version/s: 3.3.6

> ATS looking for chain certificate in the wrong place
> 
>
> Key: TS-1993
> URL: https://issues.apache.org/jira/browse/TS-1993
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Configuration, SSL
>Reporter: David Carlin
> Fix For: 3.3.6
>
>
> ATS 3.3.4 is looking for the chain certificate in the wrong location.  Here 
> is my config:
> proxy.config.ssl.server.cert.path = conf/other/ssl
> proxy.config.ssl.server.cert_chain.filename = CA.pem
> ssl_multicert.config = dest_ip=* ssl_cert_name=website.pem
> When I start ATS I see the following message indicating the root directory:
> [TrafficServer] using root directory '/root/path'
> and the following error in /var/log/messages:
> Jul  1 19:32:15 l6 traffic_server[2167]: {0x2b7a4b3e9f60} ERROR: 
> SSL::0:error:02001002:system library:fopen:No such file or 
> directory:bss_file.c:126:fopen('/root/path/conf/trafficserver/conf/other/ssl/CA.pem','r')
> It should be looking in /root/path/conf/other/ssl/CA.pem - this same config 
> worked in ATS 3.2.0
> Instead its injecting "conf/trafficserver" in the middle of the path which 
> happens to be the value of proxy.config.config_dir
> It appears to be loading the website certificate from the right location - 
> /root/path/conf/other/ssl/website.pem - I know this because if I delete the 
> file and restart ATS, I can see the ATS error where its trying to load it 
> from the correct path:
> Jul  2 14:44:33 l6 traffic_server[53961]: {0x2ae47437a540} ERROR: 
> SSL::0:error:02001002:system library:fopen:No such file or 
> directory:bss_file.c:355:fopen('/root/path/conf/other/ssl/website.pem','r')

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1993) ATS looking for chain certificate in the wrong place

[David Carlin (JIRA)]

David Carlin created TS-1993:


 Summary: ATS looking for chain certificate in the wrong place
 Key: TS-1993
 URL: https://issues.apache.org/jira/browse/TS-1993
 Project: Traffic Server
  Issue Type: Bug
  Components: Configuration, SSL
Reporter: David Carlin


ATS 3.3.4 is looking for the chain certificate in the wrong location.  Here is 
my config:

proxy.config.ssl.server.cert.path = conf/other/ssl
proxy.config.ssl.server.cert_chain.filename = CA.pem
ssl_multicert.config = dest_ip=* ssl_cert_name=website.pem

When I start ATS I see the following message indicating the root directory:

[TrafficServer] using root directory '/root/path'

and the following error in /var/log/messages:

Jul  1 19:32:15 l6 traffic_server[2167]: {0x2b7a4b3e9f60} ERROR: 
SSL::0:error:02001002:system library:fopen:No such file or 
directory:bss_file.c:126:fopen('/root/path/conf/trafficserver/conf/other/ssl/CA.pem','r')

It should be looking in /root/path/conf/other/ssl/CA.pem - this same config 
worked in ATS 3.2.0

Instead its injecting "conf/trafficserver" in the middle of the path which 
happens to be the value of proxy.config.config_dir

It appears to be loading the website certificate from the right location - 
/root/path/conf/other/ssl/website.pem - I know this because if I delete the 
file and restart ATS, I can see the ATS error where its trying to load it from 
the correct path:

Jul  2 14:44:33 l6 traffic_server[53961]: {0x2ae47437a540} ERROR: 
SSL::0:error:02001002:system library:fopen:No such file or 
directory:bss_file.c:355:fopen('/root/path/conf/other/ssl/website.pem','r')





--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1992) examine mgmt/RecordsConfig.cc, add validation where it makes sense

[JIRA]

 [ 
https://issues.apache.org/jira/browse/TS-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Galić updated TS-1992:
---

Summary: examine mgmt/RecordsConfig.cc, add validation where it makes sense 
 (was: examine ./mgmt/RecordsConfig.cc and validation where it makes sense)

> examine mgmt/RecordsConfig.cc, add validation where it makes sense
> --
>
> Key: TS-1992
> URL: https://issues.apache.org/jira/browse/TS-1992
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Igor Galić
>
> example:
> {code}
>   {RECT_CONFIG, "proxy.config.http.cache.required_headers", RECD_INT, "2", 
> RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
> {code}
> We should change this to
> {code}
>   {RECT_CONFIG, "proxy.config.http.cache.required_headers", RECD_INT, "2", 
> RECU_DYNAMIC, RR_NULL, RECC_NULL, "[0-2]", RECA_NULL}
> {code}
> which are the valid values for this config.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (TS-1992) examine ./mgmt/RecordsConfig.cc and validation where it makes sense

[JIRA]

Igor Galić created TS-1992:
--

 Summary: examine ./mgmt/RecordsConfig.cc and validation where it 
makes sense
 Key: TS-1992
 URL: https://issues.apache.org/jira/browse/TS-1992
 Project: Traffic Server
  Issue Type: Bug
  Components: Configuration
Reporter: Igor Galić


example:
{code}
  {RECT_CONFIG, "proxy.config.http.cache.required_headers", RECD_INT, "2", 
RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
{code}

We should change this to

{code}
  {RECT_CONFIG, "proxy.config.http.cache.required_headers", RECD_INT, "2", 
RECU_DYNAMIC, RR_NULL, RECC_NULL, "[0-2]", RECA_NULL}
{code}

which are the valid values for this config.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1686) Crashed in LogUtils::remove_content_type_attributes

[Leif Hedstrom (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1686:
--

Fix Version/s: (was: 3.3.6)
   3.5.0

> Crashed in LogUtils::remove_content_type_attributes
> ---
>
> Key: TS-1686
> URL: https://issues.apache.org/jira/browse/TS-1686
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Clustering, Logging
>Reporter: Yunkai Zhang
>Assignee: Uri Shachar
> Fix For: 3.5.0
>
>
> 1) enable full cluster mode
> 2) two nodes in the cluster
> 3) use jtest tool to do pressure testing
> {code}
> [E. Mgmt] log ==> [TrafficManager] using root directory '/usr'
> Layout configuration
>   --prefix = '/usr'
>  --exec_prefix = '/usr'
>   --bindir = '/usr/bin'
>  --sbindir = '/usr/sbin'
>   --sysconfdir = '/etc/trafficserver'
>  --datadir = '/usr/share/trafficserver'
>   --includedir = '/usr/include/trafficserver'
>   --libdir = '/usr/lib/trafficserver'
>   --libexecdir = '/usr/lib/trafficserver/plugins'
>--localstatedir = '/var/trafficserver'
>   --runtimedir = '/var/run/trafficserver'
>   --logdir = '/var/log/trafficserver'
>   --mandir = '/usr/share/man'
>  --infodir = '/usr/share/info'
> --cachedir = '/var/cache/trafficserver'
> [TrafficServer] using root directory '/usr'
> NOTE: Traffic Server received Sig 11: Segmentation fault
> /usr/bin/traffic_server - STACK TRACE:
> /lib64/libpthread.so.0[0x347380f4a0]
> /lib64/libc.so.6(memchr+0x1b)[0x3473481bbb]
> /usr/bin/traffic_server(_ZN8LogUtils30remove_content_type_attributesEPcPi+0x3d)[0x5fa51f]
> /usr/bin/traffic_server(_ZN13LogAccessHttp4initEv+0x1dc)[0x5d35a2]
> /usr/bin/traffic_server(_ZN3Log6accessEP9LogAccess+0x229)[0x5dc653]
> /usr/bin/traffic_server(_ZN6HttpSM12update_statsEv+0x204)[0x57a3b8]
> /usr/bin/traffic_server(_ZN6HttpSM9kill_thisEv+0x336)[0x57a06c]
> /usr/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x35e)[0x56c6e0]
> /usr/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4e36f8]
> /usr/bin/traffic_server(_ZN10HttpTunnel12main_handlerEiPv+0x15d)[0x5b4563]
> /usr/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4e36f8]
> /usr/bin/traffic_server[0x6c3275]
> /usr/bin/traffic_server[0x6c33a5]
> /usr/bin/traffic_server(_Z15write_to_net_ioP10NetHandlerP18UnixNetVConnectionP7EThread+0x72d)[0x6c4383]
> /usr/bin/traffic_server(_Z12write_to_netP10NetHandlerP18UnixNetVConnectionP14PollDescriptorP7EThread+0x84)[0x6c3c4f]
> /usr/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x716)[0x6bfff4]
> /usr/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x6c)[0x4e36f8]
> /usr/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x6e68f4]
> /usr/bin/traffic_server(_ZN7EThread7executeEv+0x472)[0x6e6eec]
> /usr/bin/traffic_server[0x6e5b00]
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (TS-1990) core at CacheContinuation::handleDisposeEvent()

[Zhao Yongming (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhao Yongming updated TS-1990:
--

Fix Version/s: 3.3.5
   Labels: crash  (was: )
Affects Version/s: 3.3.4

> core at CacheContinuation::handleDisposeEvent()
> ---
>
> Key: TS-1990
> URL: https://issues.apache.org/jira/browse/TS-1990
> Project: Traffic Server
>  Issue Type: Bug
>  Components: Cache, Clustering
>Affects Versions: 3.3.4
>Reporter: Yunkai Zhang
>  Labels: crash
> Fix For: 3.3.5
>
> Attachments: 
> 0001-TS-1990-Fix-core-at-CacheContinuation-handleDisposeE.patch
>
>
> {code}
> Core was generated by `/usr/bin/traffic_server -M --httpport 
> 8080:fd=10,80:fd=11'.
> Program terminated with signal 11, Segmentation fault.
> #0  CacheContinuation::handleDisposeEvent (event=, 
> cc=0x2b2a34575870) at ClusterCache.cc:1969
> 1969  cc->tunnel->vioTarget->reenable();
> Missing separate debuginfos, use: debuginfo-install 
> expat-2.0.1-9.1.el6.x86_64 glibc-2.12-1.47.el6.x86_64 
> keyutils-libs-1.4-3.el6.x86_64 krb5-libs-1.9-22.el6.x86_64 
> libcom_err-1.41.12-11.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 
> libselinux-2.0.94-5.2.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 
> ncurses-libs-5.7-3.20090208.el6.x86_64 openssl-1.0.0-20.el6.x86_64 
> pcre-7.8-3.1.el6.x86_64 readline-6.0-3.el6.x86_64 tcl-8.5.7-6.el6.x86_64 
> xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-27.el6.x86_64
> (gdb) l
> 1964  // Start tunnel by reenabling source and target VCs.
> 1965  cc->tunnel->in_handleDisposeEvent = true;
> 1966
> 1967  cc->tunnel->vioSource->nbytes = 
> getObjectSize(cc->tunnel->vioSource->vc_server, cc->request_opcode, 0);
> 1968  cc->tunnel->vioSource->reenable_re();
> 1969  cc->tunnel->vioTarget->reenable();
> 1970
> 1971  // Tell tunnel event we are gone
> 1972  cc->tunnel_cont->action.continuation = 0;
> 1973
> (gdb) p cc->tunnel
> $1 = (OneWayTunnel *) 0x0
> (gdb) bt
> #0  CacheContinuation::handleDisposeEvent (event=, 
> cc=0x2b2a34575870) at ClusterCache.cc:1969
> #1  0x00607421 in CacheContinuation::disposeOfDataBuffer (d= optimized out>) at ClusterCache.cc:1946
> #2  0x00619d9b in ClusterControl::free_data (this=0x2b2b201e8440) at 
> ClusterHandlerBase.cc:138
> #3  0x006130d8 in ClusterHandler::update_channels_written 
> (this=0x2b2b101e22d0, bump_unhandled_channels=)
> at ClusterHandler.cc:1570
> #4  0x006182ea in ClusterHandler::process_write (this=0x2b2b101e22d0, 
> now=1372650704886648000, only_write_control_msgs=false)
> at ClusterHandler.cc:3080
> #5  0x00618874 in ClusterHandler::mainClusterEvent 
> (this=0x2b2b101e22d0, event=, e=)
> at ClusterHandler.cc:2595
> #6  0x0061ba5c in handleEvent (this=0x2b2b101e2f90) at 
> ../../iocore/eventsystem/I_Continuation.h:146
> #7  ClusterState::IOComplete (this=0x2b2b101e2f90) at 
> ClusterHandlerBase.cc:585
> #8  0x0061bcb4 in ClusterState::doIO_write_event 
> (this=0x2b2b101e2f90, event=103, d=0x2b2a38011dd0) at 
> ClusterHandlerBase.cc:544
> #9  0x00687f11 in handleEvent (event=, 
> nh=0x2b2a2ac9e268, vc=0x2b2a38011c60) at 
> ../../iocore/eventsystem/I_Continuation.h:146
> #10 write_signal_and_update (event=, nh=0x2b2a2ac9e268, 
> vc=0x2b2a38011c60) at UnixNetVConnection.cc:153
> #11 write_signal_done (event=, nh=0x2b2a2ac9e268, 
> vc=0x2b2a38011c60) at UnixNetVConnection.cc:180
> #12 0x0068b4e7 in write_to_net_io (nh=0x2b2a2ac9e268, 
> vc=0x2b2a38011c60, thread=) at UnixNetVConnection.cc:479
> #13 0x006826d6 in NetHandler::mainNetEvent (this=0x2b2a2ac9e268, 
> event=, e=) at UnixNet.cc:394
> #14 0x006ab654 in handleEvent (this=0x2b2a2ac9b010, e=0x2b2a14164e20, 
> calling_code=5) at I_Continuation.h:146
> #15 EThread::process_event (this=0x2b2a2ac9b010, e=0x2b2a14164e20, 
> calling_code=5) at UnixEThread.cc:142
> #16 0x006abff3 in EThread::execute (this=0x2b2a2ac9b010) at 
> UnixEThread.cc:266
> #17 0x006aa5f2 in spawn_thread_internal (a=0x2b2a141cca50) at 
> Thread.cc:88
> #18 0x003984e077f1 in start_thread () from /lib64/libpthread.so.0
> #19 0x003984ae570d in clone () from /lib64/libc.so.6
> {code}
> And I have found the root cause that cc->tunnel->vioSource->reenable_re() may 
> free the tunnel in some case, the following satck trace is debuging info I 
> added inside tunnleClosedEvent():
> {code}
> [Jul  1 11:51:44.886] Server {0x2b2a2b9a7700} NOTE: ---start---
> /usr/bin/traffic_server - STACK TRACE:
> /usr/bin/traffic_server(_ZN17CacheContinuation17tunnelClosedEventEiPv+0xf0)[0x606ce0]
> /usr/bin/traffic_server(_ZN12OneWayTunnel10startEventEiPv+0x4a1)[0x5e3f51]
> /usr/bin/traffic_server(_ZN7CacheVC8calluserEi+0x2b)[0x65db7b]
> /usr/bin