[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2014-01-19 Thread Leif Hedstrom (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1956:
--

Assignee: Bryan Call

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Assignee: Bryan Call
Priority: Blocker
  Labels: Crash
 Fix For: sometime

 Attachments: backtrace.log


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2014-01-13 Thread Leif Hedstrom (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1956:
--

Fix Version/s: (was: 5.0.0)
   sometime

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Priority: Blocker
  Labels: Crash
 Fix For: sometime

 Attachments: backtrace.log


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 -

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2013-12-20 Thread Leif Hedstrom (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1956:
--

Labels: Crash  (was: )

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Priority: Blocker
  Labels: Crash
 Fix For: 5.0.0

 Attachments: backtrace.log


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP address [0]*:0
 [Jun 14

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2013-07-02 Thread Leif Hedstrom (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leif Hedstrom updated TS-1956:
--

Fix Version/s: (was: 3.3.5)
   3.3.6

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Priority: Blocker
 Fix For: 3.3.6

 Attachments: backtrace.log


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP address [0]*:0
 [Jun 14

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2013-06-14 Thread James Peach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Peach updated TS-1956:


Fix Version/s: 3.3.6

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
 Fix For: 3.3.6


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP address [0]*:0
 [Jun 14 11:54:14.565] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server].

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2013-06-14 Thread James Peach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Peach updated TS-1956:


Priority: Blocker  (was: Major)

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Priority: Blocker
 Fix For: 3.3.6


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP address [0]*:0
 [Jun 14 11:54:14.565] Server {0x2b0850514700} DEBUG: (iocore_net_server)

[jira] [Updated] (TS-1956) Under Heavy Load TS 3.3.4-dev can't (re)start

2013-06-14 Thread Tommy Lee (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommy Lee updated TS-1956:
--

Attachment: backtrace.log

Backtrace from GDB with --enable-debug

 Under Heavy Load TS 3.3.4-dev can't (re)start
 -

 Key: TS-1956
 URL: https://issues.apache.org/jira/browse/TS-1956
 Project: Traffic Server
  Issue Type: Bug
Reporter: Tommy Lee
Priority: Blocker
 Fix For: 3.3.6

 Attachments: backtrace.log


 Hi,
  We run TS in forward mode, under REALLY HEAVY load. Currently we run version 
 3.3.2-dev without problems.
  But today, we tried to upgrade to version 3.3.4-dev without lucky.
  We've noticed that, if TS restarts, it enters in this Segfault Loop.
  Below are traffic.out logs with debug .*
  I'll try to debug with GDB too, but I cannot stop this server for too long, 
 because of our operations.
   Thanks in advance.
 
 {code}
  [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fd9e0 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fdd00 on port 3128 as inbound transparent
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.563] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 [Jun 14 11:54:14.563] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.81.5:46089 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.101.4:41361 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.156.38:59164 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5413700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c00] accepted connection from 
 10.202.81.5:46089 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.35.9:51533 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.201.20:10964 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b06a5615700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015a80] accepted connection from 
 10.200.156.38:59164 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_processor) 
 NetProcessor::main_accept - port 3128,recv_bufsize 0, send_bufsize 262144, 
 sockopt 0x0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.202.148.2:44203 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Marking 
 accept server 0x20fe020 on port 3128 as inbound transparent
 [Jun 14 11:54:14.564] Server {0x2b06a5514700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015d20] accepted connection from 
 10.202.101.4:41361 transport type = 0
 [Jun 14 11:54:14.564] Server {0x2b06a5817700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c015540] accepted connection from 
 10.200.201.20:10964 transport type = 0NOTE: Traffic Server received Sig 11: 
 Segmentation fault
 /usr/local/cache-3.3.4/bin/traffic_server - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (http_tproxy) Listen 
 port inbound transparency enabled.
 [Jun 14 11:54:14.564] Server {0x2b069d226500} DEBUG: (iocore_net_accept) 
 Created accept thread #1 for port 3128
 NOTE: Traffic Server received Sig 11: Segmentation fault
 [Jun 14 11:54:14.564] Server {0x2b06a5716700} DEBUG: (http_seq) 
 [HttpAccept:mainEvent 0x2b085c0157e0] accepted connection from 
 10.200.35.9:51533 transport type = 0/usr/local/cache-3.3.4/bin/traffic_server
  - STACK TRACE: 
 [Jun 14 11:54:14.564] Server {0x2b085c120700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.131.24:65434 - *Not IP address [0]*:0
 [Jun 14 11:54:14.564] Server {0x2b0850514700} DEBUG: (iocore_net_server) 
 Connection accepted [Server]. 10.200.157.26:52514 - *Not IP address