[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107020#comment-14107020 ] Sudheer Vinukonda commented on TS-2983: --- [~amc] asked me to test the following fix and it seems to solve the issue. With the probe enabled, and with the fix from Alan, the logs did not show any corrupted requests (I also confirmed that, without Alan's fix, the logs show tons of corrupted requests). {code} diff --git a/proxy/ProtocolProbeSessionAccept.cc b/proxy/ProtocolProbeSessionAccept.cc index 46adb48..97a397a 100644 --- a/proxy/ProtocolProbeSessionAccept.cc +++ b/proxy/ProtocolProbeSessionAccept.cc @@ -30,19 +30,20 @@ struct ProtocolProbeTrampoline : public Continuation, public ProtocolProbeSessio { static const size_t minimum_read_size = 1; static const unsigned buffer_size_index = CLIENT_CONNECTION_FIRST_READ_BUFFER_SIZE_INDEX; + IOBufferReader * reader; explicit ProtocolProbeTrampoline(const ProtocolProbeSessionAccept * probe, ProxyMutex * mutex) : Continuation(mutex), probeParent(probe) { this-iobuf = new_MIOBuffer(buffer_size_index); +reader = iobuf-alloc_reader(); SET_HANDLER(ProtocolProbeTrampoline::ioCompletionEvent); } int ioCompletionEvent(int event, void * edata) { VIO * vio; -IOBufferReader * reader; NetVConnection * netvc; ProtoGroupKey key = N_PROTO_GROUPS; // use this as an invalid value. @@ -64,7 +65,6 @@ struct ProtocolProbeTrampoline : public Continuation, public ProtocolProbeSessio return EVENT_ERROR; } -reader = iobuf-alloc_reader(); ink_assert(netvc != NULL); if (!reader-is_read_avail_more_than(minimum_read_size - 1)) { {code} request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Assignee: Alan M. Carroll Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=x;%20ypcdb=xx - NONE/- - {code} After a lot of debugging, figured that the request was getting corrupted even before remap and in fact, is being parsed incorrectly at the read request state. Further analysis lead me to the commit TS-2197 (commit 30fcc2b2e698831d1a9e4db1474d8cfc202818a3 in Oct'13), which has altered the way the request is read slightly. Reverting the commit seems to have fixed the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107112#comment-14107112 ] ASF subversion and git services commented on TS-2983: - Commit 55961c89fd99546f41d66e2f83a6c47a978f7603 in trafficserver's branch refs/heads/master from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=55961c8 ] TS-2983: Fix protocol probe to not drop data. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Assignee: Alan M. Carroll Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=x;%20ypcdb=xx - NONE/- - {code} After a lot of debugging, figured that the request was getting corrupted even before remap and in fact, is being parsed incorrectly at the read request state. Further analysis lead me to the commit TS-2197 (commit 30fcc2b2e698831d1a9e4db1474d8cfc202818a3 in Oct'13), which has altered the way the request is read slightly. Reverting the commit seems to have fixed the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107114#comment-14107114 ] Alan M. Carroll commented on TS-2983: - The issue was the the IOBufferReader was initialized after data had arrived in the IOBuffer. That can lead to dropped data depending on how much the client sends initially. Moving the read init to the constructor instead of the event handler fixes the problem. See the comments on MIOBuffer::alloc_reader for more detail. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Assignee: Alan M. Carroll Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=x;%20ypcdb=xx - NONE/- - {code} After a lot of debugging, figured that the request was getting corrupted even before remap and in fact, is being parsed incorrectly at the read request state. Further analysis lead me to the commit TS-2197 (commit 30fcc2b2e698831d1a9e4db1474d8cfc202818a3 in Oct'13), which has altered the way the request is read slightly. Reverting the commit seems to have fixed the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107123#comment-14107123 ] ASF subversion and git services commented on TS-2983: - Commit 45f23caf69043174ef4e60f35b6661c08d726d4e in trafficserver's branch refs/heads/5.1.x from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=45f23ca ] TS-2983: Fix protocol probe to not drop data. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Assignee: Alan M. Carroll Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=x;%20ypcdb=xx - NONE/- - {code} After a lot of debugging, figured that the request was getting corrupted even before remap and in fact, is being parsed incorrectly at the read request state. Further analysis lead me to the commit TS-2197 (commit 30fcc2b2e698831d1a9e4db1474d8cfc202818a3 in Oct'13), which has altered the way the request is read slightly. Reverting the commit seems to have fixed the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093220#comment-14093220 ] Sudheer Vinukonda commented on TS-2983: --- Updated the logs to remove private data. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=x;%20ypcdb=xx - NONE/- - {code} After a lot of debugging, figured that the request was getting corrupted even before remap and in fact, is being parsed incorrectly at the read request state. Further analysis lead me to the commit TS-2197 (commit 30fcc2b2e698831d1a9e4db1474d8cfc202818a3 in Oct'13), which has altered the way the request is read slightly. Reverting the commit seems to have fixed the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090878#comment-14090878 ] Sudheer Vinukonda commented on TS-2983: --- Turns out that enabling probe for http also may be corrupting the requests. Apologies for the misinformation, I may not have run it long enough earlier. The https corruption was happening so often that, it was clear within a minute, but, the http issue seems to be happening with really low frequency (once/hour or so) - this may also be due to the generally lower percentage of http traffic on our host. Anyway, for now, turning off both http and https probe and we are back to trying to find the root cause. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091121#comment-14091121 ] James Peach commented on TS-2983: - FWIW, I'm strongly -1 on disabling probing on SSL connections unless we have a good explanation for the root cause of the problem. Once we have that, I'm happy to discuss it further. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090094#comment-14090094 ] Sudheer Vinukonda commented on TS-2983: --- For non-TLS and non-SSL pure http, enabling the probe seems fine even after TS-2751. The problem only happens when the probe is enabled for SSL connection. It's not very clear from the description in TS-2751 that protocol probe is also being added for SSL. I agree with [~jpeach]'s view to support SSL probe for completeness. Will try and debug the issue. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090258#comment-14090258 ] Leif Hedstrom commented on TS-2983: --- I think I'm with Bryan on this, although I have not looked at the code much. I don't think completeness is a reasonable argument here, if the patch Sudheer showed works, it seemed totally fine to me, it seemed clean to pass probe as a argument where needed. -- Leif request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090288#comment-14090288 ] Sudheer Vinukonda commented on TS-2983: --- Thanks, [~zwoop]..If everyone agrees to not probe the SSL connection, I would not mind that very much :=) [~jpe...@apache.org] - Just to understand the mystery, I was looking at the changes in TS-2751 and was wondering if the newly added ProtocolProbeTrampoline does everything, that the SSLNextProtocolTrampoline did for SSL connection prior to TS-2751. In particular, I was curious about the below comments in SSLNextProtocolAccept.cc indicating that the continuation that's handling the read event must have a mutex etc. Perhaps, I am missing something, but, it seemed to me that the new ProtocolProbeTrampoline did not seem to have such a mechanism? {code} // SSLNextProtocolTrampoline is the receiver of the I/O event generated when we perform a 0-length read on the new SSL // connection. The 0-length read forces the SSL handshake, which allows us to bind an endpoint that is selected by the // NPN extension. The Continuation that receives the read event *must* have a mutex, but we don't want to take a global // lock across the handshake, so we make a trampoline to bounce the event from the SSL acceptor to the ultimate session // acceptor. {code} request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088542#comment-14088542 ] Sudheer Vinukonda commented on TS-2983: --- [~jpeach] - I dug a bit into the old logs to find if there was any non-https request getting corrupted. I did not find a single such request. iow, all the requests that are corrupted seem to be https. Looking into the changes made in TS-2751 again, I am wondering if we really need to support protocol probe for https at all - the implementation prior to TS-2751 also seem to only support probe for http. I also verified that there's no issue with TS-2751 commit when I enable the probe for http and just revert the probe for https. Is there any specific reason that you would like the probe enabled for https? Isn't NPN support mandatory for SPDY over https? {code} diff --git a/proxy/http/HttpProxyServerMain.cc b/proxy/http/HttpProxyServerMain.cc index 9eb9291..6cdbce7 100644 --- a/proxy/http/HttpProxyServerMain.cc +++ b/proxy/http/HttpProxyServerMain.cc @@ -171,10 +171,9 @@ MakeHttpProxyAcceptor(HttpProxyAcceptor acceptor, HttpProxyPort port, unsigned // XXX the protocol probe should be a configuration option. ProtocolProbeSessionAccept *probe = new ProtocolProbeSessionAccept(); - HttpSessionAccept *http = 0; // don't allocate this unless it will be used. + HttpSessionAccept *http = new HttpSessionAccept(accept_opt); if (port.m_session_protocol_preference.intersects(HTTP_PROTOCOL_SET)) { -http = new HttpSessionAccept(accept_opt); probe-registerEndpoint(ProtocolProbeSessionAccept::PROTO_HTTP, http); } @@ -185,7 +184,7 @@ MakeHttpProxyAcceptor(HttpProxyAcceptor acceptor, HttpProxyPort port, unsigned #endif if (port.isSSL()) { -SSLNextProtocolAccept *ssl = new SSLNextProtocolAccept(probe); +SSLNextProtocolAccept *ssl = new SSLNextProtocolAccept(http); // ALPN selects the first server-offered protocol, // so make sure that we offer the newest protocol first. {code} request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088759#comment-14088759 ] Bryan Call commented on TS-2983: I don't think it is useful to probe for spdy or http/2 for TLS and it should depend on NPN/ALPN for the correct information. I do think we should fix the underlying issue for non-TLS. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088778#comment-14088778 ] James Peach commented on TS-2983: - I think it's a matter of symmetry. If you can probe for a protocol, then the underlying transport should not matter. I agree that for the specific cases of HTTP/2 and SPDY there's no point in doing it, but we should avoid the special case just on principle. If someone were to argue that we should not do protocol probing at all, I would not mind. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 Attachments: TS-2983.diff We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084710#comment-14084710 ] Sudheer Vinukonda commented on TS-2983: --- That didn't seem to help - the requests that are getting corrupted appear to be https (note the https scheme being printed in the squid log below). {code} -bash-4.1$ tail -f squid.log | grep CONNECT_FAIL | grep -v social | grep -v yahooapi 1407162766.057 0 98.155.36.127 ERR_CONNECT_FAIL 404 34 %3D345263455529%3A_uid%3D5018557808362%3Av%3D10.6.1%3Ats%3D1332174666282%3Ahc%3D45; https://YLS=v=1p=0n=0/ - NONE/- text/html https://YLS=v=1p=0n=0/ f1 f2 f3 f4 1407162775.460 23 69.9.231.80 ERR_CONNECT_FAIL 404 34 X296YWphNlZsUTA4Zy0BYWMBQU9NU2taZ3cBb2sBWlcwLQF0aXABRkJBQUFBAXNjAXdsAWZzAWpwZ3lFZHRUdmF5aQF6egE3dDQzVEJmV0E-; https://SSL=v=1s=mPCoPrtECb8X8YwI2FBKJylass7H9bW4UvloZA86q4.F0PGRjBks09cZms6f4GTCqQz7q7Qryq.ob24EFE_nzw--kv=0/ - NONE/- text/html https://SSL=v=1s=mPCoPrtECb8X8YwI2FBKJylass7H9bW4UvloZA86q4.F0PGRjBks09cZms6f4GTCqQz7q7Qryq.ob24EFE_nzw--kv=0/ f1 f2 f3 f4 1407162777.449 38 99.27.104.22 ERR_CONNECT_FAIL 404 34 azUBYQFRQUUBZwFFMjVRUjRQQlRXSU40QlFNUDNCQ09ZT0lVUQFzY2lkAUNMVDN6VGtVZ215VGkwZTVkQ3JyRzdwLm1UYy0BYWMBQUpQVnZuSmEBdGlwAWpCQUFBQQFzYwF3bAFmcwFkNWtpdG1CU3FIdHgBenoBNWw1M1RCQTdFaf=QUFBQ0FDQURCOUFJQUJBQ0FEQUVIME1nTWlNbiZ0cz0xNDA3MTYyNzQ1JnBzPTFxamVpb2xucGZWSnJiSjZDYkoyc0EtLQ--; https://SSL=v=1s=beNDORhMI9WHbcz2VlNhaW5iH8JYh.BZkgg2Pj2GelIozegoCDnLb5y0bqub72_21.m6e0Mqmj8I3R3iNtmv7Q--kv=0;%20s_sess=%2520SC_LINKS%253D%253B/ - NONE/- text/html https://SSL=v=1s=beNDORhMI9WHbcz2VlNhaW5iH8JYh.BZkgg2Pj2GelIozegoCDnLb5y0bqub72_21.m6e0Mqmj8I3R3iNtmv7Q--kv=0;%20s_sess=%2520SC_LINKS%253D%253B/ f1 f2 f3 f4 {code} I plan to review the TS-2751 commit a little more later today, but, in the mean time, we probably need to also consider a way forward - reverting this commit entirely is definitely non-trivial (too many conflicts to resolve). request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084760#comment-14084760 ] James Peach commented on TS-2983: - Oh, https-only i a good clue! request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad;
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084762#comment-14084762 ] Sudheer Vinukonda commented on TS-2983: --- Umm..I am not entirely sure if it's https-only, but, a vast majority of the corrupted logs do indicate https..i will try to dig around the logs in a bit to see if I can find a corrupted http request. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084953#comment-14084953 ] Sudheer Vinukonda commented on TS-2983: --- [~jpeach] - it looks like the change you suggested above only disables protocol probe for http. [~bcall] suggested the below additional change to disable the probe for https as well. Tried this additional suggestion and that seems to have done the trick. I've also done some basic validation to ensure that spdy is not broken when enabled via NPN (normal way of configuring spdy via server_port setting). {code} diff --git a/proxy/http/HttpProxyServerMain.cc b/proxy/http/HttpProxyServerMain.cc index c9fdbde..46ee998 100644 --- a/proxy/http/HttpProxyServerMain.cc +++ b/proxy/http/HttpProxyServerMain.cc @@ -180,7 +180,7 @@ MakeHttpProxyAcceptor(HttpProxyAcceptor acceptor, HttpProxyPort port, unsigned probe-registerEndpoint(TS_PROTO_HTTP, http); if (port.isSSL()) { -SSLNextProtocolAccept *ssl = NEW(new SSLNextProtocolAccept(probe)); +SSLNextProtocolAccept *ssl = NEW(new SSLNextProtocolAccept(http)); // ALPN selects the first server-offered protocol, // so make sure that we offer the newest protocol first. @@ -204,7 +204,7 @@ MakeHttpProxyAcceptor(HttpProxyAcceptor acceptor, HttpProxyPort port, unsigned acceptor._accept = ssl; } else { -acceptor._accept = probe; +acceptor._accept = http; } } {code} request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084039#comment-14084039 ] James Peach commented on TS-2983: - I'm surprised you were able to revert TS-2751 without breaking a ton of subsequent commits. I wonder if the protocol probe is the issue here? Is it possible to test with just the protocol probing disabled? request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084098#comment-14084098 ] Sudheer Vinukonda commented on TS-2983: --- You are right - I couldn't (and didn't) actually revert TS-2751 alone, since there are quite a few comments since. Given the nature of the problem (that the incoming client request is corrupted before any processing begins and that the corrupted request gets parsed), I just used my judgement going over the commits and the type of changes. Basically, what I did was to confirm that the issue doesn't occur when I go back to pre-TS-2751 (commit 1106086111dc0faf0568bd7bf78b3ee6f7bb344a) and when I add TS-2751 on top of that (727811ef0870701ade427b0ed374ac42daec8c54), the issue comes back. Since, it's a large commit, I didn't actually review the entire commit yet, so, do you have any specific part of the code that you would like me to comment out or modify to disable protocol probing? request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084113#comment-14084113 ] James Peach commented on TS-2983: - The part that is most suspect in that commit is where we sniff the incoming byte stream to see whether it is SDPY or HTTP. I think this change is the simplest way to disable protocol probing: {code} diff --git a/proxy/http/HttpProxyServerMain.cc b/proxy/http/HttpProxyServerMain.cc index 5c966b8..cbe9107 100644 --- a/proxy/http/HttpProxyServerMain.cc +++ b/proxy/http/HttpProxyServerMain.cc @@ -229,7 +229,8 @@ MakeHttpProxyAcceptor(HttpProxyAcceptor acceptor, HttpProxyPort port, unsigned acceptor._accept = ssl; } else { -acceptor._accept = probe; +// acceptor._accept = probe; +acceptor._accept = http; } } {code} request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083688#comment-14083688 ] Leif Hedstrom commented on TS-2983: --- This definitely sounds like a 5.1 candidate ? request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad;
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083789#comment-14083789 ] Sudheer Vinukonda commented on TS-2983: --- It turns out that TS-2197 is not really the culprit - the errors appeared to have stopped for a few minutes, when I reverted TS-2197, but, appeared again. :-( Investigating further, it looks like, the commit in TS-2751 may be causing this issue. I suspected this to begin with, but, was hoping seriously, that it was not. Given that, it's a relatively large commit, this may need to be reviewed carefully to catch the issue. For the time being, reverting TS-2751 and running the private version for an hour in production, did not bring back the errors so far - will continue to monitor and confirm/update, if there's anything else. And yes, this is definitely a blocker for our 5.0.x (and in turn, SPDY) production roll out - it would be interesting to hear if anyone else, using 5.0.x is also seeing this issue. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897
[jira] [Commented] (TS-2983) request headers, http object corrupted in 5.0.x
[ https://issues.apache.org/jira/browse/TS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083854#comment-14083854 ] Sudheer Vinukonda commented on TS-2983: --- Just to confirm that there are no more errors, since reverting TS-2751..traffic appears stable for the past few hours. request headers, http object corrupted in 5.0.x --- Key: TS-2983 URL: https://issues.apache.org/jira/browse/TS-2983 Project: Traffic Server Issue Type: Bug Components: Core Affects Versions: 5.1.0 Reporter: Sudheer Vinukonda Priority: Blocker Fix For: 5.1.0 We have run into a http header/field corruption issue on our proxy infrastructure production hosts when we enabled 5.0.x. The issue results in host header/method and other field corruption. For example, this is what we see in our squid access logs: {code} 1406999819.698 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999819.698 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 0 69.109.120.92 ERR_CONNECT_FAIL 404 0 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- - https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999834.692 4 69.109.120.92 ERR_CONNECT_FAIL 404 1825 nas=0disclaimer=2; https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ - NONE/- text/html https://AMCV_att1=MCAID%7C29B9EEB305013EE7-410900024C7D;%20ypcdb=27a2458b603f6370d295cf4130825e34/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 0 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad; https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ - NONE/- - https://YLS=v=1p=1n=0;%20ucs=bnas=1bnid=SGZJL05rb2dzRzN3dTd5UDJWdFdqUT09;%20_br_uid_2=uid%3D7216053621378%3A_uid%3D8562842167448%3Av%3D10.6.1%3Ats%3D1324745461005%3Ahc%3D11/ f1 f2 f3 f4 1406999842.475 0 75.15.121.107 ERR_CONNECT_FAIL 404 1897 435OjuZl6UyTXTs38eKhUO5C7Vs4E7FRajuU3BHWqLQxY4GMkRCgkGTvCOJPimsE1aB2W5hj_Qh70_dy7aJoas2DbmaCbIJ6UgkbkUINLhJbjDnfx3pKtVSlFca1VB6zCYSEkCswU_dULV6p2FtUs5aVRCZl2kVCmvy9esXiSqqXN1oYxAPe6l5bHM9yLOVNIRmaFTRcSHLm3e3lIp5Bx3Wismutrnp8nddMtqoz8xhLLUTVrid1YGmu0kPQxEhn8kJtm_8E2Kw48yhiy.slyxhwZPtI3rHRz42B4MrcvGBnZiglC0f0FCHYBcHVZ4B0eqCXX3hNNH_.xj9xqHfRJzRaL0c6DwuiVFiyd4UcID3uhp6e1Y5EAh0lN4_gJPFCxnyrqnzw1T31w.TuhS9Rz98ZlNHVvsN6kLnKq5JmxocK7X4rmzOKNmmY9e85vEhQS7c4fN69JtzyvdbNUWY_4x6iYU4rvIu30miN.klS8iqD_W8g_6Xl.iclrYt_AKdUJ59qhj9V54JDebWuybPyZHLB.h2ZOKzbIyO9qUiZUY.5eqiHeupe_3ZieCViVpkCVKoiEunj1bPRd9tuxcNel.GmAtDPjRCwUH3WVCz3zq34rY6VBG2EeNQyLNNTT1GEHiXQ7k_ykZeQ6qCZGSZPpqz0S5we2qshU69hR1grm1Pyxli2Pv8uZ3YqL2FUQow0zZdtejnsxosoFqfge_mWnnHq9gmXkfc0RdgUevblIr50wC6stwTwjbW2MqOo2zyrRkp99Uprxah2olhrbw7Zc2yFDuCG5A4.R_3crXB8fp6iwmdlGnDfRa1oaHXn3Kves9YbZLtp1sUmQFmPoKr7Fu0qgxSHPDagEqhQXFnV3YpZ0LHdvir7MWFKg2bzh3OlrDeaGe0KK.xl8Y2OkckdNvQ.ho6F9AElt5GBACSl2kaVKChw8xa5xUOT0x8FYzFfgGk6syEESBPxvloY7OpdDGh3LUovu00N7z2z8lhC.LvYwmEdMeuS.5qq5yAE4jGWSlXXBBgUBXqyTFJ3IZUbm1gPvOCbEVELrbEvc7n0Gw_D_TjHcbNHN7rzXqad;