Re: tcpdump and Haproxy SSL Offloading
Hi, Am 04.06.2016 um 02:14 schrieb Igor Cicimov: you can dump the symmetric keys from the browser and import them in wireshark to decrypt PFS protected TLS sessions [1] Yes in case you want to troubleshoot something generic this is a good approach but if you want to troubleshoot sessions not initiated by your self, ie particular client connection, this is practically impossible. Temporarily disabling PFS ciphers is the only way then. Extracting the symmetric key from certain sessions on the haproxy side would be an interesting feature though. Another proxy layer means that you decrypt TLS on the front-end proxy, while you sniff the plaintext traffic between the front-end and the second tier proxy. You can probably do this with a single haproxy instance recirculating the traffic through a unix socket and capture the traffic on it, but it would require some trial and error and definitely some testing. This will probably be faster but can't use tcpdump in that case. Using the loopback interface instead of a unix socket will fix this. Regards, Lukas
手动搜索客户太费精力，试试让电脑自动来搜索，夥攀奔
开发客户需主动ʌ=92;现在正是外贸旺季=5292;你不主动开发,你在=等客户的同时,你的ê=58;户已经被同行抢先=4320;发!支持QQ在线演示2376=552952
RE: tcpdump and Haproxy SSL Offloading
Hi Luca and Igor, I know there is not a simple way. In this network trace I verified an IE11 / Edge bug with preconnect sessions. This is a known problem, also if not so documented. As you can see Windows Client TCP Stack correctly send an ACK to a FIN request from HAProxy, but IE does not instruct TCP Stack to send its own FIN to HAProxy to close the TCP connection gracefully, so IE at later time erroneously try to use the TCP Connection, correctly HAProxy send a RST. In the HTTP buffer of the client there is what HAProxy sent before closing the connection (a 408 Timeout HTTP Status message), so IE erroneously read this message and wrong again, instead of closing HTTP session and retry with another session, send the 408 Timeout to the Browser, so client see 408, thinking the server does not work well… [cid:image001.png@01D1BE4E.ED11D370] [cid:image002.png@01D1BE4E.ED11D370] In this case: - 408 is not sent from backend server, so not all the traffic can be collected on backend server to analyze the issue - HAProxy send 408, but I don’t see HTTP flow, I see only HTTPS (SSL/TLS) Flow as all is encrypted from client to haproxy and haproxy do SSL termination - In this particular case we can reproduce the problem on a test machine, as we know and manage the machine, we can use the wireshark unencypting capabilities, but I think this method is not so robust as involves many changing parts Cipher/haproxy/… I have to put togheter many info (haproxy log, tcpdump trace, ecc ) to have a complete picture (ie: it would be simple if I can see and search 408 http status code in the tcpdump trace instead of know it is here from browser and form haproxy log, but not shown in wireshark as incapsulated in TLS communication. Haproxy just do different not properly application level thing for SSL termination, so I’m uncertain is haproxy or not to manage and provide unencrypted traffic, all other solution as we are seeing are prone to difficulties and instability of the trace process. I like robust solution and if possible solution independent as soon as possible from implicit (instead explicit) support by components, that is: haproxy work using unencrypting with client session keys, but I know on some version this mechanism does not work, so this introduce instability. But I’ll try FPS with client session keys solution also if I think this is complex as we have to capture all keys for all TLS sessions to see the complete traffic we take as many SSL handshake take place in that trace. I think that at the moment the best solution is to temporarily disable PFS ciphers suite, I think this is a simple reconfiguration of haproxy.cfg and haproxy daemon restart, for the test period, not so critical for us. I need for this some hints as I’m not so informed about cipher suites, in particular I didn’t found a clear specification how to identify cipher show / used by haproxy and how identify what support or not PFS. ie: if I check from a SSL tester site I see this protocols and chiper suite for haproxy SSL termination: Protocols TLS 1.2 Yes TLS 1.1 Yes TLS 1.0 Yes SSL 3No SSL 2No Cipher Suites TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 TLS_RSA_WITH_RC4_128_SHA (0x5) INSECURE 128 TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011) ECDH secp256r1 (eq. 3072 bits RSA) FS INSECURE 128 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d) 256 TLS_RSA_WITH_AES_256_CBC_SHA (0x35) 256 TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (0x84) 256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027) ECDH secp256r1 (eq. 3072 bits RSA) FS 128 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013) ECDH secp256r1 (eq. 3072 bits RSA) FS 128 TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012) ECDH secp256r1 (eq. 3072 bits RSA) FS 112 TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c) 128 TLS_RSA_WITH_AES_128_CBC_SHA (0x2f) 128 TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (0x41) 128 TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa)
★ HAProxy, Andrew left a message for you
Title: HAProxy, Andrew left a message for you See this email in Deutsch, Français, Italiano, Español, Português or 37 other languages. Andrew left a message for you You can instantly reply using our message exchange system. Check your message Some other people in the area: Download on the App Store Get it on Google Play Download from Windows Store You have received this email from Badoo Trading Limited (postal address below). If you do not wish to receive further email communications from Badoo, please click here to opt out. Badoo Trading Limited is a limited company registered in England and Wales under CRN 07540255 with its registered office at Media Village, 131-151 Great Titchfield Street, London, W1W 5BB. Follow us:
RE: tcpdump and Haproxy SSL Offloading
On 4 Jun 2016 11:53 pm, "mlist" wrote: > > Hi Luca and Igor, > > > > I know there is not a simple way. In this network trace I verified an IE11 / Edge bug with preconnect sessions. > > This is a known problem, also if not so documented. > > As you can see Windows Client TCP Stack correctly send an ACK to a FIN request from HAProxy, but IE > > does not instruct TCP Stack to send its own FIN to HAProxy to close the TCP connection gracefully, so IE at later time > > erroneously try to use the TCP Connection, correctly HAProxy send a RST. In the HTTP buffer of the client there is > > what HAProxy sent before closing the connection (a 408 Timeout HTTP Status message), so IE erroneously read this > > message and wrong again, instead of closing HTTP session and retry with another session, send the 408 > > Timeout to the Browser, so client see 408, thinking the server does not work well… > > > > > > > > > > In this case: > > - 408 is not sent from backend server, so not all the traffic can be collected on backend server to analyze the issue > > - HAProxy send 408, but I don’t see HTTP flow, I see only HTTPS (SSL/TLS) Flow as all is encrypted from client to haproxy and haproxy do SSL termination > > - In this particular case we can reproduce the problem on a test machine, as we know and manage the machine, we can use the wireshark unencypting capabilities, but I think this method is not so robust as involves many changing parts Cipher/haproxy/… > > > > I have to put togheter many info (haproxy log, tcpdump trace, ecc ) to have a complete picture (ie: it would be simple if I can see and search 408 http status code in the tcpdump trace instead of know it is here from browser and form haproxy log, but not shown in wireshark as incapsulated in TLS communication. Haproxy just do different not properly application level thing for SSL termination, so I’m uncertain is haproxy or not to manage and provide unencrypted traffic, all other solution as we are seeing are prone to difficulties and instability of the trace process. > > > > I like robust solution and if possible solution independent as soon as possible from implicit (instead explicit) support by components, that is: haproxy work using unencrypting with client session keys, but I know on some version this mechanism does not work, so this introduce instability. > > > > But I’ll try FPS with client session keys solution also if I think this is complex as we have to capture all keys for all TLS sessions to see the complete traffic we take as many SSL handshake take place in that trace. > > > > I think that at the moment the best solution is to temporarily disable PFS ciphers suite, I think this is a simple reconfiguration of haproxy.cfg and haproxy daemon restart, for the test period, not so critical for us. > > I need for this some hints as I’m not so informed about cipher suites, in particular I didn’t found a clear specification how to identify cipher show / used by haproxy and how identify what support or not PFS. > > > > ie: if I check from a SSL tester site I see this protocols and chiper suite for haproxy SSL termination: > > > > Protocols > > > >TLS 1.2 Yes > >TLS 1.1 Yes > >TLS 1.0 Yes > >SSL 3No > >SSL 2No > > > > > > Cipher Suites > > > >TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 > >TLS_RSA_WITH_RC4_128_SHA (0x5) INSECURE 128 > >TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011) ECDH secp256r1 (eq. 3072 bits RSA) FS INSECURE 128 > >TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 > >TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d) 256 > >TLS_RSA_WITH_AES_256_CBC_SHA (0x35) 256 > >TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (0x84) 256 > >TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027) ECDH secp256r1 (eq. 3072 bits RSA) FS 128 > >TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013) ECDH secp256r1 (eq. 3072 bits RSA) FS 128 > >TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012) ECDH secp256r1 (eq. 3072 bits RSA) FS 112 > >TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c) 128 > >TLS_RSA_WITH_AES_128_CBC_SHA (0x2f) 128 > >TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (0x41) 128 > >TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa) 112 > > > > In haproxy.cfg I used these cipher I found recommended: > > > > ciphers ECDHE-RSA-AES256-SHA:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM > Just remove ECDHE-RSA-AES256-SHA. Note that in your case this will leave you with rc4 cyphers only which are not secure so dont run this setup for long time.
