Hi, I'm hoping someone can help with an Exchange 2013 / NTLM Authentication question.
(My background is more on the networking side, so I'm finding my way a bit with HTTP related challenges...) We're currently running HA Proxy v1.6.4 and trying to use it in-front of an Exchange 2013 CAS set-up. The challenge that we're having is whilst everything appears to be working, we're having connectivity 'issues' when trying to migrate users to Office 365. I've been trying to read-up on this and understand what might be happening. - Our Frontend is running in HTTP mode, terminating the HTTPS connections to the outside world. - Our Backend is also running in HTTP mode, connecting back to the CAS Servers over HTTPS. The migration to Office 365 process connects to: https://mail.ourcompany.com/EWS/mrsproxy.svc >From the HA Proxy Logs, we can see that this works successfully for a number >of requests like this: Mar 23 09:12:19 ae-lb01.ourcompany.org haproxy[18151]: 132.245.40.245:56136 [23/Mar/2016:09:12:19.478] ft_exchange~ bk_exch_2013/ae-exch02 353/0/0/42/+395 200 +721 - - ---- 289/289/1/1/0 0/0 {mail.ourcompany.com||1504} {715506} "POST /EWS/mrsproxy.svc HTTP/1.1" Mar 23 09:12:19 ae-lb01. ourcompany.org haproxy[18151]: 132.245.40.245:56136 [23/Mar/2016:09:12:18.726] ft_exchange~ bk_exch_2013/ae-exch02 15/0/0/401/+416 200 +722 - - ---- 286/286/1/1/0 0/0 {mail.ourcompany.com||2435} {5620718} "POST /EWS/mrsproxy.svc HTTP/1.1" This works fine for about 500 log entries (in this instance; it appears to be random). Then we get a few 401 errors: Mar 23 09:12:19 ae-lb01.ourcompany.org haproxy[18151]: 132.245.40.245:61593 [23/Mar/2016:09:12:19.179] ft_exchange~ bk_exch_2013/ae-exch02 12/0/2/2/+16 401 +477 - - ---- 288/288/2/2/0 0/0 {mail.ourcompany.com||2435} {0} "POST /EWS/mrsproxy.svc HTTP/1.1" Mar 23 09:12:19 ae-lb01.ourcompany.org haproxy[18151]: 132.245.40.245:61593 [23/Mar/2016:09:12:19.207] ft_exchange~ bk_exch_2013/ae-exch02 6/0/0/1/+7 401 +693 - - ---- 288/288/2/2/0 0/0 {mail.ourcompany.com||0} {0} "POST /EWS/mrsproxy.svc HTTP/1.1" Interestingly, although the source IP is the same, the source tcp port changes. This is not unexpected; tcp connections will come to an end and new ones will start. The problem that we're having is that as soon as this new connection starts, we just then see a series of 401 errors from the same source IP, but different source tcp port. After doing some reading, as best as I can tell, this may be down to the NTLM / Windows Authentication that is used in the process. To understand this more, I've added some capture lines to my Frontend configuration: capture request header Host len 50 capture request header User-Agent len 500 capture request header Content-Length len 50 capture request header Authorization len 500 capture response header Content-Length len 50 capture response header WWW-Authenticate len 500 capture response header Authentication-Info len 500 After doing this and re-starting some bits, I can now see some more of the Authentication information is the logs: Mar 23 18:13:05 ae-lb01.ourcompany.org haproxy[3443]: 132.245.47.13:20099 [23/Mar/2016:18:13:05.660] ft_exchange~ bk_exch_2013/ae-exch02 8/0/1/2/+11 401 +693 - - ---- 192/192/1/1/0 0/0 {mail.ourcompany.com||0|Negotiate TlRMTVNTUAABAAAAl4II4gAAAAAAAAAAAAAAAAAAAAAGAvAjAAAADw==} {0|NTLM|} "POST /EWS/mrsproxy.svc HTTP/1.1" Mar 23 18:13:26 ae-lb01.ourcompany.org haproxy[3443]: 132.245.47.13:20099 [23/Mar/2016:18:13:05.671] ft_exchange~ bk_exch_2013/ae-exch02 7/0/0/21029/+21036 503 +482 - - ---- 194/194/1/1/0 0/0 {mail.ourcompany.com||663|Negotiate TlRMTVNTUAADAAAAGAAYAJYAAABsAWwBrgAAABIAEgBYAAAAEgASAGoAAAAaABoAfAAAABAAEAAaAgAAFYKI4gYC8CMAAAAPZvo1ootKOIM3n52pgdGLYm8AZgBmAGkAYwBlAG8AcgBnAG0AZgBhAGIAYQBkAG0AaQBuAEEATQAzAFAAUgAwADUATQBCAD} {0||} "POST /EWS/mrsproxy.svc HTTP/1.1" This was a useful insight into how Microsoft's NTLM / Windows Authentication uses the HTTP headers and 401 responses: http://www.innovation.ch/personal/ronald/ntlm.html As was: https://www.ietf.org/rfc/rfc4559.txt My conclusion is that because HA Proxy is re-using the backend to Exchange connection, the overall conversation between Office 365 and Exchange is no longer authenticated. My next step was to was to do some more reading about HA Proxy's HTTP 'connection modes'. This section in the documentation was great: http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4 As was this reference from ALOHA: https://www.haproxy.com/static/media/uploads/eng/resources/aloha_load_balancer_http_connection_mode_memo2.pdf So (finally), I'm down to my real question: - What HA Proxy HTTP 'connection' mode should I be using with a Backend that is providing an NTLM Authenticated Service? - My understand is that it should be 'tunnel mode'. - This can be configured on the backend with, 'option tunnel-mode'. - This is needed, because in v1.6 and above, the default connection mode is now 'keep alive'. Is anyone able to confirm that my understanding is correct? The reason that I'm slightly confused, as in ALOHA's guide (top of page 27), it looks like they've recommended using the 'option http-keep-alive' and 'no option tunnel-mode'. This would put the connection mode to 'keep alive'. https://www.haproxy.com/static/media/uploads/eng/resources/aloha_load_balancer_appnotes_0065_exchange_2013_deployment_guide_en.pdf So if anyone has had a similar experience, or can confirm the correct HTTP 'connection mode', that would be great. Many thanks, Graham.
