I've been trying to convert my haproxy setup (1.8.14) to by adding "alpn
h2,http/1.1" to my "bind" line in the frontend. My haproxy config is north
of 300 lines, so I'll hold off on attaching it.
My frontend selects backends using something like:
acl aerial_acl hdr(host) -m reg -i ^aerial[1-4].example.com
acl aerial_acl ssl_fc_sni -m reg -i ^aerial[1-4].example.com
use_backend aerial if aerial_acl
acl wwwl_acl hdr(host) -m reg -i ^www.example.com
acl www_acl ssl_fc_sni -m reg -i ^www.example.com
use_backend www if www_acl
default_backend www
And I have a bunch of those selecting different backends.
The base haproxy config is quite battle-tested, it has been running on my
staging environment with H2 for ~6 months. The production config is a
refinement (based off staging) that has been running for a few years in
production.
But, when I enable H2 some small number of requests seem to be going to the
wrong backend. I see something like a bunch of requests for map tiles on
one connection, then that connection gets a request that should be going to
the main web server, it has a header of "www.example.com" instead of "
aerial1.example.com", the backend server logs that the Host header was "
www.example.com", but it is the aerial backend.
This happens infrequently, and if I take the "alpn h2,http/1.1" off the
"bind" line it does not seem to happen. I have only small periods of time
when I've run it so I can't give an exhaustive list of user-agents, but I
definitely saw it happening with Chrome 70 on Windows 10, Chrome 71 on
Windows 7, Chrome 69 on Windows 10, Safari 12 on iOS 12.1, Chrome 68 on
Android...
So, in summary:
- Works fine without "alpn h2,http/1.1".
- Only some requests are mis-routed.
- It seems to be a keep-alive connection that opens to get map tiles, then
a few seconds or minutes later try to get something from the web server.
- The host header in the request seems to be correct, the backend is
logging it correct.
- The aerial tile and web servers have the same external IP address and are
routed based on the "Host" sent with the request.
Seems like haproxy is misrouting the requests based on the previous
requests made in that connection. I see other requests coming in with
different names from the same user and they have different source ports, so
in that case the browser seems to be making the connections on different
TCP connections. I'm not sure if the browsers are supposed to interleave
requests for different hostnames on the same connection (we have a ton of
services that have been consolidated onto a single IP now that we have
haproxy, so a couple dozen names resolve to the same IP).
Thoughts?
Thanks,
Sean
