Re: HTTP/2 to backend server fails health check when 'option httpchk' set
On Sun, Dec 16, 2018 at 12:01:18PM +0100, Baptiste wrote: > I don't know H2 well, but can't we forge an HTTP/2 query using tcp-check > script? Yes you can but you can't predict the result :-) Willy
Re: HTTP/2 to backend server fails health check when 'option httpchk' set
On Sun, Dec 16, 2018 at 12:20 AM Willy Tarreau wrote: > On Sat, Dec 15, 2018 at 11:11:58PM +, Nick Ramirez wrote: > > Thanks! That points me in the right direction. I found that to enable > Layer > > 7 health checks in this case, I would open another port on the web server > > that does not advertise HTTP/2 support (ALPN HTTP/1.1) or does not use > TLS > > (which also turns off HTTP/2 in the case of the Caddy web server), and > then > > use the "port" parameter on the server line to point to that port. > > > > backend webservers > > balance roundrobin > > option httpchk HEAD / > > server server1 web:443 ssl verify none alpn h2,http/1.1 check port > 80 > > > > Layer 7 health checks back up and running. :-) > > Yes definitely, if you have clear-text there it's the way to do it. > Otherwise you can do it in H1 over TLS since your server is supposed > to serve H1 if no ALPN is negociated, but it really depends how both > sides agree on this. And I would not be surprised if checks run over > H1/TLS force a new handshake to happen for regular traffic since a > single session key can be stored per server. > > Willy > > Hi, I don't know H2 well, but can't we forge an HTTP/2 query using tcp-check script? Baptiste
Re: HTTP/2 to backend server fails health check when 'option httpchk' set
On Sat, Dec 15, 2018 at 11:11:58PM +, Nick Ramirez wrote: > Thanks! That points me in the right direction. I found that to enable Layer > 7 health checks in this case, I would open another port on the web server > that does not advertise HTTP/2 support (ALPN HTTP/1.1) or does not use TLS > (which also turns off HTTP/2 in the case of the Caddy web server), and then > use the "port" parameter on the server line to point to that port. > > backend webservers > balance roundrobin > option httpchk HEAD / > server server1 web:443 ssl verify none alpn h2,http/1.1 check port 80 > > Layer 7 health checks back up and running. :-) Yes definitely, if you have clear-text there it's the way to do it. Otherwise you can do it in H1 over TLS since your server is supposed to serve H1 if no ALPN is negociated, but it really depends how both sides agree on this. And I would not be surprised if checks run over H1/TLS force a new handshake to happen for regular traffic since a single session key can be stored per server. Willy
Re[2]: HTTP/2 to backend server fails health check when 'option httpchk' set
Thanks! That points me in the right direction. I found that to enable Layer 7 health checks in this case, I would open another port on the web server that does not advertise HTTP/2 support (ALPN HTTP/1.1) or does not use TLS (which also turns off HTTP/2 in the case of the Caddy web server), and then use the "port" parameter on the server line to point to that port. backend webservers balance roundrobin option httpchk HEAD / server server1 web:443 ssl verify none alpn h2,http/1.1 check port 80 Layer 7 health checks back up and running. :-) -- Original Message -- From: "Willy Tarreau" To: "Nick Ramirez" Cc: haproxy@formilux.org Sent: 12/15/2018 10:25:42 AM Subject: Re: HTTP/2 to backend server fails health check when 'option httpchk' set Hi Nick, On Fri, Dec 14, 2018 at 10:43:04PM +, Nick Ramirez wrote: This may be something very simple that I am missing. I am using the latest HAProxy Docker image, which is using HAProxy 1.9-dev10 2018/12/08. It is using HTTP/2 to the backend web server (Caddy). It fails its health check if I uncomment the "option httpchk" line: backend webservers balance roundrobin #option httpchk server server1 web:443 check ssl verify none alpn h2 With that line commented out, it works. The project is on Github: https://github.com/NickMRamirez/experiment-haproxy-http2 Am I doing something wrong? It also works if I remove "option http-use-htx" and "alpn h2" and uncomment "option httpchk". You're not really doing anything wrong, it's just the current limitation of health checks that we've wanted to redesing for years and that deserve a year worth of work. Currently health checks are only made of a TCP string sent over the socket and checked in return. Since 1.6 or so, we introduced the ability to send this string over SSL (when "check-ssl" is set) but that's basically the limit. In fact, health checks are completely separate from the traffic. You can see them as being part of the control plane while the traffic is the data plane. You're not even forced to send them to the same IP, ports, nor protocol as your traffic. They only pre-set the same target IP and port for convenience, but that's all. I've thought we could at least implement an H2 preface+settings check but this would provide a very low value for quite some hassle to make it work for the user, so I think it would only steer the efforts away from a real redesign of better HTTP checks. However we should at the very least document this limitation more clearly for 1.9, as chances are that a number of people will want to try this :-/ Thanks, Willy
Re: HTTP/2 to backend server fails health check when 'option httpchk' set
Hi Nick, On Fri, Dec 14, 2018 at 10:43:04PM +, Nick Ramirez wrote: > This may be something very simple that I am missing. I am using the latest > HAProxy Docker image, which is using HAProxy 1.9-dev10 2018/12/08. It is > using HTTP/2 to the backend web server (Caddy). > > It fails its health check if I uncomment the "option httpchk" line: > > backend webservers > balance roundrobin > #option httpchk > server server1 web:443 check ssl verify none alpn h2 > > > With that line commented out, it works. > > The project is on Github: > https://github.com/NickMRamirez/experiment-haproxy-http2 > > Am I doing something wrong? It also works if I remove "option http-use-htx" > and "alpn h2" and uncomment "option httpchk". You're not really doing anything wrong, it's just the current limitation of health checks that we've wanted to redesing for years and that deserve a year worth of work. Currently health checks are only made of a TCP string sent over the socket and checked in return. Since 1.6 or so, we introduced the ability to send this string over SSL (when "check-ssl" is set) but that's basically the limit. In fact, health checks are completely separate from the traffic. You can see them as being part of the control plane while the traffic is the data plane. You're not even forced to send them to the same IP, ports, nor protocol as your traffic. They only pre-set the same target IP and port for convenience, but that's all. I've thought we could at least implement an H2 preface+settings check but this would provide a very low value for quite some hassle to make it work for the user, so I think it would only steer the efforts away from a real redesign of better HTTP checks. However we should at the very least document this limitation more clearly for 1.9, as chances are that a number of people will want to try this :-/ Thanks, Willy
HTTP/2 to backend server fails health check when 'option httpchk' set
This may be something very simple that I am missing. I am using the latest HAProxy Docker image, which is using HAProxy 1.9-dev10 2018/12/08. It is using HTTP/2 to the backend web server (Caddy). It fails its health check if I uncomment the "option httpchk" line: backend webservers balance roundrobin #option httpchk server server1 web:443 check ssl verify none alpn h2 With that line commented out, it works. The project is on Github: https://github.com/NickMRamirez/experiment-haproxy-http2 Am I doing something wrong? It also works if I remove "option http-use-htx" and "alpn h2" and uncomment "option httpchk". Thanks, Nick Ramirez
