On 01/02/2015 07:35 πμ, Willy Tarreau wrote: > Hello Joseph, > > I'm CCing Bhaskar since he was the one proposing the first solution, he > may have some useful insights. Other points below. > > On Thu, Jan 15, 2015 at 01:23:59PM -0800, Joseph Lynch wrote: >> Hello, >> >> I am trying to set up a health check service similar to the inetd solutions >> suggested in the documentation. Unfortunately, my backends run on different >> ports because they are being created dynamically and as far as I can tell I >> cannot include the server port in my healthcheck either as part of the >> server declaration, a header, or as part of the healthcheck uri itself. >> >> I have been trying to come up with potential solutions that are not overly >> invasive, and I think that the simplest solution is to include the server >> host and port in the existing send-state header. I have included a patch >> that I believe does this at the end of this email. Before I go off >> maintaining a local fork, I wanted to ask if the haproxy devs would be >> sympathetic to me trying to upstream this patch? > > I'm personally fine with it. As you say, it's really not invasive, so we > could merge it and even backport it into 1.5-stable. I'd slightly change > something however, I'd use "address" instead of "host" in the field, since > that's what you're copying there. "Host" could be used later to copy the > equivalent of a host name, so let's not misuse the field name. > >> As for prior art, I found a few posts on this mailing list about the >> ability to add headers to http checks. I believe that something like >> http://marc.info/?l=haproxy&m=139181606417120&w=2 would be more then what >> we need to solve this problem, but that thread seems to have died. I do >> believe that a general ability to add headers to healthchecks would be >> superior to my patch, but the general solution is significantly harder to >> pull off. > > I'd like to re-heat that thread. I didn't even remember about it, indeed > we were busy finalizing 1.5. Bhaskar, I still think your work makes sense > for 1.6, so if you still have your patch, it's probably time to resend it :-) >
If I understood Bhaskar's suggestion correctly, we could delegate health check for backend servers to a single server which does all the health checking. Am I right ? If it this is case then the downside of multiple health checks when nbproc > 1 is gone! But, I would like to see a fail-back mechanism as we have with agent check in case that single server is gone. Alternatively, we could have Bhaskar's suggestion implemented in the agent check. I am re-heating the request of delegate health checks to a central service with a fall-back mechanism in place because * Reduces checks in setups where you have servers in multiple backends * Reduces checks in setups where you have more than 1 HAProxy active server(HAProxy servers behind a Layer 4 load balancer - ECMP and etc) * Reduces checks when multi-process model is used * Reduces CPU stress on firewalls, when they are present between HAProxy and backend servers. This assumes that there are enough resources on the 'health-checker' server to sustain huge amount of requests. Which is not a big deal if 'health-checker' solution is designed correctly, meaning that backend servers push their availability to that 'health-checker' server and etc. Furthermore, 'health-checker' server should have a check in place to detect backend servers not sending their health status and declare them down after a certain period of inactivity. In case of servers located across multiple vlans, there is a edge case where backend servers are reported as healthy but HAProxy fails to send traffic to them due to missing network routes, firewall holes and etc. The main gain of this solution is that you make backend servers responsible for announcing their availability, it is a mindset change as we have used to have LBs performing the health checks and be the authoritative source of such information. Cheers, Pavlos
signature.asc
Description: OpenPGP digital signature
