Allow an agent check to be run in conjunction with one other server health check.
If the backend for a server check is not lb-agent-chk then an agent check may also be run using the agent-check parameter to a server, which sets the TCP port to be used for the agent check. e.g. server web1_1 127.0.0.1:80 check agent-port 10000 The agent-inter parameter may also be used to specify the interval and timeout for agent checks. If either the health or agent check determines that a server is down then it is marked as being down, otherwise it is marked as being up. Signed-off-by: Simon Horman <ho...@verge.net.au> --- doc/configuration.txt | 61 ++++++++++++++++++++++++++++++++++------- include/types/server.h | 1 + src/cfgparse.c | 70 +++++++++++++++++++++++++++++++++++++++++++++--- src/checks.c | 13 +++++++-- src/haproxy.c | 6 +++++ 5 files changed, 136 insertions(+), 15 deletions(-) diff --git a/doc/configuration.txt b/doc/configuration.txt index 28c5223..51020c1 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -764,11 +764,12 @@ nosplice "option splice-response". spread-checks <0..50, in percent> - Sometimes it is desirable to avoid sending health checks to servers at exact - intervals, for instance when many logical servers are located on the same - physical server. With the help of this parameter, it becomes possible to add - some randomness in the check interval between 0 and +/- 50%. A value between - 2 and 5 seems to show good results. The default value remains at 0. + Sometimes it is desirable to avoid sending agent and health checks to + servers at exact intervals, for instance when many logical servers are + located on the same physical server. With the help of this parameter, it + becomes possible to add some randomness in the check interval between 0 + and +/- 50%. A value between 2 and 5 seems to show good results. The + default value remains at 0. tune.bufsize <number> Sets the buffer size to this size (in bytes). Lower values allow more @@ -3771,6 +3772,9 @@ option lb-agent-chk information described above. Otherwise the port of the serivce will be used. + This option conflicts with the agent-port parameter to a server. + + option ldap-check Use LDAPv3 health checks for server testing May be used in sections : defaults | frontend | listen | backend @@ -7373,6 +7377,43 @@ addr <ipv4|ipv6> Supported in default-server: No +agent-inter <delay> + The "agent-inter" parameter sets the interval between two agent checks + to <delay> milliseconds. If left unspecified, the delay defaults to 2000 ms. + + Just as with every other time-based parameter, it may be entered in any + other explicit unit among { us, ms, s, m, h, d }. The "agent-inter" + parameter also serves as a timeout for agent checks "timeout check" is + not set. In order to reduce "resonance" effects when multiple servers are + hosted on the same hardware, the agent and health checks of all servers + are started with a small time offset between them. It is also possible to + add some random noise in the agent and health checks interval using the + global "spread-checks" keyword. This makes sense for instance when a lot + of backends use the same servers. + + Requires the "agent-port" parameter to be set. + + Supported in default-server: Yes + +agent-port <port> + Using the "agent-port" parameter, it becomes possible to run an agent + check in conjunction with a regular health check. In this scenario the + "agent-port" parameter specifies the TCP that an agent check should + connect to. Typically this is different to the port of the service and + health-check. + + For a description agent checks and the protocol used by them see the + description of "lb-agent-check". + + In this the agent check is run in conjunction with another check + and as such the check backend should be set to some value other than + "lb-agent-check". An alternative scenario is to run only an agent check + in which case the check backend should be set to "lb-agent-check" and + "agent-port" should not be set; in that scenario the port may be set + using the "port" parameter. + + See also the "agent-inter" parameter. + backup When "backup" is present on a server line, the server is only used in load balancing when all other non-backup servers are unavailable. Requests coming @@ -7549,11 +7590,11 @@ downinter <delay> other explicit unit among { us, ms, s, m, h, d }. The "inter" parameter also serves as a timeout for health checks sent to servers if "timeout check" is not set. In order to reduce "resonance" effects when multiple servers are - hosted on the same hardware, the health-checks of all servers are started - with a small time offset between them. It is also possible to add some random - noise in the health checks interval using the global "spread-checks" - keyword. This makes sense for instance when a lot of backends use the same - servers. + hosted on the same hardware, the agent and health checks of all servers + are started with a small time offset between them. It is also possible to + add some random noise in the agent and health checks interval using the + global "spread-checks" keyword. This makes sense for instance when a lot + of backends use the same servers. Supported in default-server: Yes diff --git a/include/types/server.h b/include/types/server.h index 3a0281b..1eec648 100644 --- a/include/types/server.h +++ b/include/types/server.h @@ -191,6 +191,7 @@ struct server { } check_common; struct check check; /* health-check specific configuration */ + struct check agent; /* agent specific configuration */ #ifdef USE_OPENSSL int use_ssl; /* ssl enabled */ diff --git a/src/cfgparse.c b/src/cfgparse.c index 25665a8..4b124ff 100644 --- a/src/cfgparse.c +++ b/src/cfgparse.c @@ -1333,9 +1333,13 @@ void init_default_instance() defproxy.defsrv.check.inter = DEF_CHKINTR; defproxy.defsrv.check.fastinter = 0; defproxy.defsrv.check.downinter = 0; + defproxy.defsrv.agent.inter = DEF_CHKINTR; + defproxy.defsrv.agent.fastinter = 0; + defproxy.defsrv.agent.downinter = 0; defproxy.defsrv.rise = DEF_RISETIME; defproxy.defsrv.fall = DEF_FALLTIME; defproxy.defsrv.check.port = 0; + defproxy.defsrv.agent.port = 0; defproxy.defsrv.maxqueue = 0; defproxy.defsrv.minconn = 0; defproxy.defsrv.maxconn = 0; @@ -4060,7 +4064,7 @@ stats_error_parsing: else if (!strcmp(args[0], "server") || !strcmp(args[0], "default-server")) { /* server address */ int cur_arg; short realport = 0; - int do_check = 0, defsrv = (*args[0] == 'd'); + int do_agent = 0, do_check = 0, defsrv = (*args[0] == 'd'); if (!defsrv && curproxy == &defproxy) { Alert("parsing [%s:%d] : '%s' not allowed in 'defaults' section.\n", file, linenum, args[0]); @@ -4106,6 +4110,7 @@ stats_error_parsing: LIST_INIT(&newsrv->actconns); LIST_INIT(&newsrv->pendconns); do_check = 0; + do_agent = 0; newsrv->state = SRV_RUNNING; /* early server setup */ newsrv->last_change = now.tv_sec; newsrv->id = strdup(args[1]); @@ -4151,11 +4156,16 @@ stats_error_parsing: goto out; } - newsrv->check.use_ssl = curproxy->defsrv.check.use_ssl; + newsrv->check.use_ssl = curproxy->defsrv.check.use_ssl; newsrv->check.port = curproxy->defsrv.check.port; newsrv->check.inter = curproxy->defsrv.check.inter; newsrv->check.fastinter = curproxy->defsrv.check.fastinter; newsrv->check.downinter = curproxy->defsrv.check.downinter; + newsrv->agent.use_ssl = curproxy->defsrv.agent.use_ssl; + newsrv->agent.port = curproxy->defsrv.agent.port; + newsrv->agent.inter = curproxy->defsrv.agent.inter; + newsrv->agent.fastinter = curproxy->defsrv.agent.fastinter; + newsrv->agent.downinter = curproxy->defsrv.agent.downinter; newsrv->rise = curproxy->defsrv.rise; newsrv->fall = curproxy->defsrv.fall; newsrv->maxqueue = curproxy->defsrv.maxqueue; @@ -4172,6 +4182,7 @@ stats_error_parsing: = curproxy->defsrv.iweight; newsrv->check.health = newsrv->rise; /* up, but will fall down at first failure */ + newsrv->agent.health = newsrv->rise; /* up, but will fall down at first failure */ cur_arg = 3; } else { @@ -4180,7 +4191,29 @@ stats_error_parsing: } while (*args[cur_arg]) { - if (!defsrv && !strcmp(args[cur_arg], "cookie")) { + if (!strcmp(args[cur_arg], "agent-inter")) { + const char *err = parse_time_err(args[cur_arg + 1], &val, TIME_UNIT_MS); + if (err) { + Alert("parsing [%s:%d] : unexpected character '%c' in 'agent-inter' argument of server %s.\n", + file, linenum, *err, newsrv->id); + err_code |= ERR_ALERT | ERR_FATAL; + goto out; + } + if (val <= 0) { + Alert("parsing [%s:%d]: invalid value %d for argument '%s' of server %s.\n", + file, linenum, val, args[cur_arg], newsrv->id); + err_code |= ERR_ALERT | ERR_FATAL; + goto out; + } + newsrv->agent.inter = val; + cur_arg += 2; + } + else if (!strcmp(args[cur_arg], "agent-port")) { + newsrv->agent.port = atol(args[cur_arg + 1]); + do_agent = 1; + cur_arg += 2; + } + else if (!defsrv && !strcmp(args[cur_arg], "cookie")) { newsrv->cookie = strdup(args[cur_arg + 1]); newsrv->cklen = strlen(args[cur_arg + 1]); cur_arg += 2; @@ -4208,6 +4241,8 @@ stats_error_parsing: if (newsrv->check.health) newsrv->check.health = newsrv->rise; + if (newsrv->agent.health) + newsrv->agent.health = newsrv->rise; cur_arg += 2; } else if (!strcmp(args[cur_arg], "fall")) { @@ -4379,6 +4414,7 @@ stats_error_parsing: newsrv->state |= SRV_MAINTAIN; newsrv->state &= ~SRV_RUNNING; newsrv->check.health = 0; + newsrv->agent.health = 0; cur_arg += 1; } else if (!defsrv && !strcmp(args[cur_arg], "observe")) { @@ -4757,6 +4793,33 @@ stats_error_parsing: newsrv->state |= SRV_CHECKED; } + if (do_agent && do_check) { + int ret; + + if (!newsrv->agent.port) { + Alert("parsing [%s:%d] : server %s has agent-inter without agent-port.\n", + file, linenum, newsrv->id); + err_code |= ERR_ALERT | ERR_FATAL; + goto out; + } + + if (newsrv->proxy->options2 & PR_O2_LB_AGENT_CHK) { + Alert("parsing [%s:%d] : server %s has agent-inter or agent-port but check type is lb-agent-chk.\n", + file, linenum, newsrv->id); + err_code |= ERR_ALERT | ERR_FATAL; + goto out; + } + + if (!newsrv->agent.inter) + newsrv->agent.inter = newsrv->check.inter; + + ret = init_check(newsrv, "Agent", &newsrv->agent, file, linenum); + if (ret) { + err_code |= ret; + goto out; + } + } + if (!defsrv) { if (newsrv->state & SRV_BACKUP) curproxy->srv_bck++; @@ -6596,6 +6659,7 @@ out_uri_auth_compat: newsrv->state |= SRV_MAINTAIN; newsrv->state &= ~SRV_RUNNING; newsrv->check.health = 0; + newsrv->agent.health = 0; } newsrv->track = srv; diff --git a/src/checks.c b/src/checks.c index 3ab8750..d763d2c 100644 --- a/src/checks.c +++ b/src/checks.c @@ -399,7 +399,7 @@ void set_server_down(struct check *check) check->health = s->rise; } - if (check->health == s->rise || s->track) { + if ((s->state & SRV_RUNNING && check->health == s->rise) || s->track) { int srv_was_paused = s->state & SRV_GOINGDOWN; int prev_srv_count = s->proxy->srv_bck + s->proxy->srv_act; @@ -466,7 +466,8 @@ void set_server_up(struct check *check) { check->health = s->rise; } - if (check->health == s->rise || s->track) { + if ((s->check.health >= s->rise && s->agent.health >= s->rise && + check->health == s->rise) || s->track) { if (s->proxy->srv_bck == 0 && s->proxy->srv_act == 0) { if (s->proxy->last_change < now.tv_sec) // ignore negative times s->proxy->down_time += now.tv_sec - s->proxy->last_change; @@ -1603,9 +1604,17 @@ int start_checks() { if (!(s->state & SRV_CHECKED)) continue; + /* A task for the primary check */ s->check.type = s->proxy->options2 & PR_O2_CHK_ANY; if (start_check_task(&s->check, mininter, nbcheck, srvpos++)) return -1; + + /* A task for a secondary agent check */ + if (s->agent.port) { + s->agent.type = PR_O2_LB_AGENT_CHK; + if (start_check_task(&s->agent, mininter, nbcheck, srvpos++)) + return -1; + } } } return 0; diff --git a/src/haproxy.c b/src/haproxy.c index 92e25c2..ac85949 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -1057,6 +1057,10 @@ void deinit(void) task_delete(s->check.task); task_free(s->check.task); } + if (s->agent.task) { + task_delete(s->agent.task); + task_free(s->agent.task); + } if (s->warmup) { task_delete(s->warmup); @@ -1067,6 +1071,8 @@ void deinit(void) free(s->cookie); free(s->check.bi); free(s->check.bo); + free(s->agent.bi); + free(s->agent.bo); free(s); s = s_next; }/* end while(s) */ -- 1.7.10.4