The goal was to serve every domain's lists from a single map file. For the
string and regex match methods that can be faked by folding the domain into
the key ("<type>:<domain>:<value>"), but map_ip cannot: its keys must be
parsable IP addresses or networks, so there is nowhere to put the domain.
The address lists therefore stay split, one map file and one configuration
rule per domain:
http-request set-var(txn.ip_grp)
src,map_ip(/etc/acl/shop.example.com/ip_list.map,ip=0) if { var(txn.site) -m
str shop.example.com }
http-request set-var(txn.ip_grp)
src,map_ip(/etc/acl/news.example.com/ip_list.map,ip=0) if { var(txn.site) -m
str news.example.com }
... one more line for every domain ...
which costs one ACL evaluation per domain on every request, one pat_ref plus
one pattern tree and lock per domain, and requires editing the configuration
whenever a domain is added or removed.
This patch adds a "conditional map" flavour of every map_* converter. Each
entry of the file holds one extra leading key, the condition, which is always
matched exactly as a string, and only the entries whose condition matches take
part in the lookup:
# <condition> <pattern> <value>
shop.example.com 10.20.10.0/23 ip=1
news.example.com 10.20.10.0/23 ip=7
http-request set-var(txn.site) req.hdr(host),lower
http-request set-var(txn.ip_grp)
src,cmap_ip(/etc/acl/ip_list.map,txn.site,ip=0)
map_ip is the case that has no workaround at all, but the flavour is provided
for every match method rather than for addresses only. Folding the domain into
the key does work for str, beg and reg, yet it leaves a prefix in front of the
pattern: end and sub then cannot constrain the domain at all, and "^" anchors
the prefix instead of the value the operator cares about. Taking the domain
out of the pattern restores all of them.
Resolving the group first is also cheaper than scanning a shared list. map_reg
walks its list, so once every domain shares one file each lookup runs the
regexes of all the other domains too; with the condition resolved by an exact
match in a tree, only the entries of that domain are considered.
The condition is taken from the second argument, which is either a variable
name, read directly with no allocation nor formatting, or a log-format string,
which lets it combine literals, sample fetches and converters. The latter has
to be quoted since it contains parenthesis and commas:
http-request set-var(txn.url_grp)
'pathq,cmap_beg(/etc/acl/beg_list.map,"url:%[var(txn.site)]",url=0)'
Implementation: the pattern_tree of a conditional expression holds condition
groups (struct pattern_cond_grp) indexed on their condition key, each of them
owning a regular sub-expression which is filled and looked up with the stock
parse/index/match functions of the underlying match method. The second key is
therefore parsed, indexed and matched exactly like in a plain map, tree lookups
included, so a conditional map is not slower than the equivalent set of
per-condition maps. Only pat_ref_push() (which picks the group a pattern is
indexed in) and pattern_exec_match() (which selects the group to look up) had
to be made condition-aware; no new match function was needed.
"get map" reports the condition it looked up and whether the map holds any
entry for it at all ('cond="shop.example.com", cond_found=yes'), which tells
an unknown condition apart from a condition whose entries simply did not match.
Nothing is added to the output of a plain map or of an ACL.
The two keys are stored space-separated as a single pat_ref key, so the whole
runtime API works unchanged: "show map" reports the three fields like the file
does, the payload syntax takes them as the three columns of the file format,
and "prepare/add @ver/commit" still replaces a map atomically. New condition
groups may be created at run time. The http-request set-map action also works,
its key log-format simply has to produce "<condition> <pattern>".
On the command line the condition is taken as its own argument rather than as
part of an escaped key, since the CLI splits its input on spaces and requiring
a backslash there is a good way to corrupt a map by hand. add/set/del/get thus
spell an entry out exactly like the file and the payload syntax do:
add map <map> <condition> <pattern> <value>
set map <map> <condition> <pattern> <value>
del map <map> <condition> <pattern>
get map <map> <condition> <sample>
"get map" takes the sample to look up rather than a pattern, as it does on a
plain map: it runs the match method, so on a beg/sub/reg/ip map the argument
that finds an entry is generally not the stored pattern.
The value is taken as everything left on the command line, the way the file and
the payload syntax take everything left on the line, so a value holding spaces
is not silently truncated to its first word. The "#<id>" form of "set map" and
"del map" addresses an entry directly and keeps its usual layout. A plain map
and an ACL are untouched: the extra argument is only consumed when the reference
carries PAT_REF_COND.
A condition may be at most 255 characters long, and a map file used by a
conditional map cannot be used by a plain map nor by an ACL (both are reported
as configuration errors).
Condition groups are reference counted and released as soon as their last
pattern goes away, so a map whose conditions come and go (a domain being
decommissioned, for instance) does not accumulate them. Without this they
would only ever be reclaimed by pat_prune_cond(), which the map path never
reaches, and "get map" would keep reporting cond_found=yes for a condition
whose entries are all gone, defeating the very distinction that field exists
for.
A reg-test covering both tree- and list-indexed match methods, the runtime
API and the release of a group with its last entry is added as
reg-tests/converter/cmap.vtc.
The whole cmap_<match_type>[_<output_type>] family mirrors the map_* one. The
"key" output type returns the pattern of the matched entry without its
condition, since the caller already knows the latter.
Care was taken to leave the existing map_* and ACL paths alone: every function
they use at run time is bit-for-bit unchanged (verified by comparing the
disassembly against a pristine tree) -- pattern_exec_match(), which ACLs use
too, sample_conv_map(), sample_conv_map_key(), acl_exec_cond(), pat_prune_gen(),
pat_delete_gen(), pat_ref_delete_by_ptr(), pat_ref_purge_range(),
pattern_new_expr(), pat_ref_append(), pat_ref_read_from_file(), and the show /
clear / prepare / commit map CLI handlers. The conditional lookup is a separate
pattern_exec_match_cond() and the cmap converters have their own load and lookup
functions rather than sharing the map_* ones.
Only these are shared, all of them at configuration-load or CLI-parse time and
all behind a "flags & PAT_REF_COND" test that a plain map never takes:
pat_ref_push(), pat_ref_read_from_file_smp(), pattern_read_from_file(),
cli_io_handler_map_lookup(), and the add / set / del / get map CLI parsers,
which consume the condition as an extra argument. On a plain map or an ACL those
four parsers keep their previous argument layout and code path.
sample_load_map()
only differs by the size passed to calloc(), as struct map_descriptor grew by
the
log-format holder.
---
include/haproxy/map-t.h | 3 +
include/haproxy/map.h | 2 +
include/haproxy/pattern-t.h | 24 ++
include/haproxy/pattern.h | 37 +++
reg-tests/converter/cmap.vtc | 134 +++++++++
src/map.c | 565 ++++++++++++++++++++++++++++++++++-
src/pattern.c | 323 +++++++++++++++++++-
7 files changed, 1069 insertions(+), 19 deletions(-)
create mode 100644 reg-tests/converter/cmap.vtc
diff --git a/include/haproxy/map-t.h b/include/haproxy/map-t.h
index d6085ee..7b9041c 100644
--- a/include/haproxy/map-t.h
+++ b/include/haproxy/map-t.h
@@ -22,12 +22,15 @@
#ifndef _HAPROXY_MAP_T_H
#define _HAPROXY_MAP_T_H
+#include <haproxy/log-t.h>
#include <haproxy/pattern-t.h>
#include <haproxy/sample-t.h>
struct map_descriptor {
struct sample_conv *conv; /* original converter descriptor */
struct pattern_head pat; /* the pattern matching associated to
the map */
+ struct lf_expr cond_fmt; /* conditional maps: log-format building
the condition,
+ only used when the argument is not a
variable name */
int do_free; /* set if <pat> is the original pat and
must be freed */
};
diff --git a/include/haproxy/map.h b/include/haproxy/map.h
index 3ec3418..ff640e7 100644
--- a/include/haproxy/map.h
+++ b/include/haproxy/map.h
@@ -35,5 +35,7 @@ struct map_reference *map_get_reference(const char
*reference);
int sample_load_map(struct arg *arg, struct sample_conv *conv,
const char *file, int line, char **err);
+int sample_load_cmap(struct arg *arg, struct sample_conv *conv,
+ const char *file, int line, char **err);
#endif /* _HAPROXY_MAP_H */
diff --git a/include/haproxy/pattern-t.h b/include/haproxy/pattern-t.h
index 35c4fd0..4509959 100644
--- a/include/haproxy/pattern-t.h
+++ b/include/haproxy/pattern-t.h
@@ -98,6 +98,12 @@ enum {
#define PAT_REF_SMP 0x04 /* Flag used if the reference contains a sample. */
#define PAT_REF_FILE 0x08 /* Set if the reference was loaded from a file */
#define PAT_REF_ID 0x10 /* Set if the reference is only an ID (not loaded
from a file) */
+#define PAT_REF_COND 0x20 /* Set if the reference is used by a conditional map
(cmap_*) */
+
+/* Maximum length of the condition key of a conditional map. Longer keys can
+ * neither be loaded nor looked up.
+ */
+#define PAT_COND_KEY_MAXLEN 255
/* This struct contain a list of reference strings for dunamically
* updatable patterns.
@@ -213,6 +219,24 @@ struct pattern_expr {
__decl_thread(HA_RWLOCK_T lock); /* lock used to protect
patterns */
};
+/* A conditional map (see the cmap_* converters) splits its entries into groups
+ * which are selected by an exact match on a leading condition key, so that the
+ * second key keeps being matched exactly like in a plain map. Each group owns
a
+ * regular sub-expression, fed with the stock parse/index/match functions of
the
+ * underlying match method, and is indexed on its condition key in the
+ * pattern_tree of the top-level expression. <node> must remain last as its key
+ * is variable-sized.
+ */
+struct pattern_cond_grp {
+ struct pattern_expr expr; /* sub-expression holding this group's
patterns */
+ unsigned int entries; /* patterns currently indexed here. The
group is
+ released once this drops back to zero,
otherwise
+ one would accumulate for every
condition ever
+ seen and "get map" would keep reporting
+ cond_found=yes for an emptied
condition. */
+ struct ebmb_node node; /* indexed on the condition key in
expr->pattern_tree */
+};
+
/* This is a list of expression. A struct pattern_expr can be used by
* more than one "struct pattern_head". this intermediate struct
* permit more than one list.
diff --git a/include/haproxy/pattern.h b/include/haproxy/pattern.h
index 570ff28..e94d740 100644
--- a/include/haproxy/pattern.h
+++ b/include/haproxy/pattern.h
@@ -24,6 +24,8 @@
#include <string.h>
+#include <import/ist.h>
+
#include <haproxy/api.h>
#include <haproxy/event_hdl.h>
#include <haproxy/pattern-t.h>
@@ -62,6 +64,36 @@ static inline int pat_find_match_name(const char *name)
*/
struct pattern *pattern_exec_match(struct pattern_head *head, struct sample
*smp, int fill);
+/* Same as above for a conditional map: only the patterns stored under the
+ * condition group whose key exactly matches <cond> are considered. Returns
+ * NULL if no group matches <cond>, if <cond> is empty or longer than
+ * PAT_COND_KEY_MAXLEN, or if the sample matches none of the group's patterns.
+ */
+struct pattern *pattern_exec_match_cond(struct pattern_head *head, struct
sample *smp,
+ int fill, struct ist cond);
+
+/* Returns the sub-expression of the condition group of <expr> whose key
+ * exactly matches <cond>, or NULL if there is none. <expr> must belong to a
+ * conditional map and the caller must hold at least a read lock on it.
+ */
+struct pattern_expr *pattern_cond_grp_expr(struct pattern_expr *expr, struct
ist cond);
+
+/* Splits the composite key of a conditional map entry, which is stored as
+ * "<cond> <pattern>", and returns the condition key. <pattern> is set to the
+ * beginning of the second key, which the loaders guarantee to be non-empty.
+ */
+static inline struct ist pat_cond_split(const char *key, const char **pattern)
+{
+ const char *sep = strchr(key, ' ');
+
+ if (!sep) {
+ *pattern = key + strlen(key);
+ return ist(key);
+ }
+ *pattern = sep + 1;
+ return ist2(key, sep - key);
+}
+
/*
*
* The following function gets "pattern", duplicate it and index it in "expr"
@@ -92,6 +124,11 @@ void pat_delete_gen(struct pat_ref *ref, struct pat_ref_elt
*elt);
*/
void pat_prune_gen(struct pattern_expr *expr);
+/* Same as above for the top-level expression of a conditional map: it also
+ * releases the condition groups.
+ */
+void pat_prune_cond(struct pattern_expr *expr);
+
/*
*
* The following functions are general purpose pattern matching functions.
diff --git a/reg-tests/converter/cmap.vtc b/reg-tests/converter/cmap.vtc
new file mode 100644
index 0000000..3915e2b
--- /dev/null
+++ b/reg-tests/converter/cmap.vtc
@@ -0,0 +1,134 @@
+varnishtest "cmap converters Test"
+
+feature ignore_unknown_macro
+
+# The conditional maps hold one extra leading key which is exactly matched
+# against a variable, so that a single file can serve entries whose second key
+# would otherwise collide (typically per-domain address or regex lists).
+
+shell {
+ printf "# cond pattern value\n\
+shop.example.com 10.20.10.0/23 ip=1\n\
+shop.example.com 192.168.0.1 ip=2\n\
+news.example.com 10.20.10.0/23 ip=99\n\
+news.example.com 101.20.20.23 ip=3\n\
+v6.example.com 2001:db8::/32 ip=4\n" > "${tmpdir}/cmap_ip.map"
+
+ printf "shop.example.com ^/api/v[0-9]+/ r=api1\n\
+shop.example.com ^/static/ r=st1\n\
+news.example.com ^/api/v[0-9]+/ r=api2\n" > "${tmpdir}/cmap_reg.map"
+}
+
+haproxy h1 -conf {
+ defaults
+ mode http
+ timeout connect "${HAPROXY_TEST_TIMEOUT-5s}"
+ timeout client "${HAPROXY_TEST_TIMEOUT-5s}"
+ timeout server "${HAPROXY_TEST_TIMEOUT-5s}"
+
+ frontend fe
+ bind "fd@${fe}"
+
+ http-request set-var(txn.site) req.hdr(x-site) if { req.hdr(x-site) -m
found }
+
+ http-request set-var(txn.ip)
req.hdr(x-ip),cmap_ip("${tmpdir}/cmap_ip.map",txn.site,ip=0)
+ http-request set-var(txn.key)
req.hdr(x-ip),cmap_ip_key("${tmpdir}/cmap_ip.map",txn.site)
+ http-request set-var(txn.reg)
path,cmap_reg("${tmpdir}/cmap_reg.map",txn.site,r=0)
+
+ http-request return status 200 hdr ip "%[var(txn.ip)]" hdr key
"%[var(txn.key)]" hdr reg "%[var(txn.reg)]"
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+ # the same pattern under two conditions yields two distinct values
+ txreq -hdr "x-site: shop.example.com" -hdr "x-ip: 10.20.11.5"
+ rxresp
+ expect resp.http.ip == "ip=1"
+ expect resp.http.key == "10.20.10.0/23"
+
+ txreq -hdr "x-site: news.example.com" -hdr "x-ip: 10.20.11.5"
+ rxresp
+ expect resp.http.ip == "ip=99"
+ expect resp.http.key == "10.20.10.0/23"
+
+ # an entry only matches under its own condition
+ txreq -hdr "x-site: shop.example.com" -hdr "x-ip: 192.168.0.1"
+ rxresp
+ expect resp.http.ip == "ip=2"
+
+ txreq -hdr "x-site: news.example.com" -hdr "x-ip: 192.168.0.1"
+ rxresp
+ expect resp.http.ip == "ip=0"
+
+ # IPv6 entries too
+ txreq -hdr "x-site: v6.example.com" -hdr "x-ip: 2001:db8::1"
+ rxresp
+ expect resp.http.ip == "ip=4"
+
+ txreq -hdr "x-site: shop.example.com" -hdr "x-ip: 2001:db8::1"
+ rxresp
+ expect resp.http.ip == "ip=0"
+
+ # unknown and unset conditions fall back to the default value
+ txreq -hdr "x-site: be9" -hdr "x-ip: 10.20.11.5"
+ rxresp
+ expect resp.http.ip == "ip=0"
+
+ txreq -hdr "x-ip: 10.20.11.5"
+ rxresp
+ expect resp.http.ip == "ip=0"
+
+ # list-based match methods are conditioned as well
+ txreq -url "/api/v2/x" -hdr "x-site: shop.example.com"
+ rxresp
+ expect resp.http.reg == "r=api1"
+
+ txreq -url "/api/v2/x" -hdr "x-site: news.example.com"
+ rxresp
+ expect resp.http.reg == "r=api2"
+
+ txreq -url "/static/a.js" -hdr "x-site: shop.example.com"
+ rxresp
+ expect resp.http.reg == "r=st1"
+
+ txreq -url "/static/a.js" -hdr "x-site: news.example.com"
+ rxresp
+ expect resp.http.reg == "r=0"
+} -run
+
+# the runtime API sees both keys as a single space-separated key
+haproxy h1 -cli {
+ send "show map ${tmpdir}/cmap_ip.map"
+ expect ~ "shop.example.com 10.20.10.0/23 ip=1"
+
+ send "get map ${tmpdir}/cmap_ip.map news.example.com 10.20.11.5"
+ expect ~ "found=yes.*key=\"news.example.com 10.20.10.0/23\",
value=\"ip=99\""
+
+ # a new condition group can be created at run time
+ send "add map ${tmpdir}/cmap_ip.map new.example.com 172.16.0.0/12 ip=5"
+ expect ~ .+
+}
+
+client c2 -connect ${h1_fe_sock} {
+ txreq -hdr "x-site: new.example.com" -hdr "x-ip: 172.20.1.1"
+ rxresp
+ expect resp.http.ip == "ip=5"
+} -run
+
+haproxy h1 -cli {
+ send "del map ${tmpdir}/cmap_ip.map new.example.com 172.16.0.0/12"
+ expect ~ .*
+
+ # the group is released with its last entry, so "get map" stops
reporting
+ # the condition as known while the other groups are untouched
+ send "get map ${tmpdir}/cmap_ip.map new.example.com 172.20.1.1"
+ expect ~ "cond_found=no"
+
+ send "get map ${tmpdir}/cmap_ip.map shop.example.com 10.20.11.5"
+ expect ~ "cond_found=yes.*found=yes"
+}
+
+client c3 -connect ${h1_fe_sock} {
+ txreq -hdr "x-site: new.example.com" -hdr "x-ip: 172.20.1.1"
+ rxresp
+ expect resp.http.ip == "ip=0"
+} -run
diff --git a/src/map.c b/src/map.c
index 772e7b1..1082839 100644
--- a/src/map.c
+++ b/src/map.c
@@ -16,7 +16,10 @@
#include <haproxy/api.h>
#include <haproxy/applet.h>
#include <haproxy/arg.h>
+#include <haproxy/cfgparse.h>
+#include <haproxy/chunk.h>
#include <haproxy/cli.h>
+#include <haproxy/log.h>
#include <haproxy/map.h>
#include <haproxy/pattern.h>
#include <haproxy/regex.h>
@@ -25,6 +28,7 @@
#include <haproxy/stats-t.h>
#include <haproxy/stconn.h>
#include <haproxy/tools.h>
+#include <haproxy/vars.h>
/* Parse an IPv4 or IPv6 address and store it into the sample.
@@ -172,6 +176,207 @@ int sample_load_map(struct arg *arg, struct sample_conv
*conv,
return 0;
}
+/* Loads the map file of a "cmap_*" converter. This is sample_load_map() above
+ * plus the condition argument taken in second position, which shifts the
+ * optional default value to arg[2].
+ */
+int sample_load_cmap(struct arg *arg, struct sample_conv *conv,
+ const char *file, int line, char **err)
+{
+ struct map_descriptor *desc = NULL;
+ const int dfl = 2;
+
+ if (!(global.mode & MODE_STARTING)) {
+ memprintf(err, "map: cannot load map at runtime");
+ goto fail;
+ }
+
+ /* create new map descriptor */
+ desc = map_create_descriptor(conv);
+ if (!desc) {
+ memprintf(err, "out of memory");
+ goto fail;
+ }
+
+ /* Initialize pattern */
+ pattern_init_head(&desc->pat);
+
+ /* the condition may be a log-format expression, prepare its holder */
+ lf_expr_init(&desc->cond_fmt);
+
+ /* This is original pattern, must free */
+ desc->do_free = 1;
+
+ /* Set the match method. A conditional map spreads its patterns over
+ * per-condition groups, which only its own prune function knows about.
+ * Everything else is indexed and matched exactly like in a plain map,
+ * within the group selected by the condition.
+ */
+ desc->pat.match = pat_match_fcts[(long)conv->private];
+ desc->pat.parse = pat_parse_fcts[(long)conv->private];
+ desc->pat.index = pat_index_fcts[(long)conv->private];
+ desc->pat.prune = pat_prune_cond;
+ desc->pat.expect_type = pat_match_types[(long)conv->private];
+
+ /* Set the output parse method. */
+ switch (desc->conv->out_type) {
+ case SMP_T_STR: desc->pat.parse_smp = map_parse_str; break;
+ case SMP_T_SINT: desc->pat.parse_smp = map_parse_int; break;
+ case SMP_T_ADDR: desc->pat.parse_smp = map_parse_ip; break;
+ default:
+ memprintf(err, "map: internal haproxy error: no default parse
case for the input type <%d>.",
+ conv->out_type);
+ goto fail;
+ }
+
+ /* The condition is built at run time. It is most often a plain variable
+ * name, which is then read directly, but anything else is compiled as a
+ * log-format expression so that it may combine literals, sample fetches
+ * and converters.
+ */
+ if (!vars_check_arg(&arg[1], NULL)) {
+ int cap = 0;
+
+ ha_free(err);
+ if (!curproxy) {
+ memprintf(err, "map: '%s' is not a valid variable name,
and a log-format "
+ "condition cannot be built outside of a
proxy section.",
+ arg[1].data.str.area);
+ goto fail;
+ }
+
+ if (curproxy->cap & PR_CAP_FE)
+ cap |= SMP_VAL_FE_HRQ_HDR;
+ if (curproxy->cap & PR_CAP_BE)
+ cap |= SMP_VAL_BE_HRQ_HDR;
+
+ if (!parse_logformat_string(arg[1].data.str.area, curproxy,
&desc->cond_fmt,
+ 0, cap, err))
+ goto fail;
+
+ chunk_destroy(&arg[1].data.str);
+ arg[1].type = ARGT_STOP;
+ }
+
+ /* Load map. */
+ if (!pattern_read_from_file(&desc->pat, PAT_REF_MAP | PAT_REF_COND,
+ arg[0].data.str.area, PAT_MF_NO_DNS,
+ 1, err, file, line))
+ goto fail;
+
+ /* the maps of type IP support a string as default value. This
+ * string can be an ipv4 or an ipv6, we must convert it.
+ */
+ if (arg[dfl].type != ARGT_STOP && desc->conv->out_type == SMP_T_ADDR) {
+ struct sample_data data;
+ if (!map_parse_ip(arg[dfl].data.str.area, &data)) {
+ memprintf(err, "map: cannot parse default ip <%s>.",
+ arg[dfl].data.str.area);
+ goto fail;
+ }
+ chunk_destroy(&arg[dfl].data.str);
+ if (data.type == SMP_T_IPV4) {
+ arg[dfl].type = ARGT_IPV4;
+ arg[dfl].data.ipv4 = data.u.ipv4;
+ } else {
+ arg[dfl].type = ARGT_IPV6;
+ arg[dfl].data.ipv6 = data.u.ipv6;
+ }
+ }
+
+ /* replace the first argument by this definition */
+ chunk_destroy(&arg[0].data.str);
+ arg[0].type = ARGT_MAP;
+ arg[0].data.map = desc;
+
+ return 1;
+ fail:
+ if (desc) {
+ lf_expr_deinit(&desc->cond_fmt);
+ free(desc);
+ }
+ return 0;
+}
+
+
+/* Runs the lookup of a conditional map: resolves the condition from the
+ * variable passed in second argument, then matches the input sample against
+ * the sole group of entries holding that condition. Returns the matched
+ * pattern, or NULL if the variable is unusable or nothing matched.
+ */
+static struct pattern *cmap_exec_match(const struct arg *arg_p, struct sample
*smp)
+{
+ struct map_descriptor *desc = arg_p[0].data.map;
+ struct sample cond;
+
+ if (arg_p[1].type != ARGT_VAR) {
+ /* the condition is a log-format expression. It is built into a
+ * local buffer since it may not be longer than a condition key
+ * anyway, which also keeps it away from the trash chunks that
+ * the lookup below is free to use.
+ */
+ char key[PAT_COND_KEY_MAXLEN + 2];
+ struct buffer *input = NULL;
+ struct pattern *pat;
+ int len;
+
+ /* sess_build_logline() dereferences the session right away. A
+ * "set-var" in the global section is evaluated without one, and
+ * curproxy is not reset when leaving a proxy section, so the
+ * load-time check above lets such a configuration through.
+ */
+ if (!smp->sess)
+ return NULL;
+
+ /* sess_build_logline() evaluates its samples through the
+ * rotating trash chunks, and our own input may well be sitting
+ * in one of them (anything coming out of a converter chain
+ * does). Building a condition that consumes trash more than
+ * once would then overwrite the very sample we are about to
+ * look up, silently returning the default value. Keep the
+ * input in a chunk of our own across the build. Non-string
+ * types are held by value in the union and need no copy.
+ */
+ if (smp->data.type == SMP_T_STR || smp->data.type == SMP_T_BIN)
{
+ input = alloc_trash_chunk();
+ if (!input)
+ return NULL;
+ if (!chunk_memcat(input, smp->data.u.str.area,
smp->data.u.str.data)) {
+ free_trash_chunk(input);
+ return NULL;
+ }
+ }
+
+ /* one extra byte so that an over-long condition is detected
+ * rather than silently truncated to a valid-looking key.
+ */
+ len = sess_build_logline(smp->sess, smp->strm, key,
sizeof(key), &desc->cond_fmt);
+
+ if (input) {
+ smp->data.u.str.area = input->area;
+ smp->data.u.str.data = input->data;
+ smp->data.u.str.size = input->size;
+ smp->flags &= ~SMP_F_CONST;
+ }
+
+ if (len <= 0 || len > PAT_COND_KEY_MAXLEN)
+ pat = NULL;
+ else
+ pat = pattern_exec_match_cond(&desc->pat, smp, 1,
ist2(key, len));
+
+ free_trash_chunk(input);
+ return pat;
+ }
+
+ smp_set_owner(&cond, smp->px, smp->sess, smp->strm, smp->opt);
+ if (!vars_get_by_desc(&arg_p[1].data.var, &cond, NULL) ||
+ !sample_convert(&cond, SMP_T_STR))
+ return NULL;
+
+ return pattern_exec_match_cond(&desc->pat, smp, 1,
+ ist2(cond.data.u.str.area,
cond.data.u.str.data));
+}
+
/* try to match input sample against map entries, returns matched entry's key
* on success
*/
@@ -197,6 +402,26 @@ static int sample_conv_map_key(const struct arg *arg_p,
struct sample *smp, void
return 0;
}
+/* same as above for a conditional map. The condition is stripped from the
+ * returned key, as the caller already knows it.
+ */
+static int sample_conv_cmap_key(const struct arg *arg_p, struct sample *smp,
void *private)
+{
+ const char *key;
+ struct pattern *pat;
+
+ pat = cmap_exec_match(arg_p, smp);
+ if (!pat)
+ return 0;
+
+ pat_cond_split(pat->ref->pattern, &key);
+ smp->data.type = SMP_T_STR;
+ smp->flags |= SMP_F_CONST;
+ smp->data.u.str.area = (char *)key;
+ smp->data.u.str.data = strlen(key);
+ return 1;
+}
+
/* try to match input sample against map entries, returns matched entry's value
* on success
*/
@@ -291,6 +516,101 @@ static int sample_conv_map(const struct arg *arg_p,
struct sample *smp, void *pr
return 1;
}
+/* try to match input sample against the entries of a conditional map,
+ * returns matched entry's value on success
+ */
+static int sample_conv_cmap(const struct arg *arg_p, struct sample *smp, void
*private)
+{
+ struct map_descriptor *desc;
+ struct pattern *pat;
+ struct buffer *str;
+
+ /* get config */
+ desc = arg_p[0].data.map;
+
+ /* Execute the match function. */
+ pat = cmap_exec_match(arg_p, smp);
+
+ /* Match case. */
+ if (pat) {
+ if (pat->data) {
+ /* In the regm case, merge the sample with the input. */
+ if ((long)private == PAT_MATCH_REGM) {
+ struct buffer *tmptrash;
+ int len;
+
+ /* Copy the content of the sample because it
could
+ be scratched by incoming get_trash_chunk */
+ tmptrash = alloc_trash_chunk();
+ if (!tmptrash)
+ return 0;
+
+ tmptrash->data = smp->data.u.str.data;
+ if (tmptrash->data > (tmptrash->size-1))
+ tmptrash->data = tmptrash->size-1;
+
+ memcpy(tmptrash->area, smp->data.u.str.area,
tmptrash->data);
+ tmptrash->area[tmptrash->data] = 0;
+
+ str = get_trash_chunk();
+ len = exp_replace(str->area, str->size,
+ tmptrash->area,
+ pat->data->u.str.area,
+ (regmatch_t *)smp->ctx.a[0]);
+ free_trash_chunk(tmptrash);
+
+ if (len == -1)
+ return 0;
+
+ str->data = len;
+ smp->data.u.str = *str;
+ return 1;
+ }
+ /* Copy sample. */
+ smp->data = *pat->data;
+ smp->flags |= SMP_F_CONST;
+ return 1;
+ }
+
+ /* Return just int sample containing 1. */
+ smp->data.type = SMP_T_SINT;
+ smp->data.u.sint = 1;
+ return 1;
+ }
+
+ /* If no default value available, the converter fails. */
+ if (arg_p[2].type == ARGT_STOP)
+ return 0;
+
+ /* Return the default value. */
+ switch (desc->conv->out_type) {
+
+ case SMP_T_STR:
+ smp->data.type = SMP_T_STR;
+ smp->flags |= SMP_F_CONST;
+ smp->data.u.str = arg_p[2].data.str;
+ break;
+
+ case SMP_T_SINT:
+ smp->data.type = SMP_T_SINT;
+ smp->data.u.sint = arg_p[2].data.sint;
+ break;
+
+ case SMP_T_ADDR:
+ if (arg_p[2].type == ARGT_IPV4) {
+ smp->data.type = SMP_T_IPV4;
+ smp->data.u.ipv4 = arg_p[2].data.ipv4;
+ } else {
+ smp->data.type = SMP_T_IPV6;
+ smp->data.u.ipv6 = arg_p[2].data.ipv6;
+ }
+ break;
+ }
+
+ return 1;
+}
+
+
/* This function is used with map and acl management. It permits to browse
* each reference. The variable <getnext> must contain the current node,
* <end> point to the root node and the <flags> permit to filter required
@@ -492,8 +812,10 @@ static int cli_io_handler_pats_list(struct appctx *appctx)
static int cli_io_handler_map_lookup(struct appctx *appctx)
{
struct show_map_ctx *ctx = appctx->svcctx;
+ struct pattern_expr *expr;
struct sample sample;
struct pattern *pat;
+ struct ist cond;
int match_method;
switch (ctx->state) {
@@ -517,9 +839,25 @@ static int cli_io_handler_map_lookup(struct appctx *appctx)
sample.data.u.str.data = ctx->chunk.data;
sample.data.u.str.area = ctx->chunk.area;
- if (ctx->expr->pat_head->match &&
- sample_convert(&sample,
ctx->expr->pat_head->expect_type))
- pat = ctx->expr->pat_head->match(&sample,
ctx->expr, 1);
+ /* on a conditional map the looked up key holds the
+ * condition followed by the pattern, just like in the
+ * map file. Only the entries of the group holding that
+ * condition are looked up.
+ */
+ expr = ctx->expr;
+ cond = IST_NULL;
+ if (ctx->ref->flags & PAT_REF_COND) {
+ const char *pattern;
+
+ cond = pat_cond_split(ctx->chunk.area,
&pattern);
+ sample.data.u.str.area = (char *)pattern;
+ sample.data.u.str.data = ctx->chunk.data -
(pattern - ctx->chunk.area);
+ expr = pattern_cond_grp_expr(ctx->expr, cond);
+ }
+
+ if (expr && expr->pat_head->match &&
+ sample_convert(&sample,
expr->pat_head->expect_type))
+ pat = expr->pat_head->match(&sample, expr, 1);
else
pat = NULL;
@@ -532,6 +870,16 @@ static int cli_io_handler_map_lookup(struct appctx *appctx)
else
chunk_appendf(&trash, "type=%s",
pat_match_names[match_method]);
+ /* conditional map: report the condition that was looked
+ * up, and whether the map holds any entry for it. This
+ * tells an unknown condition apart from a condition
+ * whose entries just didn't match.
+ */
+ if (isttest(cond))
+ chunk_appendf(&trash, ", cond=\"%.*s\",
cond_found=%s",
+ (int)istlen(cond), istptr(cond),
+ expr ? "yes" : "no");
+
/* case sensitive */
if (ctx->expr->mflags & PAT_MF_IGNORE_CASE)
chunk_appendf(&trash, ", case=insensitive");
@@ -616,6 +964,41 @@ static void cli_release_mlook(struct appctx *appctx)
}
+/* On a conditional map the key is made of two space-separated fields, the
+ * condition and the pattern. The CLI splits its input on spaces, so on the
+ * command line these arrive as two separate arguments. Join them into a
+ * freshly allocated string returned in <buf>, which the caller must free.
+ * Returns NULL if <pattern> is missing or on allocation failure.
+ */
+static char *cli_cond_key(const char *cond, const char *pattern, char **buf)
+{
+ if (!*pattern)
+ return NULL;
+ return memprintf(buf, "%s %s", cond, pattern);
+}
+
+/* Joins the arguments starting at <args> into <buf>, separated by a single
+ * space, and returns it. The map file and the payload syntax both take the
+ * value as whatever is left on the line, so the command line has to take it as
+ * whatever is left in the arguments rather than silently keep only the first
+ * one. The caller must free <buf>. Returns NULL if there is no argument left
+ * or on allocation failure.
+ */
+static char *cli_join_args(char **args, char **buf)
+{
+ int i;
+
+ if (!*args[0])
+ return NULL;
+ if (!memprintf(buf, "%s", args[0]))
+ return NULL;
+ for (i = 1; *args[i]; i++) {
+ if (!memprintf(buf, "%s %s", *buf, args[i]))
+ return NULL;
+ }
+ return *buf;
+}
+
static int cli_parse_get_map(char **args, char *payload, struct appctx
*appctx, void *private)
{
struct show_map_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
@@ -624,6 +1007,9 @@ static int cli_parse_get_map(char **args, char *payload,
struct appctx *appctx,
ha_warning("'%s %s' accessed without admin rights, this won't
be supported anymore starting from haproxy 3.3\n", args[0], args[1]);
if (strcmp(args[1], "map") == 0 || strcmp(args[1], "acl") == 0) {
+ char *ckey = NULL;
+ const char *key = args[3];
+
/* Set flags. */
if (args[1][0] == 'm')
ctx->display_flags = PAT_REF_MAP;
@@ -647,13 +1033,22 @@ static int cli_parse_get_map(char **args, char *payload,
struct appctx *appctx,
return cli_err(appctx, "Unknown ACL identifier.
Please use #<id> or <file>.\n");
}
+ /* on a conditional map the condition comes as its own argument
*/
+ if (ctx->ref->flags & PAT_REF_COND) {
+ if (!cli_cond_key(args[3], args[4], &ckey))
+ return cli_err(appctx, "This command expects
three parameters on a conditional map: "
+ "identifier, condition
and sample.\n");
+ key = ckey;
+ }
+
/* copy input string. The string must be allocated because
* it may be used over multiple iterations. It's released
* at the end and upon abort anyway.
*/
- ctx->chunk.data = strlen(args[3]);
+ ctx->chunk.data = strlen(key);
ctx->chunk.size = ctx->chunk.data + 1;
- ctx->chunk.area = strdup(args[3]);
+ ctx->chunk.area = strdup(key);
+ ha_free(&ckey);
if (!ctx->chunk.area)
return cli_err(appctx, "Out of memory error.\n");
@@ -772,7 +1167,12 @@ static int cli_parse_set_map(char **args, char *payload,
struct appctx *appctx,
ha_warning("'%s %s' accessed without admin rights, this won't
be supported anymore starting from haproxy 3.3\n", args[0], args[1]);
if (strcmp(args[1], "map") == 0) {
+ char *ckey = NULL;
+ char *cval = NULL;
+ char *key = args[3];
+ char *value = args[4];
char *err;
+ int ret;
/* Set flags. */
ctx->display_flags = PAT_REF_MAP;
@@ -786,6 +1186,24 @@ static int cli_parse_set_map(char **args, char *payload,
struct appctx *appctx,
if (!ctx->ref)
return cli_err(appctx, "Unknown map identifier. Please
use #<id> or <file>.\n");
+ /* On a conditional map the key is made of two fields which the
+ * CLI delivers as two arguments, so the new value sits one
+ * position further. The '#<id>' form addresses the entry
+ * directly and keeps the usual layout.
+ */
+ if ((ctx->ref->flags & PAT_REF_COND) &&
+ !(args[3][0] == '#' && args[3][1] == '0' && args[3][2] ==
'x')) {
+ if (!cli_cond_key(args[3], args[4], &ckey) ||
+ !cli_join_args(&args[5], &cval)) {
+ ha_free(&ckey);
+ ha_free(&cval);
+ return cli_err(appctx, "'set map' expects four
parameters on a conditional map: "
+ "map identifier,
condition, pattern and value.\n");
+ }
+ key = ckey;
+ value = cval;
+ }
+
/* If the entry identifier start with a '#', it is considered as
* pointer id
*/
@@ -807,14 +1225,14 @@ static int cli_parse_set_map(char **args, char *payload,
struct appctx *appctx,
/* Try to modify the entry. */
err = NULL;
HA_RWLOCK_WRLOCK(PATREF_LOCK, &ctx->ref->lock);
- if (!pat_ref_set_by_id(ctx->ref, ref, args[4], &err)) {
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK,
&ctx->ref->lock);
+ ret = pat_ref_set_by_id(ctx->ref, ref, value, &err);
+ HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
+ if (!ret) {
if (err)
return cli_dynerr(appctx,
memprintf(&err, "%s.\n", err));
else
return cli_err(appctx, "Failed to
update an entry.\n");
}
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
}
else {
/* Else, use the entry identifier as pattern
@@ -822,14 +1240,16 @@ static int cli_parse_set_map(char **args, char *payload,
struct appctx *appctx,
*/
err = NULL;
HA_RWLOCK_WRLOCK(PATREF_LOCK, &ctx->ref->lock);
- if (!pat_ref_set(ctx->ref, args[3], args[4], &err)) {
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK,
&ctx->ref->lock);
+ ret = pat_ref_set(ctx->ref, key, value, &err);
+ HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
+ ha_free(&ckey);
+ ha_free(&cval);
+ if (!ret) {
if (err)
return cli_dynerr(appctx,
memprintf(&err, "%s.\n", err));
else
return cli_err(appctx, "Failed to
update an entry.\n");
}
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
}
/* The set is done, send message. */
@@ -850,6 +1270,8 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
strcmp(args[1], "acl") == 0) {
const char *gen = NULL;
uint genid = 0;
+ char *cond_key = NULL;
+ char *cond_val = NULL;
int ret;
char *err;
@@ -912,6 +1334,24 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
"You must use the command 'add map' to
add values.\n");
}
+ /* On a conditional map the key is made of two fields which the
+ * CLI delivers as two arguments, so the value sits one position
+ * further. In payload mode the lines already carry the file
+ * format and are joined further down.
+ */
+ if (!payload && (ctx->ref->flags & PAT_REF_COND)) {
+ if (!cli_cond_key(args[3], args[4], &cond_key) ||
+ (ctx->display_flags == PAT_REF_MAP &&
+ !cli_join_args(&args[5], &cond_val))) {
+ ha_free(&cond_key);
+ ha_free(&cond_val);
+ return cli_err(appctx,
+ "On a conditional map this
command expects four parameters"
+ " (map identifier, condition,
pattern and value)"
+ " or one parameter (map
identifier) and a payload\n");
+ }
+ }
+
/* Add value(s). If no payload is used, key and value are read
* from the command line and only one key is set. If a payload
* is passed, one key/value pair is read per line till the end
@@ -920,8 +1360,8 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
err = NULL;
do {
- char *key = args[3];
- char *value = args[4];
+ char *key = cond_key ? cond_key : args[3];
+ char *value = cond_key ? cond_val : args[4];
size_t l;
if (payload) {
@@ -939,6 +1379,30 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
key[l] = 0;
payload++;
+ /* a conditional map's key is made of two fields
+ * which are stored space-separated. Bring them
+ * next to each other in place.
+ */
+ if (ctx->ref->flags & PAT_REF_COND) {
+ char *pat;
+ size_t l2;
+
+ payload += strspn(payload, " \t");
+ pat = payload;
+ l2 = strcspn(pat, " \t");
+ payload += l2;
+
+ if (!*payload)
+ return cli_dynerr(appctx,
memprintf(&err, "Missing value for key '%s %.*s'.\n",
+
key, (int)l2, pat));
+ payload++;
+
+ key[l] = ' ';
+ if (pat != key + l + 1)
+ memmove(key + l + 1, pat, l2);
+ key[l + 1 + l2] = 0;
+ }
+
/* value */
payload += strspn(payload, " \t");
value = payload;
@@ -955,6 +1419,8 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
HA_RWLOCK_WRLOCK(PATREF_LOCK, &ctx->ref->lock);
ret = !!pat_ref_load(ctx->ref, gen ? genid :
ctx->ref->curr_gen, key, value, -1, &err);
HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
+ ha_free(&cond_key);
+ ha_free(&cond_val);
if (!ret) {
if (err)
@@ -975,6 +1441,9 @@ static int cli_parse_add_map(char **args, char *payload,
struct appctx *appctx,
static int cli_parse_del_map(char **args, char *payload, struct appctx
*appctx, void *private)
{
struct show_map_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
+ char *ckey = NULL;
+ char *key = args[3];
+ int ret;
if ((appctx->cli_ctx.level & ACCESS_LVL_MASK) < ACCESS_LVL_ADMIN)
ha_warning("'%s %s' accessed without admin rights, this won't
be supported anymore starting from haproxy 3.3\n", args[0], args[1]);
@@ -1002,6 +1471,20 @@ static int cli_parse_del_map(char **args, char *payload,
struct appctx *appctx,
return cli_err(appctx, "Unknown ACL identifier. Please
use #<id> or <file>.\n");
}
+ /* On a conditional map the key is made of two fields which the CLI
+ * delivers as two arguments. The '#<id>' form addresses the entry
+ * directly and keeps the usual layout.
+ */
+ if ((ctx->ref->flags & PAT_REF_COND) &&
+ !(args[3][0] == '#' && args[3][1] == '0' && args[3][2] == 'x')) {
+ if (!cli_cond_key(args[3], args[4], &ckey)) {
+ ha_free(&ckey);
+ return cli_err(appctx, "On a conditional map this
command expects three parameters: "
+ "identifier, condition and
pattern.\n");
+ }
+ key = ckey;
+ }
+
/* If the entry identifier start with a '#', it is considered as
* pointer id
*/
@@ -1034,12 +1517,13 @@ static int cli_parse_del_map(char **args, char
*payload, struct appctx *appctx,
* string and try to delete the entry.
*/
HA_RWLOCK_WRLOCK(PATREF_LOCK, &ctx->ref->lock);
- if (!pat_ref_delete(ctx->ref, args[3])) {
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
+ ret = pat_ref_delete(ctx->ref, key);
+ HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
+ ha_free(&ckey);
+ if (!ret) {
/* The entry is not found, send message. */
return cli_err(appctx, "Key not found.\n");
}
- HA_RWLOCK_WRUNLOCK(PATREF_LOCK, &ctx->ref->lock);
}
/* The deletion is done, send message. */
@@ -1280,6 +1764,55 @@ static struct sample_conv_kw_list sample_conv_kws =
{ILH, {
{ "map_int_key", sample_conv_map_key, ARG1(1,STR), sample_load_map,
SMP_T_SINT, SMP_T_STR, (void *)PAT_MATCH_INT },
{ "map_ip_key", sample_conv_map_key, ARG1(1,STR), sample_load_map,
SMP_T_ADDR, SMP_T_STR, (void *)PAT_MATCH_IP },
+ /* Conditional maps. Same as above, except that each entry holds one
+ * extra leading key which is exactly matched against the condition
+ * read from the variable passed in second argument. Only the entries
+ * whose condition matches take part in the lookup.
+ *
+ * The arguments are: <file>,<cond var>[,<default value>]
+ */
+ { "cmap", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_STR },
+ { "cmap_str", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_STR },
+ { "cmap_beg", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_BEG },
+ { "cmap_sub", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_SUB },
+ { "cmap_dir", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_DIR },
+ { "cmap_dom", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_DOM },
+ { "cmap_end", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_END },
+ { "cmap_reg", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_REG },
+ { "cmap_regm", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_REGM },
+ { "cmap_int", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_SINT, SMP_T_STR , (void *)PAT_MATCH_INT },
+ { "cmap_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_ADDR, SMP_T_STR , (void *)PAT_MATCH_IP },
+
+ { "cmap_str_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_STR },
+ { "cmap_beg_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_BEG },
+ { "cmap_sub_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_SUB },
+ { "cmap_dir_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_DIR },
+ { "cmap_dom_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_DOM },
+ { "cmap_end_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_END },
+ { "cmap_reg_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_STR , SMP_T_SINT, (void *)PAT_MATCH_REG },
+ { "cmap_int_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_SINT, SMP_T_SINT, (void *)PAT_MATCH_INT },
+ { "cmap_ip_int", sample_conv_cmap, ARG3(2,STR,STR,SINT),
sample_load_cmap, SMP_T_ADDR, SMP_T_SINT, (void *)PAT_MATCH_IP },
+
+ { "cmap_str_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_STR },
+ { "cmap_beg_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_BEG },
+ { "cmap_sub_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_SUB },
+ { "cmap_dir_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_DIR },
+ { "cmap_dom_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_DOM },
+ { "cmap_end_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_END },
+ { "cmap_reg_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_ADDR, (void *)PAT_MATCH_REG },
+ { "cmap_int_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_SINT, SMP_T_ADDR, (void *)PAT_MATCH_INT },
+ { "cmap_ip_ip", sample_conv_cmap, ARG3(2,STR,STR,STR),
sample_load_cmap, SMP_T_ADDR, SMP_T_ADDR, (void *)PAT_MATCH_IP },
+
+ { "cmap_str_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_STR },
+ { "cmap_beg_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_BEG },
+ { "cmap_sub_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_SUB },
+ { "cmap_dir_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_DIR },
+ { "cmap_dom_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_DOM },
+ { "cmap_end_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_END },
+ { "cmap_reg_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_STR , SMP_T_STR , (void *)PAT_MATCH_REG },
+ { "cmap_int_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_SINT, SMP_T_STR , (void *)PAT_MATCH_INT },
+ { "cmap_ip_key", sample_conv_cmap_key, ARG2(2,STR,STR),
sample_load_cmap, SMP_T_ADDR, SMP_T_STR , (void *)PAT_MATCH_IP },
+
{ /* END */ },
}};
diff --git a/src/pattern.c b/src/pattern.c
index 4896a69..71459fc 100644
--- a/src/pattern.c
+++ b/src/pattern.c
@@ -1127,6 +1127,116 @@ void pat_prune_gen(struct pattern_expr *expr)
expr->ref->entry_cnt = 0;
}
+/*
+ *
+ * The following functions handle the condition groups of a conditional map.
+ * The groups are indexed on their condition key in the pattern_tree of the
+ * top-level expression, and each of them owns a sub-expression which is filled
+ * and looked up using the stock functions of the underlying match method.
+ *
+ */
+
+/* Returns the condition group of <expr> whose key exactly matches <cond>, or
+ * NULL if there is none. The caller must hold at least a read lock on <expr>.
+ */
+static struct pattern_cond_grp *pat_cond_grp_lookup(struct pattern_expr *expr,
struct ist cond)
+{
+ struct ebmb_node *node;
+
+ node = ebst_lookup_len(&expr->pattern_tree, istptr(cond), istlen(cond));
+ if (!node)
+ return NULL;
+ return ebmb_entry(node, struct pattern_cond_grp, node);
+}
+
+struct pattern_expr *pattern_cond_grp_expr(struct pattern_expr *expr, struct
ist cond)
+{
+ struct pattern_cond_grp *grp;
+
+ grp = pat_cond_grp_lookup(expr, cond);
+ return grp ? &grp->expr : NULL;
+}
+
+/* Returns the condition group of <expr> whose key exactly matches <cond>,
+ * creating it if it doesn't exist yet. Returns NULL on allocation error. The
+ * caller must hold a write lock on <expr>.
+ */
+static struct pattern_cond_grp *pat_cond_grp_get(struct pattern_expr *expr,
struct ist cond)
+{
+ struct pattern_cond_grp *grp;
+
+ grp = pat_cond_grp_lookup(expr, cond);
+ if (grp)
+ return grp;
+
+ grp = calloc(1, sizeof(*grp) + istlen(cond) + 1);
+ if (!grp)
+ return NULL;
+
+ /* the sub-expression is fed with the same match method and the same
+ * reference as the top-level one, it only holds a distinct set of
+ * patterns.
+ */
+ pattern_init_expr(&grp->expr);
+ LIST_INIT(&grp->expr.list);
+ grp->expr.ref = expr->ref;
+ grp->expr.pat_head = expr->pat_head;
+ grp->expr.mflags = expr->mflags;
+ HA_RWLOCK_INIT(&grp->expr.lock);
+
+ memcpy(grp->node.key, istptr(cond), istlen(cond));
+ grp->node.key[istlen(cond)] = 0;
+ grp->node.node.leaf_p = NULL;
+ ebst_insert(&expr->pattern_tree, &grp->node);
+
+ return grp;
+}
+
+void pat_prune_cond(struct pattern_expr *expr)
+{
+ struct eb_node *node, *next;
+ struct pattern_cond_grp *grp;
+
+ node = eb_first(&expr->pattern_tree);
+ while (node) {
+ next = eb_next(node);
+ eb_delete(node);
+ grp = container_of(node, struct pattern_cond_grp, node);
+ pat_prune_gen(&grp->expr);
+ free(grp);
+ node = next;
+ }
+ expr->pattern_tree = EB_ROOT;
+ LIST_INIT(&expr->patterns);
+ expr->ref->revision = rdtsc();
+ expr->ref->entry_cnt = 0;
+}
+
+/* Drops one pattern from the condition group that owns <iexpr>, and releases
+ * the group once it holds nothing anymore. Groups are created on demand by
+ * pat_ref_push() and are only reclaimed here and in pat_prune_cond(), which
the
+ * map path never reaches, so without this a group would survive every one of
+ * its entries. Does nothing on a plain map. The caller must hold the write
lock
+ * on the top-level expression, which is what protects the group tree.
+ */
+static void pat_cond_grp_release(struct pat_ref *ref, struct pattern_expr
*iexpr)
+{
+ struct pattern_cond_grp *grp;
+
+ if (!(ref->flags & PAT_REF_COND) || !iexpr)
+ return;
+
+ /* on a conditional map every pattern is indexed in a group, whose
+ * sub-expression is its first member.
+ */
+ grp = container_of(iexpr, struct pattern_cond_grp, expr);
+ if (!grp->entries || --grp->entries)
+ return;
+
+ ebmb_delete(&grp->node);
+ free(grp);
+}
+
/*
*
* The following functions are used for the pattern indexation
@@ -1463,6 +1573,10 @@ void pat_delete_gen(struct pat_ref *ref, struct
pat_ref_elt *elt)
BUG_ON(tree->ref != elt);
ebmb_delete(&tree->node);
+ /* must come after the node is unlinked, it may free the group
+ * this tree root lives in.
+ */
+ pat_cond_grp_release(ref, tree->expr);
free(tree->data);
free(tree);
}
@@ -1475,6 +1589,7 @@ void pat_delete_gen(struct pat_ref *ref, struct
pat_ref_elt *elt)
/* Delete and free entry. */
LIST_DELETE(&pat->list);
+ pat_cond_grp_release(ref, pat->expr);
if (pat->pat.sflags & PAT_SF_REGFREE)
regex_free(pat->pat.ptr.reg);
else
@@ -2019,9 +2134,28 @@ struct pat_ref_elt *pat_ref_append(struct pat_ref *ref,
const char *pattern, con
int pat_ref_push(struct pat_ref_elt *elt, struct pattern_expr *expr,
int patflags, char **err)
{
+ struct pattern_expr *iexpr = expr;
+ struct pattern_cond_grp *grp = NULL;
+ const char *text = elt->pattern;
+ struct ist cond = IST_NULL;
struct sample_data *data;
struct pattern pattern;
+ /* In a conditional map the key holds two fields: the condition, which
+ * only selects the group the pattern is indexed in, and the pattern
+ * itself, which is the only part the match method knows about.
+ */
+ if (expr->ref && (expr->ref->flags & PAT_REF_COND)) {
+ cond = pat_cond_split(elt->pattern, &text);
+ if (!istlen(cond) || istlen(cond) > PAT_COND_KEY_MAXLEN ||
!*text) {
+ memprintf(err, "'%s' is not a valid conditional key, it
must hold a "
+ "condition of 1 to %d characters and a
pattern, "
+ "separated by a space",
+ elt->pattern, PAT_COND_KEY_MAXLEN);
+ return 0;
+ }
+ }
+
/* Create sample */
if (elt->sample && expr->pat_head->parse_smp) {
/* New sample. */
@@ -2046,18 +2180,38 @@ int pat_ref_push(struct pat_ref_elt *elt, struct
pattern_expr *expr,
pattern.ref = elt;
/* parse pattern */
- if (!expr->pat_head->parse(elt->pattern, &pattern, expr->mflags, err)) {
+ if (!expr->pat_head->parse(text, &pattern, expr->mflags, err)) {
free(data);
return 0;
}
HA_RWLOCK_WRLOCK(PATEXP_LOCK, &expr->lock);
+ /* pick the condition group the pattern must be indexed in */
+ if (isttest(cond)) {
+ grp = pat_cond_grp_get(expr, cond);
+ if (!grp) {
+ HA_RWLOCK_WRUNLOCK(PATEXP_LOCK, &expr->lock);
+ memprintf(err, "out of memory");
+ free(data);
+ return 0;
+ }
+ iexpr = &grp->expr;
+ }
/* index pattern */
- if (!expr->pat_head->index(expr, &pattern, err)) {
+ if (!expr->pat_head->index(iexpr, &pattern, err)) {
+ /* a group created just above for this pattern would otherwise
+ * stay behind empty. One that already held entries must stay.
+ */
+ if (grp && !grp->entries) {
+ ebmb_delete(&grp->node);
+ free(grp);
+ }
HA_RWLOCK_WRUNLOCK(PATEXP_LOCK, &expr->lock);
free(data);
return 0;
}
+ if (grp)
+ grp->entries++;
HA_RWLOCK_WRUNLOCK(PATEXP_LOCK, &expr->lock);
return 1;
@@ -2331,6 +2485,8 @@ int pat_ref_read_from_file_smp(struct pat_ref *ref, char
**err)
char *c;
int ret = 0;
int line = 0;
+ char *cond_beg = NULL;
+ char *cond_end = NULL;
char *key_beg;
char *key_end;
char *value_beg;
@@ -2351,6 +2507,9 @@ int pat_ref_read_from_file_smp(struct pat_ref *ref, char
**err)
/* now parse all patterns. The file may contain only one pattern
* followed by one value per line. The start spaces, separator spaces
* and and spaces are stripped. Each can contain comment started by '#'
+ * A conditional map has one extra field at the beginning of the line:
+ * the condition, which is stored space-separated in front of the
+ * pattern so that both form a single key.
*/
while (fgets(trash.area, trash.size, file) != NULL) {
line++;
@@ -2379,6 +2538,36 @@ int pat_ref_read_from_file_smp(struct pat_ref *ref, char
**err)
while (*c == ' ' || *c == '\t')
c++;
+ if (ref->flags & PAT_REF_COND) {
+ /* what was read above is the condition, the pattern
+ * comes next.
+ */
+ cond_beg = key_beg;
+ cond_end = key_end;
+
+ if (cond_end - cond_beg > PAT_COND_KEY_MAXLEN) {
+ memprintf(err, "condition too long (%d chars
max) at line %d of file <%s>",
+ PAT_COND_KEY_MAXLEN, line,
ref->reference);
+ goto out_close;
+ }
+
+ key_beg = c;
+ while (*c && *c != ' ' && *c != '\t' && *c != '\n' &&
*c != '\r')
+ c++;
+
+ key_end = c;
+
+ if (key_beg == key_end) {
+ memprintf(err, "missing pattern after condition
'%.*s' at line %d of file <%s>",
+ (int)(cond_end - cond_beg), cond_beg,
line, ref->reference);
+ goto out_close;
+ }
+
+ /* strip middle spaces and tabs */
+ while (*c == ' ' || *c == '\t')
+ c++;
+ }
+
/* look for the end of the value, it is the end of the line */
value_beg = c;
while (*c && *c != '\n' && *c != '\r')
@@ -2393,6 +2582,17 @@ int pat_ref_read_from_file_smp(struct pat_ref *ref, char
**err)
*key_end = '\0';
*value_end = '\0';
+ /* join the condition and the pattern into a single key made of
+ * both fields separated by exactly one space.
+ */
+ if (ref->flags & PAT_REF_COND) {
+ size_t cond_len = cond_end - cond_beg;
+
+ memmove(cond_beg + cond_len + 1, key_beg, key_end -
key_beg + 1);
+ cond_beg[cond_len] = ' ';
+ key_beg = cond_beg;
+ }
+
/* insert values */
if (!pat_ref_append(ref, key_beg, value_beg, line)) {
memprintf(err, "out of memory");
@@ -2495,7 +2695,9 @@ int pattern_read_from_file(struct pattern_head *head,
unsigned int refflags,
if (!ref) {
chunk_printf(&trash,
"pattern loaded from file '%s' used by %s at file
'%s' line %d",
- filename, refflags & PAT_REF_MAP ? "map" : "acl",
file, line);
+ filename,
+ refflags & PAT_REF_COND ? "conditional map" :
+ refflags & PAT_REF_MAP ? "map" : "acl", file,
line);
ref = pat_ref_new(filename, trash.area, refflags);
if (!ref) {
@@ -2526,6 +2728,18 @@ int pattern_read_from_file(struct pattern_head *head,
unsigned int refflags,
else {
/* The reference already exists, check the map compatibility. */
+ /* A conditional map stores two keys per entry and cannot share
+ * its reference with a plain map or with an ACL.
+ */
+ if ((ref->flags ^ refflags) & PAT_REF_COND) {
+ memprintf(err, "The file \"%s\" is already used as a
%sconditional pattern file "
+ "and cannot be used as a %sconditional
one.",
+ filename,
+ (ref->flags & PAT_REF_COND) ? "" : "non-",
+ (refflags & PAT_REF_COND) ? "" : "non-");
+ return 0;
+ }
+
/* If the load require samples and the flag PAT_REF_SMP is not
set,
* the reference doesn't contain sample, and cannot be used.
*/
@@ -2552,6 +2766,7 @@ int pattern_read_from_file(struct pattern_head *head,
unsigned int refflags,
/* Extends display */
chunk_printf(&trash, "%s", ref->display);
chunk_appendf(&trash, ", by %s at file '%s' line %d",
+ refflags & PAT_REF_COND ? "conditional map" :
refflags & PAT_REF_MAP ? "map" : "acl", file,
line);
free(ref->display);
ref->display = strdup(trash.area);
@@ -2603,6 +2818,48 @@ int pattern_read_from_file(struct pattern_head *head,
unsigned int refflags,
return 1;
}
+/* Duplicates matched pattern <pat> and its sample data into the thread-local
+ * static holders, so that another thread may not modify them under our feet,
+ * and returns the copy. This is the same treatment pattern_exec_match() below
+ * applies inline; it is factored out here for pattern_exec_match_cond() only,
+ * so that pattern_exec_match() stays exactly as it is.
+ */
+static struct pattern *pattern_dup_static(struct pattern *pat)
+{
+ if (pat != &static_pattern) {
+ memcpy(&static_pattern, pat, sizeof(struct pattern));
+ pat = &static_pattern;
+ }
+
+ if (pat->data && (pat->data != &static_sample_data)) {
+ switch (pat->data->type) {
+ case SMP_T_STR:
+ static_sample_data.type = SMP_T_STR;
+ static_sample_data.u.str = *get_trash_chunk();
+ static_sample_data.u.str.data = pat->data->u.str.data;
+ if (static_sample_data.u.str.data >=
static_sample_data.u.str.size)
+ static_sample_data.u.str.data =
static_sample_data.u.str.size - 1;
+ memcpy(static_sample_data.u.str.area,
+ pat->data->u.str.area,
static_sample_data.u.str.data);
+
static_sample_data.u.str.area[static_sample_data.u.str.data] = 0;
+ pat->data = &static_sample_data;
+ break;
+
+ case SMP_T_IPV4:
+ case SMP_T_IPV6:
+ case SMP_T_SINT:
+ memcpy(&static_sample_data, pat->data, sizeof(struct
sample_data));
+ pat->data = &static_sample_data;
+ break;
+ default:
+ /* unimplemented pattern type */
+ pat->data = NULL;
+ break;
+ }
+ }
+ return pat;
+}
+
/* This function executes a pattern match on a sample. It applies pattern
<expr>
* to sample <smp>. The function returns NULL if the sample don't match. It
returns
* non-null if the sample match. If <fill> is true and the sample match, the
@@ -2676,6 +2933,66 @@ struct pattern *pattern_exec_match(struct pattern_head
*head, struct sample *smp
return NULL;
}
+/* Same as above, but for a conditional map: the patterns are spread over
+ * per-condition groups and only the group whose key exactly matches <cond> is
+ * looked up. This is deliberately kept separate from pattern_exec_match()
+ * above so that the regular map and ACL lookups are left untouched. Returns
+ * NULL if <cond> is empty, longer than PAT_COND_KEY_MAXLEN, holds a zero, or
+ * matches no group, and when the sample matches none of the group's patterns.
+ */
+struct pattern *pattern_exec_match_cond(struct pattern_head *head, struct
sample *smp,
+ int fill, struct ist cond)
+{
+ /* The condition key is copied because the caller may have taken it from
+ * a trash chunk which sample_convert() below is free to recycle. The
+ * tree only holds zero-terminated keys, so a key containing a zero
+ * cannot match.
+ */
+ static THREAD_LOCAL char key[PAT_COND_KEY_MAXLEN + 1];
+ struct pattern_expr_list *list;
+ struct pattern_expr *expr;
+ struct pattern *pat;
+
+ if (!head->match)
+ return NULL;
+
+ if (!istlen(cond) || istlen(cond) > PAT_COND_KEY_MAXLEN ||
+ memchr(istptr(cond), 0, istlen(cond)))
+ return NULL;
+
+ memcpy(key, istptr(cond), istlen(cond));
+ key[istlen(cond)] = 0;
+ cond = ist2(key, istlen(cond));
+
+ /* convert input to the type expected by the match method */
+ if (!sample_convert(smp, head->expect_type))
+ return NULL;
+
+ list_for_each_entry(list, &head->head, list) {
+ HA_RWLOCK_RDLOCK(PATEXP_LOCK, &list->expr->lock);
+
+ expr = pattern_cond_grp_expr(list->expr, cond);
+ if (!expr) {
+ HA_RWLOCK_RDUNLOCK(PATEXP_LOCK, &list->expr->lock);
+ continue;
+ }
+
+ /* the group's own lock is taken as well since pat_ref_set_elt()
+ * updates the samples under it
+ */
+ HA_RWLOCK_RDLOCK(PATEXP_LOCK, &expr->lock);
+ pat = head->match(smp, expr, fill);
+ if (pat)
+ pat = pattern_dup_static(pat);
+ HA_RWLOCK_RDUNLOCK(PATEXP_LOCK, &expr->lock);
+ HA_RWLOCK_RDUNLOCK(PATEXP_LOCK, &list->expr->lock);
+
+ if (pat)
+ return pat;
+ }
+ return NULL;
+}
+
/* This function prunes the pattern expressions starting at pattern_head
<head>. */
void pattern_prune(struct pattern_head *head)
{
--
2.50.1 (Apple Git-155)
