Hi,
Over a past few weeks I have been working on implementing JA3 compatible
TLS Fingerprinting[1] in the HAProxy. You can find the outcome in
attachments. Feel free to review/comment them.
Here are some choices I made which you should be aware of:
- I decided to go with a "modular" approach where you can build JA3
compatible fingerprint with available fetchers/converters rather than a
single JA3 fetcher. This makes approach more "reusable" in some other
scenarios.
- Each Client Hello related fetcher has option to include/exclude GREASE
(RFC8701) values from the output. This is mainly for backward compatibility
and ability to get "pure" data. I suspect in most cases people do not want
GREASE values to be present in the output (which is not the case right now
for cipherlist fetchers).
- exclude_grease function allocates trash on demand depending on GREASE
(RFC8701) values position in the list. We can get away without creating
trash buffer if GREASE values are present at the very beginning and/or the
very end of the list. I decided to allocate trash buffer only when it's
really needed, so that's why it's creation is "hidden" inside exlude_grease
function.
- Now ssl_capture (next to ciphersuite) contains data about extensions, ec
ciphers etc. One of the reasons I decided to merge all those values in a
single ssl_capture buffer is easier control of buffer size limit. I think
it's beneficial to have a single buffer limit for all those values rather
than separate values for each. Having said that probably
tune.ssl.capture-cipherlist-size needs to change it's name to eg.
tune.ssl.capture-buffer-limit to better reflect it's function.
- Instead of creating a new converter I decided to extend existing hex
conveter to provide a similar functionality to bin2int. I thought this
makes more sense as extended hex converter is fully backward compatible. It
has to be noted that extended hex converter is not strictly necessary to
produce JA3 TLS Fingerprint, but but might useful in some other scenarios.
Example usage:
http-request set-header X-SSL-JA3
%[ssl_fc_protocol_hello_id],%[ssl_fc_cipherlist_bin(1),bin2int(-,2)],%[ssl_fc_extlist_bin(1),bin2int(-,2)],%[ssl_fc_eclist_bin(1),bin2int(-,2)],%[ssl_fc_ecformats_bin,bin2int(-,1)]
http-request set-header X-SSL-JA3-Hash
%[req.fhdr(x-ssl-ja3),digest(md5),hex]
Question: I noticed that during Client Hello parsing we calculate xxh64
right away and store it. Isn't better to calculate it when it's actually
used?
Regards,
Marcin Deranek
[1] https://github.com/salesforce/ja3
From 649105084c7019bc48ff508376c8da137bb1f53b Mon Sep 17 00:00:00 2001
From: Marcin Deranek <marcin.dera...@booking.com>
Date: Mon, 12 Jul 2021 15:31:35 +0200
Subject: [PATCH 4/4] MEDIUM: sample: Extend functionality of hex converter
Allow hex converter to separate output with string separator
and group bytes, so more binary to hex transformations are possible.
This gives hex converter the very same functionality bin2int converter
has. Change is backward compatible.
---
doc/configuration.txt | 12 +++++++++--
src/sample.c | 49 ++++++++++++++++++++++++++++++++++++-------
2 files changed, 52 insertions(+), 9 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index c4a187722..cd96bc6e8 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -16268,11 +16268,19 @@ fix_tag_value(<tag>)
tcp-request content set-var(txn.foo) req.payload(0,0),fix_tag_value(35)
tcp-request content set-var(txn.bar) req.payload(0,0),fix_tag_value(MsgType)
-hex
+hex([<separator>],[<chunk_size>],[<truncate>])
Converts a binary input sample to a hex string containing two hex digits per
input byte. It is used to log or transfer hex dumps of some binary input data
in a way that can be reliably transferred (e.g. an SSL ID can be copied in a
- header).
+ header). <separator> is put every <chunk_size> binary input bytes if
+ specified. <truncate> flag indicates whatever binary input is truncated at
+ <chunk_size> boundaries.
+
+ Example:
+ bin(01020304050607),hex # 01020304050607
+ bin(01020304050607),hex(:,2) # 0102:0304:0506:07
+ bin(01020304050607),hex(--,2,1) # 0102--0304--0506
+ bin(0102030405060708),hex(,3,1) # 010203040506
hex2i
Converts a hex string containing two hex digits per input byte to an
diff --git a/src/sample.c b/src/sample.c
index a34aee51a..dbd62c5a9 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2113,18 +2113,53 @@ static int sample_conv_bin2int(const struct arg *args, struct sample *smp, void
return 1;
}
-static int sample_conv_bin2hex(const struct arg *arg_p, struct sample *smp, void *private)
+static int sample_conv_hex_check(struct arg *args, struct sample_conv *conv,
+ const char *file, int line, char **err)
+{
+ if (args[1].data.sint <= 0 && (args[0].data.str.data > 0 || args[2].data.sint != 0)) {
+ memprintf(err, "chunk_size needs to be positive (%lld)", args[1].data.sint);
+ return 0;
+ }
+
+ if (args[2].data.sint != 0 && args[2].data.sint != 1) {
+ memprintf(err, "Unsupported truncate value (%lld)", args[2].data.sint);
+ return 0;
+ }
+
+ return 1;
+}
+
+static int sample_conv_bin2hex(const struct arg *args, struct sample *smp, void *private)
{
struct buffer *trash = get_trash_chunk();
- unsigned char c;
+ int chunk_size = args[1].data.sint;
+ const int last = args[2].data.sint ? smp->data.u.str.data - chunk_size + 1 : smp->data.u.str.data;
+ int i;
+ int max_size;
int ptr = 0;
+ unsigned char c;
trash->data = 0;
- while (ptr < smp->data.u.str.data && trash->data <= trash->size - 2) {
- c = smp->data.u.str.area[ptr++];
- trash->area[trash->data++] = hextab[(c >> 4) & 0xF];
- trash->area[trash->data++] = hextab[c & 0xF];
+ if (args[0].data.str.data == 0 && args[2].data.sint == 0)
+ chunk_size = smp->data.u.str.data;
+ max_size = trash->size - 2 * chunk_size;
+
+ while (ptr < last && trash->data <= max_size) {
+ if (ptr) {
+ /* Separator */
+ memcpy(trash->area + trash->data, args[0].data.str.area, args[0].data.str.data);
+ trash->data += args[0].data.str.data;
+ }
+ else
+ max_size -= args[0].data.str.data;
+ /* Hex */
+ for (i = 0; i < chunk_size && ptr < smp->data.u.str.data; i++) {
+ c = smp->data.u.str.area[ptr++];
+ trash->area[trash->data++] = hextab[(c >> 4) & 0xF];
+ trash->area[trash->data++] = hextab[c & 0xF];
+ }
}
+
smp->data.u.str = *trash;
smp->data.type = SMP_T_STR;
smp->flags &= ~SMP_F_CONST;
@@ -4308,7 +4343,7 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, {
{ "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
{ "bin2int", sample_conv_bin2int, ARG3(1,STR,SINT,SINT), sample_conv_bin2int_check,SMP_T_BIN, SMP_T_STR },
- { "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "hex", sample_conv_bin2hex, ARG3(1,STR,SINT,SINT), sample_conv_hex_check, SMP_T_BIN, SMP_T_STR },
{ "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
{ "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
{ "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
--
2.32.0
From 26be68934ceb44b6ee9fa5a85e0d904c54ea7a46 Mon Sep 17 00:00:00 2001
From: Marcin Deranek <marcin.dera...@booking.com>
Date: Mon, 12 Jul 2021 15:17:04 +0200
Subject: [PATCH 3/4] MINOR: sample: Add bin2int converter
Add bin2int converter which allows to build JA3 compatible TLS
fingerprints by converting binary data into string separated
unsigned integers eg.
http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
%[ssl_fc_cipherlist_bin(1),bin2int(-,2)],\
%[ssl_fc_extlist_bin(1),bin2int(-,2)],\
%[ssl_fc_eclist_bin(1),bin2int(-,2)],\
%[ssl_fc_ecformats_bin,bin2int(-,1)]
---
doc/configuration.txt | 12 +++++++++
src/sample.c | 57 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 69 insertions(+)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 1053bbcef..c4a187722 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -16064,6 +16064,18 @@ base64
an SSL ID can be copied in a header). For base64url("URL and Filename
Safe Alphabet" (RFC 4648)) variant see "ub64enc".
+bin2int(<separator>,<chunk_size>,[<truncate>])
+ Converts a binary input sample to a string containing an unsigned integer
+ number per <chunk_size> input bytes. <separator> is put every <chunk_size>
+ binary input bytes if specified. <truncate> flag indicates whatever binary
+ input is truncated at <chunk_size> boundaries. <chunk_size> maximum value is
+ limited by the size of long long int (8 bytes).
+
+ Example:
+ bin(01020304050607),bin2int(:,2) # 258:772:1286:7
+ bin(01020304050607),bin2int(-,2,1) # 258-772-1286
+ bin(01020304050607),bin2int(,2,1) # 2587721286
+
bool
Returns a boolean TRUE if the input value of type signed integer is
non-null, otherwise returns FALSE. Used in conjunction with and(), it can be
diff --git a/src/sample.c b/src/sample.c
index d02034cf0..a34aee51a 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2057,6 +2057,62 @@ static int sample_conv_crypto_hmac(const struct arg *args, struct sample *smp, v
#endif /* USE_OPENSSL */
+static int sample_conv_bin2int_check(struct arg *args, struct sample_conv *conv,
+ const char *file, int line, char **err)
+{
+ if (args[1].data.sint <= 0 || args[1].data.sint > sizeof(unsigned long long)) {
+ memprintf(err, "chunk_size out of [1..%ld] range (%lld)", sizeof(unsigned long long), args[1].data.sint);
+ return 0;
+ }
+
+ if (args[2].data.sint != 0 && args[2].data.sint != 1) {
+ memprintf(err, "Unsupported truncate value (%lld)", args[2].data.sint);
+ return 0;
+ }
+
+ return 1;
+}
+
+static int sample_conv_bin2int(const struct arg *args, struct sample *smp, void *private)
+{
+ struct buffer *trash = get_trash_chunk();
+ const int last = args[2].data.sint ? smp->data.u.str.data - args[1].data.sint + 1 : smp->data.u.str.data;
+ int max_size = trash->size - 2;
+ int i;
+ int start;
+ int ptr = 0;
+ unsigned long long number;
+ char *pos;
+
+ trash->data = 0;
+
+ while (ptr < last && trash->data <= max_size) {
+ start = trash->data;
+ if (ptr) {
+ /* Separator */
+ memcpy(trash->area + trash->data, args[0].data.str.area, args[0].data.str.data);
+ trash->data += args[0].data.str.data;
+ }
+ else
+ max_size -= args[0].data.str.data;
+ /* Integer */
+ for (number = 0, i = 0; i < args[1].data.sint && ptr < smp->data.u.str.data; i++)
+ number = (number << 8) + (unsigned char)smp->data.u.str.area[ptr++];
+ pos = ulltoa(number, trash->area + trash->data, trash->size - trash->data);
+ if (pos)
+ trash->data = pos - trash->area;
+ else {
+ trash->data = start;
+ break;
+ }
+ }
+
+ smp->data.u.str = *trash;
+ smp->data.type = SMP_T_STR;
+ smp->flags &= ~SMP_F_CONST;
+ return 1;
+}
+
static int sample_conv_bin2hex(const struct arg *arg_p, struct sample *smp, void *private)
{
struct buffer *trash = get_trash_chunk();
@@ -4251,6 +4307,7 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, {
{ "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
+ { "bin2int", sample_conv_bin2int, ARG3(1,STR,SINT,SINT), sample_conv_bin2int_check,SMP_T_BIN, SMP_T_STR },
{ "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
{ "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
{ "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
--
2.32.0
From 44d5a24f5c202b66426f6c2045d80992ae5c2e10 Mon Sep 17 00:00:00 2001
From: Marcin Deranek <marcin.dera...@booking.com>
Date: Mon, 12 Jul 2021 14:16:55 +0200
Subject: [PATCH 1/4] MEDIUM: ssl: Capture more info from Client Hello
When we set tune.ssl.capture-cipherlist-size to a non-zero value
we are able to capture cipherlist supported by the client. To be able to
provide JA3 compatible TLS fingerprinting we need to capture more
information from Client Hello message:
- SSL Version
- SSL Extensions
- Elliptic Curves
- Elliptic Curve Point Formats
This patch allows HAProxy to capture such information and store it for
later use.
---
doc/configuration.txt | 7 +-
include/haproxy/ssl_sock-t.h | 14 +++-
src/ssl_sample.c | 4 +-
src/ssl_sock.c | 127 +++++++++++++++++++++++++++++++++--
4 files changed, 138 insertions(+), 14 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index b128718cd..c3d640ab8 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -2783,9 +2783,10 @@ tune.ssl.ssl-ctx-cache-size <number>
1000 entries.
tune.ssl.capture-cipherlist-size <number>
- Sets the maximum size of the buffer used for capturing client-hello cipher
- list. If the value is 0 (default value) the capture is disabled, otherwise
- a buffer is allocated for each SSL/TLS connection.
+ Sets the maximum size of the buffer used for capturing client hello cipher
+ list, extensions list, elliptic curves list and elliptic curve point
+ formats. If the value is 0 (default value) the capture is disabled,
+ otherwise a buffer is allocated for each SSL/TLS connection.
tune.vars.global-max-size <size>
tune.vars.proc-max-size <size>
diff --git a/include/haproxy/ssl_sock-t.h b/include/haproxy/ssl_sock-t.h
index 98390113f..8e28bf091 100644
--- a/include/haproxy/ssl_sock-t.h
+++ b/include/haproxy/ssl_sock-t.h
@@ -199,11 +199,21 @@ struct ssl_sock_msg_callback {
struct list list; /* list of registered callbacks */
};
+/* Location and size of the data in the buffer */
+struct ssl_capture_location {
+ unsigned char len;
+ size_t offset;
+};
+
/* This memory pool is used for capturing clienthello parameters. */
struct ssl_capture {
unsigned long long int xxh64;
- unsigned char ciphersuite_len;
- char ciphersuite[VAR_ARRAY];
+ unsigned int protocol_version;
+ struct ssl_capture_location ciphersuite;
+ struct ssl_capture_location extensions;
+ struct ssl_capture_location ec;
+ struct ssl_capture_location ec_formats;
+ char data[VAR_ARRAY];
};
#ifdef HAVE_SSL_KEYLOG
diff --git a/src/ssl_sample.c b/src/ssl_sample.c
index bfa61bdcd..4785171dd 100644
--- a/src/ssl_sample.c
+++ b/src/ssl_sample.c
@@ -1145,8 +1145,8 @@ smp_fetch_ssl_fc_cl_bin(const struct arg *args, struct sample *smp, const char *
smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
smp->data.type = SMP_T_BIN;
- smp->data.u.str.area = capture->ciphersuite;
- smp->data.u.str.data = capture->ciphersuite_len;
+ smp->data.u.str.area = capture->data + capture->ciphersuite.offset;
+ smp->data.u.str.data = capture->ciphersuite.len;
return 1;
}
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index e27611572..0de5a431d 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -1652,6 +1652,15 @@ static void ssl_sock_parse_clienthello(struct connection *conn, int write_p, int
struct ssl_capture *capture;
unsigned char *msg;
unsigned char *end;
+ unsigned char *extensions_end;
+ unsigned char *ec_start = NULL;
+ unsigned char *ec_formats_start = NULL;
+ unsigned char *list_end;
+ int ec_len = 0;
+ int ec_formats_len = 0;
+ unsigned int protocol_version;
+ unsigned int extension_id;
+ size_t offset = 0;
size_t rec_len;
/* This function is called for "from client" and "to server"
@@ -1714,11 +1723,18 @@ static void ssl_sock_parse_clienthello(struct connection *conn, int write_p, int
if (end < msg)
return;
- /* Expect 2 bytes for protocol version (1 byte for major and 1 byte
- * for minor, the random, composed by 4 bytes for the unix time and
- * 28 bytes for unix payload. So we jump 1 + 1 + 4 + 28.
+ /* Expect 2 bytes for protocol version
+ * (1 byte for major and 1 byte for minor)
*/
- msg += 1 + 1 + 4 + 28;
+ if (msg + 2 > end)
+ return;
+ protocol_version = (msg[0] << 8) + msg[1];
+ msg += 2;
+
+ /* Expect the random, composed by 4 bytes for the unix time and
+ * 28 bytes for unix payload. So we jump 4 + 28.
+ */
+ msg += 4 + 28;
if (msg > end)
return;
@@ -1747,10 +1763,107 @@ static void ssl_sock_parse_clienthello(struct connection *conn, int write_p, int
capture->xxh64 = XXH64(msg, rec_len, 0);
/* Capture the ciphersuite. */
- capture->ciphersuite_len = (global_ssl.capture_cipherlist < rec_len) ?
- global_ssl.capture_cipherlist : rec_len;
- memcpy(capture->ciphersuite, msg, capture->ciphersuite_len);
+ capture->ciphersuite.len = MIN(global_ssl.capture_cipherlist, rec_len);
+ capture->ciphersuite.offset = 0;
+ memcpy(capture->data, msg, capture->ciphersuite.len);
+ msg += rec_len;
+ offset += capture->ciphersuite.len;
+
+ /* Initialize other data */
+ capture->protocol_version = protocol_version;
+ capture->extensions.len = 0;
+ capture->extensions.offset = 0;
+ capture->ec.len = 0;
+ capture->ec.offset = 0;
+ capture->ec_formats.len = 0;
+ capture->ec_formats.offset = 0;
+
+ /* Next, compression methods:
+ * if present, we have to jump by length + 1 for the size information
+ * if not present, we have to jump by 1 only
+ */
+ if (msg[0] > 0)
+ msg += msg[0];
+ msg += 1;
+ if (msg > end)
+ goto store_capture;
+ /* We reached extensions */
+ if (msg + 2 > end)
+ goto store_capture;
+ rec_len = (msg[0] << 8) + msg[1];
+ msg += 2;
+ if (msg + rec_len > end || msg + rec_len < msg)
+ goto store_capture;
+ extensions_end = msg + rec_len;
+ capture->extensions.offset = offset;
+
+ /* Parse each extension */
+ while (msg + 4 < extensions_end) {
+ /* Add 2 bytes of extension_id */
+ if (global_ssl.capture_cipherlist >= offset + 2) {
+ capture->data[offset++] = msg[0];
+ capture->data[offset++] = msg[1];
+ capture->extensions.len += 2;
+ }
+ else
+ break;
+ extension_id = (msg[0] << 8) + msg[1];
+ /* Length of the extension */
+ rec_len = (msg[2] << 8) + msg[3];
+
+ /* Expect 2 bytes extension id + 2 bytes extension size */
+ msg += 2 + 2;
+ if (msg + rec_len > extensions_end || msg + rec_len < msg)
+ goto store_capture;
+ if (extension_id == 0x000a) {
+ /* Elliptic Curves */
+ list_end = msg + rec_len;
+ if (msg + 2 > list_end)
+ goto store_capture;
+ rec_len = (msg[0] << 8) + msg[1];
+ msg += 2;
+
+ if (msg + rec_len > list_end || msg + rec_len < msg)
+ goto store_capture;
+ /* Store location/size of the list */
+ ec_start = msg;
+ ec_len = rec_len;
+ }
+ else if (extension_id == 0x000b) {
+ /* Elliptic Curves Point Formats */
+ list_end = msg + rec_len;
+ if (msg + 1 > list_end)
+ goto store_capture;
+ rec_len = msg[0];
+ msg += 1;
+
+ if (msg + rec_len > list_end || msg + rec_len < msg)
+ goto store_capture;
+ /* Store location/size of the list */
+ ec_formats_start = msg;
+ ec_formats_len = rec_len;
+ }
+ msg += rec_len;
+ }
+
+ rec_len = MIN(global_ssl.capture_cipherlist - offset, ec_len);
+ if (ec_start) {
+ memcpy(capture->data + offset, ec_start, rec_len);
+ capture->ec.offset = offset;
+ capture->ec.len = rec_len;
+ offset += rec_len;
+ }
+
+ rec_len = MIN(global_ssl.capture_cipherlist - offset, ec_formats_len);
+ if (ec_formats_start) {
+ memcpy(capture->data + offset, ec_formats_start, rec_len);
+ capture->ec_formats.offset = offset;
+ capture->ec_formats.len = rec_len;
+ offset += rec_len;
+ }
+
+store_capture:
SSL_set_ex_data(ssl, ssl_capture_ptr_index, capture);
}
--
2.32.0
From 52a63089b9dacfdde727f46b2cf81b8d59678605 Mon Sep 17 00:00:00 2001
From: Marcin Deranek <marcin.dera...@booking.com>
Date: Mon, 12 Jul 2021 14:52:46 +0200
Subject: [PATCH 2/4] MINOR: sample: Expose SSL captures using new fetchers
To be able to provide JA3 compatible TLS Fingerprints we need to expose
all Client Hello captured data using fetchers. Patch provides new
and modifies existing fetchers to add ability to filter out GREASE values:
- ssl_fc_cipherlist_*
- ssl_fc_ecformats_bin
- ssl_fc_eclist_bin
- ssl_fc_extlist_bin
- ssl_fc_protocol_hello_id
---
doc/configuration.txt | 60 ++++++++++++++-----
include/haproxy/tools.h | 2 +
src/ssl_sample.c | 125 ++++++++++++++++++++++++++++++++++++++--
src/tools.c | 56 ++++++++++++++++++
4 files changed, 221 insertions(+), 22 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index c3d640ab8..1053bbcef 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -18786,27 +18786,49 @@ ssl_fc_cipher : string
Returns the name of the used cipher when the incoming connection was made
over an SSL/TLS transport layer.
-ssl_fc_cipherlist_bin : binary
- Returns the binary form of the client hello cipher list. The maximum returned
- value length is according with the value of
- "tune.ssl.capture-cipherlist-size".
+ssl_fc_cipherlist_bin([<exclude_grease>]) : binary
+ Returns the binary form of the client hello cipher list. Setting
+ <exlude_grease> to 1 will exclude GREASE (RFC8701) values from the output.
+ The maximum returned value length is limited by the shared capture buffer
+ size controlled by "tune.ssl.capture-cipherlist-size" setting.
-ssl_fc_cipherlist_hex : string
+ssl_fc_cipherlist_hex([<exclude_grease>]) : string
Returns the binary form of the client hello cipher list encoded as
- hexadecimal. The maximum returned value length is according with the value of
- "tune.ssl.capture-cipherlist-size".
-
-ssl_fc_cipherlist_str : string
- Returns the decoded text form of the client hello cipher list. The maximum
- number of ciphers returned is according with the value of
- "tune.ssl.capture-cipherlist-size". Note that this sample-fetch is only
- available with OpenSSL >= 1.0.2. If the function is not enabled, this
- sample-fetch returns the hash like "ssl_fc_cipherlist_xxh".
+ hexadecimal. Setting <exlude_grease> to 1 will exclude GREASE (RFC8701)
+ values from the output. The maximum returned value length is limited by the
+ shared capture buffer size controlled by "tune.ssl.capture-cipherlist-size"
+ setting.
+
+ssl_fc_cipherlist_str([<exclude_grease>]) : string
+ Returns the decoded text form of the client hello cipher list. Setting
+ <exlude_grease> to 1 will exclude GREASE (RFC8701) values from the output.
+ The maximum returned value length is limited by the shared capture buffer
+ size controlled by "tune.ssl.capture-cipherlist-size" setting. Note that
+ this sample-fetch is only available with OpenSSL >= 1.0.2. If the function
+ is not enabled, this sample-fetch returns the hash like
+ "ssl_fc_cipherlist_xxh".
ssl_fc_cipherlist_xxh : integer
- Returns a xxh64 of the cipher list. This hash can be return only is the value
+ Returns a xxh64 of the cipher list. This hash can return only if the value
"tune.ssl.capture-cipherlist-size" is set greater than 0, however the hash
- take in account all the data of the cipher list.
+ take into account all the data of the cipher list.
+
+ssl_fc_ecformats_bin : binary
+ Return the binary form of the client hello supported elliptic curve point
+ formats. The maximum returned value length is limited by the shared capture
+ buffer size controlled by "tune.ssl.capture-cipherlist-size" setting.
+
+ssl_fc_eclist_bin([<exclude_grease>]) : binary
+ Returns the binary form of the client hello supported elliptic curves.
+ Setting <exlude_grease> to 1 will exclude GREASE (RFC8701) values from the
+ output. The maximum returned value length is limited by the shared capture
+ buffer size controlled by "tune.ssl.capture-cipherlist-size" setting.
+
+ssl_fc_extlist_bin([<exclude_grease>]) : binary
+ Returns the binary form of the client hello extension list. Setting
+ <exlude_grease> to 1 will exclude GREASE (RFC8701) values from the output.
+ The maximum returned value length is limited by the shared capture buffer
+ size controlled by "tune.ssl.capture-cipherlist-size" setting.
ssl_fc_client_random : binary
Returns the client random of the front connection when the incoming connection
@@ -18897,6 +18919,12 @@ ssl_fc_protocol : string
Returns the name of the used protocol when the incoming connection was made
over an SSL/TLS transport layer.
+ssl_fc_protocol_hello_id : integer
+ The version of the TLS protocol by which the client wishes to communicate
+ during the session as indicated in client hello message. This value can
+ return only if the value "tune.ssl.capture-cipherlist-size" is set greater
+ than 0.
+
ssl_fc_unique_id : binary
When the incoming connection was made over an SSL/TLS transport layer,
returns the TLS unique ID as defined in RFC5929 section 3. The unique id
diff --git a/include/haproxy/tools.h b/include/haproxy/tools.h
index e376a716e..7d6686afa 100644
--- a/include/haproxy/tools.h
+++ b/include/haproxy/tools.h
@@ -973,6 +973,8 @@ static inline unsigned long long rdtsc()
struct list;
int list_append_word(struct list *li, const char *str, char **err);
+void exclude_grease(char *start, int length, char **smp_area, size_t *smp_data, unsigned int *smp_flags);
+
int dump_text(struct buffer *out, const char *buf, int bsize);
int dump_binary(struct buffer *out, const char *buf, int bsize);
int dump_text_line(struct buffer *out, const char *buf, int bsize, int len,
diff --git a/src/ssl_sample.c b/src/ssl_sample.c
index 4785171dd..80df846f6 100644
--- a/src/ssl_sample.c
+++ b/src/ssl_sample.c
@@ -1143,10 +1143,16 @@ smp_fetch_ssl_fc_cl_bin(const struct arg *args, struct sample *smp, const char *
if (!capture)
return 0;
- smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ if (args[0].data.sint)
+ exclude_grease(capture->data + capture->ciphersuite.offset, capture->ciphersuite.len,
+ &smp->data.u.str.area, &smp->data.u.str.data, &smp->flags);
+ else {
+ smp->data.u.str.area = capture->data + capture->ciphersuite.offset;
+ smp->data.u.str.data = capture->ciphersuite.len;
+ smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ }
+
smp->data.type = SMP_T_BIN;
- smp->data.u.str.area = capture->data + capture->ciphersuite.offset;
- smp->data.u.str.data = capture->ciphersuite.len;
return 1;
}
@@ -1188,6 +1194,109 @@ smp_fetch_ssl_fc_cl_xxh64(const struct arg *args, struct sample *smp, const char
return 1;
}
+static int
+smp_fetch_ssl_fc_protocol_hello_id(const struct arg *args, struct sample *smp, const char *kw, void *private)
+{
+ struct connection *conn;
+ struct ssl_capture *capture;
+ SSL *ssl;
+
+ conn = objt_conn(smp->sess->origin);
+ ssl = ssl_sock_get_ssl_object(conn);
+ if (!ssl)
+ return 0;
+
+ capture = SSL_get_ex_data(ssl, ssl_capture_ptr_index);
+ if (!capture)
+ return 0;
+
+ smp->flags = SMP_F_VOL_SESS;
+ smp->data.type = SMP_T_SINT;
+ smp->data.u.sint = capture->protocol_version;
+ return 1;
+}
+
+static int
+smp_fetch_ssl_fc_ext_bin(const struct arg *args, struct sample *smp, const char *kw, void *private)
+{
+ struct connection *conn;
+ struct ssl_capture *capture;
+ SSL *ssl;
+
+ conn = objt_conn(smp->sess->origin);
+ ssl = ssl_sock_get_ssl_object(conn);
+ if (!ssl)
+ return 0;
+
+ capture = SSL_get_ex_data(ssl, ssl_capture_ptr_index);
+ if (!capture)
+ return 0;
+
+ if (args[0].data.sint)
+ exclude_grease(capture->data + capture->extensions.offset, capture->extensions.len,
+ &smp->data.u.str.area, &smp->data.u.str.data, &smp->flags);
+ else {
+ smp->data.u.str.area = capture->data + capture->extensions.offset;
+ smp->data.u.str.data = capture->extensions.len;
+ smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ }
+
+ smp->data.type = SMP_T_BIN;
+ return 1;
+}
+
+static int
+smp_fetch_ssl_fc_ecl_bin(const struct arg *args, struct sample *smp, const char *kw, void *private)
+{
+ struct connection *conn;
+ struct ssl_capture *capture;
+ SSL *ssl;
+
+ conn = objt_conn(smp->sess->origin);
+ ssl = ssl_sock_get_ssl_object(conn);
+ if (!ssl)
+ return 0;
+
+ capture = SSL_get_ex_data(ssl, ssl_capture_ptr_index);
+ if (!capture)
+ return 0;
+
+ if (args[0].data.sint)
+ exclude_grease(capture->data + capture->ec.offset, capture->ec.len,
+ &smp->data.u.str.area, &smp->data.u.str.data, &smp->flags);
+ else {
+ smp->data.u.str.area = capture->data + capture->ec.offset;
+ smp->data.u.str.data = capture->ec.len;
+ smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ }
+
+ smp->data.type = SMP_T_BIN;
+ return 1;
+}
+
+static int
+smp_fetch_ssl_fc_ecf_bin(const struct arg *args, struct sample *smp, const char *kw, void *private)
+{
+ struct connection *conn;
+ struct ssl_capture *capture;
+ SSL *ssl;
+
+ conn = objt_conn(smp->sess->origin);
+ ssl = ssl_sock_get_ssl_object(conn);
+ if (!ssl)
+ return 0;
+
+ capture = SSL_get_ex_data(ssl, ssl_capture_ptr_index);
+ if (!capture)
+ return 0;
+
+ smp->flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ smp->data.type = SMP_T_BIN;
+ smp->data.u.str.area = capture->data + capture->ec_formats.offset;
+ smp->data.u.str.data = capture->ec_formats.len;
+ return 1;
+}
+
/* Dump the SSL keylog, it only works with "tune.ssl.keylog 1" */
#ifdef HAVE_SSL_KEYLOG
static int smp_fetch_ssl_x_keylog(const struct arg *args, struct sample *smp, const char *kw, void *private)
@@ -1533,10 +1642,14 @@ static struct sample_fetch_kw_list sample_fetch_keywords = {ILH, {
#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
{ "ssl_fc_sni", smp_fetch_ssl_fc_sni, 0, NULL, SMP_T_STR, SMP_USE_L5CLI },
#endif
- { "ssl_fc_cipherlist_bin", smp_fetch_ssl_fc_cl_bin, 0, NULL, SMP_T_STR, SMP_USE_L5CLI },
- { "ssl_fc_cipherlist_hex", smp_fetch_ssl_fc_cl_hex, 0, NULL, SMP_T_BIN, SMP_USE_L5CLI },
- { "ssl_fc_cipherlist_str", smp_fetch_ssl_fc_cl_str, 0, NULL, SMP_T_STR, SMP_USE_L5CLI },
+ { "ssl_fc_cipherlist_bin", smp_fetch_ssl_fc_cl_bin, ARG1(0,SINT), NULL, SMP_T_STR, SMP_USE_L5CLI },
+ { "ssl_fc_cipherlist_hex", smp_fetch_ssl_fc_cl_hex, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_USE_L5CLI },
+ { "ssl_fc_cipherlist_str", smp_fetch_ssl_fc_cl_str, ARG1(0,SINT), NULL, SMP_T_STR, SMP_USE_L5CLI },
{ "ssl_fc_cipherlist_xxh", smp_fetch_ssl_fc_cl_xxh64, 0, NULL, SMP_T_SINT, SMP_USE_L5CLI },
+ { "ssl_fc_protocol_hello_id",smp_fetch_ssl_fc_protocol_hello_id,0, NULL, SMP_T_SINT, SMP_USE_L5CLI },
+ { "ssl_fc_extlist_bin", smp_fetch_ssl_fc_ext_bin, ARG1(0,SINT), NULL, SMP_T_STR, SMP_USE_L5CLI },
+ { "ssl_fc_eclist_bin", smp_fetch_ssl_fc_ecl_bin, ARG1(0,SINT), NULL, SMP_T_STR, SMP_USE_L5CLI },
+ { "ssl_fc_ecformats_bin", smp_fetch_ssl_fc_ecf_bin, 0, NULL, SMP_T_STR, SMP_USE_L5CLI },
/* SSL server certificate fetches */
{ "ssl_s_der", smp_fetch_ssl_x_der, 0, NULL, SMP_T_BIN, SMP_USE_L5CLI },
diff --git a/src/tools.c b/src/tools.c
index bd6bf4edc..a3fab389c 100644
--- a/src/tools.c
+++ b/src/tools.c
@@ -4502,6 +4502,62 @@ int may_access(const void *ptr)
return 1;
}
+/* Exclude GREASE (RFC8701) values from input buffer (start/length)
+ * When possible set position/length (smp_area/smp_data) to a continuous area
+ * in the input buffer or copy to a new area and return position/length to a newly copied area.
+ * First approach is preferred as we do not copy any data, but requires GREASE values
+ * to be present at the beginning and/or at the end of the input buffer.
+ */
+void exclude_grease(char *start, int length, char **smp_area, size_t *smp_data, unsigned int *smp_flags)
+{
+ struct buffer *trash = NULL;
+ int begin = -1;
+ int end = -1;
+ int ptr = 0;
+ int size;
+ unsigned char step = 2;
+
+ while (ptr < length) {
+ if (length - ptr == 1)
+ step = 1;
+ if (step == 2 && (start[ptr] == start[ptr+1]) && ((start[ptr] & 0x0f) == 0x0a)) {
+ if (begin != -1 && end == -1)
+ end = ptr;
+ }
+ else if (begin == -1)
+ begin = ptr;
+ else if (end != -1) {
+ size = end - begin;
+ if (!trash)
+ trash = get_trash_chunk();
+ if ((trash->data + size) < trash->size) {
+ memcpy(trash->area + trash->data, start + begin, size);
+ trash->data += size;
+ }
+ else
+ break;
+ begin = ptr;
+ end = -1;
+ }
+ ptr += step;
+ }
+ if (end == -1)
+ end = ptr;
+ if (trash) {
+ size = MIN(end - begin, trash->size - trash->data);
+ memcpy(trash->area + trash->data, start + begin, size);
+ trash->data += size;
+ *smp_area = trash->area;
+ *smp_data = trash->data;
+ *smp_flags = SMP_F_VOL_SESS;
+ }
+ else {
+ *smp_area = start + begin;
+ *smp_data = end - begin;
+ *smp_flags = SMP_F_VOL_TEST | SMP_F_CONST;
+ }
+}
+
/* print a string of text buffer to <out>. The format is :
* Non-printable chars \t, \n, \r and \e are * encoded in C format.
* Other non-printable chars are encoded "\xHH". Space, '\', and '=' are also escaped.
--
2.32.0
