Hi Ricardo,
On Fri, Jul 18, 2025 at 10:15:30AM -0300, Ricardo Nabinger Sanchez wrote:
> Hello,
>
> I'm trying to do something that seems simple, at least in my head: a
> frontend accepting PROXY(2) requests and forwarding them to a PROXY2
> backend, keeping whatever TLVs come along. However, I'm finding
> myself struggling with how to make this work.
>
> For context, an extremely streamlined conf, from what I consider the
> furthest I could succeed:
>
> frontend FE
> bind :1300 accept-proxy
> use_backend BE
>
> backend BE
> server pp2 127.0.0.1:1200 send-proxy-v2
> set-proxy-v2-tlv-fmt(5) %[fc_pp_unique_id,bytes(0,8)]
>
> Now here is where it gets puzzling. This is an excerpt from strace
> which I will detail next, manually aligned (fingers crossed it will
> stay aligned in the email):
>
> 09:26:38.653517 recvfrom(10, "\r\n\r\n\x00\r\nQUIT \n
> !\x11\x00\x1c\x7f\x00\x00\x01\x7f\x00\x00\x01\x04
> L\xba\xa0\x05\x00\x08 P\x04\x02\xf7\xff\x7f\x00\x00\xe1\x00\x02\x04
> L", 44, 0, NULL, NULL) = 44 <0.000005>
> 09:26:38.653870 sendto(11, "\r\n\r\n\x00\r\nQUIT \n
> !\x11\x00\x15\x7f\x00\x00\x01\x7f\x00\x00\x01\x04
> L\xba\xa0\x05\x00\x06 P\x04\x02\xf7\xff\x7f",
> 37, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 37 <0.000031>
>
> The recvfrom, is HAProxy chopping off the PROXY2 bytes from the
> socket, since it already analyzed it in a prior MSG_PEEK. Take note
> that I inserted spaces to align things. The contents from recvfrom
> line is exactly as another application sent. It includes a PROXY2
> header, an IPv4 tuple, TLV 0x5 of size 8, containing a pointer:
> 0x00007ffff7020450, followed by another TLV, 0xE1, size 2, containing
> a u16, 0x044c (decimal 1100).
>
> An important point: my configuration is likely incorrect, or simply
> doesn't do what I wished it did in the way it is written.
>
> As for the next line, sendto(), HAProxy is not forwarding exactly what
> I would expect. In fact, I was under the impression that I didn't
> need to specify anything, that it would be simply forward the PROXY2
> message, almost untouched.
No, it does not "forward" PROXY headers. The PROXY header is decoded
on one side to fill some information that might be needed during the
processing (protocol, source IP address, etc), and is used to replace
such info with those found in the header. Then on the output, you're
asking for a PROXY header to be produced, based on some information
that are available. As you can see there's no such notion of forwarding
here. The PROXY header is really only meant to be hop-by-hop, so each
component in the chain writes the one it was configured to write.
> But in my trials, setting TLV 0x5 kind of
> propagated it. Kind of, because it did sent out 6 bytes, not the 8
> bytes received---it trimmed the nulls.
Hmm that one is interesting. I'm seeing it defined like this:
fc_pp_unique_id : string
Returns the first unique ID TLV sent by the client in the PROXY
protocol header, if any.
Since it's a string, it cannot contain \0 and will stop on them,
contrary to binary type functions which are (content+len) and can
contain zeroes.
I must say I'm a bit embarrassed, because I'm seeing that unique_id
retrieved from SSL is binary but not this one. Apparently when he
implemented it in 2.2, Tim expected it to be used only with IDs
produced on the output which are already strings (very likely in
a similar way to the HTTP unique ID we can generate), in order
to pass it into logs. But the protocol's doc says it's an opaque
byte sequence:
The value of the type PP2_TYPE_UNIQUE_ID is an opaque byte sequence of
up to 128 bytes generated by the upstream proxy that uniquely identifies
the connection.
But if we change it now to support binary, then it will be logged as
a hex string and can break some setups.
I'm seeing three possibilities short-term for you:
- you just patch the code for now to try again:
diff --git a/src/connection.c b/src/connection.c
index e8509fc2d3..8aa6a1bcc3 100644
--- a/src/connection.c
+++ b/src/connection.c
@@ -2593,6 +2593,7 @@ int smp_fetch_fc_pp_unique_id(const struct arg
*args, struct sample *smp, const
set_tlv_arg(PP2_TYPE_UNIQUE_ID, &tlv_arg);
ret = smp_fetch_fc_pp_tlv(&tlv_arg, smp, kw, private);
+ smp->data.type = SMP_T_BIN; // it's an opaque sequence of bytes
smp->flags &= ~SMP_F_NOT_LAST; // return only the first unique ID
return ret;
}
@@ -2731,7 +2732,7 @@ static struct sample_fetch_kw_list
sample_fetch_keywords = {ILH, {
{ "fc_rcvd_proxy", smp_fetch_fc_rcvd_proxy, 0, NULL, SMP_T_BOOL,
SMP_USE_L4CLI },
{ "fc_nb_streams", smp_fetch_fc_nb_streams, 0, NULL, SMP_T_SINT,
SMP_USE_L4CLI },
{ "fc_pp_authority", smp_fetch_fc_pp_authority, 0, NULL,
SMP_T_STR, SMP_USE_L4CLI },
- { "fc_pp_unique_id", smp_fetch_fc_pp_unique_id, 0, NULL,
SMP_T_STR, SMP_USE_L4CLI },
+ { "fc_pp_unique_id", smp_fetch_fc_pp_unique_id, 0, NULL,
SMP_T_BIN, SMP_USE_L4CLI },
{ "fc_pp_tlv", smp_fetch_fc_pp_tlv, ARG1(1, STR),
smp_check_tlv_type, SMP_T_STR, SMP_USE_L5CLI },
{ "fc_settings_streams_limit", smp_fetch_fc_streams_limit, 0,
NULL, SMP_T_SINT, SMP_USE_L5CLI },
{ /* END */ },
- we could create a new sample fetch function for the binary mode,
such as fc_pp_unique_id_raw(), and be done with that. We'd then
add a reference to it in the doc of the existing one.
- your application doesn't need at all to emit non-printable
bytes and you can adjust the contents (of base64/hex them).
> Now, for the questions.
>
> 1- Do I need to explicitly forward things from the PROXY2 message?
Yes because it's not "forwarded" but received and re-created.
> Is
> there a magic knob that will simply do that, which I am overlooking?
> I was under the impression that these would be treated just as HTTP
> headers: implicitly forwarded.
I don't think so, but I do understand why in certain configs it could
be desirable. I'd be a bit concerned about the end-to-end traceability
though because if you forward a PP header, you effectively make the
proxy and its predecessor completely disappear from the log chain,
which might become a problem for debugging purposes.
> 2- That data trimming was really unexpected. What am I doing wrong?
> I could sort it out on the other side (backend), but it feels
> hackish/incorrect doing so.
String vs binary. Small design bug that does exactly what it says and
documents but does not 100% match what the spec wants, and as such will
work for most users but not all. We just need to figure how to best
address it.
> 3- Of least importance and not really a question: I couldn't use
> symbolic names in the conf (e.g.: neither UNIQUE_ID or
> PP2_TYPE_UNIQUE_ID[1]; it is not clear to me which should be used and
> HAProxy complains about either).
I must confess I don't know either but wouldn't be surprised that it
doesn't decode such names. If so, that's too bad, because they don't
change often and being able to resolve them would make confs so much
more readable.
If you (or someone else) has 15mn to spend on a patch for this, I'd
happily apply it!
Cheers,
Willy
