This is an automated email from the ASF dual-hosted git repository. cmcfarlen pushed a commit to branch 10.2.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 7a4294c3350150dedbe79f54e4ed4720781cd254 Author: Brian Neradt <[email protected]> AuthorDate: Tue Apr 28 14:30:12 2026 -0500 Propagate PROXY-Protocol src to outbound surfaces (#13120) On listeners that receive PROXY-Protocol, the parsed source IP currently does not reach several origin-facing and operator-visible surfaces: the outbound `Client-ip` and `X-Forwarded-For` request headers always carry the immediate TCP peer (the CDN edge), and the outbound TPROXY local-bind address, the HostDB parent-selection affinity hash, and the Slow-Request error log all key off the TCP peer even on listeners that have opted into `:pp-clnt`. The `ssl_has_proxy_v1` debug line is also misleading: it fires for both PPv1 and PPv2 and only logs the destination. This aligns those outbound surfaces with the PROXY-Protocol source. `HttpTransact::add_client_ip_to_outgoing_request` is updated to prefer `pp_info.src_addr` whenever the user-agent connection has a parsed PROXY-Protocol header, mirroring `add_forwarded_field_to_request`, so the legacy headers agree with `Forwarded: for=` regardless of `:pp-clnt`. The outbound TPROXY local-bind, HostDB parent-selection affinity, and Slow-Request error log in `HttpSM.cc` are migrated to `t_state.effective_client_addr`, which is `:pp-clnt`-gated by construction so listeners without that flag are unaffected. The PP debug line is rewritten to print the actual PP version together with both `src` and `dst`. This extends the proxy_protocol autest with a PPv2-over-TLS session using a custom `src-addr` to lock in the new outbound-header behavior, and updates the admin-guide PROXY-Protocol page to clarify which surfaces are `:pp-clnt`-gated and which (`Client-ip`, `X-Forwarded-For`, `Forwarded: for=`) are unconditional. (cherry picked from commit 060721f3daa4e5da79ff649729f59c2fbcea426d) --- .../configuration/proxy-protocol.en.rst | 30 ++++++++++ doc/admin-guide/files/records.yaml.en.rst | 11 ++++ .../api/functions/TSHttpSsnClientAddrGet.en.rst | 64 +++++++++++++++++++++ .../api/functions/TSHttpTxnClientAddrGet.en.rst | 64 +++++++++++++++++++++ .../api/functions/TSNetVConnClientAddrGet.en.rst | 66 ++++++++++++++++++++++ src/iocore/net/SSLNetVConnection.cc | 10 +++- src/proxy/http/HttpSM.cc | 17 ++++-- src/proxy/http/HttpTransact.cc | 15 ++++- .../gold_tests/proxy_protocol/gold/access-cp.gold | 1 + .../proxy_protocol/gold/access-nocp.gold | 1 + .../proxy_protocol/proxy_protocol.test.py | 2 + .../replay/proxy_protocol_in.replay.yaml | 46 +++++++++++++++ 12 files changed, 318 insertions(+), 9 deletions(-) diff --git a/doc/admin-guide/configuration/proxy-protocol.en.rst b/doc/admin-guide/configuration/proxy-protocol.en.rst index 4263d9e771..d7fca8ac51 100644 --- a/doc/admin-guide/configuration/proxy-protocol.en.rst +++ b/doc/admin-guide/configuration/proxy-protocol.en.rst @@ -17,6 +17,8 @@ .. include:: ../../common.defs +.. default-domain:: cpp + .. _proxy-protocol: Proxy Protocol @@ -59,6 +61,27 @@ enable PROXY protocol and want to apply ACL against the IP address delivered by If you specify the server_ports flag `pp-clnt` then the client IP address used for the transaction will be the one provided by proxy protocol. +The ``pp-clnt`` flag governs whether the operator-visible "client IP" is the +PROXY-Protocol source address rather than the immediate TCP peer for the +following surfaces: + +* The squid log field ``%<chi>`` (and its derivatives ``%<chp>`` / + ``%<chh>``). +* :func:`TSHttpTxnClientAddrGet`, :func:`TSHttpSsnClientAddrGet`, and + :func:`TSNetVConnClientAddrGet` (the plugin-visible client address). +* SNI ACL evaluation, the HTTP/2 ``PEER`` ACL, SSL diagnostics, and + client-certificate validation. +* Outbound transparency: when binding the proxy's outbound socket to the + client's address (``addr_binding = FOREIGN_ADDR``), |TS| binds to the + PROXY-Protocol source address only when ``pp-clnt`` is in effect. +* HostDB parent-selection affinity: the consistent-hash key that groups + requests onto the same upstream peer is the PROXY-Protocol source only + when ``pp-clnt`` is in effect; otherwise it is the immediate TCP peer. + +The new log field ``%<rchi>`` (and its derivatives ``%<rchp>`` / +``%<rchh>``) always reports the immediate TCP peer regardless of +``pp-clnt``. + 1. HTTP Forwarded Header The client IP address in the PROXY protocol header is passed to the origin server via an HTTP `Forwarded: @@ -67,6 +90,13 @@ Detection of the PROXY protocol header is automatic. If the PROXY header precludes the request, it will automatically be parse and made available to the Forwarded: request header sent to the origin server. +The legacy outbound headers ``Client-ip:`` +(:ts:cv:`proxy.config.http.insert_client_ip`) and ``X-Forwarded-For:`` +(:ts:cv:`proxy.config.http.insert_squid_x_forwarded_for`) likewise carry +the PROXY-Protocol source address whenever a PROXY-Protocol header is +present, mirroring the ``Forwarded: for=`` parameter. This behavior is +independent of the ``pp-clnt`` listener flag. + 2. Outbound PROXY protocol See :ts:cv:`proxy.config.http.proxy_protocol_out` for configuration information. diff --git a/doc/admin-guide/files/records.yaml.en.rst b/doc/admin-guide/files/records.yaml.en.rst index 671385b9bc..55e47e08d3 100644 --- a/doc/admin-guide/files/records.yaml.en.rst +++ b/doc/admin-guide/files/records.yaml.en.rst @@ -17,6 +17,8 @@ .. include:: ../../common.defs +.. default-domain:: cpp + .. configfile:: records.yaml records.yaml @@ -844,6 +846,15 @@ pp :ref:`Proxy Protocol <proxy-protocol>` for more details on how to configure this option properly. +pp-clnt + Use the source address from the Proxy Protocol header as the client IP + address for the transaction. This affects which address is reported by the + ``%<chi>`` log field, by the plugin client-address APIs (such as + :func:`TSHttpTxnClientAddrGet`), and by SNI / HTTP/2 peer ACLs and SSL + client-certificate validation, among other surfaces. Only meaningful in + combination with ``pp``. See :ref:`Proxy Protocol <proxy-protocol>` for the + full enumeration of behaviors gated by this flag. + tr-full Fully transparent. This is a convenience option and is identical to specifying both ``tr-in`` and ``tr-out``. diff --git a/doc/developer-guide/api/functions/TSHttpSsnClientAddrGet.en.rst b/doc/developer-guide/api/functions/TSHttpSsnClientAddrGet.en.rst new file mode 100644 index 0000000000..8008f11b0a --- /dev/null +++ b/doc/developer-guide/api/functions/TSHttpSsnClientAddrGet.en.rst @@ -0,0 +1,64 @@ +.. Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed + with this work for additional information regarding copyright + ownership. The ASF licenses this file to you under the Apache + License, Version 2.0 (the "License"); you may not use this file + except in compliance with the License. You may obtain a copy of + the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing + permissions and limitations under the License. + + +.. include:: ../../../common.defs + +.. default-domain:: cpp + +TSHttpSsnClientAddrGet +********************** + +Synopsis +======== + +.. code-block:: cpp + + #include <ts/ts.h> + +.. function:: struct sockaddr const * TSHttpSsnClientAddrGet(TSHttpSsn ssnp) + +Description +=========== + +Return the socket address of the client for the HTTP session :arg:`ssnp`. +The returned pointer references storage owned by |TS| and is only valid +for the duration of the current callback; plugins that need to keep the +value across callbacks must copy it into their own storage. + +This is the session-level counterpart of :func:`TSHttpTxnClientAddrGet` +and is appropriate when the caller has a session handle but no specific +transaction (for example, in session-level hooks). + +If the listener that accepted the connection has the ``pp-clnt`` flag set +and a PROXY Protocol header was successfully parsed, the returned address +is the PROXY-Protocol source address rather than the immediate TCP peer. +Without ``pp-clnt`` the returned address is the immediate TCP peer even +when PROXY Protocol is enabled. See :ref:`Proxy Protocol <proxy-protocol>` +for the full enumeration of surfaces gated by ``pp-clnt``. + +Return Value +============ + +A pointer to the client address, or ``nullptr`` if :arg:`ssnp` is invalid +or no client address is available. + +See Also +======== + +:manpage:`TSAPI(3ts)`, +:func:`TSHttpTxnClientAddrGet`, +:func:`TSNetVConnClientAddrGet` diff --git a/doc/developer-guide/api/functions/TSHttpTxnClientAddrGet.en.rst b/doc/developer-guide/api/functions/TSHttpTxnClientAddrGet.en.rst new file mode 100644 index 0000000000..c3866bf04d --- /dev/null +++ b/doc/developer-guide/api/functions/TSHttpTxnClientAddrGet.en.rst @@ -0,0 +1,64 @@ +.. Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed + with this work for additional information regarding copyright + ownership. The ASF licenses this file to you under the Apache + License, Version 2.0 (the "License"); you may not use this file + except in compliance with the License. You may obtain a copy of + the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing + permissions and limitations under the License. + + +.. include:: ../../../common.defs + +.. default-domain:: cpp + +TSHttpTxnClientAddrGet +********************** + +Synopsis +======== + +.. code-block:: cpp + + #include <ts/ts.h> + +.. function:: struct sockaddr const * TSHttpTxnClientAddrGet(TSHttpTxn txnp) + +Description +=========== + +Return the socket address of the client that initiated the transaction +:arg:`txnp`. The returned pointer references storage owned by |TS| and is +only valid for the duration of the current callback; plugins that need to +keep the value across callbacks must copy it into their own storage. + +The returned ``struct sockaddr`` is address-family agnostic. Inspect the +``sa_family`` field (or use the ``ats_ip_*`` helpers) to dispatch on IPv4 +versus IPv6. + +If the listener that accepted the connection has the ``pp-clnt`` flag set +and a PROXY Protocol header was successfully parsed, the returned address +is the PROXY-Protocol source address rather than the immediate TCP peer. +Without ``pp-clnt`` the returned address is the immediate TCP peer even +when PROXY Protocol is enabled. See :ref:`Proxy Protocol <proxy-protocol>` +for the full enumeration of surfaces gated by ``pp-clnt``. + +Return Value +============ + +A pointer to the client address, or ``nullptr`` if :arg:`txnp` is invalid +or no client address is available. + +See Also +======== + +:manpage:`TSAPI(3ts)`, +:func:`TSHttpSsnClientAddrGet`, +:func:`TSNetVConnClientAddrGet` diff --git a/doc/developer-guide/api/functions/TSNetVConnClientAddrGet.en.rst b/doc/developer-guide/api/functions/TSNetVConnClientAddrGet.en.rst new file mode 100644 index 0000000000..0c69232ddc --- /dev/null +++ b/doc/developer-guide/api/functions/TSNetVConnClientAddrGet.en.rst @@ -0,0 +1,66 @@ +.. Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed + with this work for additional information regarding copyright + ownership. The ASF licenses this file to you under the Apache + License, Version 2.0 (the "License"); you may not use this file + except in compliance with the License. You may obtain a copy of + the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing + permissions and limitations under the License. + + +.. include:: ../../../common.defs + +.. default-domain:: cpp + +TSNetVConnClientAddrGet +*********************** + +Synopsis +======== + +.. code-block:: cpp + + #include <ts/ts.h> + +.. function:: struct sockaddr const * TSNetVConnClientAddrGet(TSVConn vc) + +Description +=========== + +Return the socket address of the client for the network virtual connection +:arg:`vc`. The returned pointer references storage owned by |TS| and is +only valid for the duration of the current callback; plugins that need to +keep the value across callbacks must copy it into their own storage. + +This is the VConn-level counterpart of :func:`TSHttpTxnClientAddrGet` and +is appropriate in low-level hooks (for example, SSL or TCP hooks) where +no HTTP session or transaction is yet associated with the connection. + +If the listener that accepted the connection has the ``pp-clnt`` flag set +and a PROXY Protocol header was successfully parsed, the returned address +is the PROXY-Protocol source address rather than the immediate TCP peer. +Without ``pp-clnt`` the returned address is the immediate TCP peer even +when PROXY Protocol is enabled. Use ``TSNetVConnRemoteAddrGet()`` if the +immediate TCP peer is always required regardless of ``pp-clnt``. See +:ref:`Proxy Protocol <proxy-protocol>` for the full enumeration of +surfaces gated by ``pp-clnt``. + +Return Value +============ + +A pointer to the client address, or ``nullptr`` if :arg:`vc` is invalid +or no client address is available. + +See Also +======== + +:manpage:`TSAPI(3ts)`, +:func:`TSHttpTxnClientAddrGet`, +:func:`TSHttpSsnClientAddrGet` diff --git a/src/iocore/net/SSLNetVConnection.cc b/src/iocore/net/SSLNetVConnection.cc index 8b599e5e4a..ab5eb9d32b 100644 --- a/src/iocore/net/SSLNetVConnection.cc +++ b/src/iocore/net/SSLNetVConnection.cc @@ -376,11 +376,15 @@ SSLNetVConnection::read_raw_data() if (this->has_proxy_protocol(buffer, &r)) { Dbg(dbg_ctl_proxyprotocol, "ssl has proxy protocol header"); if (dbg_ctl_proxyprotocol.on()) { + IpEndpoint src; + src.sa = *(this->get_proxy_protocol_src_addr()); IpEndpoint dst; dst.sa = *(this->get_proxy_protocol_dst_addr()); - ip_port_text_buffer ipb1; - ats_ip_nptop(&dst, ipb1, sizeof(ipb1)); - DbgPrint(dbg_ctl_proxyprotocol, "ssl_has_proxy_v1, dest IP received [%s]", ipb1); + ip_port_text_buffer src_ipb, dst_ipb; + ats_ip_nptop(&src, src_ipb, sizeof(src_ipb)); + ats_ip_nptop(&dst, dst_ipb, sizeof(dst_ipb)); + DbgPrint(dbg_ctl_proxyprotocol, "ssl proxy protocol v%d header parsed: src=[%s] dst=[%s]", + static_cast<int>(this->get_proxy_protocol_version()), src_ipb, dst_ipb); } } else { Dbg(dbg_ctl_proxyprotocol, "proxy protocol was enabled, but Proxy Protocol header was not present"); diff --git a/src/proxy/http/HttpSM.cc b/src/proxy/http/HttpSM.cc index 481dcf6075..1f2e7b4b15 100644 --- a/src/proxy/http/HttpSM.cc +++ b/src/proxy/http/HttpSM.cc @@ -2409,7 +2409,10 @@ HttpSM::process_hostdb_info(HostDBRecord *record) } if (record && !record->is_failed()) { - t_state.dns_info.inbound_remote_addr = &t_state.client_info.src_addr.sa; + // HostDB parent-selection hashes by inbound_remote_addr; using + // effective_client_addr keeps sharding affinity tied to the real client + // when :pp-clnt is in effect, and is the TCP peer otherwise. + t_state.dns_info.inbound_remote_addr = &t_state.effective_client_addr.sa; if (!use_client_addr) { t_state.dns_info.set_active( record->select_best_http(ts_clock::now(), t_state.txn_conf->down_server_timeout, t_state.dns_info.inbound_remote_addr)); @@ -5757,7 +5760,11 @@ HttpSM::do_http_server_open(bool raw, bool only_direct) opt.local_ip = outbound_ip; } else if (_ua.get_txn()->is_outbound_transparent()) { opt.addr_binding = NetVCOptions::FOREIGN_ADDR; - opt.local_ip = t_state.client_info.src_addr; + // Use effective_client_addr so that listeners with :pp-clnt bind the + // outbound socket to the real client IP rather than the immediate TCP + // peer; without :pp-clnt, effective_client_addr is the TCP peer, so + // existing transparent deployments are unaffected. + opt.local_ip = t_state.effective_client_addr; /* If the connection is server side transparent, we can bind to the port that the client chose instead of randomly assigning one at the proxy. This is controlled by the 'use_client_source_port' @@ -7867,7 +7874,9 @@ HttpSM::update_stats() } int status = static_cast<int>(status_code); char client_ip[INET6_ADDRSTRLEN]; - ats_ip_ntop(&t_state.client_info.src_addr, client_ip, sizeof(client_ip)); + // Log the operator-visible client IP (matches %<chi>) so the slow-request + // line is consistent with squid logs when :pp-clnt is in effect. + ats_ip_ntop(&t_state.effective_client_addr, client_ip, sizeof(client_ip)); Error("[%" PRId64 "] Slow Request: " "client_ip: %s:%u " "protocol: %s " @@ -7900,7 +7909,7 @@ HttpSM::update_stats() "sm_finish: %.3f " "plugin_active: %.3f " "plugin_total: %.3f", - sm_id, client_ip, t_state.client_info.src_addr.host_order_port(), + sm_id, client_ip, t_state.effective_client_addr.host_order_port(), _ua.get_txn() ? _ua.get_txn()->get_protocol_string() : "-1", url_string, status, unique_id_string, redirection_tries, client_response_body_bytes, fd, t_state.client_info.state, t_state.server_info.state, milestones.difference_sec(TS_MILESTONE_TLS_HANDSHAKE_START, TS_MILESTONE_TLS_HANDSHAKE_END), diff --git a/src/proxy/http/HttpTransact.cc b/src/proxy/http/HttpTransact.cc index cd1049d1a6..224c728098 100644 --- a/src/proxy/http/HttpTransact.cc +++ b/src/proxy/http/HttpTransact.cc @@ -5496,12 +5496,23 @@ HttpTransact::add_client_ip_to_outgoing_request(State *s, HTTPHdr *request) char ip_string[INET6_ADDRSTRLEN + 1] = {'\0'}; size_t ip_string_size = 0; - if (!ats_is_ip(&s->client_info.src_addr.sa)) { + // Prefer the PROXY-Protocol source address when one is present, so that + // Client-ip and X-Forwarded-For agree with Forwarded: for= regardless of + // whether the listener carries the :pp-clnt flag. + IpEndpoint src_addr = s->client_info.src_addr; + if (s->state_machine->get_ua_txn() && s->state_machine->get_ua_txn()->get_netvc()) { + const ProxyProtocol &pp = s->state_machine->get_ua_txn()->get_netvc()->get_proxy_protocol_info(); + if (pp.version != ProxyProtocolVersion::UNDEFINED) { + src_addr = pp.src_addr; + } + } + + if (!ats_is_ip(&src_addr.sa)) { return; } // Always prepare the IP string. - if (ats_ip_ntop(&s->client_info.src_addr.sa, ip_string, sizeof(ip_string)) != nullptr) { + if (ats_ip_ntop(&src_addr.sa, ip_string, sizeof(ip_string)) != nullptr) { ip_string_size += strlen(ip_string); } else { // Failure, omg diff --git a/tests/gold_tests/proxy_protocol/gold/access-cp.gold b/tests/gold_tests/proxy_protocol/gold/access-cp.gold index b84f06e4d2..28d355ac6a 100644 --- a/tests/gold_tests/proxy_protocol/gold/access-cp.gold +++ b/tests/gold_tests/proxy_protocol/gold/access-cp.gold @@ -5,3 +5,4 @@ 198.51.100.1 198.51.100.1 127.0.0.1 127.0.0.1 0 127.0.0.1 127.0.0.1 0 127.0.0.1 +198.51.100.1 198.51.100.1 127.0.0.1 diff --git a/tests/gold_tests/proxy_protocol/gold/access-nocp.gold b/tests/gold_tests/proxy_protocol/gold/access-nocp.gold index 7d705c4735..5511d71605 100644 --- a/tests/gold_tests/proxy_protocol/gold/access-nocp.gold +++ b/tests/gold_tests/proxy_protocol/gold/access-nocp.gold @@ -5,3 +5,4 @@ 127.0.0.1 198.51.100.1 127.0.0.1 127.0.0.1 0 127.0.0.1 127.0.0.1 0 127.0.0.1 +127.0.0.1 198.51.100.1 127.0.0.1 diff --git a/tests/gold_tests/proxy_protocol/proxy_protocol.test.py b/tests/gold_tests/proxy_protocol/proxy_protocol.test.py index bd008b9a6f..c596e25110 100644 --- a/tests/gold_tests/proxy_protocol/proxy_protocol.test.py +++ b/tests/gold_tests/proxy_protocol/proxy_protocol.test.py @@ -52,6 +52,8 @@ class ProxyProtocolInTest: { "proxy.config.http.proxy_protocol_allowlist": "127.0.0.1", "proxy.config.http.insert_forwarded": "for|by=ip|proto", + "proxy.config.http.insert_client_ip": 2, + "proxy.config.http.insert_squid_x_forwarded_for": 1, "proxy.config.ssl.server.cert.path": f"{self.ts.Variables.SSLDir}", "proxy.config.ssl.server.private_key.path": f"{self.ts.Variables.SSLDir}", "proxy.config.diags.debug.enabled": 1, diff --git a/tests/gold_tests/proxy_protocol/replay/proxy_protocol_in.replay.yaml b/tests/gold_tests/proxy_protocol/replay/proxy_protocol_in.replay.yaml index b8b85a2c4a..180e749660 100644 --- a/tests/gold_tests/proxy_protocol/replay/proxy_protocol_in.replay.yaml +++ b/tests/gold_tests/proxy_protocol/replay/proxy_protocol_in.replay.yaml @@ -229,3 +229,49 @@ sessions: proxy-response: status: 200 + + # Test 6: Incoming PROXY Protocol v2 on SSL port, with arbitrary source and + # destination address in PROXY message. Verifies that Client-ip and + # X-Forwarded-For (the legacy outbound header writers) carry the PROXY + # Protocol source address, in addition to the Forwarded: for= field. This + # exercises HttpTransact::add_client_ip_to_outgoing_request together with + # HttpTransactHeaders::add_forwarded_field_to_request. + - protocol: + - name: http + version: 1 + - name: tls + - name: proxy-protocol + version: 2 + src-addr: 198.51.100.1:51137 + dst-addr: 198.51.100.2:443 + - name: tcp + - name: ip + + transactions: + - client-request: + method: GET + version: "1.1" + url: /get + headers: + fields: + - [uuid, 8] + - [Host, "127.0.0.1:443"] + + proxy-request: + headers: + fields: + - [ + Forwarded, + { + value: for=198.51.100.1;by=127.0.0.1;proto=https, + as: equal, + }, + ] + - [X-Forwarded-For, { value: 198.51.100.1, as: equal }] + - [Client-ip, { value: 198.51.100.1, as: equal }] + + server-response: + status: 200 + + proxy-response: + status: 200
