PR #22558 opened by Nariman-Sayed URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22558 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22558.patch
Some WebRTC servers such as Pion send STUN packets concurrently during the DTLS handshake. Unlike OpenSSL and GnuTLS which filter non-DTLS packets internally, mbedtls passes all received UDP packets directly to its DTLS state machine, causing the handshake to fail. Fix this by using the f_recv_timeout callback in mbedtls_ssl_set_bio, which returns MBEDTLS_ERR_SSL_WANT_READ to discard non-DTLS packets such as STUN. ff_network_wait_fd_timeout is used to wait for data with a timeout, avoiding indefinite blocking if the peer stops sending, as specified by RFC 5764 Section 5.1.2. Signed-off-by: Nariman-Sayed <[email protected]> >From 2d04c8090d5629b4c26ed1de4f97ecdbb9c64797 Mon Sep 17 00:00:00 2001 From: Nariman-Sayed <[email protected]> Date: Fri, 20 Mar 2026 05:05:41 +0200 Subject: [PATCH] avformat/tls_mbedtls: fix DTLS handshake failure when receiving non-DTLS packets Some WebRTC servers such as Pion send STUN packets concurrently during the DTLS handshake. Unlike OpenSSL and GnuTLS which filter non-DTLS packets internally, mbedtls passes all received UDP packets directly to its DTLS state machine, causing the handshake to fail. Fix this by using the f_recv_timeout callback in mbedtls_ssl_set_bio, which returns MBEDTLS_ERR_SSL_WANT_READ to discard non-DTLS packets such as STUN. ff_network_wait_fd_timeout is used to wait for data with a timeout, avoiding indefinite blocking if the peer stops sending, as specified by RFC 5764 Section 5.1.2. Signed-off-by: Nariman-Sayed <[email protected]> --- libavformat/tls_mbedtls.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/libavformat/tls_mbedtls.c b/libavformat/tls_mbedtls.c index d2480ef25b..f4c9264ef9 100644 --- a/libavformat/tls_mbedtls.c +++ b/libavformat/tls_mbedtls.c @@ -376,12 +376,23 @@ static int mbedtls_send(void *ctx, const unsigned char *buf, size_t len) return handle_transport_error(h, "ffurl_write", MBEDTLS_ERR_SSL_WANT_WRITE, ret); } -static int mbedtls_recv(void *ctx, unsigned char *buf, size_t len) +static int mbedtls_recv_timeout(void *ctx, unsigned char *buf, size_t len, uint32_t timeout) { - TLSContext *tls_ctx = (TLSContext*) ctx; + TLSContext *tls_ctx = (TLSContext *) ctx; TLSShared *shr = &tls_ctx->tls_shared; URLContext *h = shr->is_dtls ? shr->udp : shr->tcp; - int ret = ffurl_read(h, buf, len); + int fd = ffurl_get_file_handle(h); + int ret; + + /* timeout is in milliseconds from mbedtls, convert to microseconds. + * Use a default of 5 seconds if timeout is 0. */ + int64_t timeout_us = timeout ? (int64_t)timeout * 1000 : 5000000; + + ret = ff_network_wait_fd_timeout(fd, 0, timeout_us, &h->interrupt_callback); + if (ret < 0) + return MBEDTLS_ERR_SSL_TIMEOUT; + + ret = ffurl_read(h, buf, len); if (ret >= 0) { if (shr->is_dtls && shr->listen && !tls_ctx->dest_addr_len) { int err_ret; @@ -394,6 +405,9 @@ static int mbedtls_recv(void *ctx, unsigned char *buf, size_t len) } av_log(tls_ctx, AV_LOG_TRACE, "Set UDP remote addr on UDP socket, now 'connected'\n"); } + /* Skip non-DTLS packets such as STUN to avoid handshake failures. */ + if (shr->is_dtls && (buf[0] < 20 || buf[0] > 63)) + return MBEDTLS_ERR_SSL_WANT_READ; return ret; } if (h->max_packet_size && len > h->max_packet_size) @@ -658,7 +672,7 @@ static int tls_open(URLContext *h, const char *uri, int flags, AVDictionary **op } // set I/O functions to use FFmpeg internal code for transport layer - mbedtls_ssl_set_bio(&tls_ctx->ssl_context, tls_ctx, mbedtls_send, mbedtls_recv, NULL); + mbedtls_ssl_set_bio(&tls_ctx->ssl_context, tls_ctx, mbedtls_send, NULL, mbedtls_recv_timeout); if (shr->is_dtls) { mbedtls_ssl_set_timer_cb(&tls_ctx->ssl_context, &tls_ctx->timer, mbedtls_timing_set_delay, mbedtls_timing_get_delay); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
