Hi Karol,
On Thu, Jul 23, 2026 at 06:07:39PM +0200, Karol Kucharski wrote:
> Hi,
>
> I tested a simple configuration where HAProxy connects to a backend
> server
> using TLS 1.3 with ktls and splice options enabled.
>
> haproxy.cfg
> ```
> global
> expose-experimental-directives
> nbthread 1
>
> defaults
> mode tcp
> timeout connect 1s
> timeout client 1s
> timeout server 1s
> option splice-request
> option splice-response
>
> frontend tcp_in
> bind *:8080
> default_backend tls_out
>
> backend tls_out
> server server 127.0.0.1:4043 ssl verify none ktls on force-tlsv13
> ```
>
> After the initial handshake phase, the server sends two
> new_session_ticket
> handshake messages.
> HAProxy tries to receive those messages with the splice syscall which
> results
> in the EINVAL error.
> There is a comment in the HAProxy source (in the raw_sock_to_pipe
> function)
> regarding such situation:
>
> ```
> * EINVAL, however, is a bit special.
> * If we're trying to splice from a KTLS
> * socket, the kernel may return EINVAL
> * to signal that the current TLS record
> * is not application data, and that we
> * have to call recvmsg() to get it.
> * This is not really an error, and doesn't
> * mean we won't be able to splice later.
> * Choosing EINVAL there is a bit unfortunate,
> * because it can mean many things, but we
> * should not get it for any other reason.
> */
> ```
>
> Using strace I've seen only repeated splice calls returning EINVAL,
> without
> any recvmsg call to handle the failed splice call.
>
> ```
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=429214144}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=429392356}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=429572280}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=429753718}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=429931308}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=430114691}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=430520235}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=430703514}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=430879870}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=431062900}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=431239849}) =
> 0
> splice(9, NULL, 11, NULL, 1073741824, SPLICE_F_MOVE|SPLICE_F_NONBLOCK) =
> -1 EINVAL (Invalid argument)
> clock_gettime(CLOCK_THREAD_CPUTIME_ID, {tv_sec=0, tv_nsec=431422559}) =
> 0
> epoll_wait(4, [], 200, 0) = 0
> ```
>
> I've tested versions 3.4.2 and git master, both with OpenSSL 3.5.6 and
> AWS-LC 5.1.0.
>
> I've added a simple hack in the raw_sock_to_pipe function:
> ```
> diff --git a/src/raw_sock.c b/src/raw_sock.c
> index 33fb8f5c3..1552e44bc 100644
> --- a/src/raw_sock.c
> +++ b/src/raw_sock.c
> @@ -137,6 +137,21 @@ int raw_sock_to_pipe(struct connection *conn, void
> *xprt_ctx, struct pipe *pipe,
> */
> if (errno != EINVAL)
> retval = -1;
> + else
> + {
> + char dbuff[1024];
> + char cbuff[64];
> + struct msghdr msg;
> + struct iovec iov;
> + memset(&msg, 0, sizeof(msg));
> + iov.iov_base = dbuff;
> + iov.iov_len = sizeof(dbuff);
> + msg.msg_control = cbuff;
> + msg.msg_controllen =
> sizeof(cbuff);
> + msg.msg_iov = &iov;
> + msg.msg_iovlen = 1;
> + (void)recvmsg(conn->handle.fd,
> &msg, 0);
> + }
> goto leave;
> }
> else if (errno == EINTR) {
> ```
> And it seems to work (HAProxy receives the data from the server and
> forwards
> it to the client).
> Of course it's not a proper solution but unfortunately I'm not familiar
> enough
> with the HAProxy architecture to make a proper fix.
Thanks a lot for reporting!
There is code in ssl_sock_to_buf() to take care of exactly that problem,
but obviously you managed, with a bit of a different timing, to get that
case when haproxy switched to using splicing already, so it will do no
good here. I'm actually working in that area, and I have a fix that
should take care of that for OpenSSL already. Now I'm gonna fix that for
AWS-LC too, and I'll let you know when I think it should now work.
Thanks again!
Olivier
