Hi,
On 3/26/24 15:50, Mina Almasry wrote:
> Add documentation outlining the usage and details of devmem TCP.
>
> Signed-off-by: Mina Almasry <almasrym...@google.com>
>
> ---
>
> v7:
> - Applied docs suggestions (Jakub).
>
> v2:
>
> - Missing spdx (simon)
> - add to index.rst (simon)
>
> ---
> Documentation/networking/devmem.rst | 256 ++++++++++++++++++++++++++++
> Documentation/networking/index.rst | 1 +
> 2 files changed, 257 insertions(+)
> create mode 100644 Documentation/networking/devmem.rst
>
> diff --git a/Documentation/networking/devmem.rst
> b/Documentation/networking/devmem.rst
> new file mode 100644
> index 000000000000..b0899e8e9e83
> --- /dev/null
> +++ b/Documentation/networking/devmem.rst
> @@ -0,0 +1,256 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=================
> +Device Memory TCP
> +=================
> +
> +
> +Intro
> +=====
> +
> +Device memory TCP (devmem TCP) enables receiving data directly into device
> +memory (dmabuf). The feature is currently implemented for TCP sockets.
> +
> +
> +Opportunity
> +-----------
> +
> +A large number of data transfers have device memory as the source and/or
> +destination. Accelerators drastically increased the prevalence of such
> +transfers. Some examples include:
> +
> +- Distributed training, where ML accelerators, such as GPUs on different
> hosts,
> + exchange data.
> +
> +- Distributed raw block storage applications transfer large amounts of data
> with
> + remote SSDs, much of this data does not require host processing.
SSDs. Much
> +
> +Typically the Device-to-Device data transfers the network are implemented as
> the
in the network
?
> +following low level operations: Device-to-Host copy, Host-to-Host network
low-level
> +transfer, and Host-to-Device copy.
> +
> +The flow involving host copies is suboptimal, especially for bulk data
> transfers,
> +and can put significant strains on system resources such as host memory
> +bandwidth and PCIe bandwidth.
> +
> +Devmem TCP optimizes this use case by implementing socket APIs that enable
> +the user to receive incoming network packets directly into device memory.
> +
> +Packet payloads go directly from the NIC to device memory.
> +
> +Packet headers go to host memory and are processed by the TCP/IP stack
> +normally. The NIC must support header split to achieve this.
> +
> +Advantages:
> +
> +- Alleviate host memory bandwidth pressure, compared to existing
> + network-transfer + device-copy semantics.
> +
> +- Alleviate PCIe bandwidth pressure, by limiting data transfer to the lowest
> + level of the PCIe tree, compared to traditional path which sends data
> through
to the
> + the root complex.
> +
> +
> +More Info
> +---------
> +
> + slides, video
> + https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html
> +
> + patchset
> + [RFC PATCH v6 00/12] Device Memory TCP
> +
> https://lore.kernel.org/netdev/20240305020153.2787423-1-almasrym...@google.com/
> +
> +
> +Interface
> +=========
> +
> +Example
> +-------
> +
> +tools/testing/selftests/net/ncdevmem.c:do_server shows an example of setting
> up
> +the RX path of this API.
> +
> +NIC Setup
> +---------
> +
> +Header split, flow steering, & RSS are required features for devmem TCP.
> +
> +Header split is used to split incoming packets into a header buffer in host
> +memory, and a payload buffer in device memory.
> +
> +Flow steering & RSS are used to ensure that only flows targeting devmem land
> on> +RX queue bound to devmem.
an RX queue
?
> +
> +Enable header split & flow steering::
> +
> + # enable header split
> + ethtool -G eth1 tcp-data-split on
> +
> +
> + # enable flow steering
> + ethtool -K eth1 ntuple on
> +
> +Configure RSS to steer all traffic away from the target RX queue (queue 15 in
> +this example)::
> +
> + ethtool --set-rxfh-indir eth1 equal 15
> +
> +
> +The user must bind a dmabuf to any number of RX queues on a given NIC using
> +netlink API::
the netlink API::
> +
> + /* Bind dmabuf to NIC RX queue 15 */
> + struct netdev_queue *queues;
> + queues = malloc(sizeof(*queues) * 1);
> +
> + queues[0]._present.type = 1;
> + queues[0]._present.idx = 1;
> + queues[0].type = NETDEV_RX_QUEUE_TYPE_RX;
> + queues[0].idx = 15;
> +
> + *ys = ynl_sock_create(&ynl_netdev_family, &yerr);
> +
> + req = netdev_bind_rx_req_alloc();
> + netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */);
> + netdev_bind_rx_req_set_dmabuf_fd(req, dmabuf_fd);
> + __netdev_bind_rx_req_set_queues(req, queues, n_queue_index);
> +
> + rsp = netdev_bind_rx(*ys, req);
> +
> + dmabuf_id = rsp->dmabuf_id;
> +
> +
> +The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf
> +that has been bound.
> +
> +Socket Setup
> +------------
> +
> +The socket must be flow steering to the dmabuf bound RX queue::
flow steered
?
> +
> + ethtool -N eth1 flow-type tcp4 ... queue 15,
> +
> +
> +Receiving data
> +--------------
> +
> +The user application must signal to the kernel that it is capable of
> receiving
> +devmem data by passing the MSG_SOCK_DEVMEM flag to recvmsg::
> +
> + ret = recvmsg(fd, &msg, MSG_SOCK_DEVMEM);
> +
> +Applications that do not specify the MSG_SOCK_DEVMEM flag will receive an
> EFAULT
> +on devmem data.
> +
> +Devmem data is received directly into the dmabuf bound to the NIC in 'NIC
> +Setup', and the kernel signals such to the user via the SCM_DEVMEM_* cmsgs::
> +
> + for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) {
> + if (cm->cmsg_level != SOL_SOCKET ||
> + (cm->cmsg_type != SCM_DEVMEM_DMABUF &&
> + cm->cmsg_type != SCM_DEVMEM_LINEAR))
> + continue;
> +
> + dmabuf_cmsg = (struct dmabuf_cmsg *)CMSG_DATA(cm);
> +
> + if (cm->cmsg_type == SCM_DEVMEM_DMABUF) {
> + /* Frag landed in dmabuf.
> + *
> + * dmabuf_cmsg->dmabuf_id is the dmabuf the
> + * frag landed on.
> + *
> + * dmabuf_cmsg->frag_offset is the offset into
> + * the dmabuf where the frag starts.
> + *
> + * dmabuf_cmsg->frag_size is the size of the
> + * frag.
> + *
> + * dmabuf_cmsg->frag_token is a token used to
> + * refer to this frag for later freeing.
> + */
> +
> + struct dmabuf_token token;
> + token.token_start = dmabuf_cmsg->frag_token;
> + token.token_count = 1;
> + continue;
> + }
> +
> + if (cm->cmsg_type == SCM_DEVMEM_LINEAR)
> + /* Frag landed in linear buffer.
> + *
> + * dmabuf_cmsg->frag_size is the size of the
> + * frag.
> + */
> + continue;
> +
> + }
> +
> +Applications may receive 2 cmsgs:
> +
> +- SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf
> indicated
> + by dmabuf_id.
> +
> +- SCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer.
> + This typically happens when the NIC is unable to split the packet at the
> + header boundary, such that part (or all) of the payload landed in host
> + memory.
> +
> +Applications may receive no SO_DEVMEM_* cmsgs. That indicates non-devmem,
> +regular TCP data that landed on an RX queue not bound to a dmabuf.
> +
> +
> +Freeing frags
> +-------------
> +
> +Frags received via SCM_DEVMEM_DMABUF are pinned by the kernel while the user
> +processes the frag. The user must return the frag to the kernel via
> +SO_DEVMEM_DONTNEED::
> +
> + ret = setsockopt(client_fd, SOL_SOCKET, SO_DEVMEM_DONTNEED, &token,
> + sizeof(token));
> +
> +The user must ensure the tokens are returned to the kernel in a timely
> manner.
> +Failure to do so will exhaust the limited dmabuf that is bound to the RX
> queue
> +and will lead to packet drops.
> +
> +
> +Implementation & Caveats
> +========================
> +
> +Unreadable skbs
> +---------------
> +
> +Devmem payloads are inaccessible to the kernel processing the packets. This
> +results in a few quirks for payloads of devmem skbs:
> +
> +- Loopback is not functional. Loopback relies on copying the payload, which
> is
> + not possible with devmem skbs.
> +
> +- Software checksum calculation fails.
> +
> +- TCP Dump and bpf can't access devmem packet payloads.
> +
> +
> +Testing
> +=======
> +
> +More realistic example code can be found in the kernel source under
> +tools/testing/selftests/net/ncdevmem.c
> +
> +ncdevmem is a devmem TCP netcat. It works very similarly to netcat, but
> +receives data directly into a udmabuf.
> +
> +To run ncdevmem, you need to run it a server on the machine under test, and
> you
it on a server
> +need to run netcat on a peer to provide the TX data.
> +
> +ncdevmem has a validation mode as well that expects a repeating pattern of
> +incoming data and validates it as such::
> +
> + # On server:
> + ncdevmem -s <server IP> -c <client IP> -f eth1 -d 3 -n 0000:06:00.0 -l \
> + -p 5201 -v 7
> +
> + # On client:
> + yes $(echo -e \\x01\\x02\\x03\\x04\\x05\\x06) | \
> + tr \\n \\0 | head -c 5G | nc <server IP> 5201 -p 5201
--
#Randy
