On 11/10/20 5:20 PM, Victor Stewart wrote: > On Wed, Nov 11, 2020 at 12:09 AM Jonathan Lemon > <[email protected]> wrote: >> >> On Sun, Nov 08, 2020 at 05:04:41PM +0000, Victor Stewart wrote: >>> hi all, >>> >>> i'm seeking input / comment on the idea of implementing full fledged >>> zerocopy UDP networking that uses persistent buffers allocated in >>> userspace... before I go off on a solo tangent with my first patches >>> lol. >>> >>> i'm sure there's been lots of thought/discussion on this before. of >>> course Willem added MSG_ZEROCOPY on the send path (pin buffers on >>> demand / per send). and something similar to what I speak of exists >>> with TCP_ZEROCOPY_RECEIVE. >>> >>> i envision something like a new flag like MSG_ZEROCOPY_FIXED that >>> "does the right thing" in the send vs recv paths. >> >> See the netgpu patches that I posted earlier; these will handle >> protocol independent zerocopy sends/receives. I do have a working >> UDP receive implementation which will be posted with an updated >> patchset. > > amazing i'll check it out. thanks. > > does your udp zerocopy receive use mmap-ed buffers then vm_insert_pfn > / remap_pfn_range to remap the physical pages of the received payload > into the memory submitted by recvmsg for reception? > > https://lore.kernel.org/io-uring/[email protected]/T/#t > > ^^ and check the thread from today on the io_uring mailing list going > into the mechanics of zerocopy sendmsg i have in mind. > > (TLDR; i think it should be io_uring "only" so that we can collapse it > into a single completion event, aka when the NIC ACKs the > transmission. and exploiting the asynchrony of io_uring is the only > way to do this? so you'd submit your sendmsg operation to io_uring and > instead of receiving a completion event when the send gets enqueued, > you'd only get it upon failure or NIC ACK). >
Do you have a working implementation? Right now, io_uring send / sendmsg can return incomplete sends (only partial buffer is sent): https://lore.kernel.org/io-uring/[email protected]/ That will need to be solved. A simple solution is to track the offset and resubmit: https://lore.kernel.org/io-uring/[email protected]/
