Re: [Linaro-mm-sig] Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-27 Thread Christian König
Am 20.07.23 um 01:24 schrieb Jason Gunthorpe: On Wed, Jul 19, 2023 at 10:57:11AM -0700, Stephen Hemminger wrote: Naive idea. Would it be possible for process to use mmap() on the GPU memory and then do zero copy TCP receive some how? Or is this what is being proposed. It could be possible,

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-19 Thread Jason Gunthorpe
On Wed, Jul 19, 2023 at 10:57:11AM -0700, Stephen Hemminger wrote: > Naive idea. > Would it be possible for process to use mmap() on the GPU memory and then > do zero copy TCP receive some how? Or is this what is being proposed. It could be possible, but currently there is no API to recover the

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-19 Thread Jakub Kicinski
On Wed, 19 Jul 2023 08:10:58 -0700 Mina Almasry wrote: > From Jakub and David's comments it sounds (if I understood correctly), > you'd like to tie the dma-buf bind/unbind functions to the lifetime of > a netlink socket, rather than a struct file like I was thinking. That > does sound cleaner, but

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-19 Thread Stephen Hemminger
On Wed, 19 Jul 2023 08:10:58 -0700 Mina Almasry wrote: > On Tue, Jul 18, 2023 at 3:45 PM Jakub Kicinski wrote: > > > > On Tue, 18 Jul 2023 16:35:17 -0600 David Ahern wrote: > > > I do not see how 1 RSS context (or more specifically a h/w Rx queue) can > > > be used properly with memory from

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-19 Thread Mina Almasry
On Tue, Jul 18, 2023 at 3:45 PM Jakub Kicinski wrote: > > On Tue, 18 Jul 2023 16:35:17 -0600 David Ahern wrote: > > I do not see how 1 RSS context (or more specifically a h/w Rx queue) can > > be used properly with memory from different processes (or dma-buf > > references). Right, my experience

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jakub Kicinski
On Tue, 18 Jul 2023 16:35:17 -0600 David Ahern wrote: > I do not see how 1 RSS context (or more specifically a h/w Rx queue) can > be used properly with memory from different processes (or dma-buf > references). When the process dies, that memory needs to be flushed from > the H/W queues. Queues

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread David Ahern
On 7/18/23 12:29 PM, Jakub Kicinski wrote: > On Tue, 18 Jul 2023 12:20:59 -0600 David Ahern wrote: >> On 7/18/23 12:15 PM, Jakub Kicinski wrote: >>> On Tue, 18 Jul 2023 15:06:29 -0300 Jason Gunthorpe wrote: netlink feels like a weird API choice for that, in particular it would be

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jakub Kicinski
On Tue, 18 Jul 2023 12:20:59 -0600 David Ahern wrote: > On 7/18/23 12:15 PM, Jakub Kicinski wrote: > > On Tue, 18 Jul 2023 15:06:29 -0300 Jason Gunthorpe wrote: > >> netlink feels like a weird API choice for that, in particular it would > >> be really wrong to somehow bind the lifecycle of a

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread David Ahern
On 7/18/23 12:15 PM, Jakub Kicinski wrote: > On Tue, 18 Jul 2023 15:06:29 -0300 Jason Gunthorpe wrote: >> netlink feels like a weird API choice for that, in particular it would >> be really wrong to somehow bind the lifecycle of a netlink object to a >> process. > > Netlink is the right API, life

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jakub Kicinski
On Tue, 18 Jul 2023 15:06:29 -0300 Jason Gunthorpe wrote: > netlink feels like a weird API choice for that, in particular it would > be really wrong to somehow bind the lifecycle of a netlink object to a > process. Netlink is the right API, life cycle of objects can be easily tied to a netlink

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jason Gunthorpe
On Tue, Jul 18, 2023 at 10:36:52AM -0700, Mina Almasry wrote: > That is specific to this proposal, and will likely be very different > in future ones. I thought the dma-buf pages approach was extensible > and the uapi belonged somewhere in dma-buf. Clearly not. The next > proposal, I think, will

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Mina Almasry
On Sun, Jul 16, 2023 at 7:41 PM Andy Lutomirski wrote: > > On 7/10/23 15:32, Mina Almasry wrote: > > * TL;DR: > > > > Device memory TCP (devmem TCP) is a proposal for transferring data to and/or > > from device memory efficiently, without bouncing the data to a host memory > > buffer. > > (I'm

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jakub Kicinski
On Sun, 16 Jul 2023 19:41:28 -0700 Andy Lutomirski wrote: > I'm wondering if a more capable if somewhat higher latency model could > work where the NIC stores received packets in its own device memory. > Then userspace (or the kernel or a driver or whatever) could initiate a > separate DMA from

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-16 Thread Andy Lutomirski
On 7/10/23 15:32, Mina Almasry wrote: * TL;DR: Device memory TCP (devmem TCP) is a proposal for transferring data to and/or from device memory efficiently, without bouncing the data to a host memory buffer. (I'm writing this as someone who might plausibly use this mechanism, but I don't

[RFC PATCH 00/10] Device Memory TCP

2023-07-10 Thread Mina Almasry
* TL;DR: Device memory TCP (devmem TCP) is a proposal for transferring data to and/or from device memory efficiently, without bouncing the data to a host memory buffer. * Problem: A large amount of data transfers have device memory as the source and/or destination. Accelerators drastically