Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-28 Thread Jonathan Cameron via
m Sethi ; Andy Currid ; > > > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > > > ; ira.we...@intel.com; navneet.si...@intel.com > > > Subject: Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory > > > > > > > >

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-28 Thread Jonathan Cameron via
i ; Andy Currid ; > > > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > > > ; ira.we...@intel.com; navneet.si...@intel.com > > > Subject: Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory > > > > > > > > > On Wed, 27 Sep 2023 10:5

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-28 Thread Jonathan Cameron via
non.zha...@gmail.com; > > peter.mayd...@linaro.org; a...@anisinha.ca; Aniket Agashe > > ; Neo Jia ; Kirti Wankhede > > ; Tarun Gupta (SW-GPU) ; > > Vikram Sethi ; Andy Currid ; > > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > > ; ira.we...@intel.com; navneet.si.

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Alex Williamson
non.zha...@gmail.com; > > peter.mayd...@linaro.org; a...@anisinha.ca; Aniket Agashe > > ; Neo Jia ; Kirti Wankhede > > ; Tarun Gupta (SW-GPU) ; > > Vikram Sethi ; Andy Currid ; > > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > > ; ira.we...@intel.com; navneet.si.

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Jason Gunthorpe
non.zha...@gmail.com; > > peter.mayd...@linaro.org; a...@anisinha.ca; Aniket Agashe > > ; Neo Jia ; Kirti Wankhede > > ; Tarun Gupta (SW-GPU) ; > > Vikram Sethi ; Andy Currid ; > > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > > ; ira.we...@intel.com; navneet

RE: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Vikram Sethi
Neo Jia ; Kirti Wankhede > ; Tarun Gupta (SW-GPU) ; > Vikram Sethi ; Andy Currid ; > qemu-...@nongnu.org; qemu-devel@nongnu.org; Gavin Shan > ; ira.we...@intel.com; navneet.si...@intel.com > Subject: Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory > > > On Wed, 27

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Alex Williamson
On Wed, 27 Sep 2023 10:53:36 -0300 Jason Gunthorpe wrote: > On Wed, Sep 27, 2023 at 12:33:18PM +0100, Jonathan Cameron wrote: > > > CXL accelerators / GPUs etc are a different question but who has one > > of those anyway? :) > > That's exactly what I mean when I say CXL will need it too. I ke

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Jason Gunthorpe
On Wed, Sep 27, 2023 at 12:33:18PM +0100, Jonathan Cameron wrote: > CXL accelerators / GPUs etc are a different question but who has one > of those anyway? :) That's exactly what I mean when I say CXL will need it too. I keep describing this current Grace & Hopper as pre-CXL HW. You can easially

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Jonathan Cameron via
On Wed, 27 Sep 2023 07:14:28 + Ankit Agrawal wrote: > > > > > > Based on the suggestions here, can we consider something like the > > > following? > > > 1. Introduce a new -numa subparam 'devnode', which tells Qemu to mark > > > the node with MEM_AFFINITY_HOTPLUGGABLE in the SRAT's memory aff

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-27 Thread Ankit Agrawal
> > > > Based on the suggestions here, can we consider something like the > > following? > > 1. Introduce a new -numa subparam 'devnode', which tells Qemu to mark > > the node with MEM_AFFINITY_HOTPLUGGABLE in the SRAT's memory affinity > > structure to make it hotpluggable. > > Is that "devnode=on

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-26 Thread Alex Williamson
On Tue, 26 Sep 2023 18:54:53 +0200 David Hildenbrand wrote: > On 26.09.23 16:52, Ankit Agrawal wrote: > > Good idea.  Fundamentally the device should not be creating NUMA > > nodes, the VM should be configured with NUMA nodes and the device > > memory associated with those nodes. >

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-26 Thread David Hildenbrand
On 26.09.23 16:52, Ankit Agrawal wrote: Good idea.  Fundamentally the device should not be creating NUMA nodes, the VM should be configured with NUMA nodes and the device memory associated with those nodes. +1. That would also make it fly with DIMMs and virtio-mem, where you would want NUMA-les

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-26 Thread Ankit Agrawal
Good idea.  Fundamentally the device should not be creating NUMA nodes, the VM should be configured with NUMA nodes and the device memory associated with those nodes. >>> >>> +1. That would also make it fly with DIMMs and virtio-mem, where you >>> would want NUMA-less nodes ass well

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-22 Thread David Hildenbrand
On 22.09.23 10:11, Ankit Agrawal wrote: Typically, a command line for a virt machine with NUMA nodes would look like : -object memory-backend-ram,id=ram-node0,size=1G \ -numa node,nodeid=0,memdev=ram-node0 \ -object memory-backend-ram,id=ram-node1,size=1G \ -numa node,nodeid

RE: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-22 Thread Ankit Agrawal
> >> > >> Typically, a command line for a virt machine with NUMA nodes would > >> look like : > >> > >> -object memory-backend-ram,id=ram-node0,size=1G \ > >> -numa node,nodeid=0,memdev=ram-node0 \ > >> -object memory-backend-ram,id=ram-node1,size=1G \ > >> -numa node,nodeid=1,cpus=

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-15 Thread David Hildenbrand
On 15.09.23 16:47, Alex Williamson wrote: On Fri, 15 Sep 2023 16:19:29 +0200 Cédric Le Goater wrote: Hello Ankit, On 9/15/23 04:45, ank...@nvidia.com wrote: From: Ankit Agrawal For devices which allow CPU to cache coherently access their memory, it is sensible to expose such memory as NUMA

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-15 Thread Alex Williamson
On Fri, 15 Sep 2023 16:19:29 +0200 Cédric Le Goater wrote: > Hello Ankit, > > On 9/15/23 04:45, ank...@nvidia.com wrote: > > From: Ankit Agrawal > > > > For devices which allow CPU to cache coherently access their memory, > > it is sensible to expose such memory as NUMA nodes separate from > >

Re: [PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-15 Thread Cédric Le Goater
Hello Ankit, On 9/15/23 04:45, ank...@nvidia.com wrote: From: Ankit Agrawal For devices which allow CPU to cache coherently access their memory, it is sensible to expose such memory as NUMA nodes separate from the sysmem node. Qemu currently do not provide a mechanism for creation of NUMA node

[PATCH v1 0/4] vfio: report NUMA nodes for device memory

2023-09-14 Thread ankita
From: Ankit Agrawal For devices which allow CPU to cache coherently access their memory, it is sensible to expose such memory as NUMA nodes separate from the sysmem node. Qemu currently do not provide a mechanism for creation of NUMA nodes associated with a vfio-pci device. Implement a mechanism