On Fri, 18 Aug 2023 11:30:55 +0000 Shreyas Shah <shreyas.shah@elastics.cloud> wrote:
> Once the cxl memory is online, how does Operating system know whether to > malloc in the cxl memory or socket attached DDR memory? If you've brought the memory up as 'normal memory' via kmem rather than the other dax options then it'll be a separate NUMA node. Hence you can control allocations using same tools used on multiple numa node systems. Jonathan > > > > [https://static1.squarespace.com/static/60dbbd6d597c966b91a3b27b/t/6104415e6177af1589fb05e4/1627668830393/elastics-cloud-logo-120.png]<https://www.elastics.cloud/> > Shreyas Shah > Founder, CTO and Chief Scientist, Elastics.cloud, Inc. > 1730 North First Street, 5th Floor, San Jose, CA 95112 > t: 408 476 3100<tel:408%20476%203100> | e: email: shreyas.shah@elastics.cloud > > ________________________________ > From: Maverickk 78 <maverickk1...@gmail.com> > Sent: Thursday, August 17, 2023 10:18 PM > To: Jonathan Cameron <jonathan.came...@huawei.com> > Cc: Jonathan Cameron via <qemu-devel@nongnu.org>; linux-...@vger.kernel.org > <linux-...@vger.kernel.org> > Subject: Re: CXL volatile memory is not listed > > Hi Jonathan, > > The use case of CXL switch will always need some sort of management > agent + FM configuring the available CXL memory connected. > > In most cases it would be bmc controller configuring MLD/MHD's to > host, and in very rare scenarios it may be one of the host interacting > with FM firmware inside the switch which would do the trick. > > Another use case is static hardcoding between CXL memory & host in > built in cxl switch > > There is no scenario where one of the host BIOS can push the select > CXL memory to itself. > > > Is my understanding correct? > > > > On Fri, 11 Aug 2023 at 19:25, Jonathan Cameron > <jonathan.came...@huawei.com> wrote: > > > > On Fri, 11 Aug 2023 08:04:26 +0530 > > Maverickk 78 <maverickk1...@gmail.com> wrote: > > > > > Jonathan, > > > > > > > More generally for the flow that would bring the memory up as system ram > > > > you would typically need the bios to have done the CXL enumeration or > > > > a bunch of scripts in the kernel to have done it. In general it can't > > > > be fully automated, because there are policy decisions to make on > > > > things like > > > > interleaving. > > > > > > BIOS CXL enumeration? is CEDT not enough? or BIOS further needs to > > > create an entry > > > in the e820 table? > > On intel platforms 'maybe' :) I know how it works on those that just > > use the nice standard EFI tables - less familiar with the e820 stuff :) > > > > CEDT says where to find the the various bits of system related CXL stuff. > > Nothing in there on the configuration that should be used such as > > interleaving > > as that depends on what the administrator wants. Or on what the BIOS has > > decided the users should have. > > > > > > > > > > > > > I'm not aware of any open source BIOSs that do it yet. So you have > > > > to rely on the same kernel paths as for persistent memory - manual > > > > configuration > > > > etc in the kernel. > > > > > > > Manual works with "cxl create regiton" :) > > Great. > > > > Jonathan > > > > > > > > On Thu, 10 Aug 2023 at 16:05, Jonathan Cameron > > > <jonathan.came...@huawei.com> wrote: > > > > > > > > On Wed, 9 Aug 2023 04:21:47 +0530 > > > > Maverickk 78 <maverickk1...@gmail.com> wrote: > > > > > > > > > Hello, > > > > > > > > > > I am running qemu-system-x86_64 > > > > > > > > > > qemu-system-x86_64 --version > > > > > QEMU emulator version 8.0.92 (v8.1.0-rc2-80-g0450cf0897) > > > > > > > > > +Cc linux-cxl as the answer is more todo with linux than qemu. > > > > > > > > > qemu-system-x86_64 \ > > > > > -m 2G,slots=4,maxmem=4G \ > > > > > -smp 4 \ > > > > > -machine type=q35,accel=kvm,cxl=on \ > > > > > -enable-kvm \ > > > > > -nographic \ > > > > > -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 \ > > > > > -device cxl-rp,id=rp0,bus=cxl.0,chassis=0,port=0,slot=0 \ > > > > > -object > > > > > memory-backend-file,id=mem0,mem-path=/tmp/mem0,size=1G,share=true \ > > > > > -device cxl-type3,bus=rp0,volatile-memdev=mem0,id=cxl-mem0 \ > > > > > -M cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=1G > > > > > > > > There are some problems upstream at the moment (probably not cxl > > > > related but > > > > I'm digging). So today I can't boot an x86 machine. (goody) > > > > > > > > > > > > More generally for the flow that would bring the memory up as system ram > > > > you would typically need the bios to have done the CXL enumeration or > > > > a bunch of scripts in the kernel to have done it. In general it can't > > > > be fully automated, because there are policy decisions to make on > > > > things like > > > > interleaving. > > > > > > > > I'm not aware of any open source BIOSs that do it yet. So you have > > > > to rely on the same kernel paths as for persistent memory - manual > > > > configuration > > > > etc in the kernel. > > > > > > > > There is support in ndctl for those enabling flows, so I'd look there > > > > for more information > > > > > > > > Jonathan > > > > > > > > > > > > > > > > > > > > > > > I was expecting the CXL memory to be listed in "System Ram", the lsmem > > > > > shows only 2G memory which is System RAM, it's not listing the CXL > > > > > memory. > > > > > > > > > > Do I need to pass any particular parameter in the kernel command line? > > > > > > > > > > Is there any documentation available? I followed the inputs provided > > > > > in > > > > > > > > > > https://lore.kernel.org/linux-mm/y+csoehvlkudn...@kroah.com/T/ > > > > > > > > > > Is there any documentation/blog listed? > > > > > > >
