On 24.11.22 10:34:35, Dan Williams wrote: > Changes since v3 [1]: > - Rework / simplify CXL to LIBNVDIMM coordination to remove a > flush_work() locking dependency from underneath the root device lock. > - Move the root device rescan to a workqueue > - Connect RCDs directly as endpoints reachable through a CXL host bridge > as a dport, i.e. drop the extra dport indirection from v3 > - Add unit test infrastructure for an RCD configuration > > [1]: http://lore.kernel.org/r/20221109104059.766720-1-rrich...@amd.com/ > > --- > > >From [PATCH v4 10/12] cxl/port: Add RCD endpoint port enumeration
> --- > > Dan Williams (9): > cxl/acpi: Simplify cxl_nvdimm_bridge probing > cxl/region: Drop redundant pmem region release handling > cxl/pmem: Refactor nvdimm device registration, delete the workqueue > cxl/pmem: Remove the cxl_pmem_wq and related infrastructure > cxl/acpi: Move rescan to the workqueue > tools/testing/cxl: Make mock CEDT parsing more robust > cxl/mem: Move devm_cxl_add_endpoint() from cxl_core to cxl_mem > cxl/port: Add RCD endpoint port enumeration > tools/testing/cxl: Add an RCH topology > > Robert Richter (2): > cxl/ACPI: Register CXL host ports by bridge device > cxl/acpi: Extract component registers of restricted hosts from RCRB > > Terry Bowman (1): > cxl/acpi: Set ACPI's CXL _OSC to indicate CXL1.1 support > > > drivers/acpi/pci_root.c | 1 > drivers/cxl/acpi.c | 105 +++++++++--- > drivers/cxl/core/core.h | 8 - > drivers/cxl/core/pmem.c | 94 +++++++---- > drivers/cxl/core/port.c | 111 +++++++------ > drivers/cxl/core/region.c | 54 ++++++ > drivers/cxl/core/regs.c | 56 +++++++ > drivers/cxl/cxl.h | 46 +++-- > drivers/cxl/cxlmem.h | 15 ++ > drivers/cxl/mem.c | 72 ++++++++ > drivers/cxl/pci.c | 13 +- > drivers/cxl/pmem.c | 351 > +++++------------------------------------ > tools/testing/cxl/Kbuild | 1 > tools/testing/cxl/test/cxl.c | 241 ++++++++++++++++++++++------ > tools/testing/cxl/test/mem.c | 40 ++++- > tools/testing/cxl/test/mock.c | 19 ++ > tools/testing/cxl/test/mock.h | 3 > 17 files changed, 712 insertions(+), 518 deletions(-) I have tested this series and the enumeration is as expected (see sysfs dump below). I see an HDM failure which I am investigating, but that seems unrelated to this series. You can add to this series my: Tested-by: Robert Richter <rrich...@amd.com> The drawback of this topology decision is that the endpoint's port is a child of root0, which is the ACPI0017's device (not the CXL host bridge): /sys/bus/cxl/devices/root0/endpoint1 I understand this is the CXL's RCD view on the pcie hierarchy here. Logically there would be a cxl host port in between, like this: /sys/bus/cxl/devices/root0/port1/endpoint2 Esp. if there are multiple hosts in the system the port relations will be lost and can only be determined using the pci dev's parent bridge. port1 could then also hold the uport's component regs with the upstream port capabilities to access e.g. status regs for RAS. Anyway, let's see how this approach flies. Thanks for looing into this. -Robert --- # find /sys/bus/cxl/devices/ -ls 265293 0 drwxr-xr-x 2 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/ 265493 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/root0 -> ../../../devices/platform/ACPI0017:00/root0 265664 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/endpoint1 -> ../../../devices/platform/ACPI0017:00/root0/endpoint1 265584 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/mem0 -> ../../../devices/pci0000:7f/0000:7f:00.0/mem0 # find /sys/bus/cxl/devices/*/ -ls 265660 0 drwxr-xr-x 2 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/endpoint1/ 265661 0 -rw-r--r-- 1 root root 4096 Nov 30 14:18 /sys/bus/cxl/devices/endpoint1/uevent 265667 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/endpoint1/driver -> ../../../../../bus/cxl/drivers/cxl_port 265669 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/endpoint1/uport -> ../../../../pci0000:7f/0000:7f:00.0/mem0 265668 0 -r-------- 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/endpoint1/CDAT 265665 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/endpoint1/subsystem -> ../../../../../bus/cxl 265662 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/endpoint1/devtype 265663 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/endpoint1/modalias 265573 0 drwxr-xr-x 4 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/mem0/ 265574 0 -rw-r--r-- 1 root root 4096 Nov 30 14:18 /sys/bus/cxl/devices/mem0/uevent 265578 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/label_storage_size 265582 0 drwxr-xr-x 2 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/mem0/pmem 265583 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/pmem/size 265576 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/firmware_version 265579 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/numa_node 265586 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/dev 265659 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/mem0/driver -> ../../../../bus/cxl/drivers/cxl_mem 265580 0 drwxr-xr-x 2 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/mem0/ram 265581 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/ram/size 265585 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/mem0/subsystem -> ../../../../bus/cxl 265575 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/serial 265577 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/mem0/payload_max 265489 0 drwxr-xr-x 3 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/root0/ 265490 0 -rw-r--r-- 1 root root 4096 Nov 30 14:18 /sys/bus/cxl/devices/root0/uevent 265660 0 drwxr-xr-x 2 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/root0/endpoint1 265661 0 -rw-r--r-- 1 root root 4096 Nov 30 14:18 /sys/bus/cxl/devices/root0/endpoint1/uevent 265667 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/root0/endpoint1/driver -> ../../../../../bus/cxl/drivers/cxl_port 265669 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/root0/endpoint1/uport -> ../../../../pci0000:7f/0000:7f:00.0/mem0 265668 0 -r-------- 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/root0/endpoint1/CDAT 265665 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/root0/endpoint1/subsystem -> ../../../../../bus/cxl 265662 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/root0/endpoint1/devtype 265663 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/root0/endpoint1/modalias 265495 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/root0/uport -> ../../ACPI0017:00 265496 0 lrwxrwxrwx 1 root root 0 Nov 30 14:21 /sys/bus/cxl/devices/root0/dport4 -> ../../../pci0000:7f 265494 0 lrwxrwxrwx 1 root root 0 Nov 30 14:18 /sys/bus/cxl/devices/root0/subsystem -> ../../../../bus/cxl 265491 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/root0/devtype 265492 0 -r--r--r-- 1 root root 4096 Nov 30 14:21 /sys/bus/cxl/devices/root0/modalias
