This series aims to address long-standing conflicts between HMEM and CXL when handling Soft Reserved memory ranges.
Reworked from Dan's patch: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/patch/?id=ab70c6227ee6165a562c215d9dcb4a1c55620d5d Previous work: https://lore.kernel.org/all/[email protected]/ Link to v4: https://lore.kernel.org/all/[email protected]/ The series is based on branch "for-7.0/cxl-init" and base-commit is base-commit: bc62f5b308cbdedf29132fe96e9d591e526527e1 [1] Hotplug works. After offlining the memory I can tear down the regions and recreate them back. dax_cxl creates dax devices and onlines memory. 850000000-284fffffff : CXL Window 0 850000000-284fffffff : region0 850000000-284fffffff : dax0.0 850000000-284fffffff : System RAM (kmem) [2] With CONFIG_CXL_REGION disabled, all the resources are handled by HMEM. Soft Reserved range shows up in /proc/iomem, no regions come up and dax devices are created from HMEM. 850000000-284fffffff : CXL Window 0 850000000-284fffffff : Soft Reserved 850000000-284fffffff : dax0.0 850000000-284fffffff : System RAM (kmem) [3] Region assembly failure works same as [2]. Before: 2850000000-484fffffff : Soft Reserved 2850000000-484fffffff : CXL Window 1 2850000000-484fffffff : dax4.0 2850000000-484fffffff : System RAM (kmem) After tearing down dax4.0 and creating it back: Logs: [ 547.847764] unregister_dax_mapping: mapping0: unregister_dax_mapping [ 547.855000] trim_dev_dax_range: dax dax4.0: delete range[0]: 0x2850000000:0x484fffffff [ 622.474580] alloc_dev_dax_range: dax dax4.1: alloc range[0]: 0x0000002850000000:0x000000484fffffff [ 752.766194] Fallback order for Node 0: 0 1 [ 752.766199] Fallback order for Node 1: 1 0 [ 752.766200] Built 2 zonelists, mobility grouping on. Total pages: 8096220 [ 752.783234] Policy zone: Normal [ 752.808604] Demotion targets for Node 0: preferred: 1, fallback: 1 [ 752.815509] Demotion targets for Node 1: null After: 2850000000-484fffffff : Soft Reserved 2850000000-484fffffff : CXL Window 1 2850000000-484fffffff : dax4.1 2850000000-484fffffff : System RAM (kmem) v5 updates: - Patch 1 dropped as its been merged for-7.0/cxl-init. - Added Reviewed-by tags. - Shared dax_cxl_mode between dax/cxl.c and dax/hmem.c and used -EPROBE_DEFER to defer dax_cxl. - CXL_REGION_F_AUTO check for resetting decoders. - Teardown all CXL regions if any one CXL region doesn't fully contain the Soft Reserved range. - Added helper cxl_region_contains_sr() to determine Soft Reserved ownership. - bus_rescan_devices() to retry dax_cxl. - Added guard(rwsem_read)(&cxl_rwsem.region). v4 updates: - No changes patches 1-3. - New patches 4-7. - handle_deferred_cxl() has been enhanced to handle case where CXL regions do not contiguously and fully cover Soft Reserved ranges. - Support added to defer cxl_dax registration. - Support added to teardown cxl regions. v3 updates: - Fixed two "From". v2 updates: - Removed conditional check on CONFIG_EFI_SOFT_RESERVE as dax_hmem depends on CONFIG_EFI_SOFT_RESERVE. (Zhijian) - Added TODO note. (Zhijian) - Included region_intersects_soft_reserve() inside CONFIG_EFI_SOFT_RESERVE conditional check. (Zhijian) - insert_resource_late() -> insert_resource_expand_to_fit() and __insert_resource_expand_to_fit() replacement. (Boris) - Fixed Co-developed and Signed-off by. (Dan) - Combined 2/6 and 3/6 into a single patch. (Zhijian). - Skip local variable in remove_soft_reserved. (Jonathan) - Drop kfree with __free(). (Jonathan) - return 0 -> return dev_add_action_or_reset(host...) (Jonathan) - Dropped 6/6. - Reviewed-by tags (Dave, Jonathan) Dan Williams (2): dax/hmem: Request cxl_acpi and cxl_pci before walking Soft Reserved ranges dax/hmem: Gate Soft Reserved deferral on DEV_DAX_CXL Smita Koralahalli (5): cxl/region: Skip decoder reset on detach for autodiscovered regions cxl/region: Add helper to check Soft Reserved containment by CXL regions dax: Introduce dax_cxl_mode for CXL coordination dax/hmem, cxl: Defer and resolve ownership of Soft Reserved memory ranges dax/hmem: Reintroduce Soft Reserved ranges back into the iomem tree drivers/cxl/core/region.c | 58 ++++++++++++++++- drivers/cxl/cxl.h | 7 ++ drivers/dax/Kconfig | 2 + drivers/dax/bus.c | 3 + drivers/dax/bus.h | 8 +++ drivers/dax/cxl.c | 9 +++ drivers/dax/hmem/hmem.c | 132 ++++++++++++++++++++++++++++++++++---- 7 files changed, 207 insertions(+), 12 deletions(-) -- 2.17.1
