Hi Bjorn, Ilpo, and the PCI community, This patch series introduces upstream Linux kernel support for PCIe Lane Margining at Receiver (LMR) per PCI Express Base Specification Revision 7.0 / 6.0 (§ 7.7.11 and § 8.4.4).
During the review of the previous v7 standalone LMR submission (https://lore.kernel.org/linux-pci/[email protected]/), Ilpo Järvinen pointed out that feature drivers resolving link partners duplicate link traversal logic that is already present in drivers such as ASPM: "This feels like duplicating similar functionality with the aspm driver that also wants to infer ends of the link when giving a pci_dev in. The aspm driver currently does that within, but it kind of duplicating pci_bus. It would be nice to avoid the duplication and have something similar for this in PCI core." In response to this feedback, the implementation is factored into a clean, modular 3-patch stack: 1. Patch 1/3 (PCI: Add pcie_get_link_endpoints() helper): Standardized, race-safe helper in drivers/pci/pci.c and include/linux/pci.h to identify both ends of a point-to-point PCIe link. Safely inspects the subordinate bus under down_read(&pci_bus_sem) and acquires a reference (pci_dev_get()), paired symmetrically with pcie_put_link_endpoints(). Filters for Function 0 (with ARI support), validates bridge ownership (child->self == pdev) to prevent ABA pointer reuse, and handles RCiEP and empty downstream ports (-ENODEV). 2. Patch 2/3 (PCI/ASPM: Add pci_aspm_inhibit() helper for temporary link state suppression): Reference-counted mechanism (aspm_inhibit_cnt) in drivers/pci/pcie/aspm.c to temporarily disable ASPM state transitions (L0s, L1, L1SS) on an active link. Guarantees callers hold down_read(&pci_bus_sem) (via lockdep_assert_held_read), eliminating deadlocks and preventing power-saving transitions while hardware link characterization is in progress. 3. Patch 3/3 (PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support): Implements the core LMR driver (CONFIG_PCIE_LMR) in drivers/pci/pcie/margin.c exposing a debugfs interface under /sys/kernel/debug/pci/pcie_lmr_<dev>/. Strictly self-contained: 0 modified lines in drivers/pci/pci.c, pci-driver.c, or core drivers/pci/pci.h. Exposes clean lifecycle hooks in drivers/pci/probe.c and drivers/pci/remove.c, supports 2D timing and voltage margining across physical receivers (0..6), enforces Function 0 endpoint filtering (§ 7.7.11), and includes an exhaustive kselftest suite (tools/testing/selftests/pcie_lmt/). - Capability Discovery (§ 7.7.11): Probed via Extended Capability ID 0x27 on Gen4+ (>= 16 GT/s) physical links. Differentiates optional support on Gen4/Gen5 from mandatory capability presence on Gen6+ (>= 64 GT/s). - Multi-Function Device Scope (§ 7.7.11): Enforces that capability registration is strictly restricted to Function 0 on Endpoints / Upstream Ports. - Payload Decoding (Table 4-77, Cmd 88h): Accurately decodes Margin Control Capabilities (Bits 0..4); preserves Bits 7:5 as reserved 0b. - Direction Encodings (§ 4.2.18.1.2): Enforces that Left/Right and Up/Down direction bits remain 0b for symmetric receivers (Bits 6 & 7 reserved 0b). - Multi-Receiver Concurrency (§ 4.2.18.2): Governed strictly by MIndErrorSampler. If MIndErrorSampler == 0b (main data sampler), at most one receiver across the link is margined at a time. - Autonomous Speed & Width Transitions (§ 7.5.3.7, § 7.5.3.17, § 8.4.4): Software sequencing disables Downstream Component before Upstream Component on enablement, and restores Upstream Component before Downstream Component on teardown. Restoration is unconditional via pcie_capability_clear_and_set_word() (serialized by pci_lock), eliminating trylock failure traps. Ingress paths (margin_enable_write and pci_lmr_exit) strictly follow the canonical PCI locking DAG: down_read(&pci_bus_sem) -> pci_dev_lock(downstream_port) -> pci_dev_lock(upstream_port) -> mutex_lock(&mdev->lock) -> spin_lock_irqsave(&aspm_lock) Key Concurrency Guarantees: 1. Hot-Swap Identity Invariance: When disabling LMR, dynamic endpoint resolution is completely bypassed. Teardown binds strictly to the session-saved mdev->partner, which is read and pinned with pci_dev_get() under mutex_lock(&mdev->lock). 2. Surprise Removal & Fault Hardening: Hardware register accesses check pci_dev_is_disconnected() and PCI_POSSIBLE_ERROR() guards, preventing MMIO or config space bus aborts when hardware is pulled. 3. Power Management Synchronization: Both link partners are pinned in D0 using pm_runtime_resume_and_get() during enablement, and symmetrically released via pm_runtime_put() on teardown. System suspend hooks into pci_pm_prepare() where all devices in the hierarchy are guaranteed to be in D0. 4. Security Teardown Order: In pci_destroy_dev(), pci_lmr_exit() is called after pci_tsm_destroy(), preserving D0 state for TSM link operations (PCIe IDE unbind and SPDM cryptographic session teardown). - Build: Clean compile on x86_64 and ARM64; make W=1 drivers/pci/ (0 warnings). - Linters: checkpatch.pl clean across all 3 patches (0 errors, 0 warnings). - Kselftest: tools/testing/selftests/pcie_lmt/pcie_lmt.sh expanded to 287 lines, covering positive and negative boundary tests (syntax clean with bash -n). - Pre-Commit AI Review (Sashiko AI): * Patch 1 (f34fdf7f5c6b): Reviewed — Clean (0 issues). * Patch 2 (6adc680a727f): Reviewed — Clean (0 issues). * Patch 3 (c62b5f92af63): Reviewed — Clean (0 issues, "No issues found."). Signed-off-by: Priyank Rathod <[email protected]> --- Changes in v2: - Decomposed monolithic LMR driver into a 3-patch stack with dedicated core helpers: * Patch 1: PCI core link endpoint discovery helper (pcie_get_link_endpoints()). * Patch 2: Core ASPM temporary inhibition helper (pci_aspm_inhibit()). * Patch 3: PCIe Lane Margining at Receiver driver and debugfs interface. - Core Scope Discipline: Confined LMR logic exclusively to drivers/pci/pcie/margin.c and include/linux/pci.h forward declarations (0 modified lines in pci.c/pci-driver.c). - Endpoint Discovery Hardening: * Handled RCiEP and empty downstream buses returning -ENODEV. * Added Function 0 resolution (pcie_find_link_upstream_func0()) supporting PCIe ARI. * Added bridge ownership verification (child->self == pdev) to prevent ABA races. * Added symmetric pcie_put_link_endpoints() to balance pci_dev_get() references. - ASPM Inhibit Hardening: * Added lockdep_assert_held_read(&pci_bus_sem) to pci_aspm_inhibit_locked(). * Added reference counting (aspm_inhibit_cnt) to support nested/concurrent calls. - Hot-Swap & Concurrency Protections: * Eliminated hot-swap identity mismatch in margin_enable_write() by binding teardown strictly to session-saved mdev->partner instead of re-evaluating endpoints. * Added mutex protection when reading and pinning mdev->partner to eliminate UAF races. * Converted autonomous speed/width restoration to unconditional register write via pcie_capability_clear_and_set_word(), eliminating trylock failure degradation. - Specification Alignments (PCIe Base Spec r7.0): * Aligned Extended Capability ID 0x27 to § 7.7.11 and § 8.4.4. * Enforced Function 0 restriction on Endpoints / Upstream Ports (§ 7.7.11). * Preserved Reserved Bits 7:5 in Command 88h capabilities response. * Implemented MIndErrorSampler-based concurrency for multi-receiver links (§ 4.2.18.2). * Corrected direction bit encoding to 0b for symmetric receivers (§ 4.2.18.1.2). - Kselftest Overhaul: * Extended test coverage from 105 to 287 lines, adding comprehensive boundary and negative validation tests. - Link to v1: https://lore.kernel.org/r/[email protected] - Link to v7 (monolithic LMR): https://lore.kernel.org/linux-pci/[email protected]/ --- Priyank Rathod (3): PCI: Add pcie_get_link_endpoints() helper PCI/ASPM: Add pci_aspm_inhibit() helper for temporary link state suppression PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support Documentation/PCI/index.rst | 1 + Documentation/PCI/pcie-lmr.rst | 174 +++ MAINTAINERS | 8 + drivers/pci/pci.c | 152 +++ drivers/pci/pci.h | 8 + drivers/pci/pcie/Kconfig | 12 + drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/aspm.c | 100 ++ drivers/pci/pcie/margin.c | 1592 ++++++++++++++++++++++++++ drivers/pci/probe.c | 1 + drivers/pci/remove.c | 2 +- include/linux/pci.h | 17 + include/uapi/linux/pci_regs.h | 18 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/pcie_lmt/Makefile | 3 + tools/testing/selftests/pcie_lmt/pcie_lmt.sh | 287 +++++ 16 files changed, 2376 insertions(+), 1 deletion(-) --- base-commit: cee9395acd8043be0644b25c34bfa86623f2b935 change-id: 20260831-pcie-link-endpoints-978e100d5d06 Best regards, -- Priyank Rathod <[email protected]>
