Public bug reported:

Hardware
========

Laptop: MSI Vector 17 HX AI
Platform: Intel Arrow Lake
Intel VMD: 8086:ad0b
VMD PCI address: 0000:00:0e.0
Subsystem: 1462:149c

Internal NVMe SSD:
Phison 1TB ESR01TBYCCA4-EDJ-2MS
Controller: Phison PS5029-E29T PCIe 4.0 NVMe
PCI ID: 1987:5029 (rev 01)
Firmware: ETFM50.0

Ubuntu: 26.04
Current/final tested kernel: 7.0.0-30-generic
Secure Boot: enabled


Problem 1: Intel VMD Bus Offset 3
================================

With the stock Ubuntu VMD driver the internal NVMe SSD is not exposed.

The kernel reports:

    vmd 0000:00:0e.0: Unknown Bus Offset Setting (3)

and no internal /dev/nvme* device is available.

Linux 6.14 was also tested on this machine and did not solve the VMD
problem.

The system firmware does not expose a usable BIOS option to disable Intel
VMD/RST.

Intel's public VMD second-rootbus patch series was integrated into the
Ubuntu VMD source:

    [PATCH v3 0/8] VMD add second rootbus support

https://www.spinics.net/lists/linux-pci/msg163096.html

After integrating the second-rootbus support, Linux successfully enumerates
the internal NVMe controller and SSD.


Problem 2: NVMe completion stalls
=================================

After the SSD became accessible through the modified VMD driver, a second
problem became visible.

Kernel messages repeatedly contained:

    nvme nvme0: I/O tag ... timeout, completion polled

The stalls occurred at approximately 30-second intervals.

A diagnostic sequential read before the workaround:

    sudo dd if=/dev/nvme0n1 of=/dev/null bs=16M count=64 status=progress

Result:

    1 GiB in approximately 121.339 seconds
    approximately 8.8 MB/s

CPU and memory were not saturated.

Disabling ASPM was tested and did not solve the problem.


ARL004 investigation
====================

The official Intel RST/VMD Windows driver distributed by MSI for this
machine was examined by static analysis for comparison.

The relevant Windows driver path performs a PCI configuration-space read
after MSI handling when the completion state requires ordering.

This behavior is consistent with the Intel Arrow Lake ARL004 erratum, where
an MSI from a VMD-owned device may pass a preceding memory write.

An experimental Linux workaround was implemented for the tested
8086:ad0b VMD/NVMe path. It performs a dummy PCI configuration-space read
before the subsequent interrupt handling path.


Result
======

With the experimental ARL004 workaround:

    sudo dd if=/dev/nvme0n1 of=/dev/null bs=16M count=64 status=progress

completed in:

    1 GiB in 0.88206 seconds
    approximately 1.2 GB/s

A subsequent 10 GiB sequential diagnostic read also completed normally.

After booting with the final tested module:

    sudo journalctl -k -b | grep -E 'timeout|completion polled'

produced no matching messages.

The approximately 136x difference above is only a diagnostic comparison on
this specific machine and is not intended as a general SSD benchmark.


Reproduction / reference implementation
=======================================

The complete investigation, tested source, separate patches, combined patch,
known-working module, SHA256 checksums, installation/bootstrap procedure and
recovery procedure are published here:

https://github.com/gvozd188/vmd-arl004

Exact reference commit:

https://github.com/gvozd188/vmd-arl004/commit/5438c83

Relevant files:

    0001-vmd-second-rootbus-intel.patch
        Integration of Intel's second-rootbus support.

    0002-vmd-ad0b-arl004-workaround.patch
        Experimental ARL004 workaround.

    vmd-arl004.patch
        Combined patch.

    vmd-ubuntu-7.0.0-30.c
        Ubuntu VMD source used as the patch base.

    vmd.c
        Final tested source.

    vmd.ko
        Known-working reference module for 7.0.0-30-generic.


Known-working source SHA256:

    0a27aa1379e8b7fc22f509b8d06ed3676a211d7bb3226a3f9b3dadfc208be7a3

Known-working module SHA256:

    b2c9eacb720d45fcaab15ff09ad3fdc2e97b8a6dd215c80fa282090ff8d2b2cd


Installation history
====================

Because the stock VMD driver could not expose the internal SSD, an existing
Ubuntu HDD from another laptop was connected to the MSI Vector through a
SATA-to-USB adapter.

The MSI Vector was booted directly from that HDD. That Ubuntu environment
was running kernel 7.0.0-30-generic.

The modified VMD driver was used there to expose the internal NVMe SSD.

Ubuntu was then manually deployed onto the dedicated Ubuntu partition of the
internal SSD without modifying the existing Windows and data partitions.

The manually deployed SSD installation initially contained kernel
7.0.0-14-generic. This kernel was not intentionally selected as a VMD
workaround; it was simply the kernel present in the manually deployed system
at that stage.

A compatible custom VMD module was installed for 7.0.0-14-generic and added
to its initramfs, allowing the first independent boot from the internal SSD.

Only after that successful SSD boot was DKMS configured.

The SSD installation was subsequently updated to 7.0.0-30-generic, where the
final VMD/ARL004 development and testing was performed.


Expected result
===============

The stock Ubuntu kernel should:

1. correctly enumerate the second-rootbus / Bus Offset 3 topology on Intel
   VMD 8086:ad0b;
2. expose the internal NVMe SSD;
3. handle NVMe completion ordering without repeated approximately 30-second
   completion timeouts.


Actual result
=============

With the stock VMD driver, the internal SSD is not exposed because Bus
Offset Setting 3 is rejected.

After adding second-rootbus support alone, the SSD becomes visible but
repeated NVMe completion stalls occur on this hardware.

The experimental ARL004 ordering workaround eliminates the observed stalls
on the tested system.


Notes
=====

The ARL004 workaround is experimental and platform-specific. I am reporting
the observed hardware behavior and the tested workaround rather than
claiming that this implementation is the appropriate final upstream fix.

The Windows driver binary is not redistributed in the GitHub repository.
Only identification information, hashes and static-analysis notes are
provided.

ProblemType: Bug
DistroRelease: Ubuntu 26.04
Package: linux-image-7.0.0-30-generic 7.0.0-30.30
ProcVersionSignature: Ubuntu 7.0.0-30.30-generic 7.0.12
Uname: Linux 7.0.0-30-generic x86_64
ApportVersion: 2.34.1-0ubuntu0.1
Architecture: amd64
AudioDevicesInUse:
 USER        PID ACCESS COMMAND
 /dev/snd/controlC1:  gvozd188   2323 F.... pipewire
                      gvozd188   2360 F.... wireplumber
 /dev/snd/controlC0:  gvozd188   2360 F.... wireplumber
 /dev/snd/seq:        gvozd188   2323 F.... pipewire
CasperMD5CheckResult: unknown
CurrentDesktop: ubuntu:GNOME
Date: Thu Sep  3 13:08:10 2026
MachineType: Micro-Star International Co., Ltd. Vector 17 HX AI A2XWIG
ProcFB: 0 i915drmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-7.0.0-30-generic 
root=UUID=fef65fb9-b1c4-4bbe-9c02-92a4a7280fee ro quiet splash iommu=pt 
crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
WifiSyslog:
 
dmi.bios.date: 04/20/2026
dmi.bios.release: 1.18
dmi.bios.vendor: American Megatrends International, LLC.
dmi.bios.version: E17S3IMS.112
dmi.board.asset.tag: Default string
dmi.board.name: MS-17S3
dmi.board.vendor: Micro-Star International Co., Ltd.
dmi.board.version: REV:1.0
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: Micro-Star International Co., Ltd.
dmi.chassis.version: N/A
dmi.modalias: 
dmi:bvnAmericanMegatrendsInternational,LLC.:bvrE17S3IMS.112:bd04/20/2026:br1.18:svnMicro-StarInternationalCo.,Ltd.:pnVector17HXAIA2XWIG:pvrREV1.0:rvnMicro-StarInternationalCo.,Ltd.:rnMS-17S3:rvrREV1.0:cvnMicro-StarInternationalCo.,Ltd.:ct10:cvrN/A:sku17S3.1:pfaVector:
dmi.product.family: Vector
dmi.product.name: Vector 17 HX AI A2XWIG
dmi.product.sku: 17S3.1
dmi.product.version: REV:1.0
dmi.sys.vendor: Micro-Star International Co., Ltd.

** Affects: linux (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug resolute wayland-session

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2166325

Title:
  Intel VMD 8086:ad0b: Bus Offset 3 prevents NVMe enumeration and ARL004
  causes NVMe completion stalls

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2166325/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to