Public bug reported:

================================================================================
TITLE
================================================================================
[regression] poweroff deadlocks in amdgpu_dm_ism_disable() — ISM backport is
missing upstream fix 3714fe2 "drm/amd/display: Fix ISM dc_lock deadlock during
suspend"

================================================================================
SUMMARY
================================================================================
Since the Idle State Manager (ISM) backport landed in linux-hwe-7.0, system
poweroff intermittently deadlocks inside the amdgpu display manager, leaving the
machine powered with displays and USB already off. A hard reset is required.

The backport carries the ISM feature commit but NOT the later upstream fix that
corrects its dc_lock ordering. That fix exists and is small.

  upstream 7540034  2026-03-30  drm/amd/display: Add Idle state manager(ISM)
  upstream 3714fe2  2026-05-19  drm/amd/display: Fix ISM dc_lock deadlock 
during suspend   <-- MISSING

Ubuntu changelogs for 7.0.0-28 through 7.0.0-31 mention only "drm/amd/display:
Add Idle state manager(ISM)". No ISM or dc_lock fix appears in any entry.
7.0.0-31 merged upstream stable v7.0.13 and v7.0.14; the fix went via drm-next
rather than stable, so it was not picked up.

================================================================================
AFFECTED / NOT AFFECTED
================================================================================
  7.0.0-14   NOT affected   amdgpu_dm_ism_* symbols absent from amdgpu.ko
  7.0.0-28   affected       symbols present (verified in the shipped module)
  7.0.0-30   affected
  7.0.0-31   affected       reproduced, trace below

Verified directly against the shipped binaries:

  $ zstd -dc /lib/modules/<ver>/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko.zst 
\
      | strings | grep -cE 
'^amdgpu_dm_ism_(enable|disable)$|^dm_ism_delayed_work_func$'
    7.0.0-14 -> 0
    7.0.0-28 -> 3
    7.0.0-31 -> 3

================================================================================
HARDWARE
================================================================================
ASRock B650M Pro RS (BIOS 4.10), Ryzen 5 7600X
XFX RX 9060 XT 16GB — RDNA 4, PCI 1002:7590, DCN 4.0.1, Display Core v3.2.369
Integrated Raphael display block also present (DCN 3.1.5)
Linux Mint 22.3 on Ubuntu 24.04 base. Stock Ubuntu HWE kernel binaries.
No third-party kernel, Mesa, or graphics repositories configured.

NOTE: LP #2152570 targeted AMD *integrated* graphics on TUXEDO laptops. This is
a desktop discrete RDNA 4 card, i.e. outside the tested scope of that patchset.

================================================================================
SYMPTOM
================================================================================
Roughly 1 poweroff in 3 stops after displays blank and USB power drops: fans,
motherboard and GPU LEDs remain powered indefinitely. Reset button required.
Not reproducible from a freshly booted session with no display activity —
consistent with a race on pending ISM delayed work.

================================================================================
EVIDENCE
================================================================================
Captured by setting kernel.hung_task_panic=1 for the shutdown window only, so
the silent hang becomes a panic that efi_pstore records across the reboot.

    INFO: task shutdown:1 blocked for more than 62 seconds.
    INFO: task kworker/u48:5:111406 blocked for more than 62 seconds.
    INFO: task kworker/u48:5:111406 is blocked on a mutex likely owned by task 
shutdown:1.
    Kernel panic - not syncing: hung_task: blocked tasks
    CPU: 1 UID: 0 PID: 93 Comm: khungtaskd Tainted: G OE 7.0.0-31-generic 
#31~24.04.1-Ubuntu

PID 1 (systemd-shutdown), holding dm->dc_lock:

    kernel_power_off
      device_shutdown
        pci_device_shutdown
          amdgpu_pci_shutdown              [amdgpu]
            amdgpu_device_suspend          [amdgpu]
              amdgpu_device_ip_suspend_phase1
                amdgpu_ip_block_suspend
                  dm_suspend               [amdgpu]
                    amdgpu_dm_ism_disable  [amdgpu]
                      disable_delayed_work_sync
                        __flush_work
                          wait_for_completion     <-- waits forever

The worker it is flushing:

    Workqueue: events_unbound dm_ism_delayed_work_func [amdgpu]
      dm_ism_delayed_work_func
        __mutex_lock_slowpath
          mutex_lock(&dm->dc_lock)                <-- owned by PID 1

ABBA deadlock. dm_suspend() holds dc_lock and synchronously flushes ISM delayed
work; that work must take dc_lock to run.

(The G/OE taint is an unrelated out-of-tree openrazer module. The trace is
entirely in-tree amdgpu.)

================================================================================
ROOT CAUSE
================================================================================
amdgpu_dm_ism_disable() explicitly requires dc_lock NOT be held:

    void amdgpu_dm_ism_disable(struct amdgpu_display_manager *dm)
    {
            lockdep_assert_not_held(&dm->dc_lock);
            drm_for_each_crtc(crtc, dm->ddev) {
                    disable_delayed_work_sync(&ism->delayed_work);
                    disable_delayed_work_sync(&ism->sso_delayed_work);
            }
    }

    static void dm_ism_delayed_work_func(struct work_struct *work)
    {
            guard(mutex)(&dm->dc_lock);
            amdgpu_dm_ism_commit_event(ism, DM_ISM_EVENT_TIMER_ELAPSED);
    }

Current upstream dm_suspend() honours this in both its reset and S3 paths, and
amdgpu_dm_fini() likewise, each carrying the comment:

    /* Quiesce ISM workers before taking dc_lock (workers take dc_lock
     * themselves; syncing under it would deadlock).
     */
    amdgpu_dm_ism_disable(dm);
    scoped_guard(mutex, &dm->dc_lock)
            amdgpu_dm_ism_force_full_power(dm);

That ordering is what commit 3714fe2 introduced. In the Ubuntu build the call
is evidently still made under the lock — hung_task's owner detection names PID 1
as the dc_lock holder while PID 1 is inside amdgpu_dm_ism_disable().

On a production kernel lockdep is disabled, so lockdep_assert_not_held() is a
no-op and the violation is silent until the race is lost.

================================================================================
REQUESTED FIX
================================================================================
Backport upstream 3714fe2 "drm/amd/display: Fix ISM dc_lock deadlock during
suspend" into linux-hwe-7.0. Optionally also 732a8ad ("Fix ISM teardown crash
from NULL dc dereference", 2026-04-17), which is from the same stabilisation
series.

Alternatively, drop the ISM patch for hardware outside the TUXEDO scope it was
verified against.

================================================================================
WORKAROUND
================================================================================
Boot 7.0.0-14, in which the ISM code is absent. There is no runtime mitigation:
ISM has no module parameter, no debugfs control, and no dcdebugmask bit. Its
only inputs are internal ism->config fields. amdgpu.dc=0 is not an option on
DCN 4.0.1.

================================================================================
HOW TO REPRODUCE / CAPTURE
================================================================================
The failure occurs inside kernel_power_off(), after journald has stopped, so it
leaves no trace in the journal. To capture it:

  1. Arm panic-on-hang for the shutdown window only (a systemd unit whose
     ExecStop sets kernel.hung_task_panic=1, kernel.panic=20).
  2. efi_pstore records the panic; systemd-pstore archives it to
     /var/lib/systemd/pstore/ on the next boot.
  3. Add ignore_loglevel to the kernel cmdline to see it on the console
     (systemd-shutdown raises console loglevel only to 5; hung_task warnings
     are level 6 and are otherwise invisible).

** Affects: linux-hwe-7.0 (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 regression-update

** Attachment added: "pstore panic record — full hung_task traces"
   
https://bugs.launchpad.net/bugs/2166940/+attachment/5999129/+files/HANG-PANIC-20260909-2313-amdgpu-dm-ism-deadlock.txt

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2166940

Title:
  [regression] poweroff deadlocks in amdgpu_dm_ism_disable() — ISM
  backport is missing upstream fix 3714fe2

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-hwe-7.0/+bug/2166940/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to