On Tue, 10 Nov 2026 07:15:00 +0000 Arun Iyengar via B4 Relay <[email protected]> wrote:
> From: Arun Iyengar <[email protected]> > > From the start of panthor_device_suspend() until panthor_device_resume() > has enabled the clocks again, the GPU clocks are off or about to be, and > the GPU power domain may be down. devfreq can still call ->target() in > that window: a write to min_freq or max_freq in sysfs, a thermal cooling > state change or a PM QoS request all end up in update_devfreq() whatever > the device state. > > With the RK3588 GPU core clock switched to the firmware-managed > SCMI_CLK_GPU (needed to reach the PVTPLL rates of the upper OPPs), such > a call is fatal. To set a PVTPLL rate TF-A programs registers in the GPU > GRF, which sit in the GPU power domain, and with the domain off that > access ends in an asynchronous SError and a kernel panic, for example > after > > echo 1000000000 > /sys/class/devfreq/fb000000.gpu/min_freq > > while the GPU is runtime suspended. > > Only apply a rate to the hardware while the device is resuming or > active, plus the suspend frequency that devfreq_suspend_device() applies > from the suspend path while the GPU is still powered. Other requests are > still accepted and recorded by devfreq, which applies them on resume or > at the next governor update. > > The RFC this is based on (Jonas Karlman's panthor SCMI clock change, > carried in Armbian's linux-rockchip pull request 380) checked the > runtime PM status instead. That > status is still RPM_SUSPENDING after panthor_devfreq_suspend() has run, > which leaves a window where the rate can be raised again right before > the power domain goes down. The devfreq suspend count cannot be used > either: panthor's first runtime resume calls devfreq_resume_device() > without a matching suspend, so the count is -1 while the GPU is active > and 0 while it is suspended. > > Suggested-by: Jonas Karlman <[email protected]> > Signed-off-by: Arun Iyengar <[email protected]> > --- > Tested on an Orange Pi 5 (RK3588S) with the GPU core clock switched to > SCMI_CLK_GPU and a 200 MHz opp-suspend (the configuration from the RFC, > not yet in mainline): writing min_freq while the GPU was runtime > suspended no longer ends in an SError, the request takes effect at the > next job (1 GHz measured), max_freq likewise; 200 suspend/resume cycles > with short GPU jobs and a 3 minute sustained load ran without faults. > With the mainline device tree (CRU clock, no suspend OPP) the SError is > not reachable and the guard only skips rate changes while the GPU is > powered down. > > This patch was developed with Claude (Anthropic) as a coding assistant: > it wrote the code and the commit message, and I reviewed and tested it > as described above. > --- > drivers/gpu/drm/panthor/panthor_devfreq.c | 42 > ++++++++++++++++++++++++++++--- > 1 file changed, 39 insertions(+), 3 deletions(-) > > diff --git a/drivers/gpu/drm/panthor/panthor_devfreq.c > b/drivers/gpu/drm/panthor/panthor_devfreq.c > index 2249b41ca4af..cab86e9c9d46 100644 > --- a/drivers/gpu/drm/panthor/panthor_devfreq.c > +++ b/drivers/gpu/drm/panthor/panthor_devfreq.c > @@ -60,20 +60,56 @@ static void panthor_devfreq_update_utilization(struct > panthor_devfreq *pdevfreq) > pdevfreq->time_last_update = now; > } > > +/* > + * Whether a new rate can be applied to the hardware now. > + * > + * From the start of panthor_device_suspend() until panthor_device_resume() > + * has enabled the clocks again, the GPU clocks are about to be disabled > + * or are off, and the GPU power domain may be down. devfreq can still > + * call ->target() in that window: sysfs > + * min_freq/max_freq writes, thermal cooling and PM QoS requests all end > + * up in update_devfreq() whatever the device state. > + * > + * Changing the rate then is pointless, and harmful where the core clock > + * is managed by firmware: on RK3588, setting SCMI_CLK_GPU to a PVTPLL > + * rate with the GPU power domain off makes TF-A access the powered-down > + * GPU GRF, which ends in an asynchronous SError. So while the device is > + * suspending only the suspend frequency, applied by > + * devfreq_suspend_device(), reaches the hardware, and nothing does > + * while it is suspended. devfreq records the other requests: they are > + * applied on resume or at the next governor update. > + */ > +static bool panthor_devfreq_can_set_rate(struct panthor_device *ptdev, > + unsigned long freq) > +{ > + struct devfreq *devfreq = ptdev->devfreq ? ptdev->devfreq->devfreq : > NULL; I'd rather use devfreq::suspend_count to detect if the devfreq component is suspended, since it gives us a better granularity than panthor_device::pm::state. > + > + switch (atomic_read(&ptdev->pm.state)) { > + case PANTHOR_DEVICE_PM_STATE_SUSPENDED: > + return false; > + case PANTHOR_DEVICE_PM_STATE_SUSPENDING: > + return devfreq && devfreq->suspend_freq && > + freq == devfreq->suspend_freq; > + default: When PANTHOR_DEVICE_PM_STATE_SUSPENDING is set, not all HW components have been clocked/put out of reset. It's probably okay for power domains, because those deps are described through device links, and the dependencies are resumed by PM core before the panthor PM hooks are called, but this is fragile, and that's another reason I think we shouldn't rely on panthor_device::pm::state for that, but on devfreq::suspend_count (or any other internal panthor_devfreq state that tracks the panthor_devfreq component status). > + return true; > + } > +} > + > static int panthor_devfreq_target(struct device *dev, unsigned long *freq, > u32 flags) > { > + struct panthor_device *ptdev = dev_get_drvdata(dev); > struct dev_pm_opp *opp; > - int err; > > opp = devfreq_recommended_opp(dev, freq, flags); > if (IS_ERR(opp)) > return PTR_ERR(opp); > dev_pm_opp_put(opp); > > - err = dev_pm_opp_set_rate(dev, *freq); > + if (!panthor_devfreq_can_set_rate(ptdev, *freq)) > + return 0; If you return zero here, the new frequency is assumed to be applied, and it messes up with the devfreq state (::previous_freq for instance) that was set on devfreq_suspend_device(). If we assume that rate changes should be rejected while we're suspended, this should probably return an error. > > - return err; > + return dev_pm_opp_set_rate(dev, *freq); There's another race between devfreq_suspend_device(), which will restore the ::resume_freq, and a concurrent devfreq_update_target() call that's here to set a new rate. Using devfreq::suspend_count won't help, because this atomic is decremented before the resume_freq is applied, and the devfreq lock is only taken around the devfreq_set_target() call. Honestly, it feels like we're trying to address the problem at the wrong level. I would expect governors to stop calling devfreq_update_target() between the DEVFREQ_GOV_SUSPEND and DEVFREQ_GOV_RESUME events (which should probably be sent before suspend_count is incremented and the suspend freq is applied). And I'd also expect some kind of locking/synchronization to prevent races between devfreq_update_target() and devfreq_{suspend,resume}_device(). > } > > static void panthor_devfreq_reset(struct panthor_devfreq *pdevfreq) > > --- > base-commit: 8deaade4a5a4ff635df91e246e42139ab2b33f6d > change-id: 20261110-panthor-suspend-clk-11d5604fe5be > > Best regards, > -- > Arun Iyengar <[email protected]> > >
