On 11/11/2026 13:05, Philipp Stanner wrote:
+Cc folks from my other RFC
On Wed, 2026-11-11 at 10:59 +0000, Tvrtko Ursulin wrote:
Currently the starting timestamp of the next job on the timeline is
adjusted to the end of a preceding job. This works if the next job has
been submitted while the preceding job was still executing, but not if it
was submitted in the window between it finishing and free job work running
the timestamp adjustment.
This can happen because the run job and free job work items share the same
ordered workqueue and run job can easily submit the next job before the
free job gets a chance to run. This over accounting window is present
especially with single credit schedulers, but not limited too.
Fix it by only adjusting the starting timestamp if it is not already after
the finish time of the preceding job.
While at it move the logic to a helper to keep indendation at bay.
Signed-off-by: Tvrtko Ursulin <[email protected]>
Fixes: 1774baa64f93 ("drm/scheduler: Change scheduled fence track v2")
In antoher mail you mentioned a userspace change which motivated this
patch. What is that about, some performance issue? So you want it in
all old kernels or not?
Not motivated but discovered while investigating job runtime accounting.
Whether is it worth backporting, assuming you mean cc: stable or not,
and ignoring the fact it would get picked up anyway due Fixes:, I
thought it is not worth it since the error per job would be very small
(how much time between two queued work items on the same workqeueue).
And despite it being accumulating, I *think*, relative to the actual GPU
times it can never grow to be significant. It would require consistent
significant spacing between run_job and free_job and/or super short GPU
jobs. It did not feel worth explicitly putting cc: stable on the patch.
>> Cc: Christian König <[email protected]>
Cc: Danilo Krummrich <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Philipp Stanner <[email protected]>
---
drivers/gpu/drm/scheduler/sched_main.c | 24 +++++++++++++++++++-----
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c
b/drivers/gpu/drm/scheduler/sched_main.c
index a030061d0390..136880fc193d 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -898,6 +898,24 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)
return IS_ERR(entity) ? NULL : entity;
}
+static void
+adjust_next_job_start(struct drm_sched_job *job, struct drm_sched_job *next)
Not nitty with that usually, but since literally all our functions
always start with drm_sched_, this one should probably be
drm_sched_job_XXX()
:shrug: for file local helpers I am not keen on long prefixes so it's
just a habit. I can change it no problem.
+{
+ struct dma_fence *next_scheduled = &next->s_fence->scheduled;
+ ktime_t ts;
+
+ if (!test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &next_scheduled->flags))
+ return;
+
+ /*
+ * If the next job has been submitted while the previous one was still
+ * running adjust it's starting timestamp to avoid accounting overlap.
+ */
+ ts = dma_fence_timestamp(&job->s_fence->finished);
+ if (ktime_after(ts, next_scheduled->timestamp))
+ next_scheduled->timestamp = ts;
+}
+
/**
* drm_sched_get_finished_job - fetch the next finished job to be destroyed
*
@@ -929,11 +947,7 @@ drm_sched_get_finished_job(struct drm_gpu_scheduler *sched)
typeof(*next), list);
if (next) {
/* make the scheduled timestamp more accurate */
- if (test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT,
- &next->s_fence->scheduled.flags))
- next->s_fence->scheduled.timestamp =
-
dma_fence_timestamp(&job->s_fence->finished);
Urgh, only now am I realizing that we just "conveniently" overwrite a
fence's internal timestamp, relying on how dma_fence works internally.
Do you know what ordering is like btw for all of this?
This is the *next* job's scheduled fence, so this sounds suspiciously
racy. Testing the bit and overwriting the timestamp, can't that race?
I have a feeling the code was assuming the default ordered workqueue
ensures either run_job or free_job runs at a time. So seeing timestamped
next fence guarantees free_job has exclusive right to the timestamp write.
That is not true though for the three drivers which use own unordered
workqueues - pvr, nouveau and panthor.
If we look at the drm_sched_run_job_work():
...
drm_sched_job_begin(sched_job);
-> puts on a pending_list for free job to find
...
drm_sched_fence_scheduled(s_fence, fence);
-> signals the scheduled fence
...
If free_job interleaves in the middle, so it sees the next job but it
looks unscheduled it will skip the correction.
But that also means timestamp is yet to be written so there is no need
to correct it. Super subtle but harmless if I am correct. At minium
needs a big fat comment though.
dma_fence_timestamp's cpu_relax() might serve as a barrier, but I can't
see ordering between bit check and overwriting the timestamp?
Yes I think you are right. As said I think there is no real bug due race
with unordered workqueues, but for formal correctness it would require a
memory barrier after test_bit, or full blown locking.
Regards,
Tvrtko
See my recent RFC where I strongly suspect that we have such race's
with fences' error codes [1].
P.
[1]
https://lore.kernel.org/dri-devel/[email protected]/