On Thu, 2026-11-12 at 12:59 +0000, Tvrtko Ursulin wrote:
> Since this is a lock inversion site we cannot lock
Which deadlock-circumvention strategies have you tried?
Pseudocode:
bool drm_sched_can_queue(struct drm_gpu_scheduler *sched,
struct drm_sched_entity *entity)
{
struct drm_sched_job *s_job;
+ lockdep_assert_held(&entity->lock)
s_job = drm_sched_entity_queue_peek(entity);
drm_sched_rq_select_entity(…)
{
retry:
spin_lock(&rq->lock);
walk_tree(…) {
if (!spin_trylock(&entity->lock) {
spin_unlock(&rq-lock);
udelay(20);
goto retry;
}
ready = !list_empty(&entity->job_list);
}
}
It appears very undesirable for someone to re-walk that tree, I get
that. But do we even know whether that would be a measurable problem in
practice? How often would it occur?
Alternatively, all the other paths that do entity_lock -> rq_lock could
be tested with trylock-patterns. They might be less impacted.
In drm_sched_entity_push_job(), for example, only occasionally are both
locks actually needed, when a job is the first job.
> but truly have to have an unlocked solution.
How do you know that?
> This is effectively replacing one unlocked read (spsc_queue_peek) with a
> different one (READ_ONCE(entity->job_credits)). Something unlocked is
> unavoidable here and I think this is more elegant and simple.
Code full of ACCESS_ONCE and RCU that is not RCU is most certainly not
elegant.
This is elegant:
spin_lock()
list_del()
spin_unlock()
With ACCESS_ONCE, manual barriers etc. pp. no one ever knows what's
accessed where with what ordering and why. It would then be better to
have the problem remain centralized in spsc_queue.h
As you say yourself, this here replaces the half-locked hard to reason
about spsc queue (we don't even know whether that thing is formally
correct) with a half-locked "locked" linked list.
It's no improvement in any way regarding our shared goal, which is a
consistently locked codebase that finally rids us of the race
conditions which are plagueing this system for >10 years
There is really no other way than either redesigning drm_sched so that
the lock inversion disappears altogether, or to address the inversion
with strategies well established in CS.
>
> And because it is an unlocked read it has to be annotated with
> READ_ONCE(). Which means the write side, even though it is done under
> the lock, is paired up and marked with WRITE_ONCE().
>
> 2. drm_sched_entity_pop_job()
>
> This is not a lock inversion site but needs to call sleepable things
> while considering the head of the job queue (->prepare_job()).
>
> This one is the only remaining reason I used list_add_tail_rcu and
> list_del_rcu, just so from here we can safely touch the first entry
> while elsewhere more are being added.
>
> It is possible to do it, as you suggest, by dropping and re-acquiring
> the lock. Note however that too would need a big fat comment why it is
> safe.
Sure, one can write code comments. No big deal. The job is not in the
list anymore. The list is lock-protected. No one can race it.
There's only one worker popping. This must always be that way, the
entire drm_sched design is founded on that principle.
> At which point, since both cases have a big fat comment explaining
> the same thing apart from the implementation details, I do not see the
> advantage of pointless lock cycling. READ_ONCE and WRITE_ONCE you
> dislike would still remain.
>
> As a side note it would be really good to eliminate the prepare_job
> callback
Yes, the user should prepare everything needed before even submitting a
job. Maybe we can begin by marking it as deprecated.
P.