Hi Andrew and Others (please read at least the part with @RT developers),
Yup, mutex_trylock() from interrupt is improper. Well dang, that's a bit silly. Presumably the 2006 spin_lock_mutex() wasn't taken with irqs-off. Ho hum, did you look at switching the kexec code back to the xchg approach?
I looked into reverting to the xchg approach, but that seems to be not a good solution anymore, because the mutex is used in many places, a lot with waiting locks and I guess that would require spinning now, if we do this with bare xchg. Instead I thought about using a spinlock, because they are supposed to be used in interrupt context as well, if I understand the documentation correctly ([1]). @RT developers Unfortunately the rt patches seem to interpret it a bit different and spin_trylock uses __rt_mutex_trylock again, with the same consequences as with the current code. I tried raw_spinlocks, but it looks like they result in a deadlock at least in the rt kernel. Thiy may be because of memory allocations in the critical sections, that are not allowed if I understand it correctly. I have no clue how to fix it at this point. Jörg [1] https://kernel.readthedocs.io/en/sphinx-samples/kernel-locking.html