On 04/04/2016 08:22 AM, Peter Zijlstra wrote:
This new form allows using hardware assisted waiting.
Requested-by: Will Deacon<will.dea...@arm.com>
Suggested-by: Linus Torvalds<torva...@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel)<pet...@infradead.org>
---
include/linux/compiler.h | 25 +++++++++++++++++++------
kernel/locking/qspinlock.c | 12 ++++++------
kernel/sched/core.c | 8 ++++----
kernel/sched/sched.h | 2 +-
kernel/smp.c | 2 +-
5 files changed, 31 insertions(+), 18 deletions(-)
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -305,21 +305,34 @@ static __always_inline void __write_once
})
/**
- * smp_cond_acquire() - Spin wait for cond with ACQUIRE ordering
+ * smp_cond_load_acquire() - (Spin) wait for cond with ACQUIRE ordering
+ * @ptr: pointer to the variable to wait on
* @cond: boolean expression to wait for
*
* Equivalent to using smp_load_acquire() on the condition variable but
employs
* the control dependency of the wait to reduce the barrier on many platforms.
*
+ * Due to C lacking lambda expressions we load the value of *ptr into a
+ * pre-named variable @VAL to be used in @cond.
+ *
* The control dependency provides a LOAD->STORE order, the additional RMB
* provides LOAD->LOAD order, together they provide LOAD->{LOAD,STORE} order,
* aka. ACQUIRE.
*/
-#define smp_cond_acquire(cond) do { \
- while (!(cond)) \
- cpu_relax(); \
- smp_rmb(); /* ctrl + rmb := acquire */ \
-} while (0)
+#ifndef smp_cond_load_acquire
+#define smp_cond_load_acquire(ptr, cond_expr) ({ \
+ typeof(ptr) __PTR = (ptr); \
+ typeof(*ptr) VAL; \
+ for (;;) { \
+ VAL = READ_ONCE(*__PTR); \
+ if (cond_expr) \
+ break; \
+ cpu_relax(); \
+ } \
+ smp_rmb(); /* ctrl + rmb := acquire */ \
+ VAL; \
+})
+#endif
Using a predefined VAR seems a bit awkward as a reader of the code may
not know where VAR comes from. How about passing in a variable to hold
the latest value of (*ptr), e.g.
#ifndef smp_cond_load_acquire
#define smp_cond_load_acquire(ptr, var, cond_expr) do { \
typeof(ptr) __PTR = (ptr); \
for (;;) { \
var = READ_ONCE(*__PTR); \
if (cond_expr) \
break; \
cpu_relax(); \
} \
smp_rmb(); /* ctrl + rmb := acquire */ \
} while (0)
#endif
Cheers,
Longman
