On Tue, Apr 13, 2021 at 10:47:03AM +0800, Huang Ying wrote:
> One typical use case of percpu_ref_tryget() family functions is as
> follows,
>
> if (percpu_ref_tryget(&p->ref)) {
> /* Operate on the other fields of *p */
> }
>
> The refcount needs to be checked before operating on the other fields
> of the data structure (*p), otherwise, the values gotten from the
> other fields may be invalid or inconsistent. To guarantee the correct
> memory ordering, percpu_ref_tryget*() needs to be the ACQUIRE
> operations.
I am not seeing the need for this.
If __ref_is_percpu() returns true, then the overall count must be non-zero
and there will be an RCU grace period between now and the time that this
count becomes zero. For the calls to __ref_is_percpu() enclosed within
rcu_read_lock() and rcu_read_unlock(), the grace period will provide
the needed ordering. (See the comment header for the synchronize_rcu()
function.)
Otherwise, when __ref_is_percpu() returns false, its caller does a
value-returning atomic read-modify-write operation, which provides
full ordering.
Either way, the required acquire semantics (and more) are already
provided, and in particular, this analysis covers the percpu_ref_tryget()
you call out above.
Or am I missing something subtle here?
Thanx, Paul
> This function implements that via using smp_load_acquire() in
> __ref_is_percpu() to read the percpu pointer.
>
> Signed-off-by: "Huang, Ying" <[email protected]>
> Cc: Tejun Heo <[email protected]>
> Cc: Kent Overstreet <[email protected]>
> Cc: "Paul E. McKenney" <[email protected]>
> Cc: Roman Gushchin <[email protected]>
> Cc: Ming Lei <[email protected]>
> Cc: Al Viro <[email protected]>
> Cc: Miaohe Lin <[email protected]>
> ---
> include/linux/percpu-refcount.h | 17 +++++++++++++----
> 1 file changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h
> index 16c35a728b4c..9838f7ea4bf1 100644
> --- a/include/linux/percpu-refcount.h
> +++ b/include/linux/percpu-refcount.h
> @@ -165,13 +165,13 @@ static inline bool __ref_is_percpu(struct percpu_ref
> *ref,
> * !__PERCPU_REF_ATOMIC, which may be set asynchronously, and then
> * used as a pointer. If the compiler generates a separate fetch
> * when using it as a pointer, __PERCPU_REF_ATOMIC may be set in
> - * between contaminating the pointer value, meaning that
> - * READ_ONCE() is required when fetching it.
> + * between contaminating the pointer value, smp_load_acquire()
> + * will prevent this.
> *
> - * The dependency ordering from the READ_ONCE() pairs
> + * The dependency ordering from the smp_load_acquire() pairs
> * with smp_store_release() in __percpu_ref_switch_to_percpu().
> */
> - percpu_ptr = READ_ONCE(ref->percpu_count_ptr);
> + percpu_ptr = smp_load_acquire(&ref->percpu_count_ptr);
>
> /*
> * Theoretically, the following could test just ATOMIC; however,
> @@ -231,6 +231,9 @@ static inline void percpu_ref_get(struct percpu_ref *ref)
> * Returns %true on success; %false on failure.
> *
> * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
> */
> static inline bool percpu_ref_tryget_many(struct percpu_ref *ref,
> unsigned long nr)
> @@ -260,6 +263,9 @@ static inline bool percpu_ref_tryget_many(struct
> percpu_ref *ref,
> * Returns %true on success; %false on failure.
> *
> * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
> */
> static inline bool percpu_ref_tryget(struct percpu_ref *ref)
> {
> @@ -280,6 +286,9 @@ static inline bool percpu_ref_tryget(struct percpu_ref
> *ref)
> * percpu_ref_tryget_live().
> *
> * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
> */
> static inline bool percpu_ref_tryget_live(struct percpu_ref *ref)
> {
> --
> 2.30.2
>
