[PATCH RFC tip/core/rcu 7/9] locking/percpu-rwsem: Fix the comments outdated by rcu_sync
From: Oleg Nesterov Update the comments broken by the previous change. Signed-off-by: Oleg Nesterov Signed-off-by: Paul E. McKenney --- kernel/locking/percpu-rwsem.c | 50 ++- 1 file changed, 11 insertions(+), 39 deletions(-) diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c index 7abc0e150a22..25b73448929c 100644 --- a/kernel/locking/percpu-rwsem.c +++ b/kernel/locking/percpu-rwsem.c @@ -38,27 +38,12 @@ void percpu_free_rwsem(struct percpu_rw_semaphore *brw) } /* - * This is the fast-path for down_read/up_read, it only needs to ensure - * there is no pending writer (atomic_read(write_ctr) == 0) and inc/dec the - * fast per-cpu counter. The writer uses synchronize_sched_expedited() to - * serialize with the preempt-disabled section below. - * - * The nontrivial part is that we should guarantee acquire/release semantics - * in case when - * - * R_W: down_write() comes after up_read(), the writer should see all - * changes done by the reader - * or - * W_R: down_read() comes after up_write(), the reader should see all - * changes done by the writer + * This is the fast-path for down_read/up_read. If it succeeds we rely + * on the barriers provided by rcu_sync_enter/exit; see the comments in + * percpu_down_write() and percpu_up_write(). * * If this helper fails the callers rely on the normal rw_semaphore and * atomic_dec_and_test(), so in this case we have the necessary barriers. - * - * But if it succeeds we do not have any barriers, atomic_read(write_ctr) or - * __this_cpu_add() below can be reordered with any LOAD/STORE done by the - * reader inside the critical section. See the comments in down_write and - * up_write below. */ static bool update_fast_ctr(struct percpu_rw_semaphore *brw, unsigned int val) { @@ -120,29 +105,15 @@ static int clear_fast_ctr(struct percpu_rw_semaphore *brw) return sum; } -/* - * A writer increments ->write_ctr to force the readers to switch to the - * slow mode, note the atomic_read() check in update_fast_ctr(). - * - * After that the readers can only inc/dec the slow ->slow_read_ctr counter, - * ->fast_read_ctr is stable. Once the writer moves its sum into the slow - * counter it represents the number of active readers. - * - * Finally the writer takes ->rw_sem for writing and blocks the new readers, - * then waits until the slow counter becomes zero. - */ void percpu_down_write(struct percpu_rw_semaphore *brw) { /* -* 1. Ensures that write_ctr != 0 is visible to any down_read/up_read -*so that update_fast_ctr() can't succeed. -* -* 2. Ensures we see the result of every previous this_cpu_add() in -*update_fast_ctr(). +* Make rcu_sync_is_idle() == F and thus disable the fast-path in +* percpu_down_read() and percpu_up_read(), and wait for gp pass. * -* 3. Ensures that if any reader has exited its critical section via -*fast-path, it executes a full memory barrier before we return. -*See R_W case in the comment above update_fast_ctr(). +* The latter synchronises us with the preceding readers which used +* the fast-past, so we can not miss the result of __this_cpu_add() +* or anything else inside their criticial sections. */ rcu_sync_enter(>rss); @@ -161,8 +132,9 @@ void percpu_up_write(struct percpu_rw_semaphore *brw) /* release the lock, but the readers can't use the fast-path */ up_write(>rw_sem); /* -* Insert the barrier before the next fast-path in down_read, -* see W_R case in the comment above update_fast_ctr(). +* Enable the fast-path in percpu_down_read() and percpu_up_read() +* but only after another gp pass; this adds the necessary barrier +* to ensure the reader can't miss the changes done by us. */ rcu_sync_exit(>rss); } -- 1.8.1.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH RFC tip/core/rcu 7/9] locking/percpu-rwsem: Fix the comments outdated by rcu_sync
From: Oleg Nesterov o...@redhat.com Update the comments broken by the previous change. Signed-off-by: Oleg Nesterov o...@redhat.com Signed-off-by: Paul E. McKenney paul...@linux.vnet.ibm.com --- kernel/locking/percpu-rwsem.c | 50 ++- 1 file changed, 11 insertions(+), 39 deletions(-) diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c index 7abc0e150a22..25b73448929c 100644 --- a/kernel/locking/percpu-rwsem.c +++ b/kernel/locking/percpu-rwsem.c @@ -38,27 +38,12 @@ void percpu_free_rwsem(struct percpu_rw_semaphore *brw) } /* - * This is the fast-path for down_read/up_read, it only needs to ensure - * there is no pending writer (atomic_read(write_ctr) == 0) and inc/dec the - * fast per-cpu counter. The writer uses synchronize_sched_expedited() to - * serialize with the preempt-disabled section below. - * - * The nontrivial part is that we should guarantee acquire/release semantics - * in case when - * - * R_W: down_write() comes after up_read(), the writer should see all - * changes done by the reader - * or - * W_R: down_read() comes after up_write(), the reader should see all - * changes done by the writer + * This is the fast-path for down_read/up_read. If it succeeds we rely + * on the barriers provided by rcu_sync_enter/exit; see the comments in + * percpu_down_write() and percpu_up_write(). * * If this helper fails the callers rely on the normal rw_semaphore and * atomic_dec_and_test(), so in this case we have the necessary barriers. - * - * But if it succeeds we do not have any barriers, atomic_read(write_ctr) or - * __this_cpu_add() below can be reordered with any LOAD/STORE done by the - * reader inside the critical section. See the comments in down_write and - * up_write below. */ static bool update_fast_ctr(struct percpu_rw_semaphore *brw, unsigned int val) { @@ -120,29 +105,15 @@ static int clear_fast_ctr(struct percpu_rw_semaphore *brw) return sum; } -/* - * A writer increments -write_ctr to force the readers to switch to the - * slow mode, note the atomic_read() check in update_fast_ctr(). - * - * After that the readers can only inc/dec the slow -slow_read_ctr counter, - * -fast_read_ctr is stable. Once the writer moves its sum into the slow - * counter it represents the number of active readers. - * - * Finally the writer takes -rw_sem for writing and blocks the new readers, - * then waits until the slow counter becomes zero. - */ void percpu_down_write(struct percpu_rw_semaphore *brw) { /* -* 1. Ensures that write_ctr != 0 is visible to any down_read/up_read -*so that update_fast_ctr() can't succeed. -* -* 2. Ensures we see the result of every previous this_cpu_add() in -*update_fast_ctr(). +* Make rcu_sync_is_idle() == F and thus disable the fast-path in +* percpu_down_read() and percpu_up_read(), and wait for gp pass. * -* 3. Ensures that if any reader has exited its critical section via -*fast-path, it executes a full memory barrier before we return. -*See R_W case in the comment above update_fast_ctr(). +* The latter synchronises us with the preceding readers which used +* the fast-past, so we can not miss the result of __this_cpu_add() +* or anything else inside their criticial sections. */ rcu_sync_enter(brw-rss); @@ -161,8 +132,9 @@ void percpu_up_write(struct percpu_rw_semaphore *brw) /* release the lock, but the readers can't use the fast-path */ up_write(brw-rw_sem); /* -* Insert the barrier before the next fast-path in down_read, -* see W_R case in the comment above update_fast_ctr(). +* Enable the fast-path in percpu_down_read() and percpu_up_read() +* but only after another gp pass; this adds the necessary barrier +* to ensure the reader can't miss the changes done by us. */ rcu_sync_exit(brw-rss); } -- 1.8.1.5 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/