From: Dave Hansen <dave.han...@linux.intel.com>
If I sit in a loop and do write()s to small tmpfs files,
__sb_end_write() is third-hottest kernel function due to its
smp_mb().
__sb_end_write() uses the barrier to avoid races with freeze_super()
and its calls to sb_wait_write(). But, now that freeze_super() is
calling synchronize_rcu() before each sb_wait_write() call, we can
use that to our advantage.
The synchronize_rcu() ensures that all __sb_end_write() will see
freeze_super()'s updates to s_writers.counter. That, in turn,
guarantees that __sb_end_write() will try to wake up any subsequent
call by freeze_super() to sb_wait_write().
Cc: Jan Kara <j...@suse.cz>
Cc: Alexander Viro <v...@zeniv.linux.org.uk>
Cc: linux-fsde...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Paul E. McKenney <paul...@linux.vnet.ibm.com>
Cc: Tim Chen <tim.c.c...@linux.intel.com>
Cc: Andi Kleen <a...@linux.intel.com>
Signed-off-by: Dave Hansen <dave.han...@linux.intel.com>
---
b/fs/super.c | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff -puN fs/super.c~selectively-do-barriers-in-__sb_end_write fs/super.c
--- a/fs/super.c~selectively-do-barriers-in-__sb_end_write 2015-06-24
17:14:35.315142611 -0700
+++ b/fs/super.c 2015-06-24 17:14:35.318142745 -0700
@@ -1146,14 +1146,23 @@ out:
*/
void __sb_end_write(struct super_block *sb, int level)
{
+ rcu_read_lock();
percpu_counter_dec(&sb->s_writers.counter[level-1]);
/*
- * Make sure s_writers are updated before we wake up waiters in
- * freeze_super().
+ * We are racing here with freeze_super()'s calls to
+ * sb_wait_write(). We want to ensure that we call
+ * wake_up() whenever one of those calls _might_ be
+ * in sb_wait_write().
+ *
+ * Since freeze_super() does a synchronize_rcu() before
+ * each of its sb_wait_write() calls, it can guarantee
+ * that it sees our update to s_writers.counter as well
+ * as that we see its update to s_writers.frozen.
*/
- smp_mb();
- if (waitqueue_active(&sb->s_writers.wait))
+ if (unlikely(sb->s_writers.frozen >= level))
wake_up(&sb->s_writers.wait);
+ rcu_read_unlock();
+
rwsem_release(&sb->s_writers.lock_map[level-1], 1, _RET_IP_);
}
EXPORT_SYMBOL(__sb_end_write);
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
