Module Name: src
Committed By: riastradh
Date: Tue Jul 16 22:45:10 UTC 2024
Modified Files:
src/common/lib/libc/arch/x86_64/atomic: atomic.S
Log Message:
amd64: Fix performance regression in uniprocessor atomics/membars.
Back in 2022, I eliminated the MFENCE hotpatch in membar_sync because
it's essentially always more expensive than LOCK ADD with no benefit
for CPU/CPU store-before-load ordering. (It is relevant only for
non-temporal stores or write-combining memory.)
https://mail-index.netbsd.org/source-changes/2022/07/30/msg140047.html
But in that change, I made a mistake and _also_ eliminated the LOCK
hotpatch on uniprocessor amd64. And our assembler gas helpfully
interprets uppercase LOCK just like lowercase lock and assembles them
the same way, so I didn't notice.
This change restores the LOCK hotpatch, so that when booting on a
uniprocessor system (or a uniprocessor guest on a multicore host),
the LOCK prefix is replaced by NOP for a cheaper instruction.
Found by puzzling over how my explanation for PR kern/57199 could
possibly be correct when (on an amd64 guest) ddb x/i membar_sync kept
showing the lock prefix even in uniprocessor boots.
To generate a diff of this commit:
cvs rdiff -u -r1.30 -r1.31 src/common/lib/libc/arch/x86_64/atomic/atomic.S
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/common/lib/libc/arch/x86_64/atomic/atomic.S
diff -u src/common/lib/libc/arch/x86_64/atomic/atomic.S:1.30 src/common/lib/libc/arch/x86_64/atomic/atomic.S:1.31
--- src/common/lib/libc/arch/x86_64/atomic/atomic.S:1.30 Tue Jul 16 22:44:38 2024
+++ src/common/lib/libc/arch/x86_64/atomic/atomic.S Tue Jul 16 22:45:10 2024
@@ -1,4 +1,4 @@
-/* $NetBSD: atomic.S,v 1.30 2024/07/16 22:44:38 riastradh Exp $ */
+/* $NetBSD: atomic.S,v 1.31 2024/07/16 22:45:10 riastradh Exp $ */
/*-
* Copyright (c) 2007 The NetBSD Foundation, Inc.
@@ -38,6 +38,13 @@
#define ALIAS(f, t) WEAK_ALIAS(f,t)
#endif
+#ifdef _HARDKERNEL
+#include <machine/frameasm.h>
+#define LOCK HOTPATCH(HP_NAME_NOLOCK, 1); lock
+#else
+#define LOCK lock
+#endif
+
.text
/* 32-bit */
