casaroli opened a new pull request, #21793:
URL: https://github.com/apache/nuttx/pull/21793

   depends-on: apache/nuttx/pull/21778
   
   ## Summary
   
   With `CONFIG_LIBC_ATOMIC_IRQ`, the atomic functions of 
`libs/libc/machine/arch_atomic.c` disable the interrupts with `up_irq_save()`. 
The 8-byte atomics of `libs/libc/machine/arch_atomic64.c` disable them with 
`spin_lock_irqsave()` with every atomics option. The compiler calls these 
functions on the 32-bit targets with no 8-byte atomic instructions (RISC-V, 
Xtensa, ARM M-profile), for example for `_Atomic long long` or 
`std::atomic<int64_t>`.
   
   The user space libc of a PROTECTED or KERNEL build has these functions too. 
The semaphore and mutex fast paths, the condition variables, the temporary 
buffers and the C11 and C++ atomics of the programs use them. User code cannot 
disable the interrupts: RISC-V traps on `csrrc mstatus`/`sstatus`, Xtensa traps 
on `rsil`, and ARM M-profile ignores `cpsid` and the `BASEPRI` write in 
unprivileged mode, so the operation is not atomic.
   
   On `esp32c3-legacy-devkit:knsh`, `nxsem_wait()` in `nuttx_user.elf` calls 
`atomic_compare_exchange_4()`, which runs `csrrc a5,mstatus,a5` in U-mode. On 
`rv-virt:knsh` (32-bit), a program that adds to a `uint64_t` with 
`__atomic_fetch_add()` is killed at the first call: `__atomic_fetch_add_8()` 
runs `csrrc a3,sstatus,a3` in U-mode.
   
   - `libs/libc/machine`: in user space, `atomic_lock()` locks the scheduler 
instead. As with `up_irq_save()`, no other thread runs on this CPU during the 
operation. The 8-byte atomics keep their spinlock for the other CPUs of an SMP 
build. They take it with `spin_trylock_notrace()` in a loop, because 
`spin_lock()` can call kernel functions (`CONFIG_SPINLOCK_INLINE=n`, the 
spinlock instrumentation).
   - `libs/libc/semaphore`: in user space with `CONFIG_LIBC_ATOMIC_IRQ`, 
`nxsem_wait()`, `nxsem_trywait()` and `nxsem_post()` skip the fast path. One 
fast path try is four system calls now (`sched_lock()` and `sched_unlock()` 
around the read and the compare and exchange); the slow path does the whole 
operation in the kernel with one system call. The new macro `NXSEM_FASTPATH` is 
the initial value of the existing `fastpath` flag.
   
   ## Impact
   
   User code of PROTECTED and KERNEL builds. The 4-byte atomics: with 
`CONFIG_LIBC_ATOMIC_IRQ` (in-tree: `esp32c3-legacy-devkit:knsh`). The 8-byte 
atomics: on every 32-bit target with no 8-byte atomic instructions (in-tree for 
example `rv-virt:knsh`, `mps2-an500:knsh`, `esp32-devkitc:knsh`, 
`esp32s3-devkit:knsh`). A user space atomic there is now two system calls; 
before, it trapped or was not atomic. The build mode test is the one that 
already selects the kernel-only code in `include/nuttx/semaphore.h` 
(`NXSEM_SET_UNINTERRUPTIBLE`) and `mm/mm_heap/mm_free.c`.
   
   Flat builds and the kernel do not change: the code is the same before and 
after (see the disassembly table below). In the user space libc of the other 
atomics options (`LIBC_ATOMIC_TOOLCHAIN`, `LIBC_ATOMIC_ARCH`), only 
`arch_atomic64.o` changes. ARMv7-A, ARMv7-R and the 64-bit targets have 8-byte 
atomic instructions: the compiler does not call these functions there. 
`CONFIG_LIBC_ATOMIC_HWSPINLOCK` does not change: its lock is kernel data, which 
user code cannot use. No ABI change. No documentation change: 
`CONFIG_PTHREAD_SPINLOCKS` still needs the toolchain or arch atomics outside a 
flat build, as `Documentation/standards/posix.rst` says.
   
   ## Testing
   
   Test program `uatomtest.c`: four `SCHED_RR` threads of the same priority 
each take a lock N times and increment a plain counter inside it 
(`sem_wait()`/`sem_post()`, `pthread_mutex_lock()`/`unlock()`), increment an 
`atomic_t` with a compare and exchange loop, or add `0x100000001` to a 
`uint64_t` with `__atomic_fetch_add()`. Where the compiler has 8-byte atomic 
instructions, the 8-byte test calls the libc `__atomic_fetch_add_8()` by its 
symbol. A lost count, two threads in the lock at once, two different halves of 
the `uint64_t`, or a semaphore count other than 1 is a FAIL.
   
   QEMU, in-tree configurations (`mps2-an500:knsh` with 
`CONFIG_RR_INTERVAL=10`):
   
   | Configuration | master | this PR |
   |---|---|---|
   | `rv-virt:knsh` (KERNEL, RV32, U-mode) | uatomtest atomic64: `hello` killed 
at the first call: Illegal instruction, MTVAL `0x1006b6f3` (`csrrc 
a3,sstatus,a3`) | uatomtest PASS (sem, mutex, atomic, atomic64); ostest status 
0 |
   | `mps2-an500:knsh` (PROTECTED, ARMv7-M) | uatomtest atomic64 PASS, also 
with 5000000 adds per thread (5.2 s) | uatomtest PASS (sem, mutex, atomic, 
atomic64); ostest status 0; 5000000 adds per thread PASS (170.9 s) |
   | `qemu-armv7a:knsh_smp` (KERNEL, SMP, 4 CPUs) | uatomtest atomic64 PASS | 
uatomtest PASS (sem, mutex, atomic, atomic64 through the SMP spinlock) |
   
   On ARMv7-M, master runs `up_irq_save()` (`msr BASEPRI`) in user mode, which 
the CPU ignores. The test does not show the race in QEMU: the code between the 
load and the store has no branch, and QEMU takes interrupts only between 
translated blocks. On the PR, each 8-byte user atomic is two system calls (33 
times slower in this test). ostest on `qemu-armv7a:knsh_smp` stops with a data 
abort in the signal handler tests on master and on this PR (not related: the 
compiler inlines the 8-byte atomics on ARMv7-A).
   
   `CONFIG_LIBC_ATOMIC_IRQ` forced on in QEMU (no board here has an IRQ atomics 
chip in a protected build):
   
   | Configuration | master | this PR |
   |---|---|---|
   | `rv-virt:knsh64` (KERNEL, U-mode) | `/system/bin/init` killed at boot: 
Illegal instruction, MTVAL `0x1007b7f3` (`csrrc a5,sstatus,a5`) | uatomtest 
PASS (sem, mutex, atomic, atomic64); ostest status 0 |
   | `mps2-an500:knsh` (PROTECTED, ARMv7-M) | uatomtest PASS (the ignored 
`cpsid`/`BASEPRI` write races in a few instructions only); ostest status 0 | 
uatomtest PASS; ostest status 0 |
   
   `-Werror` builds: `rv-virt:knsh`, `rv-virt:knsh64`, `mps2-an500:knsh`, 
`qemu-armv7a:knsh`, `qemu-armv7a:knsh_smp`, `esp32c3-legacy-devkit:knsh`, 
`esp32-devkitc:knsh`, `esp32s3-devkit:knsh`; flat: `rv-virt:nsh`, 
`qemu-armv7a:nsh`, `raspberrypi-pico:nsh`. clang 22: `arch_atomic64.c` with 
`-Werror` for user space and for the kernel of `mps2-an500:knsh` (thumbv7m), 
`rv-virt:knsh` (rv32imac) and `qemu-armv7a:knsh_smp` (ARMv7-A, SMP, 
`CONFIG_SPINLOCK`).
   
   Disassembly (`objdump -d`) of master and this PR, built in two trees:
   
   | Configuration | `libc.a` | `libkc.a` | `nuttx` | `nuttx_user.elf` |
   |---|---|---|---|---|
   | `rv-virt:nsh`, `qemu-armv7a:nsh`, `raspberrypi-pico:nsh` (flat) | same | - 
| same | - |
   | `mps2-an500:knsh` | `arch_atomic64.o` changes | same | same | same |
   | `rv-virt:knsh`, `qemu-armv7a:knsh` | `arch_atomic64.o` changes | same | 
same instructions; `.rodata` addresses differ (the apps path of the two trees) 
| - |
   | `esp32c3-legacy-devkit:knsh` | `arch_atomic.o`, `arch_atomic64.o`, 
`sem_post.o`, `sem_trywait.o` and `sem_wait.o` change | same | same | changes |
   
   On `esp32c3-legacy-devkit:knsh`, `nuttx_user.elf` has no `mstatus` access in 
the atomics any more (130 CSR instructions before, 2 after, both in 
`mm_lock_irq()`, which user code does not call). `nxsem_wait()`, 
`nxsem_trywait()` and `nxsem_post()` jump to the slow path. The user code grows 
by 1066 bytes (the atomics call `sched_lock()` and `sched_unlock()`); the 
kernel does not change.
   
   `tools/checkpatch.sh -c -u -m -g` passes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to