sdelavega opened a new issue, #22135:
URL: https://github.com/apache/nuttx/issues/22135

   **Environment:** NuttX master 295bf8c385 and 13.1.0 · Arch: xtensa 
(ESP32-S3), also seen on the simulator; the code is architecture independent · 
Area: POSIX / OS components · Host: Ubuntu 24.04 (WSL2)
   
   ### Description / Steps to reproduce the issue
   
   `pthread_cond_wait()` and `pthread_cond_clockwait()` can lose a wakeup on 
SMP. A thread that starts waiting *after* `pthread_cond_broadcast()` or 
`pthread_cond_signal()` can take the post that was meant for a thread that was 
already waiting. The thread that was already waiting then blocks with nothing 
left to wake it. POSIX requires broadcast to unblock every thread blocked on 
the condition when it is called, and signal at least one of them.
   
   Commit ef20f417f0 ("libc/pthread: Make cond wait release and block atomic", 
PR #21834) fixed this on a single CPU. Its message says: "This does not close 
the window against threads on another CPU under SMP." This issue covers that 
SMP case. I did not find an existing issue for it.
   
   #### Mechanism (master 295bf8c385)
   
   `libs/libc/pthread/pthread_condwait.c`:
   
   ```c
    96      sched_lock();
    97
    98      atomic_add(COND_WAIT_COUNT(cond), 1);
    99      ret = pthread_mutex_breaklock(mutex, &nlocks);
   100
   101      status = -nxsem_wait_uninterruptible(&cond->sem);
   102      sched_unlock();
   ```
   
   `libs/libc/pthread/pthread_condbroadcast.c` (`pthread_condsignal.c` lines 
78-81 is the same with one post):
   
   ```c
    81          if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
    ...
    87              ret = -nxsem_post(&cond->sem);
   ```
   
   `pthread_cond_clockwait()` has the same sequence. The post goes to the 
shared `cond->sem` and is not bound to a thread. Between line 99 (mutex 
released) and line 101 (blocked), the waiter is counted but not yet on the 
semaphore's wait list. `sched_lock()` keeps other threads on the same CPU out 
of that window, but not threads on another CPU:
   
   ```
   CPU0: consumer C                    CPU1: producer P
   lock(m); buffer empty
   wait_count++ (1); release m
     ... still before nxsem_wait ...   lock(m); put item
                                       broadcast: wait_count 1 -> 0, post
                                         (nobody blocked: semcount 0 -> 1)
                                       put more items until the buffer is full
                                       wait: wait_count++ (1); release m
                                       nxsem_wait: takes C's post, returns
                                       buffer still full: wait again (blocks)
   nxsem_wait: semcount 0 -> -1, blocks
   ```
   
   Both threads now sleep, with `wait_count` 2 and `semcount` -2, and the 
consumer sleeps with a full buffer. Nothing will post again. This matches the 
state in the ef20f417f0 report (`wait_count == 2`, `semcount == -2`), which was 
hit on one CPU before that commit.
   
   #### Reproducer
   
   `testing/sched/condsteal` (nuttx-apps patch at the end of this issue; I can 
open a PR for it): one mutex, one condition, a bounded buffer, producers and 
consumers that both wait on that condition in predicate loops and wake the 
other side after every item. The main thread is a watchdog: if no item moves 
for 3 s it prints what it sees and exits with status 1.
   
   ```
   condsteal [-n items] [-r rounds] [-p producers] [-c consumers] [-b slots]
             [-t stall_ms] [-T timeout_us] [-s]
   ```
   
   A failing run looks like this:
   
   ```
   nsh> condsteal -n 100000
   condsteal: 1 round(s) of 100000 items, 1 producer(s), 1 consumer(s), 4 
slots, broadcast, timeout 0 us
   condsteal: STALL in round 1 after 3000 ms: produced 172 consumed 168 of 
100000, count 4/4, waiting producers 1 consumers 1, waits 86
   condsteal: lost wakeup: a consumer waits with 4 item(s) in the buffer
   condsteal: FAILED
   ```
   
   To run it:
   
   - ESP32-S3 on Espressif QEMU (`esp-develop-9.2.2-20260417`): 
`./tools/configure.sh esp32s3-devkit:qemu_debug`, enable 
`CONFIG_TESTING_CONDSTEAL`, build, then `qemu-system-xtensa -nographic -machine 
esp32s3 -drive file=nuttx.merged.bin,if=mtd,format=raw` and run `condsteal -n 
100000` at the NSH prompt. My build host has no `genromfs`, so I also set 
`CONFIG_ETC_ROMFS=n` and `CONFIG_INIT_ENTRYPOINT="nsh_main"`, and I disabled 
networking, Wi-Fi and hostfs. With Espressif QEMU I use 
`CONFIG_ESP32S3_RX_FIFO_THRD=1`, because QEMU sometimes does not raise the UART 
RX timeout interrupt and NSH then never sees a short command. For UP, also set 
`CONFIG_SMP=n`.
   - sim: `./tools/configure.sh sim:smp`, enable `CONFIG_TESTING_CONDSTEAL`, 
run `./nuttx`, then `condsteal`.
   
   #### Observed results
   
   ESP32-S3 QEMU, `esp32s3-devkit:qemu_debug`, 2 CPUs for SMP. Each launch is 
one `condsteal -n 100000` (100,000 items) with a 3 s stall watchdog. A cell 
gives the launches that stalled, and in brackets the median number of items 
consumed before the stall.
   
   | Build | broadcast, 1P/1C | signal (`-s`), 1P/1C | broadcast, 1 slot (`-b 
1`) | broadcast, 2P/2C | timed (`-T 1000`, 1P/2C) |
   |---|---|---|---|---|---|
   | 13.1.0 (2b5c0de5e9) SMP | 100/100 (98) | 100/100 (106) | 100/100 (24) | 
0/100 | 0/50 |
   | 13.1.0 UP | 2/100 (67,458) | 5/100 (26,484) | 10/100 (32,719) | 0/100 | 
0/50 |
   | master (295bf8c385) SMP | **100/100 (170)** | **100/100 (260)** | 
**100/100 (36)** | 0/100 | 0/50 |
   | master UP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
   | master + proposed fix, SMP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
   | master + proposed fix, UP | 0/100 | 0/100 | 0/100 | 0/100 | 0/50 |
   
   - The 2-producer/2-consumer variant does not reproduce the bug, even on 
master SMP: the other threads keep posting. With one producer and one consumer, 
every launch on SMP stalls.
   - The timed variant cannot stall, because a timeout makes the waiter check 
again. It exercises timeouts racing wakeups and checks that 
`pthread_cond_destroy()` returns 0 after each round.
   - sim:smp (4 CPUs) on master: 18 of 20 launches of `condsteal -n 5000` 
stalled. With the fix: 0 of 20 stalled; 19 passed, and in 1 the sim process 
itself died (see Testing done).
   
   #### On real silicon (ESP32-S3, 2 CPUs)
   
   A dual-core ESP32-S3 (QFN56 rev v0.2, 16 MB flash), `esp32s3-devkit:nsh` 
with `CONFIG_SMP=y`, 2 CPUs (`ps` shows both idle threads), console on 
USB-Serial/JTAG. Images and logs are available on request.
   
   | Build | broadcast, 1P/1C | signal (`-s`) | broadcast, 1 slot (`-b 1`) | 
2P/2C | timed (`-T 1000`, 1P/2C) | ostest |
   |---|---|---|---|---|---|---|
   | master (295bf8c385) | passed 20 rounds, then **stalled in round 1** of a 
200-round run | **stalled in round 2**, and in round 1 on a rerun | **stalled 
in round 2** | not run | not run | status 0 (1 run) |
   | master + proposed fix | 200 rounds passed | 100 rounds passed | 100 rounds 
passed | 5 rounds passed | 5 rounds passed (21 timeouts) | status 0 in 2 of 3 
runs; 1 run failed `mutex_test: ERROR: The pthread is still running!` |
   
   Every stall printed the lost-wakeup diagnosis (a consumer waiting with items 
in the buffer, or a producer waiting with free slots). The one ostest failure 
with the fix is in `mutex_test` (timed mutex lock, no condition variable); it 
did not recur in two further runs, and I have only one master run to compare, 
so I can't yet say whether it is pre-existing.
   
   #### Proposed fix
   
   Patch: at the end of this issue (against 295bf8c385). It is a proposal for 
discussion, not yet ready to merge: see the known limits below. I can open a PR 
once the design is agreed.
   
   Each waiter queues a node on its own stack, holding its own semaphore, in 
priority order on a list in `pthread_cond_t`. It does this under a new lock in 
`pthread_cond_t`, before it releases the mutex. Under that lock, 
`pthread_cond_signal()` unlinks and posts the first node, and 
`pthread_cond_broadcast()` all of them; each posts exactly the nodes it 
unlinked. A post can only be taken by the thread it was sent to, so the 
waiter's window between releasing the mutex and blocking no longer matters, on 
any CPU, and the `sched_lock()` is no longer needed.
   
   - Timeout: the waiter takes the lock and unlinks its node. If a signal has 
already unlinked it, the post was made under the lock and is already there. The 
waiter takes it, so that no post is left on its stack, and returns 0 rather 
than ETIMEDOUT, so that the signal is not lost.
   - Cancellation (the wait is a cancellation point): handled like a timeout, 
except that a waiter that was already signalled passes the signal on with 
`pthread_cond_signal()`. POSIX requires that a canceled waiter not consume a 
signal when other threads are blocked.
   - `pthread_cond_destroy()` returns EBUSY exactly while a thread is queued. 
Today a canceled waiter is never removed from `wait_count`, so destroy returns 
EBUSY until a later signal or broadcast posts for it.
   - ABI: `pthread_cond_t` becomes `{mutex_t lock; clockid_t clockid; FAR 
struct pthread_cond_waiter_s *waiters;}` instead of `{sem_t sem; clockid_t 
clockid; int wait_count;}`. The size is the same on 32-bit targets without 
`CONFIG_LIBC_MUTEX_BACKTRACE`. It grows on 64-bit targets (a pointer instead of 
an `int`) and by the backtrace array when `CONFIG_LIBC_MUTEX_BACKTRACE > 0`. 
`PTHREAD_COND_INITIALIZER` changes accordingly.
   - Everything stays in libc. The code uses only nxmutex, nxsem and 
`sched_getparam()`, so it calls no kernel-only interface.
   
   Alternatives I considered:
   - A gate or generation scheme that keeps the single shared semaphore (as in 
pthreads-win32 or old glibc). It needs a second semaphore plus counters, and 
new waiters must block until every thread woken by the last broadcast has run. 
That is a priority inversion on the caller's mutex.
   - A waiter that finds it took a post it was not entitled to and re-posts it. 
With several such waiters this cannot be decided from counters alone, and it 
busy-waits.
   
   Known limits of the proposal (for discussion):
   - **Asynchronous termination of a waiter.** The nodes are on the waiters' 
stacks. If a waiter is terminated without running its own exit path, its node 
stays on the list and points into its freed stack. The next signal or broadcast 
then posts into that memory, and `pthread_cond_destroy()` stays EBUSY until 
then. With `CONFIG_CANCELLATION_POINTS=y`, `pthread_cancel()`, the forced 
cancellation on group exit and `task_delete()` wake a blocked waiter with 
ECANCELED, and it unlinks itself. But with `CONFIG_CANCELLATION_POINTS=n`, 
which is the Kconfig default, all three call `nxtask_terminate()` on the 
blocked waiter. So does an asynchronous-type cancel of a waiter that has not 
blocked yet. ostest's cancel test does exactly this in that configuration: test 
1a cancels a thread blocked in `pthread_cond_wait()`, and test 3 then 
broadcasts the same condition. It passed on my builds only because the freed 
stack was not yet reused. Today these cases only leave `wait_count` too high. 
The fix I
  would propose before merging is a hook in the task recovery path 
(`nxtask_recover()`), like `pthread_mutex_inconsistent()` for robust mutexes: 
the waiter records its node in its TLS, and the hook unlinks it from the 
condition variable. Signal and broadcast post under the lock, so a signaller 
that is terminated part-way leaves the not-yet-posted nodes on the list instead 
of stranding them.
   - **Process-shared condition variables across address spaces 
(`CONFIG_BUILD_KERNEL`) are not supported by this design.** The waiter nodes 
are on the waiters' stacks, which another process cannot reach. With the old 
kernel-managed semaphore this case likely worked, so this is a regression for 
it. A cheap guard would be for `pthread_cond_init()` to return ENOTSUP for a 
`PTHREAD_PROCESS_SHARED` attribute under `CONFIG_BUILD_KERNEL`, so that it 
fails loudly instead of corrupting memory. I would add that guard if the 
per-waiter design is kept, but it does not remove the regression.
   - The queue order uses the waiter's priority when it starts waiting. A 
priority change while it waits does not reorder the list, which the semaphore 
wait list does.
   
   #### Testing done
   
   - Every SMP and UP column in the table above, with and without the fix. With 
the fix that is 450 launches and 45 million items per build without a stall.
   - `ostest` from NSH on ESP32-S3 QEMU, master and master + fix, SMP and UP, 2 
runs each: all ended with `Exiting with status 0`. The condition variable, 
cancel and timed wait tests ran in each. For these runs I commented out 
`memmem_test()` and `strto_test()` in `ostest_main.c`: unmodified, ostest stops 
with an `ASSERT` in "libc tests" on this board, on master and with the fix 
alike. 13.1.0's ostest, which has no libc tests, passed on SMP and UP.
   - sim:ostest (UP, `CONFIG_TESTING_OSTEST_LOOPS=5`): within a 600 s limit, 
master and the fix both completed 2 full loops with the same output and no 
errors, then both stalled in the third loop's wdog test.
   - sim:smp under WSL2: long condsteal runs make the sim process crash 
(SIGSEGV) or hang, on master as well as with the fix. On master this also 
happens with `-T 2000`, where timeouts prevent the lost-wakeup stall. So I did 
not use sim:smp for stress.
   - All QEMU builds had `CONFIG_CANCELLATION_POINTS=n` (the `qemu_debug` 
default), so the deferred-cancellation path of the fix was not exercised there.
   - `tools/nxstyle` is clean on the changed files and on the test.
   - Real hardware: see "On real silicon" above.
   - Not tested: protected or kernel builds.
   
   ### Host information
   
   Build and test host: Ubuntu 24.04 under WSL2 (x86_64). Toolchain 
crosstool-NG `esp-14.2.0_20241119` (GCC 14.2.0). Espressif QEMU 
`esp-develop-9.2.2-20260417`.
   
   ### Patches
   
   <details><summary>Proposed fix: 
<code>0001-libc-pthread-Wake-condition-variable-waiters-through.patch</code> 
(nuttx, against 295bf8c385)</summary>
   
   ```diff
   From fee67ad868f44f8ace8ba225e1c53f1d192eccf4 Mon Sep 17 00:00:00 2001
   From: Stephen de la Vega <[email protected]>
   Date: Fri, 20 Nov 2026 14:07:33 -0500
   Subject: [PATCH] libc/pthread: Wake condition variable waiters through
    per-waiter nodes.
   
   pthread_cond_wait() counts itself in wait_count, releases the mutex and
   then blocks on the shared cond->sem.  pthread_cond_signal() and
   pthread_cond_broadcast() post that semaphore once per counted waiter.
   A post is not bound to a thread: if it is made while a counted waiter
   is still between releasing the mutex and blocking, any thread that
   reaches nxsem_wait() first takes it, including one that started waiting
   after the post.  The waiter the post was meant for then blocks with
   nothing left to wake it.  With producers and consumers on one condition
   this deadlocks: the stray wakeup goes to a thread whose predicate is
   still false, it waits again, and the thread whose predicate is true
   sleeps.
   
   Commit ef20f417f0 ("libc/pthread: Make cond wait release and block
   atomic") closes the window on one CPU with sched_lock(), but a thread
   on another CPU can still signal and then wait in it.  On ESP32-S3 QEMU
   with SMP (2 CPUs) and on sim:smp, the testing/sched/condsteal test (one
   mutex, one condition, a bounded buffer, broadcast after every item)
   stalls this way within the first few thousand items.
   
   Give every waiter a node on its stack with its own semaphore.  The node
   is queued in priority order under a new lock in pthread_cond_t before
   the mutex is released.  Under that lock, signal unlinks and posts the
   first node and broadcast all of them.  A post can only be taken by the
   thread it was sent to, so it does not matter how long that thread
   takes to block, on any CPU, and the sched_lock() window is no longer
   needed.
   
   A waiter that times out or is canceled takes the lock and unlinks its
   node.  If a signal or broadcast has already unlinked it, the post was
   made under the lock and is already there: the waiter takes it, so that
   the semaphore is not left posted on its stack.  Then a timed wait
   returns 0 (the signal is reported, not lost), and a canceled wait
   passes the signal on with pthread_cond_signal(), as POSIX requires.
   Before, a timed-out waiter could decrement wait_count after a signal
   had already counted it, leaving wait_count negative and a post that
   the next waiter took as a spurious wakeup, and a canceled waiter was
   never removed from wait_count, so pthread_cond_destroy() returned EBUSY
   until a later signal or broadcast posted for it.
   pthread_cond_destroy() now returns EBUSY exactly while a thread is
   queued.
   
   pthread_cond_t changes from {sem_t sem; clockid_t clockid; int
   wait_count} to {mutex_t lock; clockid_t clockid; FAR struct
   pthread_cond_waiter_s *waiters}.  The size is the same on 32-bit
   targets without CONFIG_LIBC_MUTEX_BACKTRACE; it grows on 64-bit targets
   (pointer instead of int) and by the backtrace array when
   CONFIG_LIBC_MUTEX_BACKTRACE > 0.  PTHREAD_COND_INITIALIZER changes
   accordingly, so code that uses it must be rebuilt.  Everything stays in
   libc and uses only nxmutex, nxsem and sched_getparam().
   
   Known limits:
   
   - A waiter that is terminated asynchronously while it waits leaves its
     node on the list, pointing into its freed stack, and the next signal
     or broadcast posts into that memory.  Deferred cancellation, the
     default with CONFIG_CANCELLATION_POINTS=y, wakes the waiter with
     ECANCELED and it unlinks itself.  But with
     CONFIG_CANCELLATION_POINTS=n (the Kconfig default), pthread_cancel()
     of a waiter, the forced cancellation on group exit, and
     task_delete() of a waiting task go through nxtask_terminate() without
     running the waiter, and so does an asynchronous cancellation of a
     waiter that is not blocked.  Before this change the same cases only
     left wait_count too high.  A fix needs a hook in the task recovery
     path, like pthread_mutex_inconsistent() for robust mutexes.
   - Process-shared condition variables used from different address
     spaces (CONFIG_BUILD_KERNEL) are not supported: the waiter nodes are
     on the waiters' stacks.  The shared kernel-managed semaphore made
     that case work before, so this is a regression for it.
   
   Signed-off-by: Stephen de la Vega <[email protected]>
   Assisted-by: Claude Code
   ---
    include/pthread.h                         |   9 +-
    libs/libc/pthread/pthread.h               |  24 ++-
    libs/libc/pthread/pthread_condbroadcast.c |  48 +++--
    libs/libc/pthread/pthread_condclockwait.c |  50 +----
    libs/libc/pthread/pthread_conddestroy.c   |  22 +--
    libs/libc/pthread/pthread_condinit.c      |  18 +-
    libs/libc/pthread/pthread_condsignal.c    |  30 ++-
    libs/libc/pthread/pthread_condwait.c      | 225 ++++++++++++++++++----
    8 files changed, 286 insertions(+), 140 deletions(-)
   
   diff --git a/include/pthread.h b/include/pthread.h
   index dca4df1008..ac5886a3d4 100644
   --- a/include/pthread.h
   +++ b/include/pthread.h
   @@ -247,11 +247,14 @@ typedef struct pthread_condattr_s pthread_condattr_t;
    #  define __PTHREAD_CONDATTR_T_DEFINED 1
    #endif
    
   +struct pthread_cond_waiter_s; /* Private to libc */
   +
    struct pthread_cond_s
    {
   -  sem_t sem;
   +  mutex_t lock;                              /* Protects waiters */
      clockid_t clockid;
   -  int wait_count;
   +  FAR struct pthread_cond_waiter_s *waiters; /* Blocked threads, highest
   +                                              * priority first */
    };
    
    #ifndef __PTHREAD_COND_T_DEFINED
   @@ -259,7 +262,7 @@ typedef struct pthread_cond_s pthread_cond_t;
    #  define __PTHREAD_COND_T_DEFINED 1
    #endif
    
   -#define PTHREAD_COND_INITIALIZER {SEM_INITIALIZER(0), CLOCK_REALTIME, 0}
   +#define PTHREAD_COND_INITIALIZER {NXMUTEX_INITIALIZER, CLOCK_REALTIME, NULL}
    
    struct pthread_mutexattr_s
    {
   diff --git a/libs/libc/pthread/pthread.h b/libs/libc/pthread/pthread.h
   index e4a2e6bd4c..3dc9230470 100644
   --- a/libs/libc/pthread/pthread.h
   +++ b/libs/libc/pthread/pthread.h
   @@ -73,8 +73,6 @@
    #  define mutex_setprioceiling(m,p,o) nxmutex_setprioceiling(m,p,o)
    #endif
    
   -#define COND_WAIT_COUNT(cond) ((FAR atomic_t *)&(cond)->wait_count)
   -
    /* Validate if a pthread_attr_t object is valid.
     *
     * This macro checks:
   @@ -142,10 +140,32 @@
         ((mutex)->mutex.sem.flags & SEM_TYPE_MUTEX))
    #endif
    
   
+/****************************************************************************
   + * Public Types
   + 
****************************************************************************/
   +
   +/* A thread blocked in pthread_cond_wait() or pthread_cond_clockwait().  The
   + * node lives on the stack of the waiting thread and is linked into
   + * pthread_cond_t::waiters while the thread waits.
   + */
   +
   +struct pthread_cond_waiter_s
   +{
   +  FAR struct pthread_cond_waiter_s *flink; /* Next waiter */
   +  sem_t sem;                               /* Posted once to wake it */
   +  int priority;                            /* Priority when it queued */
   +  bool signalled;                          /* Taken off the list by a
   +                                            * signal or broadcast */
   +};
   +
    
/****************************************************************************
     * Public Function Prototypes
     
****************************************************************************/
    
   +int pthread_cond_dowait(FAR pthread_cond_t *cond,
   +                        FAR pthread_mutex_t *mutex, clockid_t clockid,
   +                        FAR const struct timespec *abstime);
   +
    #ifndef CONFIG_PTHREAD_MUTEX_UNSAFE
    int pthread_mutex_take(FAR struct pthread_mutex_s *mutex,
                           FAR const struct timespec *abs_timeout);
   diff --git a/libs/libc/pthread/pthread_condbroadcast.c 
b/libs/libc/pthread/pthread_condbroadcast.c
   index 792ef1c422..b59aee95fe 100644
   --- a/libs/libc/pthread/pthread_condbroadcast.c
   +++ b/libs/libc/pthread/pthread_condbroadcast.c
   @@ -31,7 +31,8 @@
    #include <errno.h>
    #include <nuttx/debug.h>
    
   -#include <nuttx/atomic.h>
   +#include <nuttx/mutex.h>
   +#include <nuttx/semaphore.h>
    
    #include "pthread.h"
    
   @@ -72,25 +73,42 @@ int pthread_cond_broadcast(FAR pthread_cond_t *cond)
        }
      else
        {
   -      int wcnt = atomic_read(COND_WAIT_COUNT(cond));
   -
   -      /* Loop until all of the waiting threads have been restarted. */
   -
   -      while (wcnt > 0)
   +      FAR struct pthread_cond_waiter_s *waiter;
   +      FAR struct pthread_cond_waiter_s *next;
   +      int status;
   +
   +      /* Take every waiter off the list and post it, in priority order.
   +       * Threads that start waiting after this point queue on the now empty
   +       * list and are not woken.  The posts are made under the lock, so a
   +       * waiter that finds itself signalled under the lock knows that its
   +       * post is already there.  A node lives on the stack of its waiter
   +       * until the waiter takes the post, so read the link first.
   +       */
   +
   +      nxmutex_lock(&cond->lock);
   +      waiter = cond->waiters;
   +      cond->waiters = NULL;
   +
   +      while (waiter != NULL)
            {
   -          if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
   -            {
   -              /* Post the condition semaphore to wake up a waiting thread.
   -               * Only the highest priority waiting thread will get to 
execute
   -               */
   -
   -              ret = -nxsem_post(&cond->sem);
   +          next              = waiter->flink;
   +          waiter->signalled = true;
    
   -              /* Decrement the waiter count */
   +          /* Each node's semaphore starts at 0 and is posted only here, so
   +           * the post cannot fail; keep the first error and go on anyway,
   +           * so that the other waiters are still woken.
   +           */
    
   -              wcnt--;
   +          status = -nxsem_post(&waiter->sem);
   +          if (ret == OK)
   +            {
   +              ret = status;
                }
   +
   +          waiter = next;
            }
   +
   +      nxmutex_unlock(&cond->lock);
        }
    
      sinfo("Returning %d\n", ret);
   diff --git a/libs/libc/pthread/pthread_condclockwait.c 
b/libs/libc/pthread/pthread_condclockwait.c
   index 23687307b7..5be99f84a6 100644
   --- a/libs/libc/pthread/pthread_condclockwait.c
   +++ b/libs/libc/pthread/pthread_condclockwait.c
   @@ -36,14 +36,7 @@
    #include <assert.h>
    #include <nuttx/debug.h>
    
   -#include <nuttx/atomic.h>
   -#include <nuttx/irq.h>
   -#include <nuttx/wdog.h>
   -#include <nuttx/signal.h>
    #include <nuttx/cancelpt.h>
   -#include <nuttx/pthread.h>
   -#include <nuttx/atomic.h>
   -#include <nuttx/sched.h>
    
    #include "pthread.h"
    
   @@ -77,7 +70,6 @@ int pthread_cond_clockwait(FAR pthread_cond_t *cond,
                               FAR const struct timespec *abstime)
    {
      int ret = OK;
   -  int status;
    
      sinfo("cond=%p mutex=%p abstime=%p\n", cond, mutex, abstime);
    
   @@ -121,47 +113,7 @@ int pthread_cond_clockwait(FAR pthread_cond_t *cond,
    
      else
        {
   -      unsigned int nlocks;
   -
   -      sinfo("Give up mutex...\n");
   -
   -      sched_lock();
   -
   -      atomic_add(COND_WAIT_COUNT(cond), 1);
   -
   -      /* Give up the mutex */
   -
   -      ret = pthread_mutex_breaklock(mutex, &nlocks);
   -      if (ret == 0)
   -        {
   -          status = nxsem_clockwait_uninterruptible(&cond->sem,
   -                                                   clockid, abstime);
   -          if (status < 0)
   -            {
   -              ret = -status;
   -            }
   -        }
   -
   -      sched_unlock();
   -
   -      /* Reacquire the mutex (retaining the ret). */
   -
   -      sinfo("Re-locking...\n");
   -
   -      status = pthread_mutex_restorelock(mutex, nlocks);
   -
   -      /* When the wait times out (ETIMEDOUT), we must decrement
   -       * wait_count to maintain proper accounting.
   -       */
   -
   -      if (ret == ETIMEDOUT)
   -        {
   -          atomic_sub(COND_WAIT_COUNT(cond), 1);
   -        }
   -      else if (ret == 0)
   -        {
   -          ret = status;
   -        }
   +      ret = pthread_cond_dowait(cond, mutex, clockid, abstime);
        }
    
      leave_cancellation_point();
   diff --git a/libs/libc/pthread/pthread_conddestroy.c 
b/libs/libc/pthread/pthread_conddestroy.c
   index 8fc30ce6d9..6de189c21f 100644
   --- a/libs/libc/pthread/pthread_conddestroy.c
   +++ b/libs/libc/pthread/pthread_conddestroy.c
   @@ -30,7 +30,7 @@
    #include <nuttx/debug.h>
    #include <errno.h>
    
   -#include <nuttx/atomic.h>
   +#include <nuttx/mutex.h>
    
    #include "pthread.h"
    
   @@ -60,7 +60,6 @@
    int pthread_cond_destroy(FAR pthread_cond_t *cond)
    {
      int ret = OK;
   -  int sval = 0;
    
      sinfo("cond=%p\n", cond);
    
   @@ -75,24 +74,19 @@ int pthread_cond_destroy(FAR pthread_cond_t *cond)
       * the state of the condition variable.
       */
    
   -  else if (atomic_read(COND_WAIT_COUNT(cond)) > 0)
   -    {
   -      ret = EBUSY;
   -    }
      else
        {
   -      ret = nxsem_get_value(&cond->sem, &sval);
   -      if (ret < 0)
   -        {
   -          ret = -ret;
   -        }
   -      else if (sval < 0)
   +      nxmutex_lock(&cond->lock);
   +      if (cond->waiters != NULL)
            {
              ret = EBUSY;
            }
   -      else
   +
   +      nxmutex_unlock(&cond->lock);
   +
   +      if (ret == OK)
            {
   -          ret = -nxsem_destroy(&cond->sem);
   +          ret = -nxmutex_destroy(&cond->lock);
              if (ret == OK)
                {
                  cond->clockid = 0;
   diff --git a/libs/libc/pthread/pthread_condinit.c 
b/libs/libc/pthread/pthread_condinit.c
   index c13bf06744..516b41fb69 100644
   --- a/libs/libc/pthread/pthread_condinit.c
   +++ b/libs/libc/pthread/pthread_condinit.c
   @@ -27,7 +27,7 @@
    #include <nuttx/config.h>
    
    #include <pthread.h>
   -#include <semaphore.h>
   +#include <nuttx/mutex.h>
    #include <nuttx/debug.h>
    #include <errno.h>
    
   @@ -74,18 +74,16 @@ int pthread_cond_init(FAR pthread_cond_t *cond,
          ret = EINVAL;
        }
    
   -  /* Initialize the semaphore contained in the condition structure with
   -   * initial count = 0
   -   */
   +  /* Initialize the lock that protects the list of waiters */
    
   -  else if (sem_init(&cond->sem, 0, 0) != OK)
   -    {
   -      ret = get_errno();
   -    }
      else
        {
   -      cond->clockid = attr ? attr->clockid : CLOCK_REALTIME;
   -      cond->wait_count = 0;
   +      ret = -nxmutex_init(&cond->lock);
   +      if (ret == OK)
   +        {
   +          cond->clockid = attr ? attr->clockid : CLOCK_REALTIME;
   +          cond->waiters = NULL;
   +        }
        }
    
      sinfo("Returning %d\n", ret);
   diff --git a/libs/libc/pthread/pthread_condsignal.c 
b/libs/libc/pthread/pthread_condsignal.c
   index 39c07c977a..498431ef67 100644
   --- a/libs/libc/pthread/pthread_condsignal.c
   +++ b/libs/libc/pthread/pthread_condsignal.c
   @@ -30,7 +30,8 @@
    #include <errno.h>
    #include <nuttx/debug.h>
    
   -#include <nuttx/atomic.h>
   +#include <nuttx/mutex.h>
   +#include <nuttx/semaphore.h>
    
    #include "pthread.h"
    
   @@ -71,17 +72,28 @@ int pthread_cond_signal(FAR pthread_cond_t *cond)
        }
      else
        {
   -      int wcnt = atomic_read(COND_WAIT_COUNT(cond));
   +      FAR struct pthread_cond_waiter_s *waiter;
    
   -      while (wcnt > 0)
   +      /* Take the highest priority waiter off the list and post it.  Only
   +       * that thread can consume the post, even if it has not blocked yet.
   +       * The post is made under the lock, so a waiter that finds itself
   +       * signalled under the lock knows that its post is already there.
   +       * The node lives on the stack of the waiter until the waiter takes
   +       * the post, so the post is the last access to it.
   +       */
   +
   +      nxmutex_lock(&cond->lock);
   +      waiter = cond->waiters;
   +      if (waiter != NULL)
            {
   -          if (atomic_cmpxchg(COND_WAIT_COUNT(cond), &wcnt, wcnt - 1))
   -            {
   -              sinfo("Signalling...\n");
   -              ret = -nxsem_post(&cond->sem);
   -              break;
   -            }
   +          cond->waiters     = waiter->flink;
   +          waiter->signalled = true;
   +
   +          sinfo("Signalling...\n");
   +          ret = -nxsem_post(&waiter->sem);
            }
   +
   +      nxmutex_unlock(&cond->lock);
        }
    
      sinfo("Returning %d\n", ret);
   diff --git a/libs/libc/pthread/pthread_condwait.c 
b/libs/libc/pthread/pthread_condwait.c
   index be6326c92a..89c29c6d77 100644
   --- a/libs/libc/pthread/pthread_condwait.c
   +++ b/libs/libc/pthread/pthread_condwait.c
   @@ -32,89 +32,191 @@
    #include <errno.h>
    #include <nuttx/debug.h>
    
   -#include <nuttx/atomic.h>
    #include <nuttx/cancelpt.h>
   +#include <nuttx/mutex.h>
    #include <nuttx/pthread.h>
   -#include <nuttx/sched.h>
   +#include <nuttx/semaphore.h>
    
    #include "pthread.h"
    
   
+/****************************************************************************
   + * Private Functions
   + 
****************************************************************************/
   +
   
+/****************************************************************************
   + * Name: pthread_cond_unlink
   + *
   + * Description:
   + *   Remove a waiter that was not signalled from the list of waiters.  The
   + *   caller holds cond->lock.
   + *
   + 
****************************************************************************/
   +
   +static void pthread_cond_unlink(FAR pthread_cond_t *cond,
   +                                FAR struct pthread_cond_waiter_s *waiter)
   +{
   +  FAR struct pthread_cond_waiter_s **link = &cond->waiters;
   +
   +  while (*link != waiter)
   +    {
   +      DEBUGASSERT(*link != NULL);
   +      link = &(*link)->flink;
   +    }
   +
   +  *link = waiter->flink;
   +}
   +
    
/****************************************************************************
     * Public Functions
     
****************************************************************************/
    
    
/****************************************************************************
   - * Name: int pthread_cond_wait
   + * Name: pthread_cond_dowait
     *
     * Description:
   - *   A thread can wait for a condition variable to be signalled or 
broadcast.
   + *   Common body of pthread_cond_wait() and pthread_cond_clockwait().
   + *
   + *   Each waiter queues a node with its own semaphore on its stack, and
   + *   does so before it releases the mutex.  pthread_cond_signal() and
   + *   pthread_cond_broadcast() unlink the nodes they wake and post the
   + *   semaphores of exactly those nodes, under cond->lock.  A wakeup is thus
   + *   bound to a thread that was waiting when it was sent: a thread that
   + *   starts waiting later, on this or any other CPU, cannot consume it,
   + *   however long the waiter takes between releasing the mutex and
   + *   blocking.
     *
     * Input Parameters:
   - *   None
   + *   cond    - The condition variable to wait on
   + *   mutex   - The mutex that protects the condition, held by the caller
   + *   clockid - The clock of abstime
   + *   abstime - The absolute timeout, or NULL to wait without a timeout
     *
     * Returned Value:
   - *   None
   + *   OK (0) on success; a positive errno value on failure.  ETIMEDOUT is
   + *   returned only if the waiter was not signalled: a signal that races
   + *   with the timeout is reported as success.
     *
     * Assumptions:
   + *   The caller has validated cond, mutex and abstime, checked that it
   + *   holds the mutex, and entered the cancellation point.
     *
     
****************************************************************************/
    
   -int pthread_cond_wait(FAR pthread_cond_t *cond, FAR pthread_mutex_t *mutex)
   +int pthread_cond_dowait(FAR pthread_cond_t *cond,
   +                        FAR pthread_mutex_t *mutex, clockid_t clockid,
   +                        FAR const struct timespec *abstime)
    {
   +  struct pthread_cond_waiter_s waiter;
   +  FAR struct pthread_cond_waiter_s **link;
   +  struct sched_param param;
   +  unsigned int nlocks;
   +  bool signalled;
   +  bool released;
      int status;
      int ret;
    
   -  sinfo("cond=%p mutex=%p\n", cond, mutex);
   +  /* Queue behind the waiters of the same or higher priority, so that
   +   * signals keep waking the highest priority waiter first.
   +   */
    
   -  /* pthread_cond_wait() is a cancellation point */
   +  if (sched_getparam(0, &param) < 0)
   +    {
   +      param.sched_priority = SCHED_PRIORITY_MIN;
   +    }
    
   -  enter_cancellation_point();
   +  nxsem_init(&waiter.sem, 0, 0);
   +  waiter.priority  = param.sched_priority;
   +  waiter.signalled = false;
    
   -  /* Make sure that non-NULL references were provided. */
   +  nxmutex_lock(&cond->lock);
    
   -  if (!PTHREAD_COND_IS_VALID(cond) || !PTHREAD_MUTEX_IS_VALID(mutex))
   +  link = &cond->waiters;
   +  while (*link != NULL && (*link)->priority >= waiter.priority)
        {
   -      ret = EINVAL;
   +      link = &(*link)->flink;
        }
    
   -  /* Make sure that the caller holds the mutex */
   +  waiter.flink = *link;
   +  *link        = &waiter;
    
   -  else if (!mutex_is_hold(&mutex->mutex))
   -    {
   -      ret = EPERM;
   -    }
   -  else
   -    {
   -      unsigned int nlocks;
   +  nxmutex_unlock(&cond->lock);
    
   -      /* Give up the mutex */
   +  /* Give up the mutex.  From here on, a signal or broadcast that finds the
   +   * node posts waiter.sem, whether or not this thread has blocked yet.
   +   */
    
   -      sinfo("Give up mutex / take cond\n");
   +  sinfo("Give up mutex / take cond\n");
    
   -      /* Unlock and block atomically, or a later waiter can steal the post 
*/
   +  ret = pthread_mutex_breaklock(mutex, &nlocks);
   +  released = ret == OK;
   +  if (released)
   +    {
   +      if (abstime != NULL)
   +        {
   +          ret = -nxsem_clockwait_uninterruptible(&waiter.sem, clockid,
   +                                                 abstime);
   +        }
   +      else
   +        {
   +          ret = -nxsem_wait_uninterruptible(&waiter.sem);
   +        }
   +    }
    
   -      sched_lock();
   +  if (ret != OK)
   +    {
   +      /* Timed out, canceled, or the mutex could not be released.  Leave
   +       * the list unless a signal or broadcast has already taken the node:
   +       * then its post has been made, under cond->lock, and must be
   +       * consumed before the node leaves this stack.
   +       */
    
   -      atomic_add(COND_WAIT_COUNT(cond), 1);
   -      ret = pthread_mutex_breaklock(mutex, &nlocks);
   +      nxmutex_lock(&cond->lock);
   +      signalled = waiter.signalled;
   +      if (!signalled)
   +        {
   +          pthread_cond_unlink(cond, &waiter);
   +        }
    
   -      status = -nxsem_wait_uninterruptible(&cond->sem);
   -      sched_unlock();
   +      nxmutex_unlock(&cond->lock);
    
   -      if (ret == OK)
   +      if (signalled)
            {
   -          /* Report the first failure that occurs */
   +          do
   +            {
   +              status = nxsem_wait_uninterruptible(&waiter.sem);
   +            }
   +          while (status < 0);
    
   -          ret = status;
   +          if (ret == ETIMEDOUT)
   +            {
   +              /* The signal raced with the timeout: report the wakeup, so
   +               * that the signal is not lost.
   +               */
   +
   +              ret = OK;
   +            }
   +          else
   +            {
   +              /* A canceled thread must not consume a signal that another
   +               * waiter could take: pass it on.
   +               */
   +
   +              pthread_cond_signal(cond);
   +            }
            }
   +    }
    
   -      /* Reacquire the mutex.
   -       *
   -       * When cancellation points are enabled, we need to hold the mutex
   -       * when the pthread is canceled and cleanup handlers, if any, are
   -       * entered.
   -       */
   +  nxsem_destroy(&waiter.sem);
   +
   +  /* Reacquire the mutex.
   +   *
   +   * When cancellation points are enabled, we need to hold the mutex
   +   * when the pthread is canceled and cleanup handlers, if any, are
   +   * entered.
   +   */
    
   +  if (released)
   +    {
          sinfo("Reacquire mutex...\n");
    
          status = pthread_mutex_restorelock(mutex, nlocks);
   @@ -126,6 +228,53 @@ int pthread_cond_wait(FAR pthread_cond_t *cond, FAR 
pthread_mutex_t *mutex)
            }
        }
    
   +  return ret;
   +}
   +
   
+/****************************************************************************
   + * Name: pthread_cond_wait
   + *
   + * Description:
   + *   A thread can wait for a condition variable to be signalled or 
broadcast.
   + *
   + * Input Parameters:
   + *   None
   + *
   + * Returned Value:
   + *   None
   + *
   + * Assumptions:
   + *
   + 
****************************************************************************/
   +
   +int pthread_cond_wait(FAR pthread_cond_t *cond, FAR pthread_mutex_t *mutex)
   +{
   +  int ret;
   +
   +  sinfo("cond=%p mutex=%p\n", cond, mutex);
   +
   +  /* pthread_cond_wait() is a cancellation point */
   +
   +  enter_cancellation_point();
   +
   +  /* Make sure that non-NULL references were provided. */
   +
   +  if (!PTHREAD_COND_IS_VALID(cond) || !PTHREAD_MUTEX_IS_VALID(mutex))
   +    {
   +      ret = EINVAL;
   +    }
   +
   +  /* Make sure that the caller holds the mutex */
   +
   +  else if (!mutex_is_hold(&mutex->mutex))
   +    {
   +      ret = EPERM;
   +    }
   +  else
   +    {
   +      ret = pthread_cond_dowait(cond, mutex, cond->clockid, NULL);
   +    }
   +
      leave_cancellation_point();
      sinfo("Returning %d\n", ret);
      return ret;
   -- 
   2.43.0
   
   ```
   
   </details>
   
   <details><summary>Reproducer: <code>testing/sched/condsteal</code> 
(nuttx-apps, against c3b0f42182)</summary>
   
   ```diff
   From 85c568bd5b3076879fa5d44007a95e71a91bdd6b Mon Sep 17 00:00:00 2001
   From: Stephen de la Vega <[email protected]>
   Date: Fri, 20 Nov 2026 14:18:44 -0500
   Subject: [PATCH] testing/sched/condsteal: Add a condition variable 
lost-wakeup
    test.
   
   Producers and consumers share one bounded buffer, one mutex and one
   condition variable, and wake each other with pthread_cond_broadcast()
   (or pthread_cond_signal() with -s) after every item.  The main thread
   is a watchdog: when no item moves for the stall time (-t, 3 s by
   default) it prints what it can see (items in the buffer, how many
   producers and consumers are inside pthread_cond_wait()) and exits with
   status 1.  A consumer that waits with items in the buffer, or a producer
   that waits with free slots, means that a wakeup was lost.
   
   -T makes every wait a pthread_cond_timedwait() with a short timeout, so
   that timeouts race with wakeups; each round also checks that
   pthread_cond_destroy() succeeds afterwards.
   
   Signed-off-by: Stephen de la Vega <[email protected]>
   Assisted-by: Claude Code
   ---
    testing/sched/condsteal/CMakeLists.txt   |  41 +++
    testing/sched/condsteal/Kconfig          |  27 ++
    testing/sched/condsteal/Make.defs        |  25 ++
    testing/sched/condsteal/Makefile         |  32 ++
    testing/sched/condsteal/condsteal_main.c | 432 +++++++++++++++++++++++
    5 files changed, 557 insertions(+)
    create mode 100644 testing/sched/condsteal/CMakeLists.txt
    create mode 100644 testing/sched/condsteal/Kconfig
    create mode 100644 testing/sched/condsteal/Make.defs
    create mode 100644 testing/sched/condsteal/Makefile
    create mode 100644 testing/sched/condsteal/condsteal_main.c
   
   diff --git a/testing/sched/condsteal/CMakeLists.txt 
b/testing/sched/condsteal/CMakeLists.txt
   new file mode 100644
   index 000000000..0a631391e
   --- /dev/null
   +++ b/testing/sched/condsteal/CMakeLists.txt
   @@ -0,0 +1,41 @@
   +# 
##############################################################################
   +# apps/testing/sched/condsteal/CMakeLists.txt
   +#
   +# SPDX-License-Identifier: Apache-2.0
   +#
   +# Licensed to the Apache Software Foundation (ASF) under one or more 
contributor
   +# license agreements.  See the NOTICE file distributed with this work for
   +# additional information regarding copyright ownership.  The ASF licenses 
this
   +# file to you under the Apache License, Version 2.0 (the "License"); you 
may not
   +# use this file except in compliance with the License.  You may obtain a 
copy of
   +# the License at
   +#
   +# http://www.apache.org/licenses/LICENSE-2.0
   +#
   +# Unless required by applicable law or agreed to in writing, software
   +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
   +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
   +# License for the specific language governing permissions and limitations 
under
   +# the License.
   +#
   +# 
##############################################################################
   +
   +if(CONFIG_TESTING_CONDSTEAL)
   +
   +  set(SRCS condsteal_main.c)
   +
   +  nuttx_add_application(
   +    NAME
   +    condsteal
   +    PRIORITY
   +    ${CONFIG_TESTING_CONDSTEAL_PRIORITY}
   +    STACKSIZE
   +    ${CONFIG_TESTING_CONDSTEAL_STACKSIZE}
   +    MODULE
   +    ${CONFIG_TESTING_CONDSTEAL}
   +    COMPILE_FLAGS
   +    ${FLAGS}
   +    SRCS
   +    ${SRCS})
   +
   +endif()
   diff --git a/testing/sched/condsteal/Kconfig 
b/testing/sched/condsteal/Kconfig
   new file mode 100644
   index 000000000..4b6c11bfc
   --- /dev/null
   +++ b/testing/sched/condsteal/Kconfig
   @@ -0,0 +1,27 @@
   +#
   +# For a description of the syntax of this configuration file,
   +# see the file kconfig-language.txt in the NuttX tools repository.
   +#
   +
   +config TESTING_CONDSTEAL
   +    tristate "Condition variable lost-wakeup test"
   +    default n
   +    ---help---
   +            Producers and consumers share one bounded buffer, one mutex and
   +            one condition variable, and wake each other after every item.
   +            A watchdog reports a stall, which means that a wakeup was lost,
   +            for example when a thread that starts waiting after
   +            pthread_cond_broadcast() or pthread_cond_signal() consumes the
   +            wakeup of a thread that was already waiting.
   +
   +if TESTING_CONDSTEAL
   +
   +config TESTING_CONDSTEAL_PRIORITY
   +    int "Task priority"
   +    default 100
   +
   +config TESTING_CONDSTEAL_STACKSIZE
   +    int "Stack size"
   +    default DEFAULT_TASK_STACKSIZE
   +
   +endif
   diff --git a/testing/sched/condsteal/Make.defs 
b/testing/sched/condsteal/Make.defs
   new file mode 100644
   index 000000000..af2e8ad23
   --- /dev/null
   +++ b/testing/sched/condsteal/Make.defs
   @@ -0,0 +1,25 @@
   +############################################################################
   +# apps/testing/sched/condsteal/Make.defs
   +#
   +# SPDX-License-Identifier: Apache-2.0
   +#
   +# Licensed to the Apache Software Foundation (ASF) under one or more
   +# contributor license agreements.  See the NOTICE file distributed with
   +# this work for additional information regarding copyright ownership.  The
   +# ASF licenses this file to you under the Apache License, Version 2.0 (the
   +# "License"); you may not use this file except in compliance with the
   +# License.  You may obtain a copy of the License at
   +#
   +#   http://www.apache.org/licenses/LICENSE-2.0
   +#
   +# Unless required by applicable law or agreed to in writing, software
   +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
   +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
   +# License for the specific language governing permissions and limitations
   +# under the License.
   +#
   +############################################################################
   +
   +ifneq ($(CONFIG_TESTING_CONDSTEAL),)
   +CONFIGURED_APPS += $(APPDIR)/testing/sched/condsteal
   +endif
   diff --git a/testing/sched/condsteal/Makefile 
b/testing/sched/condsteal/Makefile
   new file mode 100644
   index 000000000..fb506786d
   --- /dev/null
   +++ b/testing/sched/condsteal/Makefile
   @@ -0,0 +1,32 @@
   +############################################################################
   +# apps/testing/sched/condsteal/Makefile
   +#
   +# SPDX-License-Identifier: Apache-2.0
   +#
   +# Licensed to the Apache Software Foundation (ASF) under one or more
   +# contributor license agreements.  See the NOTICE file distributed with
   +# this work for additional information regarding copyright ownership.  The
   +# ASF licenses this file to you under the Apache License, Version 2.0 (the
   +# "License"); you may not use this file except in compliance with the
   +# License.  You may obtain a copy of the License at
   +#
   +#   http://www.apache.org/licenses/LICENSE-2.0
   +#
   +# Unless required by applicable law or agreed to in writing, software
   +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
   +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
   +# License for the specific language governing permissions and limitations
   +# under the License.
   +#
   +############################################################################
   +
   +include $(APPDIR)/Make.defs
   +
   +PROGNAME  = condsteal
   +PRIORITY  = $(CONFIG_TESTING_CONDSTEAL_PRIORITY)
   +STACKSIZE = $(CONFIG_TESTING_CONDSTEAL_STACKSIZE)
   +MODULE    = $(CONFIG_TESTING_CONDSTEAL)
   +
   +MAINSRC = condsteal_main.c
   +
   +include $(APPDIR)/Application.mk
   diff --git a/testing/sched/condsteal/condsteal_main.c 
b/testing/sched/condsteal/condsteal_main.c
   new file mode 100644
   index 000000000..5ae81b197
   --- /dev/null
   +++ b/testing/sched/condsteal/condsteal_main.c
   @@ -0,0 +1,432 @@
   
+/****************************************************************************
   + * apps/testing/sched/condsteal/condsteal_main.c
   + *
   + * SPDX-License-Identifier: Apache-2.0
   + *
   + * Licensed to the Apache Software Foundation (ASF) under one or more
   + * contributor license agreements.  See the NOTICE file distributed with
   + * this work for additional information regarding copyright ownership.  The
   + * ASF licenses this file to you under the Apache License, Version 2.0 (the
   + * "License"); you may not use this file except in compliance with the
   + * License.  You may obtain a copy of the License at
   + *
   + *   http://www.apache.org/licenses/LICENSE-2.0
   + *
   + * Unless required by applicable law or agreed to in writing, software
   + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
   + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
   + * License for the specific language governing permissions and limitations
   + * under the License.
   + *
   + 
****************************************************************************/
   +
   +/* Lost-wakeup test for pthread condition variables.
   + *
   + * Producers and consumers share one bounded buffer, one mutex and ONE
   + * condition variable.  Each side waits in a predicate loop and wakes the
   + * other side after every change.  With a correct condition variable this
   + * cannot deadlock: pthread_cond_broadcast() must unblock every thread that
   + * is blocked on the condition when it is called, and pthread_cond_signal()
   + * at least one of them.
   + *
   + * If a wakeup is lost (for example, when a thread that starts waiting
   + * after the wakeup consumes it in place of a thread that was already
   + * waiting), a consumer can stay blocked with items in the buffer, or a
   + * producer with free space, while the other side also waits.  The main
   + * thread is a watchdog: when no item moves for the stall time, it prints
   + * the state it can see and the program exits with status 1.
   + *
   + * Usage: condsteal [-n items] [-r rounds] [-p producers] [-c consumers]
   + *                  [-b slots] [-t stall_ms] [-T timeout_us] [-s]
   + *
   + *   -T  wait with pthread_cond_timedwait() and this relative timeout, so
   + *       that timeouts race with wakeups.  A timeout only makes the thread
   + *       check its predicate again.
   + *
   + *   -s  wake with pthread_cond_signal() instead of
   + *       pthread_cond_broadcast().  Only valid with one producer and one
   + *       consumer: with more threads of one kind, a signal can legitimately
   + *       wake a thread of the same kind and the program would deadlock
   + *       even with a correct condition variable.
   + */
   +
   
+/****************************************************************************
   + * Included Files
   + 
****************************************************************************/
   +
   +#include <errno.h>
   +#include <pthread.h>
   +#include <stdbool.h>
   +#include <stdio.h>
   +#include <stdlib.h>
   +#include <string.h>
   +#include <time.h>
   +#include <unistd.h>
   +
   
+/****************************************************************************
   + * Pre-processor Definitions
   + 
****************************************************************************/
   +
   +#define CONDSTEAL_MAXTHREADS 8
   +#define CONDSTEAL_MAXSLOTS   64
   +
   
+/****************************************************************************
   + * Private Types
   + 
****************************************************************************/
   +
   +struct condsteal_s
   +{
   +  pthread_mutex_t lock;
   +  pthread_cond_t cond;
   +  unsigned long buffer[CONDSTEAL_MAXSLOTS];
   +  unsigned int slots;      /* Buffer capacity */
   +  unsigned int head;       /* Next slot to read */
   +  unsigned int count;      /* Items in the buffer */
   +  unsigned long nitems;    /* Items to move in this round */
   +  unsigned long produced;  /* Items put so far */
   +  unsigned long consumed;  /* Items taken so far */
   +  unsigned long long sum;  /* Sum of the consumed values */
   +  unsigned int pwaiting;   /* Producers inside pthread_cond_wait() */
   +  unsigned int cwaiting;   /* Consumers inside pthread_cond_wait() */
   +  unsigned long waits;     /* Calls to pthread_cond_wait() */
   +  unsigned long timeouts;  /* Waits that returned ETIMEDOUT */
   +  unsigned long tmo_us;    /* Timed wait in us, or 0: no timeout */
   +  bool use_signal;         /* Wake with signal, not broadcast */
   +};
   +
   
+/****************************************************************************
   + * Private Data
   + 
****************************************************************************/
   +
   +static struct condsteal_s g_cs;
   +
   
+/****************************************************************************
   + * Private Functions
   + 
****************************************************************************/
   +
   +static void condsteal_wake(FAR struct condsteal_s *cs)
   +{
   +  int ret;
   +
   +  if (cs->use_signal)
   +    {
   +      ret = pthread_cond_signal(&cs->cond);
   +    }
   +  else
   +    {
   +      ret = pthread_cond_broadcast(&cs->cond);
   +    }
   +
   +  if (ret != 0)
   +    {
   +      printf("condsteal: wake failed: %d\n", ret);
   +    }
   +}
   +
   +static void condsteal_wait(FAR struct condsteal_s *cs,
   +                           FAR unsigned int *waiting)
   +{
   +  int ret;
   +
   +  (*waiting)++;
   +  cs->waits++;
   +  if (cs->tmo_us > 0)
   +    {
   +      struct timespec ts;
   +
   +      clock_gettime(CLOCK_REALTIME, &ts);
   +      ts.tv_nsec += (cs->tmo_us % 1000000) * 1000;
   +      ts.tv_sec  += cs->tmo_us / 1000000 + ts.tv_nsec / 1000000000;
   +      ts.tv_nsec %= 1000000000;
   +      ret = pthread_cond_timedwait(&cs->cond, &cs->lock, &ts);
   +      if (ret == ETIMEDOUT)
   +        {
   +          cs->timeouts++;
   +          ret = 0;
   +        }
   +    }
   +  else
   +    {
   +      ret = pthread_cond_wait(&cs->cond, &cs->lock);
   +    }
   +
   +  (*waiting)--;
   +
   +  if (ret != 0)
   +    {
   +      printf("condsteal: pthread_cond_wait failed: %d\n", ret);
   +    }
   +}
   +
   +static FAR void *condsteal_producer(FAR void *arg)
   +{
   +  FAR struct condsteal_s *cs = arg;
   +
   +  pthread_mutex_lock(&cs->lock);
   +  while (cs->produced < cs->nitems)
   +    {
   +      while (cs->count == cs->slots && cs->produced < cs->nitems)
   +        {
   +          condsteal_wait(cs, &cs->pwaiting);
   +        }
   +
   +      if (cs->produced < cs->nitems)
   +        {
   +          cs->produced++;
   +          cs->buffer[(cs->head + cs->count) % cs->slots] = cs->produced;
   +          cs->count++;
   +          condsteal_wake(cs);
   +        }
   +    }
   +
   +  pthread_mutex_unlock(&cs->lock);
   +  return NULL;
   +}
   +
   +static FAR void *condsteal_consumer(FAR void *arg)
   +{
   +  FAR struct condsteal_s *cs = arg;
   +
   +  pthread_mutex_lock(&cs->lock);
   +  while (cs->consumed < cs->nitems)
   +    {
   +      while (cs->count == 0 && cs->consumed < cs->nitems)
   +        {
   +          condsteal_wait(cs, &cs->cwaiting);
   +        }
   +
   +      if (cs->consumed < cs->nitems)
   +        {
   +          cs->sum += cs->buffer[cs->head];
   +          cs->head = (cs->head + 1) % cs->slots;
   +          cs->count--;
   +          cs->consumed++;
   +          condsteal_wake(cs);
   +        }
   +    }
   +
   +  /* Let the other consumers see that the work is done */
   +
   +  pthread_cond_broadcast(&cs->cond);
   +  pthread_mutex_unlock(&cs->lock);
   +  return NULL;
   +}
   +
   +static unsigned long condsteal_ms(void)
   +{
   +  struct timespec ts;
   +
   +  clock_gettime(CLOCK_MONOTONIC, &ts);
   +  return (unsigned long)ts.tv_sec * 1000 + ts.tv_nsec / 1000000;
   +}
   +
   +static void condsteal_usage(void)
   +{
   +  printf("Usage: condsteal [-n items] [-r rounds] [-p producers] "
   +         "[-c consumers] [-b slots] [-t stall_ms] [-T timeout_us] "
   +         "[-s]\n");
   +}
   +
   
+/****************************************************************************
   + * Public Functions
   + 
****************************************************************************/
   +
   +int main(int argc, FAR char *argv[])
   +{
   +  FAR struct condsteal_s *cs = &g_cs;
   +  pthread_t threads[2 * CONDSTEAL_MAXTHREADS];
   +  unsigned long nitems = 100000;
   +  unsigned long stall_ms = 3000;
   +  unsigned long timeout_us = 0;
   +  unsigned long long expected;
   +  unsigned long rounds = 1;
   +  unsigned long round;
   +  unsigned int nproducers = 1;
   +  unsigned int nconsumers = 1;
   +  unsigned int slots = 4;
   +  unsigned int nthreads;
   +  unsigned int i;
   +  bool use_signal = false;
   +  int opt;
   +  int ret;
   +
   +  while ((opt = getopt(argc, argv, "n:r:p:c:b:t:T:s")) != -1)
   +    {
   +      switch (opt)
   +        {
   +          case 'n':
   +            nitems = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 'r':
   +            rounds = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 'p':
   +            nproducers = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 'c':
   +            nconsumers = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 'b':
   +            slots = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 't':
   +            stall_ms = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 'T':
   +            timeout_us = strtoul(optarg, NULL, 0);
   +            break;
   +
   +          case 's':
   +            use_signal = true;
   +            break;
   +
   +          default:
   +            condsteal_usage();
   +            return EXIT_FAILURE;
   +        }
   +    }
   +
   +  if (nitems == 0 || rounds == 0 || stall_ms == 0 ||
   +      nproducers == 0 || nproducers > CONDSTEAL_MAXTHREADS ||
   +      nconsumers == 0 || nconsumers > CONDSTEAL_MAXTHREADS ||
   +      slots == 0 || slots > CONDSTEAL_MAXSLOTS ||
   +      (use_signal && (nproducers != 1 || nconsumers != 1)))
   +    {
   +      condsteal_usage();
   +      return EXIT_FAILURE;
   +    }
   +
   +  expected = (unsigned long long)nitems * (nitems + 1) / 2;
   +  printf("condsteal: %lu round(s) of %lu items, %u producer(s), "
   +         "%u consumer(s), %u slots, %s, timeout %lu us\n", rounds, nitems,
   +         nproducers, nconsumers, slots,
   +         use_signal ? "signal" : "broadcast", timeout_us);
   +
   +  for (round = 1; round <= rounds; round++)
   +    {
   +      unsigned long last_moved = 0;
   +      unsigned long last_ms;
   +      bool stalled = false;
   +
   +      memset(cs, 0, sizeof(*cs));
   +      cs->slots = slots;
   +      cs->nitems = nitems;
   +      cs->use_signal = use_signal;
   +      cs->tmo_us = timeout_us;
   +      pthread_mutex_init(&cs->lock, NULL);
   +      pthread_cond_init(&cs->cond, NULL);
   +
   +      nthreads = 0;
   +      for (i = 0; i < nproducers + nconsumers; i++)
   +        {
   +          ret = pthread_create(&threads[nthreads], NULL,
   +                               i < nproducers ? condsteal_producer :
   +                                                condsteal_consumer, cs);
   +          if (ret != 0)
   +            {
   +              printf("condsteal: pthread_create failed: %d\n", ret);
   +              return EXIT_FAILURE;
   +            }
   +
   +          nthreads++;
   +        }
   +
   +      /* Watchdog: poll the progress counters */
   +
   +      last_ms = condsteal_ms();
   +      for (; ; )
   +        {
   +          unsigned long moved;
   +          bool done;
   +
   +          usleep(20000);
   +
   +          pthread_mutex_lock(&cs->lock);
   +          moved = cs->produced + cs->consumed;
   +          done = cs->consumed == cs->nitems;
   +          if (!done && moved == last_moved &&
   +              condsteal_ms() - last_ms >= stall_ms)
   +            {
   +              stalled = true;
   +              printf("condsteal: STALL in round %lu after %lu ms: "
   +                     "produced %lu consumed %lu of %lu, count %u/%u, "
   +                     "waiting producers %u consumers %u, waits %lu\n",
   +                     round, stall_ms, cs->produced, cs->consumed,
   +                     cs->nitems, cs->count, cs->slots, cs->pwaiting,
   +                     cs->cwaiting, cs->waits);
   +              if (cs->cwaiting > 0 && cs->count > 0)
   +                {
   +                  printf("condsteal: lost wakeup: a consumer waits with "
   +                         "%u item(s) in the buffer\n", cs->count);
   +                }
   +
   +              if (cs->pwaiting > 0 && cs->count < cs->slots &&
   +                  cs->produced < cs->nitems)
   +                {
   +                  printf("condsteal: lost wakeup: a producer waits with "
   +                         "%u free slot(s)\n", cs->slots - cs->count);
   +                }
   +            }
   +
   +          pthread_mutex_unlock(&cs->lock);
   +
   +          if (done || stalled)
   +            {
   +              break;
   +            }
   +
   +          if (moved != last_moved)
   +            {
   +              last_moved = moved;
   +              last_ms = condsteal_ms();
   +            }
   +        }
   +
   +      if (stalled)
   +        {
   +          /* The blocked threads end when this task exits */
   +
   +          printf("condsteal: FAILED\n");
   +          return EXIT_FAILURE;
   +        }
   +
   +      for (i = 0; i < nthreads; i++)
   +        {
   +          pthread_join(threads[i], NULL);
   +        }
   +
   +      ret = pthread_cond_destroy(&cs->cond);
   +      if (ret != 0)
   +        {
   +          printf("condsteal: round %lu: pthread_cond_destroy failed: %d\n",
   +                 round, ret);
   +          printf("condsteal: FAILED\n");
   +          return EXIT_FAILURE;
   +        }
   +
   +      pthread_mutex_destroy(&cs->lock);
   +
   +      if (cs->sum != expected)
   +        {
   +          printf("condsteal: round %lu: sum %llu, expected %llu\n",
   +                 round, cs->sum, expected);
   +          printf("condsteal: FAILED\n");
   +          return EXIT_FAILURE;
   +        }
   +
   +      if (rounds > 1 && (round % 10 == 0 || round == rounds))
   +        {
   +          printf("condsteal: round %lu ok (%lu waits, %lu timeouts)\n",
   +                 round, cs->waits, cs->timeouts);
   +        }
   +    }
   +
   +  printf("condsteal: PASSED\n");
   +  return EXIT_SUCCESS;
   +}
   -- 
   2.43.0
   
   ```
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to