On Sat, Mar 02, 2019 at 01:56:56PM +0000, Visa Hankala wrote: > On Sat, Mar 02, 2019 at 10:29:07AM +0100, Sebastien Marie wrote: > > I am experiencing dead-lock with the new futex based rwlock > > implementation commited few days ago. > > The problem happens if a read-write lock has been read-locked and > there are multiple writers waiting. The unlock routine will clear the > waiter flag and wake up a single waiter. If no new waiters emerge > before the next unlock, no wakeup will be issued. > > A simple fix is to wake all waiters, so that no wakeup is lost. > That may cause a thundering herd effect with writers though. The > implementation could encode the number of waiters in `rwlock->value', > enabling more discreet wakeups. However, that needs more testing. > > Does the following patch help?
yes, in "normal" conditions it is fine. so ok semarie@ > Index: rthread_rwlock.c > =================================================================== > RCS file: src/lib/librthread/rthread_rwlock.c,v > retrieving revision 1.12 > diff -u -p -r1.12 rthread_rwlock.c > --- rthread_rwlock.c 13 Feb 2019 13:22:14 -0000 1.12 > +++ rthread_rwlock.c 2 Mar 2019 13:35:31 -0000 > @@ -273,7 +273,7 @@ pthread_rwlock_unlock(pthread_rwlock_t * > } while (atomic_cas_uint(&rwlock->value, val, new) != val); > > if (new == UNLOCKED && (val & WAITING)) > - _wake(&rwlock->value, COUNT(val)); > + _wake(&rwlock->value, INT_MAX); > > return (0); > } For completeness, I would add that I am still seeing few dead-lock (I needed to augment the number of tries and ran under particular env) but they seems directly related to _rthread_debug() (called when running with RTHREAD_DEBUG variable set) and printf(), and not with pthread_rwlock. I prefer to mention it, but I am a bit unsure of what to do with it as _rthread_debug() is a bit uncommon in use. but maybe it deserves attention. Here a dead lock with 3 threads involved (gdb extract): - thread 1 (main): in pthread_join() > _libc__rthread_debug() - thread 2 : in pthread_rwlock_unlock() > _libc__rthread_debug() - thread 3 : in printf() (gdb) thread apply all bt Thread 3 (thread 363551): #0 sched_yield () at -:3 #1 0x00000a8c0609d9c5 in _libc__spinlock (lock=Variable "lock" is not available.) at /usr/src/lib/libc/thread/rthread.c:50 #2 0x00000a8c0607036a in _thread_flockfile (fp=Variable "fp" is not available.) at /usr/src/lib/libc/thread/rthread_file.c:211 #3 0x00000a8c060d35c4 in _libc_vfprintf (fp=0xa8c060e7c88, fmt0=0xa8993d19776 "thread[%d]: releas lock\n", ap=0xa8bbea7d240) at /usr/src/lib/libc/stdio/vfprintf.c:262 #4 0x00000a8c060a2a6f in _libc_printf (fmt=Variable "fmt" is not available.) at /usr/src/lib/libc/stdio/printf.c:44 #5 0x00000a8993d1a577 in panic () at test.c:52 #6 0x00000a8993d1a61c in c (_arg=0x0) at test.c:66 #7 0x00000a8c3c2796f1 in _rthread_start (v=Variable "v" is not available.) at /data/openbsd/src/lib/librthread/rthread.c:96 #8 0x00000a8c060cb8fb in __tfork_thread () at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:75 #9 0x0000000000000000 in ?? () Thread 2 (thread 281655): #0 sched_yield () at -:3 #1 0x00000a8c0609d9c5 in _libc__spinlock (lock=Variable "lock" is not available.) at /usr/src/lib/libc/thread/rthread.c:50 #2 0x00000a8c060705c6 in _thread_funlockfile (fp=0xa8ba97b8298) at /usr/src/lib/libc/thread/rthread_file.c:275 #3 0x00000a8c0609e1bc in _libc_fflush (fp=0xa8ba97b8298) at /usr/src/lib/libc/stdio/fflush.c:52 #4 0x00000a8c0606c89a in _libc_vdprintf (fd=Variable "fd" is not available.) at /usr/src/lib/libc/stdio/vdprintf.c:72 #5 0x00000a8c060a7f63 in _libc__rthread_debug (level=Variable "level" is not available.) at /usr/src/lib/libc/thread/rthread_debug.c:23 #6 0x00000a8c3c27ac90 in pthread_rwlock_unlock (rwlockp=Variable "rwlockp" is not available.) at /data/openbsd/src/lib/librthread/rthread_rwlock.c:264 #7 0x00000a8993d1a586 in panic () at test.c:53 #8 0x00000a8993d1a61c in c (_arg=0x0) at test.c:66 #9 0x00000a8c3c2796f1 in _rthread_start (v=Variable "v" is not available.) at /data/openbsd/src/lib/librthread/rthread.c:96 #10 0x00000a8c060cb8fb in __tfork_thread () at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:75 #11 0x0000000000000000 in ?? () Thread 1 (thread 469200): #0 sched_yield () at -:3 #1 0x00000a8c0609d9c5 in _libc__spinlock (lock=Variable "lock" is not available.) at /usr/src/lib/libc/thread/rthread.c:50 #2 0x00000a8c060702be in _thread_flockfile (fp=0x7f7ffffcfaa8) at /usr/src/lib/libc/thread/rthread_file.c:180 #3 0x00000a8c0609e11a in _libc_fflush (fp=0x7f7ffffcfaa8) at /usr/src/lib/libc/stdio/fflush.c:46 #4 0x00000a8c0606c89a in _libc_vdprintf (fd=Variable "fd" is not available.) at /usr/src/lib/libc/stdio/vdprintf.c:72 #5 0x00000a8c060a7f63 in _libc__rthread_debug (level=Variable "level" is not available.) at /usr/src/lib/libc/thread/rthread_debug.c:23 #6 0x00000a8c060410c5 in _rthread_mutex_timedlock (mutexp=Variable "mutexp" is not available.) at /usr/src/lib/libc/thread/rthread_mutex.c:163 #7 0x00000a8c060bc482 in malloc (size=56) at /usr/src/lib/libc/stdlib/malloc.c:1253 #8 0x00000a8c060703e4 in _thread_flockfile (fp=0x7f7ffffd02b8) at /usr/src/lib/libc/thread/rthread_file.c:156 #9 0x00000a8c0609e11a in _libc_fflush (fp=0x7f7ffffd02b8) at /usr/src/lib/libc/stdio/fflush.c:46 #10 0x00000a8c0606c89a in _libc_vdprintf (fd=Variable "fd" is not available.) at /usr/src/lib/libc/stdio/vdprintf.c:72 #11 0x00000a8c060a7f63 in _libc__rthread_debug (level=Variable "level" is not available.) at /usr/src/lib/libc/thread/rthread_debug.c:23 #12 0x00000a8c3c2792b6 in _rthread_reaper () at /data/openbsd/src/lib/librthread/rthread.c:260 #13 0x00000a8c3c279229 in pthread_join (thread=Variable "thread" is not available.) at /data/openbsd/src/lib/librthread/rthread.c:319 #14 0x00000a8993d1a705 in main (argc=1, argv=0x7f7ffffd0ab8) at test.c:86 3 in - Thanks. -- Sebastien Marie
