Re: pthread process-private futexes [Re: CVS: cvs.openbsd.org: src]

2019-11-01 Thread Martin Pieuchot
On 24/10/19(Thu) 13:30, Stuart Henderson wrote:
> On 2019/10/21 04:06, Martin Pieuchot wrote:
> > CVSROOT:/cvs
> > Module name:src
> > Changes by: m...@cvs.openbsd.org2019/10/21 04:06:31
> > 
> > Modified files:
> > lib/librthread : synch.h 
> > 
> > Log message:
> > Use process-private futexes to avoid the uvm_map lookup overhead.
> > 
> > While here kill unused _wait() function.
> > 
> > ok visa@
> > 
> 
> I'm going to back this one out for now - it causes hangs with Python
> (seems to be 100% reproducible). Easy/fast test case:
> 
> cd /usr/ports/graphics/py-Pillow
> make
> 
> With this commit, it hangs after
> 
> writing manifest file 'src/Pillow.egg-info/SOURCES.txt'

ltrace shows that sempahore are used.

gdb says that sem_init() is always called with `pshared' set to 0.

However python forks then sleeps in futex(2), via sem_wait(3), when
private futexes are used. 

When non-private futexes are used it it seems that threads from a
process wake up the sleeping thread from a parent process:

 84294/104794  python2.7 CALL  futex(0x1a49bbca3008,0x1,0,0,0)
 84677/611464  python2.7 CALL  futex(0x1a49bbca3008,0x2,1,0,0)

To me it seems that shared semaphore, that are advertised in librthread as
non-supported, are in fact used.

Python has a second lock implementation using mutex and condvar, I'd be
interested to know if that works.



Re: pthread process-private futexes [Re: CVS: cvs.openbsd.org: src]

2019-11-05 Thread Martin Pieuchot
On 24/10/19(Thu) 13:30, Stuart Henderson wrote:
> On 2019/10/21 04:06, Martin Pieuchot wrote:
> > CVSROOT:/cvs
> > Module name:src
> > Changes by: m...@cvs.openbsd.org2019/10/21 04:06:31
> > 
> > Modified files:
> > lib/librthread : synch.h 
> > 
> > Log message:
> > Use process-private futexes to avoid the uvm_map lookup overhead.
> > 
> > While here kill unused _wait() function.
> > 
> > ok visa@
> > 
> 
> I'm going to back this one out for now - it causes hangs with Python
> (seems to be 100% reproducible). Easy/fast test case:
> 
> cd /usr/ports/graphics/py-Pillow
> make
> 
> With this commit, it hangs after
> 
> writing manifest file 'src/Pillow.egg-info/SOURCES.txt'

robert@ figured out that Python3 doesn't have that problem.  If somebody
could figure out which upstream change fixed the issue that would be great.