[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-02 Thread Thouis (Ray) Jones
Thouis (Ray) Jones tho...@gmail.com added the comment: Further testing indicates the problem is in the filesystem itself (either the server or client, but not in python). Serializing the loops calling readdir / readdir_r fixes the problem on my system, but using either function in a large

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
New submission from Thouis (Ray) Jones tho...@gmail.com: On my system (OSX 10.6.8) using the python.org 32/64-bit build of 2.7.2, I see incorrect results from os.listdir() in a threaded program. The error is that the result of os.listdir() is missing a few files from its list. First, my use

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
Thouis (Ray) Jones tho...@gmail.com added the comment: Here is the script I use to detect the failure. % python filefinder.py /PATH/TO/LARGE/DIRECTORY/TREE (note that I was working over samba with an 8ish-level deep directory with around 25 files). Compare its final output in the FOUND

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
Thouis (Ray) Jones tho...@gmail.com added the comment: I should add the caveat that I am not completely confident that I have stress-tested the patch enough to be sure that it actually addresses the problem. It is still possible that this is an error in OSX or the remote fileserver in which

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
Changes by Thouis (Ray) Jones tho...@gmail.com: Added file: http://bugs.python.org/file23834/py272_readdir_r.v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13517

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
Thouis (Ray) Jones tho...@gmail.com added the comment: Reading through many pages discussing readdir vs. readdir_r (many on security mailing lists, a large number referring to the page linked in the patch), I get the impression that most implementations are thread-safe as long as separate

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

2011-12-01 Thread Thouis (Ray) Jones
Thouis (Ray) Jones tho...@gmail.com added the comment: It's also possible that readdir() is not reentrant with lstat() This doesn't make much sense to me. Me either. I think what I was actually seeing was multiple calls to readdir() still occurring even after placing a mutex on os.listdir

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-05 Thread Thouis (Ray) Jones
Changes by Thouis (Ray) Jones tho...@gmail.com: -- nosy: +thouis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list