[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Thouis (Ray) Jones added the comment: Further testing indicates the problem is in the filesystem itself (either the server or client, but not in python). Serializing the loops calling readdir / readdir_r fixes the problem on my system, but using either function in a large number of parallel t

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Changes by Ross Lagerwall : -- nosy: +rosslagerwall ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Charles-François Natali added the comment: And here's a post by Ulrich Drepper: http://udrepper.livejournal.com/18555.html """ readdir_r is only needed if multiple threads are using the same directory stream. I have yet to see a program where this really is the case. In this toy example the s

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Thouis (Ray) Jones added the comment: > > It's also possible that readdir() is not reentrant with lstat() > This doesn't make much sense to me. Me either. I think what I was actually seeing was multiple calls to readdir() still occurring even after placing a mutex on os.listdir due to my wrap

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Thouis (Ray) Jones added the comment: Reading through many pages discussing readdir vs. readdir_r (many on security mailing lists, a large number referring to the page linked in the patch), I get the impression that most implementations are thread-safe as long as separate threads do not call

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Charles-François Natali added the comment: > Is there any reason to believe that the problem is confined to OS X? It's a bit of a grey area. Here's what POSIX says: http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir.html """ The pointer returned by readdir() points to data which

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Changes by Thouis (Ray) Jones : Added file: http://bugs.python.org/file23834/py272_readdir_r.v2.patch ___ Python tracker ___ ___ Python-bugs-l

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Thouis (Ray) Jones added the comment: I should add the caveat that I am not completely confident that I have stress-tested the patch enough to be sure that it actually addresses the problem. It is still possible that this is an error in OSX or the remote fileserver in which a large amount of

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Ned Deily added the comment: Is there any reason to believe that the problem is confined to OS X? -- nosy: +ned.deily, neologix ___ Python tracker ___ __

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Amaury Forgeot d'Arc added the comment: The link mentioned in the patch is really interesting: http://womble.decadent.org.uk/readdir_r-advisory.html -- nosy: +amaury.forgeotdarc ___ Python tracker

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

Thouis (Ray) Jones added the comment: Here is the script I use to detect the failure. % python filefinder.py /PATH/TO/LARGE/DIRECTORY/TREE (note that I was working over samba with an 8ish-level deep directory with around 25 files). Compare its final output in the FOUND column with % find

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8

New submission from Thouis (Ray) Jones : On my system (OSX 10.6.8) using the python.org 32/64-bit build of 2.7.2, I see incorrect results from os.listdir() in a threaded program. The error is that the result of os.listdir() is missing a few files from its list. First, my use case. I work wit