[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Committed in r84489 (3.x), r84490 (3.1) and r84491 (2.7). Thank you!

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed
versions:  -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Another "+1" on the patch if it is needed.  Posix module consistently releases 
the GIL while making system calls and this is a good idea independent of this 
particular issue.

Looking at the patch made me wonder whether existing

Py_BEGIN_ALLOW_THREADS
ep = readdir(dirp);
Py_END_ALLOW_THREADS

is safe.  Apparently it is safe as was explained in

http://mail.python.org/pipermail/python-dev/2006-April/063808.html

because "The pointer returned by readdir() points to data which may be 
overwritten by another call to readdir() on the same directory stream. This 
data is not overwritten by another call to readdir() on a different directory 
stream."

It may be appropriate to add a comment near the readdir() explaining why it is 
safe in this context.  Clearly I am not the only one who found this non-obvious.

--
nosy: +Alexander.Belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

Does this patch still need review? Both Martin and Antoine already commented 
that the patch is ok, so it'd be great if someone could actually apply it...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Martin v . Löwis]

Martin v. Löwis  added the comment:

I think the bug here is really in the application, which makes a system call to 
fuse and the fuse callback in the same process. This isn't supported, and IMO 
doesn't need to be. Python makes no promises that it will accept callbacks 
while a system call is in progress.

That said, working around this problem in this specific case with the proposed 
patch is fine with me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

The patch works fine for me too. Also, I did not discover any other such 
problems for other syscalls (but I did not systematically try all os.* 
functions).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Marcin Bachry]

Marcin Bachry  added the comment:

> Still, I'd be extremly grateful if someone could tell me the trick how 
to create a backtrace in such a deadlock situation

Sorry, I should have mentioned that. In order to get backtrace you let the 
process freeze, attach gdb to it (it will freeze too), then go to 
/sys/fs/fuse/connections/ and find your fuse connection (it's most likely the 
one with non-zero value in "waiting" file). Then do "echo 1 > abort" and go 
back to gdb.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Well, now that I know what to look for, tracking down more of these 
> problems should be significantly faster and easier. Are you generally 
> going to accept similar patches for other unprotected syscalls?

Until now the rule of thumb was to consider only time-consuming syscalls
(as I said, the primary goal is optimizing multi-threaded I/O). But I
guess we can accept such patches if they have no downsides. Martin, what
do you think?

> Still, I'd be extremly grateful if someone could tell me the trick how 
> to create a backtrace in such a deadlock situation... Or am I the only 
> one for which a simple "gdb -a" does not work?

Perhaps you can try to crash the process (using `kill -ABRT`) and then
load the core dump using gdb.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

On 01/20/2010 07:19 AM, Antoine Pitrou wrote:
> Ah, thanks for the explanation. Yes indeed the patch looks ok for the
> job. You should just be aware that similar problems may appear with
> other system calls. I don't think we have ever considered that common C
> calls such as opendir() could call back into Python code through
> libraries such as FUSE.

Well, now that I know what to look for, tracking down more of these 
problems should be significantly faster and easier. Are you generally 
going to accept similar patches for other unprotected syscalls?

Still, I'd be extremly grateful if someone could tell me the trick how 
to create a backtrace in such a deadlock situation... Or am I the only 
one for which a simple "gdb -a" does not work?

Best,

-Nikolaus

--
title: os.listdir hangs since opendir() and closedir() do not release GIL -> 
os.listdir hangs since opendir() and closedir() do not release GIL

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

I have used both of them in the past, but in the end I wrote my own bindings 
(currently only available as part of the http://code.google.com/p/s3ql source 
code, but I intend to factor it out at some point), 
since neither fuse.py (http://code.google.com/p/fusepy) nor python-fuse 
(fuse.sf.net) support the FUSE low level API.

That said, I am reasonably sure that in both of them are affected by this bug 
too. However, it may not show up in practive very often, because the high level 
FUSE API by default forks into a background process before mounting the file 
system.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ah, thanks for the explanation. Yes indeed the patch looks ok for the
job. You should just be aware that similar problems may appear with
other system calls. I don't think we have ever considered that common C
calls such as opendir() could call back into Python code through
libraries such as FUSE.

By the way, it seems there are Python bindings for FUSE, in two
different flavours. You might want to look into them, and perhaps to
check whether they also experience this issue.

Le mercredi 20 janvier 2010 à 12:12 +, Nikolaus Rath a écrit :
> Nikolaus Rath  added the comment:
> 
> In this simple example, FUSE does not fork and does not start any threads.
> 
> Note that PyGILState_Ensure() cannot do anything here. What happens is this:
> 
>  - call_hello.py calls FUSE in a new thread, releasing the GIL.
>  - FUSE mounts the file system and waits for requests
>  - Meanwhile, in the main thread, call_hello.py calls opendir(), but does not 
> release the GIL
>  - FUSE receives the opendir() system call and calls the appropriate callback 
> function
>  - If the callback function is implemented in C, everything works fine.
>  - If the callback function is implemented in Python, the FUSE thread tries 
> to acquire the GIL to call the (Python) opendir() handler. But it cannot do 
> so, because the GIL is still held by the main thread (which is waiting for 
> the opendir syscall to return) => deadlock.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

In this simple example, FUSE does not fork and does not start any threads.

Note that PyGILState_Ensure() cannot do anything here. What happens is this:

 - call_hello.py calls FUSE in a new thread, releasing the GIL.
 - FUSE mounts the file system and waits for requests
 - Meanwhile, in the main thread, call_hello.py calls opendir(), but does not 
release the GIL
 - FUSE receives the opendir() system call and calls the appropriate callback 
function
 - If the callback function is implemented in C, everything works fine.
 - If the callback function is implemented in Python, the FUSE thread tries to 
acquire the GIL to call the (Python) opendir() handler. But it cannot do so, 
because the GIL is still held by the main thread (which is waiting for the 
opendir syscall to return) => deadlock.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Releasing the GIL when calling C library functions (e.g. opendir()) is not a 
requirement, it's just an optimization for slightly better multi-threading. 
Also, as shown in the backtrace, PyGILState_Ensure() is called which should 
only try to acquire the GIL if it isn't already held by the current thread; 
that is, it is designed to avoid the deadlock you are witnessing.

So, I would suggest the problem lies somewhere else. Does FUSE implicitly do a 
fork() of the current process? It's not obvious how the example shown here 
works.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Brian Curtin]

Changes by Brian Curtin :


--
keywords: +needs review
nosy: +brian.curtin
priority:  -> normal
stage:  -> patch review
versions: +Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Changes by Nikolaus Rath :


--
title: ctypes freezes/deadlocks process -> os.listdir hangs since opendir() and 
closedir() do not release GIL

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com