Re: GIL state during import
On Wed, Feb 17, 2010 at 11:59 AM, Dave Angel wrote: > Terry Reedy wrote: > > On 2/16/2010 >> 4:37 PM, Doron Tal wrote: >> >>> Is the GIL released during import statement execution when accessing the >>> file? >>> If not, is it a bug? >>> If it is not a bug, or it is here to stay for any other reason, I think >>> it should be mentioned in the documentation. >>> >> >> The CPython GIL is a CPython implementation artifact and should not be >> mentioned in the language docs (and is not, as far as I know). Were you >> thinking of something else? And why this specific feature of its behavior, >> whatever it is. >> >> tjr >> >> Threading vs. imports is mentioned at least once in the Python docs. See > http://docs.python.org/library/threading.html, section 17.2.9 (at least > in version 2.6.4) > > """While the import machinery is thread safe, there are two key > restrictions on threaded imports due to inherent limitations in the way that > thread safety is provided: > > * Firstly, other than in the main module, an import should not have > the side effect of spawning a new thread and then waiting for that > thread in any way. Failing to abide by this restriction can lead > to a deadlock if the spawned thread directly or indirectly > attempts to import a module. > * Secondly, all import attempts must be completed before the > interpreter starts shutting itself down. This can be most easily > achieved by only performing imports from non-daemon threads > created through the threading module. Daemon threads and threads > created directly with the thread module will require some other > form of synchronization to ensure they do not attempt imports > after system shutdown has commenced. Failure to abide by this > restriction will lead to intermittent exceptions and crashes > during interpreter shutdown (as the late imports attempt to access > machinery which is no longer in a valid state). > > """ > > So it may or may not use the GIL, but there are thread restrictions during > an import. The rule I try to follow is not to do anything non-trivial > during top-level code of a module, except inside the > if __name__ == "__main__": > > portion. If we're inside that portion, we're not a module, we're a script. > > Even better, import all your modules before starting any new threads. > > DaveA > > > > -- > http://mail.python.org/mailman/listinfo/python-list > No argue here. The specific problem, as I wrote in reply to Terry, is with the execfile statement. It might be a part of some online plugin machinery. In such case the nature of the cause does not allow to execute it upfront. I think that the problem can be circumvented by first reading the file followed by compile and eval, as being done in Python3 (no execfile there). --doron -- http://mail.python.org/mailman/listinfo/python-list
Re: GIL state during import
On Wed, Feb 17, 2010 at 1:01 AM, Terry Reedy wrote: > On 2/16/2010 4:37 PM, Doron Tal wrote: > >> Is the GIL released during import statement execution when accessing the >> file? >> If not, is it a bug? >> If it is not a bug, or it is here to stay for any other reason, I think >> it should be mentioned in the documentation. >> > > The CPython GIL is a CPython implementation artifact and should not be > mentioned in the language docs (and is not, as far as I know). Were you > thinking of something else? And why this specific feature of its behavior, > whatever it is. > > tjr > > > > -- > http://mail.python.org/mailman/listinfo/python-list > I'm sorry, I should have explained my case better. Quoting the documentation ( http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock ): "The lock is also released and reacquired around potentially blocking I/O operations like reading or writing a file, so that other threads can run while the thread that requests the I/O is waiting for the I/O operation to complete." Since on some cases the IO may be _very_ slow, any IO command which does not release the GIL can potentially stall the process. There are two problems (or more...), the first, it may stall the process for unacceptable duration. The second problem is that it is not an easy task to track down the root cause of this stall. In the specific case I'm dealing with, the problematic command is execfile (at the time of my initial writing I did not know that). --doron -- http://mail.python.org/mailman/listinfo/python-list
GIL state during import
Is the GIL released during import statement execution when accessing the file? If not, is it a bug? If it is not a bug, or it is here to stay for any other reason, I think it should be mentioned in the documentation. --doron -- http://mail.python.org/mailman/listinfo/python-list
Re: Most "active" coroutine library project?
On Sun, Aug 23, 2009 at 6:02 PM, Phillip B Oldham wrote: > I've been taking a look at the multitude of coroutine libraries > available for Python, but from the looks of the projects they all seem > to be rather "quiet". I'd like to pick one up to use on a current > project but can't deduce which is the most popular/has the largest > community. > > Libraries I looked at include: cogen, weightless, eventlet and > circuits (which isn't exactly coroutine-based but it's event-driven > model was intriguing). > > Firstly, are there any others I've missed? And what would the > consensus be on the which has the most active community behind it? > -- > http://mail.python.org/mailman/listinfo/python-list > If you seek also event-driven libraries: Twisted - http://twistedmatrix.com/trac/ --doron -- http://mail.python.org/mailman/listinfo/python-list
inspect.stack() performance
I use inspect.stack() to extract some info, such as file name, function name
and line number, for the sake of logging (home brew logger).
I tested to see how much time it takes to execute the command:
Python 2.4.3 (#1, Jan 21 2009, 01:11:33)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> t = timeit.Timer('inspect.stack()', 'import inspect')
>>> t.repeat(3,1000)
[0.28500604629516602, 0.28315305709838867, 0.29169297218322754]
>>>
About 280uSec per call.
If I log 1000 times per seconds than I get to spend 28% of the process time
on inspect.stack()
Is there a better/quicker way of retrieving the same info?
Thanks,
--doron
--
http://mail.python.org/mailman/listinfo/python-list
Re: Gracefully exiting CLI application
On Tue, Jul 28, 2009 at 12:52 AM, Jan Kaliszewski wrote: > As I wrote, you must use signals. Though sometimes it's a good idea > to combine these two techniques (i.e. signal handlers call sys.exit(), > then sys.exitfunc/or function registered with atexit does the actual > cleaning actions). Another way of combining signals and atexit functions is to write a signal handlers which update some QUIT_APP flag, which the application polls on occasionally. This way you'll avoid deadlocks which may happen if you have a multi-threaded application, and your main thread had just acquired a lock. --doron -- http://mail.python.org/mailman/listinfo/python-list
Re: file transfer in python
On Fri, Jun 26, 2009 at 2:38 PM, jayesh bhardwaj wrote: > i am trying to find something useful in python to transfer html files > from one terminal to other. Can this be done with some module or shall > i start coding my own module using low level socket interface. If u > know about some books on this topic or any online help then plz help. > -- > http://mail.python.org/mailman/listinfo/python-list > You can try the xmlrpclib: http://docs.python.org/library/xmlrpclib.html#binary-objects --doron -- http://mail.python.org/mailman/listinfo/python-list
Re: execfile (exec?) create non consistent locals() state
On Tue, Apr 21, 2009 at 9:13 PM, Chris Rebert wrote:
> On Tue, Apr 21, 2009 at 9:08 AM, Doron Tal
> wrote:
> > Hi,
> >
> > Recently I tried to execute a python file using execfile (exec performed
> > just the same for that reason).
> > I encountered the behavior below:
> >
> > """
> > $ cat execme.py
> > a = 2
> > $ python
> > Python 2.4.3 (#1, May 24 2008, 13:57:05)
> > [GCC 4.1.2 20070626 (Red Hat 4.1.2-14)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> def execfile_func():
> > ... execfile('execme.py')
> > ... print 'locals() = %s' % str(locals())
> > ... print a
> > ...
> >>>> execfile_func()
> > locals() = {'a': 2}
> > Traceback (most recent call last):
> > File "", line 1, in ?
> > File "", line 4, in execfile_func
> > NameError: global name 'a' is not defined
> >>>>
> > """
> >
> > After execfile, the a variable can be found in locals(), however any
> direct
> > reference (e.g., print a) fails.
> > Is it expected?
>
> Yes. See http://docs.python.org/library/functions.html#locals (emphasis
> mine):
>
> locals()
>[...]
>Warning: The contents of this dictionary should not be modified;
> ***changes may not affect the values of local variables used by the
> interpreter***.
>[...]
>
> Cheers,
> Chris
> --
> I have a blog:
> http://blog.rebertia.com
(Chris - sorry for the multiple replies, I forgot to reply all)
I actually did not attempt to modify this dictionary (the one which locals()
returns) explicitly.
The simplest assignment command, such as 'a = 2' modifies this dictionary
implicitly, and it's working perfectly well.
I expected exec to work the same, but apparently I was wrong. Is there is a
way to exec a file "more" correctly? thus avoid the need to resort to
awkward solutions such as using the locals() dictionary?
--
http://mail.python.org/mailman/listinfo/python-list
execfile (exec?) create non consistent locals() state
Hi,
Recently I tried to execute a python file using execfile (exec performed
just the same for that reason).
I encountered the behavior below:
"""
$ cat execme.py
a = 2
$ python
Python 2.4.3 (#1, May 24 2008, 13:57:05)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-14)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def execfile_func():
... execfile('execme.py')
... print 'locals() = %s' % str(locals())
... print a
...
>>> execfile_func()
locals() = {'a': 2}
Traceback (most recent call last):
File "", line 1, in ?
File "", line 4, in execfile_func
NameError: global name 'a' is not defined
>>>
"""
After execfile, the a variable can be found in locals(), however any direct
reference (e.g., print a) fails.
Is it expected?
Thanks,
Doron
--
http://mail.python.org/mailman/listinfo/python-list
Swig for Python 2.6
Hi I'm trying to wrap a c library for use with Python 2.6. I'm using swig 1.3.36, and I get the following error: linux-python/log_wrap.c: In function '_PySwigObject_type': linux-python/log_wrap.c:1680: warning: missing initializer linux-python/log_wrap.c:1680: warning: (near initialization for 'tmp.tp_version_tag') linux-python/log_wrap.c: In function '_PySwigPacked_type': linux-python/log_wrap.c:1843: warning: missing initializer linux-python/log_wrap.c:1843: warning: (near initialization for 'tmp.tp_version_tag') linux-python/log_wrap.c: In function 'swig_varlink_type': linux-python/log_wrap.c:3334: warning: missing initializer linux-python/log_wrap.c:3334: warning: (near initialization for 'tmp.tp_version_tag') What swig version should I use? If version 1.3.36 is good enough, then what can be my mistake? Thanks, doron -- http://mail.python.org/mailman/listinfo/python-list
