Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Gregory P. Smith
On Sun Feb 08 2015 at 12:21:36 AM Victor Stinner wrote: > > Le 8 févr. 2015 05:39, "Gregory P. Smith" a écrit : > > From there, in your signal handler you must try to acquire the newly > exposed keymutex and (...) > > Stop! Acquiring a lock in a signal handler is just a crazy idea. It's very > f

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Victor Stinner
This is exactly how signals are implemented in Python :-) Victor Le lundi 9 février 2015, Robert Collins a écrit : > On 9 February 2015 at 09:11, Maciej Fijalkowski > wrote: > > Hi Francis > > > > Feel free to steal most of vmprof code, it should generally work > > without requiring to patch c

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Robert Collins
On 9 February 2015 at 09:11, Maciej Fijalkowski wrote: > Hi Francis > > Feel free to steal most of vmprof code, it should generally work > without requiring to patch cpython (python 3 patches appreciated :-). > As far as timer goes - it seems not to be going anywhere, I would > rather use a backgr

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Francis Giraldeau
2015-02-08 4:01 GMT-05:00 Maciej Fijalkowski : > I'm working on vmprof (github.com/vmprof/vmprof-python) which works > for both cpython and pypy (pypy has special support, cpython is > patched on-the fly) This looks interesting. I'm working on a profiler that is similar, but not based on timer.

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Maciej Fijalkowski
Hi Francis Feel free to steal most of vmprof code, it should generally work without requiring to patch cpython (python 3 patches appreciated :-). As far as timer goes - it seems not to be going anywhere, I would rather use a background thread or something On Sun, Feb 8, 2015 at 10:03 PM, Francis

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Maciej Fijalkowski
I'm working on vmprof (github.com/vmprof/vmprof-python) which works for both cpython and pypy (pypy has special support, cpython is patched on-the fly) On Sun, Feb 8, 2015 at 6:39 AM, Gregory P. Smith wrote: > To get at the Python thread state from a signal handler (using 2.7 as a > reference her

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Victor Stinner
Le 8 févr. 2015 05:39, "Gregory P. Smith" a écrit : > From there, in your signal handler you must try to acquire the newly exposed keymutex and (...) Stop! Acquiring a lock in a signal handler is just a crazy idea. It's very far from an async signal-safe function. > So until those are fixed (hoo

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-08 Thread Victor Stinner
Le 7 févr. 2015 22:34, "Greg Ewing" a écrit : with --shared) > You might be able to use Py_AddPendingCall to schedule > what you want done outside the context of the signal > handler. I don't how it could work. You have to increment the reference counting, but also maybe increment references to o

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-07 Thread Gregory P. Smith
To get at the Python thread state from a signal handler (using 2.7 as a reference here; but i don't believe 3.4 has changed this part much) you need to modify the interpreter to expose pystate.c's "autoTLSkey" and thread.c's "struct key" as well as "keyhead" and "keymutex". >From there, in your si

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-07 Thread Greg Ewing
Maciej Fijalkowski wrote: However, you can't access thread locals from signal handlers (since in some cases it mallocs, thread locals are built lazily if you're inside the .so, e.g. if python is built with --shared) You might be able to use Py_AddPendingCall to schedule what you want done outsi

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-07 Thread Xavier de Gaye
On 02/06/2015 11:48 PM, Francis Giraldeau wrote: > 2015-02-06 6:04 GMT-05:00 Armin Rigo: > > Hi, > > On 6 February 2015 at 08:24, Maciej Fijalkowski mailto:fij...@gmail.com>> wrote: > > I don't think it's safe to assume f_code is properly filled by the > > time you might read it, d

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-07 Thread Maciej Fijalkowski
On Sat, Feb 7, 2015 at 12:48 AM, Francis Giraldeau wrote: > 2015-02-06 6:04 GMT-05:00 Armin Rigo : > >> Hi, >> >> On 6 February 2015 at 08:24, Maciej Fijalkowski wrote: >> > I don't think it's safe to assume f_code is properly filled by the >> > time you might read it, depending a bit where you f

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-06 Thread Francis Giraldeau
2015-02-06 6:04 GMT-05:00 Armin Rigo : > Hi, > > On 6 February 2015 at 08:24, Maciej Fijalkowski wrote: > > I don't think it's safe to assume f_code is properly filled by the > > time you might read it, depending a bit where you find the frame > > object. Are you sure it's not full of garbage?

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-06 Thread Armin Rigo
Hi, On 6 February 2015 at 08:24, Maciej Fijalkowski wrote: > I don't think it's safe to assume f_code is properly filled by the > time you might read it, depending a bit where you find the frame > object. Are you sure it's not full of garbage? Yes, before discussing how to do the utf8 decoding,

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-06 Thread M.-A. Lemburg
On 06.02.2015 00:27, Francis Giraldeau wrote: > I need to access frame members from within a signal handler for tracing > purpose. My first attempt to access co_filename was like this (omitting > error checking): > > PyFrameObject *frame = PyEval_GetFrame(); > PyObject *ob = PyUnicode_AsUTF8Strin

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-06 Thread Victor Stinner
Hi, 2015-02-06 0:27 GMT+01:00 Francis Giraldeau : > I need to access frame members from within a signal handler for tracing > purpose. IMO you have a big technical or design issue here. Accessing Python internals in a signal handler is not reliable. A signal can occur anytime, between two instruc

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-05 Thread Maciej Fijalkowski
Hi Francis I don't think it's safe to assume f_code is properly filled by the time you might read it, depending a bit where you find the frame object. Are you sure it's not full of garbage? Besides, are you writing a profiler, or what exactly are you doing? On Fri, Feb 6, 2015 at 1:27 AM, Franci

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-05 Thread Gregory P. Smith
On Thu Feb 05 2015 at 4:36:30 PM Francis Giraldeau < francis.girald...@gmail.com> wrote: > I need to access frame members from within a signal handler for tracing > purpose. My first attempt to access co_filename was like this (omitting > error checking): > > PyFrameObject *frame = PyEval_GetFram

Re: [Python-Dev] Encoding of PyFrameObject members

2015-02-05 Thread Chris Angelico
On Fri, Feb 6, 2015 at 10:27 AM, Francis Giraldeau wrote: > Instead, I access members directly: > char *str = PyUnicode_DATA(frame->f_code->co_filename); > size_t len = PyUnicode_GET_DATA_SIZE(frame->f_code->co_filename); > > Is it safe to assume that unicode objects co_filename and co_name are al

[Python-Dev] Encoding of PyFrameObject members

2015-02-05 Thread Francis Giraldeau
I need to access frame members from within a signal handler for tracing purpose. My first attempt to access co_filename was like this (omitting error checking): PyFrameObject *frame = PyEval_GetFrame(); PyObject *ob = PyUnicode_AsUTF8String(frame->f_code->co_filename) char *str = PyBytes_AsString