Re: PEP idea: Instrumented Python
On Oct 12, 2007, at 12:10 PM, Jean-Paul Calderone wrote: > On Fri, 12 Oct 2007 11:33:11 -0500, Erik Jones <[EMAIL PROTECTED]> > wrote: >> [snip] >> >> This got me thinking about building a module that could be included >> by projects that creates a socket and responds to messages on that >> socket in a separate thread from the main app so that you can connect >> to the app and explore its current state. Take a look at the >> evalexception module in Paste to see what he does. >> > > Or manhole in Twisted. Thanks for the tip! Erik Jones Software Developer | Emma® [EMAIL PROTECTED] 800.595.4401 or 615.292.5888 615.292.0777 (fax) Emma helps organizations everywhere communicate & market in style. Visit us online at http://www.myemma.com -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP idea: Instrumented Python
On Fri, 12 Oct 2007 11:33:11 -0500, Erik Jones <[EMAIL PROTECTED]> wrote: > [snip] > >This got me thinking about building a module that could be included >by projects that creates a socket and responds to messages on that >socket in a separate thread from the main app so that you can connect >to the app and explore its current state. Take a look at the >evalexception module in Paste to see what he does. > Or manhole in Twisted. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP idea: Instrumented Python
On Oct 12, 2007, at 10:09 AM, Christopher Nelson wrote: > I was looking at adding dtrace-like dynamic tracing to Python. > Note that this isn't dtrace itself. The basic rationale: > > 1. A lot of enterprise-level software is written in Python. It is > difficult to impossible to reproduce the customer environment in a > test lab. Sometimes applications hang mysteriously, and we are > unable to duplicate the problems in our environment. > 2. dtrace is a good design. It has little to no overhead for > collection points that are not collecting data. Emulating that > behavior allows you to test what you fly and fly what you test. > 3. Having the ability to only get trace data you want (or need) > instead of trace data for every instruction executed is very > useful. It also reduces bandwidth requirements when archiving the > traces, or shipping them out over a network connection. > 4. The ability to dynamically expand and contract the tracing scope > during a trace is vital to getting the right data. > > The general plan would be to provide collectors in all of the > public python API functions. The filtering code would also be in > the local system, for speed and bandwidth-conserving purposes. The > filters should generally be native code. The data should be > writeable to a file or to a network socket. In addition, there > should be a network socket listening for commands from a tracing > client. The client would be able to upload additional filter > scripts and start or stop tracing transparently to the running source. > > For security there should be an authentication mechanism that > allows the application to decide whether or not a connection to the > trace facility should be allowed, and a way to disable the trace > listening port on startup. (Or perhaps it should be disabled on > startup by default and enabled if so desired.) > > Finally, there should, of course, be Pythonic access to the trace > system. > > The essential differences between the current tracing system and > the proposal involve the selectable granularity, the transparent > remote access, and automatic triggering of collection points. > > Comments? Please make sure to reply to all, I am not subscribed to > the list. I've recently starting thinking of something along these lines although I don't the need to alter existing libraries or even mimic Dtrace given Python's extremely dynamic nature and reflection capabilities. Ian Bicking wrote something along these lines for Paste that is used in Pylons wherein exceptions raised by the application during debug mode drop you into an interactive debugger with a web interface -- very handy for developing webapps. This got me thinking about building a module that could be included by projects that creates a socket and responds to messages on that socket in a separate thread from the main app so that you can connect to the app and explore its current state. Take a look at the evalexception module in Paste to see what he does. Erik Jones Software Developer | Emma® [EMAIL PROTECTED] 800.595.4401 or 615.292.5888 615.292.0777 (fax) Emma helps organizations everywhere communicate & market in style. Visit us online at http://www.myemma.com -- http://mail.python.org/mailman/listinfo/python-list
PEP idea: Instrumented Python
I was looking at adding dtrace-like dynamic tracing to Python. Note that
this isn't dtrace itself. The basic rationale:
1. A lot of enterprise-level software is written in Python. It is difficult
to impossible to reproduce the customer environment in a test lab.
Sometimes applications hang mysteriously, and we are unable to duplicate the
problems in our environment.
2. dtrace is a good design. It has little to no overhead for collection
points that are not collecting data. Emulating that behavior allows you to
test what you fly and fly what you test.
3. Having the ability to only get trace data you want (or need) instead of
trace data for every instruction executed is very useful. It also reduces
bandwidth requirements when archiving the traces, or shipping them out over
a network connection.
4. The ability to dynamically expand and contract the tracing scope during a
trace is vital to getting the right data.
The general plan would be to provide collectors in all of the public python
API functions. The filtering code would also be in the local system, for
speed and bandwidth-conserving purposes. The filters should generally be
native code. The data should be writeable to a file or to a network
socket. In addition, there should be a network socket listening for
commands from a tracing client. The client would be able to upload
additional filter scripts and start or stop tracing transparently to the
running source.
For security there should be an authentication mechanism that allows the
application to decide whether or not a connection to the trace facility
should be allowed, and a way to disable the trace listening port on
startup. (Or perhaps it should be disabled on startup by default and
enabled if so desired.)
Finally, there should, of course, be Pythonic access to the trace system.
The essential differences between the current tracing system and the
proposal involve the selectable granularity, the transparent remote access,
and automatic triggering of collection points.
Comments? Please make sure to reply to all, I am not subscribed to the list.
-={C}=-
--
http://mail.python.org/mailman/listinfo/python-list
