Re: Getting better traceback info on exec and execfile - introspection?
Fernando Perez <[EMAIL PROTECTED]> writes: > So any regexp-matching based approach here is likely to be fairly brittle, > unless you restrict your tool to the standard python interpreter, and you > get some guarantee that it will always tag interactive code with > ''. Meant to mention for what it's worth. Looks like I'm not the first to use the filename == '' test. I note this in the stock pdb.py: # To be overridden in derived debuggers def defaultFile(self): """Produce a reasonable default.""" filename = self.curframe.f_code.co_filename if filename == '' and self.mainpyfile: filename = self.mainpyfile return filename I'm not sure under what conditions this equality test normally occurs though. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
Fernando Perez <[EMAIL PROTECTED]> writes: > I thought a little about this. One possibility ... Thanks. A sibling thread has the code I'm currently using. > Oh, that's because you're using %run, so your code is in complete control. > What I meant about a restriction ... Okay. > If you are interested in ipython integration, Yes I am. > I suggest the ipython-dev list as a better place for discussion: I only > monitor c.l.py on occasion, so I may well miss things here. Okay now subscribed. But interestingly I looked in the IPython PDF and didn't see mention of it when looking for contact info. I do however see it (now) listed on http://ipython.scipy.org/. Thanks again. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
"Ziga Seilnacht" <[EMAIL PROTECTED]> writes: > You should check the getFrameInfo function in zope.interface package: > http://svn.zope.org/Zope3/trunk/src/zope/interface/advice.py?rev=25177&view=markup Thanks! Just looked at that. The logic in the relevant part (if I've extracted this correctly): f_globals = frame.f_globals hasName = '__name__' in f_globals module = hasName and sys.modules.get(f_globals['__name__']) or None namespaceIsModule = module and module.__dict__ is f_globals if not namespaceIsModule: # some kind of funky exec kind = "exec" is definitely not obvious to me. exec's don't have module namespace (or something or other)? Okay. And that the way to determine the module-namespace thingy is whatever that logic is? Are the assumptions here be likely to be valid in the future? Another problem I have with that code is that it uses the Zope Public License. But the code is adapted from the Python Enterprise Application Kit (PEAK) which doesn't seem to use the Zope Public License. I'm not sure it's right to adopt a Zope Public License just for this. So instead, I followed the other avenue I suggested which is dissembling the statement around the frame. Talk about the kettle calling the pot black! Yes, I don't know if the assumptions in this method are likely to be valid in the future either. But I can use op-code examination to also help me determine if we are stopped at a def statement which I want to skip over. So here's the code I've currently settled on: from opcode import opname def op_at_frame(frame): code = frame.f_code.co_code pos = frame.f_lasti op = ord(code[pos]) return opname[op] def is_exec_stmt(frame): """Return True if we are looking at an exec statement""" return frame.f_back is not None and op_at_frame(frame.f_back)=='EXEC_STMT' re_def = re.compile(r'^\s*def\s+') def is_def_stmt(line, frame): """Return True if we are looking at a def statement""" # Should really also check that the operand is a code object return re_def.match(line) and op_at_frame(frame)=='LOAD_CONST' But it may just be because I wrote it that I find it easier to understand and more straightforward to fathom. > > And suppose instead of '' I'd like to give the value or the > > leading prefix of the value instead of the unhelpful word ''? > > How would one do that? Again, one way is to go into the outer frame > > get the source line (if that exists), parse that and interpolate > > argument to exec(file). Is there are better way? > > Py library (http://codespeak.net/py/current/doc/index.html) has some > similar functionality in the code subpackage. Again, many thanks. I'll have to study this further. It may be that exec and so on are wrapped so that it's possible to squirrel away the string before calling exec. Again, dunno. But thanks for the pointer. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
R. Bernstein wrote: . . . > which is perhaps is a little more honest since one is not really in a > file called . However the way the debugger gets this *is* > still a little hoaky in that it looks for something in the frame's > f_code.co_filename *called* . And from that it *assumes* this > is an exec, so it can't really tell the difference between an exec > command an execfile command, or a file called . But I suppose > a little more hoakiness *could* be added to look at the outer frame > and parse the source line for "exec" or "execfile". You should check the getFrameInfo function in zope.interface package: http://svn.zope.org/Zope3/trunk/src/zope/interface/advice.py?rev=25177&view=markup > And suppose instead of '' I'd like to give the value or the > leading prefix of the value instead of the unhelpful word ''? > How would one do that? Again, one way is to go into the outer frame > get the source line (if that exists), parse that and interpolate > argument to exec(file). Is there are better way? Py library (http://codespeak.net/py/current/doc/index.html) has some similar functionality in the code subpackage. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
R. Bernstein wrote: > Fernando Perez <[EMAIL PROTECTED]> writes: >> R. Bernstein wrote: > ... >> > However the frame information for exec or execfile looks like this: >> > File "", line 1, in ? >> >> That comes from how the code object was compiled: > ... >> So any regexp-matching based approach here is likely to be fairly >> brittle, unless you restrict your tool to the standard python >> interpreter, and you get some guarantee that it will always tag >> interactive code with ''. > > Thanks for the information! Alas, that's bad news. The question then > remains as to how to accurately determine if a frame is running an > exec operation and accurately get the string associated with the exec. > > Given all the introspection about python's introspection, it should be > possible. I thought a little about this. One possibility would be to check whether the argument given in the string exists as a file; if not, assume it's a string. I can't think of another way, since the string given is completely arbitrary. But someone who knows more about the internal structure of code objects may give you better tips. > I also tried pydb.py using IPython 0.7.0 (using an automatically > updated Fedora Core 5 RPM) and although I was expecting pydb.py breakage > from what you report, I didn't find any difference. > > > % ipython > Python 2.4.2 (#1, Dec 17 2005, 13:02:22) > Type "copyright", "credits" or "license" for more information. > > IPython 0.7.0 -- An enhanced Interactive Python. > ? -> Introduction to IPython's features. > %magic -> Information about IPython's 'magic' % functions. > help-> Python's own help system. > object? -> Details about 'object'. ?object also works, ?? prints more. > > In [1]: %run /usr/lib/python2.4/site-packages/pydb.py test1.py >> /home/rocky/python/test1.py(2)?() > -> import sys > (Pydb) where > -> 0 in file '/home/rocky/python/test1.py' at line 2 > ## 1 in exec cmd in globals, locals at line 1 > ## 2 run() called from file '/usr/lib/python2.4/bdb.py' at line 366 > (Pydb) > > Note entry ##1 is *not* > File "", line 1, in ? > > So somehow I guess ipython is not intercepting or changing how compile > was run here. Oh, that's because you're using %run, so your code is in complete control. What I meant about a restriction was thiking about using your pydb as the debugger called by ipython when exceptions occur. Try %pdb and then %run any script with an error in it, to be dropped in post-mortem mode inside the stack. That uses an ipython-enhanced pdb, but not your pydb. As your code matures and improves, it would be nice to, for example, make it possible to plug it into ipython as a 'better debugger'. But there, you'll see '' as the filename markers. Anyway, good luck with this. If you are interested in ipython integration, I suggest the ipython-dev list as a better place for discussion: I only monitor c.l.py on occasion, so I may well miss things here. Regards, f -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
Fernando Perez <[EMAIL PROTECTED]> writes: > R. Bernstein wrote: ... > > However the frame information for exec or execfile looks like this: > > File "", line 1, in ? > > That comes from how the code object was compiled: ... > So any regexp-matching based approach here is likely to be fairly brittle, > unless you restrict your tool to the standard python interpreter, and you > get some guarantee that it will always tag interactive code with > ''. Thanks for the information! Alas, that's bad news. The question then remains as to how to accurately determine if a frame is running an exec operation and accurately get the string associated with the exec. Given all the introspection about python's introspection, it should be possible. I also tried pydb.py using IPython 0.7.0 (using an automatically updated Fedora Core 5 RPM) and although I was expecting pydb.py breakage from what you report, I didn't find any difference. % ipython Python 2.4.2 (#1, Dec 17 2005, 13:02:22) Type "copyright", "credits" or "license" for more information. IPython 0.7.0 -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help-> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: %run /usr/lib/python2.4/site-packages/pydb.py test1.py > /home/rocky/python/test1.py(2)?() -> import sys (Pydb) where -> 0 in file '/home/rocky/python/test1.py' at line 2 ## 1 in exec cmd in globals, locals at line 1 ## 2 run() called from file '/usr/lib/python2.4/bdb.py' at line 366 (Pydb) Note entry ##1 is *not* File "", line 1, in ? So somehow I guess ipython is not intercepting or changing how compile was run here. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
R. Bernstein wrote:
> In doing the extension to the python debugger which I have here:
>
http://sourceforge.net/project/showfiles.php?group_id=61395&package_id=175827
> I came across one little thing that it would be nice to get done better.
>
> I notice on stack traces and tracebacks, an exec or execfile command
> appears as a stack entry -- probably as it should since it creates a
> new environment.
>
> However the frame information for exec or execfile looks like this:
> File "", line 1, in ?
That comes from how the code object was compiled:
>>> code = compile('error','Anything you want goes here','single')
>>> exec code
Traceback (most recent call last):
File "", line 1, in ?
File "Anything you want goes here", line 1, in ?
NameError: name 'error' is not defined
For example, in ipython:
In [1]: error
---
exceptions.NameError Traceback (most recent
call last)
/home/fperez/
NameError: name 'error' is not defined
So any regexp-matching based approach here is likely to be fairly brittle,
unless you restrict your tool to the standard python interpreter, and you
get some guarantee that it will always tag interactive code with
''.
Regards,
f
--
http://mail.python.org/mailman/listinfo/python-list
Re: Getting better traceback info on exec and execfile - introspection?
I suggested: > And suppose instead of '' I'd like to give the value or the > leading prefix of the value instead of the unhelpful word ''? > How would one do that? Again, one way is to go into the outer frame > get the source line (if that exists), parse that and interpolate > argument to exec(file). Is there are better way? I've hacked this approach into the CVS for pydb, but it is not all that accurate (although in practice I'm pleased with the result). My current regular expression is (^|\s+)exec\s+(.*) which does match simple things, but it would fail on say by *not* matching: x=1;exec "y=2";exec "z=3" and fail by *wrongly* matching on: x=" exec meeting in 5 mins" Even if I knew how to call the python parser on the line, that still might not be good enough in the presence of continuation lines. This probably should be addressed at a lower level in gathering traceback data inside Python. > Another and perhaps more direct way to distinguish exec/execfile might execfile is not a problem, just exec. -- http://mail.python.org/mailman/listinfo/python-list
