Re: [Python-Dev] Can I introspect/reflect to get arguments exec()?
Thank you for your very thoughtful and detailed explanation of what is going on and for your considerations as to how to get this done (as opposed to why it *can't *be done). It looks like DecoratorTools or more likely a customized version of it is the way to go! (The important info is above. The rest are just some geeky details) You see, there are some places where additional care may be needed in my setting. The debuggers I write use sometimes don't use just the getline module but also use my own pyficache module. I want to also cache things like file stat() info, provide a SHA1 for the text of the file, and provide colorized syntax-highlighted versions of the text when desired. But since I control pyficache, I can mirror the changes made to getline. Of course the debugger uses sys.settrace too, so the evil-ness of that is definitely not a concern. But possibly I need to make sure that since the DecoratorTools and the debugger both hook into trace hooks they play nice together and fire in the right order. And for that I created another module called tracer() which takes into account that one might want to specify priorities in the chain hook order, and that one might want a to filter out (i.e. ignore) certain calls to the hook function for specific hooks. It may be a while before I seriously get to this, but again, it is good to have in mind an approach to take. So thanks again. On Thu, Mar 28, 2013 at 1:45 AM, PJ Eby p...@telecommunity.com wrote: On Tue, Mar 26, 2013 at 11:00 PM, Rocky Bernstein ro...@gnu.org wrote: Okay. But is the string is still somewhere in the CPython VM stack? (The result of LOAD_CONST 4 above). Is there a way to pick it up from there? Maybe using C you could peek into the frame's value stack, but that's not exposed to any Python API I know of. But that still doesn't help you, because the value will be removed from the stack before exec() is actually called, which means if you go looking for it in code called from the exec (e.g. the call event itself), you aren't going to see the data. At the point that we are stopped the exec action hasn't taken place yet. That doesn't help if you're using line-level tracing events. At the beginning of the line, the data's not on the call stack yet, and by the time you enter the frame of the code being exec'd, it'll be off the stack again. Basically, there is no way to do what you're asking for, short of replacing the built-in exec function with your own version. And it still won't help you with stepping through the source of functions that are *compiled* using an exec() or compile(), or any other way of ending up with dynamically-generated code you want to debug. (Unless you use something like DecoratorTools to generate it, that is -- DecoratorTools has some facilities for caching dynamically-generated code so that it works properly with debuggers. But that has to be done by the code doing the generation, not the debugger. If the code generator uses DecoratorTools' caching support, then any debugger that uses the linecache module will Just Work. It might be nice for the stdlib should have something like this, but you could also potentially fake it by replacing the builtin eval, exec, compile, etc. functions w/versions that cache the source.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can I introspect/reflect to get arguments exec()?
[asked on comp.lang.python but no takers. So I'm bumping it up a notch.]
I have ported my Python debugger pydbgr to Python3. See [1] or [2].
Inside the debugger, when there is an exec() somewhere in the call stack,
I'd like to be able to retrieve the string parameter. With this, the
debugger can show part of the string in a call stack. Or it can show the
text when the frame is set to that exec() frame.
Going further, the debugger could write the exec string out to a temporary
file. And when reporting locations, it could report not just something like
string line 4, but also give that temporary file name which a front-end
could use as well.
So consider this code using inspect.getargvalues() and
inspect.currentframe():
import inspect
def my_exec(string):
show_args(inspect.currentframe()) # simulate exec(string)
def show_args(frame):
print(inspect.getargvalues(frame))
my_exec(show_args(inspect.currentframe()))
exec(show_args(inspect.currentframe()))
When run this is the output:
python3 exec-args.py
ArgInfo(args=['string'], varargs=None, keywords=None, locals={'string':
'show_args(inspect.currentframe())'})
ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec':
function my_exec at 0xb6f828ec,, ...
In a different setting, CPython byte-code assembly that gets generated for
running exec() is:
25 88 LOAD_GLOBAL10 (exec)
91 LOAD_CONST 4
('show_args(inspect.currentframe())')
--94 CALL_FUNCTION 1
97 POP_TOP
What's going on?
Also, I have the same question for CPython 2.6 or Python 2.7.
Thanks!
[1] http://code.google.com/p/pydbgr/
[2] http://code.google.com/p/python3-trepan/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I introspect/reflect to get arguments exec()?
On Tue, Mar 26, 2013 at 10:18 PM, Benjamin Peterson benja...@python.orgwrote:
2013/3/26 Rocky Bernstein ro...@gnu.org:
[asked on comp.lang.python but no takers. So I'm bumping it up a notch.]
I have ported my Python debugger pydbgr to Python3. See [1] or [2].
Inside the debugger, when there is an exec() somewhere in the call stack,
I'd like to be able to retrieve the string parameter. With this, the
debugger can show part of the string in a call stack. Or it can show the
text when the frame is set to that exec() frame.
Going further, the debugger could write the exec string out to a
temporary
file. And when reporting locations, it could report not just something
like
string line 4, but also give that temporary file name which a
front-end
could use as well.
So consider this code using inspect.getargvalues() and
inspect.currentframe():
import inspect
def my_exec(string):
show_args(inspect.currentframe()) # simulate exec(string)
def show_args(frame):
print(inspect.getargvalues(frame))
my_exec(show_args(inspect.currentframe()))
exec(show_args(inspect.currentframe()))
When run this is the output:
python3 exec-args.py
ArgInfo(args=['string'], varargs=None, keywords=None,
locals={'string':
'show_args(inspect.currentframe())'})
ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec':
function my_exec at 0xb6f828ec,, ...
In a different setting, CPython byte-code assembly that gets generated
for
running exec() is:
25 88 LOAD_GLOBAL10 (exec)
91 LOAD_CONST 4
('show_args(inspect.currentframe())')
--94 CALL_FUNCTION 1
97 POP_TOP
What's going on?
execing something is not the same as calling it, so there are no arguments.
Okay. But is the string is still somewhere in the CPython VM stack? (The
result of LOAD_CONST 4 above). Is there a way to pick it up from there? At
the point that we are stopped the exec action hasn't taken place yet.
--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] License for code extracted from a Python library?
Hi - I'm not sure if this is the right place to ask this, but I don't know of a more appropriate place. Sometime around 2007 I first extracted a function that was in Python's Cmd class and used that. Since then, I have modified and generalized it a bit more and turned it into a Python egg called columnizehttp://pypi.python.org/pypi/columnize. I also ported it to Ruby and made Ruby package or gem. A couple of people/companies have asked about redistributing this and have asked for a more MIT-like and less GPL-like license. I guess they will bundle this with their proprietary code. For the Python package, I used PSL Version 2, but is this legitimate? code.google.com doesn't seem to recognize in their list of licenses so that suggests what I did might not be the right thing. (So instead, code.google.com I selected the MIT License.) Can a MIT License be used for code extracted from Python's standard library? Other comments or suggestions? Thanks. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] code.interact() locals + globals?
I notice code.interact() in Python 2.6 has a parameter for locals but not globals. Should it? Ultimately, *exec* is used underneath so I don't see a technical reason why it couldn't add a global parameter. Also, although one could pass in a dictionary that is the merger or update() of locals() and globals(), this has a couple of downsides. First, if the intention is to allow update of local and global variables, combining the two into a single dictionary wreaks havoc. Even if one doesn't care about this, when the underlying *exec* is performed it will see different and wrong values for the functions locals() and globals() among other problems of this kind. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should there be a source-code checksum in module objects?
As I've mentioned, I've been re-examining from ground up the whole state of affairs in writing a debugger. One of the challenges of a debugger or any source-code analysis tool is verifying that the source-code that the tool is reporting on corresponds to the compiled object under execution. (For debuggers, this problem becomes more likely to occur when you are debugging on a computer that isn't the same as the computer where the code is running.) Is there a checksum of the source text computed and stored in compilation? If not, should there be one? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should execv() call _run_exitfuncs()? If not, should _run_exitfuncs() be private?
As a hobby I've been writing a debugger. One of the commands,restart, works by calling an execv(). You may need to do this when the program you are debugging is threaded or when one needs to ensure that all program state is reinitialized. Recently, I added remote debugging via TCP sockets and noticed that execv() in Python doesn't arrange exit hooks to get called. Should it? One can run _run_exitfuncs() from the atexit module, but the leading underscore of the function call suggests it is private. Should it be? Thanks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?
Now that there is a package mechanism (are package mechanisms?) like zipimporter that bundle source code into a single file, should the notion of a file location should be adjusted to include the package and/or importer? Is there a standard API or routine which can extract this information given a code object? A use case here I am thinking of here is in a stack trace or a debugger, or a tool which wants to show in great detail information from a code object possibly via a frame. For example does this come from a zipped egg? And if so, which one? For concreteness, here is what I did and here's what I saw. Select one of the zipimporter eggs at http://code.google.com/p/pytracer and install one of these. I did this on GNU/Linux and Python 2.5 and I look at the co_filename of one of the methods: import tracer tracer.__dict__['size'].func_code.co_filename 'build/bdist.linux-i686/egg/tracer.py' But there is no file called build/bdist.linux-686/egg/tracer.py in the filesystem. Instead there is a member tracer.py inside /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg'. It's possible I caused this egg to get built incorrectly or that setuptools has a bug which entered that misleading information. However, shouldn't there be a standard way to untangle package location, loader and member inside the package? As best as I can tell, PEP 302 which discussed importer hooks and suggests a standard way to get file data. But it doesn't address a standard way to get container package and/or loader information. Also I'm not sure there *is* a standard print string way to show member inside a package. zipimporter may insert co_filename strings like: /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg/tracer.py but the trouble with this is that it means file routines have to scan the path and notice say that /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg is a *file*, not a directory. And a file stat/reading routine needs to understand what kind of packager that is in order to get tracer.py information. (Are there any file routines in place for doing this?) Thanks. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
