Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Tue, Feb 17, 2015 at 7:31 AM, Steven D'Aprano st...@pearwood.info wrote: On Mon, Feb 16, 2015 at 07:10:21PM -0800, Devin Jeanpierre wrote: On Mon, Feb 16, 2015 at 6:15 PM, Steven D'Aprano st...@pearwood.info wrote: Here is a fork of that recipe. It uses an inner class for the new namedtuple class. The only thing which needs exec is the __new__ method. http://code.activestate.com/recipes/578918-yet-another-namedtuple/ This demonstrates a powerful truth about Python: *most of the time* you don't need to use exec or eval because the standard language features are powerful enough to solve the problem for you. Using a dynamically created inner class is *almost* enough to solve this problem, only the __new__ method defeats it. If our requirements where just a little less demanding, we could avoid exec completely. No, exec is not necessary at all. I'm not sure that I said that exec was necessary anywhere, You said that most of the time you don't need to use it. You then explained how you narrowly had to use exec in this case, and could have avoided it if the restrictions were relaxed. That sure sounds like you were saying it was necessary. Maybe you meant to say something else? but since you mention it, how about the two earlier examples I gave, timeit and doctest? Especially doctest. How would you implement doctests without exec? I already covered this in my response to Cameron: I only meant that in this particular case exec is not needed. If they had to the author could have reimplemented the argument assignment logic by hand. They chose not to because it is too hard. (And it is.) Fortunately, they don't have to go that far: signature = inspect.Signature([ inspect.Parameter(field_name, inspect.Parameter.POSITIONAL_OR_KEYWORD) for field_name in field_names]) Hmmm. Well, namedtuple was added to Python in version 2.6. [snip] So much for that idea. What? Why should what was impossible in the past stop you from doing what is possible now? -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Mon, Feb 16, 2015 at 07:10:21PM -0800, Devin Jeanpierre wrote: On Mon, Feb 16, 2015 at 6:15 PM, Steven D'Aprano st...@pearwood.info wrote: Here is a fork of that recipe. It uses an inner class for the new namedtuple class. The only thing which needs exec is the __new__ method. http://code.activestate.com/recipes/578918-yet-another-namedtuple/ This demonstrates a powerful truth about Python: *most of the time* you don't need to use exec or eval because the standard language features are powerful enough to solve the problem for you. Using a dynamically created inner class is *almost* enough to solve this problem, only the __new__ method defeats it. If our requirements where just a little less demanding, we could avoid exec completely. No, exec is not necessary at all. I'm not sure that I said that exec was necessary anywhere, but since you mention it, how about the two earlier examples I gave, timeit and doctest? Especially doctest. How would you implement doctests without exec? Of course you could write your own Python parser, and build your own Python interpreter. But that's just re-inventing exec. And it would be horribly slow. Back in the very early days of PyPy, they wrote a Python interpreter using nothing but pure Python. It was about 100 times slower than the regular Python interpreter. Another reasonable use for exec is to develop your own language or mini-language. You generate Python code, compile it, then execute it. Template engines like Mako, Jinja2 and Genshi work like this, or so I am lead to believe. If they had to the author could have reimplemented the argument assignment logic by hand. They chose not to because it is too hard. (And it is.) Fortunately, they don't have to go that far: signature = inspect.Signature([ inspect.Parameter(field_name, inspect.Parameter.POSITIONAL_OR_KEYWORD) for field_name in field_names]) Hmmm. Well, namedtuple was added to Python in version 2.6. [steve@ando ~]$ python2.6 Python 2.6.7 (r267:88850, Mar 10 2012, 12:32:58) [GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2 Type help, copyright, credits or license for more information. py from inspect import Signature Traceback (most recent call last): File stdin, line 1, in module ImportError: cannot import name Signature So much for that idea. Of course you are right that using exec is rarely the *only possible* way to solve a problem. But it's a tool like any other tool, we shouldn't be afraid to use it when it is the best tool for the job. The problem comes from people using it when it is the *wrong* tool for the job, or using it carelessly. -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] What are *appropriate* uses for exec() and eval() ?
I have heard periodically about the potential evils of using exec() and eval(), including today, on this list. I gather that the first requirement for safely using these functions is that the passed argument MUST be from a trusted source. So what would be examples where the use of these functions IS appropriate? -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Mon, Feb 16, 2015 at 11:52 AM, boB Stepp robertvst...@gmail.com wrote: I have heard periodically about the potential evils of using exec() and eval(), including today, on this list. I gather that the first requirement for safely using these functions is that the passed argument MUST be from a trusted source. So what would be examples where the use of these functions IS appropriate? Given that there are language environments that do perfectly ok without those functions, the flippant answer would be: those functions aren't necessary. But to be more serious: they'd probably be most useful when you're defining your own interactive programming environment. For example, consider a debugger or an IDE (such as IDLE. Or if you come from the Java world, imagine Eclipse). In a debugger, you're running another program, and allowing the user to do something programmatic in the context of that program. Setting breakpoints, or looking at the value of certain expressions. In this scenario, we want to be able to access the same runtime data structures that drive the running program... within the program itself! It's this introspection that drives the need for an eval or exec. Enormously powerful. Enormously dangerous in the wrong hands. That being said, almost all programs are neither debuggers nor IDEs at their heart. (Despite the joke that every program strives to become Emacs at a certain point.) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Mon, Feb 16, 2015 at 6:15 PM, Steven D'Aprano st...@pearwood.info wrote: Here is a fork of that recipe. It uses an inner class for the new namedtuple class. The only thing which needs exec is the __new__ method. http://code.activestate.com/recipes/578918-yet-another-namedtuple/ This demonstrates a powerful truth about Python: *most of the time* you don't need to use exec or eval because the standard language features are powerful enough to solve the problem for you. Using a dynamically created inner class is *almost* enough to solve this problem, only the __new__ method defeats it. If our requirements where just a little less demanding, we could avoid exec completely. No, exec is not necessary at all. If they had to the author could have reimplemented the argument assignment logic by hand. They chose not to because it is too hard. (And it is.) Fortunately, they don't have to go that far: signature = inspect.Signature([ inspect.Parameter(field_name, inspect.Parameter.POSITIONAL_OR_KEYWORD) for field_name in field_names]) def __new__(cls, *args, **kwargs): return tuple.__new__(cls, signature.bind(*args, **kwargs).arguments.values()) -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Mon, Feb 16, 2015 at 01:52:16PM -0600, boB Stepp wrote:
I have heard periodically about the potential evils of using exec()
and eval(), including today, on this list. I gather that the first
requirement for safely using these functions is that the passed
argument MUST be from a trusted source. So what would be examples
where the use of these functions IS appropriate?
The flippant reply would be there aren't any.
But that is not true.
In the broader context of programming in general, not just Python, the
use of eval/exec is incredibly powerful because it allows you to write
dynamic code that uses information available at runtime to solve
problems which cannot even be expressed at compile time.
Think about it like this:
A programming language is like a toolbox filled with tools for solving
problems. You combine those tools like Lego blocks, combining them in
different ways to make new tools. One of those Lego blocks is a robot,
called eval or exec, which you can instruct to make new tools, instead
of making them yourself.
There are various downsides: the extra complexity of telling the robot
which Lego blocks to use, instead of just directly using the blocks
yourself, means that using the robot is slower, more complex, harder to
read, error messages are less useful, and if your instructions contain
data coming from strangers, they may be able to subvert your intention,
sneak instructions into your code, and take control of the robot. But it
means you can put off the decision for which Lego block to use until
runtime when more information is available.
exec is literally a Python interpreter embedded inside Python, so if you
have a particularly hard problem to solve, one of the ways you can solve
it is to write a program to *write a program to solve it*, then use exec
to run that second program.
All this discussion has been very abstract. Here are some concrete
examples of good use of eval/exec.
In the standard library, we have the timeit module which takes a code
snippet from the user, executes it as Python code, and measures how long
it takes. There's no way to take *code* from the user except as a
string, if you type it directly Python will interpret it immediately. To
delay execution, you have to put the code inside a string, and then
later interpret the string as Python code. In other words, exec.
Likewise, we have the doctest module. Inside your function docstrings,
you can write samples of interactive output:
def spam(n):
Return n lots of spam.
spam(3)
'spam spam spam'
...
The doctest module scans the docstring, extracts anything which looks
like interactive output (starting with prompt), execs that text as
Python code, and checks that the output matches what you gave it. Your
sample code is *executable documentation*, so long as you remember to
run dotest over it, you can always be sure that the sample code is
correct.
In the collections module, there is a factory function called namedtuple
for creating record-like tuples with named fields. How it works is you
provide the name of the fields, they get plugged into a class definition
template, and the template executed as Python code, which creates a new
class. That class is returned for you to use:
py from collections import namedtuple
py Record = namedtuple(Record, x y z)
py point = Record(23, 42, 19)
py print(point)
Record(x=23, y=42, z=19)
py point.x
23
Here is the original version which eventually became part of the
collections module:
http://code.activestate.com/recipes/500261-named-tuples/
Here is a fork of that recipe. It uses an inner class for the new
namedtuple class. The only thing which needs exec is the __new__ method.
http://code.activestate.com/recipes/578918-yet-another-namedtuple/
This demonstrates a powerful truth about Python: *most of the time* you
don't need to use exec or eval because the standard language features
are powerful enough to solve the problem for you. Using a dynamically
created inner class is *almost* enough to solve this problem, only the
__new__ method defeats it. If our requirements where just a little less
demanding, we could avoid exec completely.
In some languages, if you want to define functions are runtime, the only
way to do it is to write a function template, fill in the blanks at
runtime, then exec it:
template =
def add(x):
return x + %s
namespace = {}
exec(template % 10, namespace)
addTen = namespace['add']
print(addTen(23))
With Python, going to all that trouble is unnecessary:
def factory(n):
Return a new function which adds n to its argument.
def add(x):
return x + n
return add
addTen = factory(10)
print(addTen(23))
The result is easier to read, faster, and more secure.
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On Mon, Feb 16, 2015 at 7:20 PM, Cameron Simpson c...@zip.com.au wrote:
One might use exec() to use code that is valid in one python version but not
another, when you need your program to run in both i.e. to get code that is
syntacticly invalid in one version, but to use it (conditionally) in another
version.
I only have one use case for this presently: I have a use of exec() in my
cs.py3 python2/3 compatability module:
def raise3(exc_type, exc_value, exc_traceback):
if sys.hexversion = 0x0300:
raise exc_type(exc_value).with_traceback(exc_traceback)
else:
# subterfuge to let this pass a python3 parser; ugly
exec('raise exc_type, exc_value, exc_traceback')
I'm using exec() here because a Python 3 interpreter will reject the 3
argument form of raise. Elsewhere in my code I just call cs.py3.raise3()
with the requisite arguments. Note that the string passed to exec() is
hardwired, not obtained from elsewhere in any form.
I'd try conditional imports, first:
if sys.hexversion = ...:
from .compat_py3 import raise3
else:
from .compat_py2 import raise3
But maybe this breaks with the setuptools pre-compilation shenanigans?
At any rate, I didn't mean to make a general statement. Obviously,
sometimes exec/eval is necessary. If for no other reason than because
sometimes the requirement is to use exec (e.g. if you are implementing
something equivalent to python -i, etc.).
-- Devin
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What are *appropriate* uses for exec() and eval() ?
On 16Feb2015 19:10, Devin Jeanpierre jeanpierr...@gmail.com wrote:
On Mon, Feb 16, 2015 at 6:15 PM, Steven D'Aprano st...@pearwood.info wrote:
Here is a fork of that recipe. It uses an inner class for the new
namedtuple class. The only thing which needs exec is the __new__ method.
http://code.activestate.com/recipes/578918-yet-another-namedtuple/
This demonstrates a powerful truth about Python: *most of the time* you
don't need to use exec or eval because the standard language features
are powerful enough to solve the problem for you. Using a dynamically
created inner class is *almost* enough to solve this problem, only the
__new__ method defeats it. If our requirements where just a little less
demanding, we could avoid exec completely.
No, exec is not necessary at all. If they had to the author could have
reimplemented the argument assignment logic by hand. [... example...]
I see your counter counter example and raise you another counter.
One might use exec() to use code that is valid in one python version but not
another, when you need your program to run in both i.e. to get code that is
syntacticly invalid in one version, but to use it (conditionally) in another
version.
I only have one use case for this presently: I have a use of exec() in my
cs.py3 python2/3 compatability module:
def raise3(exc_type, exc_value, exc_traceback):
if sys.hexversion = 0x0300:
raise exc_type(exc_value).with_traceback(exc_traceback)
else:
# subterfuge to let this pass a python3 parser; ugly
exec('raise exc_type, exc_value, exc_traceback')
I'm using exec() here because a Python 3 interpreter will reject the 3 argument
form of raise. Elsewhere in my code I just call cs.py3.raise3() with the
requisite arguments. Note that the string passed to exec() is hardwired, not
obtained from elsewhere in any form.
Like all sane people, I consider using exec() a code smell: if you're using it
you should consider heavily alternatives to it.
Cheers,
Cameron Simpson c...@zip.com.au
I think... Therefore I ride. I ride... Therefore I am.
- Mark Pope erec...@yarrow.wt.uwa.edu.au
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
