Re: Executing python script stored as a string
On Tuesday 01 September 2009 11:32:29 Steven D'Aprano wrote: > Possibly there is a way to have a thread halt itself after a certain > amount of time? I'm not an expert on threads, I've hardly ever used them. Not automagically, as far as I can see. You are on your own if you want to somehow kill a thread. What I do is to wrap what I want to do in a while clause: while runbool: do_what_you_want() where runbool is a global that I set to false elsewhere when I want to stop. There is of course nothing to stop you writing a thread with something like: import time start_time = time.time() while time.time() - start_time < some_parameter: do_what_you_want() Which will have the effect of running for a while and then stop. I cannot see much use use for that as it is, as it will be bloody-minded about the time, and might chop off before the whole job is done, but one could make it more intelligent, such that it keeps track of idle time, and aborts after say a second (or a minute) of not doing anything useful. Unlike you, I am a thread and queue fanatic - use them all the time. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Sep 1, 11:32 am, Steven D'Aprano
wrote:
> > But I don't quite understand why is it security
> > risk. How is it different to run:
> > exec 'format(your_hdd)'
> > than:
> > /bin/python format.py
> > ?
>
> It's not different. But read what I said -- "if the string is coming from
> an UNTRUSTED source" -- presumably you trust yourself. If you run 'exec
> "format(your_hdd)"' it is because *you* want to format your hard disk.
>
> Now imagine you have a web-app which gets a string from the user and
> calls exec on it. Then you might have this:
>
> exec "search('%d')" % user_input
>
> and the user, who is halfway across the world, enters the following
> search string:
>
> places to eat'); import os; os.system('#rm -rf /
>
> Your web app will go right ahead and erase itself. That's why you need to
> keep untrusted strings away from exec, execfile, and eval.
Ah, I see! Ok.
> No, I believe that the only way to halt that is to halt the entire
> process.
>
> Possibly there is a way to have a thread halt itself after a certain
> amount of time? I'm not an expert on threads, I've hardly ever used them.
Thank you once again!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Tue, 01 Sep 2009 01:34:33 -0700, Ecir Hana wrote:
>> You can copy the parts of the current scope into the namespace you pass
>> to exec, then later copy the revised values out again.
>>
>> But are you sure you really want to take this approach? exec is up to
>> ten times slower than just executing the code directly. And if the
>> string is coming from an untrusted source, it is a *huge* security
>> risk.
>
> I don't know if I should use exec. I don't really mind that it's slow
> (btw., why is it so?).
Because it has to parse and compile the string into a code object before
it can run it.
> But I don't quite understand why is it security
> risk. How is it different to run:
> exec 'format(your_hdd)'
> than:
> /bin/python format.py
> ?
It's not different. But read what I said -- "if the string is coming from
an UNTRUSTED source" -- presumably you trust yourself. If you run 'exec
"format(your_hdd)"' it is because *you* want to format your hard disk.
Now imagine you have a web-app which gets a string from the user and
calls exec on it. Then you might have this:
exec "search('%d')" % user_input
and the user, who is halfway across the world, enters the following
search string:
places to eat'); import os; os.system('#rm -rf /
Your web app will go right ahead and erase itself. That's why you need to
keep untrusted strings away from exec, execfile, and eval.
>> As far as I know, you can't kill threads, you can only ask them to kill
>> themselves.
>
> Also, I'm not sure if I follow. What does this mean? If a thread runs:
>
> while True:
> pass
>
> it is not possible to kill it from another thread? (Bacause it doesn't
> check whether some other thread asks to stop it..?)
No, I believe that the only way to halt that is to halt the entire
process.
Possibly there is a way to have a thread halt itself after a certain
amount of time? I'm not an expert on threads, I've hardly ever used them.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Sep 1, 5:31 am, Steven D'Aprano wrote: > But are you sure you really want to take this approach? exec is up to ten > times slower than just executing the code directly. Oh, you mean because of parsing and compiling? But otherwise it's as fast as regular python? That's perfectly ok. Or maybe I misunderstood you - what do you mean by "executing the code directly"? -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Sep 1, 5:31 am, Steven D'Aprano
wrote:
>
> You can pass in a global and local namespaces to exec as arguments:
>
> >>> x = 4
> >>> ns = {'x': 4}
> >>> exec "x += 1" in ns
> >>> x
> 4
> >>> ns['x']
>
> 5
>
> See the docs for details.
Thanks! This is very useful!
> You can copy the parts of the current scope into the namespace you pass
> to exec, then later copy the revised values out again.
>
> But are you sure you really want to take this approach? exec is up to ten
> times slower than just executing the code directly. And if the string is
> coming from an untrusted source, it is a *huge* security risk.
I don't know if I should use exec. I don't really mind that it's slow
(btw., why is it so?). But I don't quite understand why is it security
risk. How is it different to run:
exec 'format(your_hdd)'
than:
/bin/python format.py
?
> As far as I know, you can't kill threads, you can only ask them to kill
> themselves.
Also, I'm not sure if I follow. What does this mean? If a thread runs:
while True:
pass
it is not possible to kill it from another thread? (Bacause it doesn't
check whether some other thread asks to stop it..?)
> Something like this?
Well, something more like:
data = [1, 2, 3]
map(lambda x: x * 2, data)
display_data_in_editor_viewport(data) #this renders into part of main
editor window (may take some time)
> If so, I think you are making this much too complicated for such a simple
> use-case. Just publish an API which the script can use, and have the main
> text editor application specify a "script" namespace containing only that
> API. That could be a module:
>
> >>> import math # pretend this is your API shared module
> >>> exec "myvalue = 42" in math.__dict__
> >>> math.myvalue
>
> 42
>
> Then execute the text using exec, but don't bother about putting it into
> a thread or subprocess. That just makes it harder to implement, and you
> have to worry about concurrency issues.
Ok, I could try exec, thanks for the explanation. But what about those
security concerns you mentioned above?
Thanks a lot, very informative!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Tue, Sep 1, 2009 at 9:29 AM, Ecir Hana wrote: > - if I understood it correctly defining a function in the string and > exec-ing it created the function in current scope. This is something I > really don't want > Oops, missed that point. May I ask what is there you don't want, and what about putting the function inside a different module and importing? What's so bad about being "in scope"? Sincerely, -Xav -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
I'm afraid I'm not much of a help here, but was there any reason you didn't want to wrap those "string functions" inside a function, and just call the function? Cheers, -Xav -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing python script stored as a string
On Mon, 31 Aug 2009 16:29:45 -0700, Ecir Hana wrote:
> Hello,
>
> please, how to execute a python script stored as a string? But let me
> impose several limitations, so simple "exec" wont work:
>
> - if I understood it correctly defining a function in the string and
> exec-ing it created the function in current scope. This is something I
> really don't want
You can pass in a global and local namespaces to exec as arguments:
>>> x = 4
>>> ns = {'x': 4}
>>> exec "x += 1" in ns
>>> x
4
>>> ns['x']
5
See the docs for details.
> - simple exec also blocks the rest of the program
Run it in a thread.
> - I also would like the string to be able to use and return some parts
> of the caller
You can copy the parts of the current scope into the namespace you pass
to exec, then later copy the revised values out again.
But are you sure you really want to take this approach? exec is up to ten
times slower than just executing the code directly. And if the string is
coming from an untrusted source, it is a *huge* security risk.
> Couple of points:
>
> - the script in string should behave just like any other ordinary python
> script executed in separate process, except it should also know about a
> function caller "up". Nothing else. (I read that something similar is
> possible while embedding python into your C project - that you could
> invoke the VM and provide some default "imports")
If you want it to execute in a separate *process*, that's a whole
different question. If you do that, you get separation of code for free,
as well as separate namespaces. My approach would be to have a special
module "common" which subprocesses can import, to get access to the
shared functions. You will probably need to create some sort of message
passing infrastructure to get results out of the subprocess into the
parent process.
> - if the other script runs in separate process how should it call the
> remote function? And how to pass its arguments? I really hope I don't
> have to serialize every communication, maybe I should use threading
> instead of process?
If you want separate processes, they're *separate*. Threads are not.
> All I want is that running it wont block the caller
> and that it cannot modify callers code/variables/scope (apart from
> calling the predefined callers' functions). Or maybe even better, let it
> block the caller but provide a way to stop its execution?
As far as I know, you can't kill threads, you can only ask them to kill
themselves.
> - how to know that the script finished? I was thinking about atexit() -
> could it work here?
I doubt it. You would need to poll each thread to see if it has completed.
> Think of it as a text editor with a special ability to execute its
> content, while providing access of some of its functionality to the
> script.
Something like this?
In the text editor, you have contents:
text goes here
and more text
# Python script starts here
x = 'a'
up(x)
print "foo"
# Python script stops here
more text again
and the user selects lines 4 and 5 and chooses the command "Execute". The
script executes, and its output (foo) is appended to the end of the file:
text goes here
and more text
# Python script starts here
x = 'a'
up(x)
print "foo"
# Python script stops here
more text again
foo
Is this what you mean?
If so, I think you are making this much too complicated for such a simple
use-case. Just publish an API which the script can use, and have the main
text editor application specify a "script" namespace containing only that
API. That could be a module:
>>> import math # pretend this is your API shared module
>>> exec "myvalue = 42" in math.__dict__
>>> math.myvalue
42
Then execute the text using exec, but don't bother about putting it into
a thread or subprocess. That just makes it harder to implement, and you
have to worry about concurrency issues.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
