Re: pre-PEP: Simple Thunks
Mike Meyer wrote: Ron_Adam <[EMAIL PROTECTED]> writes: Here's yet another way to do it, but it has some limitations as well. import pickle def pickle_it(filename, obj, commands): try: f = open(filename, 'r') obj = pickle.load(f) f.close() except IOError: pass for i in commands: i[0](i[1]) f = open(filename, 'w') pickle.dump(obj, f) f.close() file = 'filename' L = [] opps = [ (L.append,'more data'), (L.append,'even more data') ] pickle_it(file, L, opps) Um - it doesn't look like this will work. You pass L in as the "obj" paremeter, and then it gets changed to the results of a pickle.load. However, you call L.append in the for loop, so the data will be appended to L, not obj. You'll lose an list elements that were in the pickle. Hmmm, you are correct. :-/ I was trying to not use eval(). This could be made to work, but it will get messy, which is another reason not to do it. Cheers, Ron -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Ron_Adam <[EMAIL PROTECTED]> writes: > Here's yet another way to do it, but it has some limitations as well. > > import pickle > def pickle_it(filename, obj, commands): > try: > f = open(filename, 'r') > obj = pickle.load(f) > f.close() > except IOError: > pass > for i in commands: > i[0](i[1]) > f = open(filename, 'w') > pickle.dump(obj, f) > f.close() > > file = 'filename' > L = [] > > opps = [ (L.append,'more data'), > (L.append,'even more data') ] > pickle_it(file, L, opps) Um - it doesn't look like this will work. You pass L in as the "obj" paremeter, and then it gets changed to the results of a pickle.load. However, you call L.append in the for loop, so the data will be appended to L, not obj. You'll lose an list elements that were in the pickle. http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information. -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Greg Ewing wrote:
Brian Sabbey wrote:
do f in with_file('file.txt'):
print f.read()
I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???
To sound right it would have to be something like
with_file('file.txt') as f do:
print f.read()
This is still strange since f is the arguments the thunk was called
with, e.g. the current syntax is basically:
do in = ():
I don't really know a more readable sequence of keywords, though someone
suggested 'with' and 'from', which might read something like:
with from ():
which looks okay to me, though I'm not sure that 'with' makes it clear
that this is not a normal block... I also find readability problems
when I try to stick back in.
One of the other issues is that, with the current proposal, the thunk
can be called multiple times within a function, so the keywords have to
make sense both with a single iteration interpretation and a multiple
iteration interpretation... Makes it even harder...
STeVe
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Brian Sabbey wrote:
do f in with_file('file.txt'):
print f.read()
I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???
To sound right it would have to be something like
with_file('file.txt') as f do:
print f.read()
But, while that works with this particular function
name, and others of the form "with_xxx", there are
bound to be other use cases which would require
different words or word orders in order not to sound
contrived.
It's very difficult to come up with a good syntax for
this that isn't skewed towards one kind of use case.
That's probably a large part of the reason why nothing
like it has so far been seriously considered for
adoption.
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Mon, 18 Apr 2005 21:11:52 -0700, Brian Sabbey
<[EMAIL PROTECTED]> wrote:
>Ron_Adam wrote:
>> The load and dump would be private to the data class object. Here's a
>> more complete example.
>>
>> import pickle
>> class PickledData(object):
>> def __init__(self, filename):
>> self.filename = filename
>> self.L = None
>> try:
>> self._load()
>> except IOError:
>> self.L = []
>> def _load(self):
>> f = open(self.filename, 'r')
>> self.L = pickle.load(f)
>> f.close()
>> def _update(self):
>> f = open(self.filename, 'w')
>> pickle.dump(self.L, f)
>> f.close()
>> def append(self, record):
>> self.L.append(record)
>> self._update()
>> # add other methods as needed ie.. get, sort, clear, etc...
>>
>> pdata = PickledData('filename')
>>
>> pdata.append('more data')
>> pdata.append('even more data')
>>
>> print pdata.L
>> ['more data', 'even more data']
>>
>>
>>> This has the same issues as with opening and closing files: losing the
>>> 'dump', having to always use try/finally if needed, accidentally
>>> re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
>>> be as readable as the 'pickled_file' function above since 'load' and
>>> 'dump' are separate methods that share data through 'self'.
>>
>> A few more lines to create the class, but it encapsulates the data
>> object better. It is also reusable and extendable.
>
>This class isn't reusable in the case that one wants to pickle something
>other than an array. Every type of object that one would wish to pickle
>would require its own class.
...Or in a function, or the 3 to 6 lines of pickle code someplace.
Many programs would load data when they start, and then save it when
the user requests it to be saved. So there is no one method fits all
situations. Your thunk example does handle some things better.
Here's yet another way to do it, but it has some limitations as well.
import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()
file = 'filename'
L = []
opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)
>Also, this implementation behaves differently because the object is
>re-pickled after every modification. This could be a problem when writing
>over a network, or to a shared resource.
>
>-Brian
In some cases writing to the file after ever modification would be
desired. A way around that would be to use a buffer of some sort. But
then again, that adds another level of complexity and you would have
to insure it's flushed at some point which get's back to the issue of
not closing a file.
Thanks for explaining how thunks works. I'm still undecided on
whether it should be built in feature or not.
I would rather have a way to store a block of code and pass it to a
function, then execute it at the desired time. That would solve both
the issue where you would use a thunk, and replace lambdas as well.
But I understand there's a lot of resistance to that because of the
potential abuse.
Cheers,
Ron
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Ron_Adam wrote:
The load and dump would be private to the data class object. Here's a
more complete example.
import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...
pdata = PickledData('filename')
pdata.append('more data')
pdata.append('even more data')
print pdata.L
['more data', 'even more data']
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.
This class isn't reusable in the case that one wants to pickle something
other than an array. Every type of object that one would wish to pickle
would require its own class.
Also, this implementation behaves differently because the object is
re-pickled after every modification. This could be a problem when writing
over a network, or to a shared resource.
-Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Sun, 17 Apr 2005 19:56:10 -0700, Brian Sabbey
<[EMAIL PROTECTED]> wrote:
>I also wouldn't do it that way. I don't see a class as being much better,
>though. If I understand you correctly, with classes you would have
>something like:
>
>p = Pickled('pickled.txt')
>p.load()
>p.data.append('more data')
>p.data.append('even more data')
>p.dump()
The load and dump would be private to the data class object. Here's a
more complete example.
import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...
pdata = PickledData('filename')
pdata.append('more data')
pdata.append('even more data')
print pdata.L
['more data', 'even more data']
>This has the same issues as with opening and closing files: losing the
>'dump', having to always use try/finally if needed, accidentally
>re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
>be as readable as the 'pickled_file' function above since 'load' and
>'dump' are separate methods that share data through 'self'.
A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.
Cheers,
Ron
>The motivation for thunks is similar to the motivation for generators--
>yes, a class could be used instead, but in many cases it's more work than
>should be necessary.
>
>-Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Brian Sabbey wrote:
do f in with_file('file.txt'):
print f.read()
def with_file(filename):
f = open(filename)
yield f
f.close()
for f in with_file('file.txt'):
print f.read()
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
t = "no file read yet"
for f in with_file('file.txt'):
t = f.read()
--
Serhiy Storchaka
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Sun, 17 Apr 2005 15:32:56 -0700, Brian Sabbey <[EMAIL PROTECTED]> wrote: >Bengt Richter wrote: >> Hm, one thing my syntax does, I just noticed, is allow you >> to pass several thunks to a thunk-accepter, if desired, e.g., >> (parenthesizing this time, rather than ending (): with >> dedented comma) >> >>each( >>((i): # normal thunk >>print i), >>((j): # alternative thunk >>rejectlist.append(j)), >>[1,2]) >> >> >> > >I see that it might be nice to be able to use multiple thunks, and to be >able to specify the positions in the argument list of thunks, but I think >allowing a suite inside parentheses is pretty ugly. One reason is that it >is difficult to see where the suite ends and where the argument list >begins again. I'm not sure even what the syntax would be exactly. I >suppose the suite would always have to be inside its own parentheses? >Also, you wind up with these closing parentheses far away from their >corresponding open parentheses, which is also not pretty. It's getting >too Lisp-like for my tastes. > Having had a past love affair (or at least fling) with scheme, that doesn't bother me so much ;-) The "where" specifier syntax might help, e.g., each(thk1, thk2, [1, 2]) where: thk1 = (i): # normal thunk print i thk2 = (j): # alternative thunk rejectlist.append(j) This still uses my (): expression for thunks but they don't need to be parenthesized, because their suites terminate with normal dedenting under the where: suite I.e., the 'p' of 'print i' is the left edge of the (i): suite and thus the 't' of 'thk2 = ...' ends the (i): suite. The (j): suite left edge is at the 'r' of 'rejectlist'... so anything to the left of that (excluding comments) will end the (j): suite, like normal indentation. I could have used dedent termination in the previous example too by moving the commas around to let them trigger dedent level out of the preceding suite, e.g., with args evaluated in place again: each((i): # normal thunk print i ,(j): # alternative thunk rejectlist.append(j) ,[1,2]) Of course, if you want lispy, the above simple thunks can be done in a oneliner: each(((i):print i), ((j):rejectlist.append(j)), [1,2]) I personally like the power of being able to write that, but given a clean sugar alternative, I would use it. But if it's an exclusive-or choice, I'll take primitives over sugar, because sugar never covers all the useful combinations of primitives that will turn up later. Regards, Bengt Richter -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Ron_Adam wrote:
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
Now I can re-use pickled_file whenever I have to modify a pickled file:
do data in pickled_file('pickled.txt'):
data.append('more data')
data.append('even more data')
In my opinion, that is easier and faster to write, more readable, and less
bug-prone than any non-thunk alternative.
The above looks like it's missing something to me. How does 'data'
interact with 'thunk(l)'? What parts are in who's local space?
Your example below explains it well, with 'data' renamed as 'L'. The
scope of bindings are the same in both examples, with the exception that
'data' is in the outermost namespace in the above example, and 'L' is
local to the function 'data_append' in the below example.
This might be the non-thunk version of the above.
yes
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
def data_append(L):
L.append('more data')
L.append('still more data')
pickled_file(data_append, name)
I don't think I would do it this way. I would put the data
list in a class and add a method to it to update the pickle file. Then
call that from any methods that update the data list.
I also wouldn't do it that way. I don't see a class as being much better,
though. If I understand you correctly, with classes you would have
something like:
p = Pickled('pickled.txt')
p.load()
p.data.append('more data')
p.data.append('even more data')
p.dump()
This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.
The motivation for thunks is similar to the motivation for generators--
yes, a class could be used instead, but in many cases it's more work than
should be necessary.
-Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Brian Sabbey wrote:
used, but rarely is because doing so would be awkward. Probably the
simplest real-world example is opening and closing a file. Rarely will
you see code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you
to make this abstraction in a way that is more concise and more readable
than the callback example given above:
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they
allow variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
Definitely put this example into the PEP. I didn't really understand
what you were suggesting until I saw this example. All the other ones
you gave just confused me more.
When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar. It can
be used in a flow control-like way, or as an item-in-a-list way.
Please spend some time in the PEP explaining why you chose the keywords
you chose. They gave me all the wrong intuitions about what was
supposed to be going on, and really confused me. I also got mixed up in
when you were talking about parameters to the thunk, and when you were
talking about parameters to the function that is called with the thunk
as a parameter.
I'd also like to see you start with the full example syntax, e.g.:
do in = ():
And then explain what each piece does more carefully. Something like:
"""
When a do-statement is executed, first is called with the
parameters , augmented by the thunk object, e.g.
do func(4, b=2):
...
would call
func(thunk_obj, 4, b=2)
Next, the body of the function is executed. If the thunk object is
called, then will be executed with the names in
bound to the objects with which the thunk was called, e.g.
def func(thunk):
thunk(1, y=2)
do x, y, z=4 in func():
print x, y, z
would call:
func(thunk_obj)
thunk(1, y=2)
and thus x, y and z would be bound to 1, 2 and 4 and the body of the
thunk would be executed, printing "1 2 4".
The code in is then resumed, and the process is repeated
until returns. Note that this means that each additional
call to the thunk object will cause another execution of , with
potentially different bindings for the names in .
When the function finally returns, the return value will be bound to
, e.g.:
def func(thunk):
thunk()
thunk()
return True
do r = func():
print "thunk called"
print r
would print "thunk called" twice as the body of the thunk is executed
for each call to thunk() in func, and then would print "True" in the
code following the do-statement.
"""
Not sure if I actually understood everything right, but you definitely
need a much more throrough walkthrough of what happens with a thunk --
it's not clear at all from the current pre-PEP.
STeVe
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Sun, 17 Apr 2005 15:02:12 -0700, Brian Sabbey
<[EMAIL PROTECTED]> wrote:
Brian Sabbey wrote:
> I'm kicking myself for the first example I gave in my original post in
> this thread because, looking at it again, I see now that it really gives
> the wrong impression about what I want thunks to be in python. The
> 'thunkit' function above shouldn't be in the same namespace as the thunk.
> It is supposed to be a re-usable function, for example, to acquire and
> release a resource. On the other hand, the 'foo' function is supposed to
> be in the namespace of the surrounding code; it's not re-usable. So your
> example above is pretty much the opposite of what I was trying to get
> across.
This would explain why I'm having trouble seeing it then.
> def pickled_file(thunk, name):
> f = open(name, 'r')
> l = pickle.load(f)
> f.close()
> thunk(l)
> f = open(name, 'w')
> pickle.dump(l, f)
> f.close()
>
> Now I can re-use pickled_file whenever I have to modify a pickled file:
>
> do data in pickled_file('pickled.txt'):
> data.append('more data')
> data.append('even more data')
>
> In my opinion, that is easier and faster to write, more readable, and less
> bug-prone than any non-thunk alternative.
>
The above looks like it's missing something to me. How does 'data'
interact with 'thunk(l)'? What parts are in who's local space?
This might be the non-thunk version of the above.
def pickled_file(thunk, name):
f = open(name, 'r')
l = pickle.load(f)
f.close()
thunk(l)
f = open(name, 'w')
pickle.dump(l, f)
f.close()
def data_append(L):
L.append('more data')
L.append('still more data')
pickled_file(data_append, name)
I don't think I would do it this way. I would put the data
list in a class and add a method to it to update the pickle file. Then
call that from any methods that update the data list.
>> def with_file: # no argument list, local group.
>> f = open(filename)
>> t = callback(f)
>> f.close
>>
>> def my_read(f):
>> return f.read()
>>
>> callback = my_read
>> filename = 'filename'
>> do with_file
>
> This wouldn't work since with_file wouldn't be re-usable. It also doesn't
> get rid of the awkwardness of defining a callback.
As long as the name with_file isn't rebound to something else it could
be used as often as needed. I admit there are better ways to do it
though.
Cheers,
Ron
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Bengt Richter wrote: Hm, one thing my syntax does, I just noticed, is allow you to pass several thunks to a thunk-accepter, if desired, e.g., (parenthesizing this time, rather than ending (): with dedented comma) each( ((i): # normal thunk print i), ((j): # alternative thunk rejectlist.append(j)), [1,2]) I see that it might be nice to be able to use multiple thunks, and to be able to specify the positions in the argument list of thunks, but I think allowing a suite inside parentheses is pretty ugly. One reason is that it is difficult to see where the suite ends and where the argument list begins again. I'm not sure even what the syntax would be exactly. I suppose the suite would always have to be inside its own parentheses? Also, you wind up with these closing parentheses far away from their corresponding open parentheses, which is also not pretty. It's getting too Lisp-like for my tastes. -Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Ron_Adam wrote:
On Sat, 16 Apr 2005 17:25:00 -0700, Brian Sabbey
Yes, much of what thunks do can also be done by passing a function
argument. But thunks are different because they share the surrounding
function's namespace (which inner functions do not), and because they can
be defined in a more readable way.
Generally my reason for using a function is to group and separate code
from the current name space. I don't see that as a drawback.
I agree that one almost always wants separate namespaces when defining a
function, but the same is not true when considering only callback
functions. Thunks are a type of anonymous callback functions, and so a
separate namespace usually isn't required or desired.
Are thunks a way to group and reuse expressions in the current scope?
If so, why even use arguments? Wouldn't it be easier to just declare
what I need right before calling the group? Maybe just informally
declare the calling procedure in a comment.
def thunkit: # no argument list defines a local group.
# set thunk,x and y before calling.
before()
result = thunk(x,y)
after()
def foo(x,y):
x, y = y, x
return x,y
thunk = foo
x,y = 1,2
do thunkit
print result
-> (2,1)
Since everything is in local name space, you really don't need to
pass arguments or return values.
I'm kicking myself for the first example I gave in my original post in
this thread because, looking at it again, I see now that it really gives
the wrong impression about what I want thunks to be in python. The
'thunkit' function above shouldn't be in the same namespace as the thunk.
It is supposed to be a re-usable function, for example, to acquire and
release a resource. On the other hand, the 'foo' function is supposed to
be in the namespace of the surrounding code; it's not re-usable. So your
example above is pretty much the opposite of what I was trying to get
across.
The 'do' keyword says to evaluate the group. Sort of like eval() or
exec would, but in a much more controlled way. And the group as a
whole can be passed around by not using the 'do'. But then it starts
to look and act like a class with limits on it. But maybe a good
replacement for lambas?
I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
I don't think so. My pickled_file example below can't be done as cleanly
with a class. If I were to want to ensure the closing of the pickled
file, the required try/finally could not be encapsulated in a class or
function.
You're right that, in this case, it would be better to just write
"f(stuff, 27, 28)". That example was just an attempt at describing the
syntax and semantics rather than to provide any sort of motivation. If
the thunk contained anything more than a call to 'stuff', though, it would
not be as easy as passing 'stuff' to 'f'. For example,
do f(27, 28):
print stuff()
would require one to define and pass a callback function to 'f'. To me,
'do' should be used in any situation in which a callback *could* be used,
but rarely is because doing so would be awkward. Probably the simplest
real-world example is opening and closing a file. Rarely will you see
code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you to
make this abstraction in a way that is more concise and more readable than
the callback example given above:
How would abstracting open/close help reduce bugs?
I gave two examples of bugs that one can encounter when using open/close.
Personally, I have run into the first one at least once.
I'm really used to using function calls, so anything that does things
differently tend to be less readable to me. But this is my own
preference. What is most readable to people tends to be what they use
most. IMHO
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they allow
variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more
difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
re
Re: pre-PEP: Simple Thunks
On 17 Apr 2005 01:46:14 -0700, "Kay Schluehr" <[EMAIL PROTECTED]> wrote: >Ron_Adam wrote: > >> I sort of wonder if this is one of those things that looks like it >> could be useful at first, but it turns out that using functions and >> class's in the proper way, is also the best way. (?) > >I think Your block is more low level. Yes, that's my thinking too. I'm sort of looking to see if there is a basic building blocks here that can be used in different situations but in very consistent ways. And questioning the use of it as well. >It is like copying and pasting >code-fragments together but in reversed direction: ... Yes, in a function call, you send values to a remote code block and receive back a value. The reverse is to get a remote code block, then use it. In this case the inserted code blocks variables become local, So my point is you don't need to use arguments to pass values. But you do need to be very consistent in how you use the code block. (I'm not suggesting we do this BTW) The advantage to argument passing in this case would be that it puts a control on the block that certain arguments get assigned before it gets executed. Is it possible to have a tuple argument translation independently of a function call? This would also be a somewhat lower level operation, but might be useful, for example at a certain point in a program you want to facilitate that certain values are set, you could use a tuple argument parser to do so. It could act the same way as a function call argument parser but could be used in more places and it would raise an error as expected. Basically it would be the same as: def argset(x,y,z=1): return x,y,z But done in a inline way. a,b,c = (x,y,z=1) # looks familiar doesn't it. ;-) As an inline expression it could use '%' like the string methods, something like this? (x,y,z=1)%a,b,c # point x,y,z to a,b,c (?) And combined with code chunks like this. def chunk: # code chunk, ie.. no arguments. # set (x,y) # informal arguments commented return x+y # return a value for inline use value = (x,y)%a,b: chunk# use local code chunk as body I think this might resemble some of the suggested lambda replacements. The altenative might be just to have functions name space imported as an option. def f(x,y): return x+y z = dolocal f(1,2) #But why would I want to do this? I think these pre-peps are really about doing more with less typing and don't really add anything to Python. I also feel that the additional abstraction when used only to compress code, will just make programs harder to understand. >You have to change all the environments that use the thunk e.g. >renaming variables. It is the opposite direction of creating >abstractions i.e. a method to deabstract functions: introduce less >modularity and more direct dependencies. This is the reason why those >macros are harmfull and should be abandoned from high level languages ( >using them in C is reasonable because code expansion can be time >efficient and it is also a way to deal with parametric polymorphism but >Python won't benefit from either of this issues ). > >Ciao, >Kay I agree. If a code block of this type is used it should be limited to within the function it's defined in. The advantage, if it's used in a bare bones low level way with no argument passing, would be some performance benefits over function calls in certain situations. That is, a small reusable code block without the function call overhead. But only use it in the local name space it's defined in. Otherwise use a function or a class. Cheers, Ron -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Ron_Adam wrote: > I sort of wonder if this is one of those things that looks like it > could be useful at first, but it turns out that using functions and > class's in the proper way, is also the best way. (?) I think Your block is more low level. It is like copying and pasting code-fragments together but in reversed direction: if You copy and paste a code fragment and wants to change the behaviour You probable have to change the fragment the same way in every place of occurence. This is a well known anti-pattern in software construction. If You change Your thunk somehow e.g. from def yield_thunk: yield i to def yield_thunk: yield j You have to change all the environments that use the thunk e.g. renaming variables. It is the opposite direction of creating abstractions i.e. a method to deabstract functions: introduce less modularity and more direct dependencies. This is the reason why those macros are harmfull and should be abandoned from high level languages ( using them in C is reasonable because code expansion can be time efficient and it is also a way to deal with parametric polymorphism but Python won't benefit from either of this issues ). Ciao, Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Sat, 16 Apr 2005 18:46:28 -0700, Brian Sabbey <[EMAIL PROTECTED]> wrote: [...] >> In that case, my version would just not have a do, instead defining the do >> suite >> as a temp executable suite, e.g., if instead >> >> >> we make an asignment in the suite, to make it clear it's not just a calling >> thing, e.g., >> >> do f(27, 28): >> x = stuff() >> >> then my version with explict name callable suite would be >> >> def f(thunk, a, b): >> # a == 27, b == 28 >> before() >> thunk() >> after() >> >> set_x(): >> x = stuff() # to make it plain it's not just a calling thing >> >> f(set_x, 27, 28) >> # x is now visible here as local binding >> >> but a suitable decorator and an anonymous callable suite (thunk defined my >> way ;-) would make this >> >> @f(27, 28) >> (): x = stuff() >> > >Hmm, but this would require decorators to behave differently than they do >now. Currently, decorators do not automatically insert the decorated >function into the argument list. They require you to define 'f' as: > >def f(a, b): > def inner(thunk): > before() > thunk() > after() > return inner > >Having to write this type of code every time I want an thunk-accepting >function that takes other arguments would be pretty annoying to me. > Yes, I agree. That's the way it is with the decorator expression. Maybe decorator syntax is not the way to pass thunks to a function ;-) My latest thinking is in terms of suite expressions, :: being one, and actually (): is also a suite expression, yielding a thunk. So the call to f could just be written explicitly with the expression in line: f((():x=stuff()), 27, 28) or f((): x = stuff() done = True ,27, 28) # letting the dedented ',' terminate the (): rather than parenthesizing or safe_open((f): for line in f: print f[:20] ,'datafile.txt', 'rb') That's not too bad IMO ;-) >>> >>> Thunks can also accept arguments: >>> >>> def f(thunk): >>>thunk(6,7) >>> >>> do x,y in f(): >>># x==6, y==7 >>>stuff(x,y) >> >> IMO >>@f >>(x, y): stuff(x, y) # like def foo(x, y): stuff(x, y) >> >> is clearer, once you get used to the missing def foo format >> > >OK. I prefer a new keyword because it seems confusing to me to re-use >decorators for this purpose. But I see your point that decorators can be >used this way if one allows anonymous functions as you describe, and if >one allows decorators to handle the case in which the function being >decorated is anonymous. I tend to agree now about using decorators for this. With thunk calling parameter and extra f calling parameters, in line would look like f((x,y): # x==6, y==7 stuff(x, y) ,'other' ,'f' ,args) I guess you could make a bound method to keep the thunk dothunk_n_all = f.__get__((x, y): stuff(x,y) ,type(():pass)) and then call that with whatever other parameter you specified for f do_thunk_n_all(whatever) if that seemed useful ;-) > >>> >>> The return value can be captured >>> >> This is just a fallout of f's being an ordinary function right? >> IOW, f doesn't really know thunk is not an ordinary callable too? >> I imagine that is for the CALL_FUNCTION byte code implementation to discover? > >yes > >> >>> def f(thunk): >>>thunk() >>>return 8 >>> >>> do t=f(): >>># t not bound yet >>>stuff() >>> >>> print t >>> ==> 8 >> That can't be done very well with a decorator, but you could pass an >> explicit named callable suite, e.g., >> >> thunk(): stuff() >> t = f(thunk) > >But not having to do it that way was most of the purpose of thunks. I forgot that (): is an expression t = f(():stuff()) # minimal version or final_status = safe_open((f): for line in f: print f[:20] ,'datafile.txt', 'rb') > >It wouldn't be a problem to use 'return' instead of 'continue' if people >so desired, but I find 'return' more confusing because a 'return' in 'for' >suites returns from the function, not from the suite. That is, having >'return' mean different things in these two pieces of code would be >confusing: > >for i in [1,2]: > return 10 > >do i in each([1,2]): > return 10 But in my syntax, each((i): return 10 ,[1,2]) Um, well, I guess one has to think about it ;-/ The thunk-accepter could pass the thunk a mutable arg to put a return value in, or even a returnvalue verse thunk? def accepter(thk, seq): acquire() for i in seq: thk(i, (retval):pass) if retval: break release() accepter((i, rvt): print i rvt(i==7) # is this legal? , xrange(10)) Hm, one thing my syntax does, I just noticed, is allow you to pass several thunks to a thunk-accepter, if desired, e.g., (parenthesizing this time, rather than ending (): with dedented comma) each(
Re: pre-PEP: Simple Thunks
On Sat, 16 Apr 2005 17:25:00 -0700, Brian Sabbey
<[EMAIL PROTECTED]> wrote:
>> You can already do this, this way.
>>
> def f(thunk):
>> ... before()
>> ... thunk()
>> ... after()
>> ...
> def before():
>> ... print 'before'
>> ...
> def after():
>> ... print 'after'
>> ...
> def stuff():
>> ... print 'stuff'
>> ...
> def morestuff():
>> ... print 'morestuff'
>> ...
> f(stuff)
>> before
>> stuff
>> after
> f(morestuff)
>> before
>> morestuff
>> after
>
>>
>> This works with arguments also.
>
>Yes, much of what thunks do can also be done by passing a function
>argument. But thunks are different because they share the surrounding
>function's namespace (which inner functions do not), and because they can
>be defined in a more readable way.
Generally my reason for using a function is to group and separate code
from the current name space. I don't see that as a drawback.
Are thunks a way to group and reuse expressions in the current scope?
If so, why even use arguments? Wouldn't it be easier to just declare
what I need right before calling the group? Maybe just informally
declare the calling procedure in a comment.
def thunkit: # no argument list defines a local group.
# set thunk,x and y before calling.
before()
result = thunk(x,y)
after()
def foo(x,y):
x, y = y, x
return x,y
thunk = foo
x,y = 1,2
do thunkit
print result
-> (2,1)
Since everything is in local name space, you really don't need to
pass arguments or return values.
The 'do' keyword says to evaluate the group. Sort of like eval() or
exec would, but in a much more controlled way. And the group as a
whole can be passed around by not using the 'do'. But then it starts
to look and act like a class with limits on it. But maybe a good
replacement for lambas?
I sort of wonder if this is one of those things that looks like it
could be useful at first, but it turns out that using functions and
class's in the proper way, is also the best way. (?)
>You're right that, in this case, it would be better to just write
>"f(stuff, 27, 28)". That example was just an attempt at describing the
>syntax and semantics rather than to provide any sort of motivation. If
>the thunk contained anything more than a call to 'stuff', though, it would
>not be as easy as passing 'stuff' to 'f'. For example,
>
>do f(27, 28):
> print stuff()
>
>would require one to define and pass a callback function to 'f'. To me,
>'do' should be used in any situation in which a callback *could* be used,
>but rarely is because doing so would be awkward. Probably the simplest
>real-world example is opening and closing a file. Rarely will you see
>code like this:
>
>def with_file(callback, filename):
> f = open(filename)
> callback(f)
> f.close()
>
>def print_file(file):
> print file.read()
>
>with_file(print_file, 'file.txt')
>
>For obvious reasons, it usually appears like this:
>
>f = open('file.txt')
>print f.read()
>f.close()
>
>Normally, though, one wants to do a lot more than just print the file.
>There may be many lines between 'open' and 'close'. In this case, it is
>easy to introduce a bug, such as returning before calling 'close', or
>re-binding 'f' to a different file (the former bug is avoidable by using
>'try'/'finally', but the latter is not). It would be nice to be able to
>avoid these types of bugs by abstracting open/close. Thunks allow you to
>make this abstraction in a way that is more concise and more readable than
>the callback example given above:
How would abstracting open/close help reduce bugs?
I'm really used to using function calls, so anything that does things
differently tend to be less readable to me. But this is my own
preference. What is most readable to people tends to be what they use
most. IMHO
>do f in with_file('file.txt'):
> print f.read()
>
>Thunks are also more useful than callbacks in many cases since they allow
>variables to be rebound:
>
>t = "no file read yet"
>do f in with_file('file.txt'):
> t = f.read()
>
>Using a callback to do the above example is, in my opinion, more
>difficult:
>
>def with_file(callback, filename):
> f = open(filename)
> t = callback(f)
> f.close()
> return t
>
>def my_read(f):
> return f.read()
>
>t = with_file(my_read, 'file.txt')
Wouldn't your with_file thunk def look pretty much the same as the
callback?
I wouldn't use either of these examples. To me the open/read/close
example you gave as the normal case would work fine, with some basic
error checking of course. Since Python's return statement can handle
multiple values, it's no problem to put everything in a single
function and return both the status with an error code if any, and the
result. I would keep the open, read/write, and close statements in
the same function and not split them up.
>> When I see 'do', it reminds me of 'do lo
Re: pre-PEP: Simple Thunks
Bengt Richter wrote: Good background on thunks can be found in ref. [1]. UIAM most of that pre-dates decorators. What is the relation of thunks to decorators and/or how might they interact? Hmm, I think you answered this below better than I could ;). def f(thunk): before() thunk() after() do f(): stuff() The above code has the same effect as: before() stuff() after() Meaning "do" forces the body of f to be exec'd in do's local space? What if there are assignments in f? I don't think you mean that would get executed in do's local space, that's what the thunk call is presumably supposed to do... Yes, I see now that there is an ambiguity in this example that I did not resolve. I meant that the suite of the 'do' statement gets wrapped up as an anonymous function. This function gets passed to 'f' and can be used by 'f' in the same way as any other function. Bindings created in 'f' are not visible from the 'do' statement's suite, and vice-versa. That would be quite trickier than I intended. Other arguments to 'f' get placed after the thunk: def f(thunk, a, b): # a == 27, b == 28 before() thunk() after() do f(27, 28): stuff() I'm not sure how you intend this to work. Above you implied (ISTM ;-) that the entire body of f would effectively be executed locally. But is that true? What if after after() in f, there were a last statment hi='from last statement of f' Would hi be bound at this point in the flow (i.e., after d f(27, 28): stuff() )? I didn't mean to imply that. The body of 'f' isn't executed any differently if it were called as one normally calls a function. All of its bindings are separate from the bindings in the 'do' statement's scope. I'm thinking you didn't really mean that. IOW, by magic at the time of calling thunk from the ordinary function f, thunk would be discovered to be what I call an executable suite, whose body is the suite of your do statement. yes In that case, f iself should not be a callable suite, since its body is _not_ supposed to be called locally, and other than the fact that before and after got called, it was not quite exact to say it was _equivalent_ to before() stuff() # the do suite after() yes, I said "same effect as" instead of "equivalent" so that too much wouldn't be read from that example. I see now that I should have been more clear. In that case, my version would just not have a do, instead defining the do suite as a temp executable suite, e.g., if instead we make an asignment in the suite, to make it clear it's not just a calling thing, e.g., do f(27, 28): x = stuff() then my version with explict name callable suite would be def f(thunk, a, b): # a == 27, b == 28 before() thunk() after() set_x(): x = stuff() # to make it plain it's not just a calling thing f(set_x, 27, 28) # x is now visible here as local binding but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this @f(27, 28) (): x = stuff() Hmm, but this would require decorators to behave differently than they do now. Currently, decorators do not automatically insert the decorated function into the argument list. They require you to define 'f' as: def f(a, b): def inner(thunk): before() thunk() after() return inner Having to write this type of code every time I want an thunk-accepting function that takes other arguments would be pretty annoying to me. Thunks can also accept arguments: def f(thunk): thunk(6,7) do x,y in f(): # x==6, y==7 stuff(x,y) IMO @f (x, y): stuff(x, y) # like def foo(x, y): stuff(x, y) is clearer, once you get used to the missing def foo format OK. I prefer a new keyword because it seems confusing to me to re-use decorators for this purpose. But I see your point that decorators can be used this way if one allows anonymous functions as you describe, and if one allows decorators to handle the case in which the function being decorated is anonymous. The return value can be captured This is just a fallout of f's being an ordinary function right? IOW, f doesn't really know thunk is not an ordinary callable too? I imagine that is for the CALL_FUNCTION byte code implementation to discover? yes def f(thunk): thunk() return 8 do t=f(): # t not bound yet stuff() print t ==> 8 That can't be done very well with a decorator, but you could pass an explicit named callable suite, e.g., thunk(): stuff() t = f(thunk) But not having to do it that way was most of the purpose of thunks. Thunks blend into their environment ISTM this needs earlier emphasis ;-) def f(thunk): thunk(6,7) a = 20 do x,y in f(): a = 54 print a,x,y ==> 54,6,7 IMO that's more readable as def f(thunk): thunk(6, 7) @f (x, y): # think def foo(x, y): with "def foo" missing to make it a thunk a = 54 print a,x,y IMO we need some real use cases, or we'll never be ab
Re: pre-PEP: Simple Thunks
[EMAIL PROTECTED] wrote:
Keep in mind that most of the problems come from the "space is
significant" thing, which is IMHO a very good idea, but prevents us from
putting code in expressions, like :
func( a,b, def callback( x ):
print x
)
or does it ? maybe this syntax could be made to work ?
Hmm. I'd like to think that suite-based keywords do make this example
work. One just has to let go of the idea that all arguments to a function
appear inside parentheses.
Comments on the thunks.
First of all I view code blocks as essential to a language. They are
very useful for a lot of common programming tasks (like defining callbacks in
an elegant way) :
button = create_button( "Save changes" ):
do
self.save()
However it seems your thunks can't take parameters, which to me is a
big drawback. In ruby a thunk can take parameters. Simple examples :
field = edit_field( "initial value", onsubmit={ |value| if
self.validate(value) then do something else alert( "the value is invalid" ) }
)
[1,3,4].each { |x| puts x }
Thunks can take parameters:
do value in field = edit_field("initial value"):
if self.validate(value):
something
else:
alert("the value is invalid")
But callbacks are better defined with suite-based keywords:
field = edit_field("initial value"):
def onsubmit(value):
if self.validate(value):
something
else:
alert("the value is invalid")
This has the advantage that the interface to the thunk (ie. its
parameters) are right there before your eyes instead of being buried in the
thunk invocation code inside the edit_field.
In the cases in which one is defining a callback, I agree that it would be
preferable to have the thunk parameters not buried next to 'do'.
However, I do not consider thunks a replacement for callbacks. They are a
replacement for code in which callbacks could be used, but normally aren't
because using them would be awkward. Your 'withfile' example below is a
good one. Most people wouldn't bother defining a 'withfile' function,
they would just call 'open' and 'close' individually.
So I think it's essential that thunks take parameters and return a
value (to use them effectively as callbacks).
What shall distinguish them from a simple alteration to def(): which
returns the function as a value, and an optional name ? really I don't know,
but it could be the way they handle closures and share local variables with
the defining scope. Or it could be that there is no need for two kinds of
function/blocks and so we can reuse the keyword def() :
Right, defining a function with 'def' is different than defining a thunk
because thunks share the namespace of the surrounding function, functions
do not:
x = 1
def f():
x = 2 # <- creates a new x
g(f)
print x # ==> 1
do g():
x = 2
print x # ==> 2 ( assuming 'g' calls the thunk at least once)
If you wish to modify def(), you could do, without creating any
keyword :
# standard form
f = def func( params ):
code
# unnamed code block taking params
func = def (params):
code
Note that the two above are equivalent with regard to the variable
"func", ie. func contains the defined function. Actually I find def
funcname() to be bloat, as funcname = def() has the same functionality, but
is a lot more universal.
# unnamed block taking no params
f = def:
code
I'm confused. These examples seem to do something different than what a
'do' statement would do. They create a function with a new namespace and
they do not call any "thunk-accepting" function.
***
Comments on the suite-based keywords.
Did you notice that this was basically a generalized HEREDOC syntax ?
I'd say that explicit is better than implicit, hence...
Your syntax is :
do f(a,b):
a block
passes block as the last parameter of f.
I don't like it because it hides stuff.
Yes, it hides stuff. It doesn't seem to me any worse than what is done
with 'self' when calling a method though. Once I got used to 'self'
appearing automatically as the first parameter to a method, it wasn't a
big deal for me.
I'd write :
f(a,b,@>,@>):
"""a very
large multi-line
string"""
def (x):
print x
Here the @> is a symbol (use whatever you like) to indicate
"placeholder for something which is on the next line".
Indentation indicates that the following lines are indeed argument
for the function. The : at the end of the function call is there for
coherence with the rest of the syntax.
Notice that, this way, there is no need for a "do" keyword, as the
code block created by an anonymous def() is no different that the other
parameter, in this case a multilin
Re: pre-PEP: Simple Thunks
On Sat, 16 Apr 2005, Ron_Adam wrote:
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
I'm not familiar with Ruby or Smalltalk. Could you explain this
without referring to them?
Hopefully my example below is more understandable. I realize now that I
should have provided more motivational examples.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
You can already do this, this way.
def f(thunk):
... before()
... thunk()
... after()
...
def before():
... print 'before'
...
def after():
... print 'after'
...
def stuff():
... print 'stuff'
...
def morestuff():
... print 'morestuff'
...
f(stuff)
before
stuff
after
f(morestuff)
before
morestuff
after
This works with arguments also.
Yes, much of what thunks do can also be done by passing a function
argument. But thunks are different because they share the surrounding
function's namespace (which inner functions do not), and because they can
be defined in a more readable way.
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Can you explain what 'do' does better?
Why is the 'do' form better than just the straight function call?
f(stuff, 27, 28)
The main difference I see is the call to stuff is implied in the
thunk, something I dislike in decorators. In decorators, it works
that way do to the way the functions get evaluated. Why is it needed
here?
You're right that, in this case, it would be better to just write
"f(stuff, 27, 28)". That example was just an attempt at describing the
syntax and semantics rather than to provide any sort of motivation. If
the thunk contained anything more than a call to 'stuff', though, it would
not be as easy as passing 'stuff' to 'f'. For example,
do f(27, 28):
print stuff()
would require one to define and pass a callback function to 'f'. To me,
'do' should be used in any situation in which a callback *could* be used,
but rarely is because doing so would be awkward. Probably the simplest
real-world example is opening and closing a file. Rarely will you see
code like this:
def with_file(callback, filename):
f = open(filename)
callback(f)
f.close()
def print_file(file):
print file.read()
with_file(print_file, 'file.txt')
For obvious reasons, it usually appears like this:
f = open('file.txt')
print f.read()
f.close()
Normally, though, one wants to do a lot more than just print the file.
There may be many lines between 'open' and 'close'. In this case, it is
easy to introduce a bug, such as returning before calling 'close', or
re-binding 'f' to a different file (the former bug is avoidable by using
'try'/'finally', but the latter is not). It would be nice to be able to
avoid these types of bugs by abstracting open/close. Thunks allow you to
make this abstraction in a way that is more concise and more readable than
the callback example given above:
do f in with_file('file.txt'):
print f.read()
Thunks are also more useful than callbacks in many cases since they allow
variables to be rebound:
t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()
Using a callback to do the above example is, in my opinion, more
difficult:
def with_file(callback, filename):
f = open(filename)
t = callback(f)
f.close()
return t
def my_read(f):
return f.read()
t = with_file(my_read, 'file.txt')
When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
some sort of flow control. I gather you mean it as do items in a
list, but with the capability to substitute the named function. Is
this correct?
I used 'do' because that's what ruby uses for something similar. It can
be used in a flow control-like way, or as an item-in-a-list way. For
example, you could replace 'if' with your own version (not that you would
want to):
def my_if(thunk, val):
if val:
thunk()
do my_if(a): # same as "if a:"
assert a
(replacing "else" wouldn't be possible without allowing multiple thunks.)
Or you could create your own 'for' (again, NTYWWT):
def my_for(thunk, vals):
for i in vals:
thunk(i)
do i in my_for(range(10)):
print i
-Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
Leif K-Brooks wrote: Brian Sabbey wrote: Thunk statements contain a new keyword, 'do', as in the example below. The body of the thunk is the suite in the 'do' statement; it gets passed to the function appearing next to 'do'. The thunk gets inserted as the first argument to the function, reminiscent of the way 'self' is inserted as the first argument to methods. It would probably make more sense to pass the thunk as the last argument, not as the first. That would make it easier to create functions with optional thunks, as in: def print_nums(start, end, thunk=None): for num in xrange(start, end+1): if thunk is not None: num = thunk(num) print num print_nums(1, 3) # prints 1, 2, 3 do num print_nums(1, 3): # prints 2, 4, 6 continue num * 2 That seems like a good idea to me. I suppose it also makes sense to have the thunk last because it appears after all the other arguments in the function call. Because thunks blend into their environment, a thunk cannot be used after its surrounding 'do' statement has finished Why? Ordinary functions don't have that restriction: def foo(): ... x = 1 ... def bar(): ... return x ... return bar ... foo()() 1 Thunks, as I implemented them, don't create a closure. I believe that creating a closure will require a performance penalty. Since one use of thunks is in loops, it seems that their performance may often be important. I believe that saving the thunk and calling it a later time is a somewhat hackish way to use thunks. Explicitly defining a function (perhaps with a suite-based keyword :) ) seems to me to be a more readable way to go. But, yes, it is an arbitrary restriction. If it turns out that performance isn't really affected by creating a closure, or that performance doesn't matter as much as I think it does, then this restriction could be lifted. -Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Fri, 15 Apr 2005 16:44:58 -0700, Brian Sabbey <[EMAIL PROTECTED]> wrote: > >Simple Thunks >- > >Thunks are, as far as this PEP is concerned, anonymous functions that >blend into their environment. They can be used in ways similar to code >blocks in Ruby or Smalltalk. One specific use of thunks is as a way to >abstract acquire/release code. Another use is as a complement to >generators. I'm not familiar with Ruby or Smalltalk. Could you explain this without referring to them? >A Set of Examples >= > >Thunk statements contain a new keyword, 'do', as in the example below. The >body of the thunk is the suite in the 'do' statement; it gets passed to >the function appearing next to 'do'. The thunk gets inserted as the first >argument to the function, reminiscent of the way 'self' is inserted as the >first argument to methods. > >def f(thunk): >before() >thunk() >after() > >do f(): >stuff() > >The above code has the same effect as: > >before() >stuff() >after() You can already do this, this way. >>> def f(thunk): ... before() ... thunk() ... after() ... >>> def before(): ... print 'before' ... >>> def after(): ... print 'after' ... >>> def stuff(): ... print 'stuff' ... >>> def morestuff(): ... print 'morestuff' ... >>> f(stuff) before stuff after >>> f(morestuff) before morestuff after >>> This works with arguments also. >Other arguments to 'f' get placed after the thunk: > >def f(thunk, a, b): > # a == 27, b == 28 > before() > thunk() > after() > >do f(27, 28): > stuff() Can you explain what 'do' does better? Why is the 'do' form better than just the straight function call? f(stuff, 27, 28) The main difference I see is the call to stuff is implied in the thunk, something I dislike in decorators. In decorators, it works that way do to the way the functions get evaluated. Why is it needed here? When I see 'do', it reminds me of 'do loops'. That is 'Do' involves some sort of flow control. I gather you mean it as do items in a list, but with the capability to substitute the named function. Is this correct? Cheers, Ron -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
On Fri, 15 Apr 2005 16:44:58 -0700, Brian Sabbey <[EMAIL PROTECTED]> wrote: >Here is a first draft of a PEP for thunks. Please let me know what you >think. If there is a positive response, I will create a real PEP. > >I made a patch that implements thunks as described here. It is available >at: > http://staff.washington.edu/sabbey/py_do > >Good background on thunks can be found in ref. [1]. UIAM most of that pre-dates decorators. What is the relation of thunks to decorators and/or how might they interact? > >Simple Thunks >- > >Thunks are, as far as this PEP is concerned, anonymous functions that >blend into their environment. They can be used in ways similar to code >blocks in Ruby or Smalltalk. One specific use of thunks is as a way to >abstract acquire/release code. Another use is as a complement to >generators. "blend into their environment" is not very precise ;-) If you are talking about the code executing in the local namespace as if part of a suite instead of apparently defined in a separate function, I think I would prefer a different syntax ;-) > >A Set of Examples >= > >Thunk statements contain a new keyword, 'do', as in the example below. The >body of the thunk is the suite in the 'do' statement; it gets passed to >the function appearing next to 'do'. The thunk gets inserted as the first >argument to the function, reminiscent of the way 'self' is inserted as the >first argument to methods. > >def f(thunk): >before() >thunk() >after() > >do f(): >stuff() > >The above code has the same effect as: > >before() >stuff() >after() Meaning "do" forces the body of f to be exec'd in do's local space? What if there are assignments in f? I don't think you mean that would get executed in do's local space, that's what the thunk call is presumably supposed to do... But let's get on to better examples, because this is probably confusing some, and I think there are better ways to spell most use cases than we're seeing here so far ;-) I want to explore using the thunk-accepting function as a decorator, and defining an anonymous callable suite for it to "decorate" instead of using the do x,y in deco: or do f(27, 28): format. To define an anonymous callable suite (aka thunk), I suggest the syntax for do x,y in deco: suite should be @deco (x, y):# like def foo(x, y): without the def and foo suite BTW, just dropping the def makes for a named thunk (aka callable suite), e.g. foo(x, y): suite which you could call like foo(10, 4) with the local-where-suite-was-define effect of x = 10 y = 4 suite BTW, a callable local suite also makes case switching by calling through locals()[xsuitename]() able to rebind local variables. Also, since a name is visible in an enclosing scope, it could conceivably provide a mechanism for rebinding there. E.g., def outer(): xsuite(arg): x = arg def inner(): xsuite(5) x = 2 print x # => 2 inner() print x # => 5 But it would be tricky if outer returned inner as a closure. Or if it returned xsuite, for that matter. Probably simplest to limit callable suites to the scope where they're defined. > >Other arguments to 'f' get placed after the thunk: > >def f(thunk, a, b): > # a == 27, b == 28 > before() > thunk() > after() > >do f(27, 28): > stuff() I'm not sure how you intend this to work. Above you implied (ISTM ;-) that the entire body of f would effectively be executed locally. But is that true? What if after after() in f, there were a last statment hi='from last statement of f' Would hi be bound at this point in the flow (i.e., after d f(27, 28): stuff() )? I'm thinking you didn't really mean that. IOW, by magic at the time of calling thunk from the ordinary function f, thunk would be discovered to be what I call an executable suite, whose body is the suite of your do statement. In that case, f iself should not be a callable suite, since its body is _not_ supposed to be called locally, and other than the fact that before and after got called, it was not quite exact to say it was _equivalent_ to before() stuff() # the do suite after() In that case, my version would just not have a do, instead defining the do suite as a temp executable suite, e.g., if instead we make an asignment in the suite, to make it clear it's not just a calling thing, e.g., do f(27, 28): x = stuff() then my version with explict name callable suite would be def f(thunk, a, b): # a == 27, b == 28 before() thunk() after() set_x(): x = stuff() # to make it plain it's not just a calling thing f(set_x, 27, 28) # x is now visible here as local binding but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this @f(27, 28) (): x = stuff() > >Thunks can also accept arguments: > >def f(thunk): >
Re: pre-PEP: Simple Thunks
Brian Sabbey wrote: > def get_items(thunk):# <-- "thunk-accepting function" > f = acquire() > try: > for i in f: > thunk(i) # A-OK > finally: > f.release() > > do i in get_items(): > print i Seems like You want to solve the addressed generator problem by manipulating the syntax: "make generators look more function like", because father compiler won't complain ;-) Sorry, but IMO this is hackery and has nothing to do with good language design and I consider this as extremely harmfull. Instead of making things explicit it does it the other way round and tries to make Python code more obscure. Moreover I can't notice the superiority of thunks in Your other examples over more common techniques like decorators for pre- and postconditions and the GOF command pattern. I thinks the place for such ideas are Michael Hudsons famous bytecodehacks. -1 for from me for thunks in Python. Ciao, Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Simple Thunks
I think your proposal is very interesting, I've been missing code blocks
in Python more and more as time goes by.
I'll answer to both the 'thunks" proposal and the "suite-based keywords"
proposal here.
I find the Ruby syntax rather dirty though, because it has a lot of
implicit stuff, treats code blocks as different from normal arguments,
allows passing only one code block, needs a proc keyword, has yield
execute an implicit block... all this, coming from a Python "explicit is
better than implicit" background (which makes a lot of sense) is simply
ugly.
I like your syntax but have a few comments.
I'll give you an unordered list of ideas, up to you to do what you like
with them.
Keep in mind that most of the problems come from the "space is
significant" thing, which is IMHO a very good idea, but prevents us from
putting code in expressions, like :
func( a,b, def callback( x ):
print x
)
or does it ? maybe this syntax could be made to work ?
Comments on the thunks.
First of all I view code blocks as essential to a language. They are very
useful for a lot of common programming tasks (like defining callbacks in
an elegant way) :
button = create_button( "Save changes" ):
do
self.save()
However it seems your thunks can't take parameters, which to me is a big
drawback. In ruby a thunk can take parameters. Simple examples :
field = edit_field( "initial value", onsubmit={ |value| if
self.validate(value) then do something else alert( "the value is invalid"
) } )
[1,3,4].each { |x| puts x }
This has the advantage that the interface to the thunk (ie. its
parameters) are right there before your eyes instead of being buried in
the thunk invocation code inside the edit_field.
a more complex random example :
fields['password1'] = edit_field( "Enter Password" )
fields['password2'] = edit_field( "Enter it again", onsubmit = {|value,
other_values| if value != other_values['password_1'] then alert('the two
passwords must be the same !") }
So I think it's essential that thunks take parameters and return a value
(to use them effectively as callbacks).
What shall distinguish them from a simple alteration to def(): which
returns the function as a value, and an optional name ? really I don't
know, but it could be the way they handle closures and share local
variables with the defining scope. Or it could be that there is no need
for two kinds of function/blocks and so we can reuse the keyword def() :
If you wish to modify def(), you could do, without creating any keyword
:
# standard form
f = def func( params ):
code
# unnamed code block taking params
func = def (params):
code
Note that the two above are equivalent with regard to the variable
"func", ie. func contains the defined function. Actually I find def
funcname() to be bloat, as funcname = def() has the same functionality,
but is a lot more universal.
# unnamed block taking no params
f = def:
code
***
Comments on the suite-based keywords.
Did you notice that this was basically a generalized HEREDOC syntax ?
I'd say that explicit is better than implicit, hence...
Your syntax is :
do f(a,b):
a block
passes block as the last parameter of f.
I don't like it because it hides stuff.
I'd write :
f(a,b,@>,@>):
"""a very
large multi-line
string"""
def (x):
print x
Here the @> is a symbol (use whatever you like) to indicate "placeholder
for something which is on the next line".
Indentation indicates that the following lines are indeed argument for
the function. The : at the end of the function call is there for coherence
with the rest of the syntax.
Notice that, this way, there is no need for a "do" keyword, as the code
block created by an anonymous def() is no different that the other
parameter, in this case a multiline string.
This has many advantages.
It will make big statements more readable :
instead of :
f( a,b, [some very big expression made up of nested class constructors
like a form defintion ], c, d )
write :
f( a, b, @>, c, d ):
[the very big expression goes here]
So, independently of code blocks, this already improves the readability
of big statements.
You could also use named parameters (various proposals):
f( a,b, c=@>, d=@> ):
value of c
value of d
or :
f( a,b, @*> ):
value of c
value of d
f( a,b, @**> ):
c: value of c
d: value of d
Notice how this mimics f( a,b, * ) and f(a,b, ** ) for multiple
arguments, and multiple named arguments. Do you like it ? I do. Especially
the named version whe
Re: pre-PEP: Simple Thunks
Brian Sabbey wrote: Thunk statements contain a new keyword, 'do', as in the example below. The body of the thunk is the suite in the 'do' statement; it gets passed to the function appearing next to 'do'. The thunk gets inserted as the first argument to the function, reminiscent of the way 'self' is inserted as the first argument to methods. It would probably make more sense to pass the thunk as the last argument, not as the first. That would make it easier to create functions with optional thunks, as in: def print_nums(start, end, thunk=None): for num in xrange(start, end+1): if thunk is not None: num = thunk(num) print num print_nums(1, 3) # prints 1, 2, 3 do num print_nums(1, 3): # prints 2, 4, 6 continue num * 2 Because thunks blend into their environment, a thunk cannot be used after its surrounding 'do' statement has finished Why? Ordinary functions don't have that restriction: >>> def foo(): ... x = 1 ... def bar(): ... return x ... return bar ... >>> foo()() 1 -- http://mail.python.org/mailman/listinfo/python-list
pre-PEP: Simple Thunks
Here is a first draft of a PEP for thunks. Please let me know what you think. If there is a positive response, I will create a real PEP. I made a patch that implements thunks as described here. It is available at: http://staff.washington.edu/sabbey/py_do Good background on thunks can be found in ref. [1]. Simple Thunks - Thunks are, as far as this PEP is concerned, anonymous functions that blend into their environment. They can be used in ways similar to code blocks in Ruby or Smalltalk. One specific use of thunks is as a way to abstract acquire/release code. Another use is as a complement to generators. A Set of Examples = Thunk statements contain a new keyword, 'do', as in the example below. The body of the thunk is the suite in the 'do' statement; it gets passed to the function appearing next to 'do'. The thunk gets inserted as the first argument to the function, reminiscent of the way 'self' is inserted as the first argument to methods. def f(thunk): before() thunk() after() do f(): stuff() The above code has the same effect as: before() stuff() after() Other arguments to 'f' get placed after the thunk: def f(thunk, a, b): # a == 27, b == 28 before() thunk() after() do f(27, 28): stuff() Thunks can also accept arguments: def f(thunk): thunk(6,7) do x,y in f(): # x==6, y==7 stuff(x,y) The return value can be captured def f(thunk): thunk() return 8 do t=f(): # t not bound yet stuff() print t ==> 8 Thunks blend into their environment def f(thunk): thunk(6,7) a = 20 do x,y in f(): a = 54 print a,x,y ==> 54,6,7 Thunks can return values. Since using 'return' would leave it unclear whether it is the thunk or the surrounding function that is returning, a different keyword should be used. By analogy with 'for' and 'while' loops, the 'continue' keyword is used for this purpose: def f(thunk): before() t = thunk() # t == 11 after() do f(): continue 11 Exceptions raised in the thunk pass through the thunk's caller's frame before returning to the frame in which the thunk is defined: def catch_everything(thunk): try: thunk() except: pass# SomeException gets caught here try: do catch_everything(): raise SomeException except: pass# SomeException doesn't get caught here because it was already caught Because thunks blend into their environment, a thunk cannot be used after its surrounding 'do' statement has finished: thunk_saver = None def f(thunk): global thunk_saver thunk_saver = thunk do f(): pass thunk_saver() # exception, thunk has expired 'break' and 'return' should probably not be allowed in thunks. One could use exceptions to simulate these, but it would be surprising to have exceptions occur in what would otherwise be a non-exceptional situation. One would have to use try/finally blocks in all code that calls thunks just to deal with normal situations. For example, using code like def f(thunk): thunk() prevent_core_meltdown() with code like do f(): p = 1 return p would have a different effect than using it with do f(): return 1 This behavior is potentially a cause of bugs since these two examples might seem identical at first glance. The thunk evaluates in the same frame as the function in which it was defined. This frame is accessible: def f(thunk): frame = thunk.tk_frame do f(): pass Motivation == Thunks can be used to solve most of the problems addressed by PEP 310 [2] and PEP 288 [3]. PEP 310 deals with the abstraction of acquire/release code. Such code is needed when one needs to acquire a resource before its use and release it after. This often requires boilerplate, it is easy to get wrong, and there is no visual indication that the before and after parts of the code are related. Thunks solve these problems by allowing the acquire/release code to be written in a single, re-usable function. def acquire_release(thunk): f = acquire() try: thunk(f) finally: f.release() do t in acquire_release(): print t More generally, thunks can be used whenever there is a repeated need for the same code to appear before and after other code. For example, do WaitCursor(): compute_for_a_long_time() is more organized, easier to read and less bug-prone than the code DoWaitCursor(1) compute_for_a_long_time() DoWaitCursor(-1) PEP 288 tries to overcome some of the limitations of generators. One limitation is that a 'yield' is not allowed in the 'try' block of a 'try'/'finally' statement. def get_items(): f = acquire() try: for i in f: yield i # syntax error finally: f.release() for i in get_items(): print i This code is not allowed because execution might never return after the 'yield' statement and therefore there is no way to ensure that the 'finally' block is executed. A prohibition on such yields lesse
