Re: Lambda alternative?

[Hrvoje Niksic]

Duncan Booth  writes:

> import cPickle as p
> p.dumps([])
>> '(l.'
> p.dumps([].append)
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> TypeError: expected string or Unicode object, NoneType found
>
> Not the best of examples: [].append is a built-in method rather than a 
> bound method.

Sorry about the confusion.  By bound method I meant the callable
obtained by binding a method (built-in or written in python) to a
concrete instance, as opposed to an unbound method, such as
list.append.  The reprs are different for built-in methods and for
python methods, but I consider that difference an artifact of the
CPython implementation, not a distinction mandated by the language.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Hrvoje Niksic]

"J. Cliff Dyer"  writes:

> On Thu, 2009-04-16 at 13:33 +0200, Hrvoje Niksic wrote:
>> mousemeat  writes:
>> 
>> > Correct me if i am wrong, but i can pickle an object that contains a
>> > bound method (it's own bound method).
>> 
>> No, you can't:
>> 
>> >>> import cPickle as p
>> >>> p.dumps([])
>> '(l.'
>> >>> p.dumps([].append)
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> TypeError: expected string or Unicode object, NoneType found
>
> Yes he can.  mousemeat stated that he could pickle an object that
> *contains* a bound method, not that he could pickle the method
> itself.

To pickle an object that contains something, that something still
needs to be picklable.

>>> import cPickle as p
>>> class X(object): pass
...
>>> x = X()
>>> x.foo = [].append
>>> p.dumps(x)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: expected string or Unicode object, NoneType found

> That said, you can make an instance method out of a lambda, just as
> well as any named function, and you can pickle that object, too:
[...]

The object in your example doesn't contain an instance method, its
class contains a function which gets converted to a bound method
if/when it is accessed through the instance.  Pickling the object
doesn't pickle the method at all, because it's left up to the class to
provide it after unpickling.

On the other hand, the OP had a use case where he tried to pickle an
object that actually contained lambdas as part of its data, such as:

class Foo(object):
pass

foo = Foo()
foo.a = lambda x: x+1

Now pickling foo requires pickling the lambda, which doesn't work.  My
point was that this, on its own, is no reason to avoid lambdas in
general, because by that logic one would also avoid bound methods,
which are also unpicklable.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Aaron Brady]

On Apr 17, 1:43 pm, "J. Cliff Dyer"  wrote:
> On Thu, 2009-04-16 at 13:33 +0200, Hrvoje Niksic wrote:
> > mousemeat  writes:
>
> > > Correct me if i am wrong, but i can pickle an object that contains a
> > > bound method (it's own bound method).
>
> > No, you can't:
>
> > >>> import cPickle as p
> > >>> p.dumps([])
> > '(l.'
> > >>> p.dumps([].append)
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > TypeError: expected string or Unicode object, NoneType found
>
> Yes he can.  mousemeat stated that he could pickle an object that
> *contains* a bound method, not that he could pickle the method itself.
>
> That said, you can make an instance method out of a lambda, just as well
> as any named function, and you can pickle that object, too:
snip

'Contains' here is ambiguous.  If the object contains a bound method,
that is, if a bound method is in its dictionary, you can't.

>>> import pickle as p
>>> class A: pass
...
>>> a= A()
>>> class A:
... def f( self ): print( 'f' )
...
>>> a= A()
>>> class B: pass
...
>>> b= B()
>>> b.f= a.f
>>> b.f()
f
>>> p.dumps( b )
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Programs\Python30\lib\pickle.py", line 1329, in dumps
Pickler(f, protocol).dump(obj)
_pickle.PicklingError: Can't pickle : attribute
lookup
builtins.method failed

In this example, 'b' contains a bound method, 'a.f'.  However, for
other definitions of 'contains', such as if 'b' is an instance of a
class that contains methods, you can.  But in that case, the method is
not in 'b.__dict__'.

>>> b.__dict__
{'f': >}

--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[J. Cliff Dyer]

On Thu, 2009-04-16 at 13:33 +0200, Hrvoje Niksic wrote:
> mousemeat  writes:
> 
> > Correct me if i am wrong, but i can pickle an object that contains a
> > bound method (it's own bound method).
> 
> No, you can't:
> 
> >>> import cPickle as p
> >>> p.dumps([])
> '(l.'
> >>> p.dumps([].append)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: expected string or Unicode object, NoneType found

Yes he can.  mousemeat stated that he could pickle an object that
*contains* a bound method, not that he could pickle the method itself.

That said, you can make an instance method out of a lambda, just as well
as any named function, and you can pickle that object, too:

Python 2.5.2 (r252:60911, Oct  5 2008, 19:29:17) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cPickle as p
>>> class Foo(object):
... a = lambda self, x: x+1
>>> foo.a(1)
2
>>> type(foo.a)

>>> p.dumps(foo)
'ccopy_reg\n_reconstructor\np1\n(c__main__\nFoo\np2\nc__builtin__
\nobject\np3\nNtRp4\n.'

Cheers,
Cliff


--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Duncan Booth]

Hrvoje Niksic  wrote:

>> Correct me if i am wrong, but i can pickle an object that contains a
>> bound method (it's own bound method).
> 
> No, you can't:
> 
 import cPickle as p
 p.dumps([])
> '(l.'
 p.dumps([].append)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: expected string or Unicode object, NoneType found

Not the best of examples: [].append is a built-in method rather than a 
bound method. You are correct that you can't pickle a bound method, but the 
actual error message is a bit different:

>>> C().foo
>
>>> [].append

>>> cPickle.dumps(C().foo)
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python26\lib\copy_reg.py", line 70, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects
>>> cPickle.dumps([].append)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: expected string or Unicode object, NoneType found

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Hrvoje Niksic]

mousemeat  writes:

> Correct me if i am wrong, but i can pickle an object that contains a
> bound method (it's own bound method).

No, you can't:

>>> import cPickle as p
>>> p.dumps([])
'(l.'
>>> p.dumps([].append)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: expected string or Unicode object, NoneType found
  
> (I don't mean to be argumentative, i'm trying to understand python's
> syntax a little better.)

Note that this has nothing to do with syntax, but with the
(intentional) limitations of the "pickle" module.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Aaron Brady]

On Apr 16, 5:25 am, mousemeat  wrote:
> On 16 Apr, 10:21, Hrvoje Niksic  wrote:
>
>
>
> > mousemeat  writes:
> > > Thank you for everyone's explanations, help and interest on this
> > > one.  I have reworked my code as described and promised myself not
> > > to use lambdas ever again (i still think they are an elegant idea,
> > > but if they are incompatible with frequently used modules, then the
> > > inelegance of reality quickly overshadows the elegance of the
> > > theory).
>
> > I think you're going too far concluding that lambdas are unusable.
> > lambdas are a problem only when they are stored as data attributes
> > that need to be pickled, but that's far from being the only use case.
> > You can still use them for what they're meant to be used: tiny anonymous
> > function-expressions, typically passed as parameters to functions that
> > expect a callback.
>
> > Avoiding lambdas because they're unpicklable is like avoiding bound
> > methods because they're just as unpicklable.
>
> Correct me if i am wrong, but i can pickle an object that contains a
> bound method (it's own bound method).  I cannot pickle an object that
> contains a lambda function.  Doesn't that make lambda's less
> pickleable?  (I don't mean to be argumentative, i'm trying to
> understand python's syntax a little better.)

When you pickle an instance, you get its data and the name of its
class.  If you don't have the class later, you can't unpickle it.

Pickle is intended for persistence, and the only real kinds of
persistent code are a self-contained library file, and the source,
especially given access to function-external variables, i.e.
closures.  Pickling a function is like taking an excerpt from a binary
executable.  It can only be used in highly limited contexts, and as
such not good for a standard module.

It is most likely you want to transmit the source-- either the entire
module, which you can then import dynamically, or an excerpt from it,
which is known to have no closures, etc., and carefully call 'eval' on
it.

I disagree that Andres Tremols & Peter Cogolo's 'Recipe 7.6. Pickling
Code Objects' is applicable.  It's highly specialized, delicate, and
brittle.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[mousemeat]

On 16 Apr, 10:21, Hrvoje Niksic  wrote:
> mousemeat  writes:
> > Thank you for everyone's explanations, help and interest on this
> > one.  I have reworked my code as described and promised myself not
> > to use lambdas ever again (i still think they are an elegant idea,
> > but if they are incompatible with frequently used modules, then the
> > inelegance of reality quickly overshadows the elegance of the
> > theory).
>
> I think you're going too far concluding that lambdas are unusable.
> lambdas are a problem only when they are stored as data attributes
> that need to be pickled, but that's far from being the only use case.
> You can still use them for what they're meant to be used: tiny anonymous
> function-expressions, typically passed as parameters to functions that
> expect a callback.
>
> Avoiding lambdas because they're unpicklable is like avoiding bound
> methods because they're just as unpicklable.

Correct me if i am wrong, but i can pickle an object that contains a
bound method (it's own bound method).  I cannot pickle an object that
contains a lambda function.  Doesn't that make lambda's less
pickleable?  (I don't mean to be argumentative, i'm trying to
understand python's syntax a little better.)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Hrvoje Niksic]

mousemeat  writes:

> Thank you for everyone's explanations, help and interest on this
> one.  I have reworked my code as described and promised myself not
> to use lambdas ever again (i still think they are an elegant idea,
> but if they are incompatible with frequently used modules, then the
> inelegance of reality quickly overshadows the elegance of the
> theory).

I think you're going too far concluding that lambdas are unusable.
lambdas are a problem only when they are stored as data attributes
that need to be pickled, but that's far from being the only use case.
You can still use them for what they're meant to be used: tiny anonymous
function-expressions, typically passed as parameters to functions that
expect a callback.

Avoiding lambdas because they're unpicklable is like avoiding bound
methods because they're just as unpicklable.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[mousemeat]

On 16 Apr, 09:39, Duncan Booth  wrote:
> Paul Rubin  wrote:
> > Duncan Booth  writes:
> >> dumped = dumps(air)
> >> t = loads(dumped)
> >> print t # works fine
>
> > Hmm, well, that doesn't really pickle the function; it pickles a class
> > instance that records where the class definition was and (on
> > unpickling) imports that module.  Maybe that is sufficient for this
> > purpose.
>
> It pickles a class instance which contains a reference to a function and
> on unpickling it recreates the class instance with a new reference to
> the function of the same name.
>
> That's how pickling works: it records where the class or function is
> defined and creates a new reference to the class or function of the same
> name when you unpickle. The classes and functions have to be defined at
> module scope otherwise it won't be able to reference them. You don't
> ever actually get either a class or function in the pickle so if you
> want you can update the code and the unpickled object gets the new one.
>
> >>> import pickle
> >>> class C(object):
>
>         def __init__(self, fn):
>                 self.fn = fn
>         def callfn(self):
>                 self.fn()
>
> >>> def foo():
>
>         print "Hello I'm foo"
>
> >>> inst = C(foo)
> >>> inst.callfn()
> Hello I'm foo
> >>> p = pickle.dumps(inst)
> >>> def foo():
>
>         print "I'm a new foo"
>
> >>> inst2 = pickle.loads(p)
> >>> inst2.callfn()
> I'm a new foo
> >>> inst.callfn()
>
> Hello I'm foo
>
> --
> Duncan Boothhttp://kupuguy.blogspot.com


Thank you for everyone's explanations, help and interest on this one.
I have reworked my code as described and promised myself not to use
lambdas ever again (i still think they are an elegant idea, but if
they are incompatible with frequently used modules, then the
inelegance of reality quickly overshadows the elegance of the theory).

I think my problem now is that parallel python will want to unpickle
my objects but can't find the modules.  I've asked the pp forum for
insight, but i think i'm going to have to plug away with this one.

Thanks again,

Warren
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Duncan Booth]

Paul Rubin  wrote:

> Duncan Booth  writes:
>> dumped = dumps(air)
>> t = loads(dumped)
>> print t # works fine
> 
> 
> Hmm, well, that doesn't really pickle the function; it pickles a class
> instance that records where the class definition was and (on
> unpickling) imports that module.  Maybe that is sufficient for this
> purpose.

It pickles a class instance which contains a reference to a function and 
on unpickling it recreates the class instance with a new reference to 
the function of the same name.

That's how pickling works: it records where the class or function is 
defined and creates a new reference to the class or function of the same 
name when you unpickle. The classes and functions have to be defined at 
module scope otherwise it won't be able to reference them. You don't 
ever actually get either a class or function in the pickle so if you 
want you can update the code and the unpickled object gets the new one.

>>> import pickle
>>> class C(object):
def __init__(self, fn):
self.fn = fn
def callfn(self):
self.fn()


>>> def foo():
print "Hello I'm foo"


>>> inst = C(foo)
>>> inst.callfn()
Hello I'm foo
>>> p = pickle.dumps(inst)
>>> def foo():
print "I'm a new foo"


>>> inst2 = pickle.loads(p)
>>> inst2.callfn()
I'm a new foo
>>> inst.callfn()
Hello I'm foo

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Paul Rubin]

Duncan Booth  writes:
> dumped = dumps(air)
> t = loads(dumped)
> print t # works fine


Hmm, well, that doesn't really pickle the function; it pickles a class
instance that records where the class definition was and (on
unpickling) imports that module.  Maybe that is sufficient for this
purpose.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Caleb Hattingh]

On Wed, 15 Apr 2009 15:47:05 +0200,  wrote:


But, lambda functions can't be pickled. I would like to pickle my
objects, and i would really like to use parallel python (which
requires pickling).


If you are fixated on using lambda, you could probably use Recipe 7.6:  
Pickling Code Objects in the Python Cookbook, 2nd Edition, or here:


http://www.ubookcase.com/book/Oreilly/Python.Cookbook.2nd.edition/0596007973/pythoncook2-chp-7-sect-6.html

However, it is better to use named functions as all the other posters have  
already mentioned.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Duncan Booth]

mousemeat  wrote:

> Will i then be able to pickle 'rock', and any object that may contain
> 'rock' as one of it's fields?  I'm not sure that i will, but i'll give
> it a shot.
> 
So long as the functions are defined at module level, not nested inside 
some other namespace it will work fine.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Duncan Booth]

Paul Rubin  wrote:

> Duncan Booth  writes:
>> What exactly is your objection to using a named function here?...
>> Surely that would solve your pickle problem *and* give you more
>> meaningful tracebacks in your exceptions?
> 
> I don't think it would help the pickle problem.

Why wouldn't you think that?

class material(object):
def __init__(self,density):
self.density=density

def airdensity(T):
return 10/(287*T)
air=material(airdensity)

steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

from pickle import dumps, loads

dumped = dumps(air)
t = loads(dumped)
print t # works fine

dumped = dumps(steel) # pickle.PicklingError: Can't pickle  at 0x02C78A58>: it's not found as __main__.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[mousemeat]

On 15 Apr, 16:27, Piet van Oostrum  wrote:
> > mousem...@gmail.com (m) wrote:
> >m> Hello all,
> >m> I really like the lambda function for a particular task i have: I am
> >m> writing some simulation software, working with many different
> >m> materials. Each material with many different properties, some are
> >m> temperature dependent, others aren't, some are defined by a function,
> >m> some are a lookup table, some are constant. Here is a simplified
> >m> version:
> >m> class material(object):
> >m>     def __init__(self,density):
> >m>         self.density=density
> >m> airdensity=lambda T:10/(287*T)
> >m> air=material(airdensity)
> >m> steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
> >m> steel=material(steeldensity)
> >m> rockdensity=lambda T:5000
> >m> rock=material(rockdensity)
>
> The following isn't much different (only 6 chars more per def):
>
> def airdensity(T): return 10/(287*T)
> air=material(airdensity)
>
> def steeldensity(T): return interp(T,[0,1000],[7856,7813])
> steel=material(steeldensity)
>
> def rockdensity(T): return 5000
> rock=material(rockdensity)
>
> --
> Piet van Oostrum 
> URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4]
> Private email: p...@vanoostrum.org- Hide quoted text -
>
> - Show quoted text -

Will i then be able to pickle 'rock', and any object that may contain
'rock' as one of it's fields?  I'm not sure that i will, but i'll give
it a shot.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Piet van Oostrum]

> mousem...@gmail.com (m) wrote:

>m> Hello all,
>m> I really like the lambda function for a particular task i have: I am
>m> writing some simulation software, working with many different
>m> materials. Each material with many different properties, some are
>m> temperature dependent, others aren't, some are defined by a function,
>m> some are a lookup table, some are constant. Here is a simplified
>m> version:

>m> class material(object):
>m> def __init__(self,density):
>m> self.density=density

>m> airdensity=lambda T:10/(287*T)
>m> air=material(airdensity)

>m> steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
>m> steel=material(steeldensity)

>m> rockdensity=lambda T:5000
>m> rock=material(rockdensity)

The following isn't much different (only 6 chars more per def):
  
def airdensity(T): return 10/(287*T)
air=material(airdensity)

def steeldensity(T): return interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

def rockdensity(T): return 5000
rock=material(rockdensity)

-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Paul Rubin]

Duncan Booth  writes:
> What exactly is your objection to using a named function here?...
> Surely that would solve your pickle problem *and* give you more meaningful 
> tracebacks in your exceptions?

I don't think it would help the pickle problem.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Duncan Booth]

mousem...@gmail.com wrote:

> class material(object):
> def __init__(self,density):
> self.density=density
> 
> airdensity=lambda T:10/(287*T)
> air=material(airdensity)

What exactly is your objection to using a named function here?

def airdensity(T):
return 10/(287*T)
air=material(airdensity)

Surely that would solve your pickle problem *and* give you more meaningful 
tracebacks in your exceptions?

-- 
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Lambda alternative?

[Paul Rubin]

mousem...@gmail.com writes:
> But, lambda functions can't be pickled. I would like to pickle my
> objects, and i would really like to use parallel python (which
> requires pickling).

Maybe you could use marshal instead of pickle.  Make sure to read the
documentation carefully--there are some pitfalls that can bite you.
--
http://mail.python.org/mailman/listinfo/python-list

Lambda alternative?

[mousemeat]

Hello all,

I really like the lambda function for a particular task i have: I am
writing some simulation software, working with many different
materials. Each material with many different properties, some are
temperature dependent, others aren't, some are defined by a function,
some are a lookup table, some are constant. Here is a simplified
version:

class material(object):
def __init__(self,density):
self.density=density

airdensity=lambda T:10/(287*T)
air=material(airdensity)

steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

rockdensity=lambda T:5000
rock=material(rockdensity)

I really like doing things this way, because i can then define other
properties within the class and i can always use the syntax:
specificgravity_at_273K=stone.specificgravity(273), whether
specificgravity is a function or a parameter. (not sure thats the
right nomenclature, but i think you'll know what i mean.)

But, lambda functions can't be pickled. I would like to pickle my
objects, and i would really like to use parallel python (which
requires pickling).

The best way i can see to do that is to make a new class which
inheirits the 'material' class for each physical material i want to
model, but surely i should be creating instances rather than classes.

Is this going to present some new problems that anyone can forsee? Can
anyone think of a better way to do this?  I keep reading that lambda
functions aren't particularly pythonic and that there are always
alternatives, but they seem very elegant to me.

Thanks for your time,

Warren
--
http://mail.python.org/mailman/listinfo/python-list