Re: [Tutor] map() and lambda to change class instance attribute (fwd)
That is very interesting John. Thanks! Bernard On 5/19/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Quoting Bernard Lebel <[EMAIL PROTECTED]>: > > > Well, that was a nice explanation. Thanks once again Kent! > > There is a nice (not too technical) essay on the running speeds of different > looping constructs on python.org: > > http://www.python.org/doc/essays/list2str.html > > Just FYI :-) > > -- > John. > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Well, that was a nice explanation. Thanks once again Kent! Bernard On 5/16/05, Kent Johnson <[EMAIL PROTECTED]> wrote: > Bernard Lebel wrote: > > Hi Kent, > > > > So if I undestand you right, mapping a function with map() when it is > > a built-in function will/may be faster than a for loop, but if it's a > > custom function (ie. a def one), it will most likely be slower? > > I guess I didn't proofread that last mail...what I meant is > > - Write the code for clarity first > - If you find a performance bottleneck then test different solutions to see > which is fastest for > your actual data and usage. > - Mapping a builtin function over a list is relatively fast in Python and it > is worth trying if it > meets your needs. > - Calling a function is relatively expensive in Python so if using map() > requires you to define a > helper that may wipe out its advantage. > - Testing is crucial! Guessing is only good for helping to come up with ideas > to test. > > Here is a program that tests six ways to apply a function to elements of a > list. The functions don't > create new lists except as side effects since that was the original problem. > They use map(), list > comprehension and an explicit for loop to apply str() or a Python function > returning str() to the > elements of a list. (The list already contains strings so the actual function > call should be fast.) > > > > import timeit > > data = [str(n) for n in range(100)] > > def strByMap(): > map(str, data) > > def strByListComp(): > [str(n) for n in data] > > def strByLoop(): > for n in data: > str(n) > > > def func(x): > return str(x) > > > def funcByMap(): > map(func, data) > > def funcByListComp(): > [func(n) for n in data] > > def funcByLoop(): > for n in data: > func(n) > > > def timeOne(fn): > setup = "from __main__ import " + fn.__name__ > stmt = '%s()' % (fn.__name__) > > t = timeit.Timer(stmt, setup) > secs = min(t.repeat(number=1000)) > print '%15s %s' % (fn.__name__, secs) > > for fn in [ strByMap, strByListComp, strByLoop, funcByMap, funcByListComp, > funcByLoop ]: > timeOne(fn) > > ### > > Here are the results on my computer: > > strByMap 0.0359623918682 >strByListComp 0.0581065470611 >strByLoop 0.0481150537289 >funcByMap 0.0810943849009 > funcByListComp 0.0891375859222 > funcByLoop 0.0806144356336 > > So for this test, map is fastest when calling a builtin and explicit looping > is fastest when calling > a function. With the explicit loop you might be able to inline the function, > in which case it would > be much faster than either map or list comp. > > Modifying the program slightly so that each function actually creates a list > changes the results > dramatically: > strByMap 0.0365733633744 >strByListComp 0.0579948010152 >strByLoop 0.0764722890758 >funcByMap 0.0811885309446 > funcByListComp 0.0883995032888 > funcByLoop 0.10586876265 > > Now map() is fastest in both cases, though a for loop with inlined code beats > map() with an external > function. > > Kent > > > > > > Thanks > > Bernard > > > > > > On 5/13/05, Kent Johnson <[EMAIL PROTECTED]> wrote: > > > >>Bernard Lebel wrote: > >> > >>>The authors even go as far as saysing, on page 228 (first paragraph) > >>>that map() used that way has a performance benefit and is faster than > >>>a for loop. > >> > >>That may well be correct, at least in the case where the function passed to > >>map is a builtin. > >>Mapping a builtin to over a list is extremely fast. So write the code with > >>a for loop so it is > >>clear. When you identify a performance bottleneck you can try rewriting > >>your loop using map or list > >>comprehension, which is also fast. Until then it is premature optimization. > >>For typical loops over a > >>small number of items I can't imagine you would notice the difference. > >> > >>Kent > >> > >>___ > >>Tutor maillist - Tutor@python.org > >>http://mail.python.org/mailman/listinfo/tutor > >> > > > > ___ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Bernard Lebel wrote: > Hi Kent, > > So if I undestand you right, mapping a function with map() when it is > a built-in function will/may be faster than a for loop, but if it's a > custom function (ie. a def one), it will most likely be slower? I guess I didn't proofread that last mail...what I meant is - Write the code for clarity first - If you find a performance bottleneck then test different solutions to see which is fastest for your actual data and usage. - Mapping a builtin function over a list is relatively fast in Python and it is worth trying if it meets your needs. - Calling a function is relatively expensive in Python so if using map() requires you to define a helper that may wipe out its advantage. - Testing is crucial! Guessing is only good for helping to come up with ideas to test. Here is a program that tests six ways to apply a function to elements of a list. The functions don't create new lists except as side effects since that was the original problem. They use map(), list comprehension and an explicit for loop to apply str() or a Python function returning str() to the elements of a list. (The list already contains strings so the actual function call should be fast.) import timeit data = [str(n) for n in range(100)] def strByMap(): map(str, data) def strByListComp(): [str(n) for n in data] def strByLoop(): for n in data: str(n) def func(x): return str(x) def funcByMap(): map(func, data) def funcByListComp(): [func(n) for n in data] def funcByLoop(): for n in data: func(n) def timeOne(fn): setup = "from __main__ import " + fn.__name__ stmt = '%s()' % (fn.__name__) t = timeit.Timer(stmt, setup) secs = min(t.repeat(number=1000)) print '%15s %s' % (fn.__name__, secs) for fn in [ strByMap, strByListComp, strByLoop, funcByMap, funcByListComp, funcByLoop ]: timeOne(fn) ### Here are the results on my computer: strByMap 0.0359623918682 strByListComp 0.0581065470611 strByLoop 0.0481150537289 funcByMap 0.0810943849009 funcByListComp 0.0891375859222 funcByLoop 0.0806144356336 So for this test, map is fastest when calling a builtin and explicit looping is fastest when calling a function. With the explicit loop you might be able to inline the function, in which case it would be much faster than either map or list comp. Modifying the program slightly so that each function actually creates a list changes the results dramatically: strByMap 0.0365733633744 strByListComp 0.0579948010152 strByLoop 0.0764722890758 funcByMap 0.0811885309446 funcByListComp 0.0883995032888 funcByLoop 0.10586876265 Now map() is fastest in both cases, though a for loop with inlined code beats map() with an external function. Kent > > > Thanks > Bernard > > > On 5/13/05, Kent Johnson <[EMAIL PROTECTED]> wrote: > >>Bernard Lebel wrote: >> >>>The authors even go as far as saysing, on page 228 (first paragraph) >>>that map() used that way has a performance benefit and is faster than >>>a for loop. >> >>That may well be correct, at least in the case where the function passed to >>map is a builtin. >>Mapping a builtin to over a list is extremely fast. So write the code with a >>for loop so it is >>clear. When you identify a performance bottleneck you can try rewriting your >>loop using map or list >>comprehension, which is also fast. Until then it is premature optimization. >>For typical loops over a >>small number of items I can't imagine you would notice the difference. >> >>Kent >> >>___ >>Tutor maillist - Tutor@python.org >>http://mail.python.org/mailman/listinfo/tutor >> > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Thanks Alan, that clears things up quite well. Bernard On 5/14/05, Alan Gauld <[EMAIL PROTECTED]> wrote: > > So if I undestand you right, mapping a function with map() > > when it is a built-in function will/may be faster than a for > > loop, but if it's a custom function (ie. a def one), it will > > most likely be slower? > > That's right, a builtin function, including map itself will > be written in C and so be 'fast'. So we have the trade off > between a Python for loop calling a function or a C loop > which additionally has to build a list result. But in the > builtin case we have C code calling C code which is > much faster than Python code calling C code. > > So for a function written in Python there will be less > difference and the bulk of the time is likely to be > spent in the function calls but for builtins C to C > could give a significant benefit. > > Which is faster will depend on several other factors including > what the function returns and how easily that can be packaged > into a list. > > From my personal experience I wouldn't expect map to be > much faster than a for loop, if at all. But as Kent says > you can always profile it and test it if you really need > speed. The real issue is the map call is more obscure > than an explicit loop since map() is intended to build > a list not modify an existing list! > > Alan G. > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Hi Kent, So if I undestand you right, mapping a function with map() when it is a built-in function will/may be faster than a for loop, but if it's a custom function (ie. a def one), it will most likely be slower? Thanks Bernard On 5/13/05, Kent Johnson <[EMAIL PROTECTED]> wrote: > Bernard Lebel wrote: > > The authors even go as far as saysing, on page 228 (first paragraph) > > that map() used that way has a performance benefit and is faster than > > a for loop. > > That may well be correct, at least in the case where the function passed to > map is a builtin. > Mapping a builtin to over a list is extremely fast. So write the code with a > for loop so it is > clear. When you identify a performance bottleneck you can try rewriting your > loop using map or list > comprehension, which is also fast. Until then it is premature optimization. > For typical loops over a > small number of items I can't imagine you would notice the difference. > > Kent > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Bernard Lebel wrote: > The authors even go as far as saysing, on page 228 (first paragraph) > that map() used that way has a performance benefit and is faster than > a for loop. That may well be correct, at least in the case where the function passed to map is a builtin. Mapping a builtin to over a list is extremely fast. So write the code with a for loop so it is clear. When you identify a performance bottleneck you can try rewriting your loop using map or list comprehension, which is also fast. Until then it is premature optimization. For typical loops over a small number of items I can't imagine you would notice the difference. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. Cheers Bernard On 5/13/05, Alan Gauld <[EMAIL PROTECTED]> wrote: > How bizarre. I'm astonished that Lutz/Ascher even show that as a means > of changing an attribute. As Danny said, it's definitely an abuse of > map, > and there is no advantage whatsoever, as far as I can see. In fact I > suspect that it's actually slower than an explicit loop (because of > the extra list building). > > > However I was looking into lambdas in hope to eliminate the need to > > define a function. > > And a loop also avoids the need for a function, and therefore for > a function call, which is another reason the map should be slower. > > Its completely weird that a text book should even suggest that map > be used for this! > > Alan G > Author of the Learn to Program web tutor > http://www.freenetpages.co.uk/hp/alan.gauld > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Alan Gauld wrote: >>now I always used map() to perform a looped call on a function that >>would change the attribute value, as shown in Mark Lutz & David >>Ascher's Learning Python: >> >># Perform attribute value change on a single instance >>def iterateInstances( oInstance ): >> oInstance.value = myValue >> >># Loop over list of instances >>map( iterateInstances, aListOfInstances ) >> >> > >How bizarre. I'm astonished that Lutz/Ascher even show that as a means >of changing an attribute. As Danny said, it's definitely an abuse of >map, >and there is no advantage whatsoever, as far as I can see. In fact I >suspect that it's actually slower than an explicit loop (because of >the extra list building). > > I have just been looking at http://www.python.org/moin/PythonSpeed/PerformanceTips#loops and whilst the examples do not show anything like this, they describe the map function as a for moved into C code. Could be confusing. Does not mention anything about return a list. > > >>However I was looking into lambdas in hope to eliminate the need to >>define a function. >> >> > >And a loop also avoids the need for a function, and therefore for >a function call, which is another reason the map should be slower. > >Its completely weird that a text book should even suggest that map >be used for this! > >Alan G >Author of the Learn to Program web tutor >http://www.freenetpages.co.uk/hp/alan.gauld > >___ >Tutor maillist - Tutor@python.org >http://mail.python.org/mailman/listinfo/tutor > > -- Joe Healy | Engineer OMC-International | 6 Paterson St | Abbotsford, VIC 3067 Melbourne | Australia www.omc-international.com.au Dedicated to safer and more efficient shipping. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
> now I always used map() to perform a looped call on a function that > would change the attribute value, as shown in Mark Lutz & David > Ascher's Learning Python: > > # Perform attribute value change on a single instance > def iterateInstances( oInstance ): > oInstance.value = myValue > > # Loop over list of instances > map( iterateInstances, aListOfInstances ) How bizarre. I'm astonished that Lutz/Ascher even show that as a means of changing an attribute. As Danny said, it's definitely an abuse of map, and there is no advantage whatsoever, as far as I can see. In fact I suspect that it's actually slower than an explicit loop (because of the extra list building). > However I was looking into lambdas in hope to eliminate the need to > define a function. And a loop also avoids the need for a function, and therefore for a function call, which is another reason the map should be slower. Its completely weird that a text book should even suggest that map be used for this! Alan G Author of the Learn to Program web tutor http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
-- Forwarded message -- Date: Wed, 11 May 2005 14:29:58 -0400 From: Bernard Lebel <[EMAIL PROTECTED]> To: Danny Yoo <[EMAIL PROTECTED]> Subject: Re: [Tutor] map() and lambda to change class instance attribute Hi Danny, Thanks for the answer. I have to confess that I already use map(), or should I say abuse, for this, although it is the first time I consider using lambdas. Up until now I always used map() to perform a looped call on a function that would change the attribute value, as shown in Mark Lutz & David Ascher's Learning Python: # Perform attribute value change on a single instance def iterateInstances( oInstance ): oInstance.value = myValue # Loop over list of instances map( iterateInstances, aListOfInstances ) However I was looking into lambdas in hope to eliminate the need to define a function. Cheers Bernard On 5/11/05, Danny Yoo <[EMAIL PROTECTED]> wrote: > > > On Wed, 11 May 2005, Bernard Lebel wrote: > > > Let say I have several class instances in a list, and these class > > instances have an attribute named "value", whose value is an integer. > > > > I would like to know if it is possible to loop over the list of > > instances to change their "value" attribute, using a map( ( lambda:...), > > ... ) type of loop. > > Hi Bernard, > > Hmmm... then map() is probably not what you want. map() is meant to > transform a list of things into another list of things, but isn't really > meant to mutate its input. > > It is possible to abuse map() to do what you're trying to do, using the > setattr() function: > > ## Pseudocode > map(lambda instance: setattr(instance, 'value', 42)) > ## > > but this is definitely language abuse. *grin* > > map() is not intended to be run just to affect changing assignments on its > input. It'll probably be clearest in Python just to write out the loop > explicitely. > > Best of wishes to you! > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
