Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Well, that was a nice explanation. Thanks once again Kent! Bernard On 5/16/05, Kent Johnson [EMAIL PROTECTED] wrote: Bernard Lebel wrote: Hi Kent, So if I undestand you right, mapping a function with map() when it is a built-in function will/may be faster than a for loop, but if it's a custom function (ie. a def one), it will most likely be slower? I guess I didn't proofread that last mail...what I meant is - Write the code for clarity first - If you find a performance bottleneck then test different solutions to see which is fastest for your actual data and usage. - Mapping a builtin function over a list is relatively fast in Python and it is worth trying if it meets your needs. - Calling a function is relatively expensive in Python so if using map() requires you to define a helper that may wipe out its advantage. - Testing is crucial! Guessing is only good for helping to come up with ideas to test. Here is a program that tests six ways to apply a function to elements of a list. The functions don't create new lists except as side effects since that was the original problem. They use map(), list comprehension and an explicit for loop to apply str() or a Python function returning str() to the elements of a list. (The list already contains strings so the actual function call should be fast.) import timeit data = [str(n) for n in range(100)] def strByMap(): map(str, data) def strByListComp(): [str(n) for n in data] def strByLoop(): for n in data: str(n) def func(x): return str(x) def funcByMap(): map(func, data) def funcByListComp(): [func(n) for n in data] def funcByLoop(): for n in data: func(n) def timeOne(fn): setup = from __main__ import + fn.__name__ stmt = '%s()' % (fn.__name__) t = timeit.Timer(stmt, setup) secs = min(t.repeat(number=1000)) print '%15s %s' % (fn.__name__, secs) for fn in [ strByMap, strByListComp, strByLoop, funcByMap, funcByListComp, funcByLoop ]: timeOne(fn) ### Here are the results on my computer: strByMap 0.0359623918682 strByListComp 0.0581065470611 strByLoop 0.0481150537289 funcByMap 0.0810943849009 funcByListComp 0.0891375859222 funcByLoop 0.0806144356336 So for this test, map is fastest when calling a builtin and explicit looping is fastest when calling a function. With the explicit loop you might be able to inline the function, in which case it would be much faster than either map or list comp. Modifying the program slightly so that each function actually creates a list changes the results dramatically: strByMap 0.0365733633744 strByListComp 0.0579948010152 strByLoop 0.0764722890758 funcByMap 0.0811885309446 funcByListComp 0.0883995032888 funcByLoop 0.10586876265 Now map() is fastest in both cases, though a for loop with inlined code beats map() with an external function. Kent Thanks Bernard On 5/13/05, Kent Johnson [EMAIL PROTECTED] wrote: Bernard Lebel wrote: The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. That may well be correct, at least in the case where the function passed to map is a builtin. Mapping a builtin to over a list is extremely fast. So write the code with a for loop so it is clear. When you identify a performance bottleneck you can try rewriting your loop using map or list comprehension, which is also fast. Until then it is premature optimization. For typical loops over a small number of items I can't imagine you would notice the difference. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
That is very interesting John. Thanks! Bernard On 5/19/05, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Quoting Bernard Lebel [EMAIL PROTECTED]: Well, that was a nice explanation. Thanks once again Kent! There is a nice (not too technical) essay on the running speeds of different looping constructs on python.org: http://www.python.org/doc/essays/list2str.html Just FYI :-) -- John. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Bernard Lebel wrote: Hi Kent, So if I undestand you right, mapping a function with map() when it is a built-in function will/may be faster than a for loop, but if it's a custom function (ie. a def one), it will most likely be slower? I guess I didn't proofread that last mail...what I meant is - Write the code for clarity first - If you find a performance bottleneck then test different solutions to see which is fastest for your actual data and usage. - Mapping a builtin function over a list is relatively fast in Python and it is worth trying if it meets your needs. - Calling a function is relatively expensive in Python so if using map() requires you to define a helper that may wipe out its advantage. - Testing is crucial! Guessing is only good for helping to come up with ideas to test. Here is a program that tests six ways to apply a function to elements of a list. The functions don't create new lists except as side effects since that was the original problem. They use map(), list comprehension and an explicit for loop to apply str() or a Python function returning str() to the elements of a list. (The list already contains strings so the actual function call should be fast.) import timeit data = [str(n) for n in range(100)] def strByMap(): map(str, data) def strByListComp(): [str(n) for n in data] def strByLoop(): for n in data: str(n) def func(x): return str(x) def funcByMap(): map(func, data) def funcByListComp(): [func(n) for n in data] def funcByLoop(): for n in data: func(n) def timeOne(fn): setup = from __main__ import + fn.__name__ stmt = '%s()' % (fn.__name__) t = timeit.Timer(stmt, setup) secs = min(t.repeat(number=1000)) print '%15s %s' % (fn.__name__, secs) for fn in [ strByMap, strByListComp, strByLoop, funcByMap, funcByListComp, funcByLoop ]: timeOne(fn) ### Here are the results on my computer: strByMap 0.0359623918682 strByListComp 0.0581065470611 strByLoop 0.0481150537289 funcByMap 0.0810943849009 funcByListComp 0.0891375859222 funcByLoop 0.0806144356336 So for this test, map is fastest when calling a builtin and explicit looping is fastest when calling a function. With the explicit loop you might be able to inline the function, in which case it would be much faster than either map or list comp. Modifying the program slightly so that each function actually creates a list changes the results dramatically: strByMap 0.0365733633744 strByListComp 0.0579948010152 strByLoop 0.0764722890758 funcByMap 0.0811885309446 funcByListComp 0.0883995032888 funcByLoop 0.10586876265 Now map() is fastest in both cases, though a for loop with inlined code beats map() with an external function. Kent Thanks Bernard On 5/13/05, Kent Johnson [EMAIL PROTECTED] wrote: Bernard Lebel wrote: The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. That may well be correct, at least in the case where the function passed to map is a builtin. Mapping a builtin to over a list is extremely fast. So write the code with a for loop so it is clear. When you identify a performance bottleneck you can try rewriting your loop using map or list comprehension, which is also fast. Until then it is premature optimization. For typical loops over a small number of items I can't imagine you would notice the difference. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Thanks Alan, that clears things up quite well. Bernard On 5/14/05, Alan Gauld [EMAIL PROTECTED] wrote: So if I undestand you right, mapping a function with map() when it is a built-in function will/may be faster than a for loop, but if it's a custom function (ie. a def one), it will most likely be slower? That's right, a builtin function, including map itself will be written in C and so be 'fast'. So we have the trade off between a Python for loop calling a function or a C loop which additionally has to build a list result. But in the builtin case we have C code calling C code which is much faster than Python code calling C code. So for a function written in Python there will be less difference and the bulk of the time is likely to be spent in the function calls but for builtins C to C could give a significant benefit. Which is faster will depend on several other factors including what the function returns and how easily that can be packaged into a list. From my personal experience I wouldn't expect map to be much faster than a for loop, if at all. But as Kent says you can always profile it and test it if you really need speed. The real issue is the map call is more obscure than an explicit loop since map() is intended to build a list not modify an existing list! Alan G. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
now I always used map() to perform a looped call on a function that would change the attribute value, as shown in Mark Lutz David Ascher's Learning Python: # Perform attribute value change on a single instance def iterateInstances( oInstance ): oInstance.value = myValue # Loop over list of instances map( iterateInstances, aListOfInstances ) How bizarre. I'm astonished that Lutz/Ascher even show that as a means of changing an attribute. As Danny said, it's definitely an abuse of map, and there is no advantage whatsoever, as far as I can see. In fact I suspect that it's actually slower than an explicit loop (because of the extra list building). However I was looking into lambdas in hope to eliminate the need to define a function. And a loop also avoids the need for a function, and therefore for a function call, which is another reason the map should be slower. Its completely weird that a text book should even suggest that map be used for this! Alan G Author of the Learn to Program web tutor http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Alan Gauld wrote: now I always used map() to perform a looped call on a function that would change the attribute value, as shown in Mark Lutz David Ascher's Learning Python: # Perform attribute value change on a single instance def iterateInstances( oInstance ): oInstance.value = myValue # Loop over list of instances map( iterateInstances, aListOfInstances ) How bizarre. I'm astonished that Lutz/Ascher even show that as a means of changing an attribute. As Danny said, it's definitely an abuse of map, and there is no advantage whatsoever, as far as I can see. In fact I suspect that it's actually slower than an explicit loop (because of the extra list building). I have just been looking at http://www.python.org/moin/PythonSpeed/PerformanceTips#loops and whilst the examples do not show anything like this, they describe the map function as a for moved into C code. Could be confusing. Does not mention anything about return a list. However I was looking into lambdas in hope to eliminate the need to define a function. And a loop also avoids the need for a function, and therefore for a function call, which is another reason the map should be slower. Its completely weird that a text book should even suggest that map be used for this! Alan G Author of the Learn to Program web tutor http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- Joe Healy | Engineer OMC-International | 6 Paterson St | Abbotsford, VIC 3067 Melbourne | Australia www.omc-international.com.au Dedicated to safer and more efficient shipping. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. Cheers Bernard On 5/13/05, Alan Gauld [EMAIL PROTECTED] wrote: How bizarre. I'm astonished that Lutz/Ascher even show that as a means of changing an attribute. As Danny said, it's definitely an abuse of map, and there is no advantage whatsoever, as far as I can see. In fact I suspect that it's actually slower than an explicit loop (because of the extra list building). However I was looking into lambdas in hope to eliminate the need to define a function. And a loop also avoids the need for a function, and therefore for a function call, which is another reason the map should be slower. Its completely weird that a text book should even suggest that map be used for this! Alan G Author of the Learn to Program web tutor http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Bernard Lebel wrote: The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. That may well be correct, at least in the case where the function passed to map is a builtin. Mapping a builtin to over a list is extremely fast. So write the code with a for loop so it is clear. When you identify a performance bottleneck you can try rewriting your loop using map or list comprehension, which is also fast. Until then it is premature optimization. For typical loops over a small number of items I can't imagine you would notice the difference. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] map() and lambda to change class instance attribute (fwd)
Hi Kent, So if I undestand you right, mapping a function with map() when it is a built-in function will/may be faster than a for loop, but if it's a custom function (ie. a def one), it will most likely be slower? Thanks Bernard On 5/13/05, Kent Johnson [EMAIL PROTECTED] wrote: Bernard Lebel wrote: The authors even go as far as saysing, on page 228 (first paragraph) that map() used that way has a performance benefit and is faster than a for loop. That may well be correct, at least in the case where the function passed to map is a builtin. Mapping a builtin to over a list is extremely fast. So write the code with a for loop so it is clear. When you identify a performance bottleneck you can try rewriting your loop using map or list comprehension, which is also fast. Until then it is premature optimization. For typical loops over a small number of items I can't imagine you would notice the difference. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
