Re: list comprehension question
On Oct 17, 10:22 am, Terry Reedy tjre...@udel.edu wrote: On 10/16/2012 9:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. list comprehensions specifically abbreviate the code that they are (essentially) equivalent to. res = [] for item in source: res.append(f(item)) res == [f(item) for item in source] Matrix multiplication does not fit the pattern above. The reduction is number addition rather than list appending. Dunno why you say that. Heres matrix multiply using list comprehensions: from operator import add def dot(p,q): return reduce(add, (x*y for x,y in zip(p,q))) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] which can then be 'reduced' to a one-liner if that takes your fancy -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 17/10/12 09:13:57, rusi wrote: On Oct 17, 10:22 am, Terry Reedy tjre...@udel.edu wrote: On 10/16/2012 9:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. list comprehensions specifically abbreviate the code that they are (essentially) equivalent to. res = [] for item in source: res.append(f(item)) res == [f(item) for item in source] Matrix multiplication does not fit the pattern above. The reduction is number addition rather than list appending. Dunno why you say that. Heres matrix multiply using list comprehensions: from operator import add def dot(p,q): return reduce(add, (x*y for x,y in zip(p,q))) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] which can then be 'reduced' to a one-liner if that takes your fancy I can golf it down to two lines without losing readability: def dot(p,q): return sum(x*y for x,y in zip(p,q)) def mm(a,b): return [[dot(ra, rb) for rb in zip(*b)] for ra in a] Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 10/17/2012 12:43 AM, Kevin Anthony wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. (Please don't top-post; it ruins the ordering. In these forums, put your response after the part you quote from earlier messages. Or even better, after each part you quote. Then trim off the parts you didn't reference.) list comprehensions CAN be much faster, but not necessarily. The most complex a loop, the less likely it'll help much. In any case, only the inner loop will be affected. Nesting two list comprehensions will make a trivial difference. On the other hand, Hans Mulder shows some other factoring which seems much more readable than yours. Studying (and testing) those could teach you a lot about comprehensions, as well as about the libraries that can help. Note especially what zip(*b) yields, and think about what it means. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Oct 17, 5:33 pm, Dave Angel d...@davea.name wrote: On 10/17/2012 12:43 AM, Kevin Anthony wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. list comprehensions CAN be much faster, but not necessarily. The most complex a loop, the less likely it'll help much. One-lining the comprehension seems to make a difference of about 10% out here. Maybe Ive missed something? Seems too large… # My original suggestion def dot(p,q): return sum (x*y for x,y in zip(p,q)) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] # One-liner (Thanks Hans for reminding me of sum) def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] t1=Timer(res=mm1(m,m), setup=from __main__ import mm1, m) t1.timeit(1000) 12.276363849639893 t0=Timer(res=mm(m,m), setup=from __main__ import mm, m) t0.timeit(1000) 13.453603029251099 -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Oct 17, 7:06 pm, rusi rustompm...@gmail.com wrote: On Oct 17, 5:33 pm, Dave Angel d...@davea.name wrote: On 10/17/2012 12:43 AM, Kevin Anthony wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. list comprehensions CAN be much faster, but not necessarily. The most complex a loop, the less likely it'll help much. One-lining the comprehension seems to make a difference of about 10% out here. Maybe Ive missed something? Seems too large… # My original suggestion def dot(p,q): return sum (x*y for x,y in zip(p,q)) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] # One-liner (Thanks Hans for reminding me of sum) def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] t1=Timer(res=mm1(m,m), setup=from __main__ import mm1, m) t1.timeit(1000) 12.276363849639893 t0=Timer(res=mm(m,m), setup=from __main__ import mm, m) t0.timeit(1000) 13.453603029251099 In case anyone wants to try out with the same data, I used: m = [range(i,i+30) for i in range(30)] -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 10/17/2012 10:06 AM, rusi wrote: On Oct 17, 5:33 pm, Dave Angel d...@davea.name wrote: On 10/17/2012 12:43 AM, Kevin Anthony wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. list comprehensions CAN be much faster, but not necessarily. The most complex a loop, the less likely it'll help much. One-lining the comprehension seems to make a difference of about 10% out here. Maybe Ive missed something? Seems too large… # My original suggestion def dot(p,q): return sum (x*y for x,y in zip(p,q)) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] # One-liner (Thanks Hans for reminding me of sum) def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] t1=Timer(res=mm1(m,m), setup=from __main__ import mm1, m) t1.timeit(1000) 12.276363849639893 t0=Timer(res=mm(m,m), setup=from __main__ import mm, m) t0.timeit(1000) 13.453603029251099 And I'd wager all the improvement is in the inner loop, the dot() function. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Dave Angel於 2012年10月17日星期三UTC+8下午10時37分01秒寫道： On 10/17/2012 10:06 AM, rusi wrote: On Oct 17, 5:33 pm, Dave Angel d...@davea.name wrote: On 10/17/2012 12:43 AM, Kevin Anthony wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. list comprehensions CAN be much faster, but not necessarily. The most complex a loop, the less likely it'll help much. One-lining the comprehension seems to make a difference of about 10% out here. Maybe Ive missed something? Seems too large� # My original suggestion def dot(p,q): return sum (x*y for x,y in zip(p,q)) def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] # One-liner (Thanks Hans for reminding me of sum) def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] t1=Timer(res=mm1(m,m), setup=from __main__ import mm1, m) t1.timeit(1000) 12.276363849639893 t0=Timer(res=mm(m,m), setup=from __main__ import mm, m) t0.timeit(1000) 13.453603029251099 And I'd wager all the improvement is in the inner loop, the dot() function. -- DaveA Thanks for the tips of matrix operations over some fields or rings other than the real field and the complex field. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Oct 17, 7:37 pm, Dave Angel d...@davea.name wrote: And I'd wager all the improvement is in the inner loop, the dot() function. Sorry -- red herring! Changing def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in zip(*b)] for ra in a] to def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] makes the speed diff vanish -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
rusi於 2012年10月17日星期三UTC+8下午10時50分11秒寫道： On Oct 17, 7:37 pm, Dave Angel d...@davea.name wrote: And I'd wager all the improvement is in the inner loop, the dot() function. Sorry -- red herring! Changing def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in zip(*b)] for ra in a] to def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in zip(*b)] for ra in a] makes the speed diff vanish I think a lot people don't work on computations over fields other real and complex. That is why a lot people keep complaining about the speeds of python programs executed. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 10/17/2012 3:13 AM, rusi wrote: On Oct 17, 10:22 am, Terry Reedy tjre...@udel.edu wrote: On 10/16/2012 9:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. My response is to the part Kevin could *not* convert, not the parts he did convert. I attempted to explain why he could not convert that part. list comprehensions specifically abbreviate the code that they are (essentially) equivalent to. res = [] for item in source: res.append(f(item)) res == [f(item) for item in source] Matrix multiplication does not fit the pattern above. The reduction is number addition rather than list appending. Dunno why you say that. Because it is true and because it makes an essential point about what one can and cannot sensibly do with comprehensions. They are not intended to be a replacement for *all* loops. The essential inner reduction by addition of products that Kevin was 'not able to convert' cannot be converted (with out some obnoxious trickery and *some* extra helper), so his request for a sensible conversion is futile. Heres matrix multiply using list comprehensions: plus a helper function that does the inner reduction otherwise, as I implied it should be from operator import add def dot(p,q): return reduce(add, (x*y for x,y in zip(p,q))) Right, this is the addition reduction that the OP was trying to convert to a list comp. It cannot be done and you have not done it either. Note the the vector of products is produced as a comprehension. That you left it as a 'generator expression' is not relevant. The important point is the the addition combines the products of different iterations and list comps, by their nature, cannot directly do that. def transpose(m): return zip(*m) def mm(a,b): return mmt(a, transpose(b)) def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a] This is the repeated append part of the original nested loops and that, as I said, can be re-expressed as a list comp. But that was not the part Kevin was having difficulty with and not the part I was talking about. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 17 October 2012 06:09, Dwight Hutto dwightdhu...@gmail.com wrote: On Wed, Oct 17, 2012 at 12:43 AM, Kevin Anthony kevin.s.anth...@gmail.com wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. I thought it was matrix multiplication mixed with list comprehension. Check this out real quick from the docs for list comprehension, but if it's a mixture of matrix multi, and list comp, then reply back. http://docs.python.org/tutorial/datastructures.html#list-comprehensions If you're looking for a fast solution then you should follow David's suggestion and use numpy: http://www.scipy.org/FAQ#head-045cba7978d75ec54882150fa6ad308e325a www.python-course.eu/matrix_arithmetic.php The most relevant numpy function is numpy.dot: http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html Oscar P.S. David, it was a bit fiddly for me to quote your response because the relevant part was in a post that had no context. Also your name is listed as Dwight above. That is automatically added by gmail because of the settings in your own email client. You may need to change your account settings if you want it to say David when people quote you. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 10/16/2012 09:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. here's the original code for matrix multiplcation retmatrix = Matrix(self.__row,other.__col) for m in range(0,retmatrix.__row): for n in range(0,retmatrix.__col): product = 0 for p in range(1,self.__col+1): product += (self.__matrix[m][p] * other.__matrix[p][n]) retmatrix.__matrix[m][n] = product Here is what i have so far: retmatrix.__matrix = [[ product = product + (self.__matrix[m][p]* other.__matrix[p][n]) if product else self.__matrix[m][p]* other.__matrix[p][n]) for p in range(0,self.col) for n in range(0,self.col)] for m in range(0,self.__row)] But i know that isn't correct, can someone nudge my in the right direction? The biggest thing to learn about list comprehensions is when not to use them. I can't imagine how your latter version (even if correct) is clearer than the first. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Tue, Oct 16, 2012 at 10:13 PM, Dwight Hutto dwightdhu...@gmail.com wrote: On Tue, Oct 16, 2012 at 9:54 PM, Kevin Anthony kevin.s.anth...@gmail.com wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. here's the original code for matrix multiplcation retmatrix = Matrix(self.__row,other.__col) for m in range(0,retmatrix.__row): for n in range(0,retmatrix.__col): product = 0 for p in range(1,self.__col+1): product += (self.__matrix[m][p] * other.__matrix[p][n]) retmatrix.__matrix[m][n] = product Here is what i have so far: retmatrix.__matrix = [[ product = product + (self.__matrix[m][p]* other.__matrix[p][n]) if product else self.__matrix[m][p]* other.__matrix[p][n]) for p in range(0,self.col) for n in range(0,self.col)] for m in range(0,self.__row)] But i know that isn't correct, can someone nudge my in the right direction? -- Thanks Kevin Anthony www.NoSideRacing.com Do you use Banshee? Download the Community Extensions: http://banshee.fm/download/extensions/ -- http://mail.python.org/mailman/listinfo/python-list I did this a little while back for something else, but memories get jumbled up in these molecules of data I have floating around in my mind, but maybe this will put you on the right track. -- I know this doesn't answer the python question, but it will help you algorithm out what you need to know. I do lots of interdisciplinary research, so if this doesn't help, let me know, and I'll take a refresher, and work up some code. Also, look at numpy. -- Best Regards, David Hutto CEO: http://www.hitwebdevelopment.com -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Tue, Oct 16, 2012 at 10:13 PM, Dave Angel d...@davea.name wrote: On 10/16/2012 09:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. here's the original code for matrix multiplcation retmatrix = Matrix(self.__row,other.__col) for m in range(0,retmatrix.__row): for n in range(0,retmatrix.__col): product = 0 for p in range(1,self.__col+1): product += (self.__matrix[m][p] * other.__matrix[p][n]) retmatrix.__matrix[m][n] = product Here is what i have so far: retmatrix.__matrix = [[ product = product + (self.__matrix[m][p]* other.__matrix[p][n]) if product else self.__matrix[m][p]* other.__matrix[p][n]) for p in range(0,self.col) for n in range(0,self.col)] for m in range(0,self.__row)] But i know that isn't correct, can someone nudge my in the right direction? The biggest thing to learn about list comprehensions is when not to use them. I can't imagine how your latter version (even if correct) is clearer than the first. I think he might be using the wrong function for a matrix multiplication, not that it's not workable, but there are other libraries like numpy that could help out. I wouldn't use list comprehension for this, unless it might be several lists that interact. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list -- Best Regards, David Hutto CEO: http://www.hitwebdevelopment.com -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Oct 17, 7:14 am, Dave Angel d...@davea.name wrote: On 10/16/2012 09:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. here's the original code for matrix multiplcation retmatrix = Matrix(self.__row,other.__col) for m in range(0,retmatrix.__row): for n in range(0,retmatrix.__col): product = 0 for p in range(1,self.__col+1): product += (self.__matrix[m][p] * other.__matrix[p][n]) retmatrix.__matrix[m][n] = product Here is what i have so far: retmatrix.__matrix = [[ product = product + (self.__matrix[m][p]* other.__matrix[p][n]) if product else self.__matrix[m][p]* other.__matrix[p][n]) for p in range(0,self.col) for n in range(0,self.col)] for m in range(0,self.__row)] But i know that isn't correct, can someone nudge my in the right direction? The biggest thing to learn about list comprehensions is when not to use them. I can't imagine how your latter version (even if correct) is clearer than the first. -- DaveA Try rewriting using dot from operator import mul def dot(p,q): return reduce(mul, [x*y for x,y in zip(p,q)]) # the [] can become () and avoiding object-orientation (at least to start with) -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. Thanks Kevin On Oct 16, 2012 10:14 PM, Dave Angel d...@davea.name wrote: On 10/16/2012 09:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. here's the original code for matrix multiplcation retmatrix = Matrix(self.__row,other.__col) for m in range(0,retmatrix.__row): for n in range(0,retmatrix.__col): product = 0 for p in range(1,self.__col+1): product += (self.__matrix[m][p] * other.__matrix[p][n]) retmatrix.__matrix[m][n] = product Here is what i have so far: retmatrix.__matrix = [[ product = product + (self.__matrix[m][p]* other.__matrix[p][n]) if product else self.__matrix[m][p]* other.__matrix[p][n]) for p in range(0,self.col) for n in range(0,self.col)] for m in range(0,self.__row)] But i know that isn't correct, can someone nudge my in the right direction? The biggest thing to learn about list comprehensions is when not to use them. I can't imagine how your latter version (even if correct) is clearer than the first. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Wed, Oct 17, 2012 at 12:43 AM, Kevin Anthony kevin.s.anth...@gmail.com wrote: Is it not true that list comprehension is much faster the the for loops? If it is not the correct way of doing this, i appoligize. Like i said, I'm learing list comprehension. I thought it was matrix multiplication mixed with list comprehension. Check this out real quick from the docs for list comprehension, but if it's a mixture of matrix multi, and list comp, then reply back. http://docs.python.org/tutorial/datastructures.html#list-comprehensions -- Best Regards, David Hutto CEO: http://www.hitwebdevelopment.com -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 10/16/2012 9:54 PM, Kevin Anthony wrote: I've been teaching myself list comprehension, and i've run across something i'm not able to convert. list comprehensions specifically abbreviate the code that they are (essentially) equivalent to. res = [] for item in source: res.append(f(item)) res == [f(item) for item in source] Matrix multiplication does not fit the pattern above. The reduction is number addition rather than list appending. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 29 February 2012 13:52, Johann Spies johann.sp...@gmail.com wrote: In [82]: t.append(instansie) t.append(instansie) In [83]: t t Out[83]: ['Mangosuthu Technikon'] In [84]: t = [x.alt_name for x in lys].append(instansie) t = [x.alt_name for x in lys].append(instansie) In [85]: t t In [86]: type t type t --- type(t) Out[86]: NoneType You should note that in [82], you're not doing: t = t.append(instansie) hth -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
In mailman.298.1330534919.3037.python-l...@python.org James Broadhead jamesbroadh...@gmail.com writes: On 29 February 2012 13:52, Johann Spies johann.sp...@gmail.com wrote: In [82]: t.append(instansie) t.append(instansie) In [83]: t t Out[83]: ['Mangosuthu Technikon'] In [84]: t = [x.alt_name for x in lys].append(instansie) t = [x.alt_name for x in lys].append(instansie) In [85]: t t In [86]: type t type t --- type(t) Out[86]: NoneType You should note that in [82], you're not doing: t = t.append(instansie) append() modifies the list in-place. It doesn't return anything. (and therefore its return value is None) x = [1, 2, 3] y = x.append(4) print x [1, 2, 3, 4] print y None -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Wed, Feb 29, 2012 at 5:52 AM, Johann Spies johann.sp...@gmail.com wrote: I understand the following: In [79]: instansie instansie Out[79]: 'Mangosuthu Technikon' In [80]: t = [x.alt_name for x in lys] t = [x.alt_name for x in lys] In [81]: t t Out[81]: [] In [82]: t.append(instansie) t.append(instansie) Note the lack of an accompanying Out. This means that the expression, i.e. `t.append(instansie)`, had a result of None. The list.append() method does *not* return the now-appended-to list. It is a mutator method that modifies the list object in-place; per convention, it therefore returns None to reinforce its side-effecting nature to the user; analogous methods in other languages return void. In [83]: t t Out[83]: ['Mangosuthu Technikon'] But then why does the following behave like this: In [84]: t = [x.alt_name for x in lys].append(instansie) t = [x.alt_name for x in lys].append(instansie) You didn't do anything with .append()'s useless return value before; here, you do, by assigning it to t. In [85]: t t In [86]: type t type t --- type(t) Out[86]: NoneType Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 2/29/2012 8:52 AM, Johann Spies wrote: Please post plain text, the standard for all python.org mailing lists and corresponding newsgroups, and not html. Some readers print the html as plain text, which is confusing and obnoxious. Other like mine, do skip the plain text version and print the rendered html, but in this case, the strong background colors make your post impossible for *me* to read. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Thu, 07 May 2009 13:28:10 -0400, J Kenneth King wrote: Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. They are tricky and need to be used carefully, *if at all*. IMO this means that if there's a way to do it without a nested list comprehension, then that solution should be preferred. Earlier, you claimed that list comps in general were discouraged: List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Ooops. Typo. My bad. Had it been quoted earlier it would have saved a few posts for sure. Such is the Internet. :) Emile said I'd be surprised to find routine usage frowned upon, giving you the opportunity to correct his (mis)understanding. You failed to do so, which I took as meaning that you agreed that routine usage of simple list comps were frowned upon. Now you're talking about *nested* list comps. You started off (apparently) objecting to list comps in general, because they can make readers apprehensive. Now you seem to be saying that it's only the complicated, overly-dense ones which rightly make readers apprehensive which you object to. I was rather confused by that. I use list comps all the time and said several times that I don't object to list comps, just nested ones. I suppose we're making progress if we agree that the Python docs warn against unnecessarily complicated nested list comps. Whether it discourages as well as warns is a matter of interpretation. But there's certainly no sign that list comps as a general technique are discouraged just because overly-complicated list comps are tricky to read. The same can be said about *any* piece of code. Well I did my best to make my interpretation clear. If the documentation says that nested list comps are difficult to read and should be used rarely, if at all, then I generally consider that discouraging their use. Meaning of course that in the right situation one may be able to justify their own use of a nested comp. However, for every day problems one wouldn't be encouraged to use them so liberally as some tend to do. So.. we have some sort of consensus then? This might be a rare phenomenon on Usenet... :) Cheers, J. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Terry Reedy tjre...@udel.edu writes: J Kenneth King wrote: Keep in mind that nested comprehensions are still available because they do have a use case that justifies their existence. Nested comprehensions are available because because the syntax makes them available by default and making a fiddly exception would be contrary to Python's style. A list comp creates an iterable. A list comp requires use of an iterable. Therefore, a list comp may use a list comp. However, I think the Python documentation warns against their use because people might rely on them for problems where they aren't necessary and since they are difficult to read... it can lead to difficult to read code. Whenever the expression that results in the iterable used by a list comp is sufficiently complex, readability is improved by pulling it out as a separate statement. Nested list comps are often examplex of such sufficiently complex expressions, but not the only possible one. tjr Agreed. A very succinct explanation of the point I was trying to make. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Thu, 07 May 2009 13:28:10 -0400, J Kenneth King wrote: Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. They are tricky and need to be used carefully, *if at all*. IMO this means that if there's a way to do it without a nested list comprehension, then that solution should be preferred. Earlier, you claimed that list comps in general were discouraged: List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Emile said I'd be surprised to find routine usage frowned upon, giving you the opportunity to correct his (mis)understanding. You failed to do so, which I took as meaning that you agreed that routine usage of simple list comps were frowned upon. Now you're talking about *nested* list comps. You started off (apparently) objecting to list comps in general, because they can make readers apprehensive. Now you seem to be saying that it's only the complicated, overly-dense ones which rightly make readers apprehensive which you object to. I suppose we're making progress if we agree that the Python docs warn against unnecessarily complicated nested list comps. Whether it discourages as well as warns is a matter of interpretation. But there's certainly no sign that list comps as a general technique are discouraged just because overly-complicated list comps are tricky to read. The same can be said about *any* piece of code. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano wrote: If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. and In real world, you should prefer builtin functions to complex flow statements. That's ridiculous. The example given is a special case. That's like saying Loops are hard, so in the real world, if you want a loop, find a builtin function that does what you want instead. What's the builtin function we're supposed to prefer over a complex flow statement like this? # split text into word fragments of length = 3 sentence = a sentence is a series of words new = [word[i:i+3] for word in sentence.split() for i in range(0, len(word), 3)] I often found list comprehension *more* readable than the equivalent for-loop because of its density. Seeing complex for-loop requires one step of thinking for each line, but in list comprehension I can skip some of the steps because I know what list comprehension would roughly look like. i.e. when I process this: lst = [] #1 for i in range(10): #2 lst.append(i) #3 I do 3 steps of thinking for each line but seeing this is only one step, since I know it is going to create new list (skipping step #1) and I know i will be appended to that list (skipping step #3) -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Lie Ryan wrote: Steven D'Aprano wrote: If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. and In real world, you should prefer builtin functions to complex flow statements. That's ridiculous. The example given is a special case. That's like saying Loops are hard, so in the real world, if you want a loop, find a builtin function that does what you want instead. What's the builtin function we're supposed to prefer over a complex flow statement like this? # split text into word fragments of length = 3 sentence = a sentence is a series of words new = [word[i:i+3] for word in sentence.split() for i in range(0, len(word), 3)] I often found list comprehension *more* readable than the equivalent for-loop because of its density. Seeing complex for-loop requires one step of thinking for each line, but in list comprehension I can skip some of the steps because I know what list comprehension would roughly look like. i.e. when I process this: lst = [] #1 for i in range(10): #2 lst.append(i) #3 I do 3 steps of thinking for each line but seeing this is only one step, since I know it is going to create new list (skipping step #1) and I know i will be appended to that list (skipping step #3) I don't know what I was thinking before sending that last line, here is what it was supposed to be: but in list comprehension I see this in only one step, since I know it is going to create new list (skipping step #1) and I know 'i' will be appended to that list (skipping step #3) and this also continues to nested list comprehensions -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Scott David Daniels wrote: John Posner wrote: Shane Geiger wrote: if type(el) == list or type(el) is tuple: A tiny improvement: if type(el) in (list, tuple): or (even better) if isinstance(el, (list, tuple)) However, it is my contention that you shouldn't be flattening by type -- you should know where, explicitly to flatten. If I have 3-D points as triples, a tree of points would, by this code, get flattened into a a list of numbers. If, however, I create a class with the same elements, but a method or two, suddenly flatten will produce a list of points. It is my contention that the tree abstraction should be responsible for producing its leaves, rather than a blunderbus that doesn't know where one container abstraction ends, and its contents begin. In other words, I'd like to see thigs like flatten one layer. Maybe that is why some modules like PIL or pygame accepts list of coordinates in flattened list of number instead of nested (i.e. they accepts [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] and do the grouping themself). -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. They are tricky and need to be used carefully, *if at all*. IMO this means that if there's a way to do it without a nested list comprehension, then that solution should be preferred. and In real world, you should prefer builtin functions to complex flow statements. That's ridiculous. The example given is a special case. That's like saying Loops are hard, so in the real world, if you want a loop, find a builtin function that does what you want instead. What's the builtin function we're supposed to prefer over a complex flow statement like this? It's not ridiculous and says nothing of the sort. You're jumping to conclusions. If you have a problem with it, I suggest you complain to the Python documentation people. I don't make this stuff up. Just look at many of the example solutions provided in this thread to the OP's problem. The examples in the link I provided are also good as are ones provided in the itertools documentation. Keep in mind that nested comprehensions are still available because they do have a use case that justifies their existence. However, I think the Python documentation warns against their use because people might rely on them for problems where they aren't necessary and since they are difficult to read... it can lead to difficult to read code. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
J Kenneth King wrote: Keep in mind that nested comprehensions are still available because they do have a use case that justifies their existence. Nested comprehensions are available because because the syntax makes them available by default and making a fiddly exception would be contrary to Python's style. A list comp creates an iterable. A list comp requires use of an iterable. Therefore, a list comp may use a list comp. However, I think the Python documentation warns against their use because people might rely on them for problems where they aren't necessary and since they are difficult to read... it can lead to difficult to read code. Whenever the expression that results in the iterable used by a list comp is sufficiently complex, readability is improved by pulling it out as a separate statement. Nested list comps are often examplex of such sufficiently complex expressions, but not the only possible one. tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On May 6, 2:10 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: It's precisely the indentation and colons (plus newlines) that makes nested for-loops easier to read than list-comps with multiple fors. You can get back *nearly* all the readability by splitting the list comp into multiple lines: It was less the overall readability I was commenting on, and more the claim that the listcomp required a 'back-and-forth' parsing to understand. As you've reinforced, the listcomp can be readily converted back to the multiple-line form by the inclusion of colons EOL markers, which means you can make as much sequential sense from a listcomp as you can a for-loop. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Ross wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? -- http://mail.python.org/mailman/listinfo/python-list just fyi, in python 2.6 list(itertools.chain.from_iterable(a)) would do it in python 2.5 list(itertools.chain(*a)) would do it too, but I wouldn't try it with arbitrarily long a -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On May 5, 11:36 pm, alex23 wuwe...@gmail.com wrote: Apart from the presence of 'item' at the beginning of the list comprehension as opposed to 'b.append(item)' at the end of the for-loop, how exactly does the listcomp force you to bounce [..] back and forth to follow the logic? It's precisely the fact that the item is at the beginning which is the main instigator of the bounce. To tranlate [f(x) for x in a] into a loop, you start in the middle, work your way toward the right, then bounce back to the left. I think some of us, perhaps subconsciously, get used to reading that small chunk on the right, then bouncing left. See enough small, flat LCs, and we may be conditioned to think that the general rule is to start with that small chunk on the right and bounce left. When confronted with a long, nested LC then, some of us have the instinct to read the rightmost chunk, then bounce left to the next- rightmost chunk, etc. Essentially, if you see [B A] over and over and over again, when finally confronted with LCs of more elements, it's not immediately clear that the pattern of increasing nestedness is [B A] = [C A B] = [D A B C] = etc. rather than [B A] = [C B A] = [D C B A] = etc. Maybe we're wired wrong, but there are many of us who find the second pattern more logical, which is why it is so easy for us to mess up the order of the nesting in a more complex LC. John -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. and In real world, you should prefer builtin functions to complex flow statements. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
John Yeung schreef: Essentially, if you see [B A] over and over and over again, when finally confronted with LCs of more elements, it's not immediately clear that the pattern of increasing nestedness is [B A] = [C A B] = [D A B C] = etc. rather than [B A] = [C B A] = [D C B A] = etc. Maybe we're wired wrong, but there are many of us who find the second pattern more logical, which is why it is so easy for us to mess up the order of the nesting in a more complex LC. Indeed, that is exactly the reason why I find it difficult to read nested loop comprehensions, and why I almost never use them. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 5/6/2009 6:48 AM J Kenneth King said... Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions Thanks. If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. Yes, although this is in reference to nested list comprehensions. Eg, [xx for xx in [yy for yy in [zz for zz in iterable if z] if y] if x] The section above on list comprehensions advocates their use saying: List comprehensions provide a concise way to create lists without resorting to use of map(), filter() and/or lambda. The resulting list definition tends often to be clearer than lists built using those constructs. Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. and In real world, you should prefer builtin functions to complex flow statements. That's ridiculous. The example given is a special case. That's like saying Loops are hard, so in the real world, if you want a loop, find a builtin function that does what you want instead. What's the builtin function we're supposed to prefer over a complex flow statement like this? # split text into word fragments of length = 3 sentence = a sentence is a series of words new = [word[i:i+3] for word in sentence.split() for i in range(0, len(word), 3)] -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... Chris Rebert c...@rebertia.com writes: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. Dense and difficult may be subjective to people like you or I, but you left out the to read part of that sentence. I was referring to the negative effect nested or complex list comprehensions can have on readability. b = [ item for pair in a for item in pair ] It's a clever statement, but use it once and its gone. Why not use the expanded form in a function definition and get an even shorter, but clearer statement? b = flatten(a) Boom, done. This is exactly the code above expressed in comprehension form. It's worth knowing that a list comprehension is structured identically to the equivalent for loop. So it really is neither more dense nor more difficult to read. Further, you can tell immediately from the start of the list comprehension what you've got -- in this case a list of item(s). Here with some slight changes... a = [(1, 2), (3, 4, 7), (5, 6)] [ item for j in a if len(j)==2 for item in j if item % 2 ] [1, 5] ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... b [1, 5] Thanks for the lesson in list comprehensions, but I'm well aware of how they work. List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. It's much easier to explain complex processes with a function rather than a single nested statement. YMMV, Emile It will apparently vary greatly. Depending on how much coffee I've had. ;) J -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: list comprehension question
On Fri, 2009-05-01 at 13:00 -0400, John Posner wrote: Shane Geiger wrote: if type(el) == list or type(el) is tuple: A tiny improvement: if type(el) in (list, tuple): Another alternative, which might be useful in some cases: if hasattr(el, '__iter__'): This covers all iterables, not just lists and tuples. So: flatten([1,2, xrange(3,15), 15, 16]) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16] The downside, of course, is that some iterables might be infinite (such as count), which would cause hidden breakage. But if you have one of those in your data structure, why are you trying to flatten it anyway? Cheers, Cliff -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Tue, 2009-05-05 at 12:15 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... Chris Rebert c...@rebertia.com writes: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. Dense and difficult may be subjective to people like you or I, but you left out the to read part of that sentence. I was referring to the negative effect nested or complex list comprehensions can have on readability. b = [ item for pair in a for item in pair ] It's a clever statement, but use it once and its gone. Why not use the expanded form in a function definition and get an even shorter, but clearer statement? It's also not obvious what it means. I would have thought the proper incantation would be: [item for item in pair for pair in a] but that returns [5,5,5,6,6,6]. I still haven't figured out why. The way you have to bounce your eyes back and forth in the comprehension makes it hard to read the logic. With the loop, on the other hand it is blatantly obvious which way the nesting occurs. b = flatten(a) Boom, done. This is exactly the code above expressed in comprehension form. It's worth knowing that a list comprehension is structured identically to the equivalent for loop. So it really is neither more dense nor more difficult to read. Further, you can tell immediately from the start of the list comprehension what you've got -- in this case a list of item(s). Empirically, I'd have to disagree with you. The structure is changed by the fact that item gets put at the beginning of the comprehension. With the loop, each variable gets introduced in a for statement, and then used. With the list comprehension, item gets used at the beginning, and then assigned to in the for expression at the end. Meanwhile, pair gets introduced as the loop variable in the first for expression, and iterated over in the second for expression. So do you read it left to right or right to left? Neither. You have to hold the whole expression in your mind at once. Good for short and simple comprehensions, bad as they get more complicated. Here with some slight changes... a = [(1, 2), (3, 4, 7), (5, 6)] [ item for j in a if len(j)==2 for item in j if item % 2 ] [1, 5] This, to me, looks like line noise, approaching perl in its unreadability. ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... b [1, 5] Much nicer. Thank you. Thanks for the lesson in list comprehensions, but I'm well aware of how they work. List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. It's much easier to explain complex processes with a function rather than a single nested statement. YMMV, Emile It will apparently vary greatly. Depending on how much coffee I've had. ;) J -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Tue, 05 May 2009 13:43:32 -0400, J. Cliff Dyer wrote:
On Fri, 2009-05-01 at 13:00 -0400, John Posner wrote:
Shane Geiger wrote:
if type(el) == list or type(el) is tuple:
A tiny improvement:
if type(el) in (list, tuple):
Another alternative, which might be useful in some cases:
if hasattr(el, '__iter__'):
This covers all iterables, not just lists and tuples.
Except for the ones that it doesn't cover, like strings:
hasattr('abcd', '__iter__')
False
list(iter('abcd'))
['a', 'b', 'c', 'd']
And classes that follow the sequence protocol:
class Spam:
... def __getitem__(self, index):
... if 0 = index 5:
... return spam
... raise IndexError
...
hasattr(Spam(), '__iter__')
False
list(iter(Spam()))
['spam', 'spam', 'spam', 'spam', 'spam']
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On May 6, 4:01 am, J. Cliff Dyer j...@sdf.lonestar.org wrote: The way you have to bounce your eyes back and forth in the comprehension makes it hard to read the logic. With the loop, on the other hand it is blatantly obvious which way the nesting occurs. [ item for j in a if len(j)==2 for item in j if item % 2 ] ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... Much nicer. Thank you. Apart from the presence of 'item' at the beginning of the list comprehension as opposed to 'b.append(item)' at the end of the for- loop, how exactly does the listcomp force you to bounce [..] back and forth to follow the logic? The only other difference between the two is in the layout - the indentation colons - otherwise they're structurally identical. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Tue, 05 May 2009 20:36:37 -0700, alex23 wrote: On May 6, 4:01 am, J. Cliff Dyer j...@sdf.lonestar.org wrote: The way you have to bounce your eyes back and forth in the comprehension makes it hard to read the logic. With the loop, on the other hand it is blatantly obvious which way the nesting occurs. [ item for j in a if len(j)==2 for item in j if item % 2 ] ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... Much nicer. Thank you. Apart from the presence of 'item' at the beginning of the list comprehension as opposed to 'b.append(item)' at the end of the for- loop, how exactly does the listcomp force you to bounce [..] back and forth to follow the logic? The only other difference between the two is in the layout - the indentation colons - otherwise they're structurally identical. It's precisely the indentation and colons (plus newlines) that makes nested for-loops easier to read than list-comps with multiple fors. for a in alist: for b in blist: for c in clist: if c: parrot(a, b, c) is much easier to read than the equivalent one-liner: [parrot(a, b, c) for a in alist for b in blist for c in clist if c] You can get back *nearly* all the readability by splitting the list comp into multiple lines: [parrot(a, b, c) for a in alist for b in blist for c in clist if c ] -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano wrote:
On Tue, 05 May 2009 13:43:32 -0400, J. Cliff Dyer wrote:
Except for the ones that it doesn't cover, like strings:
hasattr('abcd', '__iter__')
False
True in Python3
--
http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Ross ross.j...@gmail.com writes: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? When writing nested list comprehension, the for loops are in the same order as you would write a normal nested for loop (which is not necessarily intuitive when you first find out but is very handy in the long run I think). So write: [j for i in a for j in i] -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On May 1, 2:28 pm, Arnaud Delobelle arno...@googlemail.com wrote: Ross ross.j...@gmail.com writes: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? When writing nested list comprehension, the for loops are in the same order as you would write a normal nested for loop (which is not necessarily intuitive when you first find out but is very handy in the long run I think). So write: [j for i in a for j in i] -- Arnaud an trick a [(1, 2), (3, 4), (5, 6)] sum(a, ()) (1, 2, 3, 4, 5, 6) you may search the maillist , somebody questioned before -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Chris Rebert c...@rebertia.com writes: On Thu, Apr 30, 2009 at 5:56 PM, Ross ross.j...@gmail.com wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? Your comprehension is the identity comprehension (i.e. it effectively just copies the list as-is). What you're trying to do is difficult if not impossible to do as a comprehension. Here's another approach: b = list(itertools.chain.from_iterable(a)) And without using a library function: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 5/1/2009 7:31 AM J Kenneth King said... Chris Rebert c...@rebertia.com writes: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. b = [ item for pair in a for item in pair ] This is exactly the code above expressed in comprehension form. It's worth knowing that a list comprehension is structured identically to the equivalent for loop. So it really is neither more dense nor more difficult to read. Further, you can tell immediately from the start of the list comprehension what you've got -- in this case a list of item(s). Here with some slight changes... a = [(1, 2), (3, 4, 7), (5, 6)] [ item for j in a if len(j)==2 for item in j if item % 2 ] [1, 5] ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... b [1, 5] YMMV, Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... Chris Rebert c...@rebertia.com writes: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. b = [ item for pair in a for item in pair ] This is exactly the code above expressed in comprehension form. If the comprehension above is an abuse, then every nested list comprehension is an abuse of comprehensions so they might as well not be in the language... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
from goopy.functional import flatten # http://sourceforge.net/projects/goog-goopy/ b = [(1,2), (3,4), (5,6)] print flatten(b) #from goopy.functional import flatten # http://sourceforge.net/projects/goog-goopy/ def flatten(seq): Returns a list of the contents of seq with sublists and tuples exploded. The resulting list does not contain any sequences, and all inner sequences are exploded. For example: flatten([7,(6,[5,4],3),2,1]) [7,6,5,4,3,2,1] lst = [] for el in seq: if type(el) == list or type(el) is tuple: lst.extend(flatten(el)) else: lst.append(el) return lst Chris Rebert wrote: On Thu, Apr 30, 2009 at 5:56 PM, Ross ross.j...@gmail.com wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? Your comprehension is the identity comprehension (i.e. it effectively just copies the list as-is). What you're trying to do is difficult if not impossible to do as a comprehension. Here's another approach: b = list(itertools.chain.from_iterable(a)) And without using a library function: b = [] for pair in a: for item in pair: b.append(item) Cheers, Chris -- Shane Geiger, IT Director Council For Economic Education / www.councilforeconed.org sgei...@councilforeconed.org / 402-438-8958 Teaching Opportunity -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: list comprehension question
Shane Geiger wrote: if type(el) == list or type(el) is tuple: A tiny improvement: if type(el) in (list, tuple): -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On 5/1/2009 9:19 AM Arnaud Delobelle said... Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. b = [ item for pair in a for item in pair ] This is exactly the code above expressed in comprehension form. If the comprehension above is an abuse, then every nested list comprehension is an abuse of comprehensions so they might as well not be in the language... I hope you didn't understand me to say that this usage is abuse. That's certainly not what I mean. I have, however, examples in my code base that _would_ qualify as abuse (mostly from pre dictionary comprehension vintage where I would build a dict from within a list comp), and I have seen enough from this group to know that abuse abounds, with only slight provocation. :) Fondly-remembering-byte-code-hacks-and-introspection-ly y'rs, Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/1/2009 9:19 AM Arnaud Delobelle said... Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. b = [ item for pair in a for item in pair ] This is exactly the code above expressed in comprehension form. If the comprehension above is an abuse, then every nested list comprehension is an abuse of comprehensions so they might as well not be in the language... I hope you didn't understand me to say that this usage is abuse. Emile: sorry, the way I was quoting earlier posts was unclear. I was refering to the post you quoted in your message, which describe the nested list comprehension as abuse. As supporting evidence for my reply I included your example of nested list comprehension, which is one of the simplest possible ones one can think of. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
John Posner wrote: Shane Geiger wrote: if type(el) == list or type(el) is tuple: A tiny improvement: if type(el) in (list, tuple): or (even better) if isinstance(el, (list, tuple)) However, it is my contention that you shouldn't be flattening by type -- you should know where, explicitly to flatten. If I have 3-D points as triples, a tree of points would, by this code, get flattened into a a list of numbers. If, however, I create a class with the same elements, but a method or two, suddenly flatten will produce a list of points. It is my contention that the tree abstraction should be responsible for producing its leaves, rather than a blunderbus that doesn't know where one container abstraction ends, and its contents begin. In other words, I'd like to see thigs like flatten one layer. --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
On Thu, Apr 30, 2009 at 5:56 PM, Ross ross.j...@gmail.com wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? Your comprehension is the identity comprehension (i.e. it effectively just copies the list as-is). What you're trying to do is difficult if not impossible to do as a comprehension. Here's another approach: b = list(itertools.chain.from_iterable(a)) And without using a library function: b = [] for pair in a: for item in pair: b.append(item) Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Ross wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? -- http://mail.python.org/mailman/listinfo/python-list a = [(1,2), (3,4), (5,6)] [i for t in a for i in t] [1, 2, 3, 4, 5, 6] Or, with different spacing to make the nesting clearer: [i ... for t in a ... for i in t] [1, 2, 3, 4, 5, 6] Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 24, 5:35 am, Boris Borcic [EMAIL PROTECTED] wrote: Paul Rubin wrote: beginner [EMAIL PROTECTED] writes: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] no, that one will be [[[1,2], [f(1), f(2)]], [[3,4], [f(3), f(4)]]] eg two sublists instead of four. [map(g,a) for a in x for g in [None,f]] will do it. ...a bit too cleverly, but there's worse : list((yield a) or map(f,a) for a in x) Cheers, BB Really cool! -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
beginner [EMAIL PROTECTED] writes: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
Boris Borcic [EMAIL PROTECTED] writes: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] no, that one will be [[[1,2], [f(1), f(2)]], [[3,4], [f(3), f(4)]]] eg two sublists instead of four. Oh ugh, I misread the original request and thought the extra brackets were there. I think the following works and is reasonably intuitive: from itertools import chain y = list(chain(*([a, map(f,a)] for a in x))) But the original request itself seems a bit weird. -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
Paul Rubin wrote: beginner [EMAIL PROTECTED] writes: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] no, that one will be [[[1,2], [f(1), f(2)]], [[3,4], [f(3), f(4)]]] eg two sublists instead of four. [map(g,a) for a in x for g in [None,f]] will do it. ...a bit too cleverly, but there's worse : list((yield a) or map(f,a) for a in x) Cheers, BB -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 24, 12:44 am, beginner [EMAIL PROTECTED] wrote: On Aug 24, 12:41 am, Davo [EMAIL PROTECTED] wrote: On Aug 23, 9:24 pm, beginner [EMAIL PROTECTED] wrote: Hi All, How do I map a list to two lists with list comprehension? For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [1,2] is mapped to [1,2] and [f(1), f(2)] and [3,4] is mapped to [3,4], [f(3), f(4)]. I just can't find any way to do that with list comprension. I ended up using a loop (untested code based on real code): l=[] for y in x: l.append(y) l.append([f(z) for z in y]) Thanks, Geoffrey This may be what you want: l = [[y, [f(z) for z in y]] for y in x] But It's a bit dense. How about: l=[] for y in x: Fy = [f(z) for z in y] l.extend([y, Fy]) -- David- Hide quoted text - - Show quoted text - This is exactly what I was looking for. Thanks.- Hide quoted text - - Show quoted text - On second thought, that will generate one additional level. So it is not what I am looking for. -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 24, 5:47 am, Paul Rubin http://[EMAIL PROTECTED] wrote: Boris Borcic [EMAIL PROTECTED] writes: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] no, that one will be [[[1,2], [f(1), f(2)]], [[3,4], [f(3), f(4)]]] eg two sublists instead of four. Oh ugh, I misread the original request and thought the extra brackets were there. I think the following works and is reasonably intuitive: from itertools import chain y = list(chain(*([a, map(f,a)] for a in x))) But the original request itself seems a bit weird. Not so werid. :-) Just to put this into context, I have a list of list of objects x=[[o1, o2, o3, o4], [o5,o6]] I was trying to plot them with matplotlib. Each sub-list is a line. So I would have to re-organize the list to be like the following: v=[[o1.x, o2.x, o3.x, o4.x], [o1.y, o2.y, o3.y, o4.y], [o5.x, o6.x], [o5.y, o6.y]] So that I can pass it to plot: plot(*tuple(v)) I like the chain and map solutions. -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
Boris Borcic wrote: For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, map(f,a)] for a in x] [map(g,a) for a in x for g in [None,f]] will do it. ...a bit too cleverly, but there's worse : list((yield a) or map(f,a) for a in x) worse (in *many* ways) solutions: l = [[a, map(f,a)] for a in x] 1) s = sum(l, []) 2) from operator import add r = reduce(add, l, []) 3) a = [] for e in l: a.extend(e) -- Under construction -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [[a, [f(b) for b in a]] for a in x] -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 23, 9:24 pm, beginner [EMAIL PROTECTED] wrote: Hi All, How do I map a list to two lists with list comprehension? For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [1,2] is mapped to [1,2] and [f(1), f(2)] and [3,4] is mapped to [3,4], [f(3), f(4)]. I just can't find any way to do that with list comprension. I ended up using a loop (untested code based on real code): l=[] for y in x: l.append(y) l.append([f(z) for z in y]) Suppose f is: def f(n): return str(n) import itertools Using a list comprehension: [i for i in itertools.chain(*[(eachx, [f(y) for y in eachx]) for eachx in x])] [[1, 2], ['1', '2'], [3, 4], ['3', '4']] Using a list: list(itertools.chain(*[(eachx, [f(y) for y in eachx]) for eachx in x])) [[1, 2], ['1', '2'], [3, 4], ['3', '4']] Not so pretty in either case. -- Hope this helps, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
beginner wrote: How do I map a list to two lists with list comprehension? For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [1,2] is mapped to [1,2] and [f(1), f(2)] and [3,4] is mapped to [3,4], [f(3), f(4)]. I just can't find any way to do that with list comprension. [a for b in ((item, map(f, item)) for item in x) for a in b] [[1, 2], [f(1), f(2)], [3, 4], [f(3), f(4)]] I ended up using a loop (untested code based on real code): l=[] for y in x: l.append(y) l.append([f(z) for z in y]) Using a loop gives you clearer code in this case, so I'd use that. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 24, 12:41 am, Davo [EMAIL PROTECTED] wrote: On Aug 23, 9:24 pm, beginner [EMAIL PROTECTED] wrote: Hi All, How do I map a list to two lists with list comprehension? For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [1,2] is mapped to [1,2] and [f(1), f(2)] and [3,4] is mapped to [3,4], [f(3), f(4)]. I just can't find any way to do that with list comprension. I ended up using a loop (untested code based on real code): l=[] for y in x: l.append(y) l.append([f(z) for z in y]) Thanks, Geoffrey This may be what you want: l = [[y, [f(z) for z in y]] for y in x] But It's a bit dense. How about: l=[] for y in x: Fy = [f(z) for z in y] l.extend([y, Fy]) -- David- Hide quoted text - - Show quoted text - This is exactly what I was looking for. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: List Comprehension Question: One to Many Mapping?
On Aug 23, 9:24 pm, beginner [EMAIL PROTECTED] wrote: Hi All, How do I map a list to two lists with list comprehension? For example, if I have x=[ [1,2], [3,4] ] What I want is a new list of list that has four sub-lists: [[1,2], [f(1), f(2)], [3,4], [f(3), f(4)]] [1,2] is mapped to [1,2] and [f(1), f(2)] and [3,4] is mapped to [3,4], [f(3), f(4)]. I just can't find any way to do that with list comprension. I ended up using a loop (untested code based on real code): l=[] for y in x: l.append(y) l.append([f(z) for z in y]) Thanks, Geoffrey This may be what you want: l = [[y, [f(z) for z in y]] for y in x] But It's a bit dense. How about: l=[] for y in x: Fy = [f(z) for z in y] l.extend([y, Fy]) -- David -- http://mail.python.org/mailman/listinfo/python-list
