Re: Removing None objects from a sequence
On Dec 12, 7:51 am, Marco Mariani ma...@sferacarta.com wrote:
Filip Gruszczyński wrote:
I am not doing it, because I need it. I can as well use if not elem
is None,
I suggest if elem is not None, which is not quite the same.
They are semantically the same. In theory, Filip's would run slower
because of the time to negate the is-test, but the peephole optimizer
recognizes the opportunity to make the substitution so it works out
exactly the same:
dis(compile('x is not None', '', 'eval'))
1 0 LOAD_NAME0 (x)
3 LOAD_CONST 0 (None)
6 COMPARE_OP 9 (is not)
9 RETURN_VALUE
dis(compile('not x is None', '', 'eval'))
1 0 LOAD_NAME0 (x)
3 LOAD_CONST 0 (None)
6 COMPARE_OP 9 (is not)
9 RETURN_VALUE
Raymond
--
http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Mon, 15 Dec 2008 05:39:45 +, Lie Ryan wrote: I was just expressing the preference that operators should be composed of a single word, especially since none of the other operators are multi-words Then you should have said so, instead of introducing red-herrings about tired programmers. I must say, some years ago I probably would have agreed with you. I used to dislike a is not b and would go out of my way to write not a is b. But then I decided that was just being silly, and I've never looked back. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Steven D'Aprano wrote: On Fri, 12 Dec 2008 19:02:24 -0500, Terry Reedy wrote: ... Tim Chase wrote: If you want to literally remove None objects from a list(or mutable sequence) def deNone(alist): n=len(alist) i=j=0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j:i] = [] Contrast that with the alternative suggested by Tim: def deNone2(alist): alist[:] = [x for x in alist if x is not None] ... Here's another low-level algorithm, the classical delete items in place algorithm. Three lines, one index, lousy O(N**2) performance. def deNone3(alist): for i in xrange(len(alist)-1, -1, -1): if alist[i] is None: del alist[i] Now, let's do a shoot-out. Do they return the same thing? ... [good measurements generally reinforcing Tim's implementation] ... If you want to keep the original's method, but do it in a more Pythonic way, I would suggest: def deNone4(alist): j = 0 for val in alist: if val is not None: alist[j] = val j += 1 del alist[j :] This still loses to Tim's clearer code, but by nowhere near as much. --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Mon, 2008-12-15 at 02:11 +, Lie Ryan wrote: On Fri, 12 Dec 2008 22:55:20 +, Steven D'Aprano wrote: On Fri, 12 Dec 2008 21:18:36 +, Lie Ryan wrote: Personally, I'd prefer VB's version: foo IsNot bar or in pseudo-python foo isnot bar since that would make it less ambiguous. a is not b is no more ambiguous than 1+2*3. True, there's ambiguity if you are ignorant of the precedence rules, but that's no worse than saying that + is ambiguous if you don't know what + means. What's this 'is' operator??? It's ambiguous, it could mean ANYTHING!!! Panic panic panic panic!!! *wink* You're allowed to assume the normal conventions, and (lucky for me!) despite being Dutch Guido choose to assume the normal English convention that a is not b means the same as not (a is b) rather than a is (not b). That's probably because the use-cases for the second would be rather rare. So given the normal precedence rules of Python, there is no ambiguity. True, you have to learn the rules, but that's no hardship. *I* know about the precedence rule, but a newbie or a tired programmer might not. He might want to reverse the truth value of argument b but instead has just reversed the whole expression. Probably in a slightly convoluted code like this: if a is not(b and c): ... A newbie might make that mistake, but a tired programmer would be testing for equality rather than identity in that case, and if she weren't, she'd have to fix her code in the morning anyway. Cheers, Cliff -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Scott David Daniels: If you want to keep the original's method, but do it in a more Pythonic way, I would suggest: def deNone4(alist): j = 0 for val in alist: if val is not None: alist[j] = val j += 1 del alist[j :] For my benchmarks this is also the faster version with and without Psyco :-) (Even it's a bit slow when given a list that has no Nones). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On 2008-12-12, Filip Gruszczyński wrote: Hi! I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. And I would like to get something faster. I can use [ sth for sth in self.__sth if not sth is None ], but I don't know if that's the best way. I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. How can I get it the shortest and fastest way? There's a little hack that will remove all elements from a list that are 'False' when considered as a boolean. So None, [], '', False, etc. filter(None, myseq) an example: l = ['1', 2, 0, None, '5'] filter(None, l) ['1', 2, '5'] -- Regards, Stephen Thorne Development Engineer NetBox Blue - 1300 737 060 Scanned by the NetBox from NetBox Blue (http://netboxblue.com/) Can you afford to be without a NetBox? Find out the real cost of internet abuse with our ROI calculator. http://netboxblue.com/roi Scanned by the NetBox from NetBox Blue (http://netboxblue.com/) -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 22:55:20 +, Steven D'Aprano wrote: On Fri, 12 Dec 2008 21:18:36 +, Lie Ryan wrote: Personally, I'd prefer VB's version: foo IsNot bar or in pseudo-python foo isnot bar since that would make it less ambiguous. a is not b is no more ambiguous than 1+2*3. True, there's ambiguity if you are ignorant of the precedence rules, but that's no worse than saying that + is ambiguous if you don't know what + means. What's this 'is' operator??? It's ambiguous, it could mean ANYTHING!!! Panic panic panic panic!!! *wink* You're allowed to assume the normal conventions, and (lucky for me!) despite being Dutch Guido choose to assume the normal English convention that a is not b means the same as not (a is b) rather than a is (not b). That's probably because the use-cases for the second would be rather rare. So given the normal precedence rules of Python, there is no ambiguity. True, you have to learn the rules, but that's no hardship. *I* know about the precedence rule, but a newbie or a tired programmer might not. He might want to reverse the truth value of argument b but instead has just reversed the whole expression. Probably in a slightly convoluted code like this: if a is not(b and c): ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Mon, 15 Dec 2008 02:11:10 +, Lie Ryan wrote: So given the normal precedence rules of Python, there is no ambiguity. True, you have to learn the rules, but that's no hardship. *I* know about the precedence rule, but a newbie or a tired programmer might not. He might want to reverse the truth value of argument b but instead has just reversed the whole expression. And? A newbie or a tired programmer might not know that a^b is bit-wise xor instead of exponentiation, or that range(n) doesn't include n, or even that len(alist) returns the length of a list. There's no limit to the potential mistakes that are possible for a programmer who is tired, inexperienced, intoxicated or just plain stupid enough. What's your point? Are you expecting Python to be mistake-proof? There's a certain level of knowledge about the language necessary to program effectively, and learning that is not is a single operator is not particularly onerous. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Mon, 15 Dec 2008 03:21:21 +, Steven D'Aprano wrote: On Mon, 15 Dec 2008 02:11:10 +, Lie Ryan wrote: So given the normal precedence rules of Python, there is no ambiguity. True, you have to learn the rules, but that's no hardship. *I* know about the precedence rule, but a newbie or a tired programmer might not. He might want to reverse the truth value of argument b but instead has just reversed the whole expression. And? A newbie or a tired programmer might not know that a^b is bit-wise xor instead of exponentiation, or that range(n) doesn't include n, or even that len(alist) returns the length of a list. There's no limit to the potential mistakes that are possible for a programmer who is tired, inexperienced, intoxicated or just plain stupid enough. What's your point? Are you expecting Python to be mistake-proof? There's a certain level of knowledge about the language necessary to program effectively, and learning that is not is a single operator is not particularly onerous. I give up. It does not matter to me anyway. I was just expressing the preference that operators should be composed of a single word, especially since none of the other operators are multi-words (Special cases aren't special enough to break the rules). The only advantage of using 'is not' over 'isnot' is that we have one less keyword to deal with. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Lie Ryan lie.1...@gmail.com writes: I was just expressing the preference that operators should be composed of a single word, especially since none of the other operators are multi-words (Special cases aren't special enough to break the rules). The only advantage of using 'is not' over 'isnot' is that we have one less keyword to deal with. I had never thought of the potential for misinterpreting 'a is not b' for 'a is (not b)' before and this made me slightly uncomfortable for a bit. However there is no valid reason that I can think of to write a is (not b). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 19:02:24 -0500, Terry Reedy wrote: Tim Chase wrote: If you want to literally remove None objects from a list(or mutable sequence) def deNone(alist): n=len(alist) i=j=0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j:i] = [] blist=[None,1,None,2,None,3,None,None,4,None] deNone(blist) print(blist) # prints [1, 2, 3, 4] ...wouldn't a cleaner way of doing this just be blist=[None,1,None,2,None,3,None,None,4,None] No, making a filtered copy that is then copied back before being deleted is algorithmically much messier. My code does the minimum work necessary and is algorithmically cleaner. Er what? You're joking, right? Your function has eight lines, all of which are very low level: you're dealing with not one but TWO list indexes yourself. You have three internal names to deal with, although in fairness one is set once then used as a constant from then on. The algorithm is unclear: try explaining what you are doing in English, and see how difficult it is. Keep two indexes into the list. If the first index isn't pointing at None, copy that value to the second index, which is either the same as the first index, or pointing at None. Then advance the second index, and the first index, as needed. It will never over-write a non-None value, but just try proving it. Contrast that with the alternative suggested by Tim: def deNone2(alist): alist[:] = [x for x in alist if x is not None] It's one line, with one internal variable. You don't have to manipulate index variables. The explanation is simple: Make a new list with the non-None values and assign it in-place to the old list. Clear, succinct, and understandable, with no corner cases like empty lists to worry about. Here's another low-level algorithm, the classical delete items in place algorithm. Three lines, one index, lousy O(N**2) performance. def deNone3(alist): for i in xrange(len(alist)-1, -1, -1): if alist[i] is None: del alist[i] Now, let's do a shoot-out. Do they return the same thing? masterlist = [None]*2 + range(5) + [None]*3 + range(5, 10) + [None]*4 + [10, None, 11, None, 12, None] alist = masterlist[:] blist = masterlist[:] clist = masterlist[:] deNone(alist) deNone2(blist) deNone3(clist) alist == blist == clist True Which is faster? from timeit import Timer setup = from __main__ import deNone, deNone2, deNone3; \ ... from __main__ import masterlist as m t1 = Timer(a=m[:];deNone(a), setup) t2 = Timer(a=m[:];deNone2(a), setup) t3 = Timer(a=m[:];deNone3(a), setup) t1.repeat(number=1) [0.39079904556274414, 0.38915109634399414, 0.39700794219970703] t2.repeat(number=1) [0.17854905128479004, 0.1782989501953125, 0.17886185646057129] t3.repeat(number=1) [0.26834988594055176, 0.25835609436035156, 0.25850009918212891] Not surprisingly, Tim's version is twice as fast as yours. Surprisingly, even the deNone3 function is faster than yours. What about a real test? None of these piddly toy data sets, with 25 items. Something *real*. masterlist = masterlist*100 masterlist += range(1000) masterlist += [None]*1000 masterlist += [None, 0]*1000 masterlist += [1]*1 len(masterlist) 16500 t1.repeat(number=100) [2.1611621379852295, 1.2539350986480713, 1.2424759864807129] t2.repeat(number=100) [0.93860101699829102, 0.44704914093017578, 0.41285014152526855] t3.repeat(number=100) [4.5643420219421387, 3.216562032699585, 3.2176508903503418] Not surprisingly, my version really suffers. But so does yours: it's now three times slower than Tim's. I expect that Tim's version will look better and better until the list is so huge that memory paging to disk becomes a factor. Now, sure, most of the work in Tim's version is executed in fast C code instead of slow Python code. If we were programming in C, your version might perform better relative to Tim's. But even if performance was better, I wouldn't describe the algorithm as particularly clean. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Just to be clear, I decided to use generator by alex23, as it seems simple, short and understandable. Still reading this thread was quite interesting, thanks :-) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Steven D'Aprano: The algorithm is unclear: try explaining what you are doing in English, and see how difficult it is. That algorithm is a standard one, and quite clear. Any programmer worth his/her/hir salt has to be able to understand that. I agree that the idiom with the list comp (algorithm2) is better for Python, but you have to know that algorithm1 to use it in other languages. Now, sure, most of the work in Tim's version is executed in fast C code instead of slow Python code. If we were programming in C, your version might perform better relative to Tim's. Try Psyco too: from timeit import default_timer as clock def remove_nones1(alist): n = len(alist) i = j = 0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j : i] = [] def remove_nones2(alist): alist[:] = [x for x in alist if x is not None] def remove_nones3(alist): for i in xrange(len(alist)-1, -1, -1): if alist[i] is None: del alist[i] def test123(data): data1 = data[:] data2 = data[:] data3 = data[:] t = clock() remove_nones1(data1) print remove_nones1:, round(clock() - t, 2), s t = clock() remove_nones2(data2) print remove_nones2:, round(clock() - t, 2), s t = clock() remove_nones3(data3) print remove_nones3:, round(clock() - t, 2), s assert data1 == data2 == data3 assert data1 == data2 def test12(data): data1 = data[:] data2 = data[:] data3 = data[:] t = clock() remove_nones1(data1) print remove_nones1:, round(clock() - t, 2), s t = clock() remove_nones2(data2) print remove_nones2:, round(clock() - t, 2), s print remove_nones3: (not tested) assert data1 == data2 def gen_data(N): print N:, N data = [None]*2 + range(5) + [None]*3 + range(5, 10) + [None]*4 + \ [10, None, 11, None, 12, None] data *= N data += range(N * 10) data += [None] * (N * 10) data += [None, 0] * (N * 10) data += [1] * (N * 100) return data print Without Psyco: test123(gen_data(1000)) test12(gen_data(3)) print print With Psyco: import psyco; psyco.full() test123(gen_data(1000)) test12(gen_data(3)) Results on a Core2 2 GHz, Python 2.6.1, latest Psyco: Without Psyco: N: 1000 remove_nones1: 0.05 s remove_nones2: 0.02 s remove_nones3: 3.04 s N: 3 remove_nones1: 1.51 s remove_nones2: 0.62 s remove_nones3: (not tested) With Psyco: N: 1000 remove_nones1: 0.0 s remove_nones2: 0.01 s remove_nones3: 3.01 s N: 3 remove_nones1: 0.08 s remove_nones2: 0.33 s remove_nones3: (not tested) As you see even with Psyco a (bit) lower level style of coding wins :-) In practice if that routine is important (or if it's a generic library routine that you don't know how it will be used) you use two different algorithms according to the availability of Psyco. Something like: def remove_nones4(alist): if psyco_available: n = len(alist) i = j = 0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j : i] = [] else: alist[:] = [x for x in alist if x is not None] Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Sat, 13 Dec 2008 11:07:53 +, Steven D'Aprano wrote: Now, sure, most of the work in Tim's version is executed in fast C code instead of slow Python code. Say what??? I'm sorry, I must have been smoking crack when I wrote that. It does nothing of the sort. Tim's version is pure Python. def deNone2(alist): alist[:] = [x for x in alist if x is not None] -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Sat, 13 Dec 2008 06:00:09 -0800, bearophileHUGS wrote: Steven D'Aprano: The algorithm is unclear: try explaining what you are doing in English, and see how difficult it is. That algorithm is a standard one, and quite clear. Any programmer worth his/her/hir salt has to be able to understand that. I didn't say it can't be understood. But please, if you think it is quite clear, go ahead and explain how it works, and how you know the algorithm is correct and there are no odd corner cases where it fails. As my comp sci lecturer used to say, prove the algorithm is correct. Remember, Terry made a specific claim: the algorithm is simpler and cleaner than Tim's copy-the-values-that-aren't-None solution. It's not just clear, it's significantly clearer: No, making a filtered copy that is then copied back before being deleted is algorithmically much messier. My code does the minimum work necessary and is algorithmically cleaner. Do you agree with Terry's claim? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Filip Gruszczyński wrote: I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. That doesn't make much sense; why would iterating over the sequence take more _space_? -- Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM, Y!M erikmaxfrancis Life is a gamble so I should / Live life more carefully -- TLC -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Dec 12, 7:18 pm, Filip Gruszczyński grusz...@gmail.com wrote: Hi! I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. And I would like to get something faster. I can use [ sth for sth in self.__sth if not sth is None ], but I don't know if that's the best way. I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. How can I get it the shortest and fastest way? Rather than a list comprehension, use a generator expression: for item in (x for x in sequence if x is not None): do_something(x) This doesn't generate the intermediate list with None elements removed, rather it steps one at a time through the original sequence and only returns the non-None elements. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
I don't mean memory, but space in code ;-) I'll try this generator :) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
2008/12/12 Filip Gruszczyński grusz...@gmail.com: I don't mean memory, but space in code ;-) Trying to save printer paper for your listings, then? -- Tim Rowe -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 10:18:35 +0100, Filip Gruszczyński wrote: Hi! I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. Too much space??? seq = [x for x in seq if x is not None] It's one line, less than 40 characters. If your hard disk is so full that you're worrying about 40 characters, I suggest you need a bigger disk. And I would like to get something faster. Faster than what? What speed do we have to beat? I can use [ sth for sth in self.__sth if not sth is None ], but I don't know if that's the best way. Who cares if it's the best way? What's important is, is it good enough? It is easier to optimize correct code than to correct optimized code. Get your code working first, then worry about shaving microseconds off the runtime *if you need to*. I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. How can I get it the shortest and fastest way? You could do this: seq = filter(None, seq) but only if you know that seq doesn't contain any false objects other than None. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
I am not doing it, because I need it. I can as well use if not elem is None, but I just wanted to know, if there is some better way of doing this. I like to know :-) And I can't understand why you are becoming so aggressive because of this. Just because I asked for that, doesn't mean, that I will put some obfuscated code into my project. I just wanted to learn something new - I checked itertools, I googled a bit, now I wanted to ask here if someone knew some really cool way of this. All the other assumptions weren't really necessary. Thanks for those ideas, however. I like the last one a lot :) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Filip Gruszczyński wrote: Hi! I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. And I would like to get something faster. I can use [ sth for sth in self.__sth if not sth is None ], but I don't know if that's the best way. I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. How can I get it the shortest and fastest way? The problem with the list comprehension you quote above is that it creates a new list, which costs both time and memory. You might want to try using a generator expression instead: (sth for sth in self.__sth if not sth is None) This will give you the same sequence of values, but will produce them only as they need to be consumed, saving the memory and compute overhead of creating a second list. lst = [1, None, 3, None, five, None] mylst = [l for l in lst if l is not None] mygen = (l for l in lst if l is not None) mylst, mygen ([1, 3, 'five'], generator object at 0x92f290) for l in mylst: print l, ... 1 3 five for l in mygen: print l, ... 1 3 five regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Dec 12, 8:08 am, Filip Gruszczyński grusz...@gmail.com wrote: I am not doing it, because I need it. I can as well use if not elem is None, but I just wanted to know, if there is some better way of doing this. I like to know :-) And I can't understand why you are becoming so aggressive because of this. Just because I asked for that, doesn't mean, that I will put some obfuscated code into my project. I just wanted to learn something new - I checked itertools, I googled a bit, now I wanted to ask here if someone knew some really cool way of this. All the other assumptions weren't really necessary. Thanks for those ideas, however. I like the last one a lot :) -- Filip Gruszczyński In this case the cool way is the straightforward way: either the list versions: [x fox x in seq if x is not None] filter(lambda x: x is not None, seq) or the generator versions: (x for x in seq if x is not None) itertools.ifilter(lambda x: x is not None, seq) -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Filip Gruszczyński wrote: I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. is this too much long? from itertools import ifilter for element in ifilter(lambda x: x is not None, [0,1,2,None,3,None,4]): ... print element ... 0 1 2 3 4 -- By ZeD -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. If you slip such an error in a post, I suggest to practice some time writing correct code before having one-liner contests with your perl-loving friends :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 16:51:15 +0100, Marco Mariani wrote: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. In which way is it not the same? Has the same behavior at least: In [256]: elem = None In [257]: not elem is None Out[257]: False In [258]: elem is not None Out[258]: False In [259]: elem = 42 In [260]: not elem is None Out[260]: True In [261]: elem is not None Out[261]: True If you slip such an error in a post, I suggest to practice some time writing correct code before having one-liner contests with your perl-loving friends :) If you call something an error, make sure it really is one. :-) Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? -- Kirk Strauser The Day Companies -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Kirk Strauser wrote: So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? Cause I was tired of course, and got the not precedente not right!! Argh -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Kirk Strauser wrote: At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? I suppose that it should really be not is (cp not in), but Guido chose to follow the English pattern is not. not returns a Boolean (or originally an int) and is checks for identity, but the result of is False or is True is down to implementation details in the interpreter, so you wouldn't write is not ... to mean is (not ...) anyway, and in practice it's not a problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Kirk Strauser wrote: At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? is not is an operator, so the parse is foo (is not) None not foo is (not None) regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
alex23 wuwe...@gmail.com writes: Rather than a list comprehension, use a generator expression: for item in (x for x in sequence if x is not None): do_something(x) I much prefer for item in sequence: if x is not None: do_something(x) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 11:50:38 -0500, Steve Holden wrote: Kirk Strauser wrote: At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? is not is an operator, so the parse is foo (is not) None not foo is (not None) Personally, I'd prefer VB's version: foo IsNot bar or in pseudo-python foo isnot bar since that would make it less ambiguous. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
If you want to literally remove None objects from a list(or mutable sequence) def deNone(alist): n=len(alist) i=j=0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j:i] = [] blist=[None,1,None,2,None,3,None,None,4,None] deNone(blist) print(blist) # prints [1, 2, 3, 4] Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
If you want to literally remove None objects from a list(or mutable sequence) def deNone(alist): n=len(alist) i=j=0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j:i] = [] blist=[None,1,None,2,None,3,None,None,4,None] deNone(blist) print(blist) # prints [1, 2, 3, 4] ...wouldn't a cleaner way of doing this just be blist=[None,1,None,2,None,3,None,None,4,None] alist = blist blist[:] = [x for x in blist if x is not None] blist [1, 2, 3, 4] alist [1, 2, 3, 4] By using the slice assignment, it leaves the blist referring to the same list-object (as shown by the alist bit), and modifying it in place. This reads a lot more cleanly in my estimation. If the data-set is large, in 2.5+, you can just use a generator: blist[:] = (x for x in blist if x is not None) -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 10:08:23 -0600, Kirk Strauser wrote:
At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes:
Filip Gruszczyński wrote:
I am not doing it, because I need it. I can as well use if not elem
is None,
I suggest if elem is not None, which is not quite the same.
So what's the difference exactly? foo is not None is actually
surprising to me, since not None is True. 0 is True is False, but
0 is not None is True. Why is that?
a is not b uses a single operator to do the test.
import dis
x = compile('a is not b', '', 'single')
dis.dis(x)
1 0 LOAD_NAME0 (a)
3 LOAD_NAME1 (b)
6 COMPARE_OP 9 (is not)
9 PRINT_EXPR
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
not a is b looks like it would use two operators (is, followed by not)
but wonderfully, Python has a peephole optimization that fixes that micro
inefficiency:
x = compile('not a is b', '', 'single')
dis.dis(x)
1 0 LOAD_NAME0 (a)
3 LOAD_NAME1 (b)
6 COMPARE_OP 9 (is not)
9 PRINT_EXPR
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
So if you are using at least Python 2.5, the two expressions don't just
return the same result, they actually generate the same byte code.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Fri, 12 Dec 2008 21:18:36 +, Lie Ryan wrote: On Fri, 12 Dec 2008 11:50:38 -0500, Steve Holden wrote: Kirk Strauser wrote: At 2008-12-12T15:51:15Z, Marco Mariani ma...@sferacarta.com writes: Filip Gruszczyński wrote: I am not doing it, because I need it. I can as well use if not elem is None, I suggest if elem is not None, which is not quite the same. So what's the difference exactly? foo is not None is actually surprising to me, since not None is True. 0 is True is False, but 0 is not None is True. Why is that? is not is an operator, so the parse is foo (is not) None not foo is (not None) Personally, I'd prefer VB's version: foo IsNot bar or in pseudo-python foo isnot bar since that would make it less ambiguous. a is not b is no more ambiguous than 1+2*3. True, there's ambiguity if you are ignorant of the precedence rules, but that's no worse than saying that + is ambiguous if you don't know what + means. What's this 'is' operator??? It's ambiguous, it could mean ANYTHING!!! Panic panic panic panic!!! *wink* You're allowed to assume the normal conventions, and (lucky for me!) despite being Dutch Guido choose to assume the normal English convention that a is not b means the same as not (a is b) rather than a is (not b). That's probably because the use-cases for the second would be rather rare. So given the normal precedence rules of Python, there is no ambiguity. True, you have to learn the rules, but that's no hardship. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Filip Gruszczyński wrote: I don't mean memory, but space in code ;-) Your goal should be clarity of code, not saving keystrokes. Writing something that is compact in terms of the amount of code to write does not mean its function is clear or even that it is more efficient to run, for that matter. -- Erik Max Francis m...@alcyone.com http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM, Y!M erikmaxfrancis God grant me to contend with those that understand me. -- Thomas Fuller -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Tim Chase wrote: If you want to literally remove None objects from a list(or mutable sequence) def deNone(alist): n=len(alist) i=j=0 while i n: if alist[i] is not None: alist[j] = alist[i] j += 1 i += 1 alist[j:i] = [] blist=[None,1,None,2,None,3,None,None,4,None] deNone(blist) print(blist) # prints [1, 2, 3, 4] ...wouldn't a cleaner way of doing this just be blist=[None,1,None,2,None,3,None,None,4,None] No, making a filtered copy that is then copied back before being deleted is algorithmically much messier. My code does the minimum work necessary and is algorithmically cleaner. alist = blist blist[:] = [x for x in blist if x is not None] blist [1, 2, 3, 4] alist [1, 2, 3, 4] By using the slice assignment, it leaves the blist referring to the same list-object (as shown by the alist bit), and modifying it in place. This reads a lot more cleanly in my estimation. If the data-set is large, in 2.5+, you can just use a generator: blist[:] = (x for x in blist if x is not None) Given that this works, that the setup for slice assignment does not mess up the co-routine iteration over the same loop, this is pretty good. It amounts to deNone with the i-j loop separated into an i loop and a j loop in source and consumer co-routines. One could argue that that is even better, even if written out as def deNone2(alist): src = (item for item in alist if item is not None) j=0 try: while True: alist[j] = next(src) j += 1 except StopIteration: pass alist[j:] = [] tjr -- http://mail.python.org/mailman/listinfo/python-list
