Re: dict.has_key(x) versus 'x in dict'
Hendrik == Hendrik van Rooyen [EMAIL PROTECTED] writes: Hendrik [EMAIL PROTECTED] wrote: Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. Hendrik LOL - this is kind of weird - it was working, nobody Hendrik complained, you fiddled with it to make it faster, (just Hendrik because you could, not because you had to, or were asked to), Hendrik it became faster, and then, suddenly, retrospectively, it Hendrik became a problem No, I think you misunderstand. I was poking around in that function for other reasons, saw the k in d.keys() and realized that the wrong way to write it. Rewrote it and noticed the performance increase. What's so weird about that? Skip -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Hendrik van Rooyen schreef: [EMAIL PROTECTED] wrote: Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. LOL - this is kind of weird - it was working, nobody complained, you fiddled with it to make it faster, (just because you could, not because you had to, or were asked to), it became faster, and then, suddenly, retrospectively, it became a problem Would it have been no problem if it so happened that you were unable to make it go faster? I don't really follow that logic - but then I have never believed that I could change yesterday... Have you never experienced the following: A customer reports a bug. Upon investaging you find the source of the problem, but from studying the code you don't understand anymore how it has ever been able to function correctly. From that moment, it indeed stops working even on computers where it always have worked correctly. You fix the bug and all is well again. Very strange, but it has happened to me on a few occasions. There's probably a perfectly logical explanation for what happened, but I never found it. -- If I have been able to see further, it was only because I stood on the shoulders of giants. -- Isaac Newton Roel Schroeven -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Roel Schroeven schreef: Have you never experienced the following: A customer reports a bug. Upon investaging you find the source of the problem, but from studying the code you don't understand anymore how it has ever been able to function correctly. From that moment, it indeed stops working even on computers where it always have worked correctly. You fix the bug and all is well again. Very strange, but it has happened to me on a few occasions. There's probably a perfectly logical explanation for what happened, but I never found it. BTW this is known as a Schroedinbug. -- If I have been able to see further, it was only because I stood on the shoulders of giants. -- Isaac Newton Roel Schroeven -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Roel Schroeven [EMAIL PROTECTED] wrote: Hendrik van Rooyen schreef: [EMAIL PROTECTED] wrote: Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. LOL - this is kind of weird - it was working, nobody complained, you fiddled with it to make it faster, (just because you could, not because you had to, or were asked to), it became faster, and then, suddenly, retrospectively, it became a problem Would it have been no problem if it so happened that you were unable to make it go faster? I don't really follow that logic - but then I have never believed that I could change yesterday... Have you never experienced the following: A customer reports a bug. Upon investaging you find the source of the problem, but from studying the code you don't understand anymore how it has ever been able to function correctly. From that moment, it indeed stops working even on computers where it always have worked correctly. You fix the bug and all is well again. Very strange, but it has happened to me on a few occasions. There's probably a perfectly logical explanation for what happened, but I never found it. This is simply a manifestation of the faith that can move mountains - while everybody believed that it was working, it did, and stopped working only because of the heretical doubt of some infidel... :-)- Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
[EMAIL PROTECTED] wrote: Peter Bjoern Schliessmann wrote: Wouldn't be if k in d.keys() be the exact replacement? Peter No, 'k in d' is equivalent to 'd.has_key(k)', only with less Peter (constant) overhead for the function call. 'k in d.keys()' on the Peter other hand creates a list of keys which is then searched linearly Peter -- about the worst thing you can do about both speed and memory Peter footprint. I will admit that way back when (maybe 8 yrs ago) I actually did this in a piece of frequently executed code that's been stable for a looong time. I have no idea why I might have written it that way. Brain fart I suppose. I only noticed my mistake a couple months ago during a trip into the containing function for some other stuff. Man, was I embarrassed... why? - the main point is actually that the code worked, and was stable - that should make you proud, not embarrassed. I think that there is far too much emphasis on doing things the quickest way - as long as it works, and is fast enough, its not broken, so don't fix it... - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
I will admit that way back when (maybe 8 yrs ago) I actually did this in a piece of frequently executed code that's been stable for a looong time. I have no idea why I might have written it that way. Brain fart I suppose. I only noticed my mistake a couple months ago during a trip into the containing function for some other stuff. Man, was I embarrassed... Hendrik why? - the main point is actually that the code worked, and was Hendrik stable - that should make you proud, not embarrassed. that Hendrik there is far too much emphasis on doing things the quickest way Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. Skip -- http://mail.python.org/mailman/listinfo/python-list
RE: dict.has_key(x) versus 'x in dict'
[EMAIL PROTECTED] wrote: Hendrik why? - the main point is actually that the code worked, and was Hendrik stable - that should make you proud, not embarrassed. that Hendrik there is far too much emphasis on doing things the quickest way Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. Yep - it's so often the case that scaling problems don't turn up in developer testing, but only in the field ... and sometimes in much less obvious ways than Skip's case. I had one where the particulars of the data I was using meant that what was meant to be an O(1) HashMap lookup (yes, Java - boo) was essentially an O(n) lookup much of the time. For a lot of the data, the key chosen ended up being the same (resulting in the hashmap having very long chains). By using additional data in the key as a tie-breaker, I got back my normal-case O(1) behaviour. In this particular case though ... dict.has_key(x) x in dict.keys() one is obviously clearer than the other, as well as being faster. Of course (getting back to the original question), in current python: x in dict is obviously the clearest, and fastest method, and should always be preferred over either of the above. Tim Delaney -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
[EMAIL PROTECTED] wrote: Hendrik - as long as it works, and is fast enough, its not broken, so Hendrik don't fix it... That's the rub. It wasn't fast enough. I only realized that had been a problem once I fixed it though. LOL - this is kind of weird - it was working, nobody complained, you fiddled with it to make it faster, (just because you could, not because you had to, or were asked to), it became faster, and then, suddenly, retrospectively, it became a problem Would it have been no problem if it so happened that you were unable to make it go faster? I don't really follow that logic - but then I have never believed that I could change yesterday... ;-) - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
dict.has_key(x) versus 'x in dict'
Hello,
I've always been using the has_key() method to test if a dictionary
contains a certain key. Recently I tried the same using 'in', e.g.
d = { ... }
if k in d:
...
and found that it seems to perform a lot better when lots of key-tests
are to be performed. I also noticed that has_key() is scheduled to be
removed from future (C)Python versions.
Does the 'in' way of testing have an optimized implementation compared
to has_key()? Is that the reason has_key() is being phased out?
Thanks,
Paul
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Paul Melis wrote:
I've always been using the has_key() method to test if a dictionary
contains a certain key. Recently I tried the same using 'in', e.g.
d = { ... }
if k in d:
...
and found that it seems to perform a lot better when lots of key-tests
are to be performed. I also noticed that has_key() is scheduled to be
removed from future (C)Python versions.
Does the 'in' way of testing have an optimized implementation compared
to has_key()?
no, but full method calls are a lot slower than the under-the-hood C-level
call used by the in operator.
this is why e.g.
string[:len(prefix)] == prefix
is often a lot faster than
string.startswith(prefix)
and why
if character in string:
string = string.replace(character, replacement)
is faster than
string = string.replace(character, replacement)
if the character isn't there most of the time, despite the fact that the
replace
method doesn't actually do something if the character isn't found.
/F
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Paul Melis wrote:
I've always been using the has_key() method to test if a
dictionary contains a certain key. Recently I tried the same using
'in', e.g.
d = { ... }
if k in d:
...
Wouldn't be if k in d.keys() be the exact replacement?
Regards,
Björn
--
BOFH excuse #17:
fat electrons in the lines
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Bjoern Schliessmann wrote: Wouldn't be if k in d.keys() be the exact replacement? no. that would convert an O(1) operation to an O(n) operation, which would be rather silly. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Bjoern Schliessmann wrote:
Paul Melis wrote:
I've always been using the has_key() method to test if a
dictionary contains a certain key. Recently I tried the same using
'in', e.g.
d = { ... }
if k in d:
...
Wouldn't be if k in d.keys() be the exact replacement?
No, 'k in d' is equivalent to 'd.has_key(k)', only with less (constant)
overhead for the function call. 'k in d.keys()' on the other hand creates a
list of keys which is then searched linearly -- about the worst thing you
can do about both speed and memory footprint. Some numbers:
$ python2.5 -m timeit -sN = 1; d = dict.fromkeys(range(N)) N in d.keys()
100 loops, best of 3: 0.693 usec per loop
$ python2.5 -m timeit -sN = 1000; d = dict.fromkeys(range(N)) N in
d.keys()
1 loops, best of 3: 77.2 usec per loop # ouch!
$ python2.5 -m timeit -sN = 1; d = dict.fromkeys(range(N)) N in d
1000 loops, best of 3: 0.192 usec per loop
~ $ python2.5 -m timeit -sN = 1000; d = dict.fromkeys(range(N)) N in d
1000 loops, best of 3: 0.192 usec per loop
$ python2.5 -m timeit -sN = 1; d = dict.fromkeys(range(N)) d.has_key(N)
100 loops, best of 3: 0.376 usec per loop
$ python2.5 -m timeit -sN = 1000; d = dict.fromkeys(range(N))
d.has_key(N)
100 loops, best of 3: 0.385 usec per loop
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Fredrik Lundh wrote: [...] this is why e.g. string[:len(prefix)] == prefix is often a lot faster than string.startswith(prefix) This is interesting. In which cases does the former form perform better? [I won't stop using str.startswith anyway :) ] Regards. -- Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Paul Melis wrote: I've always been using the has_key() method to test if a dictionary contains a certain key. Recently I tried the same using 'in', e.g. I've found using the set type to be the quickest way to do many of these tasks. That leads me to another problem: how to cast / convert sets as something else afterwards What's the best (i.e. Pythonic) way to do this? I need to generate a set (lots of intersections involved), but then I need to display it sorted lstBugsChanged = [ bugId for bugId in setBugsChanged ] lstBugsChanged.sort() -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Andy Dingley wrote: I need to generate a set (lots of intersections involved), but then I need to display it sorted lstBugsChanged = [ bugId for bugId in setBugsChanged ] lstBugsChanged.sort() lstBugsChanged = sorted(setBugsChanged) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Andy Dingley wrote: I need to generate a set (lots of intersections involved), but then I need to display it sorted lstBugsChanged = [ bugId for bugId in setBugsChanged ] lstBugsChanged.sort() In Python 2.4: sorted(setBugsChanged) -- Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
I wrote: In Python 2.4: lastline.replace(, =) -- Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Peter Otten wrote: No, 'k in d' is equivalent to 'd.has_key(k)', only with less (constant) overhead for the function call. Ah, thx. Thought the x in d syntax might search in d.values() too. Regards, Björn -- BOFH excuse #12: dry joints on cable plug -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Roberto Bonvallet wrote: this is why e.g. string[:len(prefix)] == prefix is often a lot faster than string.startswith(prefix) This is interesting. In which cases does the former form perform better? no time to doublecheck right now, but iirc, last time we benchmarked this, slicing was faster when len(prefix) 300 characters or so. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Bjoern Schliessmann wrote:
Peter Otten wrote:
No, 'k in d' is equivalent to 'd.has_key(k)', only with less
(constant) overhead for the function call.
Ah, thx. Thought the x in d syntax might search in d.values() too.
I don't think it does
Python 2.4.3 (#1, Nov 19 2006, 13:16:36)
[GCC 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type help, copyright, credits or license for more information.
d={1:'a',2:'b'}
1 in d
True
'a' in d
False
Paul
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Roberto Bonvallet wrote: lstBugsChanged = [ bugId for bugId in setBugsChanged ] In Python 2.4: Hmmm. Thanks. Another reason to twist the admin's arm and get them to upgrade the last 2.3.4 boxen sorted(setBugsChanged) Out of interest, whats the Pythonic way to simply cast (sic) the set to a list, assuming I don't need it sorted? The list comprehension? -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Andy Dingley wrote: Out of interest, whats the Pythonic way to simply cast (sic) the set to a list, assuming I don't need it sorted? The list comprehension? mySet = set(myList) myList = list(mySet) -- Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Andy Dingley wrote: sorted(setBugsChanged) Out of interest, whats the Pythonic way to simply cast (sic) the set to a list, assuming I don't need it sorted? The list comprehension? list(setBugsChanged) Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Peter Otten [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Andy Dingley wrote: sorted(setBugsChanged) Out of interest, whats the Pythonic way to simply cast (sic) the set to a list, assuming I don't need it sorted? The list comprehension? list(setBugsChanged) Peter Note that this is not really a cast in the C sense of the word. list(setBugsChanged) constructs and returns a new list containing the elements of setBugsChanges, and leaves setBugsChanged unchanged. -- Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Paul Melis wrote: I don't think it does Thanks for trying, I was too lazy ;) Regards, Björn -- BOFH excuse #52: Smell from unhygienic janitorial staff wrecked the tape heads -- http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Paul Melis [EMAIL PROTECTED] wrote:
Ah, thx. Thought the x in d syntax might search in d.values() too.
I don't think it does
Python 2.4.3 (#1, Nov 19 2006, 13:16:36)
[GCC 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type help, copyright, credits or license for more information.
d={1:'a',2:'b'}
1 in d
True
'a' in d
False
It is easy enough to to check if you remember that 'in' maps to the
__contains__ method:
help({}.__contains__)
Help on built-in function __contains__:
__contains__(...)
D.__contains__(k) - True if D has a key k, else False
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.has_key(x) versus 'x in dict'
Peter Bjoern Schliessmann wrote: Wouldn't be if k in d.keys() be the exact replacement? Peter No, 'k in d' is equivalent to 'd.has_key(k)', only with less Peter (constant) overhead for the function call. 'k in d.keys()' on the Peter other hand creates a list of keys which is then searched linearly Peter -- about the worst thing you can do about both speed and memory Peter footprint. I will admit that way back when (maybe 8 yrs ago) I actually did this in a piece of frequently executed code that's been stable for a looong time. I have no idea why I might have written it that way. Brain fart I suppose. I only noticed my mistake a couple months ago during a trip into the containing function for some other stuff. Man, was I embarrassed... Skip -- http://mail.python.org/mailman/listinfo/python-list
