[Tutor] replacing a loop
Hi folks, I have the following loop (actually repeated many times ) def locChoices(self): locDS = self.eslocation.getDataSet() loc_Choices=[''] locKeys=[0] for row in locDS: loc_Choices.append(row['facility']) locKeys.append(row['pkid']) return loc_Choices,locKeys where locDS is a tuple of dicts and a row is a dict. Since I use a lot of similar loops to populate many dropdown controls I started investigating the use of list comprehensions. But I can't figure out how to use them in this loop and wonder if it will improve the performance. The data is not very big - about a thousand rows - give or take. So what do you guys think? Johnf ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
johnf wrote: > Hi folks, > > > I have the following loop (actually repeated many times ) If you have repetetive code look into ways to parameterize it, like def choices(rows, choices_column, keys_column): ... > > def locChoices(self): > locDS = self.eslocation.getDataSet() > loc_Choices=[''] > locKeys=[0] > for row in locDS: > loc_Choices.append(row['facility']) > locKeys.append(row['pkid']) > > return loc_Choices,locKeys > > where locDS is a tuple of dicts and a row is a dict. > > Since I use a lot of similar loops to populate many dropdown controls I > started investigating the use of list comprehensions. But I can't > figure out how to use them in this loop You need two loops in this case choices = [""] + [row["facility"] for row in ds] keys = [0] + [row["pkid"] for row in ds] > and wonder if it will improve > the performance. No, list comprehensions are unlikely to improve performance. > The data is not very big - about a thousand rows - > give or take. > > So what do you guys think? Are you sure (aka: did you measure that) building these lists takes a significant amount of time? If there is a GUI involved any delay you notice is more likely to stem from filling the widgets than from preparing the lists. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
On 6/24/19 10:15 AM, johnf wrote: . > > Since I use a lot of similar loops to populate many dropdown controls I > started investigating the use of list comprehensions. But I can't > figure out how to use them in this loop and wonder if it will improve > the performance. To amplify a tiny bit on what Peter said: comprehensions are a more concise way to express a loop-to-build-a-collection, but I don't believe were intended specifically as a performance improvement. They may be a little faster on the principle of "avoiding dots" - that is, the lookup every time through your loop from listname.append does take a little bit of time. And probably more significantly, append is a function that has to be called, and the setup stack frame/call/return sequence also takes some time. So depending on circumstances you may see insignificant or significant performance differences. But, again - Rule number one: only optimize when there is a proven speed bottleneck ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python type annotation question
I am little experimenting with Python type annotation today. I wrote a simple class as below: >>> from datetime import date >>> class Person: ... dob: date ... def __init__(self, dob): ... self.dob = dob ... >>> Person(11) <__main__.Person object at 0x10e078128> >>> p = Person(11) >>> p.dob 11 Although I marked dob as date type, why am I able to assign it an int? So my question how does type annotation helps, when should we use them and what advantages it brings? Thanks, Arup Rakshit a...@zeit.io ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python type annotation question
On 6/24/19 12:48 PM, Arup Rakshit wrote: > I am little experimenting with Python type annotation today. I wrote a simple > class as below: > from datetime import date class Person: > ... dob: date > ... def __init__(self, dob): > ... self.dob = dob > ... Person(11) > <__main__.Person object at 0x10e078128> p = Person(11) p.dob > 11 > > Although I marked dob as date type, why am I able to assign it an int? So my > question how does type annotation helps, when should we use them and what > advantages it brings? it's a mechansim to support static checking in various forms - tools, IDES, etc. It also serves to document your intent without having to write as much in the docstring. Python remains a "consenting adults" language, that is if you really want to do something, you can, so the type hints are not enforced at runtime. Since it's not always the case that doing something possible is a good idea, the type hints are there to make it easier to detect issues. Your annotation looks funky, by the way, I'd expect to see it written as: > ... def __init__(self, dob: date): ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
Actually I do not see a reply from Peter?? I don't have a clue what was said. I realize that performance is not a big issue in this case - of course an increase in speed is always welcome. I was more interested in a better understanding of the list comprehensions. Since I have so many of these loops I thought it might be best if I tried using comprehensions. Johnf On 6/24/19 10:48 AM, Mats Wichmann wrote: On 6/24/19 10:15 AM, johnf wrote: . Since I use a lot of similar loops to populate many dropdown controls I started investigating the use of list comprehensions. But I can't figure out how to use them in this loop and wonder if it will improve the performance. To amplify a tiny bit on what Peter said: comprehensions are a more concise way to express a loop-to-build-a-collection, but I don't believe were intended specifically as a performance improvement. They may be a little faster on the principle of "avoiding dots" - that is, the lookup every time through your loop from listname.append does take a little bit of time. And probably more significantly, append is a function that has to be called, and the setup stack frame/call/return sequence also takes some time. So depending on circumstances you may see insignificant or significant performance differences. But, again - Rule number one: only optimize when there is a proven speed bottleneck ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
Hi John, On 25/06/19 4:15 AM, johnf wrote: Hi folks, I have the following loop (actually repeated many times ) def locChoices(self): locDS = self.eslocation.getDataSet() loc_Choices=[''] locKeys=[0] for row in locDS: loc_Choices.append(row['facility']) locKeys.append(row['pkid']) return loc_Choices,locKeys where locDS is a tuple of dicts and a row is a dict. Since I use a lot of similar loops to populate many dropdown controls I started investigating the use of list comprehensions. But I can't figure out how to use them in this loop and wonder if it will improve the performance. The data is not very big - about a thousand rows - give or take. So what do you guys think? Just because this morning I needed something to kick-start my sleepy brain into being able to "think"... When most people think of 'multiples' in the context of list-comprehensions, they are talking about "nested-loops" - which are easy-enough (or at least, well-documented). However, I've often wondered about the multiple being on the 'left-hand side' of the equation/expression, and whether that is even possible? It is! Thus: python3 Python 3.6.8 (default, Mar 21 2019, 10:08:12) [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. # build the data-structure (I hope) >>> d1 = { 'pkid':1, 'facility':'here' } >>> d2 = { 'pkid':2, 'facility':'there' } >>> d3 = { 'pkid':3, 'facility':'nowhere' } >>> locDS = ( d1, d2, d3 ) # let's check that the data-format is as-described? >>> type( locDS ) >>> for row in locDS: print( row ) ... {'pkid': 1, 'facility': 'here'} {'pkid': 2, 'facility': 'there'} {'pkid': 3, 'facility': 'nowhere'} # are we on the right track? # this is the original code and approach >>> loc_Choices = [ '' ] >>> locKeys = [ 0 ] >>> for row in locDS: ... loc_Choices.append( row[ 'facility' ] ) ... locKeys.append( row[ 'pkid' ] ) ... # which (I hope) shows the existing (and desired) results >>> print( loc_Choices ) ['', 'here', 'there', 'nowhere'] >>> print( locKeys ) [0, 1, 2, 3] # we can do the list inits, cf leaving it to the list comprehension >>> loc_Choices = [ '' ] >>> locKeys = [ 0 ] # but how might we achieve this using a (single) list comprehension? >>> [ [ loc_Choices.append( row[ 'facility' ] ), locKeys.append( row[ 'pkid' ] ) ] for row in locDS ] # in (proper) code, I'd format this over two lines (at least) # correct me if I'm wrong, but these o/ps will # 'disappear into the ether' when run within a pgm... [[None, None], [None, None], [None, None]] # leaving us with 'the proof of the pudding' >>> print( loc_Choices ) ['', 'here', 'there', 'nowhere'] >>> print( locKeys ) [0, 1, 2, 3] >>> Possible considerations: 1 some sample data would have eased the way/its lack leaves room for misinterpretation 2 a list comprehension normally executes more quickly than the traditional (long-form) multi-line code-block. However, it would be worth measuring that on your particular machine h/w and Py__version__. The other 'efficiency' is readability, but "elegance" is largely a matter of (personal) taste. So that ('mine-field') I leave to you... 3 taking the multi-line code-block and splitting it into TWO separate (consecutive) list comprehensions (one for loc_Choices and the other for locKeys) will most-likely be noticeably MORE 'expensive' 4 it is no matter if row/the locDS tuple dict-elements contain more key-value pairs 5 during code review, my colleagues would delight in criticising the choice of names such as locKeys - describing them as "JavaScript" (a 'dirty word' to some), loc_Choices as contrary to PEP-8 ("flake", or whatever), etc - despite that I have a little 'set' of abbreviations with which I do the same, eg locNR. YMMV! 6 however, this simple boy will venture criticism of the inconsistency in using the underline word-separator, eg locKeys but loc_Choices. Even more potential for confusion: locChoices and loc_Choices!? 7 consider the data structures (outside of our view, here). Assuming there are more elements in each 'loc dict', might it be just as easy to leave the two pertinent elements 'there' and iterate over locDS when actually applied. Alternately, perhaps it might be better to construct a dictionary with the 'pkid's as keys and the 'facility' as values for direct-access application? Not knowing how the structure(s) will be utilised makes this pure speculation! 8 any?every time "similar loops" are found, re-factor to a function (let's see if that philosophy kicks-off a retort or two...) 9 was it Knuth who described "premature optimisation" as "evil"? Certainly I'm a believer in 'make it work before you make it better'. So, (after all that!) what are we really achieving here? With such quantities, is it critical to 'save time'? If so, is this part of the code really the greatest time-sink? (and we're back to t
Re: [Tutor] replacing a loop
Thank you - it worked! I'm glad you are no longer sleepy! Actually I wanted the naming to remain consistent with the other loops So the name of the function/method (it is in a class) caused the use of the underscore locChoices == location choices def locChoices(self) cause me to use loc_Choices in the code. I am torn about the readability of the code - at least in my mind I can read the loop a lot easier than I can read the comprehension. Of course I considered the use of a function where I passed the required parameters. But the issue is the access to the data - that getDataSet() was a performance hit when I used a passed parameter. I could have opened/accessed/retrieve the data during the opening but thought it best to use a lazy data access as needed. Today was not really about performance but more about learning to use the comprehension. You see I did attempt to do create the comprehension. But after seeing your code I realized that I had the '[' and the '{' confused. I believed I had to use '{'. I just reviewed a tutorial off the net and that was what they were using. Thanks again, Johnf Just because this morning I needed something to kick-start my sleepy brain into being able to "think"... When most people think of 'multiples' in the context of list-comprehensions, they are talking about "nested-loops" - which are easy-enough (or at least, well-documented). However, I've often wondered about the multiple being on the 'left-hand side' of the equation/expression, and whether that is even possible? It is! Thus: python3 Python 3.6.8 (default, Mar 21 2019, 10:08:12) [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information. # build the data-structure (I hope) >>> d1 = { 'pkid':1, 'facility':'here' } >>> d2 = { 'pkid':2, 'facility':'there' } >>> d3 = { 'pkid':3, 'facility':'nowhere' } >>> locDS = ( d1, d2, d3 ) # let's check that the data-format is as-described? >>> type( locDS ) >>> for row in locDS: print( row ) ... {'pkid': 1, 'facility': 'here'} {'pkid': 2, 'facility': 'there'} {'pkid': 3, 'facility': 'nowhere'} # are we on the right track? # this is the original code and approach >>> loc_Choices = [ '' ] >>> locKeys = [ 0 ] >>> for row in locDS: ... loc_Choices.append( row[ 'facility' ] ) ... locKeys.append( row[ 'pkid' ] ) ... # which (I hope) shows the existing (and desired) results >>> print( loc_Choices ) ['', 'here', 'there', 'nowhere'] >>> print( locKeys ) [0, 1, 2, 3] # we can do the list inits, cf leaving it to the list comprehension >>> loc_Choices = [ '' ] >>> locKeys = [ 0 ] # but how might we achieve this using a (single) list comprehension? >>> [ [ loc_Choices.append( row[ 'facility' ] ), locKeys.append( row[ 'pkid' ] ) ] for row in locDS ] # in (proper) code, I'd format this over two lines (at least) # correct me if I'm wrong, but these o/ps will # 'disappear into the ether' when run within a pgm... [[None, None], [None, None], [None, None]] # leaving us with 'the proof of the pudding' >>> print( loc_Choices ) ['', 'here', 'there', 'nowhere'] >>> print( locKeys ) [0, 1, 2, 3] >>> Possible considerations: 1 some sample data would have eased the way/its lack leaves room for misinterpretation 2 a list comprehension normally executes more quickly than the traditional (long-form) multi-line code-block. However, it would be worth measuring that on your particular machine h/w and Py__version__. The other 'efficiency' is readability, but "elegance" is largely a matter of (personal) taste. So that ('mine-field') I leave to you... 3 taking the multi-line code-block and splitting it into TWO separate (consecutive) list comprehensions (one for loc_Choices and the other for locKeys) will most-likely be noticeably MORE 'expensive' 4 it is no matter if row/the locDS tuple dict-elements contain more key-value pairs 5 during code review, my colleagues would delight in criticising the choice of names such as locKeys - describing them as "JavaScript" (a 'dirty word' to some), loc_Choices as contrary to PEP-8 ("flake", or whatever), etc - despite that I have a little 'set' of abbreviations with which I do the same, eg locNR. YMMV! 6 however, this simple boy will venture criticism of the inconsistency in using the underline word-separator, eg locKeys but loc_Choices. Even more potential for confusion: locChoices and loc_Choices!? 7 consider the data structures (outside of our view, here). Assuming there are more elements in each 'loc dict', might it be just as easy to leave the two pertinent elements 'there' and iterate over locDS when actually applied. Alternately, perhaps it might be better to construct a dictionary with the 'pkid's as keys and the 'facility' as values for direct-access application? Not knowing how the structure(s) will be utilised makes this pure speculation! 8 any?every time "similar loops" are found, re-factor to a function (l
Re: [Tutor] replacing a loop
On 6/24/19 4:24 PM, johnf wrote: > Thank you - it worked! I'm glad you are no longer sleepy! > > Actually I wanted the naming to remain consistent with the other loops > > So the name of the function/method (it is in a class) caused the use of > the underscore > > locChoices == location choices > > def locChoices(self) cause me to use loc_Choices in the code. > > I am torn about the readability of the code - at least in my mind I can > read the loop a lot easier than I can read the comprehension. That's actually fine. There are plenty of people who don't think comprehensions are more readable, at least unless they're really simple. It's an odd self-fulfilling prophecy: if you use them a lot, they usually start to feel more familiar. If you don't like the look of them, you don't tend to use them, and they never reach that point :) But it's also easy to abuse newer syntax - "newer must mean better" rather than "oh, here's an addition that maybe is cleaner in some cases". Do what works for you if you're the only person likely to look at the code later. If others may need to support it later, you may have to think a little harder about how to write - readability is not a binary choice. > Of course I considered the use of a function where I passed the required > parameters. But the issue is the access to the data - that getDataSet() > was a performance hit when I used a passed parameter. I could have > opened/accessed/retrieve the data during the opening but thought it best > to use a lazy data access as needed. > > Today was not really about performance but more about learning to use > the comprehension. You see I did attempt to do create the > comprehension. But after seeing your code I realized that I had the '[' > and the '{' confused. I believed I had to use '{'. I just reviewed a > tutorial off the net and that was what they were using. With curly braces it would be a dictionary comprehension. I just remembered I once saw a tutorial that I thought did a pretty nice job on comprehensions, and I had saved the link, again it's down to personal taste but take a gander at this to see if it makes sense: https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor