Re: [Tutor] replacing a loop
On 24/06/2019 17:15, johnf wrote: > def locChoices(self): > locDS = self.eslocation.getDataSet() > loc_Choices=[''] > locKeys=[0] > for row in locDS: > loc_Choices.append(row['facility']) > locKeys.append(row['pkid']) > > return loc_Choices,locKeys > ... and wonder if it will improve the performance. Because you are building two lists in one loop it probably won't do much for performance. It might even be worse. As always measure. But something else that might make a small difference is the pre-fetch of the locations. Why not: for location in self.eslocation.getDataSet(): You don't use the locDS value anywhere other than in the loop. Also the choice of method name is unfortunate in that you return both choices and keys but the method name implies only choices. Minor nit-picks... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
On 6/24/19 4:24 PM, johnf wrote:
> Thank you - it worked! I'm glad you are no longer sleepy!
>
> Actually I wanted the naming to remain consistent with the other loops
>
> So the name of the function/method (it is in a class) caused the use of
> the underscore
>
> locChoices == location choices
>
> def locChoices(self) cause me to use loc_Choices in the code.
>
> I am torn about the readability of the code - at least in my mind I can
> read the loop a lot easier than I can read the comprehension.
That's actually fine. There are plenty of people who don't think
comprehensions are more readable, at least unless they're really simple.
It's an odd self-fulfilling prophecy: if you use them a lot, they
usually start to feel more familiar. If you don't like the look of them,
you don't tend to use them, and they never reach that point :) But it's
also easy to abuse newer syntax - "newer must mean better" rather than
"oh, here's an addition that maybe is cleaner in some cases".
Do what works for you if you're the only person likely to look at the
code later. If others may need to support it later, you may have to
think a little harder about how to write - readability is not a binary
choice.
> Of course I considered the use of a function where I passed the required
> parameters. But the issue is the access to the data - that getDataSet()
> was a performance hit when I used a passed parameter. I could have
> opened/accessed/retrieve the data during the opening but thought it best
> to use a lazy data access as needed.
>
> Today was not really about performance but more about learning to use
> the comprehension. You see I did attempt to do create the
> comprehension. But after seeing your code I realized that I had the '['
> and the '{' confused. I believed I had to use '{'. I just reviewed a
> tutorial off the net and that was what they were using.
With curly braces it would be a dictionary comprehension.
I just remembered I once saw a tutorial that I thought did a pretty nice
job on comprehensions, and I had saved the link, again it's down to
personal taste but take a gander at this to see if it makes sense:
https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
Thank you - it worked! I'm glad you are no longer sleepy!
Actually I wanted the naming to remain consistent with the other loops
So the name of the function/method (it is in a class) caused the use of
the underscore
locChoices == location choices
def locChoices(self) cause me to use loc_Choices in the code.
I am torn about the readability of the code - at least in my mind I can
read the loop a lot easier than I can read the comprehension.
Of course I considered the use of a function where I passed the required
parameters. But the issue is the access to the data - that getDataSet()
was a performance hit when I used a passed parameter. I could have
opened/accessed/retrieve the data during the opening but thought it best
to use a lazy data access as needed.
Today was not really about performance but more about learning to use
the comprehension. You see I did attempt to do create the
comprehension. But after seeing your code I realized that I had the '['
and the '{' confused. I believed I had to use '{'. I just reviewed a
tutorial off the net and that was what they were using.
Thanks again,
Johnf
Just because this morning I needed something to kick-start my sleepy
brain into being able to "think"...
When most people think of 'multiples' in the context of
list-comprehensions, they are talking about "nested-loops" - which are
easy-enough (or at least, well-documented).
However, I've often wondered about the multiple being on the
'left-hand side' of the equation/expression, and whether that is even
possible?
It is!
Thus:
python3
Python 3.6.8 (default, Mar 21 2019, 10:08:12)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
# build the data-structure (I hope)
>>> d1 = { 'pkid':1, 'facility':'here' }
>>> d2 = { 'pkid':2, 'facility':'there' }
>>> d3 = { 'pkid':3, 'facility':'nowhere' }
>>> locDS = ( d1, d2, d3 )
# let's check that the data-format is as-described?
>>> type( locDS )
>>> for row in locDS: print( row )
...
{'pkid': 1, 'facility': 'here'}
{'pkid': 2, 'facility': 'there'}
{'pkid': 3, 'facility': 'nowhere'}
# are we on the right track?
# this is the original code and approach
>>> loc_Choices = [ '' ]
>>> locKeys = [ 0 ]
>>> for row in locDS:
... loc_Choices.append( row[ 'facility' ] )
... locKeys.append( row[ 'pkid' ] )
...
# which (I hope) shows the existing (and desired) results
>>> print( loc_Choices )
['', 'here', 'there', 'nowhere']
>>> print( locKeys )
[0, 1, 2, 3]
# we can do the list inits, cf leaving it to the list comprehension
>>> loc_Choices = [ '' ]
>>> locKeys = [ 0 ]
# but how might we achieve this using a (single) list comprehension?
>>> [ [ loc_Choices.append( row[ 'facility' ] ), locKeys.append( row[
'pkid' ] ) ] for row in locDS ]
# in (proper) code, I'd format this over two lines (at least)
# correct me if I'm wrong, but these o/ps will
# 'disappear into the ether' when run within a pgm...
[[None, None], [None, None], [None, None]]
# leaving us with 'the proof of the pudding'
>>> print( loc_Choices )
['', 'here', 'there', 'nowhere']
>>> print( locKeys )
[0, 1, 2, 3]
>>>
Possible considerations:
1 some sample data would have eased the way/its lack leaves room for
misinterpretation
2 a list comprehension normally executes more quickly than the
traditional (long-form) multi-line code-block. However, it would be
worth measuring that on your particular machine h/w and Py__version__.
The other 'efficiency' is readability, but "elegance" is largely a
matter of (personal) taste. So that ('mine-field') I leave to you...
3 taking the multi-line code-block and splitting it into TWO separate
(consecutive) list comprehensions (one for loc_Choices and the other
for locKeys) will most-likely be noticeably MORE 'expensive'
4 it is no matter if row/the locDS tuple dict-elements contain more
key-value pairs
5 during code review, my colleagues would delight in criticising the
choice of names such as locKeys - describing them as "JavaScript" (a
'dirty word' to some), loc_Choices as contrary to PEP-8 ("flake", or
whatever), etc - despite that I have a little 'set' of abbreviations
with which I do the same, eg locNR. YMMV!
6 however, this simple boy will venture criticism of the inconsistency
in using the underline word-separator, eg locKeys but loc_Choices.
Even more potential for confusion: locChoices and loc_Choices!?
7 consider the data structures (outside of our view, here). Assuming
there are more elements in each 'loc dict', might it be just as easy
to leave the two pertinent elements 'there' and iterate over locDS
when actually applied. Alternately, perhaps it might be better to
construct a dictionary with the 'pkid's as keys and the 'facility' as
values for direct-access application? Not knowing how the structure(s)
will be utilised makes this pure speculation!
8 any?every time "similar loops" are found, re-factor to a function
Re: [Tutor] replacing a loop
Hi John,
On 25/06/19 4:15 AM, johnf wrote:
Hi folks,
I have the following loop (actually repeated many times )
def locChoices(self):
locDS = self.eslocation.getDataSet()
loc_Choices=['']
locKeys=[0]
for row in locDS:
loc_Choices.append(row['facility'])
locKeys.append(row['pkid'])
return loc_Choices,locKeys
where locDS is a tuple of dicts and a row is a dict.
Since I use a lot of similar loops to populate many dropdown controls I
started investigating the use of list comprehensions. But I can't
figure out how to use them in this loop and wonder if it will improve
the performance. The data is not very big - about a thousand rows -
give or take.
So what do you guys think?
Just because this morning I needed something to kick-start my sleepy
brain into being able to "think"...
When most people think of 'multiples' in the context of
list-comprehensions, they are talking about "nested-loops" - which are
easy-enough (or at least, well-documented).
However, I've often wondered about the multiple being on the 'left-hand
side' of the equation/expression, and whether that is even possible?
It is!
Thus:
python3
Python 3.6.8 (default, Mar 21 2019, 10:08:12)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
# build the data-structure (I hope)
>>> d1 = { 'pkid':1, 'facility':'here' }
>>> d2 = { 'pkid':2, 'facility':'there' }
>>> d3 = { 'pkid':3, 'facility':'nowhere' }
>>> locDS = ( d1, d2, d3 )
# let's check that the data-format is as-described?
>>> type( locDS )
>>> for row in locDS: print( row )
...
{'pkid': 1, 'facility': 'here'}
{'pkid': 2, 'facility': 'there'}
{'pkid': 3, 'facility': 'nowhere'}
# are we on the right track?
# this is the original code and approach
>>> loc_Choices = [ '' ]
>>> locKeys = [ 0 ]
>>> for row in locDS:
... loc_Choices.append( row[ 'facility' ] )
... locKeys.append( row[ 'pkid' ] )
...
# which (I hope) shows the existing (and desired) results
>>> print( loc_Choices )
['', 'here', 'there', 'nowhere']
>>> print( locKeys )
[0, 1, 2, 3]
# we can do the list inits, cf leaving it to the list comprehension
>>> loc_Choices = [ '' ]
>>> locKeys = [ 0 ]
# but how might we achieve this using a (single) list comprehension?
>>> [ [ loc_Choices.append( row[ 'facility' ] ), locKeys.append( row[
'pkid' ] ) ] for row in locDS ]
# in (proper) code, I'd format this over two lines (at least)
# correct me if I'm wrong, but these o/ps will
# 'disappear into the ether' when run within a pgm...
[[None, None], [None, None], [None, None]]
# leaving us with 'the proof of the pudding'
>>> print( loc_Choices )
['', 'here', 'there', 'nowhere']
>>> print( locKeys )
[0, 1, 2, 3]
>>>
Possible considerations:
1 some sample data would have eased the way/its lack leaves room for
misinterpretation
2 a list comprehension normally executes more quickly than the
traditional (long-form) multi-line code-block. However, it would be
worth measuring that on your particular machine h/w and Py__version__.
The other 'efficiency' is readability, but "elegance" is largely a
matter of (personal) taste. So that ('mine-field') I leave to you...
3 taking the multi-line code-block and splitting it into TWO separate
(consecutive) list comprehensions (one for loc_Choices and the other for
locKeys) will most-likely be noticeably MORE 'expensive'
4 it is no matter if row/the locDS tuple dict-elements contain more
key-value pairs
5 during code review, my colleagues would delight in criticising the
choice of names such as locKeys - describing them as "JavaScript" (a
'dirty word' to some), loc_Choices as contrary to PEP-8 ("flake", or
whatever), etc - despite that I have a little 'set' of abbreviations
with which I do the same, eg locNR. YMMV!
6 however, this simple boy will venture criticism of the inconsistency
in using the underline word-separator, eg locKeys but loc_Choices. Even
more potential for confusion: locChoices and loc_Choices!?
7 consider the data structures (outside of our view, here). Assuming
there are more elements in each 'loc dict', might it be just as easy to
leave the two pertinent elements 'there' and iterate over locDS when
actually applied. Alternately, perhaps it might be better to construct a
dictionary with the 'pkid's as keys and the 'facility' as values for
direct-access application? Not knowing how the structure(s) will be
utilised makes this pure speculation!
8 any?every time "similar loops" are found, re-factor to a function
(let's see if that philosophy kicks-off a retort or two...)
9 was it Knuth who described "premature optimisation" as "evil"?
Certainly I'm a believer in 'make it work before you make it better'.
So, (after all that!) what are we really achieving here? With such
quantities, is it critical to 'save time'? If so, is this part of the
code really the greatest time-sink?
(and we're back to
Re: [Tutor] replacing a loop
Actually I do not see a reply from Peter?? I don't have a clue what was said. I realize that performance is not a big issue in this case - of course an increase in speed is always welcome. I was more interested in a better understanding of the list comprehensions. Since I have so many of these loops I thought it might be best if I tried using comprehensions. Johnf On 6/24/19 10:48 AM, Mats Wichmann wrote: On 6/24/19 10:15 AM, johnf wrote: . Since I use a lot of similar loops to populate many dropdown controls I started investigating the use of list comprehensions. But I can't figure out how to use them in this loop and wonder if it will improve the performance. To amplify a tiny bit on what Peter said: comprehensions are a more concise way to express a loop-to-build-a-collection, but I don't believe were intended specifically as a performance improvement. They may be a little faster on the principle of "avoiding dots" - that is, the lookup every time through your loop from listname.append does take a little bit of time. And probably more significantly, append is a function that has to be called, and the setup stack frame/call/return sequence also takes some time. So depending on circumstances you may see insignificant or significant performance differences. But, again - Rule number one: only optimize when there is a proven speed bottleneck ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
On 6/24/19 10:15 AM, johnf wrote: . > > Since I use a lot of similar loops to populate many dropdown controls I > started investigating the use of list comprehensions. But I can't > figure out how to use them in this loop and wonder if it will improve > the performance. To amplify a tiny bit on what Peter said: comprehensions are a more concise way to express a loop-to-build-a-collection, but I don't believe were intended specifically as a performance improvement. They may be a little faster on the principle of "avoiding dots" - that is, the lookup every time through your loop from listname.append does take a little bit of time. And probably more significantly, append is a function that has to be called, and the setup stack frame/call/return sequence also takes some time. So depending on circumstances you may see insignificant or significant performance differences. But, again - Rule number one: only optimize when there is a proven speed bottleneck ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing a loop
johnf wrote: > Hi folks, > > > I have the following loop (actually repeated many times ) If you have repetetive code look into ways to parameterize it, like def choices(rows, choices_column, keys_column): ... > > def locChoices(self): > locDS = self.eslocation.getDataSet() > loc_Choices=[''] > locKeys=[0] > for row in locDS: > loc_Choices.append(row['facility']) > locKeys.append(row['pkid']) > > return loc_Choices,locKeys > > where locDS is a tuple of dicts and a row is a dict. > > Since I use a lot of similar loops to populate many dropdown controls I > started investigating the use of list comprehensions. But I can't > figure out how to use them in this loop You need two loops in this case choices = [""] + [row["facility"] for row in ds] keys = [0] + [row["pkid"] for row in ds] > and wonder if it will improve > the performance. No, list comprehensions are unlikely to improve performance. > The data is not very big - about a thousand rows - > give or take. > > So what do you guys think? Are you sure (aka: did you measure that) building these lists takes a significant amount of time? If there is a GUI involved any delay you notice is more likely to stem from filling the widgets than from preparing the lists. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing while loop
Is this what you are looking for?
#!/usr/bin/python
'makeTextFile.py -- create text file'
import os
# get filename
#while True:
# fname = raw_input('Enter file name: ')
# if os.path.exists(fname):
# print*** ERROR: '%s' already exists % fname
# else:
# break
while True:
fname = raw_input('Enter file name: ')
try:
fobj = open(fname, 'r')
except:
break
# get file content (text) lines
all = []
print \nEnter lines ('.' by itself to quit).\n
# loop until user terminates input
while True:
entry = raw_input(' ')
if entry == '.':
break
else:
all.append(entry)
# write lines to file with NEWLINE line terminator
fobj = open(fname, 'w')
fobj.write('\n'.join(all))
fobj.close()
print 'DONE!'
--
Arvind Deshpande
On 9/8/07, Alan Gauld [EMAIL PROTECTED] wrote:
Christopher Spears [EMAIL PROTECTED] wrote
I have been asked to replace this while loop with a
try and except clause:
while True:
fname = raw_input('Enter file name: ')
if os.path.exists(fname):
print*** ERROR: '%s' already exists % fname
else:
break
I'm not sure how to do this. I looked at the back of
the book, and I don't see an exception that is raised
when a previously existing file is found. Any hints?
The loop simply detects if the file exists *or not*
If the file does not exist you exit the loop.
Can you find a way using try/except to detect
if the file does not exist?
That will replace the body of the while loop,
I can't think of any way to replace the loop itself
with try./except...
And I agree this is not an obvious place to use
try/except. Your earlier example is more typical.
--
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] replacing while loop
Christopher Spears [EMAIL PROTECTED] wrote
I have been asked to replace this while loop with a
try and except clause:
while True:
fname = raw_input('Enter file name: ')
if os.path.exists(fname):
print*** ERROR: '%s' already exists % fname
else:
break
I'm not sure how to do this. I looked at the back of
the book, and I don't see an exception that is raised
when a previously existing file is found. Any hints?
The loop simply detects if the file exists *or not*
If the file does not exist you exit the loop.
Can you find a way using try/except to detect
if the file does not exist?
That will replace the body of the while loop,
I can't think of any way to replace the loop itself
with try./except...
And I agree this is not an obvious place to use
try/except. Your earlier example is more typical.
--
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
