[issue47094] index doesn't change while looping through same elements in a list

[Dennis Sweeney]

Dennis Sweeney  added the comment:

The help text says this:

>>> help(list.index)
Help on method_descriptor:

index(self, value, start=0, stop=9223372036854775807, /)
Return first index of value.

Raises ValueError if the value is not present.

Emphasis on *first* index. Example:

>>> L = [0, 10, 20, 33, 0, 10]
>>> L.index(10)
1
>>> L[5]
10
>>> L.index(L[5]) # the same meaning as L.index(10)
1

In your code, when elm has the value 1, it's just the value 1; there's no extra 
information carried along about where that 1 came from. If elm == 1, then 
my_list.index(elm) means the same as my_list.index(1).

I'd suggest taking any further questions to either StackOverflow or 
https://discuss.python.org/c/users/

Thanks for the concern, but I'm closing this as "not a bug". Changing this 
behavior now would be backwards-incompatible and break lots of people's code.

--
nosy: +Dennis Sweeney
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47094] index doesn't change while looping through same elements in a list

[Tk44]

New submission from Tk44 :

Let us define a list where there are some duplicate elements. If we loop 
through that list as "for i in my_list" and print the index by 
my_list.index(i); the index doesn't change.

e.g.

my_list = [1,1,1,1,3]
for elm in my_list:
   print(my_list.index(elm))

==output==
0
0
0
0
4

This occurs where elements are of type string as well. Of course this can be 
overcome by using enumerate(); however I think this is a wrong behavior.

--
messages: 415785
nosy: Tugberk
priority: normal
severity: normal
status: open
title: index doesn't change while looping through same elements in a list
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue47094>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Roundup Robot]

Change by Roundup Robot :


--
nosy: +python-dev
nosy_count: 5.0 -> 6.0
pull_requests: +21165
pull_request: https://github.com/python/cpython/pull/22078

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset eefd4e04a2a1d3929d0f7978469e5b5c4e56 by Rahul Kumaresan in 
branch 'master':
bpo-39705 : sorted() tutorial example under looping techniques improved 
(GH-18999)
https://github.com/python/cpython/commit/eefd4e04a2a1d3929d0f7978469e5b5c4e56


--

___
Python tracker 
<https://bugs.python.org/issue39705>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Rahul Kumaresan]

Change by Rahul Kumaresan :


--
keywords: +patch
pull_requests: +18346
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18999

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Rahul Kumaresan]

Rahul Kumaresan  added the comment:

I would like to work on this documentation improvement task.
Please help me understand if this is not being worked on already.

--
nosy: +rahul-kumi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Eric V. Smith]

Eric V. Smith  added the comment:

That sounds like a good improvement, Raymond.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I prefer to keep the example as-is.  It is an idiomatic way to loop over sets.  
The introductory text can be modified to explain that sets eliminate 
duplicates, that sets are ordered, and that sorted() puts them back in a 
deterministic order.

--
assignee: docs@python -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Eric V. Smith]

Eric V. Smith  added the comment:

The code is converting to a set first, then calls sorted() on that set. So 
"apple" is removed when the set is created.

I'm not sure the example should throw in creating a set while it's talking 
about sorting.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39705] Tutorial, 5.6 Looping Techniques, sorted() example

[Mirwi]

New submission from Mirwi :

>>> basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
>>> for f in sorted(set(basket)):
... print(f)
...
apple
banana
orange
pear

Shouldn't 'apple' appear two times as basket is a list that allows duplicates, 
not a set?

I'm just doing my first steps into Python and may be mislead. In that case, 
sorry for the fuzz.

--
assignee: docs@python
components: Documentation
messages: 362395
nosy: docs@python, mirwi
priority: normal
severity: normal
status: open
title: Tutorial, 5.6 Looping Techniques, sorted() example
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39705>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset b6341e676af2f58f3ad9b51a0d2fb7db5a3428e3 by Raymond Hettinger 
(Miss Islington (bot)) in branch '3.8':
bpo-30826: Improve control flow examples (GH-15407) (GH-15410)
https://github.com/python/cpython/commit/b6341e676af2f58f3ad9b51a0d2fb7db5a3428e3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[miss-islington]

Change by miss-islington :


--
pull_requests: +15115
pull_request: https://github.com/python/cpython/pull/15410

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 6fcb6cfb139ade1aac6dbee0b18ca72b18cbe0d2 by Raymond Hettinger in 
branch 'master':
bpo-30826: Improve control flow examples (GH-15407)
https://github.com/python/cpython/commit/6fcb6cfb139ade1aac6dbee0b18ca72b18cbe0d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +patch
pull_requests: +15112
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15407

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32015] Asyncio looping during simultaneously socket read/write and reconnection

[Andrew Svetlov]

Andrew Svetlov <andrew.svet...@gmail.com> added the comment:


New changeset cc0961c517c31578f6a40a4dc7ea177d62c256b7 by Andrew Svetlov in 
branch '3.6':
[3.6] bpo-32015: Asyncio looping during simultaneously socket read/write an… 
(GH-4386) (#4393)
https://github.com/python/cpython/commit/cc0961c517c31578f6a40a4dc7ea177d62c256b7


--

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32015>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32015] Asyncio looping during simultaneously socket read/write and reconnection

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +4341

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32015] Asyncio looping during simultaneously socket read/write and reconnection

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32015] Asyncio looping during simultaneously socket read/write and reconnection

[Andrew Svetlov]

Andrew Svetlov <andrew.svet...@gmail.com> added the comment:


New changeset e1d62e0b7cc842d6b75b4d480391f4a94e503255 by Andrew Svetlov 
(Andrey Egorov) in branch 'master':
bpo-32015: Asyncio looping during simultaneously socket read/write an… (#4386)
https://github.com/python/cpython/commit/e1d62e0b7cc842d6b75b4d480391f4a94e503255


--
nosy: +asvetlov

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32015>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32015] Asyncio looping during simultaneously socket read/write and reconnection

[Andrey]

Change by Andrey <and...@gmail.com>:


--
title: Asyncio cycling during simultaneously socket read/write and reconnection 
-> Asyncio looping during simultaneously socket read/write and reconnection

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32015>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Looping on a list in json

[Sayth Renshaw]

no doubt tho after playing with this is that enumerate value ends up in the 
output which is a dictionary. The enumerate has no key which makes it invalid 
json if dumped.  

Not massive issue but getting the effect of enumerate without polluting output 
would be the winner.


>runner_lists = {} 
>for n, item in enumerate(result): 
># if this one is interested / not -filtered: 
>print(n, item) 
>runner_lists[n] = result[n]["RacingFormGuide"]["Event"]["Runners"] 

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping on a list in json

[Cameron Simpson]

On 04Nov2017 17:43, Sayth Renshaw  wrote:

figured it. Needed to use n to iterate when creating.


Yeah, my mistake.


runner_lists = {}
   for n, item in enumerate(result):
   # if this one is interested / not -filtered:
   print(n, item)
   runner_lists[n] = result[n]["RacingFormGuide"]["Event"]["Runners"]


That's the beauty of offering untested code: I don't have to find and fix my 
errors:-)


Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping on a list in json

[Sayth Renshaw]

Sorry

figured it. Needed to use n to iterate when creating.

runner_lists = {}
for n, item in enumerate(result):
# if this one is interested / not -filtered:
print(n, item)
runner_lists[n] = result[n]["RacingFormGuide"]["Event"]["Runners"]

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping on a list in json

[Sayth Renshaw]

> I'd just keep the interesting runners, along with their race numbers, in a 
> dict. The enumerate function is handy here. Something like (untested):
> 
>   runner_lists = {}
>   for n, item in enumerate(result):
> if this one is interested/not-filtered:
>   runner_lists[n] = result["RacingFormGuide"]["Event"]["Runners"]
> 
> and just return runner_lists. That way you know what the race numbers were 
> for 
> each list of runners.
> 

> 
> Cheers,
> Cameron Simpson 

The main issue is that enumerate doesn't enumerate on the lists when trying to 
filter.

result = meeting_id["Races"]

so yes enumerating this works, showing just printing n and item.

runner_lists = {}
for n, item in enumerate(result):
# if this one is interested / not -filtered:
print(n, item)

0 {'FeatureRaceBonusActive': 'Disabled', 'FixedPriceSummary': {'FixedPrices': 
[{'SportId': 8, 'LeagueId': 102, 'MeetingId': 1218, 'MainEventId': 650350, 
'SubEventId': 3601361, 'Status': 'F', 'StatusDescription': 'FINALISED', 
'BetTypeName': 'Win', 'EnablePlaceBetting': True}]}, 'RacingFormGuide': 
{'Copyright': .

and so on it goes through the 7 items in this file.

but including 

runner_lists = {}
for n, item in enumerate(result):
# if this one is interested / not -filtered:
print(n, item)
runner_lists[n] = result["RacingFormGuide"]["Event"]["Runners"]

## Produces

Traceback (most recent call last):
dict_keys(['RaceDay', 'ErrorInfo', 'Success'])
  File "/home/sayth/PycharmProjects/ubet_api_mongo/parse_json.py", line 31, in 

runner_lists[n] = result["RacingFormGuide"]["Event"]["Runners"]
TypeError: list indices must be integers or slices, not str


Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping on a list in json

[Sayth Renshaw]

On Sunday, 5 November 2017 09:53:37 UTC+11, Cameron Simpson  wrote:

> >I want to get a result from a largish json api. One section of the json 
> >structure returns lists of data. I am wanting to get each resulting list 
> >returned.
> >
> >This is my code.
> >import json
> >from pprint import pprint
> >
> >with open(r'/home/sayth/Projects/results/Canterbury_2017-01-20.json', 'rb') 
> >as f, open('socks3.json','w') as outfile:
> >to_read = json.load(f)
> [...]
> >meeting_id = to_read["RaceDay"]["Meetings"][0]
> >result = meeting_id["Races"]
> >#failing
> >for item in result:
> >pprint(["RacingFormGuide"]["Event"]["Runners"])
> 
> I'd just keep the interesting runners, along with their race numbers, in a 
> dict. The enumerate function is handy here. Something like (untested):
> 
>   runner_lists = {}
>   for n, item in enumerate(result):
> if this one is interested/not-filtered:
>   runner_lists[n] = result["RacingFormGuide"]["Event"]["Runners"]
> 
> and just return runner_lists. That way you know what the race numbers were 
> for 
> each list of runners.
> 
> >What is the best way to and return the data?
> 
> The basic idea is to make a small data structure of your own (just the 
> dictionary runner_lists in the example above) and fill it in with the 
> infomation you care about in a convenient and useful shape. Then just return 
> the data structure.
> 
> The actual data structure will depend on what you need to do with this later.
> 
> Cheers,


Thank you. That does seem a good approach. 

I was intending to merge other dictionary data from other dicts within the json 
structure and that's where the trouble starts i guess trying to get too much 
from json.

Thanks

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping on a list in json

[Cameron Simpson]

On 04Nov2017 14:01, Sayth Renshaw  wrote:
I want to get a result from a largish json api. One section of the json 
structure returns lists of data. I am wanting to get each resulting list 
returned.


This is my code.
import json
from pprint import pprint

with open(r'/home/sayth/Projects/results/Canterbury_2017-01-20.json', 'rb') as 
f, open('socks3.json','w') as outfile:
   to_read = json.load(f)

[...]

   meeting_id = to_read["RaceDay"]["Meetings"][0]
   result = meeting_id["Races"]
   #failing
   for item in result:
   pprint(["RacingFormGuide"]["Event"]["Runners"])


I'd just keep the interesting runners, along with their race numbers, in a 
dict. The enumerate function is handy here. Something like (untested):


 runner_lists = {}
 for n, item in enumerate(result):
   if this one is interested/not-filtered:
 runner_lists[n] = result["RacingFormGuide"]["Event"]["Runners"]

and just return runner_lists. That way you know what the race numbers were for 
each list of runners.



What is the best way to and return the data?


The basic idea is to make a small data structure of your own (just the 
dictionary runner_lists in the example above) and fill it in with the 
infomation you care about in a convenient and useful shape. Then just return 
the data structure.


The actual data structure will depend on what you need to do with this later.

Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
--
https://mail.python.org/mailman/listinfo/python-list

Looping on a list in json

[Sayth Renshaw]

Hi

I want to get a result from a largish json api. One section of the json 
structure returns lists of data. I am wanting to get each resulting list 
returned.

This is my code.
import json
from pprint import pprint

with open(r'/home/sayth/Projects/results/Canterbury_2017-01-20.json', 'rb') as 
f, open('socks3.json','w') as outfile:
to_read = json.load(f)


print(to_read.keys())
# pprint(to_read)
meet = to_read["RaceDay"]["Meetings"]
meeting_id = to_read["RaceDay"]["Meetings"][0]
pprint(meeting_id.keys())
# result = meeting_id["Races"][1]["RacingFormGuide"]["Event"]["Runners"]
result = meeting_id["Races"]
#failing
for item in result:
pprint(["RacingFormGuide"]["Event"]["Runners"])


The key to the issue is that
result = meeting_id["Races"][0]["RacingFormGuide"]["Event"]["Runners"]
result = meeting_id["Races"][1]["RacingFormGuide"]["Event"]["Runners"]
result = meeting_id["Races"][2]["RacingFormGuide"]["Event"]["Runners"]

the numbers though in the above results could go from 0 to 10.

What is the best way to and return the data?
 would just save meeting_id["Races"] to my result however there are a lot of 
other junk dictionaries and lists I am filtering.

Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

On Wed, 11 Oct 2017 10:57 pm, Stefan Ram wrote:

>  FWIW, in is book "Touch of Class" (2009) Bertrand Meyer writes:
> 
> |Such instructions are just the old goto in sheep's clothing.
> |Treat them the same way as the original:
> |
> |/Touch of Methodology/:
> | Sticking to one-entry, one-exit building blocks
> |Stay away from any "break" or similar control mechanism.

I have a great deal of respect for Meyer, but in this case I think he has it
badly wrong on both counts (`break` and "one-entry, one-exit").

Unrestricted GOTO (as in BASIC, circa 1970, where you could jump to any line
in the code, or as in assembly) is rightly considered harmful (albeit
necessary in assembly). But if Meyer means *any* sort of jump ("similar
control mechanism"), then that would rule out not just `break` and `continue`
but also:

* while loops
* for loops
* if...else
* case/switch statements
* calling subroutines (functions or procedures)
* exception handling

Remember that a function call is basically a GOTO in disguise: execution jumps
to the function, and jumps back when the function returns. In BASIC, there
was a GOSUB intermediate between a GOTO and a procedure call.

Surely Meyer doesn't mean to say we should never call functions or use
`while`, `for` or `if`. So he has to distinguish between kinds of jumps:

Good jumps (I presume):

* function calls
* if...else
* looping

Evil jumps:

* unrestricted BASIC-style GOTO/GOSUB any line number
* break/continue

Not sure:

* GOTO where there are restrictions on where you can jump
* COMEFROM

(I kid: I'm sure Meyer would oppose COMEFROM, and I expect that even
Pascal-style restricted GOTO would be on his "evil" list to avoid.)

So the question becomes, why are such harmless, simple to understand,
innocuous jumps like `break` and `continue` in the evil list, when they not
only simplify code but make it more efficient?

# with break
for i in range(2**64):
if isprime(i):
print(i, "is prime")
break

# without break
still_searching = True
for i in range(2**64):
if still_searching and isprime(i):
print(i, "is prime")
still_searching = False

# without break, attempt number 2
still_searching = True
i = 0
while still_searching and i < 2**64:
if isprime(i):
print(i, "is prime")
still_searching = False


Unrestricted jumps *into* a subroutine are hard to reason about. In general,
subroutines should have a single entry point. But the requirement for one
exit is too strict. From the caller's perspective, there is no way to tell
how many exit point a function has: subroutines are black boxes that the
caller cannot see into, and the difference between a single exit and multiple
exits is invisible.

But from the point of view of the subroutines, the rule "one exit" is like the
rule "no break" for loops: it makes the code more complex and less efficient.
If you're done, you're done, and you might as well return out of the function
rather than write boilerplate code to pad it out until you get to the very
end. With only a single condition to test, there's not much difference
between the two:

# with single exit  # with multiple exits
def foo():  def foo():
if condition:   if condition:
result = 1  return 1
else:   return 2
result = 2
return result


but as the number of decision points increase, the complexity required to keep
a single exit also increases.
  
Ironically, after telling us to stick to code with one entry and one exit,
Meyer then contradicts himself by recommending exceptions:

> |You can use exception handling as a technique of last resort
> |to handle unexpected events for which the normal control
> |structures let you down.

Even more ironically, exception handling is most similar to a COMEFROM, which
was invented as a joke.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python console menu level looping

[ROGER GRAYDON CHRISTMAN]

On Mon, Sep 24, 2017 09:41 PM, Daiyue Weng  wrote:
>
Hi, I tried to make a menu using print statements in Python 3. The code is
>as follows,
>
>print('insert data into: ')
>data_insert_method = ['new collection', 'existing collection']
>for index, elem in enumerate(data_insert_method):
>print(index, '-', elem)
>
>while 1:
>how_to_insert = input('Choose a method for inserting data: ')
>if how_to_insert:
>try:
>how_to_insert = int(how_to_insert)
>how_to_insert = data_insert_method[how_to_insert]
>break
>except ValueError:
>print('Please type in an integer corresponding to the data insert method')
>continue
>except IndexError:
>print('insert method index out of range')
>continue
>
>if how_to_insert == 'new collection':
>
>print('Set up collection name : ')
>db_name_setup = ['manual setup', 'return to the main menu']
>for index, elem in enumerate(db_name_setup):
>print(index, '-', elem)
>
>while 1:
>how_to_setup = input('Choose a method for setting up collection: ')
>if how_to_setup:
>try:
>how_to_setup = int(how_to_setup)
>how_to_setup = db_name_setup[how_to_setup]
>break
>except ValueError:
>print('Please type in an integer corresponding to the collection setup
>method')
>continue
>except IndexError:
>print('collection setup method index out of range')
>continue
>
>if how_to_setup == 'manual setup':
>print('Please type in collection name: ')
>elif how_to_setup == 'return to main menu':
>print('return to main menu')
>
>It is a 2-level menu, on the 2nd level menu 'new collection', there is an
>option - 'return to the main menu', I am wondering how to loop back to the
>1st level menu, i.e. ['new collection', 'existing collection'],
>whenever 'return
>to the main menu' is selected.
>



As one of those pesky teacher purists, my first response is 'ick'.

I see 'break' statements used for 'normal' loop exits.

I see 'continue' statements that have no effect, so serve no purpose.

I see lists being used just for display purposes, and not to actually solve any
problem

and then the question about 'how to loop back' at the end

-- if you want to go from the bottom of a loop to the top of loop,

all you have to do is make sure they are the same loop


I would take this approach:


I am inserting .'s to indent, because I do not trust my mailer

to preserve the indentation


data_insert_method = ['exit program',new collection','existing collection']   

db_name_setup = ['return to previous menu','manual setup']


first_menu = -1# feed the first loop

while first_menu != 0::

. . for counterin range(len(data_insert_method)):

. . . . index = (counter+1) % len(data_insert_method)

. . . . print(index,'-',data_insert_method(index)

. . . . # the quit option is always 0, and appears last

. . . . # obtain and verify integer input here

. . . . if first_menu == 1:# new collection

. . . . . . second_menu = -1

. . . . . . while second_menu != 0:

. . . . . . . . # the same thing as the outer loop, just nested

. . . . . . . < trim >

. . . . elif first_menu == 2:# existing collection



There are still some wide open design problems here:

It seems the only difference between 'new collection' and

'existing collection' is whether data already exists.

But the code structure as presented seems to suggest that

there will be one whole section to the program that works

on an empty set, and different whole section that works on

a non-emtpty set -- which suggests a humongous amount

of unnecessary code duplication.


Do you really expect to go back to the outer menu for

'new collection' more than once, when running this program,

or does it seem more likely that initalizing an empty data base

would happen only once at program startup, after which it

behaves just like an 'existing collection' with less data?



As someone else pointed out, your tests along the lines of

if how_to_insert == 'new collection':

are error prone, just from typing errors.If you are going to

have a user type in discrete integers, you might as well

just test using those integers.


On the other hand, if you intend to make the code's decisions

appear to be self-documenting, then skip the integers completely,

and instead use string inputs, like single letters or short keywords.

That would then eliminate any worry about ValueErrors.

A dictionary can easily associate the short inputs with the

longer method descriptions, and could even conceivably

link directly to a function object.


Roger Christman

Pennsylvania State University


 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python console menu level looping

[Cameron Simpson]

On 24Sep2017 21:41, Daiyue Weng  wrote:

Hi, I tried to make a menu using print statements in Python 3. The code is
as follows,


One thing, please try to preserve the code indenting in messages. What you 
pasted is all hard against the left side of the screen. I've tried to repair 
it.



print('insert data into: ')
data_insert_method = ['new collection', 'existing collection']
for index, elem in enumerate(data_insert_method):
 print(index, '-', elem)

while 1:


Remark: we tend to say "while True:", it reads more naturally.


how_to_insert = input('Choose a method for inserting data: ')
if how_to_insert:
 try:
   how_to_insert = int(how_to_insert)
   how_to_insert = data_insert_method[how_to_insert]
   break
 except ValueError:
   print('Please type in an integer corresponding to the data insert method')
   continue
 except IndexError:
   print('insert method index out of range')
   continue


Another remark: it is better to keep try/except around as small a piece of code 
as possible; I'd use two here, one for ValueError around the int() and one for 
IndexError around the [].



 if how_to_insert == 'new collection':
   print('Set up collection name : ')
   db_name_setup = ['manual setup', 'return to the main menu']
   for index, elem in enumerate(db_name_setup):
 print(index, '-', elem)
   while 1:
 how_to_setup = input('Choose a method for setting up collection: ')
 if how_to_setup:
   try:
 how_to_setup = int(how_to_setup)
 how_to_setup = db_name_setup[how_to_setup]
 break
   except ValueError:
 print('Please type in an integer corresponding to the collection setup 
method')
 continue
   except IndexError:
 print('collection setup method index out of range')
 continue
   if how_to_setup == 'manual setup':
 print('Please type in collection name: ')
   elif how_to_setup == 'return to main menu':
 print('return to main menu')

It is a 2-level menu, on the 2nd level menu 'new collection', there is an
option - 'return to the main menu', I am wondering how to loop back to the
1st level menu, i.e. ['new collection', 'existing collection'],
whenever 'return to the main menu' is selected.


This is where the lack of indentation in your posted code bites; I've just 
realised I've misread your logic because I misindented your code :-( Your 
"while 1:" is just a loop around the input question, not around the menu 
choices.


I would go with a flag, and use it as your loop condition. Like this:

 running_inner_menu = True
 while running_inner_menu:

So something like this (untested):

 if how_to_insert == 'new collection':
   running_inner_menu = True
   while running_inner_menu:
 print('Set up collection name : ')
 db_name_setup = ['manual setup', 'return to the main menu']
 for index, elem in enumerate(db_name_setup):
   print(index, '-', elem)
 # obtain a valid choice
 while 1:
   how_to_setup = input('Choose a method for setting up collection: ')
   if how_to_setup:
 try:
   how_to_setup = int(how_to_setup)
   how_to_setup = db_name_setup[how_to_setup]
   break
 except ValueError:
   print('Please type in an integer corresponding to the collection 
   setup thod')

   continue
 except IndexError:
   print('collection setup method index out of range')
   continue
 if how_to_setup == 'manual setup':
   print('Please type in collection name: ')
   ... etc ...
 elif how_to_setup == 'return to main menu':
   print('return to main menu')
   running_inner_menu = False

All this depends a little on how your code was originally indented i.e. exactly 
which portions of the code were within each while loop. I've guessed at 
something reasonable, but can see that there's room for different arrangements.


But basicly, keep a state variable indicating that a loop should continue. As a 
piece of terminology, a Boolean (True/False) state variable is often called a 
flag.


This has two advantages:

It means you can turn it off anywhere and next time around the loop will stop, 
whereas doing things directly with continue or break requires "structural" 
control because they only jump out of the innermost loop, and also they act 
right now, preventing other relevant stuff being done.


Also it makes your code more readable and easier to reason able: a loop which 
says "while running_inner_menu:" directly says that this loop controls a 
repeating menu, and that you can get out by changing "running_inner_menu". A 
bare "while 1:" or "while True:" requires the reader to have to scan the code 
to figure out what is being controlled.


Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
--
https://mail.python.org/mailman/listinfo/python-list

Re: python console menu level looping

[Daiyue Weng]

Sry for the unclear formatting, this is the original code with correct
format (copy from pycharm),

print('insert data into: ')
data_insert_method = ['new collection', 'existing collection']
for index, elem in enumerate(data_insert_method):
print(index, '-', elem)

while 1:
how_to_insert = input('Choose a method for inserting data: ')
if how_to_insert:
try:
how_to_insert = int(how_to_insert)
how_to_insert = data_insert_method[how_to_insert]
break
except ValueError:
print('Please type in an integer corresponding to the data
insert method')
continue
except IndexError:
print('insert method index out of range')
continue

if how_to_insert == 'new collection':

print('Set up collection name : ')
db_name_setup = ['manual setup', 'return to main menu']
for index, elem in enumerate(db_name_setup):
print(index, '-', elem)

while 1:
how_to_setup = input('Choose a method for setting up collection: ')
if how_to_setup:
try:
how_to_setup = int(how_to_setup)
how_to_setup = db_name_setup[how_to_setup]
break
except ValueError:
print('Please type in an integer corresponding to the
collection setup method')
continue
except IndexError:
print('db name setup method index out of range')
continue

if how_to_setup == 'manual setup':
print('Please type in collection name ')
elif how_to_setup == 'return to main menu':
print('return to main menu')



On 24 September 2017 at 22:11, Cameron Simpson  wrote:

> On 24Sep2017 21:41, Daiyue Weng  wrote:
>
>> Hi, I tried to make a menu using print statements in Python 3. The code is
>> as follows,
>>
>
> One thing, please try to preserve the code indenting in messages. What you
> pasted is all hard against the left side of the screen. I've tried to
> repair it.
>
> print('insert data into: ')
>> data_insert_method = ['new collection', 'existing collection']
>> for index, elem in enumerate(data_insert_method):
>>  print(index, '-', elem)
>>
>> while 1:
>>
>
> Remark: we tend to say "while True:", it reads more naturally.
>
> how_to_insert = input('Choose a method for inserting data: ')
>> if how_to_insert:
>>  try:
>>how_to_insert = int(how_to_insert)
>>how_to_insert = data_insert_method[how_to_insert]
>>break
>>  except ValueError:
>>print('Please type in an integer corresponding to the data insert
>> method')
>>continue
>>  except IndexError:
>>print('insert method index out of range')
>>continue
>>
>
> Another remark: it is better to keep try/except around as small a piece of
> code as possible; I'd use two here, one for ValueError around the int() and
> one for IndexError around the [].
>
>
>  if how_to_insert == 'new collection':
>>print('Set up collection name : ')
>>db_name_setup = ['manual setup', 'return to the main menu']
>>for index, elem in enumerate(db_name_setup):
>>  print(index, '-', elem)
>>while 1:
>>  how_to_setup = input('Choose a method for setting up collection: ')
>>  if how_to_setup:
>>try:
>>  how_to_setup = int(how_to_setup)
>>  how_to_setup = db_name_setup[how_to_setup]
>>  break
>>except ValueError:
>>  print('Please type in an integer corresponding to the collection
>> setup method')
>>  continue
>>except IndexError:
>>  print('collection setup method index out of range')
>>  continue
>>if how_to_setup == 'manual setup':
>>  print('Please type in collection name: ')
>>elif how_to_setup == 'return to main menu':
>>  print('return to main menu')
>>
>> It is a 2-level menu, on the 2nd level menu 'new collection', there is an
>> option - 'return to the main menu', I am wondering how to loop back to the
>> 1st level menu, i.e. ['new collection', 'existing collection'],
>> whenever 'return to the main menu' is selected.
>>
>
> This is where the lack of indentation in your posted code bites; I've just
> realised I've misread your logic because I misindented your code :-( Your
> "while 1:" is just a loop around the input question, not around the menu
> choices.
>
> I would go with a flag, and use it as your loop condition. Like this:
>
>  running_inner_menu = True
>  while running_inner_menu:
>
> So something like this (untested):
>
>  if how_to_insert == 'new collection':
>running_inner_menu = True
>while running_inner_menu:
>  print('Set up collection name : ')
>  db_name_setup = ['manual setup', 'return to the main menu']
>  for index, elem in enumerate(db_name_setup):
>print(index, '-', elem)
>  # obtain a valid choice
>  while 1:
>how_to_setup = input('Choose a method for setting up collection: 

Re: python console menu level looping

[Daiyue Weng]

well, in my case, there is no GUI, and we do not intend to use it since
this script is only for internal testing purposes. So I am just wondering
how to loop menu in this context.

On 24 September 2017 at 22:03, Stefan Ram  wrote:

> Daiyue Weng  writes:
> >I am wondering how to loop back to the 1st level menu
>
>   You are writing a historical kind of menu system.
>
>   One way to proceed might be a more contemporary menu
>   system like tkinter's:
>
> from tkinter import *
>
> class Main(Frame):
>
> def __init__(self, parent):
> Frame.__init__(self, parent)
> self.parent = parent
> self.initUI()
>
> def initUI(self):
> self.parent.title( "menu example" )
> self.pack( fill=BOTH, expand=1 )
> menubar = Menu( self.parent )
> self.parent.config( menu=menubar )
> mainMenu = Menu( self )
> subMenu = Menu( self )
> mainMenu.add_cascade( label='new collection', menu=subMenu )
> subMenu.add_command\
> ( label='manual setup', command = self.manual_setup )
> mainMenu.add_command\
> ( label='existing collection',
>   command = self.existing_collection )
> menubar.add_cascade( label="File", menu=mainMenu )
> self.txt = Text( self )
> self.txt.pack( fill=BOTH, expand=1 )
>
> def onOpen( self ):
> pass
>
> def manual_setup( self ):
> pass
>
> def existing_collection( self ):
> pass
>
> def manual_setup( self ):
> pass
>
> root = Tk()
>
> ex = Main( root )
> ex.pack( side="bottom" )
>
> root.geometry( "300x250+300+300" )
> root.mainloop()
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

python console menu level looping

[Daiyue Weng]

Hi, I tried to make a menu using print statements in Python 3. The code is
as follows,

print('insert data into: ')
data_insert_method = ['new collection', 'existing collection']
for index, elem in enumerate(data_insert_method):
print(index, '-', elem)

while 1:
how_to_insert = input('Choose a method for inserting data: ')
if how_to_insert:
try:
how_to_insert = int(how_to_insert)
how_to_insert = data_insert_method[how_to_insert]
break
except ValueError:
print('Please type in an integer corresponding to the data insert method')
continue
except IndexError:
print('insert method index out of range')
continue

if how_to_insert == 'new collection':

print('Set up collection name : ')
db_name_setup = ['manual setup', 'return to the main menu']
for index, elem in enumerate(db_name_setup):
print(index, '-', elem)

while 1:
how_to_setup = input('Choose a method for setting up collection: ')
if how_to_setup:
try:
how_to_setup = int(how_to_setup)
how_to_setup = db_name_setup[how_to_setup]
break
except ValueError:
print('Please type in an integer corresponding to the collection setup
method')
continue
except IndexError:
print('collection setup method index out of range')
continue

if how_to_setup == 'manual setup':
print('Please type in collection name: ')
elif how_to_setup == 'return to main menu':
print('return to main menu')

It is a 2-level menu, on the 2nd level menu 'new collection', there is an
option - 'return to the main menu', I am wondering how to loop back to the
1st level menu, i.e. ['new collection', 'existing collection'],
whenever 'return
to the main menu' is selected.

cheers
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Terry J. Reedy]

Terry J. Reedy added the comment:

I agree that the tutorial For section needs be updated to include 
non-sequences. A dict example will help with that.

I agree that the unrealistic insert mis-directs attention and like Raymond's 
replacement.  ['users.copy()' should be 'users.copy().items']

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Raymond Hettinger added the comment:

After looking at this again, I think the entire example should be removed.  We 
really don't want to encourage people to code like this (it would never make it 
through a code review).  The example itself is silly (not fully, just weird and 
lacking real-world motiviation).  The s.insert(0,x) code is an anti-pattern.  
And in general, mutating a data structure while iterating over it is a perilous 
practice leading to fragile code (many data structures ban the practice 
outright: databases, deques, dicts).

Mutating while iterating is only safe if a data structure makes explicit 
guarantees about how it iterates.  In Python, we have only a handful of such 
guarantees (you can safely mutate dict values while iterating over the keys and 
lists guarantee that the iterator looks-up consecutive indicies regardless of 
changes to the underlying list).

I propose to remove the last two paragraphs and the example, replacing them 
with clear practical advice and patterns that would pass a code review.

Something like this:

Code that modifies a collection while iterating over
that same collection can be tricky to get right.  Instead,
it is usually more straight-forward to loop over a copy
of the collection or to create a new collection.

# Strategy:  Iterate over a copy
for user, status in users.copy():
if status == 'inactive':
del users[user]

# Strategy:  Create a new collection
active_users = {}
for user, status in users.items():
if status == 'active':
active_users[user] = status

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[R. David Murray]

R. David Murray added the comment:

I don't think that helps.  The issue here is that *sequences* are iterated over 
by incrementing an integer index.  If you change the size of the list, you are 
potentially changing which value any given index points to.  Presumably the 
tutorial writer thought this was intuitive, and indeed after years of Python 
programming I find it so.  I can see how a beginner might not, though :)

What if we replaced:

  If you need to modify the sequence you are iterating over while inside the 
loop (for example to duplicate selected items), it is recommended that you 
first make a copy. Iterating over a sequence does not implicitly make a copy. 
The slice notation makes this especially convenient:

With:

  Sequence iteration is preformed by incrementing an implicit integer index 
until there are no more items in the sequence.  The sequence is *not* copied 
before iteration, so if you modify the sequence during iteration the value that 
is affected by the next iteration of the loop may not be the one you are 
expecting.  You can avoid this problem by iterating over a copy of the 
sequence, and the slice notation makes this especially convenient:

However, this section has a deeper problem.  It is introducing the 'for' 
statement, but explains what the for statement does in terms of sequences, when 
in fact the for statement now operates on any iterable, not just sequences.  
(Many Python programmers probably do not remember the time before the iteration 
protocol was added to the language :)

Fixing that problem not only requires rewriting the section, but also figuring 
out the best place to introduce the concept of the iteration protocol (which 
*might* be in this section, but it's been so long since I've looked over the 
tutorial that I can't say).

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Terry J. Reedy]

Terry J. Reedy added the comment:

Are you looking for something like:

Let it = iter(words).  When next(it) returns 'defenestrate', insertion at the 
beginning moves the original 'defenestrate' over so that next(words) returns 
'defenestrate' again.
?

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Anmol Gupta]

Anmol Gupta added the comment:

And also a small explanation for why there would be an infinite loop without 
creating a copy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Raymond Hettinger]

Raymond Hettinger added the comment:

The example would be more clear if we replaced the opaque idiom "words[:]" with 
the more explicit alternative "words.copy()".

--
assignee: docs@python -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Anmol Gupta]

Anmol Gupta added the comment:

Wrong documentaion section linked.

Correct seciton: Section 4.2 on 
https://docs.python.org/3/tutorial/controlflow.html

The last line needs more explanation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Anmol Gupta]

New submission from Anmol Gupta:

Documentation section: 
https://docs.python.org/3/reference/compound_stmts.html#for

The documentation does not explain at all why is there an infinite loop when 
not using a copy of the list.

It leaves the reader in a confused state.

Even there are questions concerning the same on stackoverlflow: 
https://stackoverflow.com/questions/44633798/loop-through-a-list-in-python-and-modify-it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30826] More details in reference 'Looping through a list in Python and modifying it'

[Anmol Gupta]

Changes by Anmol Gupta <guptaanmol...@gmail.com>:


--
assignee: docs@python
components: Documentation
nosy: Anmol Gupta, docs@python
priority: normal
severity: normal
status: open
title: More details in reference 'Looping through a list in Python and 
modifying it'
type: enhancement
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30826>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Looping [was Re: Python and the need for speed]

[Paul Rubin]

Chris Angelico  writes:
> while True:
> c = sys.stdin.read(1)
> if not c: break
> if c.isprintable(): text += c
> elif c == "\x08": text = text[:-1]
> # etc
> Can you write _that_ as a do-while?

I prefer to write that sort of thing with iterators:

 for c in iter(lambda: sys.stdin.read(1), ''):
 if c.isprintable(): text.append(c)
 elif c == '\x08': text.pop()
 ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Antoon Pardon]

Op 16-04-17 om 19:07 schreef Terry Reedy:
> On 4/16/2017 11:35 AM, Michael Torrie wrote:
>> On 04/16/2017 07:57 AM, bartc wrote:
>>> But people just don't want it.
>>>
>>> /That/ is what surprises me, when people reject things that to me are
>>> no-brainers.
>
> Whereas to me, it is a no-brainer that we are better off *without*
> multiple while/loop constructs.
>
>> I simply don't care about these missing loop constructs.
>
> I do ;-)  I consider the current simplicity a feature.
>
> > Python works
>> great for what I use it for, and apparently works well for many people.
>
> The great majority* of 'repetition with variation' is sequentially
> processing items from a collection.  Python does that nicely with 'for
> item in collection: process(item)'.  While-loops take care of
> everthing else.

Not really, unless you count on the break statement.
But if you count on that, you don't even need a while,
you can start a for loop with a generator that never
stops and use breaks.

There was a time something like the following was
seriously considered for introduction in the language.

do
part1
while condition:
part2

which would be equivalent to the following:

while True:
part1
if not condition:
break
part2

But suddenly this was no longer considered. I still
wish they had followed through. I think such a construct
comes up often enough, to have such a loop construct.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Gregory Ewing]

Ben Bacarisse wrote:


I fond the proportion on while True: loops surprising.  Is there
something about Python that encourages that kind of loop?


Maybe because for-loops take care of most of the ordinary
cases in Python, leaving while-loops to cover the weird
ones, many of which need one or more exits in the middle
somewhere.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Jussi Piitulainen]

Christian Gollwitzer writes:

> Am 18.04.17 um 08:21 schrieb Chris Angelico:
>> On Tue, Apr 18, 2017 at 4:06 PM, Christian Gollwitzer  
>> wrote:
>>> Am 18.04.17 um 02:18 schrieb Ben Bacarisse:
>>>
 Thanks (and to Grant).  IO seems to be the canonical example.  Where
 some languages would force one to write

   c = sys.stdin.read(1)
   while c == ' ':
   c = sys.stdin.read(1)
>>>
>>> repeat
>>> c  = sys.stdin.read(1)
>>> until c != ' '
>>
>> Except that there's processing code after it.
>>
>
> Sorry, I misread it then - Ben's code did NOT have it, it looks like a
> "skip the whitespace" loop.

It also reads the first character that is not whitespace, so it's not
usable to *merely* skip the whitespace.

>> while True:
>> c = sys.stdin.read(1)
>> if not c: break
>> if c.isprintable(): text += c
>> elif c == "\x08": text = text[:-1]
>> # etc
>>
>> Can you write _that_ as a do-while?
>
> No. This case OTOH looks like an iteration to me and it would be most
> logical to write
>
> for c in sys.stdin:
>  if c.isprintable(): text += c
>  elif c == "\x08": text = text[:-1]
>  # etc
>
> except that this iterates over lines. Is there an analogous iterator
> for chars? For "lines" terminated by something else than "\n"?
> "for c in get_chars(sys.stdin)" and
> "for c in get_string(sys.stdin, terminate=':')" would be nicely
> readable IMHO. Or AWK-like processing:
>
> for fields in get_fields(open('/etc/passwd'), RS='\n', FS=':'):
>   if fields[2]=='0':
>   print 'Super-User found:', fields[0]

I don't know if those exist in some standard library, but they are easy
to write, and I do it all the time. I don't need the chars one, but I do
tab-separated fields and line-separated groups of tab-separated fields,
and variations.

for s, sentence in enumerate(sentences(sys.stdin)):
for k, token in enumerate(sentence):
...
token[LEMMA] or warn('empty LEMMA', s, k, sentence)
...

The wrapper function around sys.stdin or other text source is different
depending on the data format. Sometimes it's messy, sometimes not. Any
messy details are hidden the wrapper.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Christian Gollwitzer]

Am 18.04.17 um 08:21 schrieb Chris Angelico:

On Tue, Apr 18, 2017 at 4:06 PM, Christian Gollwitzer  wrote:

Am 18.04.17 um 02:18 schrieb Ben Bacarisse:


Thanks (and to Grant).  IO seems to be the canonical example.  Where
some languages would force one to write

  c = sys.stdin.read(1)
  while c == ' ':
  c = sys.stdin.read(1)


repeat
c  = sys.stdin.read(1)
until c != ' '


Except that there's processing code after it.



Sorry, I misread it then - Ben's code did NOT have it, it looks like a 
"skip the whitespace" loop.



while True:
c = sys.stdin.read(1)
if not c: break
if c.isprintable(): text += c
elif c == "\x08": text = text[:-1]
# etc

Can you write _that_ as a do-while?


No. This case OTOH looks like an iteration to me and it would be most 
logical to write


for c in sys.stdin:
 if c.isprintable(): text += c
 elif c == "\x08": text = text[:-1]
 # etc

except that this iterates over lines. Is there an analogous iterator for 
chars? For "lines" terminated by something else than "\n"?

"for c in get_chars(sys.stdin)" and
"for c in get_string(sys.stdin, terminate=':')" would be nicely readable 
IMHO. Or AWK-like processing:


for fields in get_fields(open('/etc/passwd'), RS='\n', FS=':'):
if fields[2]=='0':
print 'Super-User found:', fields[0]


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Christian Gollwitzer :

> Am 18.04.17 um 02:18 schrieb Ben Bacarisse:
>> Python opts for
>>
>>   while True:
>>  c = sys.stdin.read(1)
>>  if c != ' ': break
>
> This loop would be the archetypical do..while or repeat...until to me.
>
> do
>   c = sys.stdin.read(1)
> while c== ' '

No, the code continues. You want to do something with c, right?

> is the most clear to me - and in fact this "while True; do something;
> break" thingy is just an idiom to fake a do..while loop in Python. C
> does have it, for example, and it is way better like this than the
> abuse of assignment and comma operator in the condition.

I do use

   do ... while

in my C code whenever a loop would end in a conditional break. I happens
exceedingly rarely, though.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Chris Angelico]

On Tue, Apr 18, 2017 at 4:06 PM, Christian Gollwitzer  wrote:
> Am 18.04.17 um 02:18 schrieb Ben Bacarisse:
>
>> Thanks (and to Grant).  IO seems to be the canonical example.  Where
>> some languages would force one to write
>>
>>   c = sys.stdin.read(1)
>>   while c == ' ':
>>   c = sys.stdin.read(1)
>>
>> Python opts for
>>
>>   while True:
>>  c = sys.stdin.read(1)
>>  if c != ' ': break
>
>
> This loop would be the archetypical do..while or repeat...until to me.
>
> do
> c = sys.stdin.read(1)
> while c== ' '
>
>
> -or-
>
> repeat
> c  = sys.stdin.read(1)
> until c != ' '

Except that there's processing code after it.

while True:
c = sys.stdin.read(1)
if not c: break
if c.isprintable(): text += c
elif c == "\x08": text = text[:-1]
# etc

Can you write _that_ as a do-while?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Christian Gollwitzer]

Am 18.04.17 um 02:18 schrieb Ben Bacarisse:


Thanks (and to Grant).  IO seems to be the canonical example.  Where
some languages would force one to write

  c = sys.stdin.read(1)
  while c == ' ':
  c = sys.stdin.read(1)

Python opts for

  while True:
 c = sys.stdin.read(1)
 if c != ' ': break


This loop would be the archetypical do..while or repeat...until to me.

do
c = sys.stdin.read(1)
while c== ' '


-or-

repeat
c  = sys.stdin.read(1)
until c != ' '

is the most clear to me - and in fact this "while True; do something; 
break" thingy is just an idiom to fake a do..while loop in Python.
C does have it, for example, and it is way better like this than the 
abuse of assignment and comma operator in the condition.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[breamoreboy]

On Tuesday, April 18, 2017 at 2:09:19 AM UTC+1, Paul Rubin wrote:
> Ben Bacarisse writes:
> > ?  I get "AttributeError: 'itertools.dropwhile' object has no attribute
> > 'next'" from your example.
> 
> Hmm, .next() worked ok for me in Python 2.7.5.  Not sure what happened.
> Maybe something went wrong with my paste.  Oh well.
> 

PEP 3114 -- Renaming iterator.next() to iterator.__next__() 
https://www.python.org/dev/peps/pep-3114/

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Ben Bacarisse :

> Python opts for
>
>   while True:
>  c = sys.stdin.read(1)
>  if c != ' ': break

I opt for that in C and bash as well.

In fact, when I start writing a loop, I first type:

while True:


Once it is done, I might notice that the loop begins:

while True:
if ...
break

and merge "while" and "if".


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Gregory Ewing :

> Marko Rauhamaa wrote:
>> What I notice in my numbers is that about one half of my while loops
>> are "while True", and about a third of my loops are while loops.
>
> Out of curiosity, what proportion of your 'while True' loops are
> infinite? (I.e. no break, return or raise in the loop.)

0%


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[eryk sun]

On Tue, Apr 18, 2017 at 1:37 AM, MRAB  wrote:
> In Python 3 it's:
>
> c = next(itertools.dropwhile(
>  lambda c: c==' ',
>  iter(lambda: sys.stdin.read(1),None)
>  ))

iter's sentinel should be an empty string.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[MRAB]

On 2017-04-18 02:09, Paul Rubin wrote:

Ben Bacarisse  writes:

?  I get "AttributeError: 'itertools.dropwhile' object has no attribute
'next'" from your example.


Hmm, .next() worked ok for me in Python 2.7.5.  Not sure what happened.
Maybe something went wrong with my paste.  Oh well.


Coming from the lazy language Haskell, I find your example natural...


Yep ;)


Your mistake was that you didn't say it was Python 2. :-)

In Python 3 it's:

c = next(itertools.dropwhile(
 lambda c: c==' ',
 iter(lambda: sys.stdin.read(1),None)
 ))
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Paul Rubin]

Ben Bacarisse  writes:
> ?  I get "AttributeError: 'itertools.dropwhile' object has no attribute
> 'next'" from your example.

Hmm, .next() worked ok for me in Python 2.7.5.  Not sure what happened.
Maybe something went wrong with my paste.  Oh well.

> Coming from the lazy language Haskell, I find your example natural...

Yep ;)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Ben Bacarisse]

Paul Rubin  writes:

> Ben Bacarisse  writes:
>>   c = sys.stdin.read(1)
>>   while c == ' ':
>>   c = sys.stdin.read(1)

(for the record: I was not suggesting this was how you'd do it but how
you'd be forced to do it in some languages)

> c = itertools.dropwhile(
>  lambda c: c==' ',
>  iter(lambda: sys.stdin.read(1),None)
>  ).next()

Did you mean

  c = next(itertools.dropwhile(lambda c: c==' ',
   iter(lambda: sys.stdin.read(1), None)))

?  I get "AttributeError: 'itertools.dropwhile' object has no attribute
'next'" from your example.

Coming from the lazy language Haskell, I find your example natural
(though the syntax is a little busy for my taste) but would it be
considered Pythonic?

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 18/04/2017 01:23, Paul Rubin wrote:

Ben Bacarisse  writes:

  c = sys.stdin.read(1)
  while c == ' ':
  c = sys.stdin.read(1)


c = itertools.dropwhile(
 lambda c: c==' ',
 iter(lambda: sys.stdin.read(1),None)
 ).next()



I tried this but it doesn't like the .next.

I wanted to see if it was any faster or slower than the more obvious loop.

(With the original loop, changing:

c=sys.stdin.read(1)
while c==' ':
c=sys.stdin.read(1)

to:

r=sys.stdin.read

c=r(1)
while c==' ':
c=r(1)

made it 75% faster (reading 10M spaces). Those 3 extra lookups per loop 
I guess (this was run inside a function otherwise the dynamics are 
different).


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Ben Bacarisse]

Marko Rauhamaa  writes:

> Ben Bacarisse :
>
>> Marko Rauhamaa  writes:
>>> What I notice in my numbers is that about one half of my while loops
>>> are "while True", and about a third of my loops are while loops.
>>
>> I fo[u]nd the proportion on while True: loops surprising. Is there
>> something about Python that encourages that kind of loop?

(Thanks for th type correction.  There is also s/of/on/.)

> Here's a typical example of such a loop in Python (ver 2):
>
> while True:
> try:
> snippet = os.read(rdfd, 1000)
> except OSError as e:
> if e.errno == errno.EAGAIN:
> return
> raise
> if not snippet:
> break
> self.stdout_snippets.append(snippet)

Thanks (and to Grant).  IO seems to be the canonical example.  Where
some languages would force one to write

  c = sys.stdin.read(1)
  while c == ' ':
  c = sys.stdin.read(1)

and an Algol-68 style language would permit one to write

  while (c := read char; c) do skip od

Python opts for

  while True:
 c = sys.stdin.read(1)
 if c != ' ': break

(Forgive the atypical choice of input primitive -- it's for illustrating
the loop style only.)


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Paul Rubin]

Ben Bacarisse  writes:
>   c = sys.stdin.read(1)
>   while c == ' ':
>   c = sys.stdin.read(1)

c = itertools.dropwhile(
 lambda c: c==' ',
 iter(lambda: sys.stdin.read(1),None)
 ).next()
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Gregory Ewing]

Marko Rauhamaa wrote:

What I notice in my numbers is that about one half of my while loops are
"while True", and about a third of my loops are while loops.


Out of curiosity, what proportion of your 'while True' loops
are infinite? (I.e. no break, return or raise in the loop.)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Grant Edwards]

On 2017-04-17, Ben Bacarisse  wrote:
> Marko Rauhamaa  writes:
>
>> Terry Reedy :
>>
>>> On 4/17/2017 3:11 AM, Marko Rauhamaa wrote:
 Here's statistics from a medium-sized project of mine:

while True:34
while : 39
for ... in ...:   158
>>>
>>> As I posted previously, the ratio of for-loops in the stdlib is about 7
>>> to 1.
>>
>> What I notice in my numbers is that about one half of my while loops are
>> "while True", and about a third of my loops are while loops.
>
> I fond the proportion on while True: loops surprising.  Is there
> something about Python that encourages that kind of loop?

That's how you read data from a socket/pipe/file.

-- 
Grant Edwards   grant.b.edwardsYow! Your CHEEKS sit like
  at   twin NECTARINES above
  gmail.coma MOUTH that knows no
   BOUNDS --

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Mikhail V]

On 17 April 2017 at 04:00, Steve D'Aprano <steve+pyt...@pearwood.info> wrote:
> On Mon, 17 Apr 2017 05:49 am, Dennis Lee Bieber wrote:
>
>> On Mon, 17 Apr 2017 02:48:08 +1000, Steve D'Aprano
>> <steve+pyt...@pearwood.info> declaimed the following:
>>
>>>On Sun, 16 Apr 2017 11:57 pm, bartc wrote:
>>>
>>>> But people just don't want it.
>>>
>>>Damn straight. Now you get it. It's not about how easy it is to implement,
>>>it's about whether it is needed and wanted. It isn't needed, and it isn't
>>>wanted.
>>>
>>
>> Let's go all the way, and just get rid of "while ".
>
> Sure, why not? If it's your language, you can make it as minimalist or
> maximalist as you like. You get to choose.
>

I think it is right not to touch anything for .py file rules.
Seriously there are much more problems, and better code editors
should adress those, not current syntax.


As for loops, well, if we speak of some 'my' language,
it could be interesting whether there are common patterns.
But is it on-topic here to speak of, say PyBasic2020 fantasy
language?

For real-time application development the amount of
"while(condition)" vs "while True" can play bad jokes
with readability.
I would find such looping schema simpler:

loop:
if contitioin 1 : contitioin_break = 1
blabla bla = bla  + foo
make something
make something
if contitioin 2 : contitioin_break = 1
if contitioin 3 : contitioin_break = 0
if contitioin_break :  *break
/loop


Anyway it is just question where to show this last line,
or to make an emphasis on it.
so I'd just leave it there, for me it helps when I imagine
a cascade of functions and my eyes go down to the
bottom.  If I have kilometers of "if"s inside the loop,
then hiding one line can become more a problem than an aid.

Constructs like "repeat 100 times {}" or other sort of
"let's pronounce it in English" found in
some languages looks sort of confusing, it takes
me time to "unwrap" their sense into a cascade
especially when there are many different keywords.


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Ben Bacarisse :

> Marko Rauhamaa  writes:
>> What I notice in my numbers is that about one half of my while loops
>> are "while True", and about a third of my loops are while loops.
>
> I fo[u]nd the proportion on while True: loops surprising. Is there
> something about Python that encourages that kind of loop?

Here's a typical example of such a loop in Python (ver 2):

while True:
try:
snippet = os.read(rdfd, 1000)
except OSError as e:
if e.errno == errno.EAGAIN:
return
raise
if not snippet:
break
self.stdout_snippets.append(snippet)

I find myself writing similar loops in C a lot, as well.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 17/04/2017 19:02, Ben Bacarisse wrote:

Marko Rauhamaa  writes:


Terry Reedy :


On 4/17/2017 3:11 AM, Marko Rauhamaa wrote:

Here's statistics from a medium-sized project of mine:

   while True:34
   while : 39
   for ... in ...:   158


As I posted previously, the ratio of for-loops in the stdlib is about 7
to 1.


What I notice in my numbers is that about one half of my while loops are
"while True", and about a third of my loops are while loops.


I fond the proportion on while True: loops surprising.  Is there
something about Python that encourages that kind of loop?



A few things:

(1) Python doesn't have the equivalent of C's comma operator, and no 
assignment in expressions. So if a condition relies on such set-up code, 
it can't do this:


   while (c=nextc())!=0:

But has to do this or use extra logic:

   while 1:
  c=nextc()
  if c==0: break

(2) There are exceptions ('raise') as an extra means of breaking out of 
such loops (as well as return, break, and exit()


(3) There's 'yield' too, as I've just seen this example:

while True:
   yield []

So there are more excuses to make use of it. Plus Python programmers may 
be more averse to using convoluted logic just to avoid a 'break' in the 
middle of a loop.



--
bartc

--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Ben Bacarisse]

Marko Rauhamaa  writes:

> Terry Reedy :
>
>> On 4/17/2017 3:11 AM, Marko Rauhamaa wrote:
>>> Here's statistics from a medium-sized project of mine:
>>>
>>>while True:34
>>>while : 39
>>>for ... in ...:   158
>>
>> As I posted previously, the ratio of for-loops in the stdlib is about 7
>> to 1.
>
> What I notice in my numbers is that about one half of my while loops are
> "while True", and about a third of my loops are while loops.

I fond the proportion on while True: loops surprising.  Is there
something about Python that encourages that kind of loop?

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Terry Reedy :

> On 4/17/2017 3:11 AM, Marko Rauhamaa wrote:
>> Here's statistics from a medium-sized project of mine:
>>
>>while True:34
>>while : 39
>>for ... in ...:   158
>
> As I posted previously, the ratio of for-loops in the stdlib is about 7
> to 1.

What I notice in my numbers is that about one half of my while loops are
"while True", and about a third of my loops are while loops.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Terry Reedy]

On 4/17/2017 3:11 AM, Marko Rauhamaa wrote:

Gregory Ewing :


bartc wrote:

Most of my loops start off as endless loops, until I can determine
the actual terminating condition, and where it best goes.


Interesting. My experience is quite different. Most of the loops I
write start off with me thinking "Now I want to do this for each
element of this collection", which maps straight to a Python for-loop.

In the rare cases where I do use a while loop, usually I have a fairly
good idea what the terminating condition is going to be.


Here's statistics from a medium-sized project of mine:

   while True:34
   while : 39
   for ... in ...:   158


As I posted previously, the ratio of for-loops in the stdlib is about 7 
to 1.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Marko Rauhamaa]

Gregory Ewing :

> bartc wrote:
>> Most of my loops start off as endless loops, until I can determine
>> the actual terminating condition, and where it best goes.
>
> Interesting. My experience is quite different. Most of the loops I
> write start off with me thinking "Now I want to do this for each
> element of this collection", which maps straight to a Python for-loop.
>
> In the rare cases where I do use a while loop, usually I have a fairly
> good idea what the terminating condition is going to be.

Here's statistics from a medium-sized project of mine:

   while True:34
   while : 39
   for ... in ...:   158


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

On Mon, 17 Apr 2017 05:49 am, Dennis Lee Bieber wrote:

> On Mon, 17 Apr 2017 02:48:08 +1000, Steve D'Aprano
>  declaimed the following:
> 
>>On Sun, 16 Apr 2017 11:57 pm, bartc wrote:
>>
>>> But people just don't want it.
>>
>>Damn straight. Now you get it. It's not about how easy it is to implement,
>>it's about whether it is needed and wanted. It isn't needed, and it isn't
>>wanted.
>>
> 
> Let's go all the way, and just get rid of "while ".

Sure, why not? If it's your language, you can make it as minimalist or
maximalist as you like. You get to choose.

Python's not your language, so it's not your choice for Python. But you can
always create your own language and implement as many or as few features as
you like.

Guido likes while loops with a condition, and Python is Guido's language.

And of course, since Python has had while loops with a condition for over 20
years now, backwards compatibility precludes removing it.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Gregory Ewing]

bartc wrote:


 > - describing the various syntax forms;
 > - explaining how they differ;
 > - tutorials for beginners showing each form;

And you don't have to explain how an endless loop should be written as 
'while True', meanwhile advising against using 'while 1'?


You don't have to mention it in the reference docs, because it's
deducible from the rest of the language rules. Nor do you need
an explicit test for it, unless there is code in the compiler
specifically to optimise that case.

Tutorials might want to mention it, but it's not strictly
necessary, as a smart enough student could figure it out for
themselves.


So, how many kinds of sequences do you have in Python?

lists
tuples
namedtuples
arrays
bytearrays
string ?
memoryview


By far the most commonly used ones are list and tuple. The
rest are designed for specialised uses, and have characteristics
that can't easily be replicated using the fundamental types,
if at all.

Alternative looping constructs, in contrast, have *no*
functional difference, their only advantage being that they
provide a slightly more succint way of expressing certain
things.


  While a; b; c; d Do e; f; g End

with the test (the value of 'd') effectively in the middle.


That would be a rather obtuse way of expressing it, though.
It looks like the whole of 'a; b; c; d' is the condition,
whereas you're really only using a, b and c for their side
effects.

The whole point of a loop-and-a-half construct is to make
the condition stand out, despite being in the middle.

This is actually the only additional looping construct that
I wouldn't mind seeing. It gets brought up now and then;
last time, if I recall, the discussion fizzled out due to
a feeling that you don't really need it much in Python,
because those kinds of loops are usually better expressed
using an iterator.

Most of my loops start off as endless loops, until I can determine the 
actual terminating condition, and where it best goes.


Interesting. My experience is quite different. Most of
the loops I write start off with me thinking "Now I want
to do this for each element of this collection", which
maps straight to a Python for-loop.

In the rare cases where I do use a while loop, usually I
have a fairly good idea what the terminating condition is
going to be. Cases where a significant amount of head
scratching is needed to figure it out are pretty rare.
But maybe you're working on different kinds of problems
from me.


Sometimes they stay as endless loops.


By design or by accident? :-)

Every program I've every written that had a deliberate
endless loop in it had exactly *one* of them, at the top
level. And I knew it would be that way before I started
writing it. (And since the only way to stop it would be
ctrl-C or equivalent, I only write such things if they're
throwaway tools for my own use.)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 16/04/2017 19:42, Chris Angelico wrote:

On Mon, Apr 17, 2017 at 4:21 AM, bartc  wrote:

Here is a function from some old CPython source that appears to be something
to do with While statements:

static int
validate_while(node *tree)
{

...

Look, no comments! Are you going to castigate the developers of CPython for
that? (I guess not.)


Can you find the equivalent in the current sources? I think it might be here:

https://github.com/python/cpython/blob/master/Grammar/Grammar#L73




while_stmt: 'while' test ':' suite ['else' ':' suite]

A single, extremely readable line. Abstract, not concrete.


How do you know my code fragment wasn't also generated? Since it was 
parsing C, the refs for that language also define the while statement 
quite neatly:


  'while' '(' expression ')' statement

but actually it was done manually. It's part of a suite of functions 
that do things in a common pattern, and would be documented as a set 
rather than line by line. And elsewhere calls to such a function would 
also be part of a chain of tests.


That's if I needed to documented it (but I can't remember how many 
parsers I've done along the same lines since c.1979).


But I've glanced through the CPython code again, and it is quite lightly 
commented. Although sometimes it gets quite chatty.



Want to show me your source for this source?


The source files I looked at were dated around 2010. I've no idea where 
they came from. I think I'd wanted to compile it, but unfortunately, 
they're not written in C, but a combination of C, make-file, 
compiler-options, and bash-script with a bunch of Linux dependencies.




Next complaint, please.


No complaint, just saying other projects are sparse on comments too. And 
my code isn't intended as open source.


--
bartc

--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Chris Angelico]

On Mon, Apr 17, 2017 at 4:21 AM, bartc  wrote:
> Here is a function from some old CPython source that appears to be something
> to do with While statements:
>
> static int
> validate_while(node *tree)
> {
> int nch = NCH(tree);
> int res = (validate_ntype(tree, while_stmt)
>&& ((nch == 4) || (nch == 7))
>&& validate_name(CHILD(tree, 0), "while")
>&& validate_test(CHILD(tree, 1))
>&& validate_colon(CHILD(tree, 2))
>&& validate_suite(CHILD(tree, 3)));
>
> if (res && (nch == 7))
> res = (validate_name(CHILD(tree, 4), "else")
>&& validate_colon(CHILD(tree, 5))
>&& validate_suite(CHILD(tree, 6)));
>
> return (res);
> }
>
> Look, no comments! Are you going to castigate the developers of CPython for
> that? (I guess not.)

Can you find the equivalent in the current sources? I think it might be here:

https://github.com/python/cpython/blob/master/Grammar/Grammar#L73

while_stmt: 'while' test ':' suite ['else' ':' suite]

A single, extremely readable line. Abstract, not concrete. And my
suspicion is that the code you're looking at above just might have
been generated from the same kind of grammar file - which means that
it is NOT the "CPython source", but rather an intermediate file. It's
the equivalent of looking at a .pyc file and complaining that it has
no comments.

Want to show me your source for this source? The best I can find (by
searching the git history (which was imported from the hg history) for
'validate_while') is the code prior to this commit:

https://github.com/python/cpython/commit/53595c

And I rather suspect, from some of the other comments in the area,
that this file may have been autogenerated. Even if it wasn't, it was
still not the "primary source", but exists merely to implement
Grammar/Grammar, as linked above.

Next complaint, please.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 16/04/2017 17:30, Steve D'Aprano wrote:

On Sun, 16 Apr 2017 10:06 pm, bartc wrote:



(The 30 Loc figure is with support for loops /in
general/ already in place, and is for /adding/ a new loop statement, in
this case 'while')


What part of *testing* and *documenting* do you not understand?



Do you have any unit tests for your compiler?  How about regression tests --
when you fix a bug, do you write a regression test to ensure it never
creeps back in?

Do you have any documentation for your compiler? Does it include doctests?
Are there any tutorials that beginners can read?

I'm guessing you don't have any of those things. The code snippet you posted
doesn't even have any comments.


Here is a function from some old CPython source that appears to be 
something to do with While statements:


static int
validate_while(node *tree)
{
int nch = NCH(tree);
int res = (validate_ntype(tree, while_stmt)
   && ((nch == 4) || (nch == 7))
   && validate_name(CHILD(tree, 0), "while")
   && validate_test(CHILD(tree, 1))
   && validate_colon(CHILD(tree, 2))
   && validate_suite(CHILD(tree, 3)));

if (res && (nch == 7))
res = (validate_name(CHILD(tree, 4), "else")
   && validate_colon(CHILD(tree, 5))
   && validate_suite(CHILD(tree, 6)));

return (res);
}

Look, no comments! Are you going to castigate the developers of CPython 
for that? (I guess not.)


Anyway I took out the one or two comments from my posted extract.


If Python added something like:

loop N times:
...

we would need *at least* the following:

- a test that `loop` was interpreted as a keyword;
- a test that `times` was interpreted as a keyword;
- a test that `loop 0 times` didn't execute the body at all;
- a test that `loop 1 times` executed the body exactly once;
- a test that `loop N times` executed the body exactly N times, for
  a few different (and probably randomly selected) values of N;
- a test that the statement handled negative integers and floats correctly;
- a test that the statement handled non-numeric values correctly
  (probably by raising an exception);
- a test that `continue` worked as expected inside this statement;
- a test that `break` worked as expected;
- a test that `return` worked as expected;
- a test that `raise` worked as expected;
- a test that signals will be caught correctly inside the statement;

and possibly more.


Well, I was talking more about just 'loop:', to repeat endlessly. Then 
half of those tests disappear. See, a dedicated statement can make 
things simpler.


The other half would be the same for loops generally as, after you get 
to the compiler output (if I can't call it byte-code), then the chances 
are that the original identity of the loop is lost.


Loop N times could be on top of While or For. Or it could have its own 
implementation where it /knows/ the loop counter is an integer, and a 
64-bit one. See, another optimisation opportunity.



BTW supporting a dedicated endless loop was only about 20 lines in
another project. Believe me, it is /nothing/. I wish other aspects were
just as trivial. It didn't even need a dedicated keyword.


Of course it's "nothing" for you, since by the evidence given you don't
bother with the hard parts. No comments, no communication with other
developers (except to gloat over how awesome you are and how stupid their
choices are), no documentation, no tests.


There are a couple of aspects I like of dynamic languages, which is the 
informality and spontaneity.


The informality with Python seems to have disappeared (with all this 
paperwork, and committees and applications to get more features added; 
about as interesting as making a proposal for a new office block).


While the spontaneity seems to have suffered too if a feature takes 
years to be accepted, even if it would only take an hour or two to add 
(on a smaller project like mine).


At least, I get the benefits immediately. If I had to wait five years 
(when I might be dead anyway) then I might as well wait forever.



The loops I use are categorised as:

* Endless


That's just a special case of "until some condition is true".


No, because there is no condition to test or think about. There should 
not be one to specify. Python requires that a dummy condition is 
provided to satisfy the language syntax (but internally it compiles to 
an endless loop, as there there is no bureaucracy telling it what to do).



* N times


That's just a special case of "over an integer sequence".


No, because you don't care about what that sequence is, nor about having 
to present it as named object to the body of the loop. Another 
optimisation opportunity.


Neither does the user really want to be bothered with writing the name 
of a variable. And providing a range() is overkill, as you just want a 
count not a sequence.





* Until some condition is true


You missed one: do you check the condition 

Re: Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

On Mon, 17 Apr 2017 03:00 am, Rustom Mody wrote:


> BTW I regard Steven's long list of things that youve missed such as
> regression tests, docs etc to be somewhat off the mark
> To see that try this experiment:
> Just add a feature to python that matters to you along with all these
> requirements: docs, tests etc etc
> Your chances of acceptance may go up marginally but most likely it will
> still not be accepted.

That's very true.

It's not enough to have working code, tests and documentation to get a
feature accepted. I didn't think I implied otherwise, but if it wasn't
clear, I'm sorry.

The feature first needs to have a reason (use-cases). It must be justified.
Unless you are Guido, "because I like it" is not enough.

When you create your own language, like Guido did with Python and Bart has
with his mystery language, your own personal taste is all it takes to get
features included. Everyone else has to justify their request, and have it
accepted by the core developers.

But even once a feature is accepted *in principle* it won't be actually
added to the language until it has tests and documentation. That's the
point I'm trying to get across to Bart. It doesn't matter how trivial the
implementation, or now much of a no-brainer it is, it still needs tests and
documentation.

In the specific case of "loop forever", I'm pretty sure that it would really
struggle to gain community favour. But even if it did, I'm pretty sure the
core devs and Guido in particular would reject it.

As I've said before, I actually like the syntax "repeat forever" -- in the
right context. In Hyperscript, it suits the rest of the language and reads
well. In Python, well, it is like grafting the head of a lion onto the
shoulder of a horse. Lions are handsome beasts, and horses are also very
attractive, individually, but making a freakish chimera of the two is not.

There are lots of ways to design programming languages, and personally I'm
particularly fond of Hyperscript's verbose English-like syntax, Forth and
other RPN stack-based languages, and Python. That's three very different
styles, and I like them all. But not all together at once.

Bart, if your aesthetic taste says different, that's your right, and good on
you. Nobody wants to tell you what syntactic features you should or
shouldn't add to your language.

I just wish you would return the favour to the Python community and stop
insisting that *your* taste is the only good taste.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Terry Reedy]

On 4/16/2017 11:35 AM, Michael Torrie wrote:

On 04/16/2017 07:57 AM, bartc wrote:

But people just don't want it.

/That/ is what surprises me, when people reject things that to me are
no-brainers.


Whereas to me, it is a no-brainer that we are better off *without* 
multiple while/loop constructs.



I simply don't care about these missing loop constructs.


I do ;-)  I consider the current simplicity a feature.

> Python works

great for what I use it for, and apparently works well for many people.


The great majority* of 'repetition with variation' is sequentially 
processing items from a collection.  Python does that nicely with 'for 
item in collection: process(item)'.  While-loops take care of everthing 
else.


*I grepped 3.6.1 .../lib/*.py with the REs '^ *while ' and '^ *for ', 
recursing into subdirectories, including tests and a few packages in 
site-packages.  These got 1389 and 9842 hits respectively. I am opposed 
to adding syntax to subdivide the 12% of looping using 'while'.  (As a 
note, '^ *while True' had 363 hits, about 1/4 of while loops.)


> I have yet to find a loop that I couldn't construct with Python's
> apparently-limited constructs.

For-loops, 'limited' to iterating through collections (iterables), cover 
at least 80% of cases.  While-loops + continue/break easily cover the 
remaining cases of linear repetition.  Branching repetition, as in naive 
Fibonacci calculation and tree processing, is more easily done with 
recursion.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Rustom Mody]

On Sunday, April 16, 2017 at 7:27:49 PM UTC+5:30, bartc wrote:

> Technically, adding this one feature to Python /is/ trivial, 
^
You are not paying attention bart and I am not likely to pursue this beyond this
post. I tried to say as are others that the substantive reasons to reject a 
feature  are mostly non-technical

BTW I regard Steven's long list of things that youve missed such as regression
tests, docs etc to be somewhat off the mark
To see that try this experiment:
Just add a feature to python that matters to you along with all these 
requirements: docs, tests etc etc
Your chances of acceptance may go up marginally but most likely it will still
not be accepted.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

On Sun, 16 Apr 2017 11:57 pm, bartc wrote:

> Yet countless other, far more elaborate features /are/ added all the time.

Indeed. Because they are needed. Because they add functionality that Python
doesn't already have, or seriously improves the interface to that
functionality.


> Technically, adding this one feature to Python /is/ trivial, for
> example, allowing while: as a synonym for while True:, but preferably
> using a new keyword such as loop. 

"Preferably using a new keyword"?

Dear gods, you really have *no clue* about the responsibilities of
maintaining a programming language used by tens of thousands of people.

Adding your new keyword will break thousands of peoples programs. That's not
to be done lightly, on a whim, just because you don't like the spelling
`while True`.


py> loop = True
  File "", line 1
loop = True
 ^
SyntaxError: invalid syntax


> Nothing else needs to be touched. 

Right. We don't need to test this new feature, because we have such awesome
mad skillz that of course it will work perfectly first time.

Nor do we need to document this new feature. We don't need to include it in
the "What's New" documentation, or include it in the tutorial, or mention
it in the list of keywords, or in the description of the grammar.

Documentation is for people who don't already know the source code because
they wrote it.

People will work it out for themselves. Eventually. After many hours, or
days, of pulling their hair out at why their working code has suddenly
stopped working after upgrading to a new Python version.

The only thing you could do that would be more irresponsible would be to
insist on introducing this new, undocumented, untested, unnecessary feature
in a bug-fix point release.

Ah, but I'm guessing that you don't understand the need for point releases
either? With only one user, yourself, who needs releases or version
numbers?


> And  
> it could have been done right at the start when the politics was simpler.
> 
> But people just don't want it.

Damn straight. Now you get it. It's not about how easy it is to implement,
it's about whether it is needed and wanted. It isn't needed, and it isn't
wanted.


> /That/ is what surprises me, when people reject things that to me are
> no-brainers.

Well, the thing is, there are two kinds of no-brainers. There are the kind
of things that are so obviously a good idea that it requires no brain to
want it.

And then there are the kinds of things that are so obviously a bad idea that
it requires no brain to want it.

Which is this, do you think?



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

. It didn't even need a dedicated keyword.

Of course it's "nothing" for you, since by the evidence given you don't
bother with the hard parts. No comments, no communication with other
developers (except to gloat over how awesome you are and how stupid their
choices are), no documentation, no tests.

 
>  > - describing the various syntax forms;
>  > - explaining how they differ;
>  > - tutorials for beginners showing each form;
> 
> And you don't have to explain how an endless loop should be written as
> 'while True', meanwhile advising against using 'while 1'?

No you don't, because `while True` or `while 1` or for that matter `while
['this list', 'is, 'a', 'true object']` are easily derived from
understanding (1) while loops; and (2) truthy values.

"repeat forever" needs documentation, because it is syntax.

(And what on earth makes you think that `while 1` needs advising against?)


>> The more choices you offer, the harder that decision becomes:
>>
>> - numeric Pascal or C-style loop
>> - foreach style loop
>> - repeat while condition (test at start)
>> - repeat until condition (test at start)
>> - do ... while condition (test at end)
>> - do ... until condition (test at end)
>> - repeat forever
> 
> So, how many kinds of sequences do you have in Python?
> 
> lists
> tuples
> namedtuples
> arrays
> bytearrays
> string ?
> memoryview
> 
> plus all those ordered types. My head is already spinning!

"Ordered types"?

You forgot bytes and deques. But you should try Java if you really want your
head to explode.

What's your point? Python has eight or ten or a dozen sequence types because
they're all different, sometimes *radically* different, and they all have
important uses that other types cannot fulfill.


> The loops I use are categorised as:
> 
> * Endless

That's just a special case of "until some condition is true".

> * N times

That's just a special case of "over an integer sequence".

> * Until some condition is true

You missed one: do you check the condition before the first loop, or at the
end of the loop? That makes a difference between the loop running zero or
more times, or one or more times.


> * Iterate over an integer sequence

And that in turn is just a special case of iterating over a set of values.

> * Iterate over a set of values of some object

Out of your five fundamental loops, you missed one important distinction,
and invented three unimportant ones.


It is true that Python doesn't support repeat...until condition loops, where
the condition is checked at the end. And I consider that a (minor) weakness
of Python.


> Other languages like to have even more elaborate schemes. That includes
> advanced uses of Python's for loop, were, for example, there are
> multiple loop variables.
> 
> I wonder how much testing that took to get it right?

Probably a lot. And it would be worth it even if it took fifty times more
testing.


[...]
>>> But very common requirements are endless loops, and repeat N times
>>> without needing an explicit counter.
>>
>> If by "very common" you mean "occasionally", I agree.
> 
> Most of my loops start off as endless loops, until I can determine the
> actual terminating condition, and where it best goes. Sometimes they
> stay as endless loops.
> 
> (Sometimes, I turn a normal statement into an endless loop with a 'do'
> prefix. This is an unusual feature but I use it quite a bit:
> 
> doswitch nextchar()   # looping version of 'switch'
> when 'A'..'Z' then 
> else exit # ie. break out of loop
> end
> )

How delightfully eccentric of you :-)


>>> Python's byte-code does at least optimise out the check that '1' is
>>> true, but that's not what the reader sees, which is 'loop while 1 is
>>> true'. And one day it will be:
>>>
>>>  while l:
>>>  body
>>>
>>> that can be mistaken for that common idiom.
>>
>> The problem there is not the loop, but the foolish use of lowercase l as
>> a variable name.
> 
> Maybe. But if 'while 1' wasn't used, then that's one less thing to
> double-check. (I just looked at a random bunch of Python code; there
> were 25 instances of 'while True:', but 40 of 'while 1:'.)

What on earth are you talking about? Even if Python had a dedicated "loop
forever" statement, "while 1" would still be perfectly legal code.


For what it's worth, there are about 170 uses of "while 1" or "while True"
in the standard library, out of about 400+ while loops in total:

[steve@ando ~]$ cd /usr/local/lib/python3.5/
[steve@ando python3.5]$ grep "while 1:" *.py | wc -l
45
[steve@ando python3.5]$ grep "while True:" *.py | wc -l
126
[steve@ando python3.5]$ grep "while .*:" *.py | wc -l
415

which is quite rare compared to for loops:

[steve@ando python3.5]$ grep "for .*:" *.py | wc -l
1435




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 16/04/2017 15:22, Chris Angelico wrote:

On Sun, Apr 16, 2017 at 11:57 PM, bartc  wrote:

Technically, adding this one feature to Python /is/ trivial, for example,
allowing while: as a synonym for while True:, but preferably using a new
keyword such as loop. Nothing else needs to be touched. And it could have
been done right at the start when the politics was simpler.

But people just don't want it.

/That/ is what surprises me, when people reject things that to me are
no-brainers.


Maybe it's not a no-brainer to them. Maybe you need to make the case that:

1) It's worth stopping people from using the word "loop" as a variable


Actually I think 'loop' could still be used as a variable, since 'name:' 
wouldn't occur in any other contexts. But a dual purpose 'loop' wouldn't 
be elegant.



2) It's worth documenting another type of loop
3) It's worth having everyone need to know how to read another type of loop


Compared to all the other stuff in Python, seriously? Lambdas, 
listcomps, generators, iterators, decorators, subclassing, class and 
instance attributes, ... no problemo.


But faced with:

  loop: print ("Hi!")

will lead to head-scratching?!


What's the benefit, aside from avoiding discussions like this?


Imagine if the discussion was instead about introducing a negate 
operator so that you could write:


  a = -b

instead of having to write:

  a = 0 - b

It wouldn't be a big deal having to do the latter, but it's nicer not to.

I just find writing 'while 1' (or while (1) or for(;;) elsewhere) an 
annoyance.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Michael Torrie]

On 04/16/2017 07:57 AM, bartc wrote:
> But people just don't want it.
> 
> /That/ is what surprises me, when people reject things that to me are 
> no-brainers.

I simply don't care about these missing loop constructs.  Python works
great for what I use it for, and apparently works well for many people.
I have yet to find a loop that I couldn't construct with Python's
apparently-limited constructs.  I guess I don't see what having those
extra looping constructs will give me in terms of productivity or even
ease of use.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Chris Angelico]

On Sun, Apr 16, 2017 at 11:57 PM, bartc  wrote:
> Technically, adding this one feature to Python /is/ trivial, for example,
> allowing while: as a synonym for while True:, but preferably using a new
> keyword such as loop. Nothing else needs to be touched. And it could have
> been done right at the start when the politics was simpler.
>
> But people just don't want it.
>
> /That/ is what surprises me, when people reject things that to me are
> no-brainers.

Maybe it's not a no-brainer to them. Maybe you need to make the case that:

1) It's worth stopping people from using the word "loop" as a variable
2) It's worth documenting another type of loop
3) It's worth having everyone need to know how to read another type of loop

What's the benefit, aside from avoiding discussions like this?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 16/04/2017 13:22, Rustom Mody wrote:

On Sunday, April 16, 2017 at 5:36:28 PM UTC+5:30, bartc wrote:

On 16/04/2017 03:51, Steve D'Aprano wrote:

On Sat, 15 Apr 2017 10:17 pm, bartc wrote:



Yes, I'm constantly surprised at this, as such syntax has a very low
cost (in my last compiler, supporting 'while' for example only added 30
lines to the project).


That's the advantage of writing your own private language and having no
users except for yourself. You get to cut corners. Python has tens of
thousands of users, and doesn't have that luxury.


Here are the lines of code in my C compiler which are necessary to
support 'while': https://pastebin.com/BYFV7EWr

(45 lines shown, but there are exactly 30 Loc if blanks are excluded.)

I'd be interested in knowing why implementing While in Python would need
significantly more. (The 30 Loc figure is with support for loops /in
general/ already in place, and is for /adding/ a new loop statement, in
this case 'while')


You walk down a mountain path (trail in US-ese?) stopping here and there to
smell a flower or spy a butterfly.
Change to driving down a mountain road — quiet for the most part — is it ok
to drive on the wrong side?
Change to an expressway is it ok to change lanes randomly, not maintain speeds
within upper (and lower!) limits?

As others have tried to point out maintaining your own 1-man-language is one 
thing
Maintaining a language
- used by industry and academia
- Used by FLOSS teams and commercial
- by kids and old-hands
- 1-man projects and large multiperson projects

is quite another

Dont get me wrong: I dont agree with everything about python.  And the 
current(est)
mess around async (and f-strings?) is (IMHO) going to cost more dearly than 
people expect...
And we will have to wait another 10 years for that decision...

My point is that you dont seem to have any estimate of the difference in
momentum between your 'its-just-another-30-lines' language and a million
user base language


Yet countless other, far more elaborate features /are/ added all the time.

Technically, adding this one feature to Python /is/ trivial, for 
example, allowing while: as a synonym for while True:, but preferably 
using a new keyword such as loop. Nothing else needs to be touched. And 
it could have been done right at the start when the politics was simpler.


But people just don't want it.

/That/ is what surprises me, when people reject things that to me are 
no-brainers.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Rustom Mody]

On Sunday, April 16, 2017 at 5:36:28 PM UTC+5:30, bartc wrote:
> On 16/04/2017 03:51, Steve D'Aprano wrote:
> > On Sat, 15 Apr 2017 10:17 pm, bartc wrote:
> 
> >> Yes, I'm constantly surprised at this, as such syntax has a very low
> >> cost (in my last compiler, supporting 'while' for example only added 30
> >> lines to the project).
> >
> > That's the advantage of writing your own private language and having no
> > users except for yourself. You get to cut corners. Python has tens of
> > thousands of users, and doesn't have that luxury.
> 
> Here are the lines of code in my C compiler which are necessary to 
> support 'while': https://pastebin.com/BYFV7EWr
> 
> (45 lines shown, but there are exactly 30 Loc if blanks are excluded.)
> 
> I'd be interested in knowing why implementing While in Python would need 
> significantly more. (The 30 Loc figure is with support for loops /in 
> general/ already in place, and is for /adding/ a new loop statement, in 
> this case 'while')

You walk down a mountain path (trail in US-ese?) stopping here and there to
smell a flower or spy a butterfly.
Change to driving down a mountain road — quiet for the most part — is it ok
to drive on the wrong side?
Change to an expressway is it ok to change lanes randomly, not maintain speeds
within upper (and lower!) limits?

As others have tried to point out maintaining your own 1-man-language is one 
thing
Maintaining a language
- used by industry and academia
- Used by FLOSS teams and commercial
- by kids and old-hands
- 1-man projects and large multiperson projects

is quite another

Dont get me wrong: I dont agree with everything about python.  And the 
current(est)
mess around async (and f-strings?) is (IMHO) going to cost more dearly than 
people expect...
And we will have to wait another 10 years for that decision...

My point is that you dont seem to have any estimate of the difference in 
momentum between your 'its-just-another-30-lines' language and a million
user base language
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[bartc]

On 16/04/2017 03:51, Steve D'Aprano wrote:

On Sat, 15 Apr 2017 10:17 pm, bartc wrote:



Yes, I'm constantly surprised at this, as such syntax has a very low
cost (in my last compiler, supporting 'while' for example only added 30
lines to the project).


That's the advantage of writing your own private language and having no
users except for yourself. You get to cut corners. Python has tens of
thousands of users, and doesn't have that luxury.


Here are the lines of code in my C compiler which are necessary to 
support 'while': https://pastebin.com/BYFV7EWr


(45 lines shown, but there are exactly 30 Loc if blanks are excluded.)

I'd be interested in knowing why implementing While in Python would need 
significantly more. (The 30 Loc figure is with support for loops /in 
general/ already in place, and is for /adding/ a new loop statement, in 
this case 'while')



For language developers with responsibilities to users, the job doesn't stop
at just implementing the feature in the language. It also needs to be
tested and documented.

As a general rule of thumb, every line of production code should expect to
add at least ten lines of test code (unit tests, regression tests,
integration tests, functional tests, doc tests, etc). So for your "30
lines" feature, that adds 300 lines of tests, and probably another page or
two of documentation:


No, we're talking about a loop. It must be just about the simplest thing 
to implement in a language (compared with a type system, or code 
generation).


BTW supporting a dedicated endless loop was only about 20 lines in 
another project. Believe me, it is /nothing/. I wish other aspects were 
just as trivial. It didn't even need a dedicated keyword.


> - describing the various syntax forms;
> - explaining how they differ;
> - tutorials for beginners showing each form;

And you don't have to explain how an endless loop should be written as 
'while True', meanwhile advising against using 'while 1'?



The more choices you offer, the harder that decision becomes:

- numeric Pascal or C-style loop
- foreach style loop
- repeat while condition (test at start)
- repeat until condition (test at start)
- do ... while condition (test at end)
- do ... until condition (test at end)
- repeat forever


So, how many kinds of sequences do you have in Python?

lists
tuples
namedtuples
arrays
bytearrays
string ?
memoryview

plus all those ordered types. My head is already spinning!

The loops I use are categorised as:

* Endless
* N times
* Until some condition is true
* Iterate over an integer sequence
* Iterate over a set of values of some object

Other languages like to have even more elaborate schemes. That includes 
advanced uses of Python's for loop, were, for example, there are 
multiple loop variables.


I wonder how much testing that took to get it right?


I don't remember the language, but I remember seeing one generalisation of
the repeat/do loop that puts the test in the middle, rather than at the
start or end of the loop. If I remember it was something like:

DO
setup code  # executed once only
REPEAT
loop body  # before the test
WHILE condition  # test
loop body  # after the test

thus combining both the

repeat while condition:
...

and

do:
...
until condition


styles of looping in one handy syntax.


Perhaps you misremembered as that looks too unwieldy to be practical. If 
implemented orthogonally, then a loop like this:


  While A Do B End

where both A and B can be any sequence of statements or expressions, 
would allow you to write:


  While a; b; c; d Do e; f; g End

with the test (the value of 'd') effectively in the middle.


But very common requirements are endless loops, and repeat N times
without needing an explicit counter.


If by "very common" you mean "occasionally", I agree.


Most of my loops start off as endless loops, until I can determine the 
actual terminating condition, and where it best goes. Sometimes they 
stay as endless loops.


(Sometimes, I turn a normal statement into an endless loop with a 'do' 
prefix. This is an unusual feature but I use it quite a bit:


   doswitch nextchar()   # looping version of 'switch'
   when 'A'..'Z' then 
   else exit # ie. break out of loop
   end
)


Python's byte-code does at least optimise out the check that '1' is
true, but that's not what the reader sees, which is 'loop while 1 is
true'. And one day it will be:

 while l:
 body

that can be mistaken for that common idiom.


The problem there is not the loop, but the foolish use of lowercase l as a
variable name.


Maybe. But if 'while 1' wasn't used, then that's one less thing to 
double-check. (I just looked at a random bunch of Python code; there 
were 25 instances of 'while True:', but 40 of 'while 1:'.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looping [was Re: Python and the need for speed]

[Ben Bacarisse]

Steve D'Aprano <steve+pyt...@pearwood.info> writes:


> I don't remember the language, but I remember seeing one generalisation of
> the repeat/do loop that puts the test in the middle, rather than at the
> start or end of the loop. If I remember it was something like:
>
> DO
> setup code  # executed once only
> REPEAT
> loop body  # before the test
> WHILE condition  # test
> loop body  # after the test
>
> thus combining both the 
>
> repeat while condition:
> ...
>
> and
>
>     do:
> ...
> until condition
>
> styles of looping in one handy syntax.

Was the setup code in the loop because that seems unnecessary (I know
this is from memory)?  I have a recollection of using something like

  do
  code before test
while cond
  code after test
  od

which has much the same effect.  Algol 68 can also do this because the
while condition can be a block (any expression can be a block in A68).
The result was not to everyone's taste.

> On Sat, 15 Apr 2017 10:17 pm, bartc wrote:

>> Of course, it's possible to overdo it; if you look at Lisp, you'll lose
>> yourself in the myriad looping options.
>
> "Myriad"? As I understand it, Lisp offers just *one* more loop construct
> than the number you agree is the "minimum" needed: five.
>
> loop
> loop for
> do
> dotimes
> dolist

I suppose it depends on how you count, but BartC did talk about
options.  Here are some of the loop options in Common Lisp:

  (loop x with ...)
  (loop for i in (...) by ...)
  (loop for i downfrom 10 above x by -2 ...)
  (loop for x = ... then (f x) ...)
  (loop across ...)
  (loop for s being symbols of ...)
  (loop for s being hash-keys in ... using ...)

That's just the start.  Then you can add "doing", "if" "when", "unless",
"return", "collect", "sum[ming]", "maximizing" and others.  Or, if you
need them, you can have "initially", "finally", "repeat", "while",
"until", "always", "never" or "thereis" parts, all the time losing (some
of) those parts you don't need.


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Looping [was Re: Python and the need for speed]

[Steve D'Aprano]

On Sat, 15 Apr 2017 10:17 pm, bartc wrote:

> On 15/04/2017 03:35, Rick Johnson wrote:
>> On Wednesday, April 12, 2017 at 8:44:30 AM UTC-5, bart...@gmail.com
>> wrote:
> 
>> At a minimum, every language should offer
>> the following four loop-forms (using Python semantics):
>>
>> while CONDITION:
>> doSomething()
>>
>> for VALUE in COLLECTION:
>> doSomething(value)
>>
>> loop(N):
>> doSomething()
>>
>> loop(N) as i:
>>doSomething(i)


What an extravagant, unnecessary waste of syntax. The last two just
duplicate the functionality of the second. AND Rick has forgotten about the
do...while form that executes the block at least once.

I've used, and to be honest really enjoyed using, a language with multiple
loop syntax like this. I can see some advantage in offering specialist
English-like loops in a programming language for non-programmers and
amateurs. But having a plethora of forms for what is essentially the same
thing is not so helpful for more skilled programmers who are more
comfortable with composing code rather than memorising fixed statements.


repeat forever:
...


is no more expressive than

while True:
...


since they're both just the same thing. You can mechanically and trivially
translate one to the other with no intelligence needed. So it becomes a
mere matter of personal taste as to which you prefer.


> Yes, I'm constantly surprised at this, as such syntax has a very low
> cost (in my last compiler, supporting 'while' for example only added 30
> lines to the project).

That's the advantage of writing your own private language and having no
users except for yourself. You get to cut corners. Python has tens of
thousands of users, and doesn't have that luxury.

For language developers with responsibilities to users, the job doesn't stop
at just implementing the feature in the language. It also needs to be
tested and documented.

As a general rule of thumb, every line of production code should expect to
add at least ten lines of test code (unit tests, regression tests,
integration tests, functional tests, doc tests, etc). So for your "30
lines" feature, that adds 300 lines of tests, and probably another page or
two of documentation:

- describing the various syntax forms;
- explaining how they differ;
- tutorials for beginners showing each form;

plus the on-going burden for every single user of the language, for ever, in
having to decide which form they need to use in any specific circumstance.

The more choices you offer, the harder that decision becomes:

- numeric Pascal or C-style loop
- foreach style loop
- repeat while condition (test at start)
- repeat until condition (test at start)
- do ... while condition (test at end)
- do ... until condition (test at end)
- repeat forever


I don't remember the language, but I remember seeing one generalisation of
the repeat/do loop that puts the test in the middle, rather than at the
start or end of the loop. If I remember it was something like:

DO
setup code  # executed once only
REPEAT
loop body  # before the test
WHILE condition  # test
loop body  # after the test

thus combining both the 

repeat while condition:
...

and

do:
...
until condition


styles of looping in one handy syntax.


> Of course, it's possible to overdo it; if you look at Lisp, you'll lose
> yourself in the myriad looping options.

"Myriad"? As I understand it, Lisp offers just *one* more loop construct
than the number you agree is the "minimum" needed: five.

loop
loop for
do
dotimes
dolist


https://www.tutorialspoint.com/lisp/lisp_loops.htm



> But very common requirements are endless loops, and repeat N times
> without needing an explicit counter. 

If by "very common" you mean "occasionally", I agree.


> The former /can/ be easily written 
> as:
> 
>  while 1:
>  body
> 
> but it's more psychological; I don't want to use an idiom to denote an
> endless loop, I want to be able to express it directly!

"Express it directly" is an idiom, and "while True" is just as direct
as "repeat forever".


> Python's byte-code does at least optimise out the check that '1' is
> true, but that's not what the reader sees, which is 'loop while 1 is
> true'. And one day it will be:
> 
>  while l:
>  body
> 
> that can be mistaken for that common idiom.

The problem there is not the loop, but the foolish use of lowercase l as a
variable name. It is simply poor programming practice to use easily
confused names, and the problem here is not the use of `while`.

No more than it is the use of `while` that makes this code bad:

while O000IlI1III111IllOO:
something()
O000IlI1III11I1II1IllOO = Fals

Re: Screwing Up looping in Generator

[Sayth Renshaw]

For completeness I was close this is the working code.

def get_list_of_names(generator_arg):
name_set = set()
for name in generator_arg:
base = os.path.basename(name.name)
filename = os.path.splitext(base)[0]
name_set.add(filename)
return name_set


def data_attr(roots):
"""Get the root object and iter items."""
for name in names:
directory = "output"
write_name = name + ".csv"
with open(os.path.join(directory, write_name), 'w', newline='') as
csvf:
race_writer = csv.writer(csvf, delimiter=','
)


thanks for your time and assistance. It's much appreciated

Sayth

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Steve D'Aprano]

On Wed, 4 Jan 2017 01:09 pm, Sayth Renshaw wrote:

> Untested as i wrote this in notepad at work but, if i first use the
> generator to create a set of filenames and then iterate it then call the
> generator anew to process file may work?

It "may" work. Or it "may not" work. It is hard to tell because we don't have 
enough context to understand your code.

Let me see if I can guess what you are doing:

> Good idea or better available?
>
> def get_list_of_names(generator_arg):
> name_set = set()
> for name in generator_arg:
> base = os.path.basename(name.name)
> filename = os.path.splitext(base)[0]
> name_set.add(filename)
> return name_set

What does "generator_arg" do? Since you iterate over it, it could be a list, a 
tuple, any other sequence, or an iterator. Why does it have to be a generator?

What is "name.name"?

I *think* that your intention is to take a list of objects that hold filenames:


["C://My Documents/data.txt", "C://My Documents/folder/image.jpg",
 "C://file.txt", "D://installer.exe", "E://folder/image.gif"]

strip off the path, strip off the file extensions, remove any duplicates, 
giving:

set(["data", "image", "file", "installer"])

(Notice that image.jpg and image.gif count as duplicates.)


If that is what you want, then I think get_list_of_names will work, except:

- the name is WRONG: it returns a set, not a list;

- the name does not describe what the function does;

- there's no documentation

- the name of the argument is misleading, it doesn't have to be a generator.


Other than that, I think the code does what I think you want.


>  for file_name in name_set:
>  directory = "output"
>  with open(os.path.join(directory, filename, 'w', newline='') as
>  csvf:
> for file in rootobs:
>   # create and write csv


Your indentation is wrong.

What's "rootobs"?

The code you show here does not have enough detail for me to even try to guess 
what it does.





--
Steve
â £Cheer up,â Ø they said, â £things could be worse.â Ø So I cheered up, and
sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Rhodri James]

On 04/01/17 02:10, Deborah Swanson wrote:
> Sayth Renshaw wrote, on January 03, 2017 5:36 PM
>>
>> So can I call the generator twice and receive the same file
>> twice in 2 for loops?
>>
>> Once to get the files name and the second to process?
>>
>>  for file in rootobs:
>> base = os.path.basename(file.name)
>> write_to = os.path.join("output",
>> os.path.splitext(base)[0] + ".csv")
>> with open(write_to, 'w', newline='') as csvf:
>> for file in rootobs:
>>   # create and write csv
>>
>> Cheers
>>
>> Sayth
>
> I don't see why not, if you let the first one run to completion and then
> do it again a second time. Assuming your generator doesn't somehow
> delete or modify the file specifications as it yields them. It would be
> helpful to see the code for rootobs, if you have it.

Ahem.  If Sayth is using the correct terminology and rootobs actually is a 
generator (not, say, a list or tuple), then no it won't work.  Once a generator 
is exhausted, it's exhausted.  Besides, the nested for-loops over the same 
iterable is a dead giveaway that something is wrong.


--
Rhodri James *-* Kynesim Ltd

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Screwing Up looping in Generator

[Deborah Swanson]

Erik wrote, on January 03, 2017 5:26 PM
> Hi,
>
> On 04/01/17 01:12, Deborah Swanson wrote:
> > The main reason you might want to catch the StopIteration
> exception is
> > to do something else before your code simply stops running. If all
> > you're doing is run a generator til it's out of gas, and that's all
> > you want it to do, then there's no need to catch anything.
>
> Ah! OK, I see where the lines are being crossed now ;) Although
> StopIteration is an exception, it is something that the 'for/iter'
> machinery handles for you under the covers. Each 'for' statement
> effectively has a 'try' block around it that catches
> 'StopIteration' and
> just terminates that particular 'for' loop and continues on with the
> remainder of your script.
>
> Raising a 'StopIteration' is an internal mechanism used to determine
> when an iterator (i.e., the thing a 'for' loop is looping over) has
> exhausted itself. It's not something a regular user is ever
> expected to
> know about let alone catch.
>
> When execution falls out of the bottom of a generator,
> StopIteration is
> raise (compared to a regular function or method returning 'None').
>
> E.

Looks like those MIT professors knew what they were teaching us after all! 
Sneaky, but they never once hinted that this was any deep dark secret. Just a 
way to deal with generators that will stop after a finite and unknown number of 
yields, and you want your code to keep going after the generator quits. I 
thought I was just regurgitating a standard approach, and surprised to get so 
much push back on it.

Seems like python uses a lot of its external functionality (what we normally 
code with) for internal behavior too. In this case, and for empty returns, 
they're raising exceptions, catching them and then doing what they want to. I 
suppose, if you don't want the method returning None, you could catch that 
exception and return something else. Or return nothing at all, which is what I 
usually want to do when those pesky Nones crop up.  But I'm not entirely sure 
how you would make it return nothing at all.

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Screwing Up looping in Generator

[Deborah Swanson]

Sayth Renshaw wrote, on January 03, 2017 5:36 PM
>
> So can I call the generator twice and receive the same file
> twice in 2 for loops?
>
> Once to get the files name and the second to process?
>
>  for file in rootobs:
> base = os.path.basename(file.name)
> write_to = os.path.join("output",
> os.path.splitext(base)[0] + ".csv")
> with open(write_to, 'w', newline='') as csvf:
> for file in rootobs:
>   # create and write csv
>
> Cheers
>
> Sayth

I don't see why not, if you let the first one run to completion and then do it 
again a second time. Assuming your generator doesn't somehow delete or modify 
the file specifications as it yields them. It would be helpful to see the code 
for rootobs, if you have it. D.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Sayth Renshaw]

Untested as i wrote this in notepad at work but, if i first use the generator 
to create a set of filenames and then iterate it then call the generator anew 
to process file may work?

Good idea or better available?

def get_list_of_names(generator_arg):
name_set = set()
for name in generator_arg:
base = os.path.basename(name.name)
filename = os.path.splitext(base)[0]
name_set.add(filename)
return name_set



 for file_name in name_set:
 directory = "output"
 with open(os.path.join(directory, filename, 'w', newline='') as csvf:
for file in rootobs:
  # create and write csv

Sayth

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Screwing Up looping in Generator

[Deborah Swanson]

Sayth Renshaw wrote, on January 03, 2017 5:55 PM
>
> On Wednesday, 4 January 2017 12:36:10 UTC+11, Sayth Renshaw  wrote:
> > So can I call the generator twice and receive the same file
> twice in 2
> > for loops?
> >
> > Once to get the files name and the second to process?
> >
> >  for file in rootobs:
> > base = os.path.basename(file.name)
> > write_to = os.path.join("output",
> os.path.splitext(base)[0] + ".csv")
> > with open(write_to, 'w', newline='') as csvf:
> > for file in rootobs:
> >   # create and write csv
> >
> > Cheers
> >
> > Sayth
>
> I just need it to write after each file however the
>
> with open(#file) as csvf:
>
> Keeps it all open until every file processed in an output
> file with the name of the first file in the generator.
>

In that case, I think you just need to devise your output file name scheme, and 
it looks like you want to use 'os.path.splitext(base)[0] + ".csv")'. Then  
output each file in the same for loop, before you go back for another input 
file. Just read in the file, do whatever you want to it, and then write it to 
the new file name, all in the same loop.

I don't see any need to loop through rootobs a second time.

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Screwing Up looping in Generator

[Deborah Swanson]

Chris Angelico wrote, on January 03, 2017 3:35 PM
>
> On Wed, Jan 4, 2017 at 10:05 AM, Deborah Swanson
>  wrote:
> > Ok, I learned how to use generators in Python 2.7.8, which may be
> > different from Python 3 for generators. But I learned from MIT's
> > online introduction to python course, and they certainly
> seem to know
> > python well. So what is the correct way to call the
> generator's next
> > yield in Python 3? We only learned to use the next function. If you
> > don't use the next function, what do you use?
>
> The built-in next function, not the next method.
>
> # don't do this
> gen.next()
>
> # do this
> next(gen)
>
> ChrisA

You speak the truth! I never doubted, but since I still have 2.7.8 on my system 
 I decided to try it out.

For a simple little Fibbonacci number generator:

def genFib ():
fibn_1 = 1 #fib(n-1)
fibn_2 = 0 #fib(n-2)
while True:
# fib(n) = fib(n-1) + fib(n-2)
next = fibn_1 + fibn_2
yield next
fibn_2 = fibn_1
fibn_1 = next

and at the console:

>>> fib = genFib()
>>> fib.next()

2.7.8 works, and cranks out as many Fibbonacci numbers as you could want.

But in 3.4.3 you get:

Traceback (most recent call last):
  File "", line 1, in 
fib.next()
AttributeError: 'generator' object has no attribute 'next'

Then, going the other way, next(fib) works in both versions.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Sayth Renshaw]

So can I call the generator twice and receive the same file twice in 2 for 
loops?

Once to get the files name and the second to process?

 for file in rootobs:
base = os.path.basename(file.name)
write_to = os.path.join("output", os.path.splitext(base)[0] + ".csv")
with open(write_to, 'w', newline='') as csvf:
for file in rootobs:
  # create and write csv

Cheers

Sayth

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Screwing Up looping in Generator

[Deborah Swanson]

Erik wrote, on January 03, 2017 3:53 PM
>
> On 03/01/17 23:05, Deborah Swanson wrote:
> > And yes, we usually used for loops for generators, unless you don't
> > know when the generator will be exhausted. As in this case,
> where the
> > number of files the generator can provide is unknown. Then
> we used the
> > while True, break on StopIteration method.
>
> Out of interest, *why* was it deemed necessary to do
> something different
> if you don't know how many items the generator will generate? Was any
> rationale given for that?
>
> for x in foo:
>bar(x)
>
> ... where foo is any iterable (something that has a __iter__ method
> defined - including generators) will just bind each value in
> turn to 'x'
> and will exit when the StopIteration exception is raised under the
> covers by the iterator that is iterating over the iterable.
>
> Some generators are infinite (and their iterator will never raise a
> StopIteration exception).
>
> E.

The main reason you might want to catch the StopIteration exception is to do 
something else before your code simply stops running. If all you're doing is 
run a generator til it's out of gas, and that's all you want it to do, then 
there's no need to catch anything. There's lots of situations where this is 
exactly what you want. It might even be the most frequent use of generators, 
though I wouldn't know.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Sayth Renshaw]

On Wednesday, 4 January 2017 12:36:10 UTC+11, Sayth Renshaw  wrote:
> So can I call the generator twice and receive the same file twice in 2 for
loops?
>
> Once to get the files name and the second to process?
>
>  for file in rootobs:
> base = os.path.basename(file.name)
> write_to = os.path.join("output", os.path.splitext(base)[0] + ".csv")
> with open(write_to, 'w', newline='') as csvf:
> for file in rootobs:
>   # create and write csv
>
> Cheers
>
> Sayth

I just need it to write after each file however the

with open(#file) as csvf:

Keeps it all open until every file processed in an output file with the name of 
the first file in the generator.

Sayth

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Screwing Up looping in Generator

[Erik]

Hi,

On 04/01/17 01:12, Deborah Swanson wrote:
> The main reason you might want to catch the StopIteration exception is
> to do something else before your code simply stops running. If all
> you're doing is run a generator til it's out of gas, and that's all you
> want it to do, then there's no need to catch anything.

Ah! OK, I see where the lines are being crossed now ;) Although StopIteration 
is an exception, it is something that the 'for/iter' machinery handles for you 
under the covers. Each 'for' statement effectively has a 'try' block around it 
that catches 'StopIteration' and just terminates that particular 'for' loop and 
continues on with the remainder of your script.

Raising a 'StopIteration' is an internal mechanism used to determine when an 
iterator (i.e., the thing a 'for' loop is looping over) has exhausted itself. 
It's not something a regular user is ever expected to know about let alone 
catch.

When execution falls out of the bottom of a generator, StopIteration is raise 
(compared to a regular function or method returning 'None').

E.

-- 
https://mail.python.org/mailman/listinfo/python-list