Re: Proposed new syntax

[Ned Batchelder]

On 8/25/17 12:10 AM, Paul Rubin wrote:
> Steve D'Aprano  writes:
>> Did __next__ cache the most recently generated value?
> No but if they're going to change stuff, they might as well actually
> improve it instead of just renaming it to break code gratutiously.

The vast majority of iteration has no need for the last-returned value
again, so why add it to every iteration? You can provide it yourself (or
propose it!) with a small wrapper class.  One of the beautiful things
about Python's iteration model is that it does just one thing well. 
This means it can be applied in more situations to more kinds of data. 
Then if you need more, you build on top of it.

--Ned.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Fri, Aug 25, 2017 at 2:10 PM, Paul Rubin  wrote:
> Steve D'Aprano  writes:
>> Did __next__ cache the most recently generated value?
>
> No but if they're going to change stuff, they might as well actually
> improve it instead of just renaming it to break code gratutiously.

Any code that called iter.next() instead of next(iter) was already
broken. Converting it to the proper notation makes it work on all
Python versions. Any code that needs to *implement* an iterator can do
so fairly simply:

class Foo(object): # you already need (object) for Py2 compat
def __iter__(self): return self
def __next__(self):
# whatever
return 42
next = __next__ # this is all you need to add

Considering that it then brings the protocol nicely in line with every
other magic-method protocol in Python, this is a Good Thing. It's one
more line of code in any class that is an iterator - not all
iterables, only custom iterators. It's no additional code in consumers
of iterators.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Steve D'Aprano  writes:
> Did __next__ cache the most recently generated value?

No but if they're going to change stuff, they might as well actually
improve it instead of just renaming it to break code gratutiously.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 25 Aug 2017 01:19 pm, Paul Rubin wrote:

> Steve D'Aprano  writes:
>> the public API is to call the next() built-in function, and the
>> implementation is the __next__ dunder.
> 
> In that case it would have been nice to make next() cache the most
> recently generated value from the iterator.  That would make lots of
> code simpler.

Huh? Why?

Did __next__ cache the most recently generated value?

The whole point of next() (whether a method or function) is that it evaluates
the next value on demand, not to return the previous value you already
evaluated earlier. I don't understand what sort of code it would make simpler.

I'm tempted to say "buggy code", but I'll give you the benefit of the doubt
first *wink*



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Steve D'Aprano  writes:
> the public API is to call the next() built-in function, and the
> implementation is the __next__ dunder.

In that case it would have been nice to make next() cache the most
recently generated value from the iterator.  That would make lots of
code simpler.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Antoon Pardon]

Op 24-08-17 om 04:02 schreef Steve D'Aprano:
>
> Unfortunately the interwebs are full of people, even mathematicians, that 
> have a
> lot of misapprehensions and misunderstandings of Gödel's Incompleteness
> Theorems. For example, there's a comment here:
>
> "It's easy to prove that ZFC is consistent in the right theory, e.g.
> ZFC+Con(ZFC)"
>
> https://philosophy.stackexchange.com/questions/28303/if-the-zfc-axioms-cannot-be-proven-consistent-how-can-we-say-for-certain-that-a
>
> apparently without the slightest awareness that this would be begging the
> question. If you assume that ZFC is consistent, of course you can prove that
> ZFC is consistent.

Which seems to suggest his point went right over your head. His point being
that Gödel's 2nd Theorem mentions the specific theory in which you can't prove
particular axioms consistent.

What you can prove and what not is not an absolute but depending on the system
you are working with and he was just illustrating that, by providing a system
in which such a proof would be trivial, because the theorem to be proven was
just one of the axioms.

Begging the question is not applicable in this kind of studies. A proposition
is provable in a particular system or not, but it is important to specify
what system you are talking about.

-- 
Antoon Pardon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Thu, 24 Aug 2017 06:21 pm, Paul Rubin wrote:

> Peter Otten <__pete...@web.de> writes:
>> Python 3 where the next() method has been renamed to __next__().
> 
> Oh cripes, you're right, it never occurred to me that py3 had broken
> .next().  I thought it was called .next() instead of .__next()
> so that it wouldn't be a dunder method.

Not so much *broken* it as *fixed* it :-)

Guido decided years ago that directly exposing next as a public method was a
mistake. Instead, like len(), str(), repr(), iter() etc. the public API is to
call the next() built-in function, and the implementation is the __next__
dunder.

I don't remember whether that decision was just for consistency with other
special methods, or whether there was some other deeper reason... 

Possibly so it was easy to add a default value to next() without having to force
every iterator class to re-implement the same default behaviour? That seems
reasonable... in Python 2.7, the next method takes no default argument:

py> iter("").next(99)
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: expected 0 arguments, got 1

but the built-in function does:

py> next(iter(""), 99)
99


In general, the public API function (whether built-in like next, or not) can do
much more than just call the dunder method, e.g. look at the various operators.


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Chris Angelico  writes:

> On Thu, Aug 24, 2017 at 8:40 PM, Ben Finney  
> wrote:
> > Paul Rubin  writes:
> >> Is something wrong with:
> >>
> >> >>> g = itertools.count().next
>
> It's on all iterators; it's the Py2 equivalent for __next__.

Yeah, when people are talking today about “Python”, unqualified by
version, I think it's a fair assumption that responses can be only
regarding Python 3 unless there's good reason otherwise :-)

So, yes there's something wrong with ‘itertools.count().next’. What's
wrong with it is that it works only on a dead-end Python that everyone
should be busily migrating away from.

(And I see Paul was later made aware of that. So, we all win! :-)

-- 
 \ “I was gratified to be able to answer promptly and I did. I |
  `\   said I didn't know.” —Mark Twain, _Life on the Mississippi_ |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Thu, Aug 24, 2017 at 8:40 PM, Ben Finney  wrote:
> Paul Rubin  writes:
>
>> Ben Finney  writes:
>> > generate_id = functools.partial(next, itertools.count())
>>
>> Is something wrong with:
>>
>> >>> g = itertools.count().next
>
> I wasn't looking for a ‘next’ method on the iterator. Is that special to
> the ‘itertools.count’ type?
>
> If so, I was attempting to give the more general solution to “how do I
> get a function that will give me the next thing from this iterator”.

It's on all iterators; it's the Py2 equivalent for __next__. When you
call next(x), Py2 calls x.next(), which - being an inconsistent name -
got renamed to the more appropriate __next__() in Py3.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Paul Rubin  writes:

> Ben Finney  writes:
> > generate_id = functools.partial(next, itertools.count())
>
> Is something wrong with:
>
> >>> g = itertools.count().next

I wasn't looking for a ‘next’ method on the iterator. Is that special to
the ‘itertools.count’ type?

If so, I was attempting to give the more general solution to “how do I
get a function that will give me the next thing from this iterator”.

-- 
 \   “If consumers even know there's a DRM, what it is, and how it |
  `\ works, we've already failed.” —Peter Lee, Disney corporation, |
_o__) 2005 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Paul Rubin :

> Ben Finney  writes:
>> generate_id = functools.partial(next, itertools.count())
>
> Is something wrong with:
>
> >>> g = itertools.count().next

I don't like lambda in Python. However, I like it better than
functools.partial.

Best of all, define an inner function.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Peter Otten <__pete...@web.de> writes:
> Python 3 where the next() method has been renamed to __next__().

Oh cripes, you're right, it never occurred to me that py3 had broken
.next().  I thought it was called .next() instead of .__next()
so that it wouldn't be a dunder method.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Peter Otten]

Paul Rubin wrote:

> Ben Finney  writes:
>> generate_id = functools.partial(next, itertools.count())
> 
> Is something wrong with:
> 
> >>> g = itertools.count().next

That question seems to be the topic of this subthread.

Other than the principle "never call a dunder method" (that I do not follow 
strictly) you accidentally bring up another argument: compatibility between 
Python 2 and Python 3 where the next() method has been renamed to 
__next__().

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Steve D'Aprano  writes:
> I've read a few people claim that disallowing multiplication from
> standard arithmetic renders it weak enough that you can prove it
> complete and correct, but since they give no proof or even evidence I
> have my doubts.

That system is called Presburger arithmetic (see Wikipedia).  It can be
proved consistent by quantifier elimination but is not powerful enough
to prove (or maybe even state) its own consistency.

Surprisingly, there are also theories that can state and prove their own
consistency:

  https://en.wikipedia.org/wiki/Self-verifying_theories
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Thu, Aug 24, 2017 at 5:47 PM, Paul Rubin  wrote:
> Ben Finney  writes:
>> generate_id = functools.partial(next, itertools.count())
>
> Is something wrong with:
>
> >>> g = itertools.count().next
> >>> g()
> 0
> >>> g()
> 1
> >>> g()
> 2
> >>> ...

That's the Python 2 version of the same thing. The next method was
renamed to __next__ to synchronize it with other, similar methods.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Ben Finney  writes:
> generate_id = functools.partial(next, itertools.count())

Is something wrong with:

>>> g = itertools.count().next
>>> g()
0
>>> g()
1
>>> g()
2
>>> ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 22 Aug 2017 07:41 pm, Marko Rauhamaa wrote:

> BTW, the main take of the metamathematical "fiasco" was that you can't
> get rid of the meta-level. There's no consistent logical system that is
> closed and encompasses everything including itself. You will always have
> to step outside your formal system and resort to hand-waving in a
> natural language.

That's not quite correct. Gödel's Incompleteness theorems only apply
to "sufficiently powerful" systems. They don't apply to systems which are too
weak. Not all such simple systems are either consistent or correct, but those
that are, may be provable as such.

Standard arithmetic is sufficiently powerful that the Incompleteness theorem
applies, but not all such systems do. I've read a few people claim that
disallowing multiplication from standard arithmetic renders it weak enough that
you can prove it complete and correct, but since they give no proof or even
evidence I have my doubts.

Unfortunately the interwebs are full of people, even mathematicians, that have a
lot of misapprehensions and misunderstandings of Gödel's Incompleteness
Theorems. For example, there's a comment here:

"It's easy to prove that ZFC is consistent in the right theory, e.g.
ZFC+Con(ZFC)"

https://philosophy.stackexchange.com/questions/28303/if-the-zfc-axioms-cannot-be-proven-consistent-how-can-we-say-for-certain-that-a

apparently without the slightest awareness that this would be begging the
question. If you assume that ZFC is consistent, of course you can prove that
ZFC is consistent.


This article has a very nice description of Gödel's theorems, the reactions of
mathematicians to it ("outrage, condescension, bafflement, fascination, and
complete disinterest"), a comparison of mathematical induction and ordinary
induction, and why, ultimately, we shouldn't be too worried by the possibility
that arithmetic is inconsistent:

http://www.mathpages.com/home/kmath347/kmath347.htm



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Stephan Houben]

Op 2017-08-23, Ben Finney schreef :
> Could you be convinced to instead do::
>
> import functools
> import itertools
>
> generate_id = functools.partial(next, itertools.count())

I certainly could be, but I was so far unaware of the desirability to do so.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 22 Aug 2017 11:42 pm, Ian Kelly wrote:

> The last line is the reason why the rich comparison methods should
> have been designed to raise NotImplementedException rather than return
> NotImplemented, but it's too late to change that.

Only if you want operators to be slow as molasses.

*wink*



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

Since this erm… discussion has also brought in Haskell
and in this case, the name, the history etc are related I thought I'd mention 
the following

Around 2015 there was a major upheaval in the Haskell community around the
socalled FTP (foldable-traversable-prelude) controversy.

In many respects this controversy is analogous and even identical to this one
and the heat there was considerably more than this thread's storm-in-a-teaspoon

Mark Lentczner resigned as Haskell's release manager¹
which in the python world would be analogous to say Raymond Hettinger saying
“Python 3 is too much of a mess; I am going to stick to python 2.2”
Along with hims went other stalwarts like Lennart Augustsson and Eric Meijer's
widely acclaimed EdX course switched from haskell to hugs²
[which is like switching to python 1.6]

The controversy somewhat oversimplified is that foldr (reduce-from-right) was
foldr : (a → b → b) → b → [a] → b
It was changed to
foldr : Foldable 풯  ⇒  (a → b → b) → b → 풯 a → b

If we think of [a] as list_of_a and 풯 a as any general list-like interface
we would see the close parallel with the divergence of opinion on this thread

¹ 
http://haskell.1045720.n5.nabble.com/quot-Excuse-me-I-think-this-i-my-stop-quot-Resigning-from-the-Platform-td5819861.html
² 
https://www.reddit.com/r/haskell/comments/3ove2e/got_the_welcome_email_from_edx_fp101x_course_hugs/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Gregory Ewing :

> Marko Rauhamaa wrote:
>> You will always have to step outside your formal system and resort to
>> hand-waving in a natural language.
>
> If the hand-waving is rigorous, this amounts to expanding your formal
> system by adding new axioms and/or rules to it.

Ultimately, it's not rigorous. You can add rigor to any number of
meta-levels, but on top of it all, you will need an informal "observer"
level. It's turtles all the way down.

For example, you can't have a set of all sets. The NBG set theory solves
the problem by introducing the concept of classes. You can have a class
of all sets. But you can't have a class of all classes.

> If the hand-waving is not rigorous, then you haven't really proved
> anything.

The mathematicians have stopped caring.

In fact, even if metamathematics were a closed, formal system, the best
it could achieve would be circular reasoning. That would still be
satisfactory and "convincing." However, no interesting system can prove
its own consistency (but it *can* prove it can't prove its own
consistency).

Recommended reading:

   https://www.amazon.com/Unprovability-Consistency-Essay-Moda
   l-Logic/dp/0521092973>


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Stephan Houben  writes:

> I have often done things like:
>
>   generate_id = itertools.count().__next__

Could you be convinced to instead do::

import functools
import itertools

generate_id = functools.partial(next, itertools.count())

-- 
 \“The right to search for truth implies also a duty; one must |
  `\  not conceal any part of what one has recognized to be true.” |
_o__) —Albert Einstein |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Marko Rauhamaa wrote:

You will always have
to step outside your formal system and resort to hand-waving in a
natural language.


If the hand-waving is rigorous, this amounts to expanding your
formal system by adding new axioms and/or rules to it.

If the hand-waving is not rigorous, then you haven't really
proved anything.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Wed, Aug 23, 2017 at 1:12 AM, Stephan Houben
 wrote:
> Op 2017-08-22, Ian Kelly schreef :
>> Careful! Python's dunder methods are reserved for use by Python.
>> They're exposed so that we can override them. Calling them directly is
>> generally considered bad style. And in this case specifically, it's
>> not equivalent.
>
> Mmm, you are correct. That's kind of a deception, really.
>
> I have often done things like:
>
>   generate_id = itertools.count().__next__
>
> but I suppose that isn't OK either, then.

That one probably won't bite you, but that doesn't mean it's correct.

General rule of thumb: Dunders are for defining, not for calling.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Stephan Houben]

Op 2017-08-22, Ian Kelly schreef :
> Careful! Python's dunder methods are reserved for use by Python.
> They're exposed so that we can override them. Calling them directly is
> generally considered bad style. And in this case specifically, it's
> not equivalent. 

Mmm, you are correct. That's kind of a deception, really.

I have often done things like:

  generate_id = itertools.count().__next__

but I suppose that isn't OK either, then.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Tue, Aug 22, 2017 at 3:58 AM, Stephan Houben
 wrote:
> Op 2017-08-11, Paul Rubin schreef :
>> I don't think we need this since we have itertools.takewhile:
>>
>>   from operator import gt
>>   from functools import partial
>>   from itertools import takewhile
>>
>>   [x + 1 for x in takewhile(partial(gt,5), (0,1,2,999,3,4))]
>>
>
> No need for partial and gt.
>
>   [x + 1 for x in takewhile((5).__gt__, (0,1,2,999,3,4))]
>
> Basically, Haskell's infix opererator sections can often be
> translated into Python by attribute access to the bound method.

Careful! Python's dunder methods are reserved for use by Python.
They're exposed so that we can override them. Calling them directly is
generally considered bad style. And in this case specifically, it's
not equivalent. Example:

import functools
import operator

@functools.total_ordering
class PseudoInt(object):
def __init__(self, value):
self.value = value
def __eq__(self, other):
return self.value == other
def __lt__(self, other):
return self.value < other

>>> PseudoInt(5) > 2
True
>>> 5 > PseudoInt(2)
True
>>> 5 > PseudoInt(7)
False
>>> operator.gt(5, PseudoInt(7))
False
>>> (5).__gt__(PseudoInt(7))
NotImplemented
>>> bool((5).__gt__(PseudoInt(7)))
True


The last line is the reason why the rich comparison methods should
have been designed to raise NotImplementedException rather than return
NotImplemented, but it's too late to change that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Pavol Lisy]

On 8/21/17, Chris Angelico  wrote:
> On Mon, Aug 21, 2017 at 12:19 PM, MRAB  wrote:
>> On 2017-08-21 03:00, Steve D'Aprano wrote:
>>>
>>> On Fri, 18 Aug 2017 04:55 pm, Marko Rauhamaa wrote:
>>>
 Is a Python implementation
 allowed to parallelize or otherwise reorder the evaluation loop?
>>>
>>>
>>> No.
>>>
>> [snip]
>>
>> Well, I suppose an implementation _could_ parallelise, or whatever,
>> _provided that_ it gave the same result.
>
> In other words, it's allowed to parallelise, just as long as
> everything happens sequentially. With arbitrary expressions, one of
> them could affect another easily.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>


import multiprocessing
def square(param):
ret = param[0] * param[0]
param[1].put("%d is done" % param[0])
return ret

pool = multiprocessing.Pool(processes=3)
m = multiprocessing.Manager()
q = m.Queue()
squares = pool.map(square, ((i, q) for i in range(10)))
print(squares)  # ordered
print([q.get() for i in range(q.qsize())])  # unordered

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
['0 is done', '2 is done', '1 is done', '3 is done', '4 is done', '5
is done', '7 is done', '6 is done', '8 is done', '9 is done']

You could collect result sequentially (ordered) but calculate it
parallel (unordered).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Stephan Houben]

Op 2017-08-11, Paul Rubin schreef :
> I don't think we need this since we have itertools.takewhile:
>
>   from operator import gt
>   from functools import partial
>   from itertools import takewhile
>
>   [x + 1 for x in takewhile(partial(gt,5), (0,1,2,999,3,4))]
>

No need for partial and gt.

  [x + 1 for x in takewhile((5).__gt__, (0,1,2,999,3,4))]

Basically, Haskell's infix opererator sections can often be
translated into Python by attribute access to the bound method.

Stephan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Paul Rubin :
> As you mention, Russell was the one who recognized the inconsistency
> in Frege's system, though I don't know how Frege took it at a personal
> level.

I read (can't remember where) that Frege was just finishing his third
and final volume when Russell's paradox was brought to his attention.
Frege was devastated, as he had spent a large part of his life on this
extensive treatise of mathematics. It turned out, though, Frege's
mistakes were fixable.

BTW, the main take of the metamathematical "fiasco" was that you can't
get rid of the meta-level. There's no consistent logical system that is
closed and encompasses everything including itself. You will always have
to step outside your formal system and resort to hand-waving in a
natural language.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Steve D'Aprano wrote:


from io import StringIO  # simulate reading from a file
myfile = StringIO('Is this the room for an argument?')
values = [myfile.read(1) for i in range(33)]
print(''.join(values))


Which is a very contrived and longwinded way to write

   print(myfile.read(33))

and would be much better written that way.

Nobody is denying that it's *possible* to write comprehensions
that rely on order of evaluation. That doesn't mean it's a good
idea to do so.


they also care
about what it contains, and that the comprehension or generator expression
iterates over its argument in a specific order.


I don't think they really care about that. What they actually
care about is that the resulting elements are in the same
*order* as the corresponding source elements. That's a
concept that doesn't inherently have anything to do with
time, so you can think about it without having to visualise
things happening sequentially.

Or at least you can as long as the comprehension doesn't
have anything like 'while' in it. If it does, then the
abstraction isn't just a bit leaky, it has a massive
iceberg-sized hole in the hull.


[format(linenum) + line for (linenum, line) in enumerate(myfile)]

I think that most people would be disturbed if the returned list didn't match
the order of lines in the file.


The order of the elements in the list, yes, but not the
order in time in which they were generated.


If we write:

[process(a) for a in (1, 2, 3)]


then we get ridiculed for being a clueless noobie who
doesn't know how to use comprehensions properly. :-)

(Actually, this being c.l.p, we just get it gently pointed
out that we're uselessly building a list of Nones and
then throwing it away, so it would be much better to
write it as a for-statement instead. But the essence is
the same.)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Rustom Mody  writes:
> Do you mean Frege or Cantor?

Frege.  Cantor was concerned with set theory, while Frege was concerned
with logic in general.  Frege's notation was different from what we use
now but the issue was about the same: unrestricted comprehension led to
contradiction.  As you mention, Russell was the one who recognized the
inconsistency in Frege's system, though I don't know how Frege took it
at a personal level.

Many of the original writings from that era are reproduced in the book
"From Frege to Gӧdel: A Source Book in Mathematical Logic".  It has good
introductions to the papers and is pretty interesting.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Sunday, August 20, 2017 at 11:00:22 AM UTC+5:30, Paul Rubin wrote:
> Rustom Mody writes:
> > Specifically the term 'comprehension' used today as a programming construct
> > traces somewhat tenuously to an axiom that Zermelo/Fraenkel formulated
> > in the 1920s
> 
> I thought went back to Frege.  Also, it appears in Zermelo set theory Z.
> ZF is Z with the Axiom of Replacement added, but Z was somewhat earlier
> than ZF.

Do you mean Frege or Cantor?

No I am not an historian, not even amateur, so my data may be quite screwed up
What I know [open to correction!]

- Cantor invented naive¹ Set Theory — late 19th century
- He was the first to discover paradoxes — Cantor's paradox — which were not
taken too seriously
- Russel discovered his paradox — generally regarded as a bigger problem
- Frege — inventor of predicate calculus — was, like Russel, interested in a
  sound and complete foundation of math
- Russell's paradox dashed Frege's hope and put him into depression

- This is about what (I know of) the relation between Frege and set theory
- From beginning 20th century upto 1930 uncovering paradoxes became a cottage 
industry
- While at the same time people like Hilbert were rebelling against the attempts
  'to expel us from the paradise created by Cantor'
- Zermelo-Fraenkel and Gödel-Bernays-Neumann were two of the principa lattempts
  at formalizing systems (called axiomatization then) so as to avoid the
  contradictions.  At the risk of over-simplification, the Gödel-Bernays system
  is haskell-like (typed) and the Zermelo-Fraenkel system is python-like
  (untyped)
- All this cottage-industry got decimated by Gödel's (2nd)² theorem showing
  that a single unitary axiomatization is impossible
  Today maybe we should say systematization? Or programming language ?  

Many people seem to think that all this is not related to computer science… 
Amusing!
[For Turing a 'computer' was a (dumb) mathematician doing a computation]


¹ If such anachronism is permissible. But then much of this thread is 
anachronistic!
² http://mathworld.wolfram.com/GoedelsSecondIncompletenessTheorem.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Terry Reedy]

On 8/20/2017 12:28 AM, Rustom Mody wrote:


Lives today in python in the fact that the russel-set gives a straightforward
syntax error and nothing more grandly profound

R = {x if x not in x}



R = {x for x not in x}


Try the actual Python syntax set builder expression and you get 
executable code:


sos = 
R = {x for x in sos if x not in x}

In Python, the expression creates a new set that is not a member of sos, 
so the x in 'x not in x' is never S, and there is no problem, as far as 
S is concerned, in evaluating 'x not in x'.


But, is R is equal to some set z in sos?  If yes, then we could identify 
R and z and say that R is in sos.  But the answer is No. Russell's 
'paradox' comes from separately insisting that the answer is also Yes.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 18 Aug 2017 04:16 pm, Paul Rubin wrote:

[...]
> Similarly we occasionally have to be aware of the procedural nature
> of Python list comprehensions, but most of the time we think of them
> in terms of the mathematical abstraction they are designed to resemble.


Thanks Paul, you've given me an insight into the disagreement. I strongly
disagree with your "most of the time" but I can see some justification for the
view.

I think you have hit the nail on the head regarding the argument over
comprehensions. Can we understand comprehensions as an abstraction? Yes we can:

result = [ ... magic happens in here ... ]

We can ignore the insides of the comprehension (I don't mean the implementation,
I mean literally the stuff between the square brackets) and treat it as a black
box that returns a list. Just as we can ignore the insides of a function call:

result = make_list(...)

Of course we can gloss over the fact that comprehensions are explicitly written
as iteration and ignore the details, just as we can ignore the details of any
piece of code:

result = []  # Oh, it's going to be a list.
for x in stuff:
...magic happens here...
print(result)  # Some sort of list. Probably.

and of course we gloss over code all the time, ignoring the details that aren't
important at the moment.

"Details, details, don't bother me with details!"

So from *that* perspective of taking a birds-eye view of the code, I'll grant
that if you don't care about the details of what the comprehension returns, we
can gloss over it and treat it as a magic black box that returns a list, and we
don't care how it was generated: recursively, iteratively, using GOTO or an
unrolled loop, or black magic, it doesn't matter and we don't care.

But only to a point. If all you care about is "it returns a list", then that's
fine. But of course people don't really care about *only* that, they also care
about what it contains, and that the comprehension or generator expression
iterates over its argument in a specific order. If we write:

[process(a) for a in (1, 2, 3)]

then we typically expect 1 to be processed before 2, and 3 last of all. If
you've ever iterated over a file in a comprehension, you have relied on that
fact, whether you realised or not, and even if process() has no side-effects
and you don't care about the order of evaluation per se, in general we care
about the order that the results are returned.

I dare say that there are cases where people would happily replace their list
comprehension with a parallel map() that doesn't guarantee either the order of
evaluation or the order the results are returned, given some sufficient speed
up. But that's a special case, not a universal: most of the time, we expect our
data to be processed in the order we give it, not in some arbitrary order, and
even if a parallel map was available we'd prefer iteration because it is more
easily understood and debugged and matches most people's expectations of
sequential processing.

For example, given:

[format(linenum) + line for (linenum, line) in enumerate(myfile)]

I think that most people would be disturbed if the returned list didn't match
the order of lines in the file.

With the simplest comprehensions, those with a single for loop, its easily to
let your eyes slide over the explicit iteration syntax and read it as some sort
of funny map syntax:

result = [expression ... iterable]  # [] means "map"

But for more complex comprehensions, that's much harder, if not impossible, and
you cannot get away from the execution details, and *that* is explicitly
written as iteration using explicit for-loops and conditional if statements.
Not as nested maps, or as recursion, or as a parallel set of function calls.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Mon, Aug 21, 2017 at 12:19 PM, MRAB  wrote:
> On 2017-08-21 03:00, Steve D'Aprano wrote:
>>
>> On Fri, 18 Aug 2017 04:55 pm, Marko Rauhamaa wrote:
>>
>>> Is a Python implementation
>>> allowed to parallelize or otherwise reorder the evaluation loop?
>>
>>
>> No.
>>
> [snip]
>
> Well, I suppose an implementation _could_ parallelise, or whatever,
> _provided that_ it gave the same result.

In other words, it's allowed to parallelise, just as long as
everything happens sequentially. With arbitrary expressions, one of
them could affect another easily.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[MRAB]

On 2017-08-21 03:00, Steve D'Aprano wrote:

On Fri, 18 Aug 2017 04:55 pm, Marko Rauhamaa wrote:


Is a Python implementation
allowed to parallelize or otherwise reorder the evaluation loop?


No.


[snip]

Well, I suppose an implementation _could_ parallelise, or whatever, 
_provided that_ it gave the same result.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 18 Aug 2017 04:55 pm, Marko Rauhamaa wrote:

> Is a Python implementation
> allowed to parallelize or otherwise reorder the evaluation loop?

No.

I initially was going to just say "Read the PEP, read the What's New from 2.0,
read the docs, notice the deliberate use of the same terminology as for-loops
and the similarity to generator expression syntax", but in fact there's a
simple demonstration for why list comps are not free to reorder the evaluation.


from io import StringIO  # simulate reading from a file
myfile = StringIO('Is this the room for an argument?')
values = [myfile.read(1) for i in range(33)]
print(''.join(values))


For all the talk of how people read list comps declaratively without caring
about the order the values are computed, if the above code snippet printed:

'nmoe sIsta  ret hhna fto?mri rugo'

instead of the expected 'Is this the room for an argument?', I believe that we
would all agree that behaviour a bug.

Perhaps not so much *here*, where we've paid for the full hour[1], but if it
happened to us in real-life code, certainly.




[1] "I'm sorry, I'm not allowed to argue unless you've paid."


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Sun, 20 Aug 2017 02:28 pm, Rustom Mody wrote:

>> >> | S = {2·x | x ∈ ℕ, x ≤ 10}
[...]
> There's more than just a nit-pick wrong with that expression

The expression is fine. It is a mathematical expression, not Haskell code, so
your example of Haskell code is irrelevant to judging maths notation.


> Here’s an actual Haskell run
> 
> Prelude> [x*2 | x <- [1..], x <= 10]
> [2,4,6,8,10,12,14,16,18,20^CInterrupted.
> Prelude>
> 
> ie after “…,20” instead of printing a ']' and giving back the  "Prelude>"
> prompt it hangs… searching in the entire set of integers > 10…
> for an integer <= 10 (!!)
> …until a Control-C is given

If only Haskell supported the Clojure syntax:

(for [x (range 1) :while (<= x 10)] x*2)

(I may have got the order of x and 10 backwards, I haven't tested the above in a
Clojure interpreter.)

http://clojuredocs.org/clojure.core/for


> What’s the conclusion?? Take your pick:
> - Functional programming is stupid
> - Haskell is not all that intelligent
> - Halting problem is unsolvable
> - Converting the borderline uncomputable notion of set builder /comprehensions
>   into the computational framework of programming is fraught with trouble
> - Sets are a harder data-structure from computational pov than lists
> - ??

None of the above. How about...?

- abstract mathematics is not programming.


[...]
> Lives today in python in the fact that the russel-set gives a straightforward
> syntax error and nothing more grandly profound
> 
 R = {x if x not in x}
>   File "", line 1
> R = {x if x not in x}
> ^
> SyntaxError: invalid syntax

Rustom, please write 100 times on the blackboard, "I cannot mechanically
translate abstract maths notation into programming languages without
understanding the semantics of both."

Thank you.

All you have demonstrated is that mechanically translating abstract maths
notation into Python code without considering the semantics of either is, in
general, doomed to fail.



> Strawman argument as usual!

If I had a dollar for every time somebody misused "strawman argument" to mean "a
valid argument against my position that I cannot counter", I'd be able to
afford to purchase a small island in the Pacific. Such as New Zealand.


> For myself, thanks to Peter's clarification that 'comprehension' is best
> thought of as 'comprise', I am now going to teach to my students:
> “Comprehension is a misspelling of comprision”

Only if you wish to mislead your students.

"How do you know that, Herr Doktor Professor Mody?"

"Oh, some random person on the internet gave me his guess as to the etymology of
the word, I made up my own word 'comprision', and put 2 and 2 together to get
7. Any other questions?"

Comprehension is not a misspelling of anything, and I don't think Peter is
correct. According to the OED, the etymology of "comprehension" comes from
French compréhension or Latin comprehensio, to seize. Comprise is also derived
from the same words, but independently. Both are first attested to in the late
Middle Ages.

By the way, this was discussed in detail at least once before:

http://code.activestate.com/lists/python-list/123564/


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Sun, Aug 20, 2017 at 2:28 PM, Rustom Mody  wrote:
> So if Chris can answer how to teach music to a tone-deaf person, I can
> consider how to answer the question of how to teach programming to a 
> math-challenged one

You don't HAVE to understand math to be a programmer. Plenty of
math-challenged people can cook. See my earlier example of recipes.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Rustom Mody  writes:
> Specifically the term 'comprehension' used today as a programming construct
> traces somewhat tenuously to an axiom that Zermelo/Fraenkel formulated
> in the 1920s

I thought went back to Frege.  Also, it appears in Zermelo set theory Z.
ZF is Z with the Axiom of Replacement added, but Z was somewhat earlier
than ZF.  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Saturday, August 19, 2017 at 9:45:48 AM UTC+5:30, Steve D'Aprano wrote:
> On Sat, 19 Aug 2017 12:59 am, Chris Angelico wrote:
> 
> > On Fri, Aug 18, 2017 at 11:46 PM, Rustom Mody  wrote:
> >> Compare the well-known haskell tutorial
> >> http://learnyouahaskell.com/starting-out
> >> whose comprehension intro starts:
> >>
> >> | If you've ever taken a course in mathematics, you've probably run into 
> >> set
> >> | comprehensions. They're normally used for building more specific sets out
> >> | of general sets. A basic comprehension for a set that contains the first
> >> | ten even | natural numbers is
> >>
> >> | S = {2·x | x ∈ ℕ, x ≤ 10}
> 
> For the record, this is not the best example to give, since the Natural 
> numbers
> ℕ are not well-defined. Some people include 0, and some do not, so there's a
> slight ambiguity to the above.
> 
> http://mathworld.wolfram.com/NaturalNumber.html
> 
> Despite that nit-pick, set builder notation is very common in maths, but not
> universal. It is taught in secondary education (high school) in Australia, but
> not to all students.

There's more than just a nit-pick wrong with that expression

Here’s an actual Haskell run

Prelude> [x*2 | x <- [1..], x <= 10]
[2,4,6,8,10,12,14,16,18,20^CInterrupted.
Prelude> 

ie after “…,20” instead of printing a ']' and giving back the  "Prelude>" prompt
it hangs… searching in the entire set of integers > 10…
for an integer <= 10 (!!)
…until a Control-C is given

What’s the conclusion?? Take your pick:
- Functional programming is stupid
- Haskell is not all that intelligent
- Halting problem is unsolvable
- Converting the borderline uncomputable notion of set builder /comprehensions
  into the computational framework of programming is fraught with trouble
- Sets are a harder data-structure from computational pov than lists
- ??

> 
> 
> 
> >> Analogous thing shown at ghci prompt:
> >>
> >> | ghci> [x*2 | x <- [1..10]]
> >> | [2,4,6,8,10,12,14,16,18,20]

So not really analogous!
[Its the first time I am reading that article/book… just searched a standard
Haskell tutorial source and posted it]

> > 
> > And what if you HAVEN'T taken a course in mathematics? What use is
> > this then? How would you teach this to a non-mathematician?
> 
> Speaking as someone who *has* taken a course of mathematics or two, I find 
> that
> Rustom's insistence in coming back to the fundamentals of set theory and the
> Zermelo–Fraenkel axioms is not terribly helpful. Even among professional
> mathematicians. Z-F and the axiom of choice and related matters are extremely
> specialised and abstract fields of little interest to the average working
> mathematician.

Dunno where you got that
My reference to Zermelo-Fraenkel was entirely from the point of tracing the 
history,
not to say that the logic-studies of a 100 years ago has any relevance to today
Specifically the term 'comprehension' used today as a programming construct
traces somewhat tenuously to an axiom that Zermelo/Fraenkel formulated in the 
1920s

Lives today in python in the fact that the russel-set gives a straightforward
syntax error and nothing more grandly profound

>>> R = {x if x not in x}
  File "", line 1
R = {x if x not in x}
^
SyntaxError: invalid syntax
>>> 
ie the first element of a comprehension must be a 'for' not

Almost…

Unfortunately python muddies the discussion by overloading predicate 'in'
and generator 'in'. So following follows the stricture above but still does not 
work 

>>> R = {x for x not in x}
  File "", line 1
R = {x for x not in x}
   ^
SyntaxError: invalid syntax
>>> 



> 
> In my experience, they're of more interest to philosophers and dilettantes 
> than
> actual mathematicians, outside of the minority working in that specific field.
> 
> Yes yes, it is absolutely fundamental to mathematics, just as quantum 
> mechanics
> is absolutely fundamental to an understanding of matter. How many bridge
> builders care about quantum mechanics?
> 
> Python is not Haskell and makes no pretence at being mathematically sound[1].
> The Zen of Python sets forth some of the design principles in the language,
> and "mathematical purity" is not one of them.
> 
> The opposite, in fact: "practicality beats purity."
> 
> To answer your (Chris') question:

Strawman argument as usual!
For myself, thanks to Peter's clarification that 'comprehension' is best
thought of as 'comprise', I am now going to teach to my students:
“Comprehension is a misspelling of comprision”
[Comprehensivesion would be more tolerable semantically than comprehension
but hurts mouth and eyes!]

> 
> When I teach comprehension syntax, I always mention set builder notation and
> say "you may have been taught this is school". I don't think I have ever come
> across someone who both *was* taught it and *remembers* so, but I'll keep
> trying. For those who don't understand set builder notation (so far, everyone
> I've tried to teach comps to) I explain them in 

Re: Proposed new syntax

[Steve D'Aprano]

On Sat, 19 Aug 2017 03:42 pm, Chris Angelico wrote:

>> [1] Assuming that mathematics actually is sound, which thanks to Gödel we
>> [know
>> is unprovable.
> 
> Harmony in audio signals is based on frequency ratios. Therefore sound
> is mathematics, and by the reflexive principle of equality,
> mathematics is sound. Sorry, Gödel, I just proved it.

All prime numbers are odd.

The primes 3, 5, 7, 11, 13, ... are not divisible by two, which makes them odd
numbers.

The prime 2 is unique in being the only prime which is divisible by two, which
makes it the oddest prime of all.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Sat, Aug 19, 2017 at 2:15 PM, Steve D'Aprano
 wrote:
> Indeed. People find imperative (recipe) algorithms easy to follow, and pure
> functional reasoning hard. I'm glad that functional programming is fashionable
> again, and hope that people will learn good habits from it, but I think that
> mathematical purity is not necessary or even helpful in the majority of
> programming tasks.
>
> I expect that languages like Haskell, like Scheme and Lisp, will be greatly
> influential but remain niche languages themselves.

Agreed. Functional programming languages teach us about recursion,
immutability, declarative programming styles, etc, all of which are
incredibly useful. I don't particularly enjoy writing Scheme code
(only ever done it because of Lilypond), but I think I'm a better
programmer for having gotten my head around it.

>
> [1] Assuming that mathematics actually is sound, which thanks to Gödel we know
> is unprovable.

Harmony in audio signals is based on frequency ratios. Therefore sound
is mathematics, and by the reflexive principle of equality,
mathematics is sound. Sorry, Gödel, I just proved it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Sat, 19 Aug 2017 12:59 am, Chris Angelico wrote:

> On Fri, Aug 18, 2017 at 11:46 PM, Rustom Mody  wrote:
>> Compare the well-known haskell tutorial
>> http://learnyouahaskell.com/starting-out
>> whose comprehension intro starts:
>>
>> | If you've ever taken a course in mathematics, you've probably run into set
>> | comprehensions. They're normally used for building more specific sets out
>> | of general sets. A basic comprehension for a set that contains the first
>> | ten even | natural numbers is
>>
>> | S = {2·x | x ∈ ℕ, x ≤ 10}

For the record, this is not the best example to give, since the Natural numbers
ℕ are not well-defined. Some people include 0, and some do not, so there's a
slight ambiguity to the above.

http://mathworld.wolfram.com/NaturalNumber.html

Despite that nit-pick, set builder notation is very common in maths, but not
universal. It is taught in secondary education (high school) in Australia, but
not to all students.



>> Analogous thing shown at ghci prompt:
>>
>> | ghci> [x*2 | x <- [1..10]]
>> | [2,4,6,8,10,12,14,16,18,20]
> 
> And what if you HAVEN'T taken a course in mathematics? What use is
> this then? How would you teach this to a non-mathematician?

Speaking as someone who *has* taken a course of mathematics or two, I find that
Rustom's insistence in coming back to the fundamentals of set theory and the
Zermelo–Fraenkel axioms is not terribly helpful. Even among professional
mathematicians. Z-F and the axiom of choice and related matters are extremely
specialised and abstract fields of little interest to the average working
mathematician.

In my experience, they're of more interest to philosophers and dilettantes than
actual mathematicians, outside of the minority working in that specific field.

Yes yes, it is absolutely fundamental to mathematics, just as quantum mechanics
is absolutely fundamental to an understanding of matter. How many bridge
builders care about quantum mechanics?

Python is not Haskell and makes no pretence at being mathematically sound[1].
The Zen of Python sets forth some of the design principles in the language,
and "mathematical purity" is not one of them.

The opposite, in fact: "practicality beats purity."

To answer your (Chris') question:

When I teach comprehension syntax, I always mention set builder notation and
say "you may have been taught this is school". I don't think I have ever come
across someone who both *was* taught it and *remembers* so, but I'll keep
trying. For those who don't understand set builder notation (so far, everyone
I've tried to teach comps to) I explain them in terms of for loops.

In fact, even if you do understand set builder notation, for more complex
examples with "if" clauses and multiple "for" loops, I maintain that you have
to think of it in terms of loops to understand it. I am extremely skeptical
that anyone could look at a comprehension like:

[expr for x in A for y in B if P for z in C if Q for w in D for v in E if R]

and understand it *without* converting it to nested loops.


> Pretty much everyone, at some point in their lives, will follow a set
> of written instructions. Most commonly, a recipe for some sort of
> food. It consists of a set of ingredients and a sequence of commands.
> This translates well into a classic imperative style

Indeed. People find imperative (recipe) algorithms easy to follow, and pure
functional reasoning hard. I'm glad that functional programming is fashionable
again, and hope that people will learn good habits from it, but I think that
mathematical purity is not necessary or even helpful in the majority of
programming tasks.

I expect that languages like Haskell, like Scheme and Lisp, will be greatly
influential but remain niche languages themselves.





[1] Assuming that mathematics actually is sound, which thanks to Gödel we know
is unprovable.

-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 18 Aug 2017 11:46 pm, Rustom Mody wrote:

> My issue is that the tutorial introduces comprehensions backwards, as though
> they are a macro for the for-loop

Python doesn't have macros as such, but if it did, that would be a good way to
describe comprehensions.

Another way would be to say that comprehensions are syntactic sugar for a
for-loop and accumulator.

You say "as though they are", but that's exactly the point -- that is precisely
what they are.


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Thu, 17 Aug 2017 03:54 pm, Marko Rauhamaa wrote:

> Ben Finney :
> 
>> The Python list comprehension syntax does not specify control flow.
> 
> I understand your point, but how do you know your statement is true?
> 
> I didn't check this, but I would imagine the list comprehension:
> 
>[ f(x) for x in I if c(x) ]
> 
> was defined as syntactic sugar for:
> 
>list(f(x) for x in I if c(x))

Not originally. List comprehensions were added to Python in version 2.0, before
generator expressions. (Generator expressions only made it into the language in
version 2.4.)

But right from 2.0, they have been explicitly defined as equivalent to for
loops:

https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Doc/whatsnew/2.0.rst#list-comprehensions

In Python 3, it would make sense for list/set/dict comprehensions to be thin
wrappers around generator expressions, but I don't know if they share
implementations that way or have separate "copy and paste" implementations. I
can't find where they are implemented.

This does leave a *tiny* bit of wriggle-room: the implementation (in C, say, or
some other language) doesn't have to use a for-loop. It could use recursion, or
a while-loop, or it might unroll the loop into a long series of imperative
statements, or use GOTO to emulate looping. So long as the Python semantics
remain the same, and the elements are generated in the same order.


> where
> 
>f(x) for x in I if c(x)
> 
> has heavily iterative semantics.

Indeed. For loops are one of the cardinal examples of iteration. It isn't a
coincidence that generator expressions and comprehensions use the same syntax.

Whatever implementation is used, the semantics must be the same as the
equivalent for loop procedural code. For example:


from io import StringIO
myfile = StringIO('Is this the room for an argument?')
values = [myfile.read(1) for i in range(33)]
assert values == list('Is this the room for an argument?')


Python is not free to generate the results in some other order.



>> List comprehensions, in their syntax, strongly connote a single
>> declarative operation. This makes them different from iteration
>> syntax.
> 
> Except for side effects, you get the same result whichever way you
> imagine it.

That's not correct, as the example of reading from a file demonstrates.


> As a side not, I have found it rather confusing that I have to say:
> 
>   [ (x, y) for x in range(5) for y in range(x) ]
> 
> instead of:
> 
>   [ (x, y) for y in range(x) for x in range(5) ]

That doesn't work because x isn't defined when you call range(x).


> After all, you must say:
> 
>   [ ((x, y) for y in range(x)) for x in range(5) ]
> 
> which, admittedly, means a different thing.

Indeed. Completely different.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Fri, Aug 18, 2017 at 11:46 PM, Rustom Mody  wrote:
> Compare the well-known haskell tutorial
> http://learnyouahaskell.com/starting-out
> whose comprehension intro starts:
>
> | If you've ever taken a course in mathematics, you've probably run into set
> | comprehensions. They're normally used for building more specific sets out of
> | general sets. A basic comprehension for a set that contains the first ten 
> even | natural numbers is
>
> | S = {2·x | x ∈ ℕ, x ≤ 10}
>
> Analogous thing shown at ghci prompt:
>
> | ghci> [x*2 | x <- [1..10]]
> | [2,4,6,8,10,12,14,16,18,20]

And what if you HAVEN'T taken a course in mathematics? What use is
this then? How would you teach this to a non-mathematician?

Pretty much everyone, at some point in their lives, will follow a set
of written instructions. Most commonly, a recipe for some sort of
food. It consists of a set of ingredients and a sequence of commands.
This translates well into a classic imperative style - for instance:

Ingredients.
100 g flour
250 g butter
1 egg

Method.
1. Sift the flour.
2. Put flour into mixing bowl.
3. Serve with caramel sauce.
4. Stir for 2 minutes.
5. Remove egg.
6. Rub the flour until sifted.
7. Stir for 2 minutes.
8. Fold the butter into the mixing bowl.
9. Pour contents of the mixing bowl into the baking dish.

Serves 1.

You might recognize that this is a part of a recipe for Fibonacci
numbers [1], but aside from a few tricky instructions like "remove
egg", it's no worse than a recipe for chocolate ripple cake [2]:

Ingredients
300g Chocolate Ripple biscuits
600mL thickened cream (for whipping)
1tsp vanilla essence
Sugar (“some”)
6-12 Lindor balls (to taste)
Rectangular plate/dish to work on (about the size of the packet of biscuits)

Directions
1. Collect ingredients. Read all the instructions and comprehend them.
2. Open the packet of chocolate biscuits. Find a broken one and eat it.
3. Taste a Lindor ball.
4. Whip the cream in a jug with plenty of room.
5. Add some sugar to the cream. If you have chocolate-flavoured sugar, use it.
... etc ...

Anyone who's worked with this kind of recipe will have no difficulty
understanding the concept of imperative code.

# Ingredients
import math
num = 42

# Mixing bowl
guess = 1

# Method
while not done:
guess = num / guess
if guess hasn't changed:
done is True

print num to the screen

And apart from a small matter of syntax and a need to break down the
concept "hasn't changed" into another variable and a comparison, this
is valid Python code. Valid *imperative* code. It's the one thing that
practically everyone, not just those elite few who actually comprehend
higher mathematics, can easily grok.

ChrisA

[1] http://www.dangermouse.net/esoteric/chef_fib.html
[2] http://rosuav.github.io/shed/ChocRippleCake
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

There is code and there are machines
There are Turing machines and Universal Turing machines
There are programs and there are programming languages
There are (il)legal programs and best/worst (software engineering) practices

As best as I can see most of us are talking of the respective latters above
while Steven insists on taking this whole discussion exclusively in terms of 
the formers

Picked up some quotes from the thread above and ALLCAPSed the phrases that
would hopefully show the difference

Paul writes
> Sure, and floating point arithmetic is inherently imprecise and
> doesn't follow the associative laws for either addition or
> multiplication. There are times when we have to be aware of those
> details. Usually, though, WE WANT TO ACT AS IF they represent the
> mathematical reals, and we read Python statements involving floats as
> if they were mathematical statements involving reals. 

Marko Rauhamaa writes:
> Nothing prevents you from using Python's comprehensions that declarative
> way. THE QUESTION IS, IS IT BAD STYLE—OR EVEN AN ERROR—to rely on the
> execution order of the comprehension loops? Is a Python implementation
> allowed to parallelize or otherwise reorder the evaluation loop?

Greg Ewing writes:
> Perhaps what I should say is that relying on side effects in
> an expression occurring in a particular order IS A BAD IDEA. (not wrong)

Ben Finney writes:
At what point will you accept the feedback: That the comprehension
syntax *does not* necessarily CONNOTE (not denote) a procedural loop, but 
instead can quite REASONABLY (not guaranteedly/necessarily) be interpreted as 
its designer intended, a single conceptual operation. 


I just want to emphasise that (for myself) Ive no quarrel with the current 
semantics
My issue is that the tutorial introduces comprehensions backwards, as though 
they are a macro for the for-loop
https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

Compare the well-known haskell tutorial 
http://learnyouahaskell.com/starting-out
whose comprehension intro starts:

| If you've ever taken a course in mathematics, you've probably run into set 
| comprehensions. They're normally used for building more specific sets out of 
| general sets. A basic comprehension for a set that contains the first ten 
even | natural numbers is

| S = {2·x | x ∈ ℕ, x ≤ 10}

Analogous thing shown at ghci prompt:

| ghci> [x*2 | x <- [1..10]]  
| [2,4,6,8,10,12,14,16,18,20] 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Marko Rauhamaa  writes:
> The question is, is it bad style—or even an error—to rely on the
> execution order of the comprehension loops? 

Bad style: IMO, yes, most of the time.  I've made use of it at
particular times.  If it's done in an obscure way it at least
deserves a code commment.

Error: no, I think the language spec is clear.  It's not like the
situation of people depending on reference counting to free resources
predictably.  Use the 'with' statement for that.

> Is a Python implementation allowed to parallelize or otherwise reorder
> the evaluation loop?

No.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Paul Rubin :

> Steve D'Aprano  writes:
>> For loops and comprehensions (in Python) are inherently procedural,
>
> Sure, and floating point arithmetic is inherently imprecise and
> doesn't follow the associative laws for either addition or
> multiplication. There are times when we have to be aware of those
> details. Usually, though, we want to act as if they represent the
> mathematical reals, and we read Python statements involving floats as
> if they were mathematical statements involving reals.

Nothing prevents you from using Python's comprehensions that declarative
way. The question is, is it bad style—or even an error—to rely on the
execution order of the comprehension loops? Is a Python implementation
allowed to parallelize or otherwise reorder the evaluation loop?

Unfortunately, the spec is rather vague:

   they are computed via a set of looping and filtering instructions

   [...]

   the elements of the new container are those that would be produced by
   considering each of the for or if clauses a block, nesting from left
   to right, and evaluating the expression to produce an element each
   time the innermost block is reached.

   https://docs.python.org/3/reference/expressions.html#display
   s-for-lists-sets-and-dictionaries>

Thus, the scholastic debate may rage on.

> Similarly we occasionally have to be aware of the procedural nature of
> Python list comprehensions, but most of the time we think of them in
> terms of the mathematical abstraction they are designed to resemble.

I understand your concern. With the introduction of a "while" in the
syntax, the beautiful illusion would be shattered.

However, I think the concept of a generator expression is more
fundamental in Pythonic thinking. Also, since Python has chosen to
specify the precize execution order in:

   { f(g(), h()) * k() : z() }

I would think comprehensions would be put in a similar straitjacket.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Steve D'Aprano  writes:
> For loops and comprehensions (in Python) are inherently procedural,

Sure, and floating point arithmetic is inherently imprecise and doesn't
follow the associative laws for either addition or multiplication.
There are times when we have to be aware of those details.  Usually,
though, we want to act as if they represent the mathematical reals, and
we read Python statements involving floats as if they were mathematical
statements involving reals.  When possible, we write in a style where
this doesn't cause problems, use double precision to decrease rounding
errors in long calculations, etc.

Similarly we occasionally have to be aware of the procedural nature
of Python list comprehensions, but most of the time we think of them
in terms of the mathematical abstraction they are designed to resemble.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Pavol Lisy]

On 8/17/17, Marko Rauhamaa  wrote:
> Pavol Lisy :
>
>> On 8/17/17, Gregory Ewing  wrote:
>>> I don't agree that the word "for" necessarily implies proceduralness.
>>
>> With this logic (I humbly think that) word "while" neither
>> necessarilly implies proceduralness.
>
> I don't see the logic.
>
> Here's a random page from an algebra textbook:
>
>A sequence {a[n]} is called *bounded* if there is some real number b,
>a *bound*, such that |a[n]| ≤ b for all n.
>
>
>Z = {(a) ∈ ℝ**∞ | a[n] = 0 for all but finitely many n }
>
> On another one:
>
><χ[i], χ[j]> = 0 if i ≠ j, and <χ[i], χ[j]> = 1 for each i
>
>
> And:
>
>For every value a of the variable x, there are at most n points of S
>whose x-coordinate is a.
>
>
> Marko
> --
> https://mail.python.org/mailman/listinfo/python-list

I meant while "implemented" like this (N is set of natural numbers) ->

   { n ∈ N | condition(m) for all m <= n }

or more correctly:
   { n ∈ N | condition(m) for all m ∈ { o ∈ N | o <= n } }
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Thu, Aug 17, 2017 at 11:05 PM, Steve D'Aprano
 wrote:
> On Fri, 18 Aug 2017 06:28 am, Ian Kelly wrote:
>> "while" implies a condition that is currently true and may at some
>> point stop being true. To me, that sequentiality does imply
>> proceduralness.
>
> For loops and comprehensions (in Python) are inherently procedural, so it's a
> good match. If you want to ignore the documented semantics of Python
> comprehensions, their fundamental similarities to generator expressions, and
> their actual behaviour, in order to say that they aren't procedural, that's
> okay, it a free planet. (Sometimes.)
>
> But if you do that, then logically you must also allow that "while" is 
> likewise
> not procedural. Otherwise you're just showing a double-standard.

I explained my logic here. You say that logically "while" is likewise
not procedural, but you didn't explain what that logic is, so I have
no idea what you're referring to.


>> "for", to quote from Merriam-Webster, indicates "equivalence in
>> exchange". A precise definition seems hard to nail down, but there's
>> no such implication there.
>
>
> Webster 1913 includes well over a dozen definitions for the word "for", so I'm
> sure that Merrian-Webster would be similar. I don't know why you cherry picked
> that one definition out of so many.

"For" in the loop sense takes the meaning that it has in "for each"
(hence why we sometimes call it a for-each loop). This is the only one
that makes sense to me in that context.

> I think this is more appropriate:
>
>8. Indicating the space or time through which an action or
>   state extends; hence, during; in or through the space or
>   time of.

The example phrase for that one is "gone for two days". I don't see
how you can possibly interpret this in the sense of a for-each loop.

> "for x in seq" indicates that x extends through the space of seq, in a manner 
> of
> speaking.

No, given the definition above, it would indicate that something
extends through the space of x. Again, this makes zero sense to me.

> Or for something more recent, and more relevant, how about foldoc, the Free
> On-line Dictionary of Computing?

Completely misses the point of consulting the dictionary in the first
place. We were discussing the implications of the use of the English
definition. A technical computing definition is irrelevant.

> But if we can do that, we can likewise do the same for "while".

Okay, I'm waiting. What is your proposed "non-procedural" definition of "while"?

> You can't have it both ways:
>
> (1) if we can ignore the proceduralness of "for", then we can ignore the
> proceduralness of "while" and put it in a comprehension;
>
> (2) if we *cannot* ignore the proceduralness of "while", then we likewise 
> cannot
> ignore the proceduralness of "for", therefore comprehensions are fundamentally
> procedural and that objection against "while" is neutered.

This claim is absurd. They're *different words* with different
definitions and entirely different connotations. They're not even the
same part of speech. You might as well be saying that if we can ignore
that "swim" is a color, then we can also ignore that "red" is a color.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 18 Aug 2017 06:28 am, Ian Kelly wrote:

> On Thu, Aug 17, 2017 at 1:19 PM, Pavol Lisy  wrote:
>> On 8/17/17, Gregory Ewing  wrote:
>>> Steve D'Aprano wrote:
 If he wanted declarative semantics, why didn't he argue for declarative
 syntax
 like "select...where", instead of choosing procedural syntax which matches
 the
 actual procedural semantics given?
>>>
>>> I don't agree that the word "for" necessarily implies proceduralness.
>>
>> With this logic (I humbly think that) word "while" neither
>> necessarilly implies proceduralness.
> 
> "while" implies a condition that is currently true and may at some
> point stop being true. To me, that sequentiality does imply
> proceduralness.

For loops and comprehensions (in Python) are inherently procedural, so it's a
good match. If you want to ignore the documented semantics of Python
comprehensions, their fundamental similarities to generator expressions, and
their actual behaviour, in order to say that they aren't procedural, that's
okay, it a free planet. (Sometimes.)

But if you do that, then logically you must also allow that "while" is likewise
not procedural. Otherwise you're just showing a double-standard.

[Aside: comprehensions are also functional in the sense that they return a
result (unlike a loop, which is imperative) but not in the sense that the order
of execution is unspecified.]


> "for", to quote from Merriam-Webster, indicates "equivalence in
> exchange". A precise definition seems hard to nail down, but there's
> no such implication there.


Webster 1913 includes well over a dozen definitions for the word "for", so I'm
sure that Merrian-Webster would be similar. I don't know why you cherry picked
that one definition out of so many. I think this is more appropriate:

   8. Indicating the space or time through which an action or
  state extends; hence, during; in or through the space or
  time of.

"for x in seq" indicates that x extends through the space of seq, in a manner of
speaking.

Or for something more recent, and more relevant, how about foldoc, the Free
On-line Dictionary of Computing?

for loop
for

A loop construct found in many procedural
   languages which repeatedly executes some instructions while a
   condition is true.

   [examples snipped]

   The for loop is an alternative way of writing a while loop
   that is convenient because the loop control logic is collected
   in a single place.  It is also closely related to the repeat
   loop.



Sounds pretty definitively procedural to me.

But as Lewis Carrol wrote:

 “When I use a word, Humpty Dumpty said, in a rather scornful tone, 
 it means just what I choose it to mean, neither more nor less.

 The question is, said Alice, whether you can make words mean so many
 different things.

 The question is, said Humpty Dumpty, which is to be master — that’s all.”


Of course we can choose any definition or meaning we like for the word "for" in
comprehensions, and insist that it doesn't actually mean what the "for" in
for-statement means. We're the master of language, not language of us!

But if we can do that, we can likewise do the same for "while".

You can't have it both ways:

(1) if we can ignore the proceduralness of "for", then we can ignore the
proceduralness of "while" and put it in a comprehension;

(2) if we *cannot* ignore the proceduralness of "while", then we likewise cannot
ignore the proceduralness of "for", therefore comprehensions are fundamentally
procedural and that objection against "while" is neutered.


There may be other objections against "while", but any that revolve around the
declarativeness or non-proceduralness of comprehensions are logically invalid.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Fri, 18 Aug 2017 01:11 am, Lew Pitcher wrote:

> Marko Rauhamaa wrote:

>> I guess we have C to blame for the redefinition of the word "for" in
>> programmers' minds.
> 
> Sorry, but that use of "for" was part of the programmer's lexicon well
> before the invention of C. Computer languages have inherited and used it
> since (at least) 1948.

That can't be right! Programming wasn't invented until 1972, when C sprung
forth, fully formed and perfect, with no predecessors, from the mind of Dennis
Ritchie!


*wink*


I wish people knew more about the history of programming.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Fri, Aug 18, 2017 at 11:13 AM, Steve D'Aprano
 wrote:
> Within a single language, it is common to have both procedural and declarative
> statements. E.g. in Python for statements are clearly procedural flow control.
> But "import" can be read as a declarative statement:
>
> "Load this module, I don't care how you do it or where you have to search or
> what order you do the searching, so long as you load this module."
>
> Arguably its not *really* declarative, because we can implicitly control the
> search order by manipulating sys.modules and sys.path and a few other ways. 
> But
> that's splitting hairs -- by that standard, SQL SELECT isn't declarative
> either, because databases often give you a way to fine-tune the query plan.

One way to figure out whether it's declarative or imperative is to
consider whether it would be legal for (a) a different Python
implementation, or (b) a different version of the same Python
implementation, to achieve the same goal in a different way. An import
statement is definitely somewhat declarative, because you can
*externally* change the mechanics of the command. (For instance,
activating a virtual environment, or otherwise messing with
PYTHONPATH.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Thu, 17 Aug 2017 10:47 pm, Gregory Ewing wrote:

> Steve D'Aprano wrote:
>> Do you consider:
>> 
>> for foo in bar:
>> if baz(foo):
>> f(foo)  # append to a list, or print, or yield, as needed
>> 
>> declarative?
> 
> No. But the reason for that is not because it has the word
> "for" in it. The reason is that it's made up of statements.
> Statements are procedural, expressions are declarative.

Not necessarily.

If we can't agree on definitions for procedural and declarative, I'm not sure
that this conversation is going to go anywhere. But the established definition
is that:

- in procedural code (although possibly "imperative" may be a better term) you
have to explicitly control the flow of program state yourself

- in declarative code you don't, you state the desired outcome and the
interpreter is free to choose its own execution flow to meet that outcome.

Whether your code is made of statements or expressions or both is not relevant.
You can have declarative statements, if the interpreter is free to perform them
in any order.


Within a single language, it is common to have both procedural and declarative
statements. E.g. in Python for statements are clearly procedural flow control.
But "import" can be read as a declarative statement:

"Load this module, I don't care how you do it or where you have to search or
what order you do the searching, so long as you load this module."

Arguably its not *really* declarative, because we can implicitly control the
search order by manipulating sys.modules and sys.path and a few other ways. But
that's splitting hairs -- by that standard, SQL SELECT isn't declarative
either, because databases often give you a way to fine-tune the query plan.


>> SQL's SELECT ... WHERE is one of the canonical examples of declarative
>> programming:
> 
> Despite the fact that "select" is a verb in English. :-)

"Select" isn't flow control.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Thu, Aug 17, 2017 at 1:19 PM, Pavol Lisy  wrote:
> On 8/17/17, Gregory Ewing  wrote:
>> Steve D'Aprano wrote:
>>> If he wanted declarative semantics, why didn't he argue for declarative
>>> syntax
>>> like "select...where", instead of choosing procedural syntax which matches
>>> the
>>> actual procedural semantics given?
>>
>> I don't agree that the word "for" necessarily implies proceduralness.
>
> With this logic (I humbly think that) word "while" neither
> necessarilly implies proceduralness.

"while" implies a condition that is currently true and may at some
point stop being true. To me, that sequentiality does imply
proceduralness.

"for", to quote from Merriam-Webster, indicates "equivalence in
exchange". A precise definition seems hard to nail down, but there's
no such implication there.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Pavol Lisy :

> On 8/17/17, Gregory Ewing  wrote:
>> I don't agree that the word "for" necessarily implies proceduralness.
>
> With this logic (I humbly think that) word "while" neither
> necessarilly implies proceduralness.

I don't see the logic.

Here's a random page from an algebra textbook:

   A sequence {a[n]} is called *bounded* if there is some real number b,
   a *bound*, such that |a[n]| ≤ b for all n.


   Z = {(a) ∈ ℝ**∞ | a[n] = 0 for all but finitely many n }

On another one:

   <χ[i], χ[j]> = 0 if i ≠ j, and <χ[i], χ[j]> = 1 for each i


And:

   For every value a of the variable x, there are at most n points of S
   whose x-coordinate is a.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Pavol Lisy]

On 8/17/17, Gregory Ewing  wrote:
> Steve D'Aprano wrote:
>> If he wanted declarative semantics, why didn't he argue for declarative
>> syntax
>> like "select...where", instead of choosing procedural syntax which matches
>> the
>> actual procedural semantics given?
>
> I don't agree that the word "for" necessarily implies proceduralness.

With this logic (I humbly think that) word "while" neither
necessarilly implies proceduralness.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Lew Pitcher]

Marko Rauhamaa wrote:

> Gregory Ewing :
>> I don't agree that the word "for" necessarily implies proceduralness.
> 
> Programming languages stole the word from math, where it is
> nonprocedural.
> 
> Really, "for" is just a preposition. In Algol, for example,
> proceduralness was not in the word "for" but in "do":
[snip]
> I guess we have C to blame for the redefinition of the word "for" in
> programmers' minds.

Sorry, but that use of "for" was part of the programmer's lexicon well 
before the invention of C. Computer languages have inherited and used it 
since (at least) 1948.

Dartmouth BASIC (1964) had "FOR"
  FOR I=1 TO 10

ALGOL-60 (1960) had "for"
  for i:=1 step 1 until 10

ALGOL-58 (1958) had "for"
  for i:=1(1)10

Superplan (1948) had "fur" (German for the English word "for")
  Für k=1(1)10 :

-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Thursday, August 17, 2017 at 8:13:24 PM UTC+5:30, Marko Rauhamaa wrote:
> Gregory Ewing :
> > I don't agree that the word "for" necessarily implies proceduralness.
> 
> Programming languages stole the word from math, where it is
> nonprocedural.
> 
> Really, "for" is just a preposition. In Algol, for example,
> proceduralness was not in the word "for" but in "do":
> 
> for p := 1 step 1 until n do
> for q := 1 step 1 until m do
> if abs(a[p, q]) > y then
> begin y := abs(a[p, q]);
> i := p; k := q
> end
> 
> https://en.wikipedia.org/wiki/ALGOL#ALGOL_60>
> 
> Pascal is similar:
> 
> for i:= 1 to 10 do writeln(i);
> 
> https://www.tutorialspoint.com/pascal/pascal_for_do_loop.htm>
> 
> As is sh:
> 
> for i in *.py; do
> mv "$i" "$i.bak"
> done
> 
> Common lisp uses "do" as well:
> 
> (setq a-vector (vector 1 nil 3 nil))
> (do ((i 0 (+ i 1)) ;Sets every null element of a-vector to zero.
>  (n (array-dimension a-vector 0)))
> ((= i n))
>   (when (null (aref a-vector i))
> (setf (aref a-vector i) 0))) =>  NIL
> 
> http://www.lispworks.com/documentation/lw60/CLHS/Body/m_do_do.
> htm>
> 
> I guess we have C to blame for the redefinition of the word "for" in
> programmers' minds.

And C’s for is just a while with a ‘finally’ clause for its inner block
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Gregory Ewing :
> I don't agree that the word "for" necessarily implies proceduralness.

Programming languages stole the word from math, where it is
nonprocedural.

Really, "for" is just a preposition. In Algol, for example,
proceduralness was not in the word "for" but in "do":

for p := 1 step 1 until n do
for q := 1 step 1 until m do
if abs(a[p, q]) > y then
begin y := abs(a[p, q]);
i := p; k := q
end

https://en.wikipedia.org/wiki/ALGOL#ALGOL_60>

Pascal is similar:

for i:= 1 to 10 do writeln(i);

https://www.tutorialspoint.com/pascal/pascal_for_do_loop.htm>

As is sh:

for i in *.py; do
mv "$i" "$i.bak"
done

Common lisp uses "do" as well:

(setq a-vector (vector 1 nil 3 nil))
(do ((i 0 (+ i 1)) ;Sets every null element of a-vector to zero.
 (n (array-dimension a-vector 0)))
((= i n))
  (when (null (aref a-vector i))
(setf (aref a-vector i) 0))) =>  NIL

http://www.lispworks.com/documentation/lw60/CLHS/Body/m_do_do.
htm>

I guess we have C to blame for the redefinition of the word "for" in
programmers' minds.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Thursday, August 17, 2017 at 5:51:45 PM UTC+5:30, Gregory Ewing wrote:
> Steve D'Aprano wrote:
> > If he wanted declarative semantics, why didn't he argue for declarative 
> > syntax
> > like "select...where", instead of choosing procedural syntax which matches 
> > the
> > actual procedural semantics given?
> 
> I don't agree that the word "for" necessarily implies proceduralness.

Thou blasphemer! Prepare to be anathematized!!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Steve D'Aprano wrote:

Do you consider:

for foo in bar:
if baz(foo):
f(foo)  # append to a list, or print, or yield, as needed

declarative?


No. But the reason for that is not because it has the word
"for" in it. The reason is that it's made up of statements.
Statements are procedural, expressions are declarative.


SQL's SELECT ... WHERE is one of the canonical examples of declarative
programming:


Despite the fact that "select" is a verb in English. :-)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Steve D'Aprano wrote:

If he wanted declarative semantics, why didn't he argue for declarative syntax
like "select...where", instead of choosing procedural syntax which matches the
actual procedural semantics given?


I don't agree that the word "for" necessarily implies proceduralness.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marko Rauhamaa]

Ben Finney :

> The Python list comprehension syntax does not specify control flow.

I understand your point, but how do you know your statement is true?

I didn't check this, but I would imagine the list comprehension:

   [ f(x) for x in I if c(x) ]

was defined as syntactic sugar for:

   list(f(x) for x in I if c(x))

where

   f(x) for x in I if c(x)

has heavily iterative semantics.

> List comprehensions, in their syntax, strongly connote a single
> declarative operation. This makes them different from iteration
> syntax.

Except for side effects, you get the same result whichever way you
imagine it.

As a side not, I have found it rather confusing that I have to say:

  [ (x, y) for x in range(5) for y in range(x) ]

instead of:

  [ (x, y) for y in range(x) for x in range(5) ]

After all, you must say:

  [ ((x, y) for y in range(x)) for x in range(5) ]

which, admittedly, means a different thing.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Steve D'Aprano  writes:

> On Thu, 17 Aug 2017 12:28 am, Ben Finney wrote:
>
> > Steve D'Aprano  writes:
> > 
> >> If he wanted declarative semantics, why didn't he argue for
> >> declarative syntax like "select...where", instead of choosing
> >> procedural syntax which matches the actual procedural semantics given?
> > 
> > The syntax “f(foo) for foo in bar if baz(foo)” is nicely declarative.
>
> Do you consider [a Python ‘for’ loop] declarative?

No, that is a syntax that can only mean iteration over the *distinct
statements* in the block.

> My understanding of "declarative" agrees with Wikipedia:

Thanks. Mine does too.

> For-loops are a classic example of explicitly specifying control flow,
> so I'm surprised that you seem to think they are declarative.

The Python for loop syntax specifies control flow.

The Python list comprehension syntax does not specify control flow.

The expression “f(foo) for foo in bar if baz(foo)” is not expressing
control flow, it is expressing a single conceptual operation.

A collection goes in, a single conceptual operation is performed, a
collection goes out. The control flow is not specified.

> SQL's SELECT ... WHERE is one of the canonical examples of declarative
> programming:
>
> SELECT *
>  FROM  Book
>  WHERE price > 100.00
>
> returns matching books (those with a price over 100) in arbitrary
> order (unless you specify the "ORDER BY" clause). But more importantly
> than the order of results, the order of data accesses is not specified
> by the SELECT statement.

Nor is the order of data accesses specified by Python's comprehension
syntax.

In both cases, of course, implementations *do* perform smaller-grained
operations in a specific control flow. You'll even find that order of
operations described in the documentation, for SQL implementations and
for Python, respectively.

> for row in Book:
> if row.price > 100:
> print(row)  # for example
>
> I think that is specifying the control flow, and therefore not
> declarative. Do you agree?

Yes, I agree.

> [...]
> > I'm responding to the proposal by pointing out that, unlike today's
> > Python, the proposed behaviour would violate those semantics.
>
> In what way?

By inserting a semantic of iteration over multiple operations, into what
is otherwise an expression with declarative semantics.

> Do you feel the same way about itertools.takewhile?

The functions in the ‘itertools’ module are not expressing a declarative
single operation; they are expressing an iteration of multiple
operations.

List comprehensions, in their syntax, strongly connote a single
declarative operation. This makes them different from iteration syntax.

-- 
 \  “Shepherds … look after their sheep so they can, first, fleece |
  `\   them and second, turn them into meat. That's much more like the |
_o__)  priesthood as I know it.” —Christopher Hitchens, 2008-10-29 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Wed, 16 Aug 2017 11:53 pm, jmp wrote:

> On 08/10/2017 04:28 PM, Steve D'Aprano wrote:
>> Every few years, the following syntax comes up for discussion, with some
>> people saying it isn't obvious what it would do, and others disagreeing and
>> saying that it is obvious. So I thought I'd do an informal survey.
>> 
>> What would you expect this syntax to return?
>> 
>> [x + 1 for x in (0, 1, 2, 999, 3, 4) while x < 5]
>> 
>> 
>> For comparison, what would you expect this to return? (Without actually
>> trying it, thank you.)
>> 
>> [x + 1 for x in (0, 1, 2, 999, 3, 4) if x < 5]
>> 
>> 
>> 
>> How about these?
>> 
>> [x + y for x in (0, 1, 2, 999, 3, 4) while x < 5 for y in (100, 200)]
>> 
>> [x + y for x in (0, 1, 2, 999, 3, 4) if x < 5 for y in (100, 200)]
>> 
>> 
>> 
>> Thanks for your comments!
>> 
> 
> [1,2,3]
> [1,2,3,4,5]
> SyntaxError("Have you tried Perl ?")
> SyntaxError("Have you tried Perl ?")
> 
> I really would not want to deal with 3 and 4.

:-)

#4 is actually valid syntax right now, and has been since Python 2.2.

py> [x + y for x in (0, 1, 2, 999, 3, 4) if x < 5 for y in (100, 200)]
[100, 200, 101, 201, 102, 202, 103, 203, 104, 204]


It is equivalent to:

for x in (0, 1, 2, 999, 3, 4):
if x < 5:
for y in (100, 200):
x + y  # collect into the list


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Thu, 17 Aug 2017 12:28 am, Ben Finney wrote:

> Steve D'Aprano  writes:
> 
>> If he wanted declarative semantics, why didn't he argue for
>> declarative syntax like "select...where", instead of choosing
>> procedural syntax which matches the actual procedural semantics given?
> 
> The syntax “f(foo) for foo in bar if baz(foo)” is nicely declarative.

Do you consider:

for foo in bar:
if baz(foo):
f(foo)  # append to a list, or print, or yield, as needed

declarative?

My understanding of "declarative" agrees with Wikipedia:

"In computer science, declarative programming is a programming paradigm ... that
expresses the logic of a computation without describing its control flow."

https://en.wikipedia.org/wiki/Declarative_programming

For-loops are a classic example of explicitly specifying control flow, so I'm
surprised that you seem to think they are declarative.


SQL's SELECT ... WHERE is one of the canonical examples of declarative
programming:

SELECT *
 FROM  Book
 WHERE price > 100.00

returns matching books (those with a price over 100) in arbitrary order (unless
you specify the "ORDER BY" clause). But more importantly than the order of
results, the order of data accesses is not specified by the SELECT statement.
Instead, the SQL compiler generates a "query plan" that specifies how to access
the data:

https://en.wikipedia.org/wiki/Query_plan

independently of the programmer. (Although many databases offer additional tools
for manually fine-tuning the query plan.)

If instead we wrote:

for row in Book:
if row.price > 100:
print(row)  # for example

I think that is specifying the control flow, and therefore not declarative. Do
you agree?



[...]
> I'm responding to the proposal by pointing out that, unlike today's
> Python, the proposed behaviour would violate those semantics.

In what way?

Do you feel the same way about itertools.takewhile?



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Marco Buttu]

On 10/08/2017 16:28, Steve D'Aprano wrote:


What would you expect this syntax to return?

[x + 1 for x in (0, 1, 2, 999, 3, 4) while x < 5]


[1, 2, 3]



For comparison, what would you expect this to return? (Without actually trying
it, thank you.)

[x + 1 for x in (0, 1, 2, 999, 3, 4) if x < 5]


[1, 2, 3, 4, 5]


--
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbu...@oa-cagliari.inaf.it

--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Steve D'Aprano  writes:

> If he wanted declarative semantics, why didn't he argue for
> declarative syntax like "select...where", instead of choosing
> procedural syntax which matches the actual procedural semantics given?

The syntax “f(foo) for foo in bar if baz(foo)” is nicely declarative.

> In order to think of list comps as declarative, you have to:
>
> - ignore the PEP;
> - ignore the documentation;
> - ignore the actual behaviour;
> - ignore the syntax.
>
> That's a lot of reality to ignore.

Python's list comprehensions behave as I expect that syntax to behave,
from the declarative, single-conceptual-operation expression model. I'm
not ignoring the behaviour or syntax.

The PEP and documentation, I will admit to ignoring (or at least have
forgotten since I first read them, now that I've got Python's actual
behaviour working with an widely known mental model).

The Python behaviour and mental model fit fine together, so I just keep
programming on that basis.

> That's why I was surprised at your attitude that list comps could
> reasonably be interpreted as a single operation. To do so requires
> ignoring everything we know about list comprehensions in Python.

I hope that clarifies. Everything about how Python's list comprehension
behaves, at a Python language level, supports the mental model of a list
comprehension as a single conceptual operation.

Does Python's behaviour *also* support the mental model of “it's like a
for loop”? Yes. Python supports multiple mental models, that should be
no surprise.

> But comprehensions? You can't even write a comprehension without being
> reminded that they are designed as an expression form of for-loops!

I'm reminded of a set operation. It works just fine that way.

> > […] your bafflement at getting answers not to the question you
> > didn't ask, but instead to the question you did ask, is becoming
> > quite strange.
>
> I don't understand this. You seem to be implying that the answer to
> the question:
>
> "What do you think Python will do if you execute a list
> comprehension?"

You didn't ask about what it *will do*, you asked about syntax that
Python does not yet support (a ‘while’ keyword in a list comprehension).

The correct answer to “What do you think Python will do if you execute
an expression with an invalid keyword in it?” is “Python will raise a
SyntaxError”.

Indeed, you got just such an answer, and it was an answer to the wrong
question as you rightly pointed out at the time.

Instead, you asked the much more interesting question: With the ‘while’
keyword in the expression, what do you think Python *would do* if it
supported such an expression?

So you're asking us not what Python *will in fact* do with that
syntax (today, it will raise an error). You're asking us what *the
proposed syntax would mean* if Python supported it.

> is best answered by predicting that Python will do something that you
> know it actually won't do.

You're asking something very similar. Because this is *proposed* syntax,
presently not valid Python, you're asking us to speculate about some
putative future Python, based on our mental models of Python semantics.

> But we really haven't established that Python's comprehension
> semantics violates any widely established programming language
> semantics.

Python's comprehension semantics do not violate those widely established
semantics, so you can stop waiting for anyone to try establishing that.

I'm responding to the proposal by pointing out that, unlike today's
Python, the proposed behaviour would violate those semantics.

-- 
 \“It is seldom that liberty of any kind is lost all at once.” |
  `\   —David Hume |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[jmp]

On 08/10/2017 04:28 PM, Steve D'Aprano wrote:

Every few years, the following syntax comes up for discussion, with some people
saying it isn't obvious what it would do, and others disagreeing and saying
that it is obvious. So I thought I'd do an informal survey.

What would you expect this syntax to return?

[x + 1 for x in (0, 1, 2, 999, 3, 4) while x < 5]


For comparison, what would you expect this to return? (Without actually trying
it, thank you.)

[x + 1 for x in (0, 1, 2, 999, 3, 4) if x < 5]



How about these?

[x + y for x in (0, 1, 2, 999, 3, 4) while x < 5 for y in (100, 200)]

[x + y for x in (0, 1, 2, 999, 3, 4) if x < 5 for y in (100, 200)]



Thanks for your comments!



[1,2,3]
[1,2,3,4,5]
SyntaxError("Have you tried Perl ?")
SyntaxError("Have you tried Perl ?")

I really would not want to deal with 3 and 4.

jm


--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 15 Aug 2017 09:10 am, Ben Finney wrote:

> Steve D'Aprano  writes:
> 
>> On Mon, 14 Aug 2017 07:59 pm, Ben Finney wrote:
>> > You began by asking what people would expect syntax to mean.
>> > 
>> > Then you expressed surprise that anyone would think a comprehension
>> > would be interpreted by the reader as a single operation.
>>
>> Yes, and I stand by that.
> 
> In the face of the designer saying that's how the feature was intended
> to be interpreted by a reader, you *remain* surprised anyone could think
> that?

Yes. What the designer intended matters not a whit, what matters is what was
actually designed. Whatever Greg intended, it didn't even last long enough to
make it into the PEP, let alone the language.

As I asked Greg earlier:

If he wanted declarative semantics, why didn't he argue for declarative syntax
like "select...where", instead of choosing procedural syntax which matches the
actual procedural semantics given?

In order to think of list comps as declarative, you have to:

- ignore the PEP;

- ignore the documentation;

- ignore the actual behaviour;

- ignore the syntax.


That's a lot of reality to ignore.

That's why I was surprised at your attitude that list comps could reasonably be
interpreted as a single operation. To do so requires ignoring everything we
know about list comprehensions in Python.

The same doesn't apply to map(), at least in Python 2. I think it is completely
reasonable for somebody to imagine that map() operated as a single operation.
They would be wrong, of course, but not egregiously wrong. There's no PEP to
contradict it, the documentation is consistent with map being an atomic
operation:

https://docs.python.org/2/library/functions.html#map

the name is the same as the functional "map", which in purely functional
languages is intended to be read as a mathematical operation which takes a
function and a list and returns a new list.

There may be subtle aspects of map in Python 2 which reveal that it is not
actually a single conceptual operation (side-effects, of course, and the fact
that you can interrupt it with Ctrl-C) but they are subtle and it is reasonable
for somebody to miss them, or to argue that they are mere accidents of
implementation.

But comprehensions? You can't even write a comprehension without being reminded
that they are designed as an expression form of for-loops! That's why I was,
and remain, perplexed that you could ignore the documentation and the behaviour
of comprehensions and believe something so egregiously wrong about them.


> I find it hard to believe you are denying facts to this extent.

I don't know what facts you think I'm ignoring.


>> Python list comprehensions are explicitly documented as being equivalent to a
>> loop, the syntax chosen uses the same keyword as a loop, the implementation
>> uses a loop, and when read as pseudo-code they explicitly claim to be
>> a loop.
>> So on what basis would anyone conclude that they aren't loops?
> 
> Python's list comprehensions are explicitly drawn from set theory, the
> syntax chosen uses the same words mathematicians use for set operations.

No they don't. Set builder notation doesn't use words at all, it uses symbols:

{n+1 | n ∈ {1, 2, 3}}

which if we were to translate into words would be:

"the set of n plus one, such that n is an element of the set 1, 2 and 3".

That's not even close to the same words as Python's comprehensions.

Aside: The interesting thing about sets in mathematics is that they're
unordered, unless it is useful for them to be thought of as ordered, in which
case they're taken as ordered. E.g. when talking about arithmetic or geometric
sequences, the unordered set of integers is taken as an ordered set, with no
change in notation or language.



[...]
>> Greg's intention simply doesn't matter. List comprehensions in Python
>> are as they are documented and implemented, not as he intended them to
>> be.
> 
> As a statement about what Python *will actually* do, of course that's
> true.
> 
> But it's not what you asked, and your bafflement at getting answers not
> to the question you didn't ask, but instead to the question you did ask,
> is becoming quite strange.

I don't understand this. You seem to be implying that the answer to the
question:

"What do you think Python will do if you execute a list comprehension?"

is best answered by predicting that Python will do something that you know it
actually won't do. I can't believe you mean it that way, so I'm more perplexed
than ever.


>> People are free to interpret Python features any way they like, but we
>> don't have to treat them seriously if their interpretation doesn't
>> match the documented semantics of the feature.
> 
> Then why ask people's interpretations at the start of this thread?

Because I wanted to know what their interpretations are, and not being
omniscient the only way to do so is to ask.

At no point did I promise that I wouldn't question their answers, if 

Re: Proposed new syntax

[Ben Finney]

Gregory Ewing  writes:

> The whole reason to write something as a comprehension is because you
> want to express it declaratively. You're saying "this is the list I
> want, I don't care how you compute it."

That's certainly a strong reason for my choosing comprehension
expressions: when I don't want to write a sequence of statements in a
loop, an expression communicates the intent (“this is one expression
with one output; think of it as one operation”).

> Likewise, the reader should be able to read it as "this is the list
> produced, and you *don't have to care* how it gets computed."

So I find the ‘while’ proposed addition to be counter to that purpose,
and describe it as confounding the ability of the reader to reason about
the expression.

> Introducing procedural elements such as "while" into a comprehension
> messes that up, because suddenly the reader *does* have to care about
> details of how it gets computed.

Thanks for stating it so well, Greg.

-- 
 \  “The good thing about science is that it's true whether or not |
  `\  you believe in it.” —Neil deGrasse Tyson, 2011-02-04 |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Steve D'Aprano wrote:
I take issue with your statement that relying on order of evaluation is always 
"a very bad idea".


Perhaps what I should say is that relying on side effects in
an expression occurring in a particular order is a bad idea.
Boolean operators are a well-understood special case that
doesn't extend to other expression elements. If you saw

   x = foo(y) + bar(z)

you wouldn't immediately assume it relied critically on
foo being called before bar -- would you?


And especially now that print is a regular function instead of a
statement, it is incredibly useful to stick a print or two in the middle of an
expression for debugging.


That's fine as a temporary hack, but I would never use that
for something the program needs to do as part of its normal
operation.


# paraphrase, not an actual quote
"Comprehensions might actually have well-defined semantics in terms
of for-loops, just as they are documented to have, but we should
pretend that they don't because some really complicated expressions
with side-effects can be confusing or buggy."


That's pretty much it, except I would delete "really
complicated". Any reliance on side effects in an expression
is confusing and bug-prone, IMO.

The whole reason to write something as a comprehension is
because you want to express it declaratively. You're saying
"this is the list I want, I don't care how you compute it."

Likewise, the reader should be able to read it as "this is
the list produced, and you *don't have to care* how it
gets computed."

Introducing procedural elements such as "while" into a
comprehension messes that up, because suddenly the reader
*does* have to care about details of how it gets computed.

And those details can be tricky. We've already seen how a
subtle error can creep in when you get a "while" and an
"if" in the wrong order!

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 15 Aug 2017 08:26 am, Gregory Ewing wrote:

> Ben Finney wrote:
>> That the comprehension
>> syntax *does not* necessarily connote a procedural loop, but instead can
>> quite reasonably be interpreted as its designer intended, a single
>> conceptual operation.
> 
> To put it another way: Consider that a comprehension is an
> expression, and I think most people would agree that relying
> on order of evaluation in an expression is a very bad idea,
> since it hurts readability and is prone to subtle bugs.

So you've stopped arguing that list comprehensions don't have a defined order
of evaluation, and are now arguing that code which relies on that order of
evaluation is hard to reason about? I suppose that's progress :-)

I take issue with your statement that relying on order of evaluation is always 
"a very bad idea". Order of evaluation in Python is well-defined, and we
frequently rely on it. There is nothing wrong with writing code like:

x = y + 1/z

x is not None and x > 1

a, b = b, a

while x in foo or x not in bar

and similar. And especially now that print is a regular function instead of a
statement, it is incredibly useful to stick a print or two in the middle of an
expression for debugging. You can't use that trick everywhere, but it is
especially useful in generator expressions as a way of seeing when its called
and yields a value:

(print('generator', x) or expression for x in something)

for example.

But really, the argument that you seem to be making:

# paraphrase, not an actual quote
"Comprehensions might actually have well-defined semantics in terms
of for-loops, just as they are documented to have, but we should
pretend that they don't because some really complicated expressions
with side-effects can be confusing or buggy."

seems pretty dubious to me.


As they say, "A man's got to know his limitations." If you program code that is
as complicated as you can understand, you won't be clever enough to debug it
when it goes wrong. That applies whether we're talking about order of
evaluation, basic arithmetic, threads, async programming, functional idioms,
levels of abstraction or spaghetti code. That makes a good argument for writing
conservative code that is the simplest thing that will work (for some
definition of "work").

It's not an argument for treating the semantics of code as different from what
those semantics actually are.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Tuesday, August 15, 2017 at 5:48:43 PM UTC+5:30, Steve D'Aprano wrote:
> On Tue, 15 Aug 2017 02:54 pm, Rustom Mody wrote:
> 
> > On Monday, August 14, 2017 at 10:35:22 PM UTC+5:30, Terry Reedy wrote:
> [...]
> >> Suppose stdin contains "a\nb\nc\nd\ne\nf\ng\n".
> >> What is the meaning of
> >> [input(f"response{i}") for i in range(6)]?
> >> In Python, the predictable result is
> >> ['a', 'b', 'c', 'd', 'e', 'f']
> >> It would not be with some of Rustom Mody's 'equivalents'.
> 
> Indeed. Terry has hit the nail on the head here. Python's documented semantics
> is predictable and deterministic, and the documented semantics of
> comprehensions are explicitly based on sequential for-loops.
> 
> To continue to insist, in contradiction to the documentation, that list comps
> might legitimately operate in arbitrary order is simply irrational. If they
> behaved as Rustom is insisting they might, it would be a bug in the
> implementation.

I know that list-comps behave like the l2r-pass-doing-appends as SPECIFIED
in the docs.
In my view the IMPLEMENTATION is correct wrt that specification
Whereas the SPECIFICATION, in being over-specific is wrong

In the same way that if someone explaining C were to say that the '+' operator
is (the SAME AS) ADD x86 instructions and then proves that case by showing gcc 
-S's output.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 15 Aug 2017 02:54 pm, Rustom Mody wrote:

> On Monday, August 14, 2017 at 10:35:22 PM UTC+5:30, Terry Reedy wrote:
[...]
>> Suppose stdin contains "a\nb\nc\nd\ne\nf\ng\n".
>> What is the meaning of
>> [input(f"response{i}") for i in range(6)]?
>> In Python, the predictable result is
>> ['a', 'b', 'c', 'd', 'e', 'f']
>> It would not be with some of Rustom Mody's 'equivalents'.

Indeed. Terry has hit the nail on the head here. Python's documented semantics
is predictable and deterministic, and the documented semantics of
comprehensions are explicitly based on sequential for-loops.

To continue to insist, in contradiction to the documentation, that list comps
might legitimately operate in arbitrary order is simply irrational. If they
behaved as Rustom is insisting they might, it would be a bug in the
implementation.


>> Or let L = .
>> This implementation of list reversal: [L.pop() for i in range(len(L))]
> 
> In languages, especially those with a clearly separated lang-spec from
> specific implementation-spec (eg C99), there is a sharp distinction made
> between - undefined behaviour
> - unspecified behaviour
> - erroneous behaviour
> 
> 
> Roughly:
> - Erroneous means compiler/runtime should flag an error
> - Undefined means implementation can format your hard disk and clear your bank
>   account

In *some* languages, not all, undefined means the implementation can format your
hard disk. Those languages include C (and perhaps C++). I don't know of any
other languages which are so hostile to the developer (and their end users) as
to specify such abominable behaviour and claim it is a feature. (There may be
others, I just don't know them.)

In general, undefined and unspecified are synonymous.


> 
https://stackoverflow.com/questions/18420753/unspecified-undefined-and-implementation-defined-behavior-wiki-for-c
> - Unspecified means not error but not spelled out
> 
> My point of suggesting those alternative implementations is precisely
> to make your examples above fall squarely into unspecified category

Terry's examples don't demonstrate unspecified behaviour. The order of execution
is completely specified. See the links to the docs I have already posted.


> Note that this would square with the informal practice seen in places like
> this ML/NG where a noob asks a question using a comprehension such as the
> ones you've used and someone more experienced pipes up "Dont do that"

Don't do that *in production code.*

[print(x, y) for x in values for y in others]

is a good way of learning how to read list comprehensions.

You're making the classic logical error of affirming the consequent:

https://en.wikipedia.org/wiki/Affirming_the_consequent

we should avoid code with unspecified behaviour

we should avoid this code, therefore it must have unspecified behaviour

Just because code showing unspecified behaviour should be avoided, doesn't mean
that all code that should be avoided has unspecified behaviour.

There are plenty of reasons for avoiding certain misuses of comprehensions that
have nothing to do with unspecified behaviour.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 15 Aug 2017 01:26 pm, Paul Rubin wrote:

> Steve D'Aprano  writes:
[...]
>> In Haskell, you cannot get the last N elements of a list without
>> allocating memory for the previous ones.
> 
> lastn n xxs@(x:xs)
>   | length (take n xs) == n-1 = xxs
>   | otherwise = lastn n xs
> 
> main = print . lastn 5 $ [1..1000]
> 
> *Main> main
> [996,997,998,999,1000]
> 
> works for me. 

Right. Did you read the Stackoverflow page I linked to? They all pretty much say
the same thing, although most of the given solutions use `drop`. Sorry if my
wording was unclear.


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Paul Rubin wrote:

Historically (in "naive set theory") we didn't bother with any of this.
We could write { S : S \not\in S } for the set of all sets that are not
members of themselves.  Is S a member of itself ("Russell's paradox")?
Either way leads to contradiction.  So the comprehension axiom schemas
for set theory had to be designed to not allow formulas like that.


Personally I think mathematicians worry overly much about that.
What it means is that not every predicate can be used to define
a set. A manifestation of the same thing in computing is that
not every program you can write down will terminate. But we
don't warp our languages in an effort to make non-terminating
programs impossible; we just accept them as a fact of life
and move on.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Mon, Aug 14, 2017 at 8:24 PM, Steve D'Aprano
 wrote:
> I think the closest analogy in Python terms would be a generator expression 
> with
> a cache. That's not *quite* the same, because Python is eagerly evaluated and
> Haskell is lazily evaluated, but the critical fact (which Ian seems to have
> completely missed) is that while evaluation can be delayed, touching each
> element *in order* from start to end cannot be. You cannot jump to an 
> arbitrary
> index in Haskell.
>
> But one thing is sure: even if evaluation is delayed, before you can access
> element N, you have to access element N-1. Whether it is implemented via
> recursion, a while-loop, or an actual for-loop, it is still logically
> equivalent to a for-loop through the elements.

I don't think that I've missed anything here. That the
*implementation* uses some kind of recursive application does not make
the *semantics* of the list comprehension logically equivalent to a
for loop.

This is about as close as you can get to a for loop with side effects
in Haskell:

Prelude Data.Traversable> forM [1..5] $ \n -> print $ n*n
1
4
9
16
25
[(),(),(),(),()]

And here's taking the last element of a list comprehension that,
according to you, should be "logically equivalent" to the above loop:

Prelude Data.Traversable> [ print $ n*n | n <- [1..5] ] !! 4
25

Wait, where are the side effects of the first four elements that had
to be recursed over to get to the last element? Clearly these are not
logically equivalent. As I said, Haskell list comprehensions are not
loops.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Monday, August 14, 2017 at 10:35:22 PM UTC+5:30, Terry Reedy wrote:
> On 8/14/2017 5:59 AM, Ben Finney wrote:
> 
> > At what point will you accept the feedback: That the comprehension
> > syntax *does not* necessarily connote a procedural loop, but instead can
> > quite reasonably be interpreted as its designer intended, a single
> > conceptual operation.
> 
> In a world where functions have 'side-effects', which is to say, 
> implicit inputs and outputs, procedural order is necessary for 
> predictable outcomes.  The 'single conception operation' is ambiguious.
> 
> Suppose stdin contains "a\nb\nc\nd\ne\nf\ng\n".
> What is the meaning of
> [input(f"response{i}") for i in range(6)]?
> In Python, the predictable result is
> ['a', 'b', 'c', 'd', 'e', 'f']
> It would not be with some of Rustom Mody's 'equivalents'.
> Or let L = .
> This implementation of list reversal: [L.pop() for i in range(len(L))]

In languages, especially those with a clearly separated lang-spec from 
specific implementation-spec (eg C99), there is a sharp distinction made between
- undefined behaviour
- unspecified behaviour
- erroneous behaviour


Roughly: 
- Erroneous means compiler/runtime should flag an error
- Undefined means implementation can format your hard disk and clear your bank 
  account
  
https://stackoverflow.com/questions/18420753/unspecified-undefined-and-implementation-defined-behavior-wiki-for-c
- Unspecified means not error but not spelled out

My point of suggesting those alternative implementations is precisely 
to make your examples above fall squarely into unspecified category

Note that this would square with the informal practice seen in places like
this ML/NG where a noob asks a question using a comprehension such as the
ones you've used and someone more experienced pipes up "Dont do that"


> Do not functional languages define comprehensions in terms of recursion,
> equivalent to for loops? 

At the risk of some over-simplification, the idea of functional languages is
to move above behavior from unspecified to erroneous category.
IMHO this is not in line with python's genus as an imperative dynamically typed
language

[In haskell, putchar, getchar etc exist and have monadic type.
Which is a funny way of saying that the type system takes cognisance of and 
rigorously enforces the side-effecting nature of such functions.
Which also means there are no side-effects; only effects, clearly flagged
with the monadic type]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Tuesday, August 15, 2017 at 3:57:22 AM UTC+5:30, Gregory Ewing wrote:
> Ben Finney wrote:
> > That the comprehension
> > syntax *does not* necessarily connote a procedural loop, but instead can
> > quite reasonably be interpreted as its designer intended, a single
> > conceptual operation.
> 
> To put it another way: Consider that a comprehension is an expression, 

A neat summary of the basis of this whole difference of opinion!

JFTR: Haskell gets its list comprehensions from Miranda.
Technically Haskell and Miranda are much the same in syntax, semantics etc (in
this respect)

However as to the informal discussion-terms around the same there is this 
difference:
In Miranda the terms used were “list comprehension” as well as
“ZF expression” (ZF standing for Zermelo, Fraenkel who formulated the
comprehension axiom(s) ) with a slight preference for the latter.

I have generally preferred “ZF expression” because as I said earlier
“comprehension”, in this context, is meaningless (in English).

Seeing this discussion, I feel the other part – “expression” – which I never 
paid attention to so far! – is probably as or more important.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Paul Rubin  writes:
>   FORALL P. [ P(0) and P(n) -> P(n+1) ]

Sorry, that was supposed to say

  FORALL P. [ (P(0) and P(n) -> P(n+1)) -> forall n. P(n) ]

FORALL quantifies over formulas and forall quantifies over numbers.

Maybe something is still missing from the above ;-).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Marko Rauhamaa  writes:
> For the non-logicians in the audience, an "axiom schema" is a generator
> pattern that produces an infinity of actual axioms. 

For non-logicians this is not worth worrying about: schemas are
basically a technical hack to get around the inability of first-order
logic to quantify over formulas.  Naively most of us think of single
axioms with second-order quantifiers, instead of schemas.

For example, arithmetic induction goes something like:

  FORALL P. [ P(0) and P(n) -> P(n+1) ]

where "FORALL P" means quantifying over all formulas P.  Since in FOL we
can only quantify over numbers and not formulas, we write down a
separate axiom for each possible formula.  Since there are infinitely
many formulas, this is an infinite axiom schema.

Historically (in "naive set theory") we didn't bother with any of this.
We could write { S : S \not\in S } for the set of all sets that are not
members of themselves.  Is S a member of itself ("Russell's paradox")?
Either way leads to contradiction.  So the comprehension axiom schemas
for set theory had to be designed to not allow formulas like that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Paul Rubin]

Steve D'Aprano  writes:
> It's quite clever, actually, in that it gives *pseudo* random-access
> with lazy evaluation. You can evaluate the Nth element without
> evaluating the earlier ones. But you can't do so without *storing* the
> previous ones, they have to be allocated, with only the actual
> evaluation being delayed.

Look at any of the memoization packages for ways around that.  Or of
course you could just write a function instead of a list...

> In Haskell, you cannot get the last N elements of a list without
> allocating memory for the previous ones.

lastn n xxs@(x:xs)
  | length (take n xs) == n-1 = xxs
  | otherwise = lastn n xs

main = print . lastn 5 $ [1..1000]

*Main> main
[996,997,998,999,1000]

works for me.  The 1000 list nodes all get allocated, but are
immediately freed, so only 5 cells have to be in memory at a time.  In
principle a fancy enough compiler optimization could get rid of all the
allocation completely.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Tue, 15 Aug 2017 01:40 am, Chris Angelico wrote:

[...]
>> A Haskell list comprehension is not a loop at all.
> 
> What if you don't take the first four, but instead take just the tenth
> element? Will the preceding elements be calculated too, or do you have
> a sparse list? 

Both. Or neither.

The preceding elements aren't *calculated*, but they are *allocated* ready for
when you need them:

Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
Prelude> squares !! 10
121.0
Prelude> :print squares
squares = (_t2::Float) : (_t3::Float) : (_t4::Float) :
  (_t5::Float) : (_t6::Float) : (_t7::Float) : (_t8::Float) :
  (_t9::Float) : (_t10::Float) : (_t11::Float) : 121.0 :
  (_t12::[Float])


Haskell lists are single-pointer linked lists.

https://wiki.haskell.org/How_to_work_on_lists

My understanding of it is that it will be implemented something like a linked
list of thunks, followed by the evaluated element, followed by an infinite
generator. In Python terms:

(lambda x=1: x**2, 
(lambda x=2: x**2, 
(lambda x=3: x**2,
... # for brevity
(lambda x=10: x**2,
(121.0, 
((x**2 for x in itertools.count(12)), None)
)
)
...
)
)
)

or something more-or-less equivalent.

It's quite clever, actually, in that it gives *pseudo* random-access with lazy
evaluation. You can evaluate the Nth element without evaluating the earlier
ones. But you can't do so without *storing* the previous ones, they have to be
allocated, with only the actual evaluation being delayed.

So its both less powerful and more powerful than Python's (x)range. Less
powerful as:

- there's no random access (you have to walk the linked list to get 
  to the Nth element, touching each element in turn);

- and you cannot access the Nth element without storing the previous
  N-1 elements.

But more powerful in that, like a generator expression, it can take arbitrary
expressions.


Here's a more interesting example:

Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
Prelude> let values = [a + 1 | a <- squares ] :: [Float]
Prelude> values !! 4
26.0
Prelude> :print values
values = (_t13::Float) : (_t14::Float) : (_t15::Float) :
 (_t16::Float) : 26.0 : (_t17::[Float])
Prelude> :print squares
squares = (_t18::Float) : (_t19::Float) : (_t20::Float) :
  (_t21::Float) : 25.0 : (_t22::[Float])

So its lazy evaluation all the way down.

I think the closest analogy in Python terms would be a generator expression with
a cache. That's not *quite* the same, because Python is eagerly evaluated and
Haskell is lazily evaluated, but the critical fact (which Ian seems to have
completely missed) is that while evaluation can be delayed, touching each
element *in order* from start to end cannot be. You cannot jump to an arbitrary
index in Haskell.

But one thing is sure: even if evaluation is delayed, before you can access
element N, you have to access element N-1. Whether it is implemented via
recursion, a while-loop, or an actual for-loop, it is still logically
equivalent to a for-loop through the elements.


In Haskell, you cannot get the last N elements of a list without allocating
memory for the previous ones. Here's a Stackoverflow post about it:

https://stackoverflow.com/questions/17252851/how-do-i-take-the-last-n-elements-of-a-list


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Steve D'Aprano  writes:

> On Mon, 14 Aug 2017 07:59 pm, Ben Finney wrote:
> > You began by asking what people would expect syntax to mean.
> > 
> > Then you expressed surprise that anyone would think a comprehension
> > would be interpreted by the reader as a single operation.
>
> Yes, and I stand by that.

In the face of the designer saying that's how the feature was intended
to be interpreted by a reader, you *remain* surprised anyone could think
that?

I find it hard to believe you are denying facts to this extent.

> Python list comprehensions are explicitly documented as being equivalent to a
> loop, the syntax chosen uses the same keyword as a loop, the implementation
> uses a loop, and when read as pseudo-code they explicitly claim to be
> a loop.
> So on what basis would anyone conclude that they aren't loops?

Python's list comprehensions are explicitly drawn from set theory, the
syntax chosen uses the same words mathematicians use for set operations.

I'm not saying what you wrote above is incorrect. But it is *also* true
what I wrote.

Since this is a thread you began by asking how people would interpret
some syntax, *both of these are relevant*, even if incompatible.

> > The designer of that feature expressed that yes, the intention was
> > that it be interpreted as a single conceptual operation, not a
> > looping sequence of operations.
>
> What matters is how comprehensions actually *are*, not what Greg
> intended.

When it comes to the matter you actually raised – how a person reading
the syntax will interpret it – implementation is barely at play.

What matters much more is the connotations and baggage of language and
syntax not in the Python implementation, but *in the reader's mind*.

That's why it is an ongoing non sequitur that, when people honestly say
what leads them to interpret the syntax a particular way, you then point
to the Python implementation.

> Greg's intention simply doesn't matter. List comprehensions in Python
> are as they are documented and implemented, not as he intended them to
> be.

As a statement about what Python *will actually* do, of course that's
true.

But it's not what you asked, and your bafflement at getting answers not
to the question you didn't ask, but instead to the question you did ask,
is becoming quite strange.

> People are free to interpret Python features any way they like, but we
> don't have to treat them seriously if their interpretation doesn't
> match the documented semantics of the feature.

Then why ask people's interpretations at the start of this thread?

The Python executable can be freely designed to interpret syntax any way
its designers choose. But those are *choices*, and we can expect the
designers to inform those choices from the existing connotations and
baggage of language and syntax from outside and prior to Python — and
even from outside any programming languages.

If the design of Python's semantics sufficiently violates an established
interpretation of language or syntax, we don't have to treat it
seriously and can rightly call it a design bug.

-- 
 \ “Having sex with Rachel is like going to a concert. She yells a |
  `\  lot, and throws frisbees around the room; and when she wants |
_o__)more, she lights a match.” —Steven Wright |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Gregory Ewing]

Ben Finney wrote:

That the comprehension
syntax *does not* necessarily connote a procedural loop, but instead can
quite reasonably be interpreted as its designer intended, a single
conceptual operation.


To put it another way: Consider that a comprehension is an
expression, and I think most people would agree that relying
on order of evaluation in an expression is a very bad idea,
since it hurts readability and is prone to subtle bugs.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Terry Reedy]

On 8/14/2017 5:59 AM, Ben Finney wrote:


At what point will you accept the feedback: That the comprehension
syntax *does not* necessarily connote a procedural loop, but instead can
quite reasonably be interpreted as its designer intended, a single
conceptual operation.


In a world where functions have 'side-effects', which is to say, 
implicit inputs and outputs, procedural order is necessary for 
predictable outcomes.  The 'single conception operation' is ambiguious.


Suppose stdin contains "a\nb\nc\nd\ne\nf\ng\n".
What is the meaning of
[input(f"response{i}") for i in range(6)]?
In Python, the predictable result is
['a', 'b', 'c', 'd', 'e', 'f']
It would not be with some of Rustom Mody's 'equivalents'.
Or let L = .
This implementation of list reversal: [L.pop() for i in range(len(L))]

Do not functional languages define comprehensions in terms of recursion, 
equivalent to for loops?


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

http://www.cosc.canterbury.ac.nz/greg.ewing/python/listcomp/

Greg's 1999 announcement.
If you see the "semantics are equivalent" Steven wins. 
If you focus on "like other languages" then it's... well not quite equivalent! 

We can take our pick!

For myself, as I earlier said, if python disagrees with math/other-langs, I 
would typically go with the majoritarian view
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Tue, Aug 15, 2017 at 1:54 AM, Ian Kelly  wrote:
> On Mon, Aug 14, 2017 at 9:40 AM, Chris Angelico  wrote:
>> On Tue, Aug 15, 2017 at 1:33 AM, Ian Kelly  wrote:
>>> On Sun, Aug 13, 2017 at 8:36 AM, Steve D'Aprano
 Sure. In Haskell, comprehensions are *implicit* loops, rather than 
 explicit like
 in Python.
>>>
>>> No, they aren't. Haskell list comprehensions use lazy evaluation.
>>> Here's an example of an infinite list comprehension:
>>>
>>> Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
>>> Prelude> :print squares
>>> squares = (_t1::[Float])
>>>
>>> You might say that this is more like a generator expression but the
>>> result is in fact a list. We can evaluate the first four elements:
>>>
>>> Prelude> print $ take 4 squares
>>> [1.0,4.0,9.0,16.0]
>>>
>>> And then see that these have been lazily evaluated in the list:
>>>
>>> Prelude> :print squares
>>> squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t2::[Float])
>>>
>>> A Haskell list comprehension is not a loop at all.
>>
>> What if you don't take the first four, but instead take just the tenth
>> element? Will the preceding elements be calculated too, or do you have
>> a sparse list? If the latter, it's not really comparable to a Python
>> list, but to some sort of cached mapping from input values to output
>> values, which in Python I would implement as a dict with a __missing__
>> method. And if the former, well, that's still going to have a loop,
>> and it's definitely like a genexp, but genexps have more functionality
>> than they do in Python.
>
> The answer is, it depends. If it can avoid evaluating the earlier
> elements it will:
>
> Prelude> squares !! 10
> 121.0
> Prelude> :print squares
> squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t3::Float) : (_t4::Float) :
>   (_t5::Float) : (_t6::Float) : (_t7::Float) : (_t8::Float) : 121.0 :
>   (_t9::[Float])
>
> But sometimes it can't:
>
> Prelude> let triangle = [ (i,j) | i <- [1..], j <- [1..i*i] ]
> Prelude> triangle !! 10
> (3,6)
> Prelude> :print triangle
> triangle = (1,1) : (2,1) : (2,2) : (2,3) : (2,4) : (3,1) : (3,2) :
>(3,3) : (3,4) : (3,5) : (3,6) : (_t10::[(Integer, Integer)])

Well, no, it doesn't depend. It evaluates only those elements that get
requested. You just requested all the earlier elements as part of
calculating a triangle. :)

So it's really nothing to do with a Python list, nor a list
comprehension. It's like this lazy map:

class Map(dict):
def __init__(self, func, *coll):
self.func = func
self.coll = coll # Collections, not iterables
def __missing__(self, key):
self[key] = self.func(*(coll[key] for coll in self.coll))
return self[key]

>>> squares = Map(lambda x: x*x, range(1))
>>> squares[5]
25
>>> squares[1000]
100
>>> squares
{5: 25, 1000: 100}

Except that the indices used are restricted, presumably. to counting
numbers. But still, what you call a lazy list is a function (in the
mathematical sense), but not a true collection.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Mon, Aug 14, 2017 at 9:40 AM, Chris Angelico  wrote:
> On Tue, Aug 15, 2017 at 1:33 AM, Ian Kelly  wrote:
>> On Sun, Aug 13, 2017 at 8:36 AM, Steve D'Aprano
>>> Sure. In Haskell, comprehensions are *implicit* loops, rather than explicit 
>>> like
>>> in Python.
>>
>> No, they aren't. Haskell list comprehensions use lazy evaluation.
>> Here's an example of an infinite list comprehension:
>>
>> Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
>> Prelude> :print squares
>> squares = (_t1::[Float])
>>
>> You might say that this is more like a generator expression but the
>> result is in fact a list. We can evaluate the first four elements:
>>
>> Prelude> print $ take 4 squares
>> [1.0,4.0,9.0,16.0]
>>
>> And then see that these have been lazily evaluated in the list:
>>
>> Prelude> :print squares
>> squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t2::[Float])
>>
>> A Haskell list comprehension is not a loop at all.
>
> What if you don't take the first four, but instead take just the tenth
> element? Will the preceding elements be calculated too, or do you have
> a sparse list? If the latter, it's not really comparable to a Python
> list, but to some sort of cached mapping from input values to output
> values, which in Python I would implement as a dict with a __missing__
> method. And if the former, well, that's still going to have a loop,
> and it's definitely like a genexp, but genexps have more functionality
> than they do in Python.

The answer is, it depends. If it can avoid evaluating the earlier
elements it will:

Prelude> squares !! 10
121.0
Prelude> :print squares
squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t3::Float) : (_t4::Float) :
  (_t5::Float) : (_t6::Float) : (_t7::Float) : (_t8::Float) : 121.0 :
  (_t9::[Float])

But sometimes it can't:

Prelude> let triangle = [ (i,j) | i <- [1..], j <- [1..i*i] ]
Prelude> triangle !! 10
(3,6)
Prelude> :print triangle
triangle = (1,1) : (2,1) : (2,2) : (2,3) : (2,4) : (3,1) : (3,2) :
   (3,3) : (3,4) : (3,5) : (3,6) : (_t10::[(Integer, Integer)])
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Chris Angelico]

On Tue, Aug 15, 2017 at 1:33 AM, Ian Kelly  wrote:
> On Sun, Aug 13, 2017 at 8:36 AM, Steve D'Aprano
>> Sure. In Haskell, comprehensions are *implicit* loops, rather than explicit 
>> like
>> in Python.
>
> No, they aren't. Haskell list comprehensions use lazy evaluation.
> Here's an example of an infinite list comprehension:
>
> Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
> Prelude> :print squares
> squares = (_t1::[Float])
>
> You might say that this is more like a generator expression but the
> result is in fact a list. We can evaluate the first four elements:
>
> Prelude> print $ take 4 squares
> [1.0,4.0,9.0,16.0]
>
> And then see that these have been lazily evaluated in the list:
>
> Prelude> :print squares
> squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t2::[Float])
>
> A Haskell list comprehension is not a loop at all.

What if you don't take the first four, but instead take just the tenth
element? Will the preceding elements be calculated too, or do you have
a sparse list? If the latter, it's not really comparable to a Python
list, but to some sort of cached mapping from input values to output
values, which in Python I would implement as a dict with a __missing__
method. And if the former, well, that's still going to have a loop,
and it's definitely like a genexp, but genexps have more functionality
than they do in Python.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ian Kelly]

On Sun, Aug 13, 2017 at 8:36 AM, Steve D'Aprano
 wrote:
> On Sat, 12 Aug 2017 01:02 am, Ian Kelly wrote:
>
>> On Thu, Aug 10, 2017 at 11:45 PM, Steve D'Aprano
>>  wrote:
>
>>> Comprehension syntax makes the sequential loop explicit: the loop is right
>>> there in the syntax:
>>>
>>> [expr for x in iterable]
>>
>> This is a peculiarity of Python.
>
> Yes? We're talking about Python? I haven't accidentally been posting to
> comp.lang.anything-but-python have I? *wink*
>
> I am not referring to syntax from other languages. (One wonders what a list
> comprehension in Whitespace would look like...) We're talking about Python,
> which prefers explicit English-like executable pseudo-code over implicit
> cryptic mathematical symbolic expressions.

With respect, your own post that I was responding to discussed
"traditional functional programming idioms". Why are traditional
map/filter on the table for discussion but not traditional list
comprehensions?

In any case, this thread started with the question of what we would
expect a particular list comprehension syntax to look like. While
existing Python syntax is certainly important to understanding that
(and that was the basis for my own initial response stemming from the
knowledge that Python maps fors and ifs to nested blocks and I would
expect while to work in the same way), for many Python users their
understanding of list comprehensions may not necessarily be rooted in
Python syntax, and so I think it is also important to consider how
list comprehensions are viewed in other contexts.

> But for the record, this is not a peculiarity of Python by any means. I count 
> at
> least a dozen other languages which use an explicit "for" in their
> comprehension syntax:

Yes, as I noted in my reply.

> Boo, Ceylon, Clojure, CoffeeScript, Common Lisp, Cobra, Elixir, F#, Mozilla's
> Javascript, Julia, Perl6, Racket and Scala.

I think some of these take influence from Python. Common Lisp is a bit
of an oddball; it doesn't actually have list comprehensions as a
distinct syntax. The example given there is just a standard loop using
the "collect" keyword.

The Javascript example should be removed from that page. The syntax
didn't make it into the standard and Mozilla's own documentation has a
big red warning saying not to use it. They'll probably eventually
remove it from Gecko.

> Sure. In Haskell, comprehensions are *implicit* loops, rather than explicit 
> like
> in Python.

No, they aren't. Haskell list comprehensions use lazy evaluation.
Here's an example of an infinite list comprehension:

Prelude> let squares = [ x ** 2 | x <- [1 ..] ] :: [Float]
Prelude> :print squares
squares = (_t1::[Float])

You might say that this is more like a generator expression but the
result is in fact a list. We can evaluate the first four elements:

Prelude> print $ take 4 squares
[1.0,4.0,9.0,16.0]

And then see that these have been lazily evaluated in the list:

Prelude> :print squares
squares = 1.0 : 4.0 : 9.0 : 16.0 : (_t2::[Float])

A Haskell list comprehension is not a loop at all.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Steve D'Aprano]

On Mon, 14 Aug 2017 07:59 pm, Ben Finney wrote:

> Steven D'Aprano  writes:
> 
>> On Sun, 13 Aug 2017 21:06:08 -0700, Rustom Mody wrote:
>>
>> > Here's a bunch of different ways in which a mapping comprehension
>> > could be implemented:
>>
>> Not in Python they couldn't be
> 
> You began by asking what people would expect syntax to mean.
> 
> Then you expressed surprise that anyone would think a comprehension
> would be interpreted by the reader as a single operation.

Yes, and I stand by that.

Python list comprehensions are explicitly documented as being equivalent to a
loop, the syntax chosen uses the same keyword as a loop, the implementation
uses a loop, and when read as pseudo-code they explicitly claim to be a loop.

So on what basis would anyone conclude that they aren't loops? That's a genuine
question, not a rhetorical question. I meant it when I said I could understand
people imagining that the Python 2 versions of map() or filter() could be
implemented as something other than a straightforward sequential loop, but I'm
puzzled how anyone could imagine that of list comprehensions and generator
expressions.

The author of the PEP which introduced list comps to the language, Barry Warsaw,
wrote:

"They [comprehensions] would nest in the same way for loops and if statements
nest now."

Guido added to the PEP:

"The form [... for x... for y...] nests, with the last index varying fastest,
just like nested for loops."

https://www.python.org/dev/peps/pep-0202/


> The designer of that feature expressed that yes, the intention was that
> it be interpreted as a single conceptual operation, not a looping
> sequence of operations.

What matters is how comprehensions actually *are*, not what Greg intended.

- The PEP states that comprehensions are equivalent to loops; if Greg 
  intended different, he should have written the PEP to say so.

- The syntax chosen is that of a procedural for-loop, not a declarative 
  "select where" statement.

- The semantics of the comprehension is explicitly documented as equivalent
  to nested for-loops.

- Guido blessed both the syntax and the semantics.

Greg's intention simply doesn't matter. List comprehensions in Python are as
they are documented and implemented, not as he intended them to be.


> Your latest counter has been that Python means something special, beyond
> what the feature's designer intended, and it's implemented as a looping
> sequence of operations.

The implementation is the least of my arguments, since it could change to a
while loop without affecting the semantics of comprehensions and generator
expressions. Or a series of unrolled loops, or to use goto or comefrom for that
matter.

The critical point is that the actual semantics, as stated in the PEP and the
documentation is a for-loop, and the syntax chosen is self-documenting as a
for-loop.

Greg may have intended that comprehensions being equivalent to nested for-loops
be taken as a mere analogy, but that's not the path Python has taken. It is not
an analogy, but a language promise that all comprehensions will be semantically
equivalent to for-loops.



[...]
> You were apparently, at the start of this thread, honestly seeking to
> know how people interpret the syntax.

Indeed.

About half the people who responded interpreted it with the same way semantics
as the Clojure list comprehensions which inspired it. Some people interpreted
it as an infinite loop. You claimed that it was impossible to understand; one
person claimed that it would cause unending confusion; one person saw no
difference between "while" and "if", and one person seemed to believe that even
the existing "if" clause should be a syntax error.



> At what point will you accept the feedback: That the comprehension
> syntax *does not* necessarily connote a procedural loop, but instead can
> quite reasonably be interpreted as its designer intended, a single
> conceptual operation.

People are free to interpret Python features any way they like, but we don't
have to treat them seriously if their interpretation doesn't match the
documented semantics of the feature.

I might honestly, strongly believe that

del x

causes the object referenced by x to be garbage collected. I can say that's how
it works in other languages, I can say that's what the designer intended all
those many years ago, I can even insist that there is no other logical way for
del to operate and any other behaviour would be inconceivable and a source of
unending confusion.

Nevertheless, that isn't how del operates, it isn't how it is documented, and my
opinion is simply wrong. If I gave a code review where I insisted that:

x = y = some_object()
del x
print(y)

is unsafe because the object has been garbage collected, you would be right to
ignore that feedback. Not because you don't like my review, but because it is
based on a misconception and is simply factually wrong.



-- 
Steve
“Cheer up,” they said, 

Re: Proposed new syntax

[Steve D'Aprano]

On Mon, 14 Aug 2017 09:03 pm, Rustom Mody wrote:

> For myself, if the python docs contradict the last 100 years or prior art/math
> history, the latter takes precedence

When theory disagrees with the facts, you ignore the facts?


> All I want to say is the gratuitous incidental resemblance of for-loops
> with comprehensions confuses not just beginners but even the python
> implementers:

You have misinterpreted what you are seeing.

The Python 2 variable leak has nothing to do with whether comprehensions are
for-loops or not. It is a scoping issue: list comps previously shared the same
scope as their surrounding, but no longer.


> However this leak still remains:
> 
 fl = [(lambda x : x + i) for i in range(5)]
 [f(2) for f in fl]
> [6, 6, 6, 6, 6]

This has nothing to do with list comprehensions or for-loops. We can unroll the
loop and demonstrate the same behaviour:

fl = []
i = 0
fl.append(lambda x: x + i)
i = 1
fl.append(lambda x: x + i)
i = 2
fl.append(lambda x: x + i)
i = 3
fl.append(lambda x: x + i)
i = 4
fl.append(lambda x: x + i)
[f(2) for f in fl]


It's not a "leak", it's not "bugginess", it has nothing to do with for loops or
comprehensions. It's just the standard, normal name resolution.

The example in the list comprehension is *slightly* different: the functions
there are closures over i, whereas in the unrolled version the functions simply
do a global name lookup.

But the behaviour is the same because both use late binding semantics: the value
of i isn't copied into the function at the time of function creation, it is
retrieved only when needed when the function is called.

Haskell (it seems) uses early binding instead.

Which is "correct"? Both are. Both early and late binding are reasonable design
choices, with good reasons for choosing either. Python uses early binding for
function default values, and late binding for closures. You might argue for the
other choice, but the decision is made and won't change. And neither has
anything to do with for-loops.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Rustom Mody]

On Monday, August 14, 2017 at 3:30:39 PM UTC+5:30, Ben Finney wrote:
> Steven D'Aprano writes:
> 
> > On Sun, 13 Aug 2017 21:06:08 -0700, Rustom Mody wrote:
> >
> > > Here's a bunch of different ways in which a mapping comprehension
> > > could be implemented:
> >
> > Not in Python they couldn't be
> 
> You began by asking what people would expect syntax to mean.
> 
> Then you expressed surprise that anyone would think a comprehension
> would be interpreted by the reader as a single operation.
> 
> The designer of that feature expressed that yes, the intention was that
> it be interpreted as a single conceptual operation, not a looping
> sequence of operations.
> 
> Your latest counter has been that Python means something special, beyond
> what the feature's designer intended, and it's implemented as a looping
> sequence of operations.
> 
> So it's normal to point out that Python's implementation is just one way
> that it could be implemented, hence the reader can reasonably expect
> that it's not the way Python implemented it.
> 
> You were apparently, at the start of this thread, honestly seeking to
> know how people interpret the syntax.
> 
> At what point will you accept the feedback: That the comprehension
> syntax *does not* necessarily connote a procedural loop, but instead can
> quite reasonably be interpreted as its designer intended, a single
> conceptual operation.



I thought that this – Steven's OP – was some joke or prank question.
As best as I could make out all the initial responders giving some supposed 
meaning to the various 'comprehensions' seemed to be playing along
Since I had nothing cute to say… remained quiet

Then in quick succession Terry, Cecil, yourself gave responses that I would give
in seriousness if anything, better than what I could come up with
So once again I had nothing to add

I just put in some history to say that yes, you can take the expansion
of comprehensions as loops-with-appends given in the python docs as God's Word

For myself, if the python docs contradict the last 100 years or prior art/math 
history,
the latter takes precedence 

All I want to say is the gratuitous incidental resemblance of for-loops
with comprehensions confuses not just beginners but even the python 
implementers:

This leak

>>> [x for x in range(5)]
[0, 1, 2, 3, 4]
>>> x
4
>>> 

got corrected from python2 to 3
>>> [x for x in range(5)]
[0, 1, 2, 3, 4]
>>> x
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'x' is not defined

However this leak still remains:

>>> fl = [(lambda x : x + i) for i in range(5)]
>>> [f(2) for f in fl] 
[6, 6, 6, 6, 6]

Compare to haskell:

Prelude> let fl = [(\ x -> x + i) | i <- [0..4]]
Prelude> [f 2 | f <- fl]
[2,3,4,5,6]
Prelude> 

I strongly suspect this bugginess to be related to Greg's original idea of using
that for-loop-containing-append as an informal off-the-cuff explanation 
becoming 
the formal definition


[And I am still unsure whether this OP is a prank trick-question… ]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Ben Finney]

Steven D'Aprano  writes:

> On Sun, 13 Aug 2017 21:06:08 -0700, Rustom Mody wrote:
>
> > Here's a bunch of different ways in which a mapping comprehension
> > could be implemented:
>
> Not in Python they couldn't be

You began by asking what people would expect syntax to mean.

Then you expressed surprise that anyone would think a comprehension
would be interpreted by the reader as a single operation.

The designer of that feature expressed that yes, the intention was that
it be interpreted as a single conceptual operation, not a looping
sequence of operations.

Your latest counter has been that Python means something special, beyond
what the feature's designer intended, and it's implemented as a looping
sequence of operations.

So it's normal to point out that Python's implementation is just one way
that it could be implemented, hence the reader can reasonably expect
that it's not the way Python implemented it.

You were apparently, at the start of this thread, honestly seeking to
know how people interpret the syntax.

At what point will you accept the feedback: That the comprehension
syntax *does not* necessarily connote a procedural loop, but instead can
quite reasonably be interpreted as its designer intended, a single
conceptual operation.

-- 
 \“The priesthood have, in all ancient nations, nearly |
  `\ monopolized learning.” —John Adams, _Letters to John Taylor_, |
_o__) 1814 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list