Hi Steve, Thank you for so detailed comments.
My comments also below interleaved with yours.
At 2018-02-16 08:57:40, "Steven D'Aprano" <st...@pearwood.info> wrote:
>Hi fhsxfhsx, and welcome.
>
>My comments below, interleaved with yours.
>
>
>On Thu, Feb 15, 2018 at 01:56:44PM +0800, fhsxfhsx wrote:
>
>[quoted out of order]
>> And I hope the discussion could focus more on whether we should allow
>> assigning temporary variables in comprehensions rather than how to
>> solve the specific example I mentioned above.
>
>Whether or not to allow this proposal will depend on what alternate
>solutions to the problem already exist, so your specific example is very
>relevant. Any proposed change has to compete with existing solutions.
>>
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>> alist.append(f(x))
>> because the previous one shows every element of the list explicitly so
>> that we don't need to handle `append` mentally.
>
>While I personally agree with you, many others disagree. I know quite a
>few experienced, competent Python programmers who avoid list
>comprehensions because they consider them harder to read and reason
>about. They consider a regular for-loop better precisely because you do
>see the explicit call to append.
>
>(In my experience, those of us who get functional-programming idioms
>often forget that others find them tricky.)
>
>The point is that list comprehensions are already complex enough that
>they are difficult for many people to learn, and some people never come
>to grips with them. Adding even more features comes with a cost.
>
>The bottom line is that it isn't clear to me that allowing local
>variables inside comprehensions will make them more readable.
>
To be frank, I had not thought of this before.
However, in my opinion, when considering adding a new syntax, we care more
about the marginal cost.
I mean, I think it is the functional-programming way which is tricky, but
allowing a new syntax would not make things worse.
Well, that's just a guess, maybe only those who are puzzled with comprehensions
can give us an answer.
> >> But when it comes to something like >> [f(x) + g(f(x)) for x in range(10)]
> >> >> you find you have to sacrifice some readableness if you don't want two
> >> >> f(x) which might slow down your code. > >The usual comments about
> >> premature optimisation apply here. > >Setting a new comprehension variable
> >> is not likely to be free, and may even be >more costly than calling f(x)
> >> twice if f() is a cheap expression: > > [x+1 + some_func(x+1) for x in
> >> range(10)] > >could be faster than > > [y + some_func(y) for x in
> >> range(10) let y = x + 1] > >or whatever syntax we come up with. >
It is true. But since there are still so many cases where a temporary variable
is faster.
Also, even without let-clause, one can write a for-loop with a temporary
variable which slow down the code.
So, it seems that "setting a new comprehension variable may even be more
costly" does not show any uselessness of temporary variables in comprehensions.
> >> Someone may argue that one can write >> [y + g(y) for y in [f(x) for x in
> >> range(10)]] > >Indeed. This would be the functional-programming solution,
> >> and I >personally think it is an excellent one. The only changes are that
> >> I'd >use a generator expression for the intermediate value, avoiding the
> >> need >to make a full list, and I would lay it out more nicely, using
> >> >whitespace to make the structure more clear: > > result = [y + g(y) for y
> >> in > (f(x) for x in range(10)) > ] >
In my opinion,
[
y + g(y)
for x in range(10)
let y = f(x)
]
is better because it's more corresponding to a for-loop
for x in range(10):
y = f(x)
result.append(y + g(y))
In my opinion, comprehensions are not real functional-programming because there
is not even a function. Though there're similarities, map and filter are real
functional-programming. Since the similarity between for-clause in
comprehensions and the for-loop, I think it's better to write comprehensions
more close to for-loop.
I don't know but I guess maybe it can also help those who fear comprehensions
better embrace them?
>
>> but it's not as clear as to show what `y` is in a subsequent clause,
>> not to say there'll be another temporary list built in the process.
>
>There's no need to build the temporary list. Use a generator
>comprehension. And I disagree that the value of y isn't as clear.
>
>An alternative is simply to refactor your list comprehension. Move the
>calls to f() and g() into a helper function:
>
>def func(x):
> y = f(x)
> return y + g(y)
>
>and now you can write the extremely clear comprehension
>
>[func(x) for x in range(10)]
>
>that needs no extra variable.
>
I think it can be a goods idea if there's a name to `func` which is easy to
understand, or `func` is put close to the comprehension and is in a obvious
place.
But I feel it's not for the case I gave in another mail to Paul,
https://mail.python.org/pipermail/python-ideas/2018-February/048997.html,
(I'm sorry that the example is quite long, and I don't hope to copy it here)
To me, it can be confusing to have several `func` when I have several lists at
the same time and have to transform them each in a similar but different way.
> > >[...] >> In a word, what I'm arguing is that we need a way to assign
> > >temporary >> variables in a comprehension. > >"Need" is very strong. I
> > >think that the two alternatives I mention above >cover 95% of the cases
> > >where might use a local variable in a >comprehension. And of the remaining
> > >cases, many of them will be so >complex that they should be re-written as
> > >an explicit for-loop. So in my >opinion, we're only talking about a "need"
> > >to solve the problem for a >small proportion of cases: > >- most
> > >comprehensions don't need a local variable (apart from > the loop
> > >variable) at all; > >- of those which do need a local variable, most can
> > >be easily > solved using a nested comprehension or a helper function; > >-
> > >of those which cannot be solved that way, most are complicated > enough
> > >that they should use a regular for-loop; > >- leaving only a small number
> > >of cases which are complicated enough > to genuinely benefit from local
> > >variables but not too complicated. > >So this is very much a borderline
> > >feature. Occasionally it would be >"nice to have", but on the negative
> > >side: > >- it adds complexity to the language; > >- makes comprehensions
> > >harder to read; > >- and people will use it unnecessarily where there is
> > >no readability > or speed benefit (premature optimization again). > >It is
> > >not clear to me that we should burden *all* Python programmers >with
> > >additional syntax and complexity of an *already complex* feature >for such
> > >a marginal improvement. > > >> In my opinion, code like >> [y + g(y) for x
> > >in range(10) **some syntax for `y=f(x)` here**] >> is more natural than
> > >any solution we now have. >> And that's why I pro the new syntax, it's
> > >clear, explicit and readable > >How can you say that the new syntax is
> > >"clear, explicit and readable" >when you haven't proposed any new syntax
> > >yet? > >For lack of anything better, I'm going to suggest "let y = f(x)"
> > >as the >syntax, although personally I don't like it even a bit. > >Where
> > >should the assignment go? > > [(y, y**2) let y = x+1 for x in (1, 2, 3,
> > >4)] > > [(y, y**2) for x in (1, 2, 3, 4) let y = x+1] > >I think they're
> > >both pretty ugly, but I can't think of anything else. > >Can we rename the
> > >loop variable, or is that an error? > > [(x, x**2) let x = x+1 for x in
> > >(1, 2, 3, 4)] > >How do they interact when you have multiple loops and
> > >if-clauses? > > [(w, w**2) for x in (1, 2, 3, 4) let y = x+1 > for a in
> > >range(y) let z = a+1 if z > 2 > for b in range(z) let w = z+1] > > >For
> > >simplicity, perhaps we should limit any such local assignments to >the
> > >very end of the comprehension: > > [expression for name in sequence >
> > ><zero or more for-loops and if-clauses> > <zero or more let-clauses> > ] >
> > >>but that means we can't optimise this sort of comprehension: > >
> > >[expression for x in sequence > for y in (something_expensive(x) +
> > >function(something_expensive(x)) > ] > >Or these: > > [expression for x in
> > >sequence > if something_expensive(x) or condition(something_expensive(x))
> > >> ] > > >I think these are very hard questions to answer. >
I think the assignment should be treated equally as for-clause and if-clause
which means
[(y, y**2) for x in (1, 2, 3, 4) let y = x+1]
would be a right syntax.
And
[(x, x**2) for x in (1, 2, 3, 4) let x = x+1]
would not cause an error because we can also write
[(x, x**2) for x in (1, 2, 3, 4) for x in (4, 3, 2, 1)]
now.
I didn't see any problem in
[(w, w**2) for x in (1, 2, 3, 4) let y = x+1
for a in range(y) let z = a+1 if z > 2
for b in range(z) let w = z+1]
In my opinion, it would behave the same as
for x in (1, 2, 3, 4):
y = x+1
for a in range(y):
z = a+1
if z > 2:
for b in range(z):
w = z+1
mylist.append((w, w**2))
According to my understanding, the present for-clause and if-clause does
everything quite similar to this nested way,
> >-- >Steve >_______________________________________________ >Python-ideas
> >mailing list >Python-ideas@python.org
> >>https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct:
> >http://python.org/psf/codeofconduct/_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
