Re: [Python-ideas] Temporary variables in comprehensions

[Neil Girdhar]

On Fri, Feb 23, 2018 at 1:42 PM Kyle Lahnakoski 
wrote:

>
>
> On 2018-02-23 12:44, Neil Girdhar wrote:
>
> On Fri, Feb 23, 2018 at 12:35 PM Kyle Lahnakoski 
> wrote:
>
>>
>> > [
>> > (w, w**2)
>> > for x in (1, 2, 3, 4)
>> > let y = x+1
>> > for a in range(y)
>> > let z = a+1
>> > if z > 2
>> > for b in range(z)
>> > let w = z+1
>> > ]
>>
>> which is a short form for:
>>
>> > def stuff():
>> > for x in (1, 2, 3, 4):
>> > y = x+1
>> > for a in range(y):
>> > z = a+1
>> > if z > 2:
>> > for b in range(z):
>> > w = z+1
>> > yield (w, w**2)
>> >
>> > list(stuff())
>>
>
> Is it that much shorter that it's worth giving up the benefit of
> indentation?
>
>>
>>
> Saving the indentation? Oh yes, for sure!  This code reads like a story,
> the indentation is superfluous to that story.  Should we add it to Python?
> I don't know; I quick scan through my own code, and I do not see much
> opportunity for list comprehensions of this complexity.
>

That's a good thing that you are not writing code like this.  I don't agree
that the indentation is "superfluous".  It makes the code easy to read.
Anyway, Google's Python style guide also agrees that very long
comprehensions like that are not worth it. (
https://google.github.io/styleguide/pyguide.html?showone=List_Comprehensions#List_Comprehensions
)

Either my data structures are not that complicated, or I have try/except
> blocks inside a loop, or I am using a real query language (like SQL).
> pythonql seems to solve all these problems well enough (
> https://github.com/pythonql/pythonql).
>

> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/KwZtO4rpAGE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/KwZtO4rpAGE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Neil Girdhar]

On Fri, Feb 23, 2018 at 12:35 PM Kyle Lahnakoski 
wrote:

> I believe list comprehensions are difficult to read because they are not
> formatted properly. For me, list comprehension clauses are an
> expression, followed by clauses executed in the order. Any list
> comprehension with more than one clause should be one-line-per clause.
>
> Examples inline:
>
> On 2018-02-15 19:57, Steven D'Aprano wrote:
> > Where should the assignment go? [(y, y**2) let y = x+1 for x in (1, 2,
> > 3, 4)] [(y, y**2) for x in (1, 2, 3, 4) let y = x+1]
>
> Since y is a function of x, it must follow the for clause:
>
> > [
> > (y, y ** 2)
> > for x in (1, 2, 3, 4)
> > let y = x + 1
> > ]
>
> > How do they interact when you have multiple loops and if-clauses? [(w,
> > w**2) for x in (1, 2, 3, 4) let y = x+1 for a in range(y) let z = a+1
> > if z > 2 for b in range(z) let w = z+1]
>
> They are applied in order:
>
> > [
> > (w, w**2)
> > for x in (1, 2, 3, 4)
> > let y = x+1
> > for a in range(y)
> > let z = a+1
> > if z > 2
> > for b in range(z)
> > let w = z+1
> > ]
>
> which is a short form for:
>
> > def stuff():
> > for x in (1, 2, 3, 4):
> > y = x+1
> > for a in range(y):
> > z = a+1
> > if z > 2:
> > for b in range(z):
> > w = z+1
> > yield (w, w**2)
> >
> > list(stuff())
>

Is it that much shorter that it's worth giving up the benefit of
indentation?

>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/KwZtO4rpAGE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Kyle Lahnakoski]

I believe list comprehensions are difficult to read because they are not
formatted properly. For me, list comprehension clauses are an
expression, followed by clauses executed in the order. Any list
comprehension with more than one clause should be one-line-per clause.

Examples inline:

On 2018-02-15 19:57, Steven D'Aprano wrote:
> Where should the assignment go? [(y, y**2) let y = x+1 for x in (1, 2,
> 3, 4)] [(y, y**2) for x in (1, 2, 3, 4) let y = x+1] 

Since y is a function of x, it must follow the for clause:

> [
>     (y, y ** 2)
>     for x in (1, 2, 3, 4)
>     let y = x + 1
> ]

> How do they interact when you have multiple loops and if-clauses? [(w,
> w**2) for x in (1, 2, 3, 4) let y = x+1 for a in range(y) let z = a+1
> if z > 2 for b in range(z) let w = z+1] 

They are applied in order:

> [
> (w, w**2)
> for x in (1, 2, 3, 4)
> let y = x+1
> for a in range(y)
> let z = a+1
> if z > 2
> for b in range(z)
> let w = z+1
> ]

which is a short form for:

> def stuff():
> for x in (1, 2, 3, 4):
> y = x+1
> for a in range(y):
> z = a+1
> if z > 2:
> for b in range(z):
> w = z+1
> yield (w, w**2)
>
> list(stuff())


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Kyle Lahnakoski]

On 2018-02-17 11:23, fhsxfhsx wrote:
> [
>   {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>   }
>   for goods_id in goods_id_list
>   for goods is Goods.get_by_id(goods_id)
>   for gc is GoodsCategory.get_by_id(goods.category_id)
> ]

in the short term, it seems for...in [...] is good enough:

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     for goods in [Goods.get_by_id(goods_id)]
>     for gc in [GoodsCategory.get_by_id(goods.category_id)]
> ]

I personally would like to see with...as... syntax allowed in list
comprehensions, despite `with` being limited to context managers to date.

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     with Goods.get_by_id(goods_id) as goods
>     with GoodsCategory.get_by_id(goods.category_id) as gc
> ]

..,or maybe `let` reads easier?

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     let goods = Goods.get_by_id(goods_id)
>     let gc = GoodsCategory.get_by_id(goods.category_id)
> ]




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Neil Girdhar]

You should give an actual motivating example.  I think none of these 
suggestions are more readable than just writing things out as a for loop.  
You argue that you want to avoid appending to a result list.  In that case, 
I suggest writing your pattern as a generator function.

Best,

Neil

On Thursday, February 15, 2018 at 2:03:31 AM UTC-5, fhsxfhsx wrote:
>
> As far as I can see, a comprehension like 
> alist = [f(x) for x in range(10)]
> is better than a for-loop
> for x in range(10):
>   alist.append(f(x))
> because the previous one shows every element of the list explicitly so 
> that we don't need to handle `append` mentally.
>
> But when it comes to something like
> [f(x) + g(f(x)) for x in range(10)]
> you find you have to sacrifice some readableness if you don't want two 
> f(x) which might slow down your code.
>
> Someone may argue that one can write
> [y + g(y) for y in [f(x) for x in range(10)]]
> but it's not as clear as to show what `y` is in a subsequent clause, not 
> to say there'll be another temporary list built in the process.
> We can even replace every comprehension with map and filter, but that 
> would face the same problems.
>
> In a word, what I'm arguing is that we need a way to assign temporary 
> variables in a comprehension.
> In my opinion, code like
> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> is more natural than any solution we now have.
> And that's why I pro the new syntax, it's clear, explicit and readable, 
> and is nothing beyond the functionality of the present comprehensions so 
> it's not complicated.
>
> And I hope the discussion could focus more on whether we should allow 
> assigning temporary variables in comprehensions rather than how to solve 
> the specific example I mentioned above.
>
>
>  
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Steven D'Aprano]

On Sun, Feb 18, 2018 at 12:23:28AM +0800, fhsxfhsx wrote:
> Thank you Paul, what you said is enlightening and I agree on most part of it.
> 
> I'll propose two candidate syntaxs.
> 1. `with ... as ...`
> This syntax is more paralles as there would be `for` and `with` 
> clause as well as `for` and `with` statement. However, the 
> existing `with` statement is semantically different from this one, 
> although similar.

I don't think they are even a little bit similar. The existing `with` 
statement is for controlling cleanup code (using a context manager). 
This proposal doesn't have anything to do with context managers or 
cleanup code. It's just a different way to spell "name = value" inside 
comprehensions.


> 2. `for ... is ...`
> This syntax is more uniform as the existing `for` clause make an 
> iterator which is a special kind of variable. However, I'm afraid 
> this syntax might be confused with `for ... in ...` as they differ 
> only on one letter.

Indeed.

And frankly, treated as English grammar, "for value is name" doesn't 
make sense and is horribly ugly to my eyes:

result = [x for value in sequence for value+1 is x]


> And here is an example which appears quite often in my code where I 
> think a new syntax can help a lot: Suppose I have an list of goods 
> showing by their ids in database, and I need to transform the ids into 
> json including information from two tables, `Goods` and 
> `GoodsCategory`, where the first table recording `id`, `name` and 
> `category_id` indicating which category the goods belongs to, the 
> second table recording `id`, `name` and `type` to the categories.
>
> With the new syntax, I can write
> [
>   {
> 'id': goods.id,
> 'name': goods.name,
> 'category': gc.name,
> 'category_type': gc.type,
>   }
>   for goods_id in goods_id_list
>   for goods is Goods.get_by_id(goods_id)
>   for gc is GoodsCategory.get_by_id(goods.category_id)
> ]
> 
> And I cannot think of any good solutions as this one without it.

I can of a few, starting with the most simple: write a helper function. 
Not every problem needs to be solved with new syntax.

def dict_from_id(goods_id):
goods = Goods.get_by_id(goods_id)
gc = GoodsCategory.get_by_id(goods.category_id)
return {'id': goods.id,
'name': goods.name,
'category': gc.name,
'category_type': gc.type
}

result = [dict_from_id(goods_id) for goods_id in goods_id_list]

That's much nicer to read, you can document and test the dict_from_id() 
function, no new systax is required, it is easy to refactor, and I very 
much doubt that adding one extra function call is going to be a 
significant slowdown compared to the cost of two calls to get_by_id() 
methods and constructing a dict.

(And if as you add more fields to the dict, the overhead of the function 
call becomes an even smaller proportion.)

Or you can make this a method of the goods object, which is arguably a 
better OO design. Let the goods object be responsible for creating the 
dict.

result = [Goods.get_by_id(goods_id).make_dict() for goods_id in goods_id_list]
# or if you prefer
result = [goods.make_dict() for goods in map(Goods.get_by_id, goods_id_list)]

Here is a third solution: use a for-loop iterating over a single-item 
tuple to get the effect of a local assignment to a temporary variable:

result = [{ 
   # dict display truncated for brevity...
   } 
   for goods_id in goods_id_list
   for goods in (Goods.get_by_id(goods_id),)
   for gc in (GoodsCategory.get_by_id(goods.category_id),)
   ]

If you don't like the look of single-item tuples (foo,) you can use 
single-item lists instead [x] but they are a tiny bit slower to create.

Serhiy has suggested that the interpreter can optimize the single-item 
loop to make it as fast as a bare assignment:

https://bugs.python.org/issue32856

I think this is a neat trick, although Yuri thinks it is an ugly hack 
and doesn't want to encourage it. Neat or ugly, I think it is better 
than "for value is name".


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

Thanks so much for the comments and the collect on this syntax!

Comments on *previous talk*
The list also mentioned some other previous proposals, so I myself search it 
and find that there're even more in the mailing list since 2008.
https://mail.python.org/pipermail/python-ideas/2008-August/001842.html

Comments on *previous talk: PEP*
The PEP seems to be about an explicit temporary namespace where any objects 
(including functions, classes, etc.) could be held.
However, I find that this syntax may not work for temporary variables in 
comprehensions.

We might expect this syntax work like
[?.y+2 for x in range(5) given: y = x+1]
But notice that the proposed `given` statement here appeared in comprehensions, 
where syntax changes in comprehensions are needed, and nothing about this is 
mentioned in the PEP.
What's more, the proposed syntax is a statement like `for` statement, `if` 
statement, rather than a for-clause or if-clause in comprehensions. In my 
opinion, it's not a good idea to have a statement in comprehensions. Instead, 
another given-clause should be added if one wants to write code like above, 
doing assignments in comprehensions, which can look like:
[?.y+2 for x in range(5) given y = x+1]
So the ? symbol seems useless here, so maybe
[y+2 for x in range(5) given y = x+1]
make it quite similar to the `where` syntax you proposed.

So, I agree with you that it is a good idea to have a new PEP, though for a 
different reason.
 
Comments on *Choice of syntax*
I gave two candidate syntaxs in 
https://mail.python.org/pipermail/python-ideas/2008-August/001842.html, one 
said `for ... in ...`, another said `with ... as ...`.
The first has the same problem as `for ... = ...` you proposed. And the biggest 
problem I think the second will face is the difference in semantic between the 
`with` statement and it.
When it comes to `where ... = ...`, there are one possible problem I can think 
of.
`where` is not now a keyword in Python. There are WHERE clauses in SQL, so in 
many modules including peewee and SQLAlchemy, `where` is an important method. 
The new syntax would cause quite incompatibilities.

Personally I agreed with you that postfix notation would have advantage over 
prefix notation. Other than `where`, `with` is quite readable in my opinion. So 
maybe `with ... = ...` can be another candidate?

Plus the `given ... = ...` I mentioned above, there are several more candidates 
now. Personally I perfer `with ... = ...`, because `with` is a python keyword 
so it would be good for backward compatibility.

*About comprehensions and expressions*
You gave `print(y+2 where y = x+1)` as an example, I think it should be 
clarified that it's not, or at least, does not look like a comprehension, but 
an expression. It should give an object `y+2` rather than a list or a 
generator. (otherwise what list can it give?)
There are for-clause and if-clause for comprehensions, and if-clause (aka 
ternary operators) for expressions. So, In my opinion, there should be 
additional discuss and examples to show that it's helpful to have such syntax.

For the where-clause in expressions, I think we could refer to how python 
handles if-clause.
The following setences are legal:
[x if x else 0 for x in mylist if x > 10]
The following illegal:
[x if x for x in mylist if x > 10]
[x if x else 0 for x in mylist if x > 10 else 10]
That's to say, the two kinds of if-clause, one is only used in expressions (aka 
`ternary operator`), the other is only used in comprehensions. They slightly 
differ in syntax.

The where-clause might work in similar ways. Then
[z+2 where z = y+1 for x in mylist where y = x+1]
means
[(z+2 where z=y+1) for x in mylist where y = x+1]
where the parenthesis (also the part before the first `for`) is a expression, 
the rest is comprehension clauses.
To be more accurate, the new syntax would be:

test: where_test ['if' where_test 'else' test] | lambdef
where_test: or_test | ( '(' or_test 'where' NAME '=' testlist_star_expr ')' )

Mandatory parenthesis in where_test is to resolve the ambiguity in code like
print(y+2 where y=x+1 if x>0 else x-1 if x>1 else 0)
It could be analysed like
print((y+2 where y=x+1 if x>0 else x-1) if x>1 else 0)
or
print(y+2 where y=(x+1 if x>0 else x-1 if x>1 else 0)).
I guess thektulu may have mandatory parenthesis for the same reason.

I haven't check the new syntax very carefully so there might be other 
ambiguities.

Another example is
print(y+2 if x>0 else y-2 where y=x+1)
with mandatory parenthesis, one must write
print((y+2 if x>0 else y-2 where y=x+1))
or
print(y+2 if x>0 else (y-2 where y=x+1))

However, it might still confuse many people. I wonder whether it's a good idea 
to have such syntax.

It would be much easier to add assignments in comprehensions.
comp_iter: comp_for | comp_if | comp_where
comp_where: 'where' NAME '=' testlist_star_expr [comp_iter]

Comments on *Goals of the new syntax*
I have a real-world example in 

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

Hi Steve, Thank you for so detailed comments.
My comments also below interleaved with yours.






At 2018-02-16 08:57:40, "Steven D'Aprano"  wrote:
>Hi fhsxfhsx, and welcome.
>
>My comments below, interleaved with yours.
>
>
>On Thu, Feb 15, 2018 at 01:56:44PM +0800, fhsxfhsx wrote:
>
>[quoted out of order]
>> And I hope the discussion could focus more on whether we should allow 
>> assigning temporary variables in comprehensions rather than how to 
>> solve the specific example I mentioned above.
>
>Whether or not to allow this proposal will depend on what alternate 
>solutions to the problem already exist, so your specific example is very 
>relevant. Any proposed change has to compete with existing solutions.
>>
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so 
>> that we don't need to handle `append` mentally.
>
>While I personally agree with you, many others disagree. I know quite a 
>few experienced, competent Python programmers who avoid list 
>comprehensions because they consider them harder to read and reason 
>about. They consider a regular for-loop better precisely because you do 
>see the explicit call to append.
>
>(In my experience, those of us who get functional-programming idioms 
>often forget that others find them tricky.)
>
>The point is that list comprehensions are already complex enough that 
>they are difficult for many people to learn, and some people never come 
>to grips with them. Adding even more features comes with a cost.
>
>The bottom line is that it isn't clear to me that allowing local 
>variables inside comprehensions will make them more readable.
>

To be frank, I had not thought of this before.
However, in my opinion, when considering adding a new syntax, we care more 
about the marginal cost.
I mean, I think it is the functional-programming way which is tricky, but 
allowing a new syntax would not make things worse.
Well, that's just a guess, maybe only those who are puzzled with comprehensions 
can give us an answer.


> >> But when it comes to something like >> [f(x) + g(f(x)) for x in range(10)] 
> >> >> you find you have to sacrifice some readableness if you don't want two 
> >> >> f(x) which might slow down your code. > >The usual comments about 
> >> premature optimisation apply here. > >Setting a new comprehension variable 
> >> is not likely to be free, and may even be >more costly than calling f(x) 
> >> twice if f() is a cheap expression: > > [x+1 + some_func(x+1) for x in 
> >> range(10)] > >could be faster than > > [y + some_func(y) for x in 
> >> range(10) let y = x + 1] > >or whatever syntax we come up with. >
It is true. But since there are still so many cases where a temporary variable 
is faster. 
Also, even without let-clause, one can write a for-loop with a temporary 
variable which slow down the code.
So, it seems that "setting a new comprehension variable may even be more 
costly" does not show any uselessness of temporary variables in comprehensions.


> >> Someone may argue that one can write >> [y + g(y) for y in [f(x) for x in 
> >> range(10)]] > >Indeed. This would be the functional-programming solution, 
> >> and I >personally think it is an excellent one. The only changes are that 
> >> I'd >use a generator expression for the intermediate value, avoiding the 
> >> need >to make a full list, and I would lay it out more nicely, using 
> >> >whitespace to make the structure more clear: > > result = [y + g(y) for y 
> >> in > (f(x) for x in range(10)) > ] >

In my opinion,
[
y + g(y)
for x in range(10)
let y = f(x)
]
is better because it's more corresponding to a for-loop
for x in range(10):
y = f(x)
result.append(y + g(y))
In my opinion, comprehensions are not real functional-programming because there 
is not even a function. Though there're similarities, map and filter are real 
functional-programming. Since the similarity between for-clause in 
comprehensions and the for-loop, I think it's better to write comprehensions 
more close to for-loop.
I don't know but I guess maybe it can also help those who fear comprehensions 
better embrace them?


>
>> but it's not as clear as to show what `y` is in a subsequent clause, 
>> not to say there'll be another temporary list built in the process.
>
>There's no need to build the temporary list. Use a generator 
>comprehension. And I disagree that the value of y isn't as clear.
>
>An alternative is simply to refactor your list comprehension. Move the 
>calls to f() and g() into a helper function:
>
>def func(x):
>y = f(x)
>return y + g(y)
>
>and now you can write the extremely clear comprehension
>
>[func(x) for x in range(10)]
>
>that needs no extra variable.
>
I think it can be a goods idea if there's a name to `func` which is easy to 
understand, or `func` is put close to 

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

You are right and actually I sometimes did the same thing in a temporary script 
such as in ipython.
Because in my opinion, it's not really elegant code as one may be puzzled for 
the list `[f(x)]`.
Well, although that's quite subjective.

And also I test the code and find another `for` clause can make time cost about 
1.5 times in my computer, even when I optimize `[f(x)]` to a generator `(f(x), 
)`. Though that's not very big issue if you don't care about efficiency that 
much. But a temporary list or generator is redundant here. `[f(x)]` can be an 
alternative, but I think it is worth a new syntax.







At 2018-02-15 18:11:47, "Stephan Houben"  wrote:

Note that you can already do:

 [y + g(y) for x in range(10) for y in [f(x)]]


i.e. for y  in [expr]

does exactly what the OP wants.

No new syntax needed.


If you hang out on python-list , you'll soon notice

that many newbies struggle already with the list comprehension

syntax.

It's a mini-language which is almost, but not entirely,

exactly unlike normal Python code.

Let's not complicate it further.



Stephan

 



2018-02-15 10:53 GMT+01:00 Evpok Padding :

For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
I don't really see what the issue is, if you really want to make it shorter,

you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
case where I like `map` more than comprehensions.


For more complex case, just define a intermediate generator along the lines

```
f_samples = (f(x) for x in range(10))
[y+g(y) for y in f_samples]
```
Which does exactly the same thing but
  - Is more readable and explicit

  - Has no memory overhead thanks to lazy evaluation
(btw, you should consider generators for your nested comprenshions)


While I am sometimes in the same state of mind, wishing for variables in
comprehensions seems to me like a good indicator that your code needs
refactoring.


Best,


E


On 15 February 2018 at 10:32, Jamie Willis  
wrote:
>
> I +1 this at surface level; Both Haskell list comprehensions and Scala for 
> comprehensions have variable assignment in them, even between iterating and 
> this is often very useful. Perhaps syntax can be generalised as:
>
> [expr_using_x_and_y
>  for i in is
>   x = expr_using_i
>  for j in is
>   y = expr_using_j_and_x]
>
> This demonstrates the scope of each assignment; available in main result and 
> then every clause that follows it.
>
> Sorry to op who will receive twice, forgot reply to all
>
> On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
>>
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so that 
>> we don't need to handle `append` mentally.
>>
>> But when it comes to something like
>> [f(x) + g(f(x)) for x in range(10)]
>> you find you have to sacrifice some readableness if you don't want two f(x) 
>> which might slow down your code.
>>
>> Someone may argue that one can write
>> [y + g(y) for y in [f(x) for x in range(10)]]
>> but it's not as clear as to show what `y` is in a subsequent clause, not to 
>> say there'll be another temporary list built in the process.
>> We can even replace every comprehension with map and filter, but that would 
>> face the same problems.
>>
>> In a word, what I'm arguing is that we need a way to assign temporary 
>> variables in a comprehension.
>> In my opinion, code like
>> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
>> is more natural than any solution we now have.
>> And that's why I pro the new syntax, it's clear, explicit and readable, and 
>> is nothing beyond the functionality of the present comprehensions so it's 
>> not complicated.
>>
>> And I hope the discussion could focus more on whether we should allow 
>> assigning temporary variables in comprehensions rather than how to solve the 
>> specific example I mentioned above.
>>
>>
>>  
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

Thank you Paul, what you said is enlightening and I agree on most part of it.

I'll propose two candidate syntaxs.
1. `with ... as ...`
This syntax is more paralles as there would be `for` and `with` clause as 
well as `for` and `with` statement. However, the existing `with` statement is 
semantically different from this one, although similar.
2. `for ... is ...`
This syntax is more uniform as the existing `for` clause make an iterator 
which is a special kind of variable. However, I'm afraid this syntax might be 
confused with `for ... in ...` as they differ only on one letter.

I like the latter one better. Other proposes are absolutely welcome.

And here is an example which appears quite often in my code where I think a new 
syntax can help a lot:
Suppose I have an list of goods showing by their ids in database, and I need to 
transform the ids into json including information from two tables, `Goods` and 
`GoodsCategory`, where the first table recording `id`, `name` and `category_id` 
indicating which category the goods belongs to, the second table recording 
`id`, `name` and `type` to the categories.
With the new syntax, I can write
[
  {
'id': goods.id,
'name': goods.name,
'category': gc.name,
'category_type': gc.type,
  }
  for goods_id in goods_id_list
  for goods is Goods.get_by_id(goods_id)
  for gc is GoodsCategory.get_by_id(goods.category_id)
]

And I cannot think of any good solutions as this one without it.
To generalize this case, for each element of the list, I need two temporary 
variables (`goods` and `gc` in my case), and each one was used twice.


And reply to the two past discussions you mentioned,
1.https://mail.python.org/pipermail/python-ideas/2011-April/009863.html
This mail gave a solution to modify function `f` to keep the result. The 
weak point is obvious, you must modify the function `f`.
2.https://mail.python.org/pipermail/python-ideas/2012-January/013468.html
This mail wrote
>The important thing is that you name the thing you care about before using it.
>I think this is a very natural way of writing: first you give the thing you
>care about a name, then you refer to it by name.
However, that's a problem every comprehension faces, not a problem drawn by the 
new syntax.







At 2018-02-15 18:08:46, "Paul Moore"  wrote:
>On 15 February 2018 at 05:56, fhsxfhsx  wrote:
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so that
>> we don't need to handle `append` mentally.
>
>... as long as the code inside the comprehension remains relatively
>simple. It's easy to abuse comprehensions to the point where they are
>less readable than a for loop, but that's true of a lot of things, so
>isn't a specific problem with comprehensions.
>
>> But when it comes to something like
>> [f(x) + g(f(x)) for x in range(10)]
>> you find you have to sacrifice some readableness if you don't want two f(x)
>> which might slow down your code.
>
>Agreed. I hit that quite often.
>
>> Someone may argue that one can write
>> [y + g(y) for y in [f(x) for x in range(10)]]
>> but it's not as clear as to show what `y` is in a subsequent clause, not to
>> say there'll be another temporary list built in the process.
>> We can even replace every comprehension with map and filter, but that would
>> face the same problems.
>
>That is a workaround (and one I'd not thought of before) but I agree
>it's ugly, and reduces readability. Actually, factoring out the inner
>comprehension like Evpok Padding suggests:
>
>f_samples = (f(x) for x in range(10))
>[y+g(y) for y in f_samples]
>
>is very readable and effective, IMO, so it's not *that* obvious that
>local names are beneficial.
>
>> In a word, what I'm arguing is that we need a way to assign temporary
>> variables in a comprehension.
>
>"We need" is a bit strong here. "It would be useful to have" is
>probably true for some situations.
>
>> In my opinion, code like
>> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
>> is more natural than any solution we now have.
>> And that's why I pro the new syntax, it's clear, explicit and readable, and
>> is nothing beyond the functionality of the present comprehensions so it's
>> not complicated.
>
>The problem is that you haven't proposed an actual syntax here, just
>that one should be invented. There have been discussions on this in
>the past (a quick search found
>https://mail.python.org/pipermail/python-ideas/2011-April/009863.html
>and https://mail.python.org/pipermail/python-ideas/2012-January/013468.html,
>for example).
>
>> And I hope the discussion could focus more on whether we should allow
>> assigning temporary variables in comprehensions rather than how to solve the
>> specific example I mentioned above.
>
>The problem isn't so much "whether we 

Re: [Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

A generator can be a good idea, however, I wonder if it's really readable to 
have a `f_samples`.
And the structure is the same as in
[y + g(y) for y in [f(x) for x in range(10)]]
if you replace the list with a generator. And it's similar for other solutions 
you mentioned.
Well, I know that it can be quite different for everyone to determine whether a 
piece of code is readable or not, maybe it's wise to wait for more opinions.

There's another problem that, as you distinguished the `simple` and `more 
complex` case, the offered solutions seem very specific, I'm not sure if 
there's a universal solution for every case of temporary variables in 
comprehensions. If not, I think it's too high cost to reject a new syntax if 
you need to work out a new solution every time.







At 2018-02-15 17:53:21, "Evpok Padding"  wrote:

For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
I don't really see what the issue is, if you really want to make it shorter,

you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
case where I like `map` more than comprehensions.


For more complex case, just define a intermediate generator along the lines

```
f_samples = (f(x) for x in range(10))
[y+g(y) for y in f_samples]
```
Which does exactly the same thing but
  - Is more readable and explicit

  - Has no memory overhead thanks to lazy evaluation
(btw, you should consider generators for your nested comprenshions)


While I am sometimes in the same state of mind, wishing for variables in
comprehensions seems to me like a good indicator that your code needs
refactoring.


Best,


E


On 15 February 2018 at 10:32, Jamie Willis  
wrote:
>
> I +1 this at surface level; Both Haskell list comprehensions and Scala for 
> comprehensions have variable assignment in them, even between iterating and 
> this is often very useful. Perhaps syntax can be generalised as:
>
> [expr_using_x_and_y
>  for i in is
>   x = expr_using_i
>  for j in is
>   y = expr_using_j_and_x]
>
> This demonstrates the scope of each assignment; available in main result and 
> then every clause that follows it.
>
> Sorry to op who will receive twice, forgot reply to all
>
> On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
>>
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so that 
>> we don't need to handle `append` mentally.
>>
>> But when it comes to something like
>> [f(x) + g(f(x)) for x in range(10)]
>> you find you have to sacrifice some readableness if you don't want two f(x) 
>> which might slow down your code.
>>
>> Someone may argue that one can write
>> [y + g(y) for y in [f(x) for x in range(10)]]
>> but it's not as clear as to show what `y` is in a subsequent clause, not to 
>> say there'll be another temporary list built in the process.
>> We can even replace every comprehension with map and filter, but that would 
>> face the same problems.
>>
>> In a word, what I'm arguing is that we need a way to assign temporary 
>> variables in a comprehension.
>> In my opinion, code like
>> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
>> is more natural than any solution we now have.
>> And that's why I pro the new syntax, it's clear, explicit and readable, and 
>> is nothing beyond the functionality of the present comprehensions so it's 
>> not complicated.
>>
>> And I hope the discussion could focus more on whether we should allow 
>> assigning temporary variables in comprehensions rather than how to solve the 
>> specific example I mentioned above.
>>
>>
>>  
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Martin Teichmann]

Hi list,

> But when it comes to something like
> [f(x) + g(f(x)) for x in range(10)]
> you find you have to sacrifice some readableness if you don't want two
f(x)
> which might slow down your code.
>
> Someone may argue that one can write
> [y + g(y) for y in [f(x) for x in range(10)]]

personally I think that the biggest problem readability-wise is that "for"
is a post-fix operator, which makes generators much harder to read. It's
also very different from normal for loops, which have the "for" at the top.
IMHO generators would be much easier to read with a prefix for, as in

[for x in range(10): f(x) + g(f(x))]

also nested generators get nicer like that:

[for y in (for x in range(10): f(x)):
  y + g(y)]

one could critique here that we shouldn't use colons in expressions, but
that boat has sailed: we use them for lambdas. We do not write

 sq = x**2 lambda x

and I am not proposing that.

Also if/else could be written with colons, but I am not sure whether that's
actually nicer:

val = (if attr is None: 5 else: attr + 3)

but it certainly is in case of ifs in generators:

[for x in range(10):
 if x % 3 != 2: x]

which could be problematic to parse if you compare that to

[for x in range(10):
 if x % 3 == 2: x - 1
 else: x + 1]

one could even dig out the often-proposed always-rejected
except-in-expression:

[for x in range(10):
 try: f(x)
 except WhateverError: None]

or even a with:

[for x in file_names:
 with open(x) as f:
 f.read()]


Greetings

Martin
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Steven D'Aprano]

On Thu, Feb 15, 2018 at 10:13:37AM +, Jamie Willis wrote:

> I'm not sure it does indicate a need for refactoring, I'd argue it's quite
> a common pattern, at least in functional languages from which this
> construct arises.
> 
> In fact, in those languages, there are laws that govern interactions with
> the comprehensions (though this comes from monads and monads perhaps don't
> quite apply to pythons model). These laws define behaviour that is expected
> equivalent by users;

Python is not a functional language, and the usual Pythonic solution to 
an overly complex or inefficient functional expression is to refactor 
into non-functional style.

In my experience, most Python users have never even heard of monads, and 
those who have, few grok them. (I know I don't.)


> [x for x in xs] = xs
> [f(x) for x in [x]] = f(x)
> [g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]

I think these should be:

[x for x in xs] = list(xs)# not to be confused with [x]
[f(x) for x in [x]] = [f(x)]
[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in [f(x)]]

 
> Even though that last law isn't completely analogous to the given example
> from the OP,

It can be used though. He has:

[f(x) + g(f(x)) for x in xs]

which can be written as

[y + g(y) for x in xs for y in [f(x)]]


Here's an example:

# calls ord() twice for each x
py> [ord(x) + ord(x)**2 for x in "abc"]
[9506, 9702, 9900]

# calls ord() once for each x
py> [y + y**2 for x in "abc" for y in [ord(x)]]
[9506, 9702, 9900]

And one last version:

py> [y + y**2 for y in [ord(x) for x in "abc"]]
[9506, 9702, 9900]

In production, I'd change the last example to use a generator 
comprehension.


> the transformation he wants to be able to do does arise from
> the laws. So it could be argued that not being able to flatten the
> comprehension down via law 3 is unexpected behaviour 

Law 3 does apply, and I'm not sure what you mean by the statement that 
we can't flatten the comprehension down.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Robert Vanden Eynde]

Hello, talking about this syntax :

[y+2 for x in range(5) let y = x+1]

*Previous talks*
I've had almost exactly the same idea in June 2017, see subject "variable
assignment in functional context here: https://mail.python.org/
pipermail/python-ideas/2017-June/subject.html.

Currently this can easily be done by iterating over a list of size 1:

[y+2 for x in range(5) for y in [x+1]]

(Other ways exist, see section "current possibilities" below).

This comparison would answer a lot of questions like "can we write [x+2 for
x in range(5) for x in [x+1]], so [x+2 for x in range(5) let x = x+1]" the
new variable would indeed shadow the old one.

In June 2017 I introduced the "let" syntax using the existing "for"
keyword, using the "=" instead of a "in" like this:

[y+2 for x in range(5) for y = x+1]

The only difference I introduced was that it would be logical to accept the
syntax in any expression:

x = 5
print(y+2 for y = x + 1)

or with the "let" keyword:

x = 5
print(y+2 let y = x + 1)

*Previous talk: Pep*
In the June conversation one conclusion was that someone wrote a pep (
https://www.python.org/dev/peps/pep-3150/) in 2010 that's still pending
(not exactly the same syntax but the same idea, he used a "given:" syntax)
that looked like that :

print(y+2 given:
y=x+1)

*Previous talk: GitHub implementation on Cython*
Another conclusion was that someone on GitHub implemented a "where"
statement in Cython (url: https://github.com/thektulu/cpython/commit/
9e669d63d292a639eb6ba2ecea3ed2c0c23f2636) where one could write :

print(y+2 where y = x+1)

So in a list comprehension:

[y+2 where y = x + 1 for x in range(5)]

As the author thektulu said "just compile and have fun".

*New syntax in list comprehension*
However, we probably would like to be able to use this "where" after the
for:

[y+2 for x in range(5) where y = x+1]

This would allow the new variable to be used in further "for" and "if"
statement :

[y+z for x in range(5) where y = x+1 for z in range(y+1)]

*Choice of syntax*
Initially I thought re-using the "for" keyword would be a good idea for
backward comptability (a variable named "where" in old code wouldn't be a
problem), however some people pointed out that the python Grammar wouldn't
be LL1, when reading the "for" token it wouldn't be able to choose directly
if the rest would be a "for in" or "for =" so actually introducing a
dedicated keyword is probably better, like this :

[y+2 for x in range(5) where y = x+1]
print(y+2 where y = x+1)

The "where" keyword is very readable in my opinion, very close to English
sentences we use. Sometimes we introduce a new variable before using it,
generally using "let" or after using it using "where". For example "let y =
x+1; print(y+2)". Or "print(y+2 where y = x+1)".

The first syntax is chosen in the "let in" syntax in haskell :

print(let y = x+2 in y+2)

Or chained:
print(let x = 2 in let y = x+1 in y+2)

But Haskell user would probably line break for clarity :

print(let x = 2 in
 let y = x+1 in
 y+2)

A postfix notation using "where" would probably be less verbose in my
opinion. Another example of "postfix notation" python has is with the "a if
condition else b" so adding a new one wouldn't be surprising. Furthermore,
the postfix notation is preferred in the context of "presenting the result
first, then the implementation" (context discussed already in the 2010
pep), the "presenting the result first" is also a goal of the list
comprehension, indeed one does write [x+3 for x in range(5)] and not [for x
in range(5): x+3], the latter would be more "imperative programming" style,
and would be translated to a normal loop.

The problem of chaining without parenthesis, how to remove the parenthesis
in the following statement ?

print((y+2 where y = x+1) where x = 2)

We have two options :

print(y+2 where x = 2 where y = x+1)
print(y+2 where y = x+1 where x = 2)

The first option would be probably closer to the way multiple "for" are
linked in a list comprehension:

[y+2 for x in range(5) for y in [x+1]]

But the second option would be more "present result first" and more close
to the parenthesized version, the user would create new variables as they
go, "I want to compute y+2 but hey, what is y ? It's x+1 ! But what is x ?
It's 5 !)). However, keeping the same order as in "multiple for in list
comprehension" is better, so I'd choose first option.

In the implementation on GitHub of thektulu the parenthesis are mandatory.

Another syntax issue would probably surprise some users, the following
statement, parenthesized :

[(y+2 where y = x+1) for x in range(5)]

Would have two totally legal ways to be done:

[y+2 where y = x+1 for x in range(5)]
[y+2 for x in range(5) where y = x+1]

The first one is a consequence of the "where" keyword usable in an
expression and the second one is a consequence of using it in a list
comprehension.

However I think it doesn't break the "there is only one obvious way to do
it", because depending on 

Re: [Python-ideas] Temporary variables in comprehensions

[Terry Reedy]

On 2/15/2018 8:37 PM, Chris Barker - NOAA Federal wrote:


Back to one of your examples:

[f(x) for x in [x]]

What does that mean???

for x in seq

Means iterate through seq, and assign each item to the name x.

If that seq has x in it — I’m not sure that is even legal python — the 
scope in a comprehension confuses me.


But that is the equivalent is something like:

it= iter(seq)
while True:
     Try:
         x = next(it)
     Except StopIteration:
         Break

(Excuse the caps — hard to write code on a phone)

So not sure how x gets into that sequence before the loop starts.


Reusing a previously bound name as an iteration variable is a bad idea. 
It works in 3.x because the outermost iterable, but only the outermost 
iterable, is pre-calculated before executing the comprehension.  Hence 
'x in [x]' sometimes works, and sometimes not.  ('Outermost' is topmost 
in nested loops, left most in comprehension.)


>>> x = 2
>>> [x*x for x in [x]]
[4]
>>> [x*y for x in [3] for y in [x]]  # here, local x is 3, not 2
[9]
>>> [x*y for y in [x] for x in [3]]
[6]
>>> [x*y for y in [3] for x in [x]]
Traceback (most recent call last):
  File "", line 1, in 
[x*y for y in [3] for x in [x]]
  File "", line 1, in 
[x*y for y in [3] for x in [x]]
UnboundLocalError: local variable 'x' referenced before assignment
>>> [z*y for y in [3] for z in [x]]  # no confusion here
[6]

To put it another way, l = [x for x in [x]] is actually calculated as
_temp = [x]; l = [x for x in _temp].  In general, other iterables cannot 
be precalculated since they might depend on prior iteration variables.



--
Terry Jan Reedy


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Chris Barker - NOAA Federal]

> Setting a new comprehension variable is not likely to be free, and may even be
> more costly than calling f(x) twice if f() is a cheap expression:
>
>[x+1 + some_func(x+1) for x in range(10)]
>
> could be faster than
>
>[y + some_func(y) for x in range(10) let y = x + 1]

A bit of a nit — function call overhead is substantial in python, so
if that is an actual function, rather than a simple expression, it’ll
likely be slower to call it twice for any but trivially small
iterables.


>[(y, y**2) let y = x+1 for x in (1, 2, 3, 4)]

Do we need the let?

[ g(y) for y = f(x) for c in seq]

Or, with expressions:

[y + y**2 for y = x+1 for x in (1,2,3)]

Maybe that would be ambiguous— I haven’t thought carefully about it.

-CHB
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Steven D'Aprano]

Hi fhsxfhsx, and welcome.

My comments below, interleaved with yours.


On Thu, Feb 15, 2018 at 01:56:44PM +0800, fhsxfhsx wrote:

[quoted out of order]
> And I hope the discussion could focus more on whether we should allow 
> assigning temporary variables in comprehensions rather than how to 
> solve the specific example I mentioned above.

Whether or not to allow this proposal will depend on what alternate 
solutions to the problem already exist, so your specific example is very 
relevant. Any proposed change has to compete with existing solutions.


> As far as I can see, a comprehension like
> alist = [f(x) for x in range(10)]
> is better than a for-loop
> for x in range(10):
>   alist.append(f(x))
> because the previous one shows every element of the list explicitly so 
> that we don't need to handle `append` mentally.

While I personally agree with you, many others disagree. I know quite a 
few experienced, competent Python programmers who avoid list 
comprehensions because they consider them harder to read and reason 
about. They consider a regular for-loop better precisely because you do 
see the explicit call to append.

(In my experience, those of us who get functional-programming idioms 
often forget that others find them tricky.)

The point is that list comprehensions are already complex enough that 
they are difficult for many people to learn, and some people never come 
to grips with them. Adding even more features comes with a cost.

The bottom line is that it isn't clear to me that allowing local 
variables inside comprehensions will make them more readable.


> But when it comes to something like
> [f(x) + g(f(x)) for x in range(10)]
> you find you have to sacrifice some readableness if you don't want two 
> f(x) which might slow down your code.

The usual comments about premature optimisation apply here.

Setting a new comprehension variable is not likely to be free, and may even be 
more costly than calling f(x) twice if f() is a cheap expression:

[x+1 + some_func(x+1) for x in range(10)]

could be faster than

[y + some_func(y) for x in range(10) let y = x + 1]

or whatever syntax we come up with.


> Someone may argue that one can write
> [y + g(y) for y in [f(x) for x in range(10)]]

Indeed. This would be the functional-programming solution, and I 
personally think it is an excellent one. The only changes are that I'd 
use a generator expression for the intermediate value, avoiding the need 
to make a full list, and I would lay it out more nicely, using 
whitespace to make the structure more clear:

result = [y + g(y) for y in 
 (f(x) for x in range(10))
 ]


> but it's not as clear as to show what `y` is in a subsequent clause, 
> not to say there'll be another temporary list built in the process.

There's no need to build the temporary list. Use a generator 
comprehension. And I disagree that the value of y isn't as clear.

An alternative is simply to refactor your list comprehension. Move the 
calls to f() and g() into a helper function:

def func(x):
y = f(x)
return y + g(y)

and now you can write the extremely clear comprehension

[func(x) for x in range(10)]

that needs no extra variable.



[...]
> In a word, what I'm arguing is that we need a way to assign temporary 
> variables in a comprehension.

"Need" is very strong. I think that the two alternatives I mention above 
cover 95% of the cases where might use a local variable in a 
comprehension. And of the remaining cases, many of them will be so 
complex that they should be re-written as an explicit for-loop. So in my 
opinion, we're only talking about a "need" to solve the problem for a 
small proportion of cases:

- most comprehensions don't need a local variable (apart from 
  the loop variable) at all;

- of those which do need a local variable, most can be easily 
  solved using a nested comprehension or a helper function;

- of those which cannot be solved that way, most are complicated
  enough that they should use a regular for-loop;

- leaving only a small number of cases which are complicated enough
  to genuinely benefit from local variables but not too complicated.

So this is very much a borderline feature. Occasionally it would be 
"nice to have", but on the negative side:

- it adds complexity to the language;

- makes comprehensions harder to read;

- and people will use it unnecessarily where there is no readability 
  or speed benefit (premature optimization again).

It is not clear to me that we should burden *all* Python programmers 
with additional syntax and complexity of an *already complex* feature 
for such a marginal improvement.


> In my opinion, code like
> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> is more natural than any solution we now have.
> And that's why I pro the new syntax, it's clear, explicit and readable

How can you say that the new syntax is "clear, explicit and readable" 
when you haven't proposed any 

Re: [Python-ideas] Temporary variables in comprehensions

[Chris Barker]

On Thu, Feb 15, 2018 at 2:13 AM, Jamie Willis  wrote:

>  These laws define behaviour that is expected equivalent by users;
>
> [x for x in xs] = xs
>

OK -- that's the definition...


> [f(x) for x in [x]] = f(x)
>

well, not quite:

[f(x) for x in [x]] = [f(x)]

Using x in two places where they mean different things makes this odd, but
yes, again the definition (of a list comp, and a length-1 sequence)

[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]
>

well, no. using two for expressions yields the outer product -- all
combinations:

In [*14*]: xs = range(3)


In [*15*]: [(x,y) *for* x *in* xs *for* y *in* xs]

Out[*15*]: [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1),
(2, 2)]

so the result is a length len(seq1)*len(seq(2)) list. Or in this case, a
len(xs)**2.

But nesting the comps applies one expression, and then the other, yielding
a length len(xs) list.

but you wrote:

[g(y) for x in xs for y in f(x)]

which I'm not sure what you were expecting, as f(x) is not a sequence
(probably)...

To play with your examples:

Define some functions that make it clear what's been applied:

In [*16*]: *def* f(x):

...: *return* "f(*{}*)".format(x)

...:


In [*17*]: *def* g(x):

...: *return* "g(*{}*)".format(x)

and a simple sequence to use:

In [*18*]: xs = range(3)

Now your examples:

In [*19*]: [x *for* x *in* xs]

Out[*19*]: [0, 1, 2]


In [*20*]: [f(x) *for* x *in* [x]]

Out[*20*]: ['f(5)']


In [*21*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*21*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']

OK -- all good f applied, then g, but then the last one:

In [*27*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

Out[*27*]:

['g(f)',

 'g(()',

 'g(0)',

 'g())',

 'g(f)',

 'g(()',

 'g(1)',

 'g())',

 'g(f)',

 'g(()',

 'g(2)',

 'g())']

in this case, f(x) is returning a string, which is a sequence, so you get
that kind odd result. But what if f(x) was a simple scalr function:

In [*29*]: *def* f(x):

...: *return* 2*x

Then you just get an error:

In [*30*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

---

TypeError Traceback (most recent call last)

 in ()

> 1 [g(y) for x in xs for y in f(x)]


 in (.0)

> 1 [g(y) for x in xs for y in f(x)]


TypeError: 'int' object is not iterable

The the nested comp is what is desired here:

In [*31*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*31*]: ['g(0)', 'g(2)', 'g(4)']

Except you probably want a generator expression in the inner loop to avoid
bulding an extra list:


In [*33*]:  [g(y) *for* y *in* (f(x) *for* x *in* xs)]

Out[*33*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']


So back to the OP's example:

In [*34*]: [f(x) + g(f(x)) *for* x *in* range(10)]

Out[*34*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

that is best done with comps as:

In [*36*]: [fx + g(fx) *for* fx *in* (f(x) *for* x *in* range(10))]

Out[*36*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

which really doesn't seem bad to me. And if the function names are longer
-- which they should be, you might want to use a temp as suggested earlier:

In [*41*]: fx = (f(x) *for* x *in* range(10))


In [*42*]: [x + g(x) *for* x *in* fx]

Out[*42*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

The truth is, comprehensions really are a bit wordy, if you are doing a lot
of this kind of thing (at least with numbers), you might be happier with an
array-oriented language or library, such as numpy:

In [*46*]: *import* *numpy* *as* *np*


In [*47*]: *def* f(x):

...: *return* x * 2

...:


In [*48*]: *def* g(x):

...: *return* x * 3

...:

...:


In [*49*]: xs = np.arange(3)


In [*50*]: f(xs) + g(f(xs))

Out[*50*]: array([ 0,  8, 16])


is pretty compact, and can be "optimized with a temp:

In [*51*]: fx = f(xs)

...: fx + g(fx)

...:

Out[*51*]: array([ 0,  8, 16])

pretty simple  isn't it?

So this gets back to -- does anyone have a suggestion for a syntax for
comprehensions that would make this substantially clearer, more readable,
or more compact?

I'm guessing not :-)

(the compact bit comes from having to type the "for x in" part twice -- it
does *feel* a bit unnecessary. which is why I like numpy -- no "for" at all
:-)

(I'm still trying to figure out why folks think map() or filter() help
here...)

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   

Re: [Python-ideas] Temporary variables in comprehensions

[Jamie Willis]

I'm not sure it does indicate a need for refactoring, I'd argue it's quite
a common pattern, at least in functional languages from which this
construct arises.

In fact, in those languages, there are laws that govern interactions with
the comprehensions (though this comes from monads and monads perhaps don't
quite apply to pythons model). These laws define behaviour that is expected
equivalent by users;

[x for x in xs] = xs
[f(x) for x in [x]] = f(x)
[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]

Even though that last law isn't completely analogous to the given example
from the OP, the transformation he wants to be able to do does arise from
the laws. So it could be argued that not being able to flatten the
comprehension down via law 3 is unexpected behaviour and in order to
achieve this you'd need a form of assignment in the comprehension or suffer
inefficiencies.

But that's probably just my functional brain talking...

On 15 Feb 2018 9:53 am, "Evpok Padding"  wrote:

> For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
> I don't really see what the issue is, if you really want to make it
> shorter,
> you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
> case where I like `map` more than comprehensions.
>
> For more complex case, just define a intermediate generator along the lines
> ```
> f_samples = (f(x) for x in range(10))
> [y+g(y) for y in f_samples]
> ```
> Which does exactly the same thing but
>   - Is more readable and explicit
>   - Has no memory overhead thanks to lazy evaluation
> (btw, you should consider generators for your nested comprenshions)
>
> While I am sometimes in the same state of mind, wishing for variables in
> comprehensions seems to me like a good indicator that your code needs
> refactoring.
>
> Best,
>
> E
>
> On 15 February 2018 at 10:32, Jamie Willis 
> wrote:
> >
> > I +1 this at surface level; Both Haskell list comprehensions and Scala
> for comprehensions have variable assignment in them, even between iterating
> and this is often very useful. Perhaps syntax can be generalised as:
> >
> > [expr_using_x_and_y
> >  for i in is
> >   x = expr_using_i
> >  for j in is
> >   y = expr_using_j_and_x]
> >
> > This demonstrates the scope of each assignment; available in main result
> and then every clause that follows it.
> >
> > Sorry to op who will receive twice, forgot reply to all
> >
> > On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
> >>
> >> As far as I can see, a comprehension like
> >> alist = [f(x) for x in range(10)]
> >> is better than a for-loop
> >> for x in range(10):
> >>   alist.append(f(x))
> >> because the previous one shows every element of the list explicitly so
> that we don't need to handle `append` mentally.
> >>
> >> But when it comes to something like
> >> [f(x) + g(f(x)) for x in range(10)]
> >> you find you have to sacrifice some readableness if you don't want two
> f(x) which might slow down your code.
> >>
> >> Someone may argue that one can write
> >> [y + g(y) for y in [f(x) for x in range(10)]]
> >> but it's not as clear as to show what `y` is in a subsequent clause,
> not to say there'll be another temporary list built in the process.
> >> We can even replace every comprehension with map and filter, but that
> would face the same problems.
> >>
> >> In a word, what I'm arguing is that we need a way to assign temporary
> variables in a comprehension.
> >> In my opinion, code like
> >> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> >> is more natural than any solution we now have.
> >> And that's why I pro the new syntax, it's clear, explicit and readable,
> and is nothing beyond the functionality of the present comprehensions so
> it's not complicated.
> >>
> >> And I hope the discussion could focus more on whether we should allow
> assigning temporary variables in comprehensions rather than how to solve
> the specific example I mentioned above.
> >>
> >>
> >>
> >>
> >>
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >>
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Stephan Houben]

Note that you can already do:

 [y + g(y) for x in range(10) for y in [f(x)]]

i.e. for y  in [expr]
does exactly what the OP wants.
No new syntax needed.

If you hang out on python-list , you'll soon notice
that many newbies struggle already with the list comprehension
syntax.
It's a mini-language which is almost, but not entirely,
exactly unlike normal Python code.
Let's not complicate it further.

Stephan


2018-02-15 10:53 GMT+01:00 Evpok Padding :

> For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
> I don't really see what the issue is, if you really want to make it
> shorter,
> you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
> case where I like `map` more than comprehensions.
>
> For more complex case, just define a intermediate generator along the lines
> ```
> f_samples = (f(x) for x in range(10))
> [y+g(y) for y in f_samples]
> ```
> Which does exactly the same thing but
>   - Is more readable and explicit
>   - Has no memory overhead thanks to lazy evaluation
> (btw, you should consider generators for your nested comprenshions)
>
> While I am sometimes in the same state of mind, wishing for variables in
> comprehensions seems to me like a good indicator that your code needs
> refactoring.
>
> Best,
>
> E
>
> On 15 February 2018 at 10:32, Jamie Willis 
> wrote:
> >
> > I +1 this at surface level; Both Haskell list comprehensions and Scala
> for comprehensions have variable assignment in them, even between iterating
> and this is often very useful. Perhaps syntax can be generalised as:
> >
> > [expr_using_x_and_y
> >  for i in is
> >   x = expr_using_i
> >  for j in is
> >   y = expr_using_j_and_x]
> >
> > This demonstrates the scope of each assignment; available in main result
> and then every clause that follows it.
> >
> > Sorry to op who will receive twice, forgot reply to all
> >
> > On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
> >>
> >> As far as I can see, a comprehension like
> >> alist = [f(x) for x in range(10)]
> >> is better than a for-loop
> >> for x in range(10):
> >>   alist.append(f(x))
> >> because the previous one shows every element of the list explicitly so
> that we don't need to handle `append` mentally.
> >>
> >> But when it comes to something like
> >> [f(x) + g(f(x)) for x in range(10)]
> >> you find you have to sacrifice some readableness if you don't want two
> f(x) which might slow down your code.
> >>
> >> Someone may argue that one can write
> >> [y + g(y) for y in [f(x) for x in range(10)]]
> >> but it's not as clear as to show what `y` is in a subsequent clause,
> not to say there'll be another temporary list built in the process.
> >> We can even replace every comprehension with map and filter, but that
> would face the same problems.
> >>
> >> In a word, what I'm arguing is that we need a way to assign temporary
> variables in a comprehension.
> >> In my opinion, code like
> >> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> >> is more natural than any solution we now have.
> >> And that's why I pro the new syntax, it's clear, explicit and readable,
> and is nothing beyond the functionality of the present comprehensions so
> it's not complicated.
> >>
> >> And I hope the discussion could focus more on whether we should allow
> assigning temporary variables in comprehensions rather than how to solve
> the specific example I mentioned above.
> >>
> >>
> >>
> >>
> >>
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >>
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Paul Moore]

On 15 February 2018 at 05:56, fhsxfhsx  wrote:
> As far as I can see, a comprehension like
> alist = [f(x) for x in range(10)]
> is better than a for-loop
> for x in range(10):
>   alist.append(f(x))
> because the previous one shows every element of the list explicitly so that
> we don't need to handle `append` mentally.

... as long as the code inside the comprehension remains relatively
simple. It's easy to abuse comprehensions to the point where they are
less readable than a for loop, but that's true of a lot of things, so
isn't a specific problem with comprehensions.

> But when it comes to something like
> [f(x) + g(f(x)) for x in range(10)]
> you find you have to sacrifice some readableness if you don't want two f(x)
> which might slow down your code.

Agreed. I hit that quite often.

> Someone may argue that one can write
> [y + g(y) for y in [f(x) for x in range(10)]]
> but it's not as clear as to show what `y` is in a subsequent clause, not to
> say there'll be another temporary list built in the process.
> We can even replace every comprehension with map and filter, but that would
> face the same problems.

That is a workaround (and one I'd not thought of before) but I agree
it's ugly, and reduces readability. Actually, factoring out the inner
comprehension like Evpok Padding suggests:

f_samples = (f(x) for x in range(10))
[y+g(y) for y in f_samples]

is very readable and effective, IMO, so it's not *that* obvious that
local names are beneficial.

> In a word, what I'm arguing is that we need a way to assign temporary
> variables in a comprehension.

"We need" is a bit strong here. "It would be useful to have" is
probably true for some situations.

> In my opinion, code like
> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> is more natural than any solution we now have.
> And that's why I pro the new syntax, it's clear, explicit and readable, and
> is nothing beyond the functionality of the present comprehensions so it's
> not complicated.

The problem is that you haven't proposed an actual syntax here, just
that one should be invented. There have been discussions on this in
the past (a quick search found
https://mail.python.org/pipermail/python-ideas/2011-April/009863.html
and https://mail.python.org/pipermail/python-ideas/2012-January/013468.html,
for example).

> And I hope the discussion could focus more on whether we should allow
> assigning temporary variables in comprehensions rather than how to solve the
> specific example I mentioned above.

The problem isn't so much "whether we should allow it" as "can we find
a syntax that is acceptable", and only then "does the new syntax give
sufficient benefit to be worth adding". New syntax has a pretty high
cost, and proposals that don't suggest explicit syntax will get stuck
because you can't judge whether adding the capability is "worth it"
without being clear on what the cost is - particularly when the
benefit is relatively small (which this is).

Agreed that it's important to focus on the general problem, but part
of the discussion *will* include arguing as to why the existing
workarounds and alternatives are less acceptable than new syntax. And
that will need to include discussion of specific cases. Generally, in
that sort of discussion, artificial examples like "y=f(x)" don't fare
well because it's too easy to end up just debating subjective views on
"readability". If you can provide examples from real-world code that
clearly demonstrate the cost in terms of maintainability of the
existing workarounds, that will help your argument a lot. Although
you'll need to be prepared for questions like "would you be willing to
drop support for versions of Python older than 3.8 in order to get
this improvement?" - it's surprisingly hard to justify language (as
opposed to library) changes when you really stop and think about it.
Which is not to say that it can't be done, just that it's easy to
underestimate the effort needed.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Evpok Padding]

For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
I don't really see what the issue is, if you really want to make it shorter,
you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
case where I like `map` more than comprehensions.

For more complex case, just define a intermediate generator along the lines
```
f_samples = (f(x) for x in range(10))
[y+g(y) for y in f_samples]
```
Which does exactly the same thing but
  - Is more readable and explicit
  - Has no memory overhead thanks to lazy evaluation
(btw, you should consider generators for your nested comprenshions)

While I am sometimes in the same state of mind, wishing for variables in
comprehensions seems to me like a good indicator that your code needs
refactoring.

Best,

E

On 15 February 2018 at 10:32, Jamie Willis 
wrote:
>
> I +1 this at surface level; Both Haskell list comprehensions and Scala
for comprehensions have variable assignment in them, even between iterating
and this is often very useful. Perhaps syntax can be generalised as:
>
> [expr_using_x_and_y
>  for i in is
>   x = expr_using_i
>  for j in is
>   y = expr_using_j_and_x]
>
> This demonstrates the scope of each assignment; available in main result
and then every clause that follows it.
>
> Sorry to op who will receive twice, forgot reply to all
>
> On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
>>
>> As far as I can see, a comprehension like
>> alist = [f(x) for x in range(10)]
>> is better than a for-loop
>> for x in range(10):
>>   alist.append(f(x))
>> because the previous one shows every element of the list explicitly so
that we don't need to handle `append` mentally.
>>
>> But when it comes to something like
>> [f(x) + g(f(x)) for x in range(10)]
>> you find you have to sacrifice some readableness if you don't want two
f(x) which might slow down your code.
>>
>> Someone may argue that one can write
>> [y + g(y) for y in [f(x) for x in range(10)]]
>> but it's not as clear as to show what `y` is in a subsequent clause, not
to say there'll be another temporary list built in the process.
>> We can even replace every comprehension with map and filter, but that
would face the same problems.
>>
>> In a word, what I'm arguing is that we need a way to assign temporary
variables in a comprehension.
>> In my opinion, code like
>> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
>> is more natural than any solution we now have.
>> And that's why I pro the new syntax, it's clear, explicit and readable,
and is nothing beyond the functionality of the present comprehensions so
it's not complicated.
>>
>> And I hope the discussion could focus more on whether we should allow
assigning temporary variables in comprehensions rather than how to solve
the specific example I mentioned above.
>>
>>
>>
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Jamie Willis]

I +1 this at surface level; Both Haskell list comprehensions and Scala for
comprehensions have variable assignment in them, even between iterating and
this is often very useful. Perhaps syntax can be generalised as:

[expr_using_x_and_y
 for i in is
  x = expr_using_i
 for j in is
  y = expr_using_j_and_x]

This demonstrates the scope of each assignment; available in main result
and then every clause that follows it.

Sorry to op who will receive twice, forgot reply to all

On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:

> As far as I can see, a comprehension like
> alist = [f(x) for x in range(10)]
> is better than a for-loop
> for x in range(10):
>   alist.append(f(x))
> because the previous one shows every element of the list explicitly so
> that we don't need to handle `append` mentally.
>
> But when it comes to something like
> [f(x) + g(f(x)) for x in range(10)]
> you find you have to sacrifice some readableness if you don't want two
> f(x) which might slow down your code.
>
> Someone may argue that one can write
> [y + g(y) for y in [f(x) for x in range(10)]]
> but it's not as clear as to show what `y` is in a subsequent clause, not
> to say there'll be another temporary list built in the process.
> We can even replace every comprehension with map and filter, but that
> would face the same problems.
>
> In a word, what I'm arguing is that we need a way to assign temporary
> variables in a comprehension.
> In my opinion, code like
> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> is more natural than any solution we now have.
> And that's why I pro the new syntax, it's clear, explicit and readable,
> and is nothing beyond the functionality of the present comprehensions so
> it's not complicated.
>
> And I hope the discussion could focus more on whether we should allow
> assigning temporary variables in comprehensions rather than how to solve
> the specific example I mentioned above.
>
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Temporary variables in comprehensions

[fhsxfhsx]

As far as I can see, a comprehension like
alist = [f(x) for x in range(10)]
is better than a for-loop
for x in range(10):
  alist.append(f(x))
because the previous one shows every element of the list explicitly so that we 
don't need to handle `append` mentally.

But when it comes to something like
[f(x) + g(f(x)) for x in range(10)]
you find you have to sacrifice some readableness if you don't want two f(x) 
which might slow down your code.

Someone may argue that one can write
[y + g(y) for y in [f(x) for x in range(10)]]
but it's not as clear as to show what `y` is in a subsequent clause, not to say 
there'll be another temporary list built in the process.
We can even replace every comprehension with map and filter, but that would 
face the same problems.

In a word, what I'm arguing is that we need a way to assign temporary variables 
in a comprehension.
In my opinion, code like
[y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
is more natural than any solution we now have.
And that's why I pro the new syntax, it's clear, explicit and readable, and is 
nothing beyond the functionality of the present comprehensions so it's not 
complicated.

And I hope the discussion could focus more on whether we should allow assigning 
temporary variables in comprehensions rather than how to solve the specific 
example I mentioned above.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/