Re: [Python-ideas] Fixing class scope brainstorm

[Nick Coghlan]

On 27 March 2018 at 01:11, Guido van Rossum  wrote:
> Honestly that sounds way too complex. In my ideal world, only functions
> defined with 'def' are methods whose __get__ auto-binds the first argument
> (unless @classmethod or @staticmethod is present), and do not get to "see"
> the class scope. This is how you define methods after all. Lambdas used at
> class scope do not get this benefit, and they see the class scope as just
> another closure; ditto for comprehensions and genexprs.
>
> I don't think that the "is the call delayed" idea is the proper way to
> distinguish the two cases here. IMO the key concept is "is it used to define
> a method". I betcha if you see a lambda here it's because there's some
> calculation going on whose result will be stored as a class variable.

Aye, I don't disagree with that. The implicit functions used in the
comprehension & generator expression cases are just potentially
simpler to handle, as we don't care about their API signatures, which
means we can freely pollute their APIs with eager name bindings if we
choose to do so.

Lambda expressions are different, since their public API matters, so
we can't mess with it freely to pass in extra name references.

What we potentially *could* do though is allow implicit additions to
their __closure__ attributes in a way that isn't permitted for regular
function definitions, such that in the following code:

class C:
x = 1
f = staticmethod(lambda: print(x))

we would do the following things differently for lambdas (vs def functions):

1. We'd pass "x" down as a candidate for nonlocal name resolution
while compiling the lambda
2. If "x" did get referenced from a lambda, we'd start allowing
non-optimised code objects to have a list of cell vars (the way
functions already do), and define a new "STORE_CLASSDEREF" opcode (as
the counterpart to LOAD_CLASSDEREF)

What STORE_CLASSDEREF would need to do is write updates both to the
current locals namespace (so the expectations of a class body are met)
*and* to the nominated cell object (so any references from lambda
expressions are updated).

The biggest downside I'd see to this is that for an incredibly long
time, we've pushed the view that "lambda expressions are just regular
functions that don't know their own name". Making it so that lambdas
can close over class attributes breaks that equivalence, and if we
were to use the cell based approach I suggest above, the seams would
be visible in the case where a lambda expression references an
attribute that gets rebound after class creation:

>>> C.f()
1
>>> C.x = 2
>>> C.f() # With a cell based approach, this wouldn't change
1

All of those potential complexities are specific to lambda functions
though - since the implicit functions created for lambda expressions
and generator expressions are called and then immediately thrown away,
they don't need a persistent cell reference to the closed over
variable name.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Nick Coghlan]

On 27 March 2018 at 01:57, Guido van Rossum  wrote:
> On Mon, Mar 26, 2018 at 7:57 AM, Nick Coghlan  wrote:
>> By contrast, the sublocals idea strives to keep the *lifecycle* impact
>> of naming a subexpression as negligible as possible - while a named
>> subexpression might live a little longer than it used to as an
>> anonymous subexpression (or substantially longer in the case of
>> compound statement headers), it still wouldn't survive past the end of
>> the statement where it appeared.
>
>
> But this is not new: if you use a for-loop to initialize some class-level
> structure  you have the same problem. There is also a standard solution
> (just 'del' it).

Right, but that's annoying, too, and adds "Am I polluting a namespace
I care about?" to something that would ideally be a purely statement
local consideration (and currently is for comprehensions and generator
expressions).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Joao S. O. Bueno]

>
> Honestly that sounds way too complex. In my ideal world, only functions
> defined with 'def' are methods whose __get__ auto-binds the first argument
> (unless @classmethod or @staticmethod is present), and do not get to "see"
> the class scope. This is how you define methods after all. Lambdas used at
> class scope do not get this benefit, and they see the class scope as just
> another closure; ditto for comprehensions and genexprs.
>
> I don't think that the "is the call delayed" idea is the proper way to
> distinguish the two cases here. IMO the key concept is "is it used to define
> a method". I betcha if you see a lambda here it's because there's some
> calculation going on whose result will be stored as a class variable.


Nevertheless, treating lambdas differently from "def functions" in this context
alone can be complicated. Even if not the most intuitive thing on writing code,
everywhere else lambda s can't even be distinguished from "regular
functions" once
created. And I have created a lot of "one line methods"  as lambdas
over the years.
(that is a lot of code that would be broke with no reason). Even
calling "type" with a
couple "lambda" expressionsin the namespace parameter is something
very useful, and
might be crippled if lambda's can't be methods at all (although that
is a different context,
I don't think things should be changed at this point)


But there are two different things about lambdas in class bodies:

- enabling a lambda to "see" the outer class scope: that is ok, as it
would just make
things simpler, and won't break existing code, as this ability did not
exist before.

- preventing a lambda to become a method though its `__get__`: this breakes
stuff. As it is not incompatible with a lambda that can "see" the
class scope, I see no reason for doing that
(code might not work if a lambda with cell classes pointing to the class body,
but my feeling is that it would just work correctly if such cells are
permitted to start with)


  js
 -><-
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Joao S. O. Bueno]

On 27 March 2018 at 10:56, Nick Coghlan  wrote:
> ... Making it so that lambdas
> can close over class attributes breaks that equivalence, and if we
> were to use the cell based approach I suggest above, the seams would
> be visible in the case where a lambda expression references an
> attribute that gets rebound after class creation:
>
> >>> C.f()
> 1
> >>> C.x = 2
> >>> C.f() # With a cell based approach, this wouldn't change
> 1
>

Yes - but that would be the intention of the code beign written as in
your example -

class C:
x = 1
f = staticmethod(lambda: print(x))

While, the classic behavior can be attained by doing:

class C:
 x = 1
 f = classmethod(lambda cls: print(cls.x))

And the behavior in both cases if one of no-surprises for me.  For
coders who don't
have the mechanism of class creation in their mind, that could come as
 a surprise, but it is
better than being inconsistent.

TL;DR: +1 for your approach - I am just saying your perceived drawback
is not one IMHO.

> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings

[Ethan Furman]

On 03/25/2018 09:46 AM, Ethan Furman wrote:

On 03/24/2018 09:24 PM, Nick Coghlan wrote:


No, the fact that the expression defining the outermost iterable gets evaluated 
in the outer scope is behaviour that's
explicitly tested for in the regression test suite.

The language reference spells out that this is intentional for generator 
expressions, where it has the added benefit of
reporting errors in the outermost iterable expression at the point where the 
genexp is defined, rather than at the point
where it gets iterated over: 
https://docs.python.org/3/reference/expressions.html#generator-expressions

Independently of the pragmatic "getting them to work sensibly at class scope" 
motivation, comprehensions inherit those
semantics by way of the intended semantic equivalence between "[x for x in sequence]" and 
"list(x for x in sequence)".


Thank you (everyone!) for your patience.

Using the latest example from Angelico and myself:

--> d = 9
... def func(_iter):
... ... body of comprehension
--> func((d as e))

The sticking point is the `func((d as e))`, which to my mind should happen 
inside the comprehension, but needs to happen
outside -- and the reason it has to happen outside is so that the interpretor 
can verify that `d` actually exists;
however, I think we can have both:

--> def func(_iter):
... ...
... e = d
... ...
--> d
--> func()

This way, the assignment does not leak, but the referenced name is still looked 
up and verified to exist outside the
function call.

I don't know how easy/difficult that would be to implement.  At this point a 
simple confirmation that I understand and
that in theory the above would solve both issues is what I'm looking for.  Or, 
of course, the reasons why the above
would not, in theory, work.


Any insights here?  I would rather not say the same things in another thread if we can resolve this question here and be 
done with it.  :)


--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Kyle Lahnakoski]

On 2018-03-23 06:01, Chris Angelico wrote:
> https://www.python.org/dev/peps/pep-0572/
>

A suggestion:

Under the rejected "Special-casing comprehensions", you show
"prefix-local-name-bindings": Name bindings that appear before the loop,
like:

> stuff = [(y, x/y) where y = f(x) for x in range(5)]


Please add mention of rejecting "postfix-local-name-bindings": Name
bindings that happen after the loop.  For example:

> stuff = [(y, x/y) for x in range(5) where y = f(x)]


Since all the same reasoning applies to both prefix and postfix
variations, maybe distinguishing between prefix and postfix can be done
in the last paragraph of "Special-casing comprehensions".

Thank you.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Guido van Rossum]

On Tue, Mar 27, 2018 at 7:00 AM, Nick Coghlan  wrote:

> On 27 March 2018 at 01:57, Guido van Rossum  wrote:
> > On Mon, Mar 26, 2018 at 7:57 AM, Nick Coghlan 
> wrote:
> >> By contrast, the sublocals idea strives to keep the *lifecycle* impact
> >> of naming a subexpression as negligible as possible - while a named
> >> subexpression might live a little longer than it used to as an
> >> anonymous subexpression (or substantially longer in the case of
> >> compound statement headers), it still wouldn't survive past the end of
> >> the statement where it appeared.
> >
> >
> > But this is not new: if you use a for-loop to initialize some class-level
> > structure  you have the same problem. There is also a standard solution
> > (just 'del' it).
>
> Right, but that's annoying, too, and adds "Am I polluting a namespace
> I care about?" to something that would ideally be a purely statement
> local consideration (and currently is for comprehensions and generator
> expressions).
>

The standard reply here is that if you can't tell at a glance whether
that's the case, your code is too complex. The Zen of Python says
"Namespaces are one honking great idea -- let's do more of those!" and in
this case that means refactor into smaller namespaces, i.e.
functions/methods.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Paul Moore]

On 27 March 2018 at 15:32, Joao S. O. Bueno  wrote:
> Yes - but that would be the intention of the code beign written as in
> your example -
>
> class C:
> x = 1
> f = staticmethod(lambda: print(x))
>
> While, the classic behavior can be attained by doing:
>
> class C:
>  x = 1
>  f = classmethod(lambda cls: print(cls.x))
>
> And the behavior in both cases if one of no-surprises for me. For
> coders who don't have the mechanism of class creation in their
> mind, that could come as a surprise, but it is better than being
> inconsistent.

I wouldn't describe myself as "having the mechanism of class creation
in my mind", but I'm not 100% sure that's a necessary criterion here.
In an ideal world, Python's semantics is supposed to be intuitive,
which means that understanding subtle details shouldn't be necessary
to anticipate the behaviour of certain constructs.

Looking at the following definitions:

class C:
x = 1

y1 = x
y2 = (lambda: x)()

def meth1(self):
return self.x
meth2 = lambda self: self.x

@staticmethod
def sm1():
return x
sm2 = staticmethod(lambda: x)

@classmethod
def cm1(cls):
return cls.x
cm2 = classmethod(lambda cls: cls.x)

I would expect meth1 and meth2 to be equivalent. I'd expect cm1 and
cm2 to be equivalent. I would *not* expect sm1 to work, and I'd expect
sm2 to fail for the same reason - namely that sm1 has no access to the
class or an instance of it, so it should not be able to reference x.
And so we should get NameError.

These to me don't imply any sort of "understanding of how classes are
created" - they simply need an understanding of how methods (normal,
static and class) get access to the class/instance. They also require
that you *don't* expect a class statement to create a scope that can
be closed over. I didn't really think about that before I started
analysing this code, but once I did, I realised that I've never
expected that.

So my reaction to a bare nonlocal variable reference in a method
(whether defined in a def statement or as a lambda) would be "wait,
what would that mean? I guess it's the global". I wouldn't even be
looking at the x defined in the class at that point.

The definition of y2 follows that rule as well - even though the fact
that it's not defining a method, but it's "just" a bare lambda,
slightly muddies the water.

The odd one out is y1. I actually can't quite explain the logic that
allows y1 to refer to the value of x, even though it's the most
"natural" case. As I said, I don't think of a class as defining a new
scope, rather I think of it as bundling together a set of statements
that will be run to populate the class (maybe that's "having the
mechanism of class creation in my mind"?). So I guess y1 = x is just
normal statement sequencing.

So I guess I'm saying that I don't really see a problem with the
current behaviour here. Classes *don't* create a scope. So you don't
close over class variables.

For further emphasis of this point of view:

@staticmethod
def sm1():
nonlocal x
return x

gives

nonlocal x
^
SyntaxError: no binding for nonlocal 'x' found

which again I can understand as "there's no outer scope containing x".

Having said all this, I guess you could say that I'm too close to the
current behaviour to see its deficiencies. Maybe that's true. But I
don't actually *use* any of this on a regular basis. I worked out all
of the above based on intuition from how I understood Python's class
model, plus a few small experiments that mostly just confirmed what I
expected.

If adding the ability to refer to a bare x in sm1/sm2 or y2 [1] means
complicating the behaviour to the point where my mental model needs to
involve injecting arguments into nested scopes, I'm a strong -1.If
there's a need to "fix" comprehensions, then I'd much rather that we
do so in a way that doesn't change the current behaviour or execution
model. Specifically, I'd actually much *rather* that these 3 cases
continue giving NameError.

The comprehension cases:

  y3 = [x+i for i in (0,1,2)] #1
  y4 = [1+i for i in (x, x+1)] #2

(#1 fails with NameError, and #2 works) strike me as odd corner cases
that are a bit odd, but are not worth disrupting the normal cases for.
And they should be viewed as oddities in how comprehensions look names
up, and not as peculiarities of class scope.

Sorry - I intended that to be an "it's all pretty simple and obvious"
comment, but it turned into a rather long post. But I do think the
basic idea remains simple.

Paul

[1] The other cases don't need to refer to a bare x, they already have
perfectly good ways of accessing the class variable x.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Joao S. O. Bueno]

Well, there is an idiom to "keep everything as is", and work around
the current limitations:

class A:
   def b():
   x = 1
   d = [i + x for i in range(2)]
   return locals()
   locals().update(b())
   del b

Maybe if we could find a syntactic sugar for this idiom
(with an abuse of the `with` keyword, for example), people could be happy,
with class bodies working by default as they are now, and enabling the
reuse of defined members by using the special syntax -

class A:
   with class:
  x = 1
  d = [i + x for in range(2)]





On 27 March 2018 at 12:27, Paul Moore  wrote:
> On 27 March 2018 at 15:32, Joao S. O. Bueno  wrote:
>> Yes - but that would be the intention of the code beign written as in
>> your example -
>>
>> class C:
>> x = 1
>> f = staticmethod(lambda: print(x))
>>
>> While, the classic behavior can be attained by doing:
>>
>> class C:
>>  x = 1
>>  f = classmethod(lambda cls: print(cls.x))
>>
>> And the behavior in both cases if one of no-surprises for me. For
>> coders who don't have the mechanism of class creation in their
>> mind, that could come as a surprise, but it is better than being
>> inconsistent.
>
> I wouldn't describe myself as "having the mechanism of class creation
> in my mind", but I'm not 100% sure that's a necessary criterion here.
> In an ideal world, Python's semantics is supposed to be intuitive,
> which means that understanding subtle details shouldn't be necessary
> to anticipate the behaviour of certain constructs.
>
> Looking at the following definitions:
>
> class C:
> x = 1
>
> y1 = x
> y2 = (lambda: x)()
>
> def meth1(self):
> return self.x
> meth2 = lambda self: self.x
>
> @staticmethod
> def sm1():
> return x
> sm2 = staticmethod(lambda: x)
>
> @classmethod
> def cm1(cls):
> return cls.x
> cm2 = classmethod(lambda cls: cls.x)
>
> I would expect meth1 and meth2 to be equivalent. I'd expect cm1 and
> cm2 to be equivalent. I would *not* expect sm1 to work, and I'd expect
> sm2 to fail for the same reason - namely that sm1 has no access to the
> class or an instance of it, so it should not be able to reference x.
> And so we should get NameError.
>
> These to me don't imply any sort of "understanding of how classes are
> created" - they simply need an understanding of how methods (normal,
> static and class) get access to the class/instance. They also require
> that you *don't* expect a class statement to create a scope that can
> be closed over. I didn't really think about that before I started
> analysing this code, but once I did, I realised that I've never
> expected that.
>
> So my reaction to a bare nonlocal variable reference in a method
> (whether defined in a def statement or as a lambda) would be "wait,
> what would that mean? I guess it's the global". I wouldn't even be
> looking at the x defined in the class at that point.
>
> The definition of y2 follows that rule as well - even though the fact
> that it's not defining a method, but it's "just" a bare lambda,
> slightly muddies the water.
>
> The odd one out is y1. I actually can't quite explain the logic that
> allows y1 to refer to the value of x, even though it's the most
> "natural" case. As I said, I don't think of a class as defining a new
> scope, rather I think of it as bundling together a set of statements
> that will be run to populate the class (maybe that's "having the
> mechanism of class creation in my mind"?). So I guess y1 = x is just
> normal statement sequencing.
>
> So I guess I'm saying that I don't really see a problem with the
> current behaviour here. Classes *don't* create a scope. So you don't
> close over class variables.
>
> For further emphasis of this point of view:
>
> @staticmethod
> def sm1():
> nonlocal x
> return x
>
> gives
>
> nonlocal x
> ^
> SyntaxError: no binding for nonlocal 'x' found
>
> which again I can understand as "there's no outer scope containing x".
>
> Having said all this, I guess you could say that I'm too close to the
> current behaviour to see its deficiencies. Maybe that's true. But I
> don't actually *use* any of this on a regular basis. I worked out all
> of the above based on intuition from how I understood Python's class
> model, plus a few small experiments that mostly just confirmed what I
> expected.
>
> If adding the ability to refer to a bare x in sm1/sm2 or y2 [1] means
> complicating the behaviour to the point where my mental model needs to
> involve injecting arguments into nested scopes, I'm a strong -1.If
> there's a need to "fix" comprehensions, then I'd much rather that we
> do so in a way that doesn't change the current behaviour or execution
> model. Specifically, I'd actually much *rather* that these 3 cases
> continue giving NameError.
>
> The comprehension cases:
>
>   y3 = [x+i for i in (0,1,2)] #1
>   y4 = 

Re: [Python-ideas] Fixing class scope brainstorm

[Paul Moore]

On 27 March 2018 at 16:51, Joao S. O. Bueno  wrote:
> Well, there is an idiom to "keep everything as is", and work around
> the current limitations:
>
> class A:
>def b():
>x = 1
>d = [i + x for i in range(2)]
>return locals()
>locals().update(b())
>del b
>
> Maybe if we could find a syntactic sugar for this idiom
> (with an abuse of the `with` keyword, for example), people could be happy,
> with class bodies working by default as they are now, and enabling the
> reuse of defined members by using the special syntax -
>
> class A:
>with class:
>   x = 1
>   d = [i + x for in range(2)]

I'd actually like to see some real world use cases to get a feel for
whether this is even worth worrying about. (I'm not saying it isn't,
just that it's hard to get a feel for the importance based on
artificial examples).

BTW, for an alternative workaround that avoids nested scopes altogether:

>>> import operator
>>> from functools import partial
>>> class C:
...   x = 1
...   d = list(map(partial(operator.add, x), range(2)))
...
>>> c = C()
>>> c.d
[1, 2]

No, I'm not suggesting that's clearer - but it does (at least in my
mind) make it more obvious that the problem is with comprehensions,
not with class scopes.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Guido van Rossum]

On Tue, Mar 27, 2018 at 6:56 AM, Nick Coghlan  wrote:

> [...] The implicit functions used in the
> comprehension & generator expression cases are just potentially
> simpler to handle, as we don't care about their API signatures, which
> means we can freely pollute their APIs with eager name bindings if we
> choose to do so. [...]
>

Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
issues for comprehensions and genexprs.

There may still be breakage, when the code defines a global x that is
overridden by a class-level x, and a class-level comprehension references x
assuming it to be the global. So we need to tread carefully even here --
but this case is weird already:

x = 42
class C:
x = [1, 2, 3]
z = [x+y for y in x]  # [43, 44, 45]

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Eric Fahlgren]

On Tue, Mar 27, 2018 at 9:52 AM, Paul Moore  wrote:

> I'd actually like to see some real world use cases to get a feel for
> whether this is even worth worrying about. (I'm not saying it isn't,
> just that it's hard to get a feel for the importance based on
> artificial examples).
>

​The only reason I brought it up was because I ran into this about three
weeks ago porting some old Python 2 to 3, where there was a class level
comprehension that referenced a class variable on the lhs.  I simply
removed it by enumerating the cases by hand, no big deal, but it did take
me a while to figure out why ​that no longer worked.

My specific case looked approximately like this:

class Plugin:
plugin_dir = 'somepath'
plugin_names = [os.join(plugin_dir, name) for name in ('list', 'of',
'names')]
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Ivan Levkivskyi]

On 27 March 2018 at 18:19, Guido van Rossum  wrote:

> On Tue, Mar 27, 2018 at 6:56 AM, Nick Coghlan  wrote:
>
>> [...] The implicit functions used in the
>> comprehension & generator expression cases are just potentially
>> simpler to handle, as we don't care about their API signatures, which
>> means we can freely pollute their APIs with eager name bindings if we
>> choose to do so. [...]
>>
>
> Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
> issues for comprehensions and genexprs.
>

Removing the implicit function scope in comprehensions is something I
wanted for long time.
It would not only "fix" the scoping, but will also fix the yield inside
comprehensions.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Ethan Furman]

On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:

On 27 March 2018 at 18:19, Guido van Rossum wrote:



Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope 
issues for comprehensions and genexprs.


Removing the implicit function scope in comprehensions is something I wanted 
for long time.
It would not only "fix" the scoping, but will also fix the yield inside 
comprehensions.


Can we do it without leaking names?

--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Guido van Rossum]

On Tue, Mar 27, 2018 at 11:43 AM, Ethan Furman  wrote:

> On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:
>
>> On 27 March 2018 at 18:19, Guido van Rossum wrote:
>>
>
> Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
>>> issues for comprehensions and genexprs.
>>>
>>
>> Removing the implicit function scope in comprehensions is something I
>> wanted for long time.
>> It would not only "fix" the scoping, but will also fix the yield inside
>> comprehensions.
>>
>
> Can we do it without leaking names?
>

Assuming you're concerned about leaking names out of the comprehension into
the class scope, that shouldn't be a problem. (The solution actually
involves leaking names *into* the comprehension scope, but I'm not sure
that should be called "leaking". :-)

I do notice that we probably can't easily solve this using the existing
closure machinery ("cells") because if we were to have cells in the class
dict, it would confuse everything else that looks in the class namespace.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Paul Moore]

On 27 March 2018 at 19:43, Ethan Furman  wrote:
> On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:
>>
>> On 27 March 2018 at 18:19, Guido van Rossum wrote:
>
>>> Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
>>> issues for comprehensions and genexprs.
>>
>>
>> Removing the implicit function scope in comprehensions is something I
>> wanted for long time.
>> It would not only "fix" the scoping, but will also fix the yield inside
>> comprehensions.
>
> Can we do it without leaking names?

To me, that would be the ideal. I assume there are significant
technical challenges, though, as otherwise I'd have thought that would
have been the approach taken when Python 3 fixed the name leaking
issue from Python 2.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Ivan Levkivskyi]

On 27 March 2018 at 19:43, Ethan Furman  wrote:

> On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:
>
>> On 27 March 2018 at 18:19, Guido van Rossum wrote:
>>
>
> Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
>>> issues for comprehensions and genexprs.
>>>
>>
>> Removing the implicit function scope in comprehensions is something I
>> wanted for long time.
>> It would not only "fix" the scoping, but will also fix the yield inside
>> comprehensions.
>>
>
> Can we do it without leaking names?
>
>
If you mean this

[i for i in range(5)]

i  # NameError

then yes, this is possible. Serhiy outlined the implementation few moths
ago. The rough idea is to use automatic re-naming.
The only problem with this is that if someone will step into debugger one
will see a name like .0.i instead of i.
But this can be solved in the debuggers.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Ivan Levkivskyi]

On 27 March 2018 at 19:47, Paul Moore  wrote:

> On 27 March 2018 at 19:43, Ethan Furman  wrote:
> > On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:
> >>
> >> On 27 March 2018 at 18:19, Guido van Rossum wrote:
> >
> >>> Hm, so maybe we shouldn't touch lambda, but we can at least fix the
> scope
> >>> issues for comprehensions and genexprs.
> >>
> >>
> >> Removing the implicit function scope in comprehensions is something I
> >> wanted for long time.
> >> It would not only "fix" the scoping, but will also fix the yield inside
> >> comprehensions.
> >
> > Can we do it without leaking names?
>
> To me, that would be the ideal. I assume there are significant
> technical challenges, though, as otherwise I'd have thought that would
> have been the approach taken when Python 3 fixed the name leaking
> issue from Python 2.
>
>
Yes, this will be certainly a big PR, but if we agree to do this, I
volunteer to make the PR.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Guido van Rossum]

On Tue, Mar 27, 2018 at 11:51 AM, Ivan Levkivskyi 
wrote:

> On 27 March 2018 at 19:43, Ethan Furman  wrote:
>
>> On 03/27/2018 11:12 AM, Ivan Levkivskyi wrote:
>>
>>> On 27 March 2018 at 18:19, Guido van Rossum wrote:
>>>
>>
>> Hm, so maybe we shouldn't touch lambda, but we can at least fix the scope
 issues for comprehensions and genexprs.

>>>
>>> Removing the implicit function scope in comprehensions is something I
>>> wanted for long time.
>>> It would not only "fix" the scoping, but will also fix the yield inside
>>> comprehensions.
>>>
>>
>> Can we do it without leaking names?
>>
>>
> If you mean this
>
> [i for i in range(5)]
>
> i  # NameError
>
> then yes, this is possible. Serhiy outlined the implementation few moths
> ago. The rough idea is to use automatic re-naming.
> The only problem with this is that if someone will step into debugger one
> will see a name like .0.i instead of i.
> But this can be solved in the debuggers.
>

Oh, sorry, I misread what you were talking about. You're proposing going
back to the Python 2 shared namespace. I'm not at all excited about that,
and I'm not convinced that adjusting debuggers to hide the name mangling is
effective -- it does nothing about other forms of introspection. Also
depending on how PEP 572 falls there may be assignments in there. Plus
there may be object lifetime consequences.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fixing class scope brainstorm

[Greg Ewing]

Guido van Rossum wrote:
I do notice that we probably can't easily solve this using the existing 
closure machinery ("cells") because if we were to have cells in the 
class dict, it would confuse everything else that looks in the class 
namespace.


Maybe it could be done with a new kind of cell object that
holds a reference to the class dict and an attribute name.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Cammil Taank]

>
> ... From here,
> the most important concern and question is: Is there any other syntax
> or related proposal that ought to be mentioned here?


I am not sure if this is valid, but perhaps this is an alternative syntax
which might be simpler:

``name! expr``

So for example, instead of:

stuff = [[(f(x) as y), x/y] for x in range(5)]

stuff = [[y! f(x), x/y] for x in range(5)]

As far as I can tell there would be no conflicts with the current uses of "!".

One potential source of ambiguity would be in:

x = y! a + b # should y be a or (a + b)?

I think this is solved by requiring the target expression to be
non-greedy. If you want a larger named expression, you can always use
parenthesis. i.e. ``x = y! (z + z)``

I feel brevity and minimised punctuation are important for the
adoption of statement-locals, and personally I feel it reads well. I
also happen to prefer the name preceding the expression, though I
suspect this is quite subjective.

Also, apologies if I have grossly misunderstood something.

Cammil
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Rob Cliffe via Python-ideas]

On 27/03/2018 16:22, Guido van Rossum wrote:
On Tue, Mar 27, 2018 at 7:00 AM, Nick Coghlan > wrote:


On 27 March 2018 at 01:57, Guido van Rossum mailto:gu...@python.org>> wrote:
> On Mon, Mar 26, 2018 at 7:57 AM, Nick Coghlan
mailto:ncogh...@gmail.com>> wrote:
>> By contrast, the sublocals idea strives to keep the *lifecycle* impact
>> of naming a subexpression as negligible as possible - while a named
>> subexpression might live a little longer than it used to as an
>> anonymous subexpression (or substantially longer in the case of
>> compound statement headers), it still wouldn't survive past the
end of
>> the statement where it appeared.
>
>
> But this is not new: if you use a for-loop to initialize some
class-level
> structure  you have the same problem. There is also a standard
solution
> (just 'del' it).

Right, but that's annoying, too, and adds "Am I polluting a namespace
I care about?" to something that would ideally be a purely statement
local consideration (and currently is for comprehensions and generator
expressions).


The standard reply here is that if you can't tell at a glance whether 
that's the case, your code is too complex. The Zen of Python says 
"Namespaces are one honking great idea -- let's do more of those!" and 
in this case that means refactor into smaller namespaces, i.e. 
functions/methods.


This is not always satisfactory.  If your for-loop uses 20 
already-defined-locals, do you want to refactor it into a function with 
20 parameters?

Regards
Rob Cliffe
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Steven D'Aprano]

On Wed, Mar 28, 2018 at 12:08:24AM +0100, Rob Cliffe via Python-ideas wrote:
> 
> On 27/03/2018 16:22, Guido van Rossum wrote:

> >The standard reply here is that if you can't tell at a glance whether 
> >that's the case, your code is too complex. The Zen of Python says 
> >"Namespaces are one honking great idea -- let's do more of those!" and 
> >in this case that means refactor into smaller namespaces, i.e. 
> >functions/methods.
> >
> This is not always satisfactory.  If your for-loop uses 20 
> already-defined-locals, do you want to refactor it into a function with 
> 20 parameters?

The standard reply here is that if your for-loop needs 20 locals, your 
function is horribly over-complex and you may need to rethink your 
design.

And if you don't think "20 locals" is too many, okay, how about 50? 100? 
1000? At some point we'll all agree that the function is too complex.

We don't have an obligation to solve every problem of excess complexity, 
especially when the nominal solution involves adding complexity 
elsewhere.

For 25 years, the solution to complex functions in Python has been to 
refactor or simplify them. That strategy has worked well in practice, 
not withstanding your hypothetical function.

If you genuinely do have a function that is so highly coupled with so 
many locals that it is hard to refactor, then you have my sympathy but 
we have no obligation to add a band-aid for it to the language.

Putting the loop variable in its own scope doesn't do anything about the 
real problem: you have a loop that needs to work with twenty other local 
variables. Any other modification to the loop will run into the same 
problem: you have to check the rest of the function to ensure you're not 
clobbering one of the twenty other variables. Special-casing the loop 
variable seems hardly justified.

If there is a justification for introducing sub-local scoping, then I 
think it needs to be something better than pathologically over-complex 
functions.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Rob Cliffe via Python-ideas]

On 28/03/2018 01:19, Steven D'Aprano wrote:

On Wed, Mar 28, 2018 at 12:08:24AM +0100, Rob Cliffe via Python-ideas wrote:

On 27/03/2018 16:22, Guido van Rossum wrote:

The standard reply here is that if you can't tell at a glance whether
that's the case, your code is too complex. The Zen of Python says
"Namespaces are one honking great idea -- let's do more of those!" and
in this case that means refactor into smaller namespaces, i.e.
functions/methods.


This is not always satisfactory.  If your for-loop uses 20
already-defined-locals, do you want to refactor it into a function with
20 parameters?

The standard reply here is that if your for-loop needs 20 locals, your
function is horribly over-complex and you may need to rethink your
design.

And if you don't think "20 locals" is too many, okay, how about 50? 100?
1000? At some point we'll all agree that the function is too complex.

We don't have an obligation to solve every problem of excess complexity,
especially when the nominal solution involves adding complexity
elsewhere.

For 25 years, the solution to complex functions in Python has been to
refactor or simplify them. That strategy has worked well in practice,
not withstanding your hypothetical function.

If you genuinely do have a function that is so highly coupled with so
many locals that it is hard to refactor, then you have my sympathy but
we have no obligation to add a band-aid for it to the language.
It's a fact of life that some tasks *are* complicated.  I daresay most 
aren't, or don't need to be, but some are.


Putting the loop variable in its own scope doesn't do anything about the
real problem: you have a loop that needs to work with twenty other local
variables. Any other modification to the loop will run into the same
problem: you have to check the rest of the function to ensure you're not
clobbering one of the twenty other variables. Special-casing the loop
variable seems hardly justified.

If there is a justification for introducing sub-local scoping, then I
think it needs to be something better than pathologically over-complex
functions.


But putting the loop variable in its own scope solves one problem: it 
ensures that the variable is confined to that loop, and you don't have 
to worry about whether a variable of the same name occurs elsewhere in 
your function.  In other words it increases local transparency (I'm not 
sure that's the right phrase, but I'm struggling to bring a more 
appropriate one to mind) and hence increases readability.
(I understand your point about being able to inspect the for-loop 
variable after the for-loop has terminated - I've probably done it 
myself - but it's a matter of opinion whether that convenience outweighs 
the cleanliness of confining the for-variable's scope.)

Regards
Rob Cliffe
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Guido van Rossum]

This thread is dead.

On Tue, Mar 27, 2018 at 5:40 PM, Rob Cliffe via Python-ideas <
python-ideas@python.org> wrote:

>
>
> On 28/03/2018 01:19, Steven D'Aprano wrote:
>
>> On Wed, Mar 28, 2018 at 12:08:24AM +0100, Rob Cliffe via Python-ideas
>> wrote:
>>
>>> On 27/03/2018 16:22, Guido van Rossum wrote:
>>>
 The standard reply here is that if you can't tell at a glance whether
 that's the case, your code is too complex. The Zen of Python says
 "Namespaces are one honking great idea -- let's do more of those!" and
 in this case that means refactor into smaller namespaces, i.e.
 functions/methods.

 This is not always satisfactory.  If your for-loop uses 20
>>> already-defined-locals, do you want to refactor it into a function with
>>> 20 parameters?
>>>
>> The standard reply here is that if your for-loop needs 20 locals, your
>> function is horribly over-complex and you may need to rethink your
>> design.
>>
>> And if you don't think "20 locals" is too many, okay, how about 50? 100?
>> 1000? At some point we'll all agree that the function is too complex.
>>
>> We don't have an obligation to solve every problem of excess complexity,
>> especially when the nominal solution involves adding complexity
>> elsewhere.
>>
>> For 25 years, the solution to complex functions in Python has been to
>> refactor or simplify them. That strategy has worked well in practice,
>> not withstanding your hypothetical function.
>>
>> If you genuinely do have a function that is so highly coupled with so
>> many locals that it is hard to refactor, then you have my sympathy but
>> we have no obligation to add a band-aid for it to the language.
>>
> It's a fact of life that some tasks *are* complicated.  I daresay most
> aren't, or don't need to be, but some are.
>
>>
>> Putting the loop variable in its own scope doesn't do anything about the
>> real problem: you have a loop that needs to work with twenty other local
>> variables. Any other modification to the loop will run into the same
>> problem: you have to check the rest of the function to ensure you're not
>> clobbering one of the twenty other variables. Special-casing the loop
>> variable seems hardly justified.
>>
>> If there is a justification for introducing sub-local scoping, then I
>> think it needs to be something better than pathologically over-complex
>> functions.
>>
>>
>> But putting the loop variable in its own scope solves one problem: it
> ensures that the variable is confined to that loop, and you don't have to
> worry about whether a variable of the same name occurs elsewhere in your
> function.  In other words it increases local transparency (I'm not sure
> that's the right phrase, but I'm struggling to bring a more appropriate one
> to mind) and hence increases readability.
> (I understand your point about being able to inspect the for-loop variable
> after the for-loop has terminated - I've probably done it myself - but it's
> a matter of opinion whether that convenience outweighs the cleanliness of
> confining the for-variable's scope.)
> Regards
> Rob Cliffe
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[BrenBarn]

On 03/23/2018 03:01 AM, Chris Angelico wrote:

Apologies for letting this languish; life has an annoying habit of
getting in the way now and then.


	My simple response to all of this is that it's not worth it.  Each new 
example convinces me more and more that in almost every case, sublocal 
assignments DECREASE readability as long as they occur inline.  If the 
statement is very simple, the sublocal assignments make it complex.  If 
it is complex, they do not aid in seeing parallelism between different 
pieces that reuse the same value, because the sublocal assignment itself 
creates an asymmetry.


	The only alternatives that I see as increasing readability are the 
"rejected" alternatives in which the sublocal assignment is moved "out 
of order" so that all references to it look the same and are separated 
from the (single) assignment --- i.e., the variants of the form "x = a+b 
with a='foo', b='bar'".


	(I think someone already mentioned this, but these variants, even if 
rejected, probably shouldn't be placed under the header of 
"special-casing comprehensions".  Extracting the assignment to a 
with-clause makes sense outside of comprehensions too.  It would make 
more sense to label them as "out of order" or "non-inline" or perhaps 
"cleft assignment", by analogy with cleft constructions in natural 
language.)


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/