from:"Eric V. Smith"

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-28 Thread Eric V. Smith


On 3/26/2020 9:10 PM, Ethan Furman wrote:
First off, thank you for being so patient -- trying to champion a PEP 
can be exhausting.


On 03/26/2020 05:22 PM, Dennis Sweeney wrote:

So now if I understand the dilemma up to this point we have:

Benefits of writing ``return self`` in the PEP:
    a - Makes it clear that the optimization of not copying is allowed
    b - Makes it clear that ``self.__class__.__getitem__`` isn't used

Benefits of writing ``return self[:]`` in the PEP:
    c - Makes it clear that returning self is an implementation detail
    d - For subclasses not overriding ``__getitem__`` (the majority 
of cases), makes
   it clear that this method will return a base str like the 
other str methods.


Did I miss anything?


The only thing you missed is that, for me at least, points A, C, and D 
are not at all
clear from the example code.  If I wanted to be explicit about the 
return type being

`str` I would write:

 return str(self)   # subclasses are coerced to str


That does seem like the better solution, including the comment.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/43GEOBNJYN22QLYMG2PGN3KDOSQQHR6E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-25 Thread Eric V. Smith


On 3/25/2020 1:36 PM, Dennis Sweeney wrote:

I'm removing the tuple feature from this PEP. So now, if I understand
correctly, I don't think there's disagreement about behavior, just about
how that behavior should be summarized in Python code.

I think that's right.

Ethan Furman wrote:

It appears that in CPython, self[:] is self is true for base
str
  objects, so I think return self[:] is consistent with (1) the premise
  that returning self is an implementation detail that is neither mandated
  nor forbidden, and (2) the premise that the methods should return base
  str objects even when called on str subclasses.

The Python interpreter in my head sees self[:] and returns a copy.
A
note that says a str is returned would be more useful than trying to
exactly mirror internal details in the Python "roughly equivalent" code.

I think I'm still in the camp that ``return self[:]`` more precisely prescribes
the desired behavior. It would feel strange to me to write ``return self``
and then say "but you don't actually have to return self, and in fact
you shouldn't when working with subclasses". To me, it feels like

 return (the original object unchanged, or a copy of the object,
 depending on implementation details,
 but always make a copy when working with subclasses)

is well-summarized by

return self[:]

especially if followed by the text

 Note that ``self[:]`` might not actually make a copy -- if the affix
 is empty or not found, and if ``type(self) is str``, then these methods
 may, but are not required to, make the optimization of returning ``self``.
 However, when called on instances of subclasses of ``str``, these
 methods should return base ``str`` objects, not ``self``.

...which is a necessary explanation regardless. Granted, ``return self[:]``
isn't perfect if ``__getitem__`` is overridden, but at the cost of three
characters, the Python gains accuracy over both the optional nature of
returning ``self`` in all cases and the impossibility (assuming no dunders
are overridden) of returning self for subclasses. It also dissuades readers
from relying on the behavior of returning self, which we're specifying is
an implementation detail.

Is that text explanation satisfactory?


Yes, that makes sense to me. I haven't had time to review the most 
recent updates, and I'll probably wait until you update it one more time.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VE65KII4ZAQV4BR4RACCIOQ27BWRPNYI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-24 Thread Eric V. Smith


On 3/24/2020 7:21 PM, Dennis Sweeney wrote:

It seems that there is a consensus on the names ``removeprefix`` and 
``removesuffix``. I will update the PEP accordingly. I'll also simplify sample 
Python implementation to primarily reflect *intent* over strict type-checking 
correctness, and I'll adjust the accompanying commentary accordingly.

Lastly, since the issue of multiple prefixes/suffixes is more controversial and 
seems that it would not affect how the single-affix cases would work, I can 
remove that from this PEP and allow someone else with a stronger opinion about 
it to propose and defend a set of semantics in a different PEP. Is there any 
objection to deferring this to a different PEP?


No objection. I think that's a good idea.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LPTDYL6ZX47D26B4TGZWR5K6I5PWX77U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-24 Thread Eric V. Smith


On 3/24/2020 3:30 PM, Steve Dower wrote:

On 24Mar2020 1849, Brett Cannon wrote:

-1 on "cut*" because my brain keeps reading it as "cute".
+1 on "trim*" as it is clear what's going on and no confusion with 
preexisting methods.

+1 on "remove*" for the same reasons as "trim*".

And if no consensus is reached in this thread for a name I would 
assume the SC is going to ultimately decide on the name if the PEP is 
accepted as the burden of being known as "the person who chose 
_those_ method names on str" is more than any one person should have 
bear. ;)


-1 on "cut*" (feels too much like what .partition() does)
-0 on "trim*" (this is the name used in .NET instead of "strip", so I 
foresee new confusion)

+1 on "remove*" (because this is exactly what it does)

I actually prefer "without*" because it seems more descriptive, but I 
don't expect it to get any traction.


So "remove" would get my +1.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KQNVH74JAZBXLD6YNQSHRQ6UEIKTTMVQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-23 Thread Eric V. Smith


On 3/23/2020 12:02 PM, Rhodri James wrote:

On 23/03/2020 14:50, Dan Stromberg wrote:


I tend to be mistrustful of code that tries to guess the best thing 
to do,

when something expected isn't found.

How about:

def cutprefix(self: str, pre: str, raise_on_no_match: bool=False, /) 
-> str:

 if self.startswith(pre):
 return self[len(pre):]
 if raise_on_no_match:
 raise ValueError('prefix not found')
 return self[:]


I'm firmly of the opinion that the functions should either raise or 
not, and should definitely not have a parameter to switch behaviours. 
Probably it should do nothing; if the programmer needs to know that 
the prefix wasn't there, cutprefix() probably wasn't the right thing 
to use anyway.


Agreed, and I think we shouldn't raise. If raising is important, the 
user can write a trivial wrapper that raises if no substitution was 
done. Let's not over-complicate this.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NYJMGSLPW4KFOYUT6ZWL6PSKXCF3EDHM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-22 Thread Eric V. Smith


On 3/22/2020 12:25 PM, Paul Ganssle wrote:


Sorry, I think I accidentally left out a clause here - I meant that 
the rationale for /always returning a 'str'/ (as opposed to returning 
a subclass) is missing, it just says in the PEP:


The only difference between the real implementation and the above is 
that, as with other string methods like replace, the methods will 
raise a TypeError if any of self, pre or suf is not an instace of 
str, and will cast subclasses of str to builtin str objects.


I think the rationale for these differences is not made entirely 
clear, specifically the "and will cast subclasses of str to builtin 
str objects" part.
Agreed. I don't understand the rationale, either. If we stick with it, 
it should definitely be stated. And if we don't, that reason should be 
explained, too.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B4AGI7TJU5HC7VMYEO7VK63LTDMU7Q4M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-22 Thread Eric V. Smith


On 3/22/2020 1:42 AM, Nick Coghlan wrote:

On Sun, 22 Mar 2020 at 15:13, Cameron Simpson  wrote:

On 21Mar2020 12:45, Eric V. Smith  wrote:

On 3/21/2020 12:39 PM, Victor Stinner wrote:

Well, if CPython is modified to implement tagged pointers and supports
storing a short strings (a few latin1 characters) as a pointer, it may
become harder to keep the same behavior for "x is y" where x and y are
strings.

Are you suggesting that it could become impossible to write this
function:

 def myself(o):
 return o

and not be able to rely on "o is myself(o)"? That seems... a pretty
nasty breaking change for the language.

Other way around - because strings are immutable, their identity isn't
supposed to matter, so it's possible that functions that currently
return the exact same object in some cases may in the future start
returning a different object with the same value.

Right now, in CPython, with no tagged pointers, we return the full
existing pointer wherever we can, as that saves us a data copy. With
tagged pointers, the pointer storage effectively *is* the instance, so
you can't really replicate that existing "copy the reference not the
storage" behaviour any more.

That said, it's also possible that identity for tagged pointers would
be value based (similar to the effect of the small integer cache and
string interning), in which case the entire question would become
moot.

Either way, the PEP shouldn't be specifying that a new object *must*
be returned, and it also shouldn't be specifying that the same object
*can't* be returned.


Agreed. I think the PEP should say that a str will be returned (in the 
event of a subclass, assuming that's what we decide), but if the 
argument is exactly a str, that it may or may not return the original 
object.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JHM7T6JZU56PWYRJDG45HMRBXE3CBXMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Eric V. Smith

On 3/21/2020 2:09 PM, Steven D'Aprano wrote:

On Sat, Mar 21, 2020 at 12:15:21PM -0400, Eric V. Smith wrote:

On 3/21/2020 11:20 AM, Ned Batchelder wrote:

Why be so prescriptive? The semantics of these functions should be
about what the resulting string contains. Leave it to implementors to
decide when it is OK to return self or not.

I agree with Ned -- whether the string object is returned unchanged or a
copy is an implementation decision, not a language decision.

[Eric]

The only reason I can think of is to enable the test above: did a
suffix/prefix removal take place? That seems like a useful thing.

We don't make this guarantee about string identity for any other string
method, and CPython's behaviour varies from method to method:

py> s = 'a b c'
py> s is s.strip()
True
py> s is s.lower()
False

and version to version:

py> s is s.replace('a', 'a') # 2.7
False
py> s is s.replace('a', 'a') # 3.5
True

I've never seen anyone relying on this behaviour, and I don't expect
these new methods will change that. Thinking that `is` is another way of
writing `==`, yes, I see that frequently. But relying on object identity
to see whether a new string was created by a method, no.
Agreed. I think the PEP should just write the Python pseudo-code without
copying, and it should mention that whether or not the original string
is returned is an implementation detail. Then I think the actual
documentation should just omit any discussion of it, like the existing
docs for lstrip().

If you want to know whether a prefix/suffix was removed, there's a more
reliable way than identity and a cheaper way than O(N) equality. Just
compare the length of the string before and after. If the lengths are
the same, nothing was removed.

That's a good point. This should probably go in the PEP, and maybe the
documentation.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/SOLOVVZXIGEAFZKX223YEAWLP6G5TSB7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Eric V. Smith

On 3/21/2020 12:50 PM, Victor Stinner wrote:

In that case, the PEP should advice to use .startwith() or .endswith()
explicitly if the caller requires to know if the string is going to be
modified. Example:

modified = False
# O(n) complexity where n=len("prefix:")
if line.startswith("prefix:"):
line = line.cutprefix("prefix: ")
modified = True

It should be more efficient than:

old_line = line
line = line.cutprefix("prefix: ")
modified = (line != old_line) # O(n) complexity where n=len(line)

since the checked prefix is usually way shorter than the whole string.

Agreed (except the string passed to startswith should be the same as the
one used in cutprefix!).

Eric

Victor

Le sam. 21 mars 2020 à 17:45, Eric V. Smith a écrit :

On 3/21/2020 12:39 PM, Victor Stinner wrote:

Well, if CPython is modified to implement tagged pointers and supports
storing a short strings (a few latin1 characters) as a pointer, it may
become harder to keep the same behavior for "x is y" where x and y are
strings.

Good point. And I guess it's still a problem for interned strings, since
even a copy could be the same object:

>>> s = 'for'
>>> s[:] is 'for'
True

So I now agree with Ned, we shouldn't be prescriptive here, and we
should explicitly say in the PEP that there's no way to tell if the
strip/cut/whatever took place, other than comparing via equality, not
identity.

Eric

Victor

Le sam. 21 mars 2020 à 17:23, Eric V. Smith a écrit :

On 3/21/2020 11:20 AM, Ned Batchelder wrote:

On 3/20/20 9:34 PM, Cameron Simpson wrote:

On 20Mar2020 13:57, Eric Fahlgren wrote:

On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney

wrote:

If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix, then
``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix has
been removed. If ``s`` does not have ``pre`` as a prefix, an
unchanged copy of ``s`` is returned. In summary, ``s.cutprefix(pre)``
is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else s``.

The second sentence above unambiguously states that cutprefix
returns 'an
unchanged *copy*', but the example contradicts that and shows that
'self'
may be returned and not a copy. I think it should be reworded to
explicitly allow the optimization of returning self.

My versions of these (plain old functions) return self if unchanged,
and are explicitly documented as doing so.

This has the concrete advantage that one can test for nonremoval if
the suffix with "is", which is very fast, instead of == which may not
be.

So one writes (assuming methods):

prefix = cutsuffix(s, 'abc')
if prefix is s:
... no change
else:
... definitely changed, s != prefix also

I am explicitly in favour of returning self if unchanged.

Why be so prescriptive? The semantics of these functions should be
about what the resulting string contains. Leave it to implementors to
decide when it is OK to return self or not.

The only reason I can think of is to enable the test above: did a
suffix/prefix removal take place? That seems like a useful thing. I
think if we don't specify the behavior one way or the other, people are
going to rely on Cpython's behavior here, consciously or not.

Is there some python implementation that would have a problem with the
"is" test, if we were being this prescriptive? Honest question.

Of course this would open the question of what to do if the suffix is
the empty string. But since "'foo'.startswith('')" is True, maybe we'd
have to return a copy in that case. It would be odd to have
"s.startswith('')" be true, but "s.cutprefix('') is s" also be True. Or,
since there's already talk in the PEP about what happens if the
prefix/suffix is the empty string, and if we adopt the "is" behavior
we'd add more details there. Like "if the result is the same object as
self, it means either the suffix is the empty string, or self didn't
start with the suffix".

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/HYSZSIAZA7QNPR43OCBXPWPAZKYTXB7L/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/APEHOJXRWSC6LU2A7FPFNLO352WMHAGU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Eric V. Smith

On 3/21/2020 12:39 PM, Victor Stinner wrote:

Good point. And I guess it's still a problem for interned strings, since
even a copy could be the same object:

>>> s = 'for'
>>> s[:] is 'for'
True

Eric

Victor

Le sam. 21 mars 2020 à 17:23, Eric V. Smith a écrit :

On 3/21/2020 11:20 AM, Ned Batchelder wrote:

On 3/20/20 9:34 PM, Cameron Simpson wrote:

On 20Mar2020 13:57, Eric Fahlgren wrote:

On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney

wrote:

My versions of these (plain old functions) return self if unchanged,
and are explicitly documented as doing so.

This has the concrete advantage that one can test for nonremoval if
the suffix with "is", which is very fast, instead of == which may not
be.

So one writes (assuming methods):

prefix = cutsuffix(s, 'abc')
if prefix is s:
... no change
else:
... definitely changed, s != prefix also

I am explicitly in favour of returning self if unchanged.

Why be so prescriptive? The semantics of these functions should be
about what the resulting string contains. Leave it to implementors to
decide when it is OK to return self or not.

Is there some python implementation that would have a problem with the
"is" test, if we were being this prescriptive? Honest question.

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/WDDY4Z6B775ZUKBXDJT57Q5REY2ZGSXN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Eric V. Smith


On 3/21/2020 11:20 AM, Ned Batchelder wrote:

On 3/20/20 9:34 PM, Cameron Simpson wrote:

On 20Mar2020 13:57, Eric Fahlgren  wrote:
On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney 


wrote:


If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix, then
``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix has
been removed.  If ``s`` does not have ``pre`` as a prefix, an
unchanged copy of ``s`` is returned.  In summary, ``s.cutprefix(pre)``
is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else s``.



The second sentence above unambiguously states that cutprefix 
returns 'an
unchanged *copy*', but the example contradicts that and shows that 
'self'

may be returned and not a copy.  I think it should be reworded to
explicitly allow the optimization of returning self.


My versions of these (plain old functions) return self if unchanged, 
and are explicitly documented as doing so.


This has the concrete advantage that one can test for nonremoval if 
the suffix with "is", which is very fast, instead of == which may not 
be.


So one writes (assuming methods):

   prefix = cutsuffix(s, 'abc')
   if prefix is s:
   ... no change
   else:
   ... definitely changed, s != prefix also

I am explicitly in favour of returning self if unchanged.


Why be so prescriptive? The semantics of these functions should be 
about what the resulting string contains.  Leave it to implementors to 
decide when it is OK to return self or not.


The only reason I can think of is to enable the test above: did a 
suffix/prefix removal take place? That seems like a useful thing. I 
think if we don't specify the behavior one way or the other, people are 
going to rely on Cpython's behavior here, consciously or not.


Is there some python implementation that would have a problem with the 
"is" test, if we were being this prescriptive? Honest question.


Of course this would open the question of what to do if the suffix is 
the empty string. But since "'foo'.startswith('')" is True, maybe we'd 
have to return a copy in that case. It would be odd to have 
"s.startswith('')" be true, but "s.cutprefix('') is s" also be True. Or, 
since there's already talk in the PEP about what happens if the 
prefix/suffix is the empty string, and if we adopt the "is" behavior 
we'd add more details there. Like "if the result is the same object as 
self, it means either the suffix is the empty string, or self didn't 
start with the suffix".


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HYSZSIAZA7QNPR43OCBXPWPAZKYTXB7L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Looking for a sponsor and feedback on PEP 616: string methods for removing prefixes and suffixes

2020-03-20 Thread Eric V. Smith


I'll sponsor it.

Eric

On 3/20/2020 11:29 AM, Dennis Sweeney wrote:

Hello all! I'm a relatively new contributor looking for a Core Dev sponsor for 
the following PEP:

https://github.com/python/peps/pull/1332

Related:
- Python-Ideas Thread: 
https://mail.python.org/archives/list/python-id...@python.org/thread/RJARZSUKCXRJIP42Z2YBBAEN5XA7KEC3/
- Bug Tracker Issue: https://bugs.python.org/issue39939
- Github PR for implementation: https://github.com/python/cpython/pull/18939

Abstract


This is a proposal to add two new methods, ``cutprefix`` and
``cutsuffix``, to the APIs of Python's various string objects.  In
particular, the methods would be added to Unicode ``str`` objects,
binary ``bytes`` and ``bytearray`` objects, and
``collections.UserString``.

If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix, then
``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix has
been removed.  If ``s`` does not have ``pre`` as a prefix, an
unchanged copy of ``s`` is returned.  In summary, ``s.cutprefix(pre)``
is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else s``.

The behavior of ``cutsuffix`` is analogous: ``s.cutsuffix(suf)`` is
roughly equivalent to
``s[:-len(suf)] if suf and s.endswith(suf) else s``.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XC3D3QGONENQ7PIAUM2SNNEP5BWA6Q4J/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6ZLKLYJV6VTK4ZKM5ZCILSXBJL67L3MS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to respond to repeated bad ideas

2020-03-03 Thread Eric V. Smith


On 3/3/2020 2:18 AM, Serhiy Storchaka wrote:

03.03.20 01:03, Skip Montanaro пише:
Atm we don't have an index of ideas, apart from pep 3099, and I'm 
not sure we can make one (can we?), so I do not see a way to prevent 
this from happening.


Maybe an informational PEP which briefly lists rejected ideas?


There is a risk to accept this PEP as it happened to PEP 572 which was 
initially created with a similar purpose.


And the walrus operator was also mentioned in PEP 3099 as being not in 
scope for Py3K.


But I think as a rule, a new contributor is unlikely to get such an idea 
accepted, so there's not much risk in listing rejected ideas, although 
I'm not sure a PEP is the right place for it. Although it may be as good 
a place as we've got.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FND45AVVE2IPGEBZXIVKAUERLPXCG77S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request to postpone some Python 3.9 incompatible changes to Python 3.10

2020-01-24 Thread Eric V. Smith

On 1/24/2020 9:14 AM, Miro Hrončok wrote:

On 24. 01. 20 14:02, Eric V. Smith wrote:
I think the concern is that with removing so many deprecated
features, we're effectively telling libraries that if they want to
support 3.9, they'll have stop supporting 2.7. And many library
authors aren't willing to do that yet. Will they be willing to in
another year? I can't say.

The concern is not that they don't want to drop 2.7 support, but that
is is a nontrivail task to actaually do and we cannot expect them to
do it within the first couple weeks of 2020. While at the same time,
we want them to support 3.9 since the early development versisons in
order to eb able to detect regressions early in the dev cycle.

Ah. So in 3.8, they kept code that had deprecation warnings so that they
could be compatible with 2.7. They'd like to now drop that code and be
3.9-only compatible, but they don't have enough time to do that because
they couldn't start that work as long as they were supporting 2.7. Do I
have that right?

If so, I'd be okay with postponing the removal of the deprecated code
until 3.10. But I don't think we should postpone it if the driver is so
that libraries can remain 2.7 compatible. That could go on forever. This
postponement would be a one-time thing.

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/VPKPA5JW2G22LB7A4OWESIL6O25GSOIK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request to postpone some Python 3.9 incompatible changes to Python 3.10

2020-01-24 Thread Eric V. Smith


On 1/24/2020 5:50 AM, Ivan Levkivskyi wrote:
On Fri, 24 Jan 2020 at 10:05, Victor Stinner > wrote:



The proposal is to give one year to project maintainers to drop Python
2.7 support, since Python 2.7 end of support just happened a few weeks
ago (2020-01-01).


IMO creating this kind of "gray areas" in support and deprecation 
issues is bad.
What this will create is just more sources for arguing/debates. Once 
deprecation or EoL schedule is set,
it is best to align to it. Discussions about the schedules should 
happen when setting them, not when

the deadline is coming.

Also I am not sure it is really worth it. For example, importing ABCs 
directly from collections was deprecated 8 years ago,

what would 1 extra year change?

I think the concern is that with removing so many deprecated features, 
we're effectively telling libraries that if they want to support 3.9, 
they'll have stop supporting 2.7. And many library authors aren't 
willing to do that yet. Will they be willing to in another year? I can't 
say.


Eric


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DKZBXOYUPJCL3PUXWDJOJUEPG4WDYNF7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Documenting Python's float.str()

2020-01-21 Thread Eric V. Smith

On 1/21/2020 2:02 PM, Serhiy Storchaka wrote:

21.01.20 12:37, Eric V. Smith пише:
Yes (I wrote a lot of that), but '.17g' doesn't mean to always show 
17 digits. See 
https://github.com/python/cpython/blob/master/Python/pystrtod.c#L825 
where the repr (which is format_code =='r') is translated to 
format_code = 'g' and precision = 17.

But I was wrong about them being equivalent: 'g' will drop the 
trailing '.0' if it exists, and repr() will not (via flags = 
Py_DTSF_ADD_DOT_0).

This is not the only difference between '.17g' and repr().

>>> '%.17g' % 1.23456789
'1.23456788'
>>> format(1.23456789, '.17g')
'1.23456788'
>>> repr(1.23456789)
'1.23456789'

Huh. That's interesting. Thanks!

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BJKU6GO65UOGCHQN2BCDTFH3XWXIYRD7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Documenting Python's float.str()

2020-01-21 Thread Eric V. Smith

On 1/21/2020 1:32 PM, Chris Angelico wrote:

On Wed, Jan 22, 2020 at 4:03 AM Eric V. Smith  wrote:

The reason repr adds the '.0' that 'g' does not is to avoid this problem:

  >>> type(eval(repr(17.0))) == type(17.0)
True
  >>> type(eval(format(17.0, '.17g'))) == type(17.0)
False

The OP wasn't asking about eval, though, but about float. If you're
depending on the ability to eval the repr of a float, you also have to
concern yourself with inf and nan, which are not builtin names. But I
believe float(repr(x)) == x for any float x.

None the less, it's why repr adds the '.0' that 'g' does not.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7OMWVNHERBVTS56KD4ENQ4PHYFXXPHFV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Documenting Python's float.str()

2020-01-21 Thread Eric V. Smith

On 1/21/2020 11:52 AM, Steven D'Aprano wrote:

I don't really care whether there's documentation for __str__() or
__repr__() or something else.  I'm just thinking that there should
be some way to guarantee a well defined "useful" float output
formatting.

https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting

https://docs.python.org/3/library/string.html#format-string-syntax

Thanks.  For some reason nobody in #python pointed me to the 'g' format
type.  That resolves my issue.

Unfortunately, because 'g' can strip the trailing ".0" floats
formatted with it no longer satisfy the float->str->float
immutability property.

I don't see why. Any string you get back from %g ought to convert back
to a float without loss of precision, the trailing '.0' should not
affect it. Can you give an example where it does?

It seems to work for me.

 py> x = 94.0
 py> float('%g' % x) == x
 True

The reason repr adds the '.0' that 'g' does not is to avoid this problem:

>>> type(eval(repr(17.0))) == type(17.0)
True
>>> type(eval(format(17.0, '.17g'))) == type(17.0)
False

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HMOBPGLJBPVTHSTDCXVUHKSJHJKC2Z6D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Documenting Python's float.str()

2020-01-21 Thread Eric V. Smith


On 1/21/2020 4:32 AM, Serhiy Storchaka wrote:

21.01.20 10:37, Eric V. Smith пише:
For what it's worth, float's repr internally uses a format of '.17g'. 
So, format(value, '.17g') will be equal to repr(f), where f is any 
float.


It was in Python 2, but since Python 3.1 it returns the shortest 
unambiguous representation, which may be shorter than 17 digits.


https://docs.python.org/3/whatsnew/3.1.html#other-language-changes
https://bugs.python.org/issue1580

Yes (I wrote a lot of that), but '.17g' doesn't mean to always show 17 
digits. See 
https://github.com/python/cpython/blob/master/Python/pystrtod.c#L825 
where the repr (which is format_code =='r') is translated to format_code 
= 'g' and precision = 17.


But I was wrong about them being equivalent: 'g' will drop the trailing 
'.0' if it exists, and repr() will not (via flags = Py_DTSF_ADD_DOT_0).


And I'm almost positive that 2.7 also uses short float repr, but doesn't 
have str == repr for floats. But 2.7 is dead to me (despite that fact 
that I use it extensively for one client!), so I'm not going to bother 
double checking.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FILPSQ3FOJF5IZNL4BEZSKJJS6RNKT4A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Documenting Python's float.str()

2020-01-21 Thread Eric V. Smith

On 1/20/2020 10:59 PM, Karl O. Pinc wrote:

Hello,

There appears to be extremely minimal documentation on how floats are
formatted on output. All I really see is that float.__str__() is
float.__repr__(). So that means that float->str->float does not
result in a different value.

It would be nice if the output format for float was documented, to the
extent this is possible. #python suggested that I propose a patch,
but I see no way to write a documentation patch without having any
clue about what Python promises, whether in the CPython implementation
or as part of a specification for Python.

What are the promises Python makes about the str() of a float? Will
it produce 1.0 today and 1.0e0 or +1.0 tomorrow? When is the result
in exponential notation and when not? Does any of this depend on the
underlying OS or hardware or Python implementation? Etc.
For what it's worth, float's repr internally uses a format of '.17g'.
So, format(value, '.17g') will be equal to repr(f), where f is any float.

I think (but don't exactly recall, it's been a while) that you'll get
different values if sys.float_repr_style is 'short' or not. I don't know
if any current systems don't support 'short'.

I don't know if this is documented. I'm also not sure if this is
considered a CPython implementation detail or not, but I would argue
that it is.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/N2O4V4MUG6VYT4HRFRFOIH6NJHVPA6DQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

2019-12-16 Thread Eric V. Smith

On 12/16/2019 3:05 AM, Kyle Stanley wrote:

Chris Angelico wrote:
> ANY object can be passed to str() in order to get some sort of valid
> printable form. The awkwardness comes from the fact that str()
> performs double duty - it's both "give me a printable form of this
> object" and "decode these bytes into text".

While it does make sense for str() to be able to give some form of 
printable form for any object, I suppose that I just don't consider 
something like this: "b'\\xc3\\xa1'" to be overly useful, at least for 
any practical purposes. Can anyone think of a situation where you 
would want a string representation of a bytes object instead of 
decoding it?

Debugging. I sometimes do things like: print('\n'.join(str(thing) for 
thing in lst)), or various variations on this. This is especially useful 
when maybe something in the list is a bytes object where I was expecting 
a string.

I'm not saying it's the best practice, but calling str() on an object is 
a currently a guaranteed way of making a string out of it, and I don't 
think we can change it.

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5B46FTPOWDWNWDD7UL2HCDGSVPCSUUR3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-03 Thread Eric V. Smith


On 12/3/2019 3:35 PM, Christian Heimes wrote:

On 03/12/2019 21.04, Ethan Furman wrote:

On 12/03/2019 09:31 AM, Guido van Rossum wrote:


I think it’s too soon to worry about this. I don’t see a reason to
harass people who maintain code based that were just recently migrated.

I'm happy to go with this, since my libraries still do the 2/3 straddle.

Do we want to set a date/version where we will drop code that only
exists to make 2/3 projects easier?

How about when the last major Linux distro with Python 2 reaches end of
general support? According to Wikipedia that would be 2023 for Ubuntu
18.04 LTS, 2024 for RHEL 8.0, and 2028 for SUSE SLES 15.


I agree we should wait a long time before removing the 'u' prefix. If 
I'm using a library that uses 'u', but that library is no longer 
maintained, when do we want to break my app, just because I use this 
library? Given that 'u' requires so little code to implement, I don't 
see what's to be gained by removing it. Maybe document it as deprecated 
(and why), but that's about it.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/44QX46DVMO7MDXKNPXXLEIADNUTR7ZMS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pass the Python thread state to internal C functions

2019-11-13 Thread Eric V. Smith


On 11/12/2019 5:03 PM, Victor Stinner wrote:

Hi,

Are you ok to modify internal C functions to pass explicitly tstate?


The last time we discussed this, there was pushback due to performance 
concerns. I don't recall if that was actually measured, or just a vague 
unease.


I've long advocated (mostly to myself, and Larry when he would listen!) 
that we should do this. I agree with Petr that not breaking existing 
APIs is of course critical. A parallel set of APIs is needed. But the 
existing APIs should become thin wrappers, until Python 5000 (aka never) 
when they can go away.


And this not only helps with being explicit, it should help with 
testing. No more depending on some hidden global state.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TFCJUGXONTYQXYDHG2OSWLCFNZBAHUFT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migrate typing in builtin

2019-10-08 Thread Eric V. Smith

Ivan (who you cc'd) is a good start!

Eric

On 10/8/2019 8:35 AM, Philippe Prados wrote:

Glups.

I am not an expert of Pyhton source code. May be after this patch ;-)

I think I should discuss with the authors of the module typing, to 
identify the best strategy.

Who is he ?

Philippe

Le mar. 8 oct. 2019 à 10:19, Ivan Levkivskyi > a écrit :

You will need to rewrite most of things in C.

--
Ivan

On Tue 8 Oct 2019, 08:53 Philippe Prados, mailto:philippe.pra...@gmail.com>> wrote:

Ok,

But _GenericAlias and dependencies are written with Python
(Lib/typing.py), not with C.
So, I must rewrite the _GenericAlias in C or it's possible to
merge the C and Python in builtin and add a direct reference to
_GenericAlias with C, and add the reference in builtin module ?

Philippe

Le lun. 7 oct. 2019 à 22:58, Random832 mailto:random...@fastmail.com>> a écrit :

On Mon, Oct 7, 2019, at 12:02, Philippe Prados wrote:
 > Because this PEP propose to accept, for all classes
 > assert isinstance("", int | str)
 > assert issubclass(int, int | str)
 > and add an operator __or__() for type type.
 > def f(list: List[int | str], param: int | None) -> float
| str:
 >     pass

Oh, sorry, I didn't realize that this also included the |
operator, I thought this was just for isinstance("",
Union[int, str]).

___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/GD7WXPD26VUPMZT6WAATCJJBB42DDYYQ/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4K7WZ3RBGG7K6E6XK65MS44VQYZIKQS2/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UDRLYXGQVAGMF26TYPKTMMQVQKG7GGI7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again

2019-09-29 Thread Eric V. Smith


On 9/29/2019 5:13 PM, Victor Stinner wrote:

It seems simpler to me to pass the structure size rather than the
Python version. It avoids the risk of updating the structure without
update the Python version. I also avoids to have to change the Python
version immediately when PyConfig is modified. 


In Win32, Microsoft does this a lot. For example, 
https://docs.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndclassexa



The main risk of
sizeof(PyConfig) comes if we *remove* a field and add a new field of
the same size: the structure size doesn't change... But in my
experience, we only add new ways to configure Pyhon, we never remove
old ones :-D


I agree this is unlikely.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J2NYQCXYNNTSG7HBXWXZDZH7K7BMLMKT/

[Python-Dev] Re: The Python 2 death march

2019-09-10 Thread Eric V. Smith

I think Ned's concerns are real, and the sunset-python-2 page could use 
some wordsmithing to explain that even though support ends on Jan 1, 
there will be a wrap-up release some time after that, but it won't 
incorporate any changes made or proposed after Jan 1.

Another place where that document could be improved is in clarifying the 
different uses of the word "volunteers". For me, that's the most 
confusing part as I was reading it. As near as I can tell, these are 
different groups, or at least largely disjoint:

 * "We are volunteers who make and take care of the Python programming
   language"
 * "most volunteers will not help fix them"
 * "If you need free help from volunteers"

Unfortunately I'm heading out of town and don't have time to help clean 
this up.

Eric

On 9/10/2019 5:09 PM, Jacqueline Kazil wrote:
*RE: Ned's comments -- *That is the same reaction I had when I read 
through this thread.

*RE: Tal's comment - *I could see this making sense as an explanation.

*RE: Guido's comment*
This makes me think that April 2020 is not a thing. And if Ben is 
supporting solo, then can people email him directly with issues? 😂😂😂

*On a serious note...*
I would like clarification, so we (dev community and PSF) have a 
shared understanding of the direction and are sending the same messaging.

-Jackie
PSF Board of Directors

On Tue, Sep 10, 2019 at 4:20 PM Tal Einat > wrote:

On Tue, Sep 10, 2019 at 10:03 PM Ned Batchelder
mailto:n...@nedbatchelder.com>> wrote:
>
> I'm not looking forward to answering questions from the public
about why
> the PSF is writing dire and specific warnings like "We have
decided that
> January 1, 2020, will be the day that we sunset Python 2," while the
> core devs are planning a release four months after that. It
won't help
> Python's credibility, and may convince some people that they
don't have
> to take the date seriously..

To me it seems pretty clear: On Jan 1st 2020, the 2.7.x branch will no
longer receive fixes for any *new* bugs or security issues, nor other
improvements. I would expect that be the time of the code freeze for
the first release candidate, not the time of the final release. While
we will still fix issues *introduced in 2.7.18 since 2.7.17* before
the final 2.7.18 release, we won't address any other bugs or security
issues and won't backport anything *new* from 3.x. (I may have some
details not exactly correct here, but I hope the gist is correct.)

I'm sure the wording could be improved, but generally this seems
entirely reasonable to me.

- Tal Einat
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/KSU2A5SDGDHCLPKA7BSW2PH5OIZVUOCB/

--
Jacqueline Kazil | @jackiekazil

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WOAYYHGIQG7W4DUXBJHBOXEE44C2THDV/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JJWQYZZTTOWELRGSHYLG5FKKCIMRSI2K/

[Python-Dev] Re: python3 -bb and hash collisions

2019-09-10 Thread Eric V. Smith

On 9/10/2019 2:12 PM, Chris Angelico wrote:

On Wed, Sep 11, 2019 at 12:47 AM Daniel Holth wrote:

On Sat, Jun 22, 2019 at 2:48 AM Serhiy Storchaka wrote:

22.06.19 01:08, Daniel Holth пише:

Thanks. I think I might like an option to disable str(bytes) without
disabling str != bytes. Unless the second operation would also corrupt
output.

Came across this kind of set in the hyper http library which uses a set
to accept certain headers with either str or bytes keys.

Does that library support Python 2? If it is true than you have a
problem, because u'abc' == b'abc' in Python 2 and u'abc' != b'abc' in
Python 3.

If it is Python 3 only, you can just ignore BytesWarning. It was added
purely to help to catch subtle bugs in transition to Python 3. In
future, after Python 2 be out of use, BytesWarning will become deprecated.

I stopped using Python 3 after learning about str(bytes) by finding it in my
corrupted database. Ever since then I've been anxious about changing to the new
language, since it makes it so easy to convert from bytes to unicode by
accident without specifying a valid encoding. So I would like to see a future
where str(bytes) is effectively removed. I started working on a pull request
that adds an API to toggle str(bytes) at runtime with a thread local (instead
of requiring a command line argument), so you could do with no_str_bytes(): if
you were worried about the feature, but got a bit stuck in the internals.

Python has, for as long as I've known it, permitted you to call str()
on literally any object - if there's no other string form, you get its
repr. Breaking this would break all manner of debugging techniques.

Right. Think of print(bytes).

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-14 Thread Eric V. Smith

On 8/14/2019 11:02 AM, Random832 wrote:

On Mon, Aug 12, 2019, at 15:15, Terry Reedy wrote:

Please no more combinations. The presence of both legal and illegal
combinations is already a mild nightmare for processing and testing.
idlelib.colorizer has the following re to detest legal combinations

stringprefix = r"(?i:r|u|f|fr|rf|b|br|rb)?"

More advanced syntax highlighting editors have to handle each string type separately
anyway, because they highlight (valid) backslash-escapes and f-string formatters.
The proposed 'v-string' type would need separate handling even in a simplistic
editor like IDLE, because it's different at the basic level of \" not ending
the string (whereas, for better or worse, all current string types have exactly the
same rules for how to find the end delimiter)

The reason I defined f-strings as I did is so that lexer/parsers
(editors, syntax highlighters, other implementations, etc.) could easily
ignore them, at least as a first pass. They're literally like all other
strings to the lexer. Python's lexer/parser says that a string is:

- some optional letters, making the string prefix
- an opening quote or triple quote
- some optional chars, with \ escaping
- a matching closing quote or triple quote

The parser then validates the string prefix ('f' is okay, 'b' is okay,
'fb' isn't okay, 'x' isn't okay, etc.) It then operates on the contents
of the string, based on what the string prefix tell it to do.

So all an alternate lexer/parser has to do is add 'f' to the valid
string prefixes, and it could then at least skip over f-strings.
Somewhere in my notes I have 3 or 4 examples of projects that did this,
and voila: they "supported" f-strings. Imagine a syntax highlighter that
didn't want to highlight the inside of an f-string.

The proposed v-strings would indeed break this. I'm opposed to them for
this reason, among others.

That all said, I am considering moving f-string parsing into the CPython
parser. That would let you say things like:

f'some text {ord('a')}'

I'm not sure that's a great idea, but I've discussed it with several
alternate implementations, and with authors of several editors, and they
seem okay with it. I'm following Guido's parser experiment with some
interest, to see how it might interact with this proposal. Might they
also be okay with v-strings? Maybe. But it seems like a lot of hassle
for a very minor feature.

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-12 Thread Eric V. Smith


On 8/12/2019 2:52 AM, Greg Ewing wrote:

Eric V. Smith wrote:
I'm not in any way serious about this. I just want people to realize 
how many wacky combinations there would be.


It doesn't matter how many combinations there are, as long as
multiple prefixes combine in the way you would expect, which
they do as far as I can see.


In general I agree, although there's some cognitive overhead to which 
combinations are valid or not. There's no "fu" strings, for example.


But for reading code that doesn't matter, so your point stands.

Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VD4BZYX2UV2GBU22PSZKDQAANQ43EZ54/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-11 Thread Eric V. Smith

On 8/11/2019 4:18 PM, Glenn Linderman wrote:

On 8/11/2019 2:50 AM, Steven D'Aprano wrote:

On Sat, Aug 10, 2019 at 12:10:55PM -0700, Glenn Linderman wrote:

Or invent "really raw" in some spelling, such as rr"c:\directory\"
or e for exact, or x for exact, or "c:\directory\"

And that brings me to the thought that if   \e  wants to become an
escape for escape, that maybe there should be an "extended escape"
prefix... if you want to use more escapes, define   ee"string where \\
can only be used as an escape or escaped character, \e means the ASCII
escape character, and \ followed by a character with no escape
definition would be an error."

Please no.

We already have b-strings, r-strings, u-strings, f-strings, br-strings,
rb-strings, fr-strings, rf-strings, each of which comes in four
varieties (single quote, double quote, triple single quote and triple
double quote). Now you're talking about adding rr-strings, v-strings
(Greg suggested that) and ee-strings, presumably some or all of which
will need b*- and *b- or f*- and *f- varieties too.

Don't forget the upper & lower case varieties :)

And all orders!

>>> _all_string_prefixes()
{'', 'b', 'BR', 'bR', 'B', 'rb', 'F', 'RF', 'rB', 'FR', 'Rf', 'Fr', 
'RB', 'f', 'r', 'rf', 'rF', 'R', 'u', 'fR', 'U', 'Br', 'Rb', 'fr', 'br'}

>>> len(_all_string_prefixes())
25

And if you add just 'bv' and 'fv', it's 41:

{'', 'fr', 'Bv', 'BR', 'F', 'rb', 'Fv', 'VB', 'vb', 'vF', 'br', 'FV', 
'vf', 'FR', 'fV', 'bV', 'Br', 'Vb', 'Rb', 'RF', 'bR', 'r', 'R', 'Vf', 
'fv', 'U', 'RB', 'B', 'rB', 'vB', 'Fr', 'rF', 'fR', 'Rf', 'BV', 'VF', 
'bv', 'b', 'u', 'f', 'rf'}

There would be no need for 'uv' (not needed for backward compatibility) 
or 'rv' (can't be both raw and verbatim).

I'm not in any way serious about this. I just want people to realize how 
many wacky combinations there would be. And heaven forbid we ever add 
some combination of 3 characters. If 'rfv' were actually also valid, you 
get to 89:

{'', 'br', 'vb', 'fR', 'F', 'rFV', 'fRv', 'fV', 'rVF', 'Rfv', 'u', 
'vRf', 'fVR', 'rfV', 'Fvr', 'vrf', 'fVr', 'vB', 'Vb', 'Rvf', 'Fv', 'Fr', 
'FVr', 'B', 'rVf', 'FVR', 'vfr', 'VB', 'VrF', 'BR', 'VRf', 'vfR', 'FR', 
'Br', 'RFV', 'Rf', 'fvR', 'f', 'rb', 'VfR', 'VFR', 'fr', 'vFR', 'VRF', 
'frV', 'bR', 'b', 'FrV', 'r', 'R', 'RVF', 'FV', 'rvF', 'FRV', 'Vrf', 
'rvf', 'FRv', 'Frv', 'vF', 'bV', 'VF', 'fv', 'RF', 'RB', 'rB', 'vRF', 
'RFv', 'RVf', 'Rb', 'Vfr', 'vrF', 'rf', 'Bv', 'vf', 'rF', 'U', 'bv', 
'FvR', 'RfV', 'Vf', 'VFr', 'vFr', 'fvr', 'BV', 'rFv', 'rfv', 'fRV', 
'frv', 'RvF'}

If only we could deprecate upper case prefixes!

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B26LJOLLKKVDSQR6ZUVZKSFCU4WNXYC5/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-11 Thread Eric V. Smith


On 8/10/2019 10:30 PM, Rob Cliffe via Python-Dev wrote:



On 10/08/2019 23:30:18, Greg Ewing wrote:

Rob Cliffe via Python-Dev wrote:


Also, the former is simply more *informative* - it tells the reader 
that baz is expected to be a directory, not a file.


On Windows you can usually tell that from the fact that filenames
almost always have an extension, and directory names almost never
do.

Usually, but not always.  I have not infrequently used files with a 
blank extension.
I can't recall using a directory name with an extension (but I can't 
swear that I never have).


I most commonly see this with bare git repositories .git. And 
I've created directory names with "extensions" for my own use.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PJ4TPHOY6ZIWI5CQ56J3BYWTEBFYNMJU/

[Python-Dev] Re: An f-string issue [Was: Re: Re: What to do about invalid escape sequences]

2019-08-10 Thread Eric V. Smith

n 8/10/2019 7:46 PM, Glenn Linderman wrote:
Because of the "invalid escape sequence" and "raw string" discussion,
when looking at the documentation, I also noticed the following
description for f-strings:

Escape sequences are decoded like in ordinary string literals (except
when a literal is also marked as a raw string). After decoding, the
grammar for the contents of the string is:

followed by lots of stuff, followed by
Backslashes are not allowed in format expressions and will raise an
error:

f"newline: {ord('\n')}" # raises SyntaxError

What I don't understand is how, if f-strings are processed AS
DESCRIBED, how the \n is ever seen by the format expression.
If I recall correctly, the mentioned decoding is happening on the string
literal parts of the f-strings (above, the "newline: " part), not the
expression parts (inside the {}). But it's been a while and I don't
recall all of the details.

The description is that they are first decoded like ordinary strings,
and then parsed for the internal grammar containing {} expressions to
be expanded. If that were true, the \n in the above example would
already be a newline character, and the parsing of the format
expression would not see the backslash. And if it were true, that
would actually be far more useful for this situation.

So given that it is not true, why not? And why go to the extra work of
prohibiting \ in the format expressions?

It's a future-proofing thing. See the discussion at
https://mail.python.org/archives/list/python-dev@python.org/thread/EVXD72IYUN2APF2443OMADKA5WJTOKHD/
It has pointers to other parts of the discussion.

At some point, I'm planning on switching the parsing of f-strings from
the custom parser (see Python/ast.c, FstringParser_ConcatFstring()) to
having the python parser itself parse the f-strings. This will be
similar to PEP 536, which doesn't have much detail, but does describe
some of the motivations.

The PEP 498, of course, has an apparently more accurate description,
that the {} parsing actually happens before the escape processing.
Perhaps this avoids making multiple passes over the string to do the
work, as the literal pieces and format expression pieces have to be
separate in the generated code, but that is just my speculation: I'd
like to know the real reason.

Should the documentation be fixed to make the description more
accurate? If so, I'd be glad to open an issue.

Sure. I'm always in favor of accuracy. The f-string documentation was a
last-minute rush job that could have used a lot more editing, and more
eyes are always welcome.

But it will take a fair amount of research to understand it well enough
to document it in more detail.

The PEP further contains the inaccurate statement:

Like all raw strings in Python, no escape processing is done for raw
f-strings:

not mentioning the actual escape processing that is done for raw
strings, regarding \" and \'.

It should probably just say it uses the same rules as raw strings.

Eric

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Eric V. Smith


On 8/9/2019 2:28 PM, Jonathan Goble wrote:

On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan  wrote:

I find the "Our deprecation warnings were even less visible than
normal" argument for extending the deprecation period compelling.

Outsider's 2 cents from reading this discussion (with no personal
experience with this warning):

I am perplexed at the opinion, seemingly espoused by multiple people
in this thread, that because a major part of the problem is that the
warnings were not visible enough, somehow the proposed solution is
making them not visible enough again? It's too late, in my
understanding, in the 3.8 cycle to add a new feature like a change to
how these warnings are produced (it seems a significant change to the
.pyc structure is needed to emit them at runtime), so this supposed
"solution" is nothing but kicking the can down the road. When 3.9
rolls around, public exposure to the problem of invalid escape
sequences will still be approximately what it is now (because if
nobody saw the warnings in 3.7, they certainly won't see them in 3.8
with this "fix"), so you'll end up with the same complaints about
SyntaxWarning that started this discussion, end up back on
DeprecationWarning for 3.9 (hopefully with support for emitting them
at runtime instead of just compile-time), then have to wait until
3.10/4.0 for SyntaxWarning and eventually the next version to actually
make them errors.


Yes, I think that's the idea: Deprecation warning in 3.9, but more 
visible that what 3.7 has. That is, not just at compile time but at run 
time. What's required to make that happen is an open question.



It seems to me, in my humble but uneducated opinion, that if people
are not seeing the warnings, then continuing to give them warnings
they won't see isn't a solution to anything. Put the warning front and
center. The argument of third-party packages will always be an issue,
even if we wait ten years. So put these warnings front and center now
so package and code maintainers actually see it, and I'll bet the
problematic escape sequences get fixed rather quickly.

What am I missing here?


Hopefully the warnings in 3.9 would be more visible that what we saw in 
3.7, so that library authors can take notice and do something about it 
before 3.10 rolls around.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GGZY7B2WFHVXRQ7NVTHGC2F4L5RJIKDI/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Eric V. Smith


On 8/9/2019 11:41 AM, Tim Peters wrote:

[Guido]

I don't see how this debate can avoid a vote in the Steering Council.


FWIW, I found Nick's last post wholly persuasive:  back off to
SyntaxError for now, and think about adding a more specific exception
later for _all_ cases (not just walrus) in which a scope conflict
isn't allowed (e.g., his example of declaring a function's formal
argument `global` in the function).


Also FWIW: I agree with Nick, except I don't see the point of having an 
exception that likely no one is ever going to catch. What's the scenario 
under which you'd want to catch a ScopeConflictException? If it only 
exists to give more info to a human, just improve the message that goes 
with SyntaxError in that case.


But to answer Guido's question: I think this is a fine issue for the SC 
to vote on.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2TDXXUCOEAZYFPCVGOVUWZSCQ24DNGMZ/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-08 Thread Eric V. Smith

On 8/8/2019 7:28 PM, Barry Warsaw wrote:

On Aug 8, 2019, at 14:58, Steven D'Aprano wrote:

It's not a syntax error. There's nothing wrong with the syntax per-say:
we still have ``target := expression``. There's a problem with the
*semantics* not the syntax.

I’m not sure that distinction is meaningful though. What you wrote is
disallowed, so you have to change your code (i.e. syntax) to avoid the error.

I agree with Barry here. For all practical purposes, it's a syntax
error. And if we had a more powerful parser, maybe it really would be a
syntax error. See Guido's recent PEG parser articles
(https://medium.com/@gvanrossum_83706/peg-parsers-7ed72462f97c) where he
discusses raising SyntaxError after the parser is finished, in a
subsequent pass. Is it really a syntax error if pgen doesn't object to
it? In current CPython, the answer is yes.

But to the original point: I think the real reason to make this just a
SyntaxError, and not a subclass (TargetScopeError) is this: would you
ever catch TargetScopeError? If not, why would it need to be its own
class? It should just be SyntaxError with a descriptive enough text that
the user can figure out what's wrong.

I don't see anyone ever writing:

try:
.. something ..
except TargetScopeError:
.. handle this ..
except SyntaxError:
.. handle this differently ..

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-08 Thread Eric V. Smith

On 8/5/2019 4:30 PM, raymond.hettin...@gmail.com wrote:

Thanks for weighing in. I think this is an important usability discussion.
IMO it is the number one issue affecting the end user experience with this
release. If we could get more people to actively use the beta release, the
issue would stand-out front and center. But if people don't use the beta in
earnest, we won't have confirmation until it is too late.

We really don't have to go this path. Arguably, the implicit conversion of
'\latex' to '\\latex' is a feature that has existed for three decades, and now
we're deciding to turn it off to define existing practices as errors. I don't
think any commercial product manager would allow this to occur without a lot of
end user testing.

As much as I'd love to force this change through [0], it really does
seem like we're forcing it. Especially given Nathaniel's point about the
discoverability problems with compile-time warnings, I think we should
delay a visible warning about this. Possibly in 3.9 we can do something
about making these warnings visible at run time, not just compile time.
I had a similar problems with f-strings (can't recall the details now,
since resolved), and the compile-time-only nature made it difficult to
notice. I realize a run-time warning for this would require a fair bit
of work that might not be worth it.

I think Raymond's point goes beyond this. I think he's proposing that we
never make this change. I'm sympathetic to that, too. But the first step
is to change 3.8's behavior to not make this visible. That is, we should
restore the 3.7 warning behavior.

Eric

[0] And the real reason I'd like this is so we can add \e
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/UADZYIYTPGNRELG477F3SSRB3K7R2J75/

[Python-Dev] Re: Comparing dict.values()

2019-07-26 Thread Eric V. Smith

On 7/26/2019 8:24 AM, Greg Ewing wrote:

Steven D'Aprano wrote:
But the basic semantics hasn't really changed: two (multi)sets of 
values are equal if they have the same individual values, regardless 
of order.

Why regardless of order? Dicts have an ordering nowadays. Why
shouldn't that ordering be reflected in the algorithm for
comparing their values()?

Because it's already the case that order doesn't matter when comparing 
dicts and their keys (and presumably items, but I didn't check):

>>> {1:2,2:3} == {2:3,1:2}
True
>>> list({1:2,2:3}.keys())
[1, 2]
>>> list({2:3,1:2}.keys())
[2, 1]
>>> {2:3,1:2}.keys() == {1:2,2:3}.keys()
True
>>>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JOCJL5GWPUVSTMHZ2PSF52FNZJKBXCEN/

[Python-Dev] Re: Comparing dict.values()

2019-07-25 Thread Eric V. Smith




On 7/25/2019 4:19 PM, Kyle Stanley wrote:

Serhiy Storchaka wrote:

Actually, the == operator cannot return NotImplemented.


Thanks for the clarification. What is the reason for this limitation and is it 
only possible for the `==` operator to return one of `None`, `False`, or 
`True`? It seems like it would be useful for it to be able to return 
`NotImplemented` in situations such as this.


Because no one is testing for it. And just using it in a boolean context 
will return True.



Also, I think that I may have had some misconceptions with regards to the 
relationship between the `__eq__()` method and the `==` operator. I know they 
are not the same, but isn't the result of the `==` operator based on a 
transformation of the result from `__eq__()`?


"Based on", yes. If all of the options return NotImplemented, it falls 
back on identity comparison. I can't find this in the Python 3 docs, but 
it's no doubt somewhere.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LLE35RZQF3WMLJMMIIREUGJNMIVQBN3P/

[Python-Dev] Re: Comparing dict.values()

2019-07-25 Thread Eric V. Smith

7/25/2019 6:00 AM, Eric V. Smith wrote:

On 7/25/2019 4:27 AM, Kyle Stanley wrote:

Serhiy Storchaka wrote:
Is there any precedence of raising an exception in the equality
comparison?

Does 3 == "3" returning False make more sense to you?

Personally, I don't find ``3 == "3"`` to be an equivalent comparison
to ``d0.values() == d1.values()``. Generally, it makes sense when
comparing two items of different types, they are not going to be
equivalent (except in cases such as ``3 == 3.0``, but in that case
they are both subtypes of numeric).

I don't know that an exception would be the best behavior to suit this
situation (or for anything using ``__eq__`` for that matter), but
returning ``False`` seems to be a bit misleading. Instead, I think
that either returning the ``NotImplemented`` constant or ``None``
would provide far more useful information to the user, without the
hindrance of causing an exception. I'm leaning more favorably towards
``NotImplemented`` because it explicitly tells the user "Hey, that
equality comparison isn't implemented".

That makes things worse. Now the comparison is always true in a boolean
context. And presumably you'd want __ne__ to also return NotImplemented,
so then both __eq__ and __ne__ would be true, since bool(NotImplemented)
is True.

I might have to take that back. I hadn't factored in what the == and !=
machinery does, beyond calling __eq__ or __ne__.

[Python-Dev] Re: Comparing dict.values()

2019-07-25 Thread Eric V. Smith

On 7/25/2019 4:27 AM, Kyle Stanley wrote:

Serhiy Storchaka wrote:

Is there any precedence of raising an exception in the equality comparison?
Does 3 == "3" returning False make more sense to you?

Personally, I don't find ``3 == "3"`` to be an equivalent comparison to
``d0.values() == d1.values()``. Generally, it makes sense when comparing two items of
different types, they are not going to be equivalent (except in cases such as ``3 ==
3.0``, but in that case they are both subtypes of numeric).

I don't know that an exception would be the best behavior to suit this situation (or for
anything using ``__eq__`` for that matter), but returning ``False`` seems to be a bit
misleading. Instead, I think that either returning the ``NotImplemented`` constant or
``None`` would provide far more useful information to the user, without the hindrance of
causing an exception. I'm leaning more favorably towards ``NotImplemented`` because it
explicitly tells the user "Hey, that equality comparison isn't implemented".

[Python-Dev] Re: Comparing dict.values()

2019-07-24 Thread Eric V. Smith


On 7/24/2019 1:30 PM, Brett Cannon wrote:

Serhiy Storchaka wrote:


o you propose? What is the use case for it?
If you want to compare dict value views as ordered sequences, it can be
surprised that d1.values() != d2.values() when d1 == d2. It will
be
inconsistent with orderless comparison of keys() and items(). If
you
want to compare them as unordered sequences, the computation complexity
of the operation will be quadratic.
Note also, that while in Python 2 always d.values() == d.values(), it
is possible that d1.keys() != d2.keys() and d1.values() !=
d2.values() when d1 == d2. Python 3 is more consistent.
When I saw this I thought, "it should be like `set(d1.values()) == 
set(d2.values())`", but has been pointed out there's no guarantee that 
all values will be hashable. After that I have no expectation since 
order isn't guaranteed.


I think this is one of those cases where it's superficially surprising 
when you don't think about all the ramifications, but once you 
understand the complexity of the problem then it becomes more clear 
that it isn't straight-forward.


To me a doc update for dict.values() stating that the iterator can't 
be compared and a brief mention as to why would be the best solution 
for this.


(I hope my quoting is correct.)

I agree with Brett: let's just document this and not make any code changes.

If someone really has a use case, which I haven't seen, then they can 
write their own comparison using constraints specific to their data: 
perhaps their values are hashable, for example.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C774AG3OQRG6XCZ4JKVP6UYDX6DELMQR/

[Python-Dev] Re: Replacing 4 underscores with a $ sign, idea for a PEP

2019-07-21 Thread Eric V. Smith

You’ll have to manually re-send it. 

--
Eric V. Smith
(301) 502-0945 cell

> On Jul 21, 2019, at 11:23 AM, ka...@beholdingeye.com wrote:
> 
> Sorry for posting to the wrong list! If this could be moved I'd appreciate it.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/MCD6LDSLNDA4DANEJEDT7FQR4HSCTU36/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QVYBOPDF5DXM5PF6ZDBFYEXYULTG32QN/

Re: [Python-Dev] Python in next Windows 10 update

2019-05-21 Thread Eric V. Smith

That’s great, Steve. Thanks for all of the work (by you and others) on this. 

--
Eric V. Smith
True Blade Systems, Inc
(301) 859-4544

> On May 21, 2019, at 4:30 PM, Steve Dower  wrote:
> 
> Hi all
> 
> Just sharing this here because I think it's important for us to be aware of 
> it - I'm not trying to promote or sell anything here :) (Those who were at 
> the language summit have seen this already.)
> 
> In the next Windows 10 update that starts rolling out today, we (Microsoft) 
> have added "python.exe" and "python3.exe" commands that are installed on PATH 
> *by default* and will open the Microsoft Store at the page where we (Python 
> core team) publish our build.
> 
> This makes it a 1-2 click process to get from a clean machine to having a 
> usable Python install ("python.exe" -> opens Store -> "Get it Free" -> 
> "python.exe" now works!)
> 
> The associated blog post:
> 
> https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/
> 
> Here are answers to a few questions that I assume will come up, at least from 
> this audience that understands the issues better than most:
> 
> * if someone had installed Python and put it on PATH with our installer, this 
> new command *does not* interfere
> * if someone had manually modified their own PATH, they *may* see some 
> interference (but we [Microsoft] decided this was an acceptable risk)
> * the Python 3.7 installed from the store will not auto-update to 3.8, but 
> when 3.8 is released we (Microsoft) will update the redirect to point at it
> * if you pass arguments to the redirect command, it just exits with an error 
> code - you only get the Store page if you run it without arguments
> * once the Store package is installed, the redirect command is replaced (this 
> required a new feature in the OS). If you install with the regular installer 
> and update PATH, or active a venv, it will add it *before* the redirect. So 
> these scenarios should be all good.
> 
> I'm happy to answer other questions here. The long-term contact for this 
> integration is python (at) microsoft.com, which right now will come to me.
> 
> And on a personal note, I'm very excited that we (Microsoft) got the approval 
> to do this. Getting *anything* added to Windows is a big task, so it's a 
> reflection of the popularity and support for Python that's growing within 
> Microsoft that we were able to make this happen. That's due to every 
> contributor, both to the core runtime and the ecosystem. I hope this will 
> only help us improve the availability of Python for users and make it an 
> easier choice for dev tasks in the future.
> 
> Cheers,
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ast changes for "debug" f-strings

2019-05-20 Thread Eric V. Smith

On 5/20/2019 10:32 AM, Batuhan Taskaya wrote:

 > This strictly speaking isn't necessary. I could have added another
Constant node for "x=" and left FormattedValue alone. I didn't for three
reasons: it was expedient; it didn't require a lot of surgery to
f-string parsing, which the extra Constant node would require; and it
allowed the Python/ast_unparse.c code to produce a string that was more
consistent with input string.

Agreed.

 > Does anyone care that f'{x=}' would become f'x={x!r}' if I removed
expr_text from the FormattedValue node?

Yes, when i was implementing f-string debugging support to Berker's 
astor project
the roundtrip tests i wrote is failing because of it adds an extra `!r` 
to end. Then

i realized you added a new field (expr_text) for that.

 > I'm not sure how much we care about all of this, but let me know if you
have a strong feeling about it.

I don't think we should complicate this. The current version is very 
simple and understandable.

I think the salient question is: does the lack of expr_text make 
anything more difficult for anyone? As Serhiy said in the other email, 
lots of things are lost while round-tripping, this would just be another 
one. Anyone really interested in high-fidelity round trips already can't 
use the ast module to unparse from.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ast changes for "debug" f-strings

2019-05-20 Thread Eric V. Smith

I added an optional "expr_text" field to the FormattedValue node, which 
represents the text of the expression in a "debug" f-string expression. 
So in f'{x=}', expr_text would be "x=".


This strictly speaking isn't necessary. I could have added another 
Constant node for "x=" and left FormattedValue alone. I didn't for three 
reasons: it was expedient; it didn't require a lot of surgery to 
f-string parsing, which the extra Constant node would require; and it 
allowed the Python/ast_unparse.c code to produce a string that was more 
consistent with input string.


Now that I have more time, I'm rethinking this decision. I'd rather not 
have the extra churn that modifying the ast node will cause to 
downstream code. Someone (Serhiy?) already pointed this out somewhere, 
but now I can't find the original message.


But the issue of the fidelity of ast_unparse is still there. I think the 
only place this code is used is "from __future__ import annotations", 
but for all I know there's a plan to use it elsewhere.


Does anyone care that f'{x=}' would become f'x={x!r}' if I removed 
expr_text from the FormattedValue node?


This isn't the only place where ast_unparse isn't exact with what it 
produces. It's not designed to give the exact source text back, just 
something equivalent.


But it's not clear to me how equivalent it needs to be. Even if I did 
add the extra Constant node, then you'd have the following discrepancy:


f'value: {x=}' would produce:
[Expr(value=JoinedStr(values=[Constant(value='value: ', kind=None), 
Constant(value='x=', kind=None), FormattedValue(value=Name(id='x', 
ctx=Load()), conversion=114, format_spec=None, expr_text='x=')]))]


Note that there are two consecutive Constant nodes here. So this:

f'value: x={x!r}' would produce the slightly different:
[Expr(value=JoinedStr(values=[Constant(value='value: x=', kind=None), 
FormattedValue(value=Name(id='x', ctx=Load()), conversion=114, 
format_spec=None)]))]


Note that there is a single Constant node, with the combined string in it.

I'm not sure how much we care about all of this, but let me know if you 
have a strong feeling about it.


My suggestion for the 3.8b1 release is to add the extra Constant node 
and remove expr_text from the FormattedValue node. This would at least 
mean that there's no change to the ast definitions due to this feature.


If we decide that the two consecutive Constant nodes generated by 
f'value: {x=}' need to be combined, then we can do that post 3.8b1. But 
it's a lot more work to combine the nodes while the f-string is being 
compiled.


Let me know your thoughts, since time is of the essence.

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Redoing failed PR checks

2019-05-08 Thread Eric V. Smith

I think you can close and reopen the PR. That’s what I’m trying on my blocked 
PR. 

--
Eric V. Smith
True Blade Systems, Inc
(301) 859-4544

> On May 8, 2019, at 10:07 AM, Mark Shannon  wrote:
> 
> Hi,
> 
> How do I redo a failed PR check?
> The appveyor failure for https://github.com/python/cpython/pull/13181 appears 
> to be spurious, but there is no obvious way to redo it.
> 
> BTW, this is not the first time I've seen a PR blocked by a spurious appveyor 
> failure.
> 
> Cheers,
> Mark.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Easier debugging with f-strings

2019-05-07 Thread Eric V. Smith


On 5/7/19 8:39 PM, Steven D'Aprano wrote:

Disclaimer: this topic seems to have been split over at least two issues
on the bug tracker, a Python-Ideas thread from 2018, Discourse (I
think...) and who knows what else. I haven't read it all, so excuse me
if I'm raising something already discussed.


Yeah, it's all over the place. At this stage, I think discussing it here 
is best.




On Mon, May 06, 2019 at 08:39:41PM -0400, Eric V. Smith wrote:


After that lightning talk, Larry and I talked about it some more, and
for a number of reasons decided that it would make more sense if the
syntax used an = sign. So we came up with f"{foo=}", which would also
produce "foo='Hello'".

The reasons for the change are:
- Having '=' in the expression is a better mnemonic than !d.
- By not using a conversion starting with !, we can compose = with the
existing ! conversions, !r, !s, and the rarely used !a.
- We can let the user have a little more control of the resulting string.


You're going to hate me for bike-shedding, but I really don't like using
= as a defacto unary postfix operator. To me, spam= is always going to
look like it was meant to be spam== and the second half got
accidentally deleted.

I don't have a better suggestion, sorry.


I think = makes a great mnemonic for what's happening. I realize it 
looks a little funny at first, but just wait until you start confusing 
it with := (hah!). Everyone else I've discussed it with likes =.



In an earlier draft, back when this was spelled !d, you specifically
talked about whitespace. Does this still apply?


Yes, it does.


spam = 42
f'{spam=}'  # returns 'spam=42'
f'{spam =}'  # returns 'spam =42'
f'{spam = }'  # returns 'spam = 42' I guess?


All correct.


f'{spam+1=}'  # returns 'spam+1=41' I guess?


I'm not proposing to redefine addition by one, so this is actually 
'spam+1=43' :)


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Easier debugging with f-strings

2019-05-07 Thread Eric V. Smith

Hi, Tim. 

Your name came up frequently as a target for this use case. I think we’ve come 
up with a pretty good solution. 

Now if I could only convince my clients to upgrade from 2.7! I’ve heard about 
these f-string things, and I think I would enjoy using them. 

Eric

> On May 7, 2019, at 12:50 PM, Tim Peters  wrote:
> 
> [Larry Hastings ]
>> Guido just stopped by--we're all at the PyCon 2019 dev sprints--and we had
>> a chat about it.  Guido likes it but wanted us to restore a little of the 
>> magical
>> behavior we had in "!d": now, = in f-strings will default to repr (!r), 
>> unless
>> you specify a format spec. If you specify a format spec it will always 
>> default
>> to format.  And naturally if you specify an explicit conversion function (!r 
>> !s !a)
>> it will use that.
>> 
>> This makes !f irrelevant, so we're removing it.
>> 
>> Here's the thinking: 99% of the time the user will just use {foo=}, and for 
>> that
>> you want repr.  After that, 0.99% of the time the user will want a format 
>> spec
>> that applies directly the value.  It's exceedingly unlikely that someone will
>> want a format spec, but want it to apply to repr(value) and not value 
>> itself.  So
>> that's possible (f'{foo=!r:20}').  But the default behavior is the most 
>> common
>> case at every step.
> 
> +1.  Perfect!  I'm one of the 0.99% who will frequently use this to
> display floats, and really wants them to show as "0.99%" rather than
> "0.9913499340289%".
> 
> BTW, that Guido person has made enough decent contributions by now
> that I think he should be asked whether he wants to become a core dev!
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Easier debugging with f-strings

2019-05-07 Thread Eric V. Smith


On 5/7/19 9:49 AM, Barry Warsaw wrote:

On May 7, 2019, at 03:52, Steve Holden  wrote:


What's not to like?


My only complaint is that you steadfastly refuse use Guido’s time machine keys 
to make this available in 3.7.


Open a feature request here: https://github.com/asottile/future-fstrings

!

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Easier debugging with f-strings

2019-05-06 Thread Eric V. Smith

Last fall Larry Hastings made a suggestion for adding a way to make 
so-called "print-based debugging" easier with f-strings. Basically the 
approach is that f-strings would be able to produce the text of the 
expression and the value of that expression, without repeating the 
expression in the f-sting. No more writing f'foo={foo}, bar={bar}'. foo 
and bar should each only be in there once each!


At PyCon US 2019 I did a lightning talk about this, suggesting the 
syntax of !d, so that if foo="Hello", then f"{foo!d}" would produce 
"foo='Hello'". That is, it's the text of the expression, followed by an 
equal sign, followed by the repr of the expression. I have implemented 
this and a PR exists. Arbitrary expressions are allowed. I heard from 
core devs and end users after this talk, and all were positive.


After that lightning talk, Larry and I talked about it some more, and 
for a number of reasons decided that it would make more sense if the 
syntax used an = sign. So we came up with f"{foo=}", which would also 
produce "foo='Hello'".


The reasons for the change are:
- Having '=' in the expression is a better mnemonic than !d.
- By not using a conversion starting with !, we can compose = with the 
existing ! conversions, !r, !s, and the rarely used !a.

- We can let the user have a little more control of the resulting string.

Another thing I like about this approach over !d is that the patch is 
simpler, because there are fewer special cases. And because there are 
fewer special cases in the code, I think the proposal is easier to 
explain than !d (in particular how it interacts (or doesn't!) with 
format specifiers).


There's a good rationale here, along with a PR: 
https://bugs.python.org/issue36817.


My plan is to commit this change before 3.8b1. If anyone would like to 
discuss it at PyCon, I'll be around until about 10:30 am on Tuesday. 
I'll be in the CPython sprint room, and I'll be watching bpo, too.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode identifiers in test files?

2019-05-04 Thread Eric V. Smith


On 5/4/19 3:54 AM, Eric V. Smith wrote:

On 5/4/19 2:48 AM, Serhiy Storchaka wrote:

04.05.19 05:46, Eric V. Smith пише:

Is there a policy against using Unicode identifiers in test files?

As part of adding !d to f-strings, there's a code path that's only
executed if the text of the expression is non-ascii. The easiest way
to exercise it, and the way I found a bug, is by using an identifier
with Unicode chars. I know we have a policy against this in Lib/, but
what about Lib/test/?

I could work around this with exec-ing some strings, but that seems
like added confusion that I'd avoid with a real Unicode identifier.


Could you use string literals with non-ascii characters? They are more
used in tests.


Hi, Serhiy.

I could and will, yes: thanks for the suggestion. But for this specific
feature, I also want to test with simple variable-only expressions. I'm
either going to use a unicode identifier in the code, or eval one in a
string. Doing the eval dance just seems like extra work for some sort of
purity that I don't think is important in a test, unless someone can
think of a good reason for it.


And I just noticed that PEP 3131 has an exception for tests in its 
section that says the stdlib can't contain unicode identifiers:


https://www.python.org/dev/peps/pep-3131/#policy-specification

So, since it's the most direct and probably safest thing to do for a 
test, I'm going to use a unicode identier.


Thanks, all, for your ideas. Especially Greg for reminding me to add the 
coding comment.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode identifiers in test files?

2019-05-04 Thread Eric V. Smith


On 5/4/19 2:48 AM, Serhiy Storchaka wrote:

04.05.19 05:46, Eric V. Smith пише:

Is there a policy against using Unicode identifiers in test files?

As part of adding !d to f-strings, there's a code path that's only
executed if the text of the expression is non-ascii. The easiest way
to exercise it, and the way I found a bug, is by using an identifier
with Unicode chars. I know we have a policy against this in Lib/, but
what about Lib/test/?

I could work around this with exec-ing some strings, but that seems
like added confusion that I'd avoid with a real Unicode identifier.


Could you use string literals with non-ascii characters? They are more
used in tests.


Hi, Serhiy.

I could and will, yes: thanks for the suggestion. But for this specific 
feature, I also want to test with simple variable-only expressions. I'm 
either going to use a unicode identifier in the code, or eval one in a 
string. Doing the eval dance just seems like extra work for some sort of 
purity that I don't think is important in a test, unless someone can 
think of a good reason for it.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Unicode identifiers in test files?

2019-05-03 Thread Eric V. Smith


Is there a policy against using Unicode identifiers in test files?

As part of adding !d to f-strings, there's a code path that's only 
executed if the text of the expression is non-ascii. The easiest way to 
exercise it, and the way I found a bug, is by using an identifier with 
Unicode chars. I know we have a policy against this in Lib/, but what 
about Lib/test/?


I could work around this with exec-ing some strings, but that seems like 
added confusion that I'd avoid with a real Unicode identifier.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] configparser: should optionxform be idempotent?

2019-03-08 Thread Eric V. Smith


> On Mar 8, 2019, at 12:12 AM, Steven D'Aprano  wrote:
> 
>> On Fri, Mar 08, 2019 at 12:56:13PM +1300, Greg Ewing wrote:
>> 
>> In any case, the word is easy enough to avoid in this case.
> 
> I don't think we should avoid using standard terminology even if we can. 
> Knowledge of standard terminology is useful, and we don't generally make 
> a practice of talking about (e.g.) "simultaneously running subtasks" 
> when we can say "threads" instead.
> 
> You are happy to use the jargon terms "function" and "canonical form" 
> without explanation, which I think proves that one person's jargon is 
> another's obvious, clear, precise technical terminology.
> 
> 
>> We could say something like:
>> 
>>   "The optionxform function transforms option names to a
>>   canonical form. If the name is already in canonical form,
>>   it should be returned unchanged."
> 
> How about:
> 
>"The optionxform function transforms option names to a
>canonical form. This should be an idempotent function: 
>if the name is already in canonical form, it should be 
>returned unchanged."

I’d prefer something less passive than “it should remain unchanged” (as my high 
school English teacher would say: “by whom?”). Something like “If optionxform 
is called on a name that is already in canonical form, then it should return 
that name unchanged”.  Then add something like “That is, optionxform should be 
idempotent”.

Eric

> 
> 
> requires six extra words, but it uses the correct technical term which 
> will be familiar to some proportion of users, while also explaining the 
> term for those who aren't familiar with it. We all win!
> 
> 
> -- 
> Steven
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add more SyntaxWarnings?

2019-01-24 Thread Eric V. Smith


On 1/24/2019 6:16 PM, Eric V. Smith wrote:

On 1/24/2019 5:52 PM, Chris Angelico wrote:

+1. If it's something that the peephole optimizer is already allowed
to change (eg "1"+"1" is constant-folded) and there is absolutely no
way that it can ever be changed at run time, then raising at compile
time can't hurt [1]. It'd be as implementation-dependent and
version-dependent as the peephole optimizer itself.


It would be a change if the code is never called. I'm not sure we care 
about code that's never called, but it is a change.


Which Chris already pointed out, in some text that I inadvertently 
deleted. I didn't catch his meaning, which is my fault.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add more SyntaxWarnings?

2019-01-24 Thread Eric V. Smith


On 1/24/2019 5:52 PM, Chris Angelico wrote:

On Fri, Jan 25, 2019 at 9:42 AM Steven D'Aprano  wrote:

We could say that implementations are allowed to raise errors at compile
time instead of run time, but aren't required to. Then it becomes a
matter of "quality of implementation".

For literal ints, strings, None, etc we can tell at compile time that an
error will occur. All of these cannot fail to raise (short of changing
the interpreter, in which case you're not using Python anymore):

 1 + "1"  # raises TypeError
 None[x]  # TypeError
 1.234(x)  # TypeError
 "spam".idnex("a")  # AttributeError

In these specific cases, there is nothing wrong with the *syntax*, but a
compiler should be permitted to immediately raise the same exception
that would otherwise occur at run time. This is a form of peephole
optimization, I guess.


+1. If it's something that the peephole optimizer is already allowed
to change (eg "1"+"1" is constant-folded) and there is absolutely no
way that it can ever be changed at run time, then raising at compile
time can't hurt [1]. It'd be as implementation-dependent and
version-dependent as the peephole optimizer itself.


It would be a change if the code is never called. I'm not sure we care 
about code that's never called, but it is a change.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C API changes

2018-11-26 Thread Eric V. Smith


On 11/26/2018 7:08 PM, Larry Hastings wrote:

On 11/23/18 5:15 AM, Armin Rigo wrote:

Also FWIW, my own 2 cents on the topic of changing the C API: let's
entirely drop ``PyObject *`` and instead use more opaque
handles---like a ``PyHandle`` that is defined as a pointer-sized C
type but is not actually directly a pointer.  The main difference this
would make is that the user of the API cannot dereference anything
from the opaque handle, nor directly compare handles with each other
to learn about object identity.  They would work exactly like Windows
handles or POSIX file descriptors.


Why would this be better than simply returning the pointer? Sure, it 
prevents ever dereferencing the pointer and messing with the object, it 
is true.  So naughty people would be prevented from messing with the 
object directly instead of using the API as they should.  But my 
understanding is that the implementation would be slightly 
slower--there'd be all that looking up objects based on handles, and 
managing the handle namespace too.  I'm not convinced the nice-to-have 
of "you can't dereference the pointer anymore" is worth this runtime 
overhead.


Or maybe you have something pretty cheap in mind, e.g. "handle = pointer 
^ 49"?  Or even "handle = pointer ^ (random odd number picked at 
startup)" to punish the extra-naughty?


I thought the important part of the proposal was to have multiple 
PyHandles that point to the same PyObject (you couldn't "directly 
compare handles with each other to learn about object identity"). But 
I'll admit I'm not sure why this would be a win. Then of course they 
couldn't be regular pointers.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] bpo-34203: FAQ now recommends python 3.x over 2.x (GH-9796)

2018-10-12 Thread Eric V. Smith


> On Oct 12, 2018, at 7:03 PM, Martin Panter  wrote:
> 
>> On 12/10/2018, Eric V. Smith  wrote:
>>> On 10/12/2018 5:17 AM, Tal Einat wrote:
>>> 
>>>  The latest stable releases can always be found on the `Python download page
>>> -<https://www.python.org/downloads/>`_.  There are two recommended 
>>> production-ready
>>> -versions at this point in time, because at the moment there are two 
>>> branches of
>>> -stable releases: 2.x and 3.x.  Python 3.x may be less useful than 2.x, 
>>> since
>>> -currently there is more third party software available for Python 2 than 
>>> for
>>> -Python 3.  Python 2 code will generally not run unchanged in Python 3.
>>> +<https://www.python.org/downloads/>`_.  There are two production-ready 
>>> version
>>> +of Python: 2.x and 3.x, but the recommended one at this times is Python 
>>> 3.x.
>> 
>> This should be "time", not "times". I'd fix it, but I'm unsure if this
>> is being backported or not, and I don't want to mess up any merges
>> before they're done.
> 
> Or just remove “at this time[s]”; it doesn’t add much.
> 
> Also, “two . . . version” should be changed back to plural: “two . . .
> versions”.

See https://github.com/python/cpython/pull/9821 for some updates. 

Eric 

> 
>> I do think this should backported to 3.7, 3.6, and 2.7.
>> 
>>> +Although Python 2.x is still widely used, `it will not be
>>> +maintained after January 1, 2020
>>> <https://www.python.org/dev/peps/pep-0373/>`_.
>>> +Python 2.x was known for having more third-party libraries available,
>>> however,
>>> +by the time of this writing, most of the widely used libraries support
>>> Python 3.x,
>> 
>> Should probably be "at the time of this writing".
>> 
>>> +and some are even dropping the Python 2.x support.
>> 
>> And this would read better as "are even dropping support for Python 2.x".
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] bpo-34203: FAQ now recommends python 3.x over 2.x (GH-9796)

2018-10-12 Thread Eric V. Smith


On 10/12/2018 5:17 AM, Tal Einat wrote:


  The latest stable releases can always be found on the `Python download page
-`_.  There are two recommended 
production-ready
-versions at this point in time, because at the moment there are two branches of
-stable releases: 2.x and 3.x.  Python 3.x may be less useful than 2.x, since
-currently there is more third party software available for Python 2 than for
-Python 3.  Python 2 code will generally not run unchanged in Python 3.
+`_.  There are two production-ready version
+of Python: 2.x and 3.x, but the recommended one at this times is Python 3.x.


This should be "time", not "times". I'd fix it, but I'm unsure if this 
is being backported or not, and I don't want to mess up any merges 
before they're done.


I do think this should backported to 3.7, 3.6, and 2.7.


+Although Python 2.x is still widely used, `it will not be
+maintained after January 1, 2020 `_.
+Python 2.x was known for having more third-party libraries available, however,
+by the time of this writing, most of the widely used libraries support Python 
3.x,


Should probably be "at the time of this writing".


+and some are even dropping the Python 2.x support.


And this would read better as "are even dropping support for Python 2.x".

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Postponed annotations break inspection of dataclasses

2018-09-27 Thread Eric V. Smith

Yes, it’s https://bugs.python.org/issue34776

--
Eric

> On Sep 27, 2018, at 12:05 PM, Ivan Levkivskyi  wrote:
> 
> Do we have a b.p.o. issue about this? If no, then I would recommend to open 
> one, so that we will not loose track of this.
> 
> --
> Ivan
> 
> 
> 
>> On Sat, 22 Sep 2018 at 16:32, David Hagen  wrote:
>> The new postponed annotations have an unexpected interaction with 
>> dataclasses. Namely, you cannot get the type hints of any of the data 
>> classes methods.
>> 
>> For example, I have some code that inspects the type parameters of a class's 
>> `__init__` method. (The real use case is to provide a default serializer for 
>> the class, but that is not important here.)  
>> 
>> ```
>> from dataclasses import dataclass
>> from typing import get_type_hints
>> 
>> class Foo:
>> pass
>> 
>> @dataclass
>> class Bar:
>> foo: Foo
>> 
>> print(get_type_hints(Bar.__init__))
>> ```
>> 
>> In Python 3.6 and 3.7, this does what is expected; it prints `{'foo': > '__main__.Foo'>, 'return': }`.
>> 
>> However, if in Python 3.7, I add `from __future__ import annotations`, then 
>> this fails with an error:
>> 
>> ```
>> NameError: name 'Foo' is not defined
>> ```
>> 
>> I know why this is happening. The `__init__` method is defined in the 
>> `dataclasses` module which does not have the `Foo` object in its 
>> environment, and the `Foo` annotation is being passed to `dataclass` and 
>> attached to `__init__` as the string `"Foo"` rather than as the original 
>> object `Foo`, but `get_type_hints` for the new annotations only does a name 
>> lookup in the module where `__init__` is defined not where the annotation is 
>> defined.
>> 
>> I know that the use of lambdas to implement PEP 563 was rejected for 
>> performance reasons. I could be wrong, but I think this was motivated by 
>> variable annotations because the lambda would have to be constructed each 
>> time the function body ran. I was wondering if I could motivate storing the 
>> annotations as lambdas in class bodies and function signatures, in which the 
>> environment is already being captured and is code that usually only runs 
>> once.
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/levkivskyi%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Postponed annotations break inspection of dataclasses

2018-09-22 Thread Eric V. Smith


On 9/22/2018 12:41 PM, Guido van Rossum wrote:
This is a good catch -- thanks for bringing it up. I'm adding Eric Smith 
(author of dataclasses) and Ivan Levkivskyi (co-author of typing) as 
well as Łukasz Langa (author of PEP 563) to the thread to see if they 
have further insights.


I don't see Ivan and Łukasz cc'd, so I'm adding them here.

Personally I don't think it's feasible to change PEP 563 to use lambdas 
(if it were even advisable, which would be a long discussion), but I do 
think we might be able to make small improvements to the dataclasses 
and/or typing modules to make sure your use case works.


Probably a bugs.python.org  issue is a better 
place to dive into the details than python-dev.


Agreed that opening a bug would be good.

And then I'll ruin that suggestion by answering here, too:

I think this problem is endemic to get_type_hints(). I've never 
understood how you're supposed to use the globals and locals arguments 
to it, but this works:


print(get_type_hints(Bar.__init__, globals()))

as does:

print(get_type_hints(Bar.__init__, Bar.__module__))

But that seems like you'd have to know a lot about how a class were 
declared in order to call get_type_hints on it. I'm not sure __module__ 
is always correct (but again, I haven't really thought about it).


The docs for get_type_hints() says: "In addition, forward references 
encoded as string literals are handled by evaluating them in globals and 
locals namespaces."


Every once in a while someone will bring up the idea of delayed 
evaluation, and the answer is always "use a lambda". If we ever wanted 
to do something more with delayed evaluation, this is a good use case 
for it.


Eric



Thanks again,

--Guido (top-poster in chief)

On Sat, Sep 22, 2018 at 8:32 AM David Hagen > wrote:


The new postponed annotations have an unexpected interaction with
dataclasses. Namely, you cannot get the type hints of any of the
data classes methods.

For example, I have some code that inspects the type parameters of a
class's `__init__` method. (The real use case is to provide a
default serializer for the class, but that is not important here.)

```
from dataclasses import dataclass
from typing import get_type_hints

class Foo:
     pass

@dataclass
class Bar:
     foo: Foo

print(get_type_hints(Bar.__init__))
```

In Python 3.6 and 3.7, this does what is expected; it prints
`{'foo': , 'return': }`.

However, if in Python 3.7, I add `from __future__ import
annotations`, then this fails with an error:

```
NameError: name 'Foo' is not defined
```

I know why this is happening. The `__init__` method is defined in
the `dataclasses` module which does not have the `Foo` object in its
environment, and the `Foo` annotation is being passed to `dataclass`
and attached to `__init__` as the string `"Foo"` rather than as the
original object `Foo`, but `get_type_hints` for the new annotations
only does a name lookup in the module where `__init__` is defined
not where the annotation is defined.

I know that the use of lambdas to implement PEP 563 was rejected for
performance reasons. I could be wrong, but I think this was
motivated by variable annotations because the lambda would have to
be constructed each time the function body ran. I was wondering if I
could motivate storing the annotations as lambdas in class bodies
and function signatures, in which the environment is already being
captured and is code that usually only runs once.
___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido )


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-19 Thread Eric V. Smith


On 9/19/2018 9:25 PM, Barry Warsaw wrote:

On Sep 19, 2018, at 20:34, Gregory P. Smith  wrote:


There's ongoing work to rewrite zipimport.c in python using zipfile itself


Great timing!  Serhiy’s rewrite of zipimport in Python has just landed in 3.8, 
although it doesn’t use zipfile.  What’s in git now is a pretty straightforward 
translation from the original C, so it could use some clean ups (and I think 
Serhiy is planning that).  So the problem you describe should be easier to fix 
now in 3.8.  It would be interesting to see if we can squeeze more performance 
and better behavior out of it now that it’s in Python.


You don't hear "better performance" and "now that it's in Python" 
together very often! Although I agree with your point: it's like how we 
tried and failed to make progress on namespace packages when import was 
written in C, and then once it was in Python it was easy to add the 
functionality.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-13 Thread Eric V. Smith


On 9/13/2018 8:04 PM, Victor Stinner wrote:


Le ven. 14 sept. 2018 à 00:09, Eric V. Smith  a écrit :

f'{type(obj)}' becomes type(obj).__format__(''), so you can return
something other than __str__ or __repr__ does. It's only by convention
that an object's __format__ returns __str__: it need not do so.

What's New in Python 3.7 contains:


object.__format__(x, '') is now equivalent to str(x) rather than 
format(str(self), '').
(Contributed by Serhiy Storchaka in bpo-28974.)

https://bugs.python.org/issue28974

Oh, I didn't know that a type is free to change this behavior: return
something different than str(obj) if the format spec is an empty
string.


True! That issue was specific to object.__format__, not any other 
classes implementation of __format__.



So are you suggesting to change type(obj).__format__('') to return the
fully qualified name instead of repr(type)?


I'm not suggesting it, I'm saying it's possible. It indeed might be the 
most useful behavior.



So "%s" % type(obj) would use repr(), but "{}".format(type(obj)) and
f"{type(obj)}" would return the fully qualified name?



"%s" % type(obj) would use str(), not repr.

You could either:
- keep with convention and have type(obj).__format__('') return 
type(obj).__str__(), while type(obj).__format__('#') (or what other char you 
want to use) return the qualname;
or
- just have type(obj).__format__('') return the qualname, if that's the more 
useful behavior.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-13 Thread Eric V. Smith


On 9/13/2018 5:52 PM, Petr Viktorin wrote:

On 09/13/18 14:08, Victor Stinner wrote:
Le jeu. 13 sept. 2018 à 16:01, Eric V. Smith  a 
écrit :

* Add !t conversion to format string


I'm strongly opposed to this. This !t conversion would not be widely
applicable enough to be generally useful, and would need to be exposed
in the f-string and str.format() documentation, even though 99% of
programmers would never need or see it.


(I'm thinking aloud.)

In the Python code base, I found 115 lines using type(obj).__name__
and 228 lines using obj.__class__.__name__.

[...]

"!t" is not a big improvement over ":T" and "type(obj)".


I'm not sure if type(obj) or obj.__class__ should be used, but I can
say that they are different: obj.__class__ can be overriden:

[...]



Moreover, it's also possible to override the "type" symbol in the
global or local scope:

[...]

I don't think either of those are problematic. If you override 
`__class__` or `type`, things will behave weirdly, and that's OK.



One advantage of having a builtin formatter would be to always use
internally the builtin type() function to get the type of an object,
or not use "type()" in the current scope. The second advantage is to
prevent the need of having to decide between type(obj) and
obj.__class__ :-)



raise TypeError(f"must be str, not {obj!t}")

Should be written as:
raise TypeError(f"must be str, not {type(obj)}")

[...]


Do you want to modify str(type) to return a value different than 
repr(type)?


Or maybe it's just a typo and you wanted to write f"{type(obj):T}"?


Yes, AFAIK that was a typo.


f'{type(obj)}' becomes type(obj).__format__(''), so you can return 
something other than __str__ or __repr__ does. It's only by convention 
that an object's __format__ returns __str__: it need not do so.


Eric




I think "T" is a good idea, but I think you're adding in obj vs
type(obj) just because of the borrowed reference issue in Py_TYPE().
That issue is so much larger than string formatting the type of an
object that it shouldn't be addressed here.


Right, that's a side effect of the discussion on the C API. It seems
like Py_TYPE() has to go in the new C API. Sorry, the rationale is not
written down yet, but Dino convinced me that Py_TYPE() has to go :-)


I'll be happy when we get rid of Py_TYPE and get to use moving garbage 
collectors... but now is not the time.
The API for "%T" should be "give me the type". The best way to do that 
might change in the future.



But at this point, we're bikeshedding. I think all the relevant voices 
have been heard.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-13 Thread Eric V. Smith


On 9/12/2018 8:33 PM, Victor Stinner wrote:


Hi,

For the type name, sometimes, we only get a type (not an instance),
and we want to format its FQN. IMHO we need to provide ways to format
the FQN of a type for *types* and for *instances*. Here is my
proposal:

* Add !t conversion to format string


I'm strongly opposed to this. This !t conversion would not be widely 
applicable enough to be generally useful, and would need to be exposed 
in the f-string and str.format() documentation, even though 99% of 
programmers would never need or see it. The purpose of the conversions 
is not to save you from making a function call when you know the type of 
the arguments. The purpose was specifically to convert arguments to 
strings so that your format specifier could always use the string 
formatting mini-language. It was more useful in str.format(), where the 
format string might be written separately (as user input or a 
translation, say) and not know the types of the arguments. You can (and 
I have!) argued that the conversions are completely unneeded in f-strings.


raise TypeError(f"must be str, not {obj!t}")

Should be written as:
raise TypeError(f"must be str, not {type(obj)}")



* Add ":T" format to type.__format__()
As you know (I've read the patch) this is just "T". I mention it here 
for future readers. They should understand that the ":" is a 
str.format() and f-string construct, and is unknown to __format__().


That said, I think this is a good idea. type.__format__() could also 
understand "#"  to specify qualname.



* Add "%t" and "%T" formatters to PyUnicode_FromUnicodeV()
I think "T" is a good idea, but I think you're adding in obj vs 
type(obj) just because of the borrowed reference issue in Py_TYPE(). 
That issue is so much larger than string formatting the type of an 
object that it shouldn't be addressed here.



* Add a read-only type.__fqn__ property

I'm not sure of the purpose of this. When in your examples is it used?

# Python: "!t" for instance
raise TypeError(f"must be str, not {obj!t}")

/* C: "%t" for instance */
PyErr_Format(PyExc_TypeError, "must be str, not %t", obj);


/* C: "%T" for type */
PyErr_Format(PyExc_TypeError, "must be str, not %T", mytype);

# Python: ":T" for type
raise TypeError(f"must be str, not {mytype!T}")


Open question: Should we also add "%t" and "%T" formatters to the str
% args operator at the Python level?


No. Again, I think any formatting of type names should not be in a 
widely used interface, and should remain in our type-specific interface, 
__format__. %-formatting has no per-type extensibility, and I don't 
think we should start adding codes for every possible use case. Format 
codes for datetimes would be way more useful that %t, and I'd be opposed 
to adding them, too. (I realize my analogy is stretched, because every 
object has a type. But still.)


Eric



I have a proof-of-concept implementation:
https://github.com/python/cpython/pull/9251

Victor

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Schedule of the Python 3.7.1 release?

2018-09-05 Thread Eric V. Smith


On 9/5/2018 6:22 AM, Christian Heimes wrote:

On 2018-09-05 11:56, Victor Stinner wrote:

Hi,

Someone asked somewhere (oops, I forgot where!) when is Python 3.7.1
scheduled. I wanted to reply when I saw that it was scheduled for 2
months ago:

"3.7.1: 2018-07-xx"
https://www.python.org/dev/peps/pep-0537/#maintenance-releases

Is there any blocker for 3.7.1? I fixed dozens of bugs in the 3.7
branch since 3.7.0 has been released. I'm ashamed of most of them,
since I introduced regressions in 3.7. (If you insist, I can name them
;-))

Note: the latest Python activity of Ned Deily, our 3.7 release
manager, was an email sent to python-committers mi-July.


Hi,

can we do a release after the core dev sprints, please? I have like to
discuss and land some TLS 1.3 / OpenSSL 1.1.1 related changes and
improvements first.


Agreed. I have at least one dataclasses bug I'm planning on addressing, 
and I'd like to get some feedback on it at the sprints.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 EOL date

2018-08-23 Thread Eric V. Smith


On 8/23/2018 8:14 PM, Barry Warsaw wrote:

On Aug 23, 2018, at 15:23, Eric V. Smith  wrote:


On 8/23/2018 4:30 PM, Victor Stinner wrote:

Hi,
The reference is the PEP 373 "Python 2.7 Release Schedule". See the
"Update" section:
https://www.python.org/dev/peps/pep-0373/#update


We could probably make it more clear in this section and/or in 
https://www.python.org/dev/peps/pep-0373/#id4 that there will be no 2.7 
releases of any kind after 2020-01-01: no bugfix, no security, and no source 
releases after that date.

I'll whip something up if there's general agreement.


+1 from me, but IIRC, Benjamin has said that the final 2.7 release may not 
happen *exactly* on January 1st (due to holidays and such), but that we should 
still consider that the “hammer date” when all support stops.  We should get 
some confirmation from Benjamin (the 2.7 release manager) on the (tentative) 
exact final release date, and then codify that in the PEP.  I’d be very happy 
if that, or say December 31 2019 were the actual last release date.


Agreed on getting Benjamin's blessing. We can discuss it in a few weeks.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 EOL date

2018-08-23 Thread Eric V. Smith


On 8/23/2018 4:30 PM, Victor Stinner wrote:

Hi,

The reference is the PEP 373 "Python 2.7 Release Schedule". See the
"Update" section:
https://www.python.org/dev/peps/pep-0373/#update


We could probably make it more clear in this section and/or in 
https://www.python.org/dev/peps/pep-0373/#id4 that there will be no 2.7 
releases of any kind after 2020-01-01: no bugfix, no security, and no 
source releases after that date.


I'll whip something up if there's general agreement.

Eric



Victor

2018-08-23 20:53 GMT+02:00 Collin Anderson :

Hi All,

Sorry if this has been mentioned before, but I noticed the Python 2.7 EOL
date was recently set to Jan 1st, 2020.

My understanding was Python releases get 5 years of support from their
initial release, and Python 2.7 was extended an additional 5 years.

Python 2.7 was originally released on 2010-07-03, and with an original EOL
of 2015-07-03. Extended 5 years, shouldn't the EOL be 2020-07-03?

Also, this statement is a little unclear to me:


Specifically, 2.7 will receive bugfix support until January 1, 2020. All
2.7 development work will cease in 2020.


This statement makes it sound like bugfixes end on Jan 1st, but seems to
leave open the possibility that security fixes could continue through the
year.

Thanks!
Collin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Finding Guido's replacement

2018-07-24 Thread Eric V. Smith


On 7/24/2018 4:32 PM, Gregory P. Smith wrote:
On Tue, Jul 24, 2018 at 12:27 PM Abdur-Rahmaan Janhangeer 
mailto:arj.pyt...@gmail.com>> wrote:


not googler i mean google. they requested a change to a pep


"They"?  nah.  "Google LLC" did not request anything. People who 
happen to be working for Google on a Google owned project asked a 
question seeking clarification / codification of some details.  There 
is nothing more to read into that.  Adam was merely introducing 
himself by stating some background info as he hasn't participated on 
the list much in the past.


-gps | destroyer of conspiracies

PS The number of core devs / committers who are Googlers has been high 
for well over a decade (myself included).  As is true for many other 
large open source friendly companies as well.  I believe Microsoft has 
the most /active/ committers employed at the moment.


I think we should be concerned that fully 1/3 of True Blade's staff are 
core developers. Why are they so interested in Python? What nefarious 
end could possibly motivate them to devote so many company resources to 
its development?


Eric
VP, True Blade


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Time for 3.4.9 and 3.5.6

2018-07-08 Thread Eric V. Smith


On 7/8/2018 8:35 PM, Terry Reedy wrote:

On 7/8/2018 1:05 PM, Ivan Pozdeev via Python-Dev wrote:
I'll use this opportunity to remind you that 3.4 build is broken -- it 
can't be built from start to installer with the instructions given 
because of outside factors (CPython has migrated from Hg to Git). 
https://bugs.python.org/issue31623 about this was ignored (see 
https://bugs.python.org/issue31623#msg303708 for supplemental fixes).


If this isn't something considered needing a fix, the claim that 3.4 
is supported in any shape and form is but a pretense


Another wild exaggeration that inhibits me, and I suspect others, from 
attending to your legitimate issue.


Yes, thanks for writing this, Terry. Given Ivan's previous behavior on 
his "Drop/deprecate Tkinter?" thread, and combined with this thread, I'm 
unlikely to spend my free time on his particular issue here.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Call for prudence about PEP-572

2018-07-08 Thread Eric V. Smith


On 7/8/2018 1:59 PM, Chris Angelico wrote:

On Mon, Jul 9, 2018 at 3:55 AM, Eric V. Smith  wrote:

I agree with Chris in this case. That said, there is at least one place
where the grammar does forbid you from doing something that would otherwise
make be allowable: decorators.


@lookup[0]

   File "", line 1
 @lookup[0]
^
SyntaxError: invalid syntax

But this works:


new_decorator = lookup[0]
@new_decorator

... def f(): pass

Thus, the idea of restricting the type of expression that can be used in
particular circumstances is not without precedent, and should not be
dismissed at face value. That is, unless we want to remove the restriction
on decorators, which I'm okay with, too. I have occasionally wanted to do
something more complicated with a decorator, and used the workaround above.



This is true. I wasn't around when decorator syntax was discussed;
what were the reasons for this being the way it is? It isn't simply
"'@' test".


I was around, but I don't recall the exact reasoning. Something along 
the lines of YAGNI, I believe.


The first reference I found to it is 
https://mail.python.org/pipermail/python-ideas/2009-September/005623.html, 
although surely there are older ones, and even this email references an 
older super-confusing use of lambdas in decorators.


In any event, I see no reason to restrict where assignment expressions 
can be used.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Call for prudence about PEP-572

2018-07-08 Thread Eric V. Smith

On 7/8/2018 1:23 PM, Chris Angelico wrote:

On Mon, Jul 9, 2018 at 3:14 AM, Giampaolo Rodola'  wrote:

On Sun, Jul 8, 2018 at 6:45 PM Steve Holden  wrote:

But the PEP 8 spellings are

 foo(x=1)

and

f(x := 1).

The extra spacing makes it obvious that this isn't a regular named
argument.

What if the author of the code I'm reading didn't respect PEP-8? I don't
think it's fair to invoke PEP-8 as a counter-measure to obviate a syntax
which can clearly be mistaken with something else simply by omitting 2
spaces. Not to mention that I don't see why anyone would want to declare a
variable in there in the first place.

It's not about why someone would want to assign inside a function
call. It's about why it should be forbidden. Perhaps nobody has a good
reason to use THlS_OBJECT as a variable name, and it's potentially
very confusing; but should the grammar of Python forbid it? No.
Because there is no value in forbidding it.

Python's grammar has a number of weird edge cases due to necessity
(for instance, "for x in a if cond else b:" works, but not in a
comprehension), and when there's an actual conflict, sure, you can say
"but nobody would ever want to do that, so we'll forbid it". In this
case, there is no conflict.

I agree with Chris in this case. That said, there is at least one place 
where the grammar does forbid you from doing something that would 
otherwise make be allowable: decorators.

>>> @lookup[0]
  File "", line 1
@lookup[0]
   ^
SyntaxError: invalid syntax

But this works:

>>> new_decorator = lookup[0]
>>> @new_decorator
... def f(): pass

Thus, the idea of restricting the type of expression that can be used in 
particular circumstances is not without precedent, and should not be 
dismissed at face value. That is, unless we want to remove the 
restriction on decorators, which I'm okay with, too. I have occasionally 
wanted to do something more complicated with a decorator, and used the 
workaround above.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Call for prudence about PEP-572

2018-07-08 Thread Eric V. Smith


On 7/8/2018 5:41 AM, Giampaolo Rodola' wrote:
As for "assert" what I'm concern about is the proliferation of things 
like this:

     class Foo:
         def __init__(self):
             assert self.x := fun1()
             assert self.y := fun2()
             assert self.z := fun3()

When I look at that my brain tells me that the main subject of the line 
is "assert", not the assignment, but maybe it's just because I'm not 
used to it. That aside there's the question of what to do when "python 
-O" switch is used. With this in place "-O" would acquire a new meaning, 
as it would disable "assert" statements AND assignments.


It has always meant "disable the assert statement and therefore any side 
effects the expression has". It's just that now the side effects are 
more obvious, or maybe easier to create. Even pre-572 I've been bitten 
by this, I'm ashamed to admit.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-27 Thread Eric V. Smith


> On Jun 27, 2018, at 9:49 AM, Steven D'Aprano  wrote:
> 
>> On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
>>> On 6/27/2018 7:08 AM, Chris Angelico wrote:
>>> It gets funnier with nested loops. Or scarier. I've lost the ability
>>> to distinguish those two.
>>> 
>>> def test():
>>>spam = 1
>>>ham = 2
>>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
>>>return vars
>>> 
>>> Wanna guess what that's gonna return?
>> 
>> I'm not singling out Chris here, but these discussions would be easier 
>> to follow and more illuminating if the answers to such puzzles were 
>> presented when they're posed.
> 
> You can just copy and paste the function into the interactive 
> interpreter and run it :-)

Not on my phone when I’m riding a bus, I can’t. I’m trying to more or less 
follow the discussion, but the “guess what this will do” aspect of the 
discussion makes it hard.

Eric 

> 
> But where's the fun in that? The point of the exercise is to learn first 
> hand just how complicated it is to try to predict the *current* scope 
> behaviour of comprehensions. Without the ability to perform assignment 
> inside them, aside from the loop variable, we've managed to avoid 
> thinking too much about this until now.
> 
> It also demonstrates the unrealisticness of treating comprehensions as a 
> separate scope -- they're hybrid scope, with parts of the comprehension 
> running in the surrounding local scope, and parts running in an sublocal 
> scope.
> 
> Earlier in this thread, Nick tried to justify the idea that 
> comprehensions run in their own scope, no matter how people think of 
> them -- but that's an over-simplification, as Chris' example above 
> shows. Parts of the comprehension do in fact behave exactly as the naive 
> model would suggest (even if Nick is right that other parts don't).
> 
> As complicated and hairy as the above example is, (1) it is a pretty 
> weird thing to do, so most of us will almost never need to consider it; 
> and (2) backwards compatibility requires that we live with it now (at 
> least unless we introduce a __future__ import).
> 
> If we can't simplify the scope of comprehensions, we can at least 
> simplify the parts that actually matters. What matters are the loop 
> variables (already guaranteed to be sublocal and not "leak" out of the 
> comprehension) and the behaviour of assignment expressions (open to 
> discussion).
> 
> Broadly speaking, there are two positions we can take:
> 
> 1. Let the current implementation of comprehensions as an implicit 
> hidden function drive the functionality; that means we duplicate the 
> hairiness of the locals() behaviour seen above, although it won't be 
> obvious at first glance.
> 
> What this means in practice is that assignments will go to different 
> scopes depending on *where* they are in the comprehension:
> 
>[ expr   for x in iter1  for y in iter2  if cond   ...]
>[ BB for x in AA for y in BB if BB ...]
> 
> Assignments in the section marked "AA" will be in the local scope; 
> assignments in the BB sections will be in the sublocal scope. That's 
> not too bad, up to the point you try to assign to the same name in 
> AA and BB. And then you are likely to get confusing hard to 
> debug UnboundLocalErrors.
> 
> 
> 2. Or we can keep the current behaviour for locals and the loop 
> variables, but we can keep assignment expressions simple by ensuring 
> they always bind to the enclosing scope. Compared to the complexity of 
> the above, we have the relatively straight forward:
> 
>[ AA for x in AA for y in AA if AA ...]
> 
> The loop variables continue to be hidden away in the invisible, implicit 
> comprehension function, where they can't leak out, while explicit 
> assignments to variables (using := or given or however it is spelled) 
> will always go into the surrounding local scope, like they do in every 
> other expression.
> 
> Does it matter that the implementation of this requires an implicit 
> nonlocal declaration for each assignment? No more than it matters that 
> comprehensions themselves require an implicit function.
> 
> And what we get out of this is simpler semantics at the Python level:
> 
> - Unless previous declared global, assignment expressions always bind to 
> the current scope, even if they're inside a comprehension;
> 
> - and we don't have to deal with the oddity that different bits of a 
> comprehension run in different scopes (unless we go out of o

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-27 Thread Eric V. Smith


On 6/27/2018 7:08 AM, Chris Angelico wrote:

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
 spam = 1
 ham = 2
 vars = [key1+key2 for key1 in locals() for key2 in locals()]
 return vars

Wanna guess what that's gonna return?


I'm not singling out Chris here, but these discussions would be easier 
to follow and more illuminating if the answers to such puzzles were 
presented when they're posed.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] 3.7.0 / 3.6.6 Update: all systems go for final releases!

2018-06-26 Thread Eric V. Smith

Congrats, Ned. Thank you for all of your hard work!

--
Eric

> On Jun 26, 2018, at 2:39 AM, Ned Deily  wrote:
> 
> A quick update: after many months we are at the finish line. We are on
> track (mixing metaphors) to release 3.7.0 (and 3.6.6) this week on
> 2018-06-27. Since 3.7.0rc1 shipped 2 weeks ago, I am aware of only two
> noteworthy regressions that have been identified and now fixed. Since
> the issues for both have the potential to impact some (but small)
> subsets of 3.7.0 users and the fixes for both are straightforward and
> appear to be low-risk, I am planning to cherry-pick the fixes for them
> into 3.7.0 final without either another release candidate cycle or
> waiting for 3.7.1. There may be some doc fixes that get cherry-picked
> as well. At the moment, there are no plans for any bug cherry-picks for
> 3.6.6 final.
> 
> As you know, a new feature release is a big deal and something for all
> of us to be proud of.  A new feature release also has various, mostly
> minor, impacts to lots of different parts of our development
> infrastructure: to multiple branches of the cpython repo, to
> documentation builds, to different parts of the python.org web site,
> etc. You will start to see some of the changes roll out over the next 24
> to 36 hours and it may take some time until everything is in place.
> So please be patient until the official release announcement goes out
> before reporting release-related issues. Also be advised that over the
> same period, there may be a few brief periods where commit access to
> various cpython branches is blocked in order to do the necessary
> release engineering. If you run into this, for example when trying to
> merge a PR, please try again in a few hours.
> 
> Thanks and more later!
> 
> https://bugs.python.org/issue33851
> https://bugs.python.org/issue33932
> 
> --
>  Ned Deily
>  n...@python.org -- []
> 
> ___
> python-committers mailing list
> python-committ...@python.org
> https://mail.python.org/mailman/listinfo/python-committers
> Code of Conduct: https://www.python.org/psf/codeofconduct/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stable ABI

2018-06-03 Thread Eric V. Smith


On 6/3/2018 10:55 AM, Christian Tismer wrote:

On 03.06.18 13:18, Ronald Oussoren wrote:




On 3 Jun 2018, at 12:03, Christian Tismer  wrote:

...



I have written a script that scans all relevant header files
and analyses all sections which are reachable in the limited API
context.
All macros that don't begin with an underscore which contain
a "->tp_" string are the locations which will break.

I found exactly 7 locations where this is the case.

My PR will contain the 7 fixes plus the analysis script
to go into tools. Preparind that in the evening.


Having tests would still be nice to detect changes to the stable ABI when they 
are made.

Writing those tests is quite some work though, especially if those at least 
smoke test the limited ABI by compiling snippets the use all symbols that 
should be exposed by the limited ABI. Writing those tests should be fairly 
simple for someone that knows how to write C extensions, but is some work.

Writing a tests that complain when the headers expose symbols that shouldn’t be 
exposed is harder, due to the need to parse headers (either by hacking 
something together using regular expressions, or by using tools like gccxml or 
clang’s C API).


What do you mean?
My script does that with all "tp_*" type fields.
What else would you want to check?


I think Ronald is saying we're trying to answer a few questions:

1. Did we accidentally drop anything from the stable ABI?

2. Did we add anything to the stable ABI that we didn't mean to?

3. (and one of mine): Does the stable ABI already contain things that we 
don't expect it to?


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why aren't escape sequences in literal strings handled by the tokenizer?

2018-05-17 Thread Eric V. Smith


On 5/17/2018 3:01 PM, Larry Hastings wrote:



I fed this into tokenize.tokenize():

b''' x = "\u1234" '''

I was a bit surprised to see \U in the output.  Particularly because 
the output (t.string) was a *string* and not *bytes*.


For those (like me) who have no idea how to use tokenize.tokenize's 
wacky interface, the test code is:


list(tokenize.tokenize(io.BytesIO(b''' x = "\u1234" ''').readline))

Maybe I'm making a parade of my ignorance, but I assumed that string 
literals were parsed by the parser--just like everything else is parsed 
by the parser, hey it seems like a good place for it--and in particular 
that the escape sequence substitutions would be done in the tokenizer.  
Having stared at it a little, I now detect a whiff of "this design 
solved a real problem".  So... what was the problem, and how does this 
design solve it?


I assume the intent is to not throw away any information in the lexer, 
and give the parser full access to the original string. But that's just 
a guess.


BTW, my use case is that I hoped to use CPython's tokenizer to parse 
some Python-ish-looking text and handle double-quoted strings for me.  
*Especially* all the escape sequences--leveraging all CPython's support 
for funny things like \U{penguin}.  The current behavior of the 
tokenizer makes me think it'd be easier to roll my own!


Can you feed the token text to the ast?

>>> ast.literal_eval('"\u1234"')
'ሴ'

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What is the rationale behind source only releases?

2018-05-16 Thread Eric V. Smith


On 5/16/18 4:34 AM, Serhiy Storchaka wrote:

16.05.18 07:35, Alex Walters пише:

[1]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence


And I wish that every author who suggested the idea for Python was
familiar with the Chesterton's fence principle.


Indeed! It's refreshing. Thanks, Alex.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572 and f-strings

2018-05-12 Thread Eric V. Smith


> On May 12, 2018, at 9:03 AM, Chris Angelico  wrote:
> 
>> On Sat, May 12, 2018 at 9:11 PM, Eric V. Smith  wrote:
>> I don't think it matters to its acceptance, but PEP 572 should at least
>> mention that the := syntax means that you cannot use assignment expressions
>> in f-strings.
>> 
>> As I wrote in a python-ideas email, f'{x:=4}' already has a defined meaning
>> (even if no one is using it).
> 
> As with lambda functions, you can't write them the simple way.
> However, you can parenthesize it, and then it works fine.
> 
>>>> f"@{(lambda: 42)}@"
> '@ at 0x7f09e18c4268>@'
>>>> f"@{(y := 1)}@"
> '@1@'
>>>> y
> 1
> 

Yes, but just as PEP 498 mentions lambdas, I think 572 should mention 
f-strings, and point out this workaround. 

Eric


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 572 and f-strings

2018-05-12 Thread Eric V. Smith

I don't think it matters to its acceptance, but PEP 572 should at least 
mention that the := syntax means that you cannot use assignment 
expressions in f-strings.


As I wrote in a python-ideas email, f'{x:=4}' already has a defined 
meaning (even if no one is using it).


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Slow down...

2018-05-08 Thread Eric V. Smith


> On May 8, 2018, at 11:48 AM, Brett Cannon  wrote:
> 
> 
> 
>> On Tue, 8 May 2018 at 08:26 Craig Rodrigues  wrote:
>>> On Mon, May 7, 2018 at 2:24 PM Barry Warsaw  wrote:
>>> On May 7, 2018, at 11:49, Craig Rodrigues  wrote:
>>> > 
>>> > Would it be reasonable to request a 10 year moratorium on making changes 
>>> > to the core Python language,
>>> > and for the next 10 years only focus on things that do not require core 
>>> > language changes,
>>> > such as improving/bugfixing existing libraries, writing new libraries, 
>>> > improving tooling, improving infrastructure (PyPI),
>>> > improving performance, etc., etc.?
>>> 
>>> IMHO, no.
>>> 
>>> I don’t believe that the way for Python to remain relevant and useful for 
>>> the next 10 years is to cease all language evolution.  Who knows what the 
>>> computing landscape will look like in 5 years, let alone 10?  Something as 
>>> arbitrary as a 10 year moratorium is (again, IMHO) a death sentence for the 
>>> language.
>>> 
>>> But I do think it makes sense to think about how Python-the-language and 
>>> CPython-the-reference implementation can better balance the desire to 
>>> evolve vs the need to concentrate on improving what we’ve got.  With that 
>>> in mind, it does make sense to occasionally use a moratorium release to 
>>> focus on tech debt, cleaning up the stdlib, improve performance, etc.
>>> 
>>> CPython’s 18 month release cycle has served us well for a long time, but I 
>>> do think it’s time to discuss whether it will still be appropriate moving 
>>> forward.  I’m not saying it is or isn’t, but with the release of 3.7, I 
>>> think it’s a great time to explore our options.
>>> 
>> 
>> 10 years is a long time for many types of applications, such as web server 
>> and desktop applications
>> where regular and somewhat frequent upgrades can happen.
>> However, I have worked on embedding Python in networking and storage devices.
>> It is true that many of these types of devices can also have their software 
>> upgraded,
>> but often the frequency of such upgrades is much slower than for 
>> conventional web server and desktop applications.
>> Upgrades of these devices usually spans user-space and kernel/device 
>> drivers, so
>> upgrades are usually done more cautiously and less frequently.
>> 
>> For Python 2.x, the ship has sailed.  However, 10 years from now, if the 
>> Python language
>> is pretty much the same as Python 3.7 today, that would be nice.
> 
> Then feel free to stay on Python 3.7. We have versioned releases so people 
> can choose to do that. :)

Also, you can pay people to support old versions, and sometimes even backport 
features you need. So I think this use case is already covered. You just can’t 
expect to hold up language development because you want to have a stable 
supported version. 

Eric 

>  
>> 
>> I'm not stuck on the number "10 years", but I am just throwing it out there 
>> as a draft proposal.
>> So even 5-8 year moratorium would be nice to strive for.
> 
> Timespans of that length are still too long to freeze the language. Look at 
> it this way: node.js 0.10.0 was released 5 years ago and now it's a thing. If 
> we had not moved forward and added async/await in Python 3.5 -- which was 
> only 3 years ago -- but instead froze ourselves for 5 years would we be 
> considered relevant in the networking world like we are, or viewed as 
> somewhat as a dinosaur?
> 
> I realize the embedded world moves at a different pace (as well as other 
> groups of developers), but that doesn't mean we have to move at the speed of 
> our slowest adopters to punish those willing and wanting newer features.
>  
>> 
>> 
>> Outside of the embedded space, I will give another example where folks in 
>> industry are behind.
>> I don't want to pick on a particular vendor, but from April 24-26, I 
>> attended training sessions at RedisConf18 in San Francisco.
>> During the training sessions, multiple sample Python code examples were 
>> given for accessing the Redis database.
>> The instructor thought that the code examples worked in Python 3, but in 
>> fact, they only worked in Python 2 mostly due to
>> bytes/unicode issues.  During the class, I fixed the examples for Python 3 
>> and submitted the patches to the instructor,
>> who gratefully accepted my patches.  However, there are going to be many, 
>> many users of Redis out there who
>> maybe will not upgrade their Python code for newer versions of Python for 
>> many years.
> 
> Why is Redis specifically going to be behind specifically? Are they embedding 
> the interpreter?
>  
>> 
>> Besides Redis users, I am seeing all sorts of communities and companies 
>> which are behind in terms of having working
>> code and documentation that works on latest Python 3.  It is going to take 
>> YEARS for all these communities and companies
>> to catch up (if ever).
> 
> And that's fine. If they want to continue to maintain Python 2 and stay on 
> it, or simply stick with our final release

Re: [Python-Dev] PEP 575: Unifying function/method classes

2018-05-03 Thread Eric V. Smith

On 5/3/2018 6:22 AM, Jeroen Demeyer wrote:

On 2018-05-03 11:30, Victor Stinner wrote:

Please don't queue backward incompatible changes for Python 4.0. You
should use the regular deprecation process.

I don't really see how that can be done here. As Stefan said

The problem is that this
change does not really fit into the deprecation cycle since there is no
specific use case to warn about.

The PEP proposes to change an implementation detail. It's really hard to
determine at runtime whether code is relying on that implementation
detail. We could insert a DeprecationWarning in some places, but those
would mostly be false positives (a DeprecationWarning is shown but the
code won't break).

On top of that, there is no way to show a DeprecationWarning for code
like "type(x) is foo".

Deprecating doesn't necessarily involve a DeprecationWarning, although
if possible it should, of course. It could just be a documented
deprecation. We've done this before, although I can't think of an
example off the top of my head (which I realize is not exactly helpful).

"If you're doing a type check involving C functions, and you're doing it
, change it to because we're going to deprecate the
old way in version x.y". Of course this assumes both and way> can coexist for several versions, which itself might not be possible.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572)

2018-04-27 Thread Eric V. Smith


On 4/27/2018 8:28 AM, Steven D'Aprano wrote:

On Fri, Apr 27, 2018 at 08:13:20AM +0200, Andrea Griffini wrote:

Now we got standard library features requiring type annotation


We do? Did I miss them? Which std lib features are you referring to?

(That's not a rhetorical question -- maybe I have missed something.)


Presumably dataclasses and typing.NamedTuple.

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Timeline for Pull request reviews in 3.8

2018-04-06 Thread Eric V. Smith


Hi, Anthony.

On 4/5/2018 10:57 PM, Anthony Flury via Python-Dev wrote:
Can anyone enlighten me on what the expected time-line is for reviewing 
pull requests made on 3.8.


I can give you the timeline for releases. Unfortunately, volunteer time 
is short for reviews.


I made a few simple fixes in Early March - and I understand everyone is 
busy.


If you mention the issues, someone might take a look at them.


What is the time line and cut off dates for backports to 3.7 and 3.6.


Both are closed to new features and open for bug fixes. PEP 494 has the 
3.6 schedule and PEP 537 has the 3.7 schedule. For 3.6 the last binary 
release is currently scheduled for 2018-12-03.


I also made a documentation change (based on a open bug report) into 
2.7, and I am keen to understand the planned time-line for those too.


PEP 373 has the 2.7 release schedule.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

2018-03-29 Thread Eric V. Smith


On 3/29/2018 12:13 PM, Nick Coghlan wrote:

On 29 March 2018 at 21:50, Eric V. Smith  wrote:

#1 seems so complex as to not be worth it, given the likely small overall
impact of the optimization to a large program. If the speedup really is
sufficiently important for a particular piece of code, I'd suggest just
rewriting the code to use f-strings, and the author could then determine if
the transformation breaks anything. Maybe write a 2to3 like tool that would
identify places where str.format or %-formatting could be replaced by
f-strings? I know I'd run it on my code, if it existed. Because the
optimization can only work code with literals, I think manually modifying
the source code is an acceptable solution if the possible change in
semantics implied by #3 are unacceptable.


While more projects are starting to actively drop Python 2.x support,
there are also quite a few still straddling the two different
versions. The "rewrite to f-strings" approach requires explicitly
dropping support for everything below 3.6, whereas implicit
optimization of literal based formatting will work even for folks
preserving backwards compatibility with older versions.


Sure. But 3.6 will be 3 years old before this optimization is released. 
I've been seeing 3.4 support dropping off, and expect to see 3.5 follow 
suit by the time 3.8 is released. Although maybe the thought is to do 
this in a bug-fix release? If we're changing semantics at all, that 
seems like a non-starter.



As far as the semantics go, perhaps it would be possible to explicitly
create a tuple as part of the implementation to ensure that the
arguments are still evaluated in order, and everything gets calculated
exactly once? This would have the benefit that even format strings
that used numbered references could be optimised in a fairly
straightforward way.

 '{}{}'.format(a, b)

would become:

 _hidden_ref = (a, b)
 f'{_hidden_ref[0]}{_hidden_ref[1]}'

while:

 '{1}{0}'.format(a, b)

would become:

 _hidden_ref = (a, b)
 f'{_hidden_ref[1]}{_hidden_ref[0]}'

This would probably need to be implemented as Serhiy's option 1
(generating a distinct AST node), which in turn leads to 2a: adding
extra stack manipulation opcodes in order to more closely replicate
str.format semantics.


I still think the complexity isn't worth it, but maybe I'm a lone voice 
on this.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

2018-03-29 Thread Eric V. Smith


On 3/29/2018 6:17 AM, Jeff Allen wrote:
My credentials for this are that I re-worked str.format in Jython quite 
extensively, and I followed the design of f-strings a bit when they were 
introduced, but I haven't used them to write anything.


Thanks for your work on Jython. And hop on the f-string bandwagon!

The difference Serhiy identifies emerges (I think) because in the 
conventional interpretation of a format call, the arguments of format 
are evaluated left-to right (all of them) and then formatted in the 
order references are encountered to these values in a tuple or 
dictionary. In an f-string expressions are evaluated as they are 
encountered. A more testing example is therefore perhaps:


     '{1} {0}'.format(a(), b()) # E1

     f'{b()}{a()}'  # E2


I think I would be very surprised to find b called before a in E1 
because of the general contract on the meaning of method calls. I'm 
assuming that's what an AST-based optimisation would do? There's no 
reason in E2 to call them in any other order than b then a and the 
documentation tells me they are.


But do I expect a() to be called before the results of b() are 
formatted? In E1 I definitely expect that. In E2 I don't think I'd be 
surprised either way. Forced to guess, I would guess that b() would be 
formatted and in the output buffer before a() was called, since it gives 
the implementation fewer things to remember. Then I hope I would not 
depend on this guesswork. Strictly-speaking the documentation doesn't 
say when the result is formatted in relation to the evaluation of other 
expressions, so there is permission for Serhiy's idea #2.


I don't think we should restrict f-strings to having to evaluate all of 
the expressions before formatting. But, if we do restrict it, we should 
document whatever the order is in 3.6 and add tests to ensure the 
behavior doesn't change.


I think the (internal) AST change implied in Serhiy's idea #1 is the 
price one has to pay *if* one insists on optimising str.format().


str.format just a method like any other. The reasons would have to be 
very strong to give it special-case semantics. I agree that the cases 
are rare in which one would notice a difference. (Mostly I think it 
would be a surprise during debugging.) But I think users should be able 
to rely on the semantics of call. Easier optimisation doesn't seem to me 
a strong enough argument.


This leaves me at:
1: +1
2a, 2b: +0
3: -1


#1 seems so complex as to not be worth it, given the likely small 
overall impact of the optimization to a large program. If the speedup 
really is sufficiently important for a particular piece of code, I'd 
suggest just rewriting the code to use f-strings, and the author could 
then determine if the transformation breaks anything. Maybe write a 2to3 
like tool that would identify places where str.format or %-formatting 
could be replaced by f-strings? I know I'd run it on my code, if it 
existed. Because the optimization can only work code with literals, I 
think manually modifying the source code is an acceptable solution if 
the possible change in semantics implied by #3 are unacceptable.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

2018-03-28 Thread Eric V. Smith

I’d vote #3 as well. 

--
Eric

> On Mar 28, 2018, at 11:27 AM, Serhiy Storchaka  wrote:
> 
> There is a subtle semantic difference between str.format() and "equivalent" 
> f-string.
> 
>'{}{}'.format(a, b)
>f'{a}{b}'
> 
> In the former case b is evaluated before formatting a. This is equivalent to
> 
>t1 = a
>t2 = b
>t3 = format(t1)
>t4 = format(t2)
>r = t3 + t4
> 
> In the latter case a is formatted before evaluating b. This is equivalent to
> 
>t1 = a
>t2 = format(t1)
>t3 = b
>t4 = format(t3)
>r = t2 + t4
> 
> In most cases this doesn't matter, but when implement the optimization that 
> transforms the former expression to the the latter one ([1], [2]) we have to 
> make a decision what to do with this difference.
> 
> 1. Keep the exact semantic of str.format() when optimize it. This means that 
> it should be transformed into AST node different from the AST node used for 
> f-strings. Either introduce a new AST node type, or add a boolean flag to 
> JoinedStr.
> 
> 2. Change the semantic of f-strings. Make it closer to the semantic of 
> str.format(): evaluate all subexpressions first than format them. This can be 
> implemented in two ways:
> 
> 2a) Add additional instructions for stack manipulations. This will slow down 
> f-strings.
> 
> 2b) Introduce a new complex opcode that will replace FORMAT_VALUE and 
> BUILD_STRING. This will speed up f-strings.
> 
> 3. Transform str.format() into an f-string with changing semantic, and ignore 
> this change. This is not new. The optimizer already changes semantic. 
> Non-optimized "if a and True:" would call bool(a) twice, but optimized code 
> calls it only once.
> 
> [1] https://bugs.python.org/issue28307
> [2] https://bugs.python.org/issue28308
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] descriptor __set_name__ and dataclasses

2018-03-26 Thread Eric V. Smith


On 3/26/18 11:10 AM, Eric V. Smith wrote:

On 3/26/18 11:08 AM, Nick Coghlan wrote:

On 27 March 2018 at 00:40, Eric V. Smith  wrote:



Would it be feasible to define `Field.__set_name__`, and have that
call `default.__set_name__` when the latter exists, and be a no-op
otherwise?


A clever idea! I'll look in to it.


It looks like that does work. Thank, Nick!

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] descriptor __set_name__ and dataclasses

2018-03-26 Thread Eric V. Smith


On 3/26/18 11:08 AM, Nick Coghlan wrote:

On 27 March 2018 at 00:40, Eric V. Smith  wrote:

https://bugs.python.org/issue33141 points out an interesting issue with
dataclasses and descriptors.

Given this code:

from dataclasses import *

class D:
"""A descriptor class that knows its name."""
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is not None:
return 1
return self


@dataclass
class C:
d: int = field(default=D(), init=False)

C.d.name is not set, because d.__set_name__ is never called. However, in
this case:

class X:
d: int = D()

X.d.name is set to 'd' when d.__set_name__ is called during type.__new__.

The problem of course, is that in the dataclass case, when class C is
initialized, and before the decorator is called, C.d is set to a Field()
object, not to D(). It's only when the dataclass decorator is run that I
change C.d from a Field to the value of D(). That means that the call to
d.__set_name__(C, 'd') is skipped. See
https://www.python.org/dev/peps/pep-0487/#implementation-details for details
on how type.__new__ works.

The only workaround I can think of is to emulate the part of PEP 487 where
__set_name__ is called. I can do this from within the @dataclass decorator
when I'm initializing C.d. I'm not sure how great this solution is, since
it's moving the call from class creation time to class decorator time. I
think in 99+% of cases this would be fine, but you could likely write code
that depends on side effects of being called during type.__new__.

Unless anyone has strong objections, I'm going to make the call to
__set_name__ in the @datacalss decorator. Since this is such a niche use
case, I don't feel strongly that it needs to be in today's beta release, but
if possible I'll get it in. I already have the patch written. And if it does
get in but the consensus is that it's a bad idea, we can back it out.


Would it be feasible to define `Field.__set_name__`, and have that
call `default.__set_name__` when the latter exists, and be a no-op
otherwise?


A clever idea! I'll look in to it.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] descriptor __set_name__ and dataclasses

2018-03-26 Thread Eric V. Smith

https://bugs.python.org/issue33141 points out an interesting issue with 
dataclasses and descriptors.


Given this code:

from dataclasses import *

class D:
"""A descriptor class that knows its name."""
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is not None:
return 1
return self


@dataclass
class C:
d: int = field(default=D(), init=False)

C.d.name is not set, because d.__set_name__ is never called. However, in 
this case:


class X:
d: int = D()

X.d.name is set to 'd' when d.__set_name__ is called during type.__new__.

The problem of course, is that in the dataclass case, when class C is 
initialized, and before the decorator is called, C.d is set to a Field() 
object, not to D(). It's only when the dataclass decorator is run that I 
change C.d from a Field to the value of D(). That means that the call to 
d.__set_name__(C, 'd') is skipped. See 
https://www.python.org/dev/peps/pep-0487/#implementation-details for 
details on how type.__new__ works.


The only workaround I can think of is to emulate the part of PEP 487 
where __set_name__ is called. I can do this from within the @dataclass 
decorator when I'm initializing C.d. I'm not sure how great this 
solution is, since it's moving the call from class creation time to 
class decorator time. I think in 99+% of cases this would be fine, but 
you could likely write code that depends on side effects of being called 
during type.__new__.


Unless anyone has strong objections, I'm going to make the call to 
__set_name__ in the @datacalss decorator. Since this is such a niche use 
case, I don't feel strongly that it needs to be in today's beta release, 
but if possible I'll get it in. I already have the patch written. And if 
it does get in but the consensus is that it's a bad idea, we can back it 
out.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fix strncpy warning with gcc 8 (#5840)

2018-03-06 Thread Eric V. Smith


On 3/6/2018 6:52 AM, Serhiy Storchaka wrote:

06.03.18 12:34, Xiang Zhang пише:
https://github.com/python/cpython/commit/efd2bac1564f8141a4eab1bf8779b412974b8d69 


commit: efd2bac1564f8141a4eab1bf8779b412974b8d69
branch: master
author: Siddhesh Poyarekar 
committer: Xiang Zhang 
date: 2018-03-06T18:34:35+08:00
summary:

Fix strncpy warning with gcc 8 (#5840)

The length in strncpy is one char too short and as a result it leads
to a build warning with gcc 8.  Comment out the strncpy since the
interpreter aborts immediately after anyway.

files:
M Python/pystrtod.c

diff --git a/Python/pystrtod.c b/Python/pystrtod.c
index 9bf936386210..601f7c691edf 100644
--- a/Python/pystrtod.c
+++ b/Python/pystrtod.c
@@ -1060,8 +1060,8 @@ format_float_short(double d, char format_code,
  else {
  /* shouldn't get here: Gay's code should always return
 something starting with a digit, an 'I',  or 'N' */
-    strncpy(p, "ERR", 3);
-    /* p += 3; */
+    /* strncpy(p, "ERR", 3);
+   p += 3; */
  Py_UNREACHABLE();
  }
  goto exit;



I think this code was added for purpose. In the case of programming 
error we could get meaningful value in post-mortal debugging. But after 
replacing assert(0) with Py_UNREACHABLE this perhaps lost a sense.


If this code is no longer needed it is better to remove it than keeping 
an obscure comment.


What are your thoughts @warsaw and @ericvsmith?

Py_UNREACHABLE was added in issue31338 by Barry. The original code was 
added in issue1580 by Eric Smith (maybe it was copied from other place).


Mark Dickinson and/or I wrote that. I agree that leaving the two 
commented out lines is confusing. I suggest deleting them, and of course 
leave the comment about Gay's code.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-28 Thread Eric V. Smith


On 3/1/2018 1:02 AM, Nick Coghlan wrote:
I'm assuming you meant "3.7.0b2" here (and similarly alpha->beta for the 
other version numbers below)


Oops, yes. Thanks.


So going back to original questions above, my suggestions are:

1. What happens when a frozen dataclass inherits from a non-frozen
dataclass? The result is a frozen dataclass, and all fields are
non-writable. No non-fields can be added. This is a reversion to the
3.7.0a1 behavior.

2. What happens when a non-frozen dataclass inherits from a frozen
dataclass? The result is a frozen dataclass, and all fields are
non-writable. No non-fields can be added. This is a reversion to the
3.7.0a1 behavior. I'd also be okay with this case being an error,
and you'd have to explicitly mark the derived class as frozen. This
is the 3.7.0a2 behavior.

3. What happens when a non-dataclass inherits from a frozen
dataclass? The fields that are in the dataclass are non-writable,
but new non-field attributes can be added. This is new behavior.

4. Can new non-field attributes be created for frozen dataclasses?
No. This is existing behavior.


+1 from me for the variant of this where dataclasses inheriting from 
frozen data classes must explicitly mark themselves as frozen (at least 
for now). That way we leave the door open to allowing a variant where a 
non-frozen dataclass that inherits from a frozen dataclass can set 
"hash=False" on all of the new fields it adds to avoid becoming frozen 
itself.


I tend to agree. It's not like this is a huge burden, and at least the 
author is acknowledging that the class will end up frozen.




I'm hoping this change isn't so large that we can't get it in to
3.7.0a3 next month.


I think this qualifies as the beta period serving its intended purpose 
(i.e. reviewing and refining the behaviour of already added features, 
without allowing completely new features).


Thanks.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-27 Thread Eric V. Smith

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:

When working on the docs for dataclasses, something unexpected came up.  If a 
dataclass is specified to be frozen, that characteristic is inherited by 
subclasses which prevents them from assigning additional attributes:

 >>> @dataclass(frozen=True)
 class D:
 x: int = 10

 >>> class S(D):
 pass

 >>> s = S()
 >>> s.cached = True
 Traceback (most recent call last):
   File "", line 1, in 
 s.cached = True
   File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
 line 448, in _frozen_setattr
 raise FrozenInstanceError(f'cannot assign to field {name!r}')
 dataclasses.FrozenInstanceError: cannot assign to field 'cached'

Other immutable classes in Python don't behave the same way:

 >>> class T(tuple):
 pass

 >>> t = T([10, 20, 30])
 >>> t.cached = True

 >>> class F(frozenset):
 pass

 >>> f = F([10, 20, 30])
 >>> f.cached = True

 >>> class B(bytes):
 pass

 >>> b = B()
 >>> b.cached = True

I'll provide some background, then get in to the dataclasses design 
issues. Note that I'm using "field" here in the PEP 557 sense.

There are some questions to resolve:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass?

2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass?

3. What happens when a non-dataclass inherits from a frozen dataclass?

4. Can new non-field attributes be created for frozen dataclasses?

I think it's useful to look at what attrs does. Unsurprisingly, attrs 
works the way the dataclasses implementation in 3.7.0a1 works:

- If a frozen attrs class inherits from a non-frozen attrs class, the 
result is a frozen attrs class.

- If a non-frozen attrs class inherits from a frozen attrs class, the 
result is a frozen attrs class.

- For a frozen attrs class, you may not assign to any field, nor create 
new non-field instance attributes.

- If a non-attrs class derives from a frozen attrs class, then you 
cannot assign to or create any non-field instance attributes. This is 
because they override the class's __setattr__ to always raise. This is 
the case that Raymond initially brought up on this thread (but for 
dataclasses, of course).

As I said, this is also how 3.7.0a1 dataclasses also works. The only 
difference between this and 3.7.0.a2 is that I prohibited inheriting a 
non-frozen dataclass from a frozen one, and also prohibited the 
opposite: you can't inherit a frozen dataclass from a non-frozen 
dataclass. This was just a stop-gap measure to give us more wiggle room 
for future changes. But this does nothing to address Raymond's concern 
about non-dataclasses deriving from frozen dataclasses.

A last piece of background info on how dataclasses and attrs work: the 
most derived class implements all of the functionality. They never call 
in to the base class to do anything. The base classes just exist to 
provide the list of fields.

If frozen dataclasses only exist to protect fields that belong to the 
hash, then my suggestion is to change the implementation of frozen class 
to use properties instead of overwriting __setattr__ (Nick also 
suggested this). This would allow you to create non-field attributes. If 
the purpose is really to prevent any attributes from being added or 
modified, then I think __setattr__ should stay but we should change it 
to allow non-dataclass subclasses to add non-field instance attributes 
(addressing Raymond's concern).

I think we shouldn't allow non-field instance attributes to be added to 
a frozen dataclass, although if anyone has a strong feeling about it, 
I'd like to hear it.

So, given a frozen dataclass "C" with field names in "field_names", I 
propose changing __setattr__ to be:

def __setattr__(self, name, value):
if type(self) is C or name in field_names:
raise FrozenInstanceError(f'cannot assign to field {name!r}')
super(cls, self).__setattr__(name, value)

In the current 3.7.0a2 implementation of frozen dataclasses, __setattr__ 
always raises. The change is the test and then call to 
super().__setattr__ if it's a derived class. The result is an exception 
if either self is an instance of C, or if self is an instance of a 
derived class, but the attribute being set is a field of C.

So going back to original questions above, my suggestions are:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior.

2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior. I'd also be okay with this case being an error, and

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-26 Thread Eric V. Smith

On 2/22/18 9:43 PM, Nick Coghlan wrote:

On 22 February 2018 at 20:55, Eric V. Smith  wrote:

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:

When working on the docs for dataclasses, something unexpected came up.
If a dataclass is specified to be frozen, that characteristic is inherited
by subclasses which prevents them from assigning additional attributes:

  >>> @dataclass(frozen=True)
  class D:
  x: int = 10

  >>> class S(D):
  pass

  >>> s = S()
  >>> s.cached = True
  Traceback (most recent call last):
File "", line 1, in 
  s.cached = True
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
line 448, in _frozen_setattr
  raise FrozenInstanceError(f'cannot assign to field {name!r}')
  dataclasses.FrozenInstanceError: cannot assign to field 'cached'

This is because "frozen-ness" is implemented by adding __setattr__ and
__delattr__ methods in D, which get inherited by S.

Other immutable classes in Python don't behave the same way:

  >>> class T(tuple):
  pass

  >>> t = T([10, 20, 30])
  >>> t.cached = True

  >>> class F(frozenset):
  pass

  >>> f = F([10, 20, 30])
  >>> f.cached = True

  >>> class B(bytes):
  pass

  >>> b = B()
  >>> b.cached = True

The only way I can think of emulating this is checking in __setattr__ to see
if the field name is a field of the frozen class, and only raising an error
in that case.

If you were going to do that then it would likely make more sense to
convert the frozen fields to data descriptors, so __setattr__ only
gets called for attempts to add new attributes.

Then for the `frozen=False` case, the decorator could force
__setattr__ and __delattr__ back to the default implementations from
object, rather than relying on the behaviour inherited from base
classes.

I guess it depends on what we're trying to do. By using data 
descriptors, we'd be saying "the fields can't be modified, but other 
parts of the class can". For example, what should happen here:

@dataclass(frozen=True)
class F:
   i: int

f = F(10)
f.i = 0   # should be an error
f.j = 0   # should this be an error? It's not a field.

I was hoping to get all of this finished today, by a2, but it looks like 
that's not going to happen.

A related issue is that dataclasses derived from frozen dataclasses are
automatically "promoted" to being frozen.

@dataclass(frozen=True)

... class A:
... i: int
...

@dataclass

... class B(A):
... j: int
...

b = B(1, 2)
b.j = 3

Traceback (most recent call last):
   File "", line 1, in 
   File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
_frozen_setattr
 raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

Maybe it should be an error to declare B as non-frozen?

It would be nice to avoid that, as a mutable subclass of a frozen base
class could be a nice way to model hashable-but-mutable types:

 >>> @dataclass(frozen=True) # This is the immutable/hashable bit
 ... class A:
 ... i: int
 ...
 >>> @dataclass # This adds the mutable-but-comparable parts
 ... class B(A):
 ... j: int
 ... __hash__ = A.__hash__

However, disallowing this case for now *would* be a reasonable way to
postpone making a decision until 3.8 - in the meantime, folks would
still be able to experiment by overriding __setattr__ and __delattr__
after the dataclass decorator sets them.

For today, I'm going to make it an error to derive a frozen dataclass 
from a non-frozen dataclass, and also an error to derive a non-frozen 
dataclass from a frozen dataclass. With any luck (and Guido and Ned 
willing), we can address this by a3.

Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-26 Thread Eric V. Smith

I've opened https://bugs.python.org/issue32953 to track this.

On 2/22/18 5:55 AM, Eric V. Smith wrote:

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:

When working on the docs for dataclasses, something unexpected came
up.  If a dataclass is specified to be frozen, that characteristic is
inherited by subclasses which prevents them from assigning additional
attributes:

 >>> @dataclass(frozen=True)
 class D:
 x: int = 10

 >>> class S(D):
 pass

 >>> s = S()
 >>> s.cached = True
 Traceback (most recent call last):
   File "", line 1, in 
 s.cached = True
   File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
line 448, in _frozen_setattr
 raise FrozenInstanceError(f'cannot assign to field {name!r}')
 dataclasses.FrozenInstanceError: cannot assign to field 'cached'

This is because "frozen-ness" is implemented by adding __setattr__ and
__delattr__ methods in D, which get inherited by S.

...

A related issue is that dataclasses derived from frozen dataclasses are
automatically "promoted" to being frozen.

@dataclass(frozen=True)

... class A:
... i: int
...

@dataclass

... class B(A):
... j: int
...

b = B(1, 2)
b.j = 3

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452,
in _frozen_setattr
raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

This is tricky to fix.

Here's the problem with a inheriting a non-frozen dataclass from a 
frozen one. Consider class Y in this example:

@dataclass
class X:
x: int

@dataclass
class Y(X):
y: int

Y's __init__ looks like:

def __init__(self, x, y):
self.x = x
self.y = y

That is, all of the initializing for Y and its base classes which are 
dataclasses is done in Y's __init__. There are a number of reasons for 
this, including performance and not knowing how to call the base 
classes' __init__ methods.

If Y is frozen, then the __init__ currently looks like:

def __init__(self, x, y):
object.__setattr__(self, 'x', x)
object.__setattr__(self, 'y', y)

If X is frozen but Y isn't, then it should look like:

def __init__(self, x, y):
object.__setattr__(self, 'x', x)
self.y = y

But I currently can't generate the code that way, because I don't know 
if a base dataclass is frozen. That information is not saved on a dataclass.

I think the right thing to do is to record with each dataclass if it is 
frozen or not, so that derived classes can generate the correct code. 
Another option would be to always use object.__setattr__, but that will 
hurt performance in the common case.

As long as I'm saving if a dataclass is frozen, I should save all of the 
dataclass parameters on the class. Since it's per-class, it's not a lot 
of overhead.

I should probably separate the two issues here: 1) deriving a 
non-dataclass from a frozen dataclass and 2) deriving a non-frozen 
dataclass from a frozen dataclass, but since they're somewhat related 
I'm going to keep them together for the time being. #1 was Raymond's 
initial report.

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Exhaustively test dataclass hashing when no hash= value is provided. This is in anticipation of changing how non-default hashing is handled. (GH-5834) (GH-5889)

2018-02-25 Thread Eric V. Smith


Thanks. Do you know if this gets backported to 3.7?

Eric.

On 2/25/2018 1:36 PM, Terry Reedy wrote:

On 2/25/2018 11:56 AM, Eric V. Smith wrote:


+    # specify any value in the deecorator).


I fixed this through the web interface.  The CI now knows that a comment 
change does not need testing.  I could not request a review from 
ericvsmith though.


tjr


___
Python-checkins mailing list
python-check...@python.org
https://mail.python.org/mailman/listinfo/python-checkins

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-23 Thread Eric V. Smith

On 2/22/18 9:43 PM, Nick Coghlan wrote:

On 22 February 2018 at 20:55, Eric V. Smith  wrote:

A related issue is that dataclasses derived from frozen dataclasses are
automatically "promoted" to being frozen.

@dataclass(frozen=True)

... class A:
... i: int
...

@dataclass

... class B(A):
... j: int
...

b = B(1, 2)
b.j = 3

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
_frozen_setattr
raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

Maybe it should be an error to declare B as non-frozen?

It would be nice to avoid that, as a mutable subclass of a frozen base
class could be a nice way to model hashable-but-mutable types:

>>> @dataclass(frozen=True) # This is the immutable/hashable bit
... class A:
... i: int
...
>>> @dataclass # This adds the mutable-but-comparable parts
... class B(A):
... j: int
... __hash__ = A.__hash__

However, disallowing this case for now *would* be a reasonable way to
postpone making a decision until 3.8 - in the meantime, folks would
still be able to experiment by overriding __setattr__ and __delattr__
after the dataclass decorator sets them.

Yes, that's what I was thinking. If we can't come up with a good 
solution for this now, make it an error and think about it for 3.8.

I'm doing some experimenting on this and Raymond's case and I'll post 
some ideas this weekend.

Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

2018-02-22 Thread Eric V. Smith

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:

When working on the docs for dataclasses, something unexpected came up.  If a 
dataclass is specified to be frozen, that characteristic is inherited by 
subclasses which prevents them from assigning additional attributes:

 >>> @dataclass(frozen=True)
 class D:
 x: int = 10

 >>> class S(D):
 pass

 >>> s = S()
 >>> s.cached = True
 Traceback (most recent call last):
   File "", line 1, in 
 s.cached = True
   File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
 line 448, in _frozen_setattr
 raise FrozenInstanceError(f'cannot assign to field {name!r}')
 dataclasses.FrozenInstanceError: cannot assign to field 'cached'

This is because "frozen-ness" is implemented by adding __setattr__ and 
__delattr__ methods in D, which get inherited by S.

Other immutable classes in Python don't behave the same way:

 >>> class T(tuple):
 pass

 >>> t = T([10, 20, 30])
 >>> t.cached = True

 >>> class F(frozenset):
 pass

 >>> f = F([10, 20, 30])
 >>> f.cached = True

 >>> class B(bytes):
 pass

 >>> b = B()
 >>> b.cached = True

The only way I can think of emulating this is checking in __setattr__ to 
see if the field name is a field of the frozen class, and only raising 
an error in that case.

A related issue is that dataclasses derived from frozen dataclasses are 
automatically "promoted" to being frozen.

>>> @dataclass(frozen=True)
... class A:
... i: int
...
>>> @dataclass
... class B(A):
... j: int
...
>>> b = B(1, 2)
>>> b.j = 3
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 
452, in _frozen_setattr

raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

Maybe it should be an error to declare B as non-frozen?

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses, frozen and __post_init__

2018-02-20 Thread Eric V. Smith


> On Feb 20, 2018, at 5:38 PM, Guido van Rossum  wrote:
> 
>> On Tue, Feb 20, 2018 at 1:37 PM, Eric V. Smith  wrote:
>>> On 2/17/2018 2:35 PM, Guido van Rossum wrote:
>>> PS. I have to ponder why frozen dataclasses don't use `__new__`.
>> 
>> As I'm sure everyone is now aware after the rest of this discussion: it's 
>> because the returned object isn't really immutable.
>> 
>> That said, I have threatened to create a decorator similar to 
>> typing.NamedTuple that has the @dataclass features (except maybe 
>> __post_init__) and returns a class that does inherit from tuple, in which 
>> case it really would use `__new__`. I'll save that for 3.8, if it ever 
>> happens. There's a lot there to think about, first. For now, 
>> typing.NamedTuple is the way to go if you want something based on namedtuple 
>> yet using type hints.
> 
> But then the class would also inherit a bunch of misfeatures from tuple (like 
> being indexable and having a length).

Right. That’s the “lots to think about” part. It might just be unworkable. 

> It would be nicer if it used __slots__ instead. (Also, the problem with 
> __slots__ is the same as the problem with inheriting from tuple, and it 
> should just be solved right, somehow.)

Agreed. 

Eric___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

< 1 2 3 4 5 6 >

101 - 200 of 538 matches

Mail list logo