Re: [Python-Dev] Slow down...

[Lukasz Langa]

> On May 7, 2018, at 9:57 AM, Brett Cannon  wrote:
> 
> 
> 
> On Mon, 7 May 2018 at 09:55 Ivan Levkivskyi  > wrote:
> On 7 May 2018 at 17:32, Brett Cannon  > wrote:
> On Mon, 7 May 2018 at 08:18 João Santos  > wrote:
> Hi,
> 
> I would like to see this go even further and have a tick-tock approach to 
> python versions, i.e. adopt new syntax and other large changes on one version 
> (for example odd versions) and polish everything up in the next (even 
> versions).
> 
> [...], so I would advise we stick to the discussion on a moratorium [...]
> 
> 
> Btw the upcoming Language Summit may be a good opportunity for such 
> discussion.
> 
> If it's not already on the schedule for discussion then the best you are 
> going to get is a lightning talk to bring up the idea which will definitely 
> not enough time ;) . Otherwise the schedule is full and locked down at this 
> point.

FWIW I'm hearing the 3.8 release manager has a short talk related to this 
subject!

- Ł___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[Lukasz Langa]

> On May 2, 2018, at 8:57 PM, INADA Naoki  wrote:
> 
> Recently, I reported how stdlib slows down `import requests`.
> https://github.com/requests/requests/issues/4315#issuecomment-385584974
> 
> For Python 3.8, my ideas for faster startup time are:
> 
> * Add lazy compiling API or flag in `re` module.  The pattern is compiled
> when first used.

How about go the other way and allow compiling at Python *compile*-time? That 
would actually make things faster instead of just moving the time spent around.

I do see value in being less eager in Python but I think the real wins are 
hiding behind ahead-of-time compilation.

- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Boundaries between numbers and identifiers

[Lukasz Langa]

> On Apr 26, 2018, at 11:37 AM, Serhiy Storchaka  wrote:
> 
> I propose to change the Python syntax by adding a requirement that there 
> should be a whitespace or delimiter between a numeric literal and the 
> following keyword.

-1

This would make Python 3.8 reject code due to stylistic preference. Code that 
it actually can unambiguously parse today.

I agree that a formatting style that omits whitespace between numerals and 
other tokens is terrible.  However, if you start downright rejecting it, you 
will likely punish the wrong people.  Users of third-party libraries will be 
met with random parsing errors in files they have no control over.  This is not 
helpful.

And given BPO-8 the standard library tokenizer would have to keep parsing 
those things as is.

Making 0or[] working again is also not worth it since that's been broken since 
Python 2.6 and hopefully nobody is running Python 2.5-only code anymore.

What we should instead is to make the standard library tokenizer reflect the 
behavior of Python 2.6+.

-- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reserve ':=' for type-inferred variable initialization (was PEP 572)

[Lukasz Langa]

> On Apr 26, 2018, at 11:00 AM, Fatty Morgan  wrote:
> 
> I would like to urge you to reconsider the use of the token ':='
> for assignment expressions.
> 
> The natural interpretation of 'name := expr' is a PEP 526
> type-annotated variable initialization 'name : T = expr' with the
> type annotation T omitted, the tokens ':' and '=' coalesced, and
> the implied type T inferred as 'type(expr)'.

PEP 484 type checkers *always* infer the type of an assignment
based on the right-hand side's value.

The main reason we require PEP 526 annotations at all is when
your initial assignment is not useful for this inference.

The two most common examples of this are `None` and empty
containers.  In those cases the type checker is unable to infer
what the *intended* value should be, so it urges the programmer
to explicitly state it.

In this sense, `a := []` or `a := None` doesn't provide any
additional information.  And `a = 1` is already enough to determine
what the expected type is.


> The following should probably be discussed in python-ideas and
> would only be relevant for a more distant future, but while I
> have your attention, here are a couple of thoughts:

Discussing this here will add to noise for Chris and the BDFL.
Consider re-posting in python-ideas.

-- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Lukasz Langa]

[Uncle T]
> So, to match your sarcasm, here's mine:  try using a feature for what
> it's good at instead of for what it's bad at ;-)

Yes, this is the fundamental wisdom.  Judging which is which is left as an
exercise to the programmer.

With this, I'm leaving the discussion.  With Guido and you on board for PEP
572, I feel that Chris' streak is indeed about to break.

Some parting hair-splitting follows.


> [Uncle T]
>>> One language feature conspicuous by absence in newbie
>>> confusions was, consistently, assignment expressions.  Read any book
>>> or tutorial for such a language, and you'll find very little space
>>> devoted to them too.
> 
> [Łukasz Langa ]
>> Well, you have an entire code style built around this feature called Yoda
>> conditions. You teach people on Day 1 to never ever confuse == with =. Some
>> compilers even warn about this because so many people did it wrong.
> 
[Uncle T]
> Sorry, I couldn't follow that.

You implied that newbies don't have to even know about assignments in
expressions.  I wanted to demonstrate that this isn't really the case because
mistaking `=` for `==` is a relatively common occurence for newbies.  If you
want to argue that it isn't, I'd like to point out that the WordPress code
style *requires* Yoda conditions because it was enough of a hindrance.  ESLint
(a JavaScript linter) also has a warning about assignment in a conditional.


[Uncle T]]
> In languages like C that use easily
> confused operator symbols, sure, people are forever typing "=" when
> they mean "==".  That's nothing to do with whether they _understand_
> what the different operators do, though.

What you're saying is true.  But for it to be true, newbies *have to* learn the
distinction, and the fact that yes, sometimes the programmer indeed meant to
put a single `=` sign in the conditional.  That's why we'll end up with the
Pascal assignment operator.  And that *is* a thing that you will have to
explain to newbies when they encounter it for the first time.  Sadly, googling
for a colon followed by an equal sign isn't trivial if you don't know what
you're looking for.


[Łukasz]
>> Well, you can also use it as a statement. But don't!
> 
[Uncle T]]
> Why not?  _Every_ expression in Python can be used as a statement.
> Nothing forbids it, and that's even (very!) useful at an interactive
> prompt.

Because it suggests different intent, because it's limited, because it's slower
at runtime, and because PEP 572 says so itself.


> At this point I think you must have a lower opinion of Python
> programmers than I have ;-)  If adding even a dozen characters to a
> line makes it exceed a reasonable line-length guide, the code was
> almost certainly too confusingly dense to begin with.

Around 5% of if and elif statements in the standard library don't fit a single
line *as is*.  Sure, that's a low percentage but that's over 1,000 statements.
If you're putting an `if` statement in a method, you are already starting out
with 71 characters left on the line.  Four of those are going to be taken by
"if", a space, and the colon.  Adding a parenthesized assignment expression
takes at least 10% of that available space.

The silver lining for me is that this makes the environment riper for
auto-formatting.


-- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3?

[Lukasz Langa]

> On Apr 2, 2018, at 2:13 PM, Antoine Pitrou <solip...@pitrou.net> wrote:
> 
> On Mon, 2 Apr 2018 13:48:46 -0700
> Lukasz Langa <luk...@langa.pl> wrote:
>> Pickle protocol version 4.0 was originally defined back in PEP 3154 and 
>> shipped as part of Python 3.4 back in 2011. Yet it's still not the default.
> 
> Because we want pickles produced with the default to be readable by
> earlier Python 3 versions.
> (the same reason protocol 0 stayed the default throughout the Python 2
> lifetime)

Alright, so that means we can easily do this for Python 3.8, right? I mean, 
following Christian's logic, Python 3.3 is already dead, with its final release 
done in February 2016 and support dropped in September 2017 per PEP 398.

I think we need to get past thinking about "Python 2" vs. "Python 3". This 
frame of mind creates space for another mythical release of Python that will 
break all the compatibilities, something we promised not to do. A moving 
backward compatibility window that includes the last release still under 
security fixes seems like a good new framework for this.

What do you think?

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Why is pickle.DEFAULT_PROTOCOL still 3?

[Lukasz Langa]

Pickle protocol version 4.0 was originally defined back in PEP 3154 and shipped 
as part of Python 3.4 back in 2011. Yet it's still not the default. There's a 
number of things that would run faster with it like multiprocessing.

This is too late for 3.7 which is a shame but can we at least bump it for 3.8?

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Lukasz Langa]

To add a counter-example that I'm painfully familiar with: the old Thrift for 
Python makes its (mutable) structures hashable. This is "useful" because you 
can memoize functions that take Thrift structures as arguments. You can key 
dictionaries with them. And so on.

Unfortunately, more often then not this same structure is passed around and 
inevitably gets mutated in place. Everything breaks. What should be memoized 
isn't (and in pathological cases, what shouldn't be memoized is), dictionary 
access *using the same object* raises unexpected key errors but iterating on 
the dictionary reveals the key.

It's clearly clowntown and in hindsight we shouldn't have enabled this 
behavior. But we can't go back since too much code relies on hashability now 
and it's "mostly" fine.

As a solution, the new asyncio-only Thrift implementation for Python uses 
C-level structures to make sure they are truly immutable in Python. We hash 
them and cache them like there's no tomorrow.

- Ł


> On Feb 5, 2018, at 11:54 PM, Ivan Levkivskyi  wrote:
> 
> Just wanted to add my 5 cents here. I am a bit surprised how people are 
> scared by adding `__hash__` to mutable classes.
> From my experience it is quite normal, I was always thinking about `hash()` 
> as hashing a _value_,
> and never as hashing _identity_, if I would need the latter, there is a 
> different function for this, `id()`.
> Moreover, I often did this in situations where dataclasses would be useful: a 
> class with several fields,
> necessary dunders, and few other methods (record-like classes).
> My motivation was mostly speed-up by memorization.
> To be fair my use cases were mostly about some tree-like strictures, but this 
> is probably a coincidence.
> 
> FWIW it is not a super-safe practice, but neither super-dangerous.
> 
> --
> Ivan
> 
> 
> 
> On 5 February 2018 at 22:56, Nick Coghlan  > wrote:
> On 6 February 2018 at 03:47, Guido van Rossum  > wrote:
> > If there's going to be an API for it, it should be in the class, not
> > something that mutates the class afterwards.
> 
> Something I realised after posting the __class__ setting idea is that
> you can actually use a comparable trick to inject an unsafe hash from
> the frozen version into the mutable version:
> 
> >>> from dataclasses import dataclass
> >>> @dataclass
> ... class Example:
> ... a: int
> ... b: int
> ...
> >>> c = Example(1, 2)
> >>> hash(c)
> Traceback (most recent call last):
>  File "", line 1, in 
> TypeError: unhashable type: 'Example'
> 
> >>> @dataclass(frozen=True)
> ... class LockedExample(Example):
> ... pass
> ...
> >>> Example.__hash__ = LockedExample.__hash__
> >>> hash(c)
> 3713081631934410656
> 
> So "unsafe_hash=True" would just be a shorthand spelling of that which
> skips creating the full frozen version of the class (and with the
> explicit parameter, we can better document the risks of making
> something hashable without also freezing it post-creation).
> 
> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   ncogh...@gmail.com    |   
> Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/levkivskyi%40gmail.com 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl 
> 


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guido's Python 1.0.0 Announcement from 27 Jan 1994

[Lukasz Langa]

> On 27 Jan, 2018, at 5:10 PM, Dan Stromberg  wrote:
> 
> We probably should (if possible) create an archive (with dates) of
> very old (or all, actually) versions of CPython, analogous to what The
> Unix Heritage Society does for V5, V7, etc., but for CPython...
> 
> Or is there one already?  I found a bunch of 1.x's, but no 0.x's.
> What I found was at http://legacy.python.org/download/releases/src/

If I remember correctly, Dave Beazley, who went on this particular adventure a 
few months back, concluded that other releases are lost forever due to FTPs and 
their mirrors going offline over time. He did find a tarball of 0.9.1 
reconstructed by Andrew Dalke from usenet posts.

Read on, this is pretty fascinating: 
https://twitter.com/dabeaz/status/934590421984075776 


- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guido's Python 1.0.0 Announcement from 27 Jan 1994

[Lukasz Langa]

> On 27 Jan, 2018, at 12:52 PM, Simon Cross  
> wrote:
> 
> Python Party Proposal!

Oh, that's okay then. For a second there I got reminded of the dreadful days of 
trying to get dial-up to work on Linux with a winmodem. PPP. Shudder.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Merging the implementation of PEP 563

[Lukasz Langa]

Hi all,
Serhiy looks busy these days. I'd appreciate somebody looking at and hopefully 
merging https://github.com/python/cpython/pull/4390 
. Everything there was reviewed by 
Serhiy except for the latest commit.

This should be ready to merge and maybe tweak in the beta stage. I'd like to 
avoid merging it myself but I'd really hate missing the deadline.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] HP-UX pr not feeling the love

[Lukasz Langa]

Hi Rob,
thanks for your patch. CPython core developers, as volunteers, have limited 
resources available to maintain Python. Those resources are not only time, they 
are also mental resources necessary to make a change in Python as well as 
actual physical resources. Supporting a platform requires all three:

1. You need time to make a platform work initially, and then continuous effort 
to keep it working, fixing regressions, including this platform in new 
features, etc.
2. You need mental resources to manage additional complexity that comes from 
#ifdef sprinkled through the code, cryptic configure/Makefile machinery, etc.
3. You need access to machines running the given operating system to be able to 
test if your changes are compatible.

This is why we are keeping the list of supported platforms relatively short. In 
fact, in time we're cutting support for less popular platforms that we couldn't 
keep running. Details in https://www.python.org/dev/peps/pep-0011/ 
. Look, just in 3.7 we're dropping 
IRIX and systems without threads.

As you're saying, while your current PR is relatively innocent, more are needed 
to make it work. If those require more drastic changes in our codebase, we 
won't be able to accept them due to reasons stated above.

I understand where you're coming from. If you're serious about this, we would 
need to see the full extent of changes required to make Python 3.7 work on HP 
UX, preferably minimal. We would also need a buildbot added to our fleet (see 
http://buildbot.python.org/ ) that would ensure 
the build stays green. Finally, we would need you to think whether you could 
provide the patches that keep the build green for a significant period of time 
(counted in years).

- Ł



> On Dec 6, 2017, at 7:22 AM, Rob Boehne  wrote:
> 
> Hello,
> 
> Back in June I was fired up to get my diverse set of platforms all running 
> Python 3, but quickly ran into issues and submitted a PR.
> 
> https://github.com/python/cpython/pull/2519 
> 
> 
> It seems as though this HP-UX specific change isn’t getting much 
> consideration, which probably isn’t a big deal.  What may be more important 
> is that I’ve stopped trying to contribute, and if I really need Python 3 on 
> HP-UX, AIX, Sparc Solaris or other operating systems, I’ll have to hack it 
> together myself and maintain  my own fork, while presumably others do the 
> same.  At the same time I’m working hard to convince management that we 
> shouldn’t create technical debt by maintaining patches to all the tools we 
> use, and that we should get these changes accepted into the upstream repos.
> 
> Could someone have a look at this PR and possibly merge?
> 
> Thanks,
> 
> Rob Boehne
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev 
> 
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl 
> 



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Lukasz Langa]

+1 to (3), I like the type error idea, too.

I don't care much about naming... but if I were to bikeshed this, I'd go for

isdataclass (like issubclass)
isdatainstance (like isinstance)

- Ł


> On Nov 30, 2017, at 12:35 PM, Carl Meyer  wrote:
> 
> On 11/29/2017 05:02 PM, Guido van Rossum wrote:
>> I tried to look up the discussion but didn't find much except that you
>> flagged this as an issue. To repeat, your concern is that isdataclass()
>> applies to *instances*, not classes, which is how Eric has designed it,
>> but you worry that either through the name or just because people don't
>> read the docs it will be confusing. What do you suppose we do? I think
>> making it work for classes as well as for instances would cause another
>> category of bugs (confusion between cases where a class is needed vs. an
>> instance abound in other situations -- we don't want to add to that).
>> Maybe it should raise TypeError when passed a class (unless its
>> metaclass is a dataclass)? Maybe it should be renamed to
>> isdataclassinstance()? That's a mouthful, but I don't know how common
>> the need to call this is, and people who call it a lot can define their
>> own shorter alias.
> 
> Yeah, I didn't propose a specific fix because I think there are several
> options (all mentioned in this thread already), and I don't really have
> strong feelings about them:
> 
> 1) Keep the existing function and name, let it handle either classes or
> instances. I agree that this is probably not the best option available,
> though IMO it's still marginally better than the status quo).
> 
> 2) Punt the problem by removing the function; don't add it to the public
> API at all until we have demonstrated demand.
> 
> 3) Rename it to "is_dataclass_instance" (and maybe also keep a separate
> "is_dataclass" for testing classes directly). (Then there's also the
> choice about raising TypeError vs just returning False if a function is
> given the wrong type; I think TypeError is better.)
> 
> Carl
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] What's the status of PEP 505: None-aware operators?

[Lukasz Langa]

Hi Mark,
it looks like the PEP is dormant for over two years now. I had multiple people 
ask me over the past few days about it though, so I wanted to ask if this is 
moving forward.

I also cc python-dev to see if anybody here is strongly in favor or against 
this inclusion. If the idea itself is uncontroversial, I could likely find 
somebody interested in implementing it. If not for 3.7 then at least for 3.8.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 563: Postponed Evaluation of Annotations (Draft 3)

[Lukasz Langa]

Based on the feedback I gather in early November,
I'm publishing the third draft for consideration on python-dev.
I hope you like it!

A nicely formatted rendering is available here:
https://www.python.org/dev/peps/pep-0563/

The full list of changes between this version and the previous draft
can be found here:
https://github.com/ambv/static-annotations/compare/python-dev1...python-dev2

- Ł



PEP: 563
Title: Postponed Evaluation of Annotations
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa 
Discussions-To: Python-Dev 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Sep-2017
Python-Version: 3.7
Post-History: 1-Nov-2017, 21-Nov-2017
Resolution:


Abstract


PEP 3107 introduced syntax for function annotations, but the semantics
were deliberately left undefined.  PEP 484 introduced a standard meaning
to annotations: type hints.  PEP 526 defined variable annotations,
explicitly tying them with the type hinting use case.

This PEP proposes changing function annotations and variable annotations
so that they are no longer evaluated at function definition time.
Instead, they are preserved in ``__annotations__`` in string form.

This change is going to be introduced gradually, starting with a new
``__future__`` import in Python 3.7.


Rationale and Goals
===

PEP 3107 added support for arbitrary annotations on parts of a function
definition.  Just like default values, annotations are evaluated at
function definition time.  This creates a number of issues for the type
hinting use case:

* forward references: when a type hint contains names that have not been
  defined yet, that definition needs to be expressed as a string
  literal;

* type hints are executed at module import time, which is not
  computationally free.

Postponing the evaluation of annotations solves both problems.

Non-goals
-

Just like in PEP 484 and PEP 526, it should be emphasized that **Python
will remain a dynamically typed language, and the authors have no desire
to ever make type hints mandatory, even by convention.**

This PEP is meant to solve the problem of forward references in type
annotations.  There are still cases outside of annotations where
forward references will require usage of string literals.  Those are
listed in a later section of this document.

Annotations without forced evaluation enable opportunities to improve
the syntax of type hints.  This idea will require its own separate PEP
and is not discussed further in this document.

Non-typing usage of annotations
---

While annotations are still available for arbitrary use besides type
checking, it is worth mentioning that the design of this PEP, as well
as its precursors (PEP 484 and PEP 526), is predominantly motivated by
the type hinting use case.

In Python 3.8 PEP 484 will graduate from provisional status.  Other
enhancements to the Python programming language like PEP 544, PEP 557,
or PEP 560, are already being built on this basis as they depend on
type annotations and the ``typing`` module as defined by PEP 484.
In fact, the reason PEP 484 is staying provisional in Python 3.7 is to
enable rapid evolution for another release cycle that some of the
aforementioned enhancements require.

With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.


Implementation
==

In Python 4.0, function and variable annotations will no longer be
evaluated at definition time.  Instead, a string form will be preserved
in the respective ``__annotations__`` dictionary.  Static type checkers
will see no difference in behavior, whereas tools using annotations at
runtime will have to perform postponed evaluation.

The string form is obtained from the AST during the compilation step,
which means that the string form might not preserve the exact formatting
of the source.  Note: if an annotation was a string literal already, it
will still be wrapped in a string.

Annotations need to be syntactically valid Python expressions, also when
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
Annotations can only use names present in the module scope as postponed
evaluation using local names is not reliable (with the sole exception of
class-level names resolved by ``typing.get_type_hints()``).

Note that as per PEP 526, local variable annotations are not evaluated
at all since they are not accessible outside of the function's closure.

Enabling the future behavior in Python 3.7
--

The functionality described above can be enabled starting from Python
3.7 using the following special import::

from __future__ import annotations

A reference implementation of this functionality is available
`on GitHub `_.


Resolving Type Hints at Runtime
===

To resolve an 

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

See my response to Mark's e-mail. I agree that special-casing outermost strings 
is not generic enough of a solution and will be removing this special handling 
from the PEP and the implementation.

- Ł

> On Nov 19, 2017, at 2:38 PM, Koos Zevenhoven  wrote:
> 
> On Mon, Nov 13, 2017 at 11:59 PM, Brett Cannon  > wrote:
> ​[..]​
> On Sun, Nov 12, 2017, 10:22 Koos Zevenhoven,  > wrote:​​
> 
> There's two thing I don't understand here:
> 
> * What does it mean to preserve the string verbatim? No matter how I read it, 
> I can't tell if it's with quotes or without.
> 
> Maybe I'm missing some context.
> 
> I believe the string passes through unchanged (i.e. no quotes). Think of the 
> PEP as simply turning all non-string annotations into string ones.
> 
> 
> ​Ok, maybe that was just wishful thinking on my part ;-).
> 
> More info in the other threads, for example:
> 
> https://mail.python.org/pipermail/python-dev/2017-November/150642.html 
> 
> https://mail.python.org/pipermail/python-dev/2017-November/150637.html 
> 
> 
> -- Koos
> 
> 
> --
> + Koos Zevenhoven + http://twitter.com/k7hoven  +
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 563 (Postponed Evaluation of Annotations)

[Lukasz Langa]

I agree with you. The special handling of outermost strings vs. strings 
embedded inside annotations bugged me a lot. Now you convinced me that this 
functionality should be moved to `get_type_hints()` and the __future__ import 
shouldn't try to special-case this one instance, while leaving others as is.

I will be amending the PEP accordingly.

- Ł

> On Nov 19, 2017, at 10:56 AM, Mark Shannon  wrote:
> 
> Hi,
> 
> Overall I am strongly in favour of this PEP. It pretty much cures all the 
> ongoing pain of using PEP 3017 annotations for type hints.
> 
> There is one thing I don't like however, and that is treating strings as if 
> the quotes weren't there.
> While this seems like a superficial simplification to make transition easier, 
> it introduces inconsistency and will ultimately make both implementing and 
> using type hints harder.
> 
> Having the treatment of strings depend on their depth in the AST seems 
> confusing and unnecessary:
> "List[int]" becomes 'List[int]' # quotes removed
> but
> List["int"] becomes 'List["int"]' # quoted retained
> 
> Also,
> 
> T = "My unparseable annotation"
> def f()->T: pass
> 
> would remain legal, but
> 
> def f()->"My unparseable annotation"
> 
> would become illegal.
> 
> The change in behaviour between the above two code snippets is already 
> confusing enough without making one of them a SyntaxError.
> 
> Using annotations for purposes other than type hinting is legal and has been 
> for quite a while.
> Also, PEP 484 type-hints are not the only type system in the Python 
> ecosystem. Cython has a long history of using static type hints.
> 
> For tools other than MyPy, the inconsistent quoting is onerous and will 
> require double-quoting to prevent a parse error.
> For example
> 
> def foo()->"unsigned int": ...
> 
> will become illegal and require the cumbersome
> 
> def foo()->'"unsigned int"': ...
> 
> Cheers,
> Mark.
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Show DeprecationWarning in debug mode?

[Lukasz Langa]

Merged. Thanks! ✨  ✨

- Ł

> On Nov 20, 2017, at 7:01 AM, Victor Stinner  wrote:
> 
> 2017-11-18 18:13 GMT+01:00 Brett Cannon :
>> +1 from me as well.
> 
> Ok, I created https://bugs.python.org/issue32088 and
> https://github.com/python/cpython/pull/4474 to implement the proposed
> change.
> 
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On 10 Nov, 2017, at 4:48 PM, Guido van Rossum <gu...@python.org> wrote:
> 
> On Fri, Nov 10, 2017 at 1:20 AM, Lukasz Langa <luk...@langa.pl 
> <mailto:luk...@langa.pl>> wrote:
> Alright, we're on bikeshed territory now. Finally! :-)
> 
> I was always thinking about this as "static annotations". The fact they're 
> strings at runtime is irrelevant for most people who will use this future. 
> They don't want string annotations, they want them to not be evaluated on 
> import time... they want them to be static. Also, "static typing" et al. I 
> think it has a nice vibe to it.
> 
> I admit "annotations" is too broad but "static_annotations" (or 
> "string_annotations" ¯\_(ツ)_/¯ ) will be the longest __future__ name so far. 
> That was my main motivation behind using the shorter name. And a bit of 
> megalomania I guess.
> 
> I don't mind the long name. Of all the options so far I really only like 
> 'string_annotations' so let's go with that.

Done.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

Alright, we're on bikeshed territory now. Finally! :-)

I was always thinking about this as "static annotations". The fact they're 
strings at runtime is irrelevant for most people who will use this future. They 
don't want string annotations, they want them to not be evaluated on import 
time... they want them to be static. Also, "static typing" et al. I think it 
has a nice vibe to it.

I admit "annotations" is too broad but "static_annotations" (or 
"string_annotations" ¯\_(ツ)_/¯ ) will be the longest __future__ name so far. 
That was my main motivation behind using the shorter name. And a bit of 
megalomania I guess.

- Ł

> On 9 Nov, 2017, at 7:30 PM, Guido van Rossum  wrote:
> 
> So... Łukasz?
> 
> On Thu, Nov 9, 2017 at 6:11 PM, Nick Coghlan  > wrote:
> On 10 November 2017 at 05:51, Guido van Rossum  > wrote:
> > If we have to change the name I'd vote for string_annotations -- "lazy" has
> > too many other connotations (e.g. it might cause people to think it's the
> > thunks). I find str_annotations too abbreviated, and stringify_annotations
> > is too hard to spell.
> 
> Aye, I'd be fine with "from __future__ import string_annotations" -
> that's even more explicitly self-documenting than either of my
> suggestions.
> 
> --
> --Guido van Rossum (python.org/~guido )



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Reminder: PEP 479's __future__ about to become the default behavior

[Lukasz Langa]

This is according to https://www.python.org/dev/peps/pep-0479/#transition-plan 
 but looking at 
Objects/genobject.c that hasn't been implemented yet. Is this as simple as 
removing the `else` clause here?

https://github.com/python/cpython/blob/master/Objects/genobject.c#L277-L298 


- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 7, 2017, at 2:33 PM, Nick Coghlan <ncogh...@gmail.com> wrote:
> 
> On 8 November 2017 at 04:41, Lukasz Langa <luk...@langa.pl> wrote:
>> 4. How do we even version this library then? Probably like this: 3.7.0.0,
>> 3.7.0.1, 3.7.1.0, and so on. But that depends on answers to the other
>> questions above.
> 
> Something you may want to consider is switching to CalVer for typing
> itself, such that we end up saying something like "Python 3.7.0
> includes typing 2017.12.1".

You don't need to sell me on CalVer, all my private packages use this 
versioning scheme (just with the shorthand 17) :-)

And yes, this is a good suggestion.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 7, 2017, at 9:39 AM, Brett Cannon  wrote:
> 
> On Tue, 7 Nov 2017 at 03:34 Paul Moore  > wrote:
> 
> Because type annotations are a development-time feature, they should
> *not* require a dependency in the final deployment (apart from Python
> itself). However, because they are a language syntax feature they are
> of necessity written in the application source. And type specification
> of anything more complex than basic types (for example, List[int])
> requires classes defined in the typing module. Therefore, typing must
> be in the stdlib so that use of type annotations by the developer
> doesn't impose a runtime dependency on the end user.
> 
> That's not necessarily true if Lukasz's PEP lands and annotations become 
> strings. The dependency would only need to be installed if someone chose to 
> introspect the annotations and then "instantiate" them into actual objects. 
> And that only comes up if someone does it from outside by a 3rd-party, who 
> would then need to install the type annotation dependencies themselves.

PEP 563 is the first step there but it's not enough. For this idea to work, 
`typing.TYPE_CHECKING` would need to be moved to the Python runtime, so that 
you can do

if TYPE_CHECKING:
from typing import *

More importantly, you would put type aliases, type variables and new-types in 
this `if` block, too.

But even if all this is true, there's still two remaining features that require 
runtime availability:

1. `cast()`; and
2. creating generic classes by subclassing `Generic[T]` or any of its 
subclasses. And soon enough, Protocols.

I hope I'm not forgetting any other cases. For `cast()` I have an idea how to 
move it to a variable annotation. For generic classes, there's no magic, you 
need `typing` at runtime. Fortunately, that latter case is (I hope?) relatively 
unlikely.

All the above is summarized here:
https://github.com/python/typing/issues/496 


- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 6, 2017, at 8:01 PM, Nick Coghlan  wrote:
> 
> As I indicated in my comment there, I'm now wondering if there might
> be an opportunity here whereby we could use the *dataclasses* module
> to define a stable non-provisional syntactically compatible subset of
> the typing module, and require folks to start explicitly depending on
> the typing module (as a regular PyPI dependency) if they want anything
> more sophisticated than that (Unions, Generics, TypeVars, etc).

I have an issue open about essentially this idea:
https://github.com/python/typing/issues/496


The consensus is that this is too expensive to do this in time for Python 3.7. 
On the other hand, not doing anything is terrible in terms of usability: now 
users will be forced to use both `typing` (a frozen non-provisional version in 
the stdlib) and `typing_extensions` (new features that we couldn't add directly 
to `typing`).


This is the reason why Guido asked about moving `typing` out of the standard 
library. Currently it seems that the best thing that we can do is to ship an 
upgradeable version of `typing` with Python 3.7.0. There are many details to be 
figured out here:

1. Does that mean `typing` is going to be provisional forever? The answer is 
surprisingly "yes" since it makes sense to keep updating the bundled `typing` 
in 3.7.1, 3.7.2, etc. We need to update PEP 411 to talk about this new case.

2. Does that mean `typing` can now become backwards incompatible randomly? The 
answer should be "no" and already is "no". If you look at typing.py, it already 
ships in Python 3.6 with some crazy dances that makes it work even on Python 
3.3.

3. Does that mean that Debian is going to rip it out and make people install a 
`python-typing` .deb? Sadly, probably yes. We need to figure out what that 
means for us.

4. How do we even version this library then? Probably like this: 3.7.0.0, 
3.7.0.1, 3.7.1.0, and so on. But that depends on answers to the other questions 
above.

5. What will happen to typing_extensions and mypy_extensions? They can probably 
be folded into typing 3.7.0.0. This is a huge win for usability.

6. Where will we track issues for it? Which repo will be the source of truth? 
Fortunately, we already have https://github.com/python/typing/ 



There's more things to figure out for sure. And the clock is ticking. But this 
is a great case study to figure out since more packages in the standard library 
could use a release model like this, and maybe in the future this will finally 
lead to unbundling of the standard library from the runtime. Yeah, it's a long 
shot and the dependency graph between libraries in the standard library is 
rather extensive. But this is the first step.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On Nov 5, 2017, at 11:28 PM, Nick Coghlan <ncogh...@gmail.com> wrote:
> 
> On 6 November 2017 at 16:36, Lukasz Langa <luk...@langa.pl> wrote:
> 
> - compile annotations like a small nested class body (but returning
> the expression result, rather than None)
> - emit MAKE_THUNK instead of the expression's opcodes
> - emit STORE_ANNOTATION as usual
> 

Is the motivation behind creating thunks vs. reusing lambdas just the 
difference in handling class-level scope? If so, would it be possible to just 
modify lambdas to behave thunk-like there? It sounds like this would strictly 
broaden the functionality of lambdas, in other words, wouldn't create backwards 
incompatibility for existing code.

Reusing lambdas (with extending them to support class-level scoping) would be a 
less scary endeavor than introducing a brand new language construct.

With my current understanding I still think stringification is both easier to 
implement and understand by end users. The main usability win of thunks/lambdas 
is not very significant: evaluating them is as easy as calling them whereas 
strings require typing.get_type_hints(). I still think being able to access 
function-local state at time of definition is only theoretically useful.

What would be significant though is if thunk/lambdas helped fixing forward 
references in general. But I can't really see how that could work.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On 5 Nov, 2017, at 9:55 PM, Nick Coghlan  wrote:
> 
> Python's name resolution rules are already ridiculously complicated,
> and PEP 563 is proposing to make them *even worse*, purely for the
> sake of an optional feature primarily of interest to large enterprise
> users.

Solving forward references in type annotations is one of the two explicit goals 
of the PEP. That alone changes how name resolution works. It sounds like you're 
-1 on that idea alone?


> That's enough to leave nested classes as the main problematic case,
> since they can't see each other's attributes by default, and the
> proposed injected locals semantics in PEP 563 don't get this right
> either (they only account for MRO-based name resolution, not lexical
> nesting, even though the PEP claims the latter is expected to work)


You're right! I went too far here. I meant to be as compatible as possible with 
how regular attribute access works for nested classes. Originally I didn't want 
to support any locals at all for simplicity. I was convinced this is important 
(documented in "Rejected ideas") but overdid the example.

Since your example that illustrates the problem shows that those things fail 
for regular attribute lookup, too, that can be simply fixed in the PEP. I did 
that here:
https://github.com/ambv/static-annotations/blob/master/pep-0563.rst#backwards-compatibility
 


Note that I am still including the "def method(self) -> D.field2:" example as 
valid as including a class' own name in the locals() chain map provided by 
get_type_hints() is going to be trivial. In fact, I think I'll add this to 
get_type_hints() independently of this PEP since users of string literal 
forward references probably don't expect it to fail for nested classes.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On 4 Nov, 2017, at 6:32 PM, Nick Coghlan  wrote:
> 
> The PEP's current attitude towards this is "Yes, it will break, but
> that's OK, because it doesn't matter for the type annotation use case,
> since static analysers will still understand it". Adopting such a
> cavalier approach towards backwards compatibility with behaviour that
> has been supported since Python 3.0 *isn't OK*, since it would mean we
> were taking the step from "type annotations are the primary use case"
> to "Other use cases for function annotations are no longer supported".

Well, this is what the PEP literally says in "Deprecation policy":

> In Python 4.0 this will become the default behavior. Use of annotations 
> incompatible with this PEP is no longer supported.

The rationale here is that type annotations as defined by PEP 484 and others is 
the only notable use case. Note that "type annotations" includes things like 
data classes, auto_attribs in attrs, the dependency injection frameworks 
mentioned before, etc. Those are compatible with PEP 484. So, despite the open 
nature of annotations since Python 3.0, no alternative use case emerged that 
requires eager evaluation and access to local state. PEP 563 is addressing the 
pragmatic issue of improving usability of type annotations, instead of worrying 
about some unknown theoretically possible use case.

While function annotations were open to arbitrary use, typing was increasingly 
hinted (pun not intended) as *the* use case for them:

1. From Day 1, type checking is listed as the number one intended use case in 
PEP 3107 (and most others listed there are essentially type annotations by any 
other name).
2. PEP 484 says "We do hope that type hints will eventually become the sole use 
for annotations", and that "In order for maximal compatibility with offline 
type checking it may eventually be a good idea to change interfaces that rely 
on annotations to switch to a different mechanism, for example a decorator."
3. Variable annotations in PEP 526 were designed with type annotations as the 
sole stated purpose.

PEP 563 simply brings this multi-PEP dance to its logical conclusion, stating 
in "Rationale and Goals" that "uses for annotations incompatible with the 
aforementioned PEPs should be considered deprecated." The timeline for full 
deprecation is Python 4.0.


> The only workaround I can see for that breakage is that instead of
> using strings, we could instead define a new "thunk" type that
> consists of two things:
> 
> 1. A code object to be run with eval()
> 2. A dictionary mapping from variable names to closure cells (or None
> for not yet resolved references to globals and builtins)

This is intriguing.

1. Would that only be used for type annotations? Any other interesting things 
we could do with them?
2. It feels to me like that would make annotations *heavier* at runtime instead 
of leaner, since now we're forcing the relevant closures to stay in memory.
3. This form of lazy evaluation seems pretty implicit to me for the reader. 
Peter Ludemann's example of magic logging.debug() is a case in point here.

All in all, unless somebody else is ready to step up and write the PEP on this 
subject (and its implementation) right now, I think this idea will miss Python 
3.7.


> Now, even without the introduction of the IndirectAttributeCell
> concept, this is amenable to a pretty simple workaround:
> 
> A = Optional[int]
> class C:
>field: A = 1
>def method(self, arg: A) -> None: ...
>C.A = A
>del A

This is a poor workaround, worse in fact than using a string literal as a 
forward reference. This is more verbose and error-prone. Decorators address the 
same construct and their wild popularity suggests that this notation is 
inferior.


> But I genuinely can't see how breaking annotation evaluation at class
> scope can be seen as a deal-breaker for the implicit lambda based
> approach without breaking annotation evaluation for nested functions
> also being seen as a deal-breaker for the string based approach.

The main reason to use type annotations is readability, just like decorators. 
While there's nothing stopping the programmer to write:

class C: ...
def method(self, arg1): ...
method.__annotations__ = {'arg1': str, 'return': int}
C.__annotations__ = {'attribute1': ...}

...this notation doesn't fit the bill. Since nested classes and types embedded 
as class attributes are popular among type hinting users, supporting this case 
is a no-brainer. On the other hand, if you have a factory function that 
generates some class or function, then you either:

1. Use annotations in the generated class/function for type checking; OR
2. Add annotations in the generated class/function for them to be preserved in 
__annotations__ for some future runtime use.

In the former case, you are unlikely to use local state. But even if you were, 
that doesn't matter since the static type checker 

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On 4 Nov, 2017, at 11:43 AM, Peter Ludemann via Python-Dev 
>  wrote:
> 
> If type annotations are treated like implicit lambdas, then that's a first 
> step to something similar to Lisp's "special forms". A full generalization of 
> that would allow, for example, logging.debug to not evaluate its args unless 
> debugging is turned on (I use a logging.debug wrapper that allows lambdas as 
> args, and evaluates them if debugging is turned on).

Interestingly enough, at Facebook we found out that using f-strings is *faster* 
at runtime than the lazy form of logging.log("format with %s and %d", arg1, 
arg2), including for cases when the log message is not emitted.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On 4 Nov, 2017, at 3:39 AM, Paul Moore <p.f.mo...@gmail.com> wrote:
> 
> Lukasz Langa said:
>> So, the difference is in perceived usability. It's psychological.
> 
> Please, let's not start the "not in the stdlib isn't an issue" debate
> again. If I concede it's a psychological issue, will you concede that
> the fact that it's psychological doesn't mean that it's not a real,
> difficult to solve, problem for some people? I'm also willing to
> concede that it's a *minority* problem, if that helps. But can we stop
> dismissing it as a non-existent problem?

Paul, if you read the words I wrote in my e-mail verbatim, you will note that I 
am not saying it's not real or it's not important. Quite the opposite. Can you 
elaborate what made you think that my assertion that the issue is psychological 
made you think I'm being dismissive? To me it looks like you're aggressively 
agreeing with me, so I'd like to understand what caused your reaction.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default

[Lukasz Langa]

> On 5 Nov, 2017, at 6:29 PM, Oleg Broytman  wrote:
> 
> On Sun, Nov 05, 2017 at 09:20:12PM -0500, Yury Selivanov 
>  wrote:
>> Big +1 from me.  The whole point of DeprecationWarnings is to be
>> visible
> 
> The whole point of DeprecationWarnings is to be visible to
> developers while in reality they will be visible to users -- and what
> the users would do with the warnings?

Complain to the authors and make them remove the issue.

https://github.com/requests/requests/issues/3954 

https://github.com/scikit-learn-contrib/sklearn-pandas/issues/76 

https://github.com/pandas-dev/pandas/issues/5824 

https://github.com/pypa/setuptools/issues/472 



+1 to re-enable this from me, too. At Facebook we are running unit tests and 
development builds with warnings. Just be aware that for some applications that 
will spew a lot. Python 3.6's warnings on invalid escapes is a big source of 
this.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On 3 Nov, 2017, at 4:01 PM, Victor Stinner  wrote:
> 
> The question is if you would only need  or  pip install typing>.
> 
> If typing is removed from the stdlib, you can still use it in your
> application. It's "just" another dependency no?


The ideal situation is that something is built-in and just works, examples: 
dicts, lists, sorted().

So, if you have to import it to use it, it's still great but less seamless, 
current example: regular expressions. Let's say Guido suggests we should import 
sorted, dict, and list before use. Not a big deal, right? I mean, how many 
applications do you know that don't use any other imports?

Finally, if you have to find a third-party package, add it to requirements.txt 
and manage the dependency forward, that's even less seamless. The standard 
library has a pretty conservative approach to backwards compatibility. On the 
other hand, third-party libraries often don't. Sure, there's noble exceptions 
but the general feel is that you need to be more careful with dependencies from 
PyPI. If somebody suggested that regular expressions or dictionaries should be 
moved to PyPI, in my book that would suggest strange things will start 
happening in the future.

So, the difference is in perceived usability. It's psychological.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

> On Nov 2, 2017, at 1:13 PM, Stéfane Fermigier  wrote:
> 
> Another common use case is dependency injection / IoC:
> 
> - Injector (https://github.com/alecthomas/injector 
> ):
> - Flask-Injector (ok it's the same underlying injector):

Pretty cool! This is already using `get_type_hints()` so it's perfectly 
compatible with PEP 563:

https://github.com/alecthomas/injector/blob/master/injector.py#L915

> - Apistar components (https://github.com/encode/apistar#components 
>  ):

This is using `inspect` directly so will have to migrate to `get_type_hints()` 
later. But, as the PEP mentions, it should already be using `get_type_hints()` 
since people are free to use forward references even today.

Thanks for flagging those use cases!


- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Migrate python-dev to Mailman 3?

[Lukasz Langa]

+1 committers

> On Nov 1, 2017, at 7:54 PM, Barry Warsaw  wrote:
> 
> On Nov 1, 2017, at 19:42, Guido van Rossum  wrote:
>> 
>> Maybe we should try it on some other list too? I know it works "in 
>> principle" and I'd love for all Python mailing lists to migrate, but I'd 
>> like to have some more experience with community mailing lists before 
>> tackling python-dev.
> 
> What about core-workflow or committers?  I think some of the criteria are:
> 
> * Gets a fair bit of traffic, but not too much
> * Is okay with a little bit of downtime for the migration
> * Willing to put up with any transient migration snafus
> * Amenable to trying the new UI
> * Has the BDFL as a member :)
> 
> -Barry
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Migrate python-dev to Mailman 3?

[Lukasz Langa]

> On Oct 30, 2017, at 4:00 AM, Victor Stinner  wrote:
> 
> Except of Antoine Pitrou, does everybody else like the new UI? :-)

I also much prefer MM3 and HyperKitty.

The old pipermail tree looks more inviting (I like the concise tree) but it's 
deceiving. When you actually start going through an entire discussion, it's 
easy to lose track unless posters use quotations neatly.

The search functionality of HyperKitty alone is worth it.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 530

[Lukasz Langa]

The original poster is an elementary school student. To keep the list clean, I 
responded 1:1 in a more inviting manner. Hopefully the person will succeed 
installing and learning Python :-)

- Ł

> On Oct 28, 2017, at 2:33 PM, Terry Reedy  wrote:
> 
> On 10/27/2017 4:43 PM, London wrote:
>> can you help me get idol for my computer
> 
> Post questions about using python on python-list and include information 
> about what OS you are running and what version of Python you want.
> 
> --
> Terry Jan Reedy
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 511 (code transformers) rejected

[Lukasz Langa]

I find this sad. In the JavaScript community the existence of Babel is very 
important for the long-term evolution of the language independently from the 
runtime. With Babel, JavaScript programmers can utilize new language syntax 
while being able to deploy on dated browsers. While there's always some 
experimentation, I doubt our community would abuse the new syntactic freedom 
that the PEP provided.

Then again, maybe we should do what Babel did, e.g. release a tool like it 
totally separately from the runtime.

- Ł

> On Oct 17, 2017, at 1:23 PM, Victor Stinner  wrote:
> 
> Hi,
> 
> I rejected my own PEP 511 "API for code transformers" that I wrote in
> January 2016:
> 
> https://github.com/python/peps/commit/9d8fd950014a80324791d7dae3c130b1b64fdace
> 
> Rejection Notice:
> 
> """
> This PEP was rejected by its author.
> 
> This PEP was seen as blessing new Python-like programming languages
> which are close but incompatible with the regular Python language. It
> was decided to not promote syntaxes incompatible with Python.
> 
> This PEP was also seen as a nice tool to experiment new Python features,
> but it is already possible to experiment them without the PEP, only with
> importlib hooks. If a feature becomes useful, it should be directly part
> of Python, instead of depending on an third party Python module.
> 
> Finally, this PEP was driven was the FAT Python optimization project
> which was abandonned in 2016, since it was not possible to show any
> significant speedup, but also because of the lack of time to implement
> the most advanced and complex optimizations.
> """
> 
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Lukasz Langa]

Based on positive feedback on python-ideas back in September,
I'm publishing the second draft for consideration on python-dev.
I hope you like it!

A nicely formatted rendering is available here:
https://www.python.org/dev/peps/pep-0563/

(Just make sure you're looking at the version that has
"Post-History: 1-Nov-2017" as you might have an older version
cached by your browser, or you might be seeing a newer
version if you get to this e-mail in the future.)

- Ł



PEP: 563
Title: Postponed Evaluation of Annotations
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa 
Discussions-To: Python-Dev 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Sep-2017
Python-Version: 3.7
Post-History: 1-Nov-2017
Resolution:


Abstract


PEP 3107 introduced syntax for function annotations, but the semantics
were deliberately left undefined.  PEP 484 introduced a standard meaning
to annotations: type hints.  PEP 526 defined variable annotations,
explicitly tying them with the type hinting use case.

This PEP proposes changing function annotations and variable annotations
so that they are no longer evaluated at function definition time.
Instead, they are preserved in ``__annotations__`` in string form.

This change is going to be introduced gradually, starting with a new
``__future__`` import in Python 3.7.


Rationale and Goals
===

PEP 3107 added support for arbitrary annotations on parts of a function
definition.  Just like default values, annotations are evaluated at
function definition time.  This creates a number of issues for the type
hinting use case:

* forward references: when a type hint contains names that have not been
  defined yet, that definition needs to be expressed as a string
  literal;

* type hints are executed at module import time, which is not
  computationally free.

Postponing the evaluation of annotations solves both problems.

Non-goals
-

Just like in PEP 484 and PEP 526, it should be emphasized that **Python
will remain a dynamically typed language, and the authors have no desire
to ever make type hints mandatory, even by convention.**

This PEP is meant to solve the problem of forward references in type
annotations.  There are still cases outside of annotations where
forward references will require usage of string literals.  Those are
listed in a later section of this document.

Annotations without forced evaluation enable opportunities to improve
the syntax of type hints.  This idea will require its own separate PEP
and is not discussed further in this document.

Non-typing usage of annotations
---

While annotations are still available for arbitrary use besides type
checking, it is worth mentioning that the design of this PEP, as well
as its precursors (PEP 484 and PEP 526), is predominantly motivated by
the type hinting use case.

With Python 3.7, PEP 484 graduates from provisional status.  Other
enhancements to the Python programming language like PEP 544, PEP 557,
or PEP 560, are already being built on this basis as they depend on
type annotations and the ``typing`` module as defined by PEP 484.

With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.


Implementation
==

In Python 4.0, function and variable annotations will no longer be
evaluated at definition time.  Instead, a string form will be preserved
in the respective ``__annotations__`` dictionary.  Static type checkers
will see no difference in behavior, whereas tools using annotations at
runtime will have to perform postponed evaluation.

If an annotation was already a string, this string is preserved
verbatim.  In other cases, the string form is obtained from the AST
during the compilation step, which means that the string form preserved
might not preserve the exact formatting of the source.

Annotations need to be syntactically valid Python expressions, also when
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
Annotations can only use names present in the module scope as postponed
evaluation using local names is not reliable (with the sole exception of
class-level names resolved by ``typing.get_type_hints()``).

Note that as per PEP 526, local variable annotations are not evaluated
at all since they are not accessible outside of the function's closure.

Enabling the future behavior in Python 3.7
--

The functionality described above can be enabled starting from Python
3.7 using the following special import::

from __future__ import annotations


Resolving Type Hints at Runtime
===

To resolve an annotation at runtime from its string form to the result
of the enclosed expression, user code needs to evaluate the string.

For code that uses type hints, the
``typing.get_type_hints(obj, globalns=None, localns=None)`` function
correctly 

Re: [Python-Dev] Generator objects and list comprehensions?

[Lukasz Langa]

> On Feb 2, 2017, at 2:17 AM, Anders Munch  wrote:
> 
> Give Python 2 a little more credit.

We are, it told you what your issue was: yield outside a function. Consider:

  >>> def f():
  ...   l = [(yield 1) for x in range(10)]
  ...   print(l)
  >>> gen = f()
  >>> for i in range(11):
  ...   gen.send(i or None)
  ...
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

But this is a very convoluted "feature" and likely people don't expect *this* 
to be what's happening.

- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Lukasz Langa]

> On Jan 31, 2017, at 11:40 AM, Wang, Peter Xihong 
>  wrote:
> 
> Regarding to the performance difference between "re" and "regex" and 
> packaging related options, we did a performance comparison using Python 3.6.0 
> to run some micro-benchmarks in the Python Benchmark Suite 
> (https://github.com/python/performance 
> ):
> 
> Results in ms, and the lower the better (running on Ubuntu 15.10)
>   re  regex (via pip install regex, 
> and a replacement of "import re" with "import regex as re")
> bm_regex_compile.py   229 298
> bm_regex_dna.py   171 267
> bm_regex_effbot.py2.773.04
> bm_regex_v8.py24.814.1
> This data shows "re" is better than "regex" in term of performance in 3 out 
> of 4 above micro-benchmarks.


This is very informative, thank you! This clearly shows we should rather pursue 
the PyPI route (with a documentation endorsement and possible bundling for 3.7) 
than full-blown replacement.

However, this benchmark is incomplete in the sense that it only checks the 
compatibility mode of `regex`, whereas it's the new mode that lends the biggest 
performance gains. So, providing checks for the other engine would show us the 
full picture. We'd need to add checks that prove the regular expressions in 
said benchmarks end up with equivalent matches, to be sure we're testing the 
same thing.

- Ł___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] re performance

[Lukasz Langa]

> On Jan 30, 2017, at 6:26 AM, Barry Warsaw  wrote:
> 
> Actually, I think pkg_resources would make an excellent candidate.  The
> setuptools crew is working on a branch that would allow for setuptools and
> pkg_resources to be split, which would be great for other reasons.  Splitting
> them may mean that pkg_resources could eventually be added to the stdlib, but
> as an intermediate step, it could also test out this idea.  It probably has a
> lot less of the baggage that you outline.

FWIW this is what we do at Facebook. We ship pkg_resources with everything but 
not setuptools. So there's definitely value in that. It's a little surprising 
we haven't been doing that yet, given that zipped applications need it.

- Ł
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

[Lukasz Langa]

> On Jan 23, 2017, at 12:10 PM, Brett Cannon  wrote:
> 
> 
> 
> On Mon, 23 Jan 2017 at 04:27 INADA Naoki  > wrote:
> On Mon, Jan 23, 2017 at 8:33 PM, Victor Stinner
> > wrote:
> > 2017-01-23 12:25 GMT+01:00 INADA Naoki  > >:
> >> I gave advice to use 'List[User]' instead of List[User] to the team of
> >> the project,
> >> if the team think RAM usage or boot speed is important.
> >
> > I would prefer a Python option (ex: "-o noannotation" command line
> > option) to opt-out annotations rather than having to write annotations
> > in strings, which is IMHO more "ugly".
> 
> So basically the equivalent of -OO for docstrings? Maybe this can be the 
> final motivator for some of us to come up with a design to generalize -O or 
> something as it keeps coming up.

Yes, please. We've talked about generalizing this for years now. FWIW, I know 
of projects that run with -OO for the memory wins stemming from docstrings and 
had to codemod assert statements into a "prod_assert" function call to achieve 
this. If you think docstrings aren't that much, multiply this by a few hundred 
processes on a box and it ends up being a substantial win to strip them out.

> >
> > Victor
> 
> Personally speaking, I hope annotations are just static hint, and
> makes zero overhead at runtime.
> (startup time, memory consumption, and execution speed).
> 
> Local variable annotations are nothing but info in the AST. Parameter 
> annotations and class annotations are stored on their respective objects so 
> there's memory usage from that and the construction of them at object 
> creation time, but that's it (e.g. the cost of creating func.__annotations__ 
> when the function object is created is all you pay for performance-wise). And 
> using strings will make those introspective attributes difficult to use, 
> hence why I don't think people have said to do that everywhere.

I suggested making all annotations just strings at runtime and PEP 484 still 
lists this as a possible course for the future. So far Guido blocked this on a 
legitimate question: how much do type hints actually cost? Nobody knows yet, 
the biggest annotated codebase is at Dropbox and this is using comments (so no 
runtime cost).

>  
> Anyway, many users are starting to use typing, for code completion or
> static checking.
> And very few user noticed it affects to performance of `isinstance(x,
> collections.Sequence)`.
> Python 3.7 may be too slow to help them.
> Can't we skip abc registration of typing.List[MyClass] completely?
> 
> I'm sorry if it's silly idea.  I don't know about background of
> current typing.py design. And I
> don't use abc much too.
> 
> Since isinstance() checks are expected to be rare I don't think anyone has 
> worried too much about the performance beyond the initial work to introduce 
> ABCs and __instancecheck__.

Similar to the above, I would advise against crippling functionality unless we 
prove this is affecting performance in a significant way.

- Ł___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making the new dtrace support work on OS X

[Lukasz Langa]

Looks like function-entry and function-return give you the C-level frame names 
for some reason. This was implemented on OS X 10.11 if that makes any 
difference. I will look at this in the evening, the laptop I'm on now is macOS 
Sierra with SIP which cripples dtrace.

> On Jan 12, 2017, at 5:08 AM, David Cournapeau  wrote:
> 
> Hi,
> 
> I was excited to see official dtrace support for python 3.6.0 on OS X, but I 
> have not been able to make it work:
> 
> 1. I built my own python from sources on OS X 10.9,  with the --with-dtrace 
> support
> 2. if I launch `python3.6 -q &` and then `sudo dtrace -l -P python$!`, I get 
> the following output:
> 
>ID   PROVIDERMODULE  FUNCTION NAME
>  2774 python48084 python3.6  _PyEval_EvalFrameDefault 
> function-entry
>  2775 python48084 python3.6  _PyEval_EvalFrameDefault 
> function-return
>  2776 python48084 python3.6   collect gc-done
>  2777 python48084 python3.6   collect gc-start
>  2778 python48084 python3.6  _PyEval_EvalFrameDefault line
> 
> Which looks similar but not the same as the example given in the doc at 
> https://docs.python.org/dev/howto/instrumentation.html#enabling-the-static-markers
>  
> 
> 
> 3. When I try to test anything with the given call_stack.d example, I can't 
> make it work at all:
> 
> """
> # script.py
> def start():
> foo()
> 
> def foo():
> pass
> 
> start()
> """
> 
> I am not very familiar with dtrace, so maybe I a missing a step, there is a 
> documentation bug, or it depends on which OS X version you are using ?
> 
> Thanks,
> David
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com