On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra <jelle.zijls...@gmail.com> wrote:
> > > 2017-11-12 3:40 GMT-08:00 Ethan Smith <et...@ethanhs.me>: > >> Hello, >> >> I re-wrote my PEP to have typing opt-in be per-package rather than >> per-distribution. This greatly simplifies things, and thanks to the >> feedback and suggestions of Nick Coghlan, it is entirely compatible with >> older packaging tooling. >> >> The main idea is there are two types of packages: >> - types are packaged with runtime code (inline or stubs in the same >> package) >> - types are in a separate package (a third party or maintainer wants to >> ship type information, but not with runtime code). >> >> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/ >> >> And as always, duplicated below. >> >> Cheers, >> >> Ethan Smith >> >> --------------------------------------------------- >> >> PEP: 561 >> Title: Distributing and Packaging Type Information >> Author: Ethan Smith <et...@ethanhs.me> >> Status: Draft >> Type: Standards Track >> Content-Type: text/x-rst >> Created: 09-Sep-2017 >> Python-Version: 3.7 >> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017 >> >> >> Abstract >> ======== >> >> PEP 484 introduced type hinting to Python, with goals of making typing >> gradual and easy to adopt. Currently, typing information must be distributed >> manually. This PEP provides a standardized means to leverage existing tooling >> to package and distribute type information with minimal work and an ordering >> for type checkers to resolve modules and collect this information for type >> checking. >> >> >> Rationale >> ========= >> >> Currently, package authors wish to distribute code that has inline type >> information. Additionally, maintainers would like to distribute stub files >> to keep Python 2 compatibility while using newer annotation syntax. However, >> there is no standard method to distribute packages with type information. >> Also, if one wished to ship stub files privately the only method available >> would be via setting ``MYPYPATH`` or the equivalent to manually point to >> stubs. If the package can be released publicly, it can be added to >> typeshed [1]_. However, this does not scale and becomes a burden on the >> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases >> of the tool using typeshed. >> >> PEP 484 has a brief section on distributing typing information. In this >> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for >> shipping stub files. However, manually adding a path to stub files for each >> third party library does not scale. The simplest approach people have taken >> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type >> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy >> and Django). >> >> >> Definition of Terms >> =================== >> >> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are >> to be interpreted as described in RFC 2119. >> >> "inline" - the types are part of the runtime code using PEP 526 and 3107 >> syntax. >> >> "stubs" - files containing only type information, empty of runtime code. >> >> "Distributions" are the packaged files which are used to publish and >> distribute >> a release. [3]_ >> >> "Module" a file containing Python runtime code or stubbed type information. >> >> "Package" a directory or directories that namespace Python modules. >> >> >> Specification >> ============= >> >> There are several motivations and methods of supporting typing in a package. >> This PEP recognizes three (3) types of packages that users of typing wish to >> create: >> >> 1. The package maintainer would like to add type information inline. >> >> 2. The package maintainer would like to add type information via stubs. >> >> 3. A third party or package maintainer would like to share stub files for >> a package, but the maintainer does not want to include them in the source >> of the package. >> >> This PEP aims to support these scenarios and make them simple to add to >> packaging and deployment. >> >> The two major parts of this specification are the packaging specifications >> and the resolution order for resolving module type information. The type >> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec >> of PEP 484 [2]_. >> >> New third party stub libraries SHOULD distribute stubs via the third party >> packaging methods proposed in this PEP in place of being added to typeshed. >> Typeshed will remain in use, but if maintainers are found, third party stubs >> in typeshed MAY be split into their own package. >> >> >> Packaging Type Information >> -------------------------- >> >> In order to make packaging and distributing type information as simple and >> easy as possible, packaging and distribution is done through existing >> frameworks. >> >> Package maintainers who wish to support type checking of their code MUST add >> a ``py.typed`` file to their package supporting typing. This marker is >> recursive, if a top-level package includes it, all sub-packages MUST support >> type checking as well. To have this file installed with the package, >> maintainers can use existing packaging options such as ``package_data`` in >> distutils, shown below. >> >> Distutils option example:: >> >> ... >> package_data = { >> 'pkg': ['py.typed'], >> }, >> ... >> >> For namespace packages, the ``py.typed`` file should be in the submodules of >> the namespace, to avoid conflicts and for clarity. >> >> Stub Only Packages >> '''''''''''''''''' >> >> For package maintainers wishing to ship stub files containing all of their >> type information, it is preferred that the ``*.pyi`` stubs are alongside the >> corresponding ``*.py`` files. However, the stubs can also be put in a >> separate >> package and distributed separately. Third parties can also find this method >> useful if they wish to distribute stub files. The name of the stub package >> MUST follow the scheme ``pkg_stubs`` for type stubs for the package named >> ``pkg``. The normal resolution order of checking ``*.pyi`` before ``*.py`` >> will be maintained. >> >> This is very minor, but what do you think of using "pkg-stubs" instead of > "pkg_stubs" (using a hyphen rather than an underscore)? This would make the > name illegal to import as a normal Python package, which makes it clear > that it's not a normal package. Also, there could be real packages named > "_stubs". > I suppose this makes sense. I checked PyPI and as of a few weeks ago there were no packages with the name pattern, but I like the idea of making it explicitly non-runtime importable. I cannot think of any reason not to do it, and the avoidance of confusion about the package being importable is a benefit. I will make the change with my next round of edits. > Third parties seeking to distribute stub files are encouraged to contact the >> maintainer of the package about distribution alongside the package. If the >> maintainer does not wish to maintain or package stub files or type >> information >> inline, then a third party stub only package can be created. >> >> In addition, stub only distributions SHOULD indicate which version(s) >> of the runtime package are supported by indicating the runtime distribution's >> version(s) through normal dependency data. For example, if there was a >> stub package ``flyingcircus_stubs``, it can indicate the versions of the >> runtime ``Flyingcircus`` distribution supported through ``install_requires`` >> in distutils based tools, or the equivalent in other packaging tools. >> >> >> Type Checker Module Resolution Order >> ------------------------------------ >> >> The following is the order that type checkers supporting this PEP SHOULD >> resolve modules containing type information: >> >> 1. User code - the files the type checker is running on. >> >> 2. Stubs or Python source manually put in the beginning of the path. Type >> checkers SHOULD provide this to allow the user complete control of which >> stubs to use, and patch broken stubs/inline types from packages. >> >> 3. Stub packages - these packages can supersede the installed packages. >> They can be found at ``pkg_stubs`` for package ``pkg``. >> >> 4. Inline packages - if there is nothing overriding the installed >> package, and it opts into type checking, inline types SHOULD be used. >> >> 5. Typeshed (if used) - Provides the stdlib types and several third party >> libraries. >> >> Type checkers that check a different Python version than the version they run >> on MUST find the type information in the ``site-packages``/``dist-packages`` >> of that Python version. This can be queried e.g. >> ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also >> recommended >> that the type checker allow for the user to point to a particular Python >> binary, in case it is not in the path. >> >> >> Implementation >> ============== >> >> The proposed scheme of indicating support for typing is completely backwards >> compatible, and requires no modification to tooling. A sample package with >> inline types is available [typed_pkg]_, as well as a sample package checker >> [pkg_checker]_ which reads the metadata of installed packages and reports on >> their status as either not typed, inline typed, or a stub package. >> >> >> Acknowledgements >> ================ >> >> This PEP would not have been possible without the ideas, feedback, and >> support >> of Ivan Levkivskyi, Jelle Zijlstra, Nick Coghlan, Daniel F Moisset, Nathaniel >> Smith, and Guido van Rossum. >> >> >> Version History >> =============== >> >> * 2017-11-12 >> >> * Rewritten to use existing tooling only >> * No need to indicate kind of type information in metadata >> * Name of marker file changed from ``.typeinfo`` to ``py.typed`` >> >> * 2017-11-10 >> >> * Specification re-written to use package metadata instead of >> distribution >> metadata. >> * Removed stub only packages and merged into third party packages spec. >> * Removed suggestion for typecheckers to consider checking runtime >> versions >> * Implementations updated to reflect PEP changes. >> >> * 2017-10-26 >> >> * Added implementation references. >> * Added acknowledgements and version history. >> >> * 2017-10-06 >> >> * Rewritten to use .distinfo/METADATA over a distutils specific command. >> * Clarify versioning of third party stub packages. >> >> * 2017-09-11 >> >> * Added information about current solutions and typeshed. >> * Clarify rationale. >> >> >> References >> ========== >> .. [1] Typeshed (https://github.com/python/typeshed) >> >> .. [2] PEP 484, Storing and Distributing Stub Files >> >> (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) >> >> .. [3] PEP 426 definitions >> (https://www.python.org/dev/peps/pep-0426/) >> >> .. [typed_pkg] Sample typed package >> (https://github.com/ethanhs/sample-typed-package) >> >> .. [pkg_checker] Sample package checker >> (https://github.com/ethanhs/check_typedpkg) >> >> Copyright >> ========= >> >> This document has been placed in the public domain. >> >> >> >> .. >> Local Variables: >> mode: indented-text >> indent-tabs-mode: nil >> sentence-end-double-space: t >> fill-column: 70 >> coding: utf-8 >> End: >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/jelle. >> zijlstra%40gmail.com >> >> >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com