Accepted! Thanks for your patience, Paul, and thanks everyone for their feedback.
I know there are still a few small edits to the PEP, but those don't affect my acceptance. Congrats! --Guido On Thu, Feb 26, 2015 at 9:05 AM, Paul Moore <p.f.mo...@gmail.com> wrote: > On 24 February 2015 at 18:24, Guido van Rossum <gu...@python.org> wrote: > > Here's my review. I really like where this is going but I have a few > > questions and suggestions (I can't help myself :-). > > OK, I've updated both the PEP and the patch based on follow-up > discussions. I think (again!) it is ready to go. > > I've included the updated PEP inline below, it'll be available at > peps.python.org as soon as someone has a chance to upload it. > > Thanks to everyone for the various comments. If I've missed anything > that someone thinks I'd said I would change, please let me know. > > Paul > > PEP: 441 > Title: Improving Python ZIP Application Support > Version: $Revision$ > Last-Modified: $Date$ > Author: Daniel Holth <dho...@gmail.com>, > Paul Moore <p.f.mo...@gmail.com> > Discussions-To: > https://mail.python.org/pipermail/python-dev/2015-February/138277.html > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 30 March 2013 > Post-History: 30 March 2013, 1 April 2013, 16 February 2015 > > Improving Python ZIP Application Support > ======================================== > > Python has had the ability to execute directories or ZIP-format > archives as scripts since version 2.6 [1]_. When invoked with a zip > file or directory as its first argument the interpreter adds that > directory to sys.path and executes the ``__main__`` module. These > archives provide a great way to publish software that needs to be > distributed as a single file script but is complex enough to need to > be written as a collection of modules. > > This feature is not as popular as it should be mainly because it was > not promoted as part of Python 2.6 [2]_, so that it is relatively > unknown, but also because the Windows installer does not register a > file extension (other than ``.py``) for this format of file, to associate > with the launcher. > > This PEP proposes to fix these problems by re-publicising the feature, > defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications" > and "Windowed Python ZIP Applications", and providing some simple > tooling to manage the format. > > A New Python ZIP Application Extension > ====================================== > > The terminology "Python Zip Application" will be the formal term used > for a zip-format archive that contains Python code in a form that can > be directly executed by Python (specifically, it must have a > ``__main__.py`` file in the root directory of the archive). The > extension ``.pyz`` will be formally associated with such files. > > The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python > Zip Applications" with the platform launcher so they can be executed. > A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a > windowed application, indicating whether the console should appear > when running the app. > > On Unix, it would be ideal if the ``.pyz`` extension and the name > "Python Zip Application" were registered (in the mime types database?). > However, such an association is out of scope for this PEP. > > Python Zip applications can be prefixed with a ``#!`` line > pointing to the correct Python interpreter and an optional > explanation:: > > #!/usr/bin/env python3 > # Python application packed with zipapp module > (binary contents of archive) > > On Unix, this allows the OS to run the file with the correct > interpreter, via the standard "shebang" support. On Windows, the > Python launcher implements shebang support. > > However, it is always possible to execute a ``.pyz`` application by > supplying the filename to the Python interpreter directly. > > As background, ZIP archives are defined with a footer containing > relative offsets from the end of the file. They remain valid when > concatenated to the end of any other file. This feature is completely > standard and is how self-extracting ZIP archives and the bdist_wininst > installer format work. > > > Minimal Tooling: The zipapp Module > ================================== > > This PEP also proposes including a module for working with these > archives. The module will contain functions for working with Python > zip application archives, and a command line interface (via ``python > -m zipapp``) for their creation and manipulation. > > More complete tools for managing Python Zip Applications are > encouraged as 3rd party applications on PyPI. Currently, pyzzer [5]_ > and pex [6]_ are two such tools. > > Module Interface > ---------------- > > The zipapp module will provide the following functions: > > ``create_archive(source, target=None, interpreter=None, main=None)`` > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Create an application archive from *source*. The source can be any > of the following: > > * The name of a directory, in which case a new application archive > will be created from the content of that directory. > * The name of an existing application archive file, in which case the > file is copied to the target. The file name should include the > ``.pyz`` extension, if required. > * A file object open for reading in bytes mode. The content of the > file should be an application archive, and the file object is > assumed to be positioned at the start of the archive. > > The *target* argument determines where the resulting archive will be > written: > > * If it is the name of a file, the archive will be written to that > file. > * If it is an open file object, the archive will be written to that > file object, which must be open for writing in bytes mode. > * If the target is omitted (or None), the source must be a directory > and the target will be a file with the same name as the source, with > a ``.pyz`` extension added. > > The *interpreter* argument specifies the name of the Python > interpreter with which the archive will be executed. It is written as > a "shebang" line at the start of the archive. On Unix, this will be > interpreted by the OS, and on Windows it will be handled by the Python > launcher. Omitting the *interpreter* results in no shebang line being > written. If an interpreter is specified, and the target is a > filename, the executable bit of the target file will be set. > > The *main* argument specifies the name of a callable which will be > used as the main program for the archive. It can only be specified if > the source is a directory, and the source does not already contain a > ``__main__.py`` file. The *main* argument should take the form > "pkg.module:callable" and the archive will be run by importing > "pkg.module" and executing the given callable with no arguments. It > is an error to omit *main* if the source is a directory and does not > contain a ``__main__.py`` file, as otherwise the resulting archive > would not be executable. > > If a file object is specified for *source* or *target*, it is the > caller's responsibility to close it after calling create_archive. > > When copying an existing archive, file objects supplied only need > ``read`` and ``readline``, or ``write`` methods. When creating an > archive from a directory, if the target is a file object it will be > passed to the ``zipfile.ZipFile`` class, and must supply the methods > needed by that class. > > ``get_interpreter(archive)`` > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Returns the interpreter specified in the shebang line of the > *archive*. If there is no shebang, the function returns ``None``. > The *archive* argument can be a filename or a file-like object open > for reading in bytes mode. > > > Command Line Usage > ------------------ > > The zipapp module can be run with the python ``-m`` flag. The command > line interface is as follows:: > > python -m zipapp directory [options] > > Create an archive from the given directory. An archive will > be created from the contents of that directory. The archive > will have the same name as the source directory with a .pyz > extension. > > The following options can be specified: > > -o archive / --output archive > > The destination archive will have the specified name. The > given name will be used as written, so should include the > ".pyz" extension. > > -p interpreter / --python interpreter > > The given interpreter will be written to the shebang line > of the archive. If this option is not given, the archive > will have no shebang line. > > -m pkg.mod:fn / --main pkg.mod:fn > > The source directory must not have a __main__.py file. The > archiver will write a __main__.py file into the target > which calls fn from the module pkg.mod. > > The behaviour of the command line interface matches that of > ``zipapp.create_archive()``. > > In addition, it is possible to use the command line interface to work > with an existing archive:: > > python -m zipapp app.pyz --show > > Displays the shebang line of an archive. Output is of the > form > > Interpreter: /usr/bin/env > or > Interpreter: <none> > > and is intended for diagnostic use, not for scripts. > > python -m zipapp app.pyz -o newapp.pyz [-p interpreter] > > Copy app.pyz to newapp.pyz, modifying the shebang line based > on the -p option (as for creating an archive, no -p option > means remove the shebang line). Specifying a destination is > mandatory. > > In-place modification of an archive is *not* supported, as the > risk of damaging archives is too great for a simple tool. > > As noted, the archives are standard zip files, and so can be unpacked > using any standard ZIP utility or Python's zipfile module. For this > reason, no interfaces to list the contents of an archive, or unpack > them, are provided or needed. > > FAQ > --- > > Are you sure a standard ZIP utility can handle ``#!`` at the beginning? > Absolutely. The zipfile specification allows for arbitrary data to > be prepended to a zipfile. This feature is commonly used by > "self-extracting zip" programs. If your archive program can't > handle this, it is a bug in your archive program. > > Isn't zipapp just a very thin wrapper over the zipfile module? > Yes. If you prefer to build your own Python zip application > archives using other tools, they will work just as well. The > zipapp module is a convenience, nothing more. > > Why not use just use a ``.zip`` or ``.py`` extension? > Users expect a ``.zip`` file to be opened with an archive tool, and > expect a ``.py`` file to contain readable text. Both would be > confusing for this use case. > > How does this compete with existing package formats? > The sdist, bdist and wheel formats are designed for packaging of > modules to be installed into an existing Python installation. > They are not intended to be used without installing. The > executable zip format is specifically designed for standalone use, > without needing to be installed. They are in effect a multi-file > version of a standalone Python script. > > Rejected Proposals > ================== > > Convenience Values for Shebang Lines > ------------------------------------ > > Is it worth having "convenience" forms for any of the common > interpreter values? For example, ``-p 3`` meaning the same as ``-p > "/usr/bin/env python3"``. It would save a lot of typing for the > common cases, as well as giving cross-platform options for people who > don't want or need to understand the intricacies of shebang handling > on "other" platforms. > > Downsides are that it's not obvious how to translate the > abbreviations. For example, should "3" mean "/usr/bin/env python3", > "/usr/bin/python3", "python3", or something else? Also, there is no > obvious short form for the key case of "/usr/bin/env python" (any > available version of Python), which could easily result in scripts > being written with overly-restrictive shebang lines. > > Overall, this seems like there are more problems than benefits, and as > a result has been dropped from consideration. > > Registering ``.pyz`` as a Media Type > ------------------------------------ > > It was suggested [3]_ that the ``.pyz`` extension should be registered > in the Unix database of extensions. While it makes sense to do this > as an equivalent of the Windows installer registering the extension, > the ``.py`` extension is not listed in the media types database [4]_. > It doesn't seem reasonable to register ``.pyz`` without ``.py``, so > this idea has been omitted from this PEP. An interested party could > arrange for *both* ``.py`` and ``.pyz`` to be registered at a future > date. > > Default Interpreter > ------------------- > > The initial draft of this PEP proposed using ``/usr/bin/env python`` > as the default interpreter. Unix users have problems with this > behaviour, as the default for the python command on many distributions > is Python 2, and it is felt that this PEP should prefer Python 3 by > default. However, using a command of ``python3`` can result in > unexpected behaviour for Windows users, where the default behaviour of > the launcher for the command ``python`` is commonly customised by users, > but the behaviour of ``python3`` may not be modified to match. > > As a result, the principle "in the face of ambiguity, refuse to guess" > has been invoked, and archives have no shebang line unless explicitly > requested. On Windows, the archives will still be run (with the > default Python) by the launcher, and on Unix, the archives can be run > by explicitly invoking the desired Python interpreter. > > Command Line Tool to Manage Shebang Lines > ----------------------------------------- > > It is conceivable that users would want to modify the shebang line for > an existing archive, or even just display the current shebang line. > This is tricky to do so with existing tools (zip programs typically > ignore prepended data totally, and text editors can have trouble > editing files containing binary data). > > The zipapp module provides functions to handle the shebang line, but > does not include a command line interface to that functionality. This > is because it is not clear how to provide one without the resulting > interface being over-complex and potentially confusing. Changing the > shebang line is expected to be an uncommon requirement. > > Reference Implementation > ======================== > > A reference implementation is at http://bugs.python.org/issue23491. > > References > ========== > > .. [1] Allow interpreter to execute a zip file > (http://bugs.python.org/issue1739468) > > .. [2] Feature is not documented > (http://bugs.python.org/issue17359) > > .. [3] Discussion of adding a .pyz mime type on python-dev > (https://mail.python.org/pipermail/python-dev/2015-February/138338.html > ) > > .. [4] Register of media types > (http://www.iana.org/assignments/media-types/media-types.xhtml) > > .. [5] pyzzer - A tool for creating Python-executable archives > (https://pypi.python.org/pypi/pyzzer) > > .. [6] pex - The PEX packaging toolchain > (https://pypi.python.org/pypi/pex) > > The discussion of this PEP took place on the python-dev mailing list, > in the thread starting at > https://mail.python.org/pipermail/python-dev/2015-February/138277.html > > Copyright > ========= > > This document has been placed into the public domain. > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com