Hi, Here is round two of my take on python policy. I have incorporate the correction offered by various people, and read the documents for python-central and python-support, and incorporated my understanding of those into this document.
So, this is my take on the new python policy, based on the analysis of the new Python policy draft and dh_python, and is supposed to be a rough specification of what the python policy is supposed to be (based on current dh_python behaviour). The current analysis, and future updates, are to be found at http://www.golden-gryphon.com/software/manoj-policy/ This can still use a lot of polishing. I am including a text version below. manoj
Packaging with the new Python policy A package developers view Manoj Srivastava Copyright (c) 2006 Manoj Srivastava Revision History Revision 1.0 25 Jul 2006 Obstacles to conformance with the new python policy. While the new Python policy, specifically the [1]"Packaged Modules" chapter, contains the elements that must be present in the debian/control filename, it is not very explicit about how the values are to be substituted. The Debian Wiki falls back on calling dh_python, which is not helpful in understanding the actual logic to be followed. This article is an attempt to correct this gap in documentation. -------------------------------------------------------------------------- Table of Contents 1. [2]Introduction 1.1. [3]Categorization of Python software 2. [4]Recipe for developers 2.1. [5]General Notes 2.1.1. [6]Python versions supported by the source 2.1.2. [7]Supported versions in the binary package 2.1.3. [8]Depends: 2.1.4. [9]Provides 2.1.5. [10]Build-Depends: 2.2. [11]Script 2.2.1. [12]Supported versions 2.2.2. [13]Depends: 2.3. [14]Private Pure Python Modules 2.3.1. [15]Byte compilation 2.3.2. [16]Supported versions 2.3.3. [17]Depends: 2.4. [18]Private Extension 2.4.1. [19]Supported versions 2.4.2. [20]Depends: 2.5. [21]Public pure Python Module 2.5.1. [22]Byte compiling 2.5.2. [23]Supported versions 2.5.3. [24]Depends: 2.5.4. [25]Provides: 2.6. [26]Public Extension 2.6.1. [27]Supported versions 2.6.2. [28]Depends: 2.6.3. [29]Provides 1. Introduction While trying to update SELinux packages, I ran across problems in trying to determine if my packages were complying with the new python policy: any practical tips for packaging generally devolved to the statement "Oh, just run dh_python". This is my attempt to offer more concrete tips for packaging, by reverse engineering dh_python for the specifications and tips. -------------------------------------------------------------------------- 1.1. Categorization of Python software Program/script This consists of software directly called by an end user of external program, and is independently interpreted by the Python interpreter. Usually starts with the magic bytes #!, with the interpreter being /usr/bin/python* or /usr/bin/env python*. Modules This is code included in python "programs/scripts", and not invoked directly (serving as library modules in compiled languages). Modules can be categorized under two orthogonal criteria: firstly, based on the whether or not they are implemented purely in python, like so: Pure Python Module These are python source code, to be interpreted by the Python interpreter just like program/script code is, and may work across many versions of Python. Extension Module Extensions are C code compiled and linked against a specific version of the libpython library, and so can only be used by one version of Python. Another way of categorizing modules is based on whether or not they are available for use by third party scripts/modules. Public Public modules are available for use in other Python scripts or modules using the import directive. They are installed in one of the directories /usr/lib/pythonX.Y/site-packages /usr/lib/pythonX.Y /var/lib/python-support/pythonX.Y . Packages must not directly install anything in the directory /var/lib/python-support/pythonX.Y, that is reserved for the utility python-support. Packages may install public Python modules in directories specific to Python packaging utilities -- which specify the directories under which such modules should be droppped, and the the structure of these directories is defined by the utilities themselves. Please note that these directories are not in the path for Python, and are not available for modules to be imported from. At the time of writing, such uility specific directories include: /usr/share/pycentral /usr/share/python-support . Private Private modules are modules that are used internally by a program, and are generally only accessible to that specific program or suite of programs included in the same package. They are not intended to be shared with third party programs. They are installed in special directories, for example: /usr/lib/<package> /usr/share/<package> /usr/lib/games/<package> /usr/share/games/<package> -------------------------------------------------------------------------- 2. Recipe for developers 2.1. General Notes This is essentially a restatement of policy and dh_python internals to allow one to look at the type of python components being shipped in the package, and get an idea of what the expectation of policy is, based on both the policy and dh_python. My notes on the internal implementation of dh_python are also [30]provided for reference. There is certain amount of duplication in the sections below, but I have elected to leave it in place, so each section on each type of module is self contained. One of the major features of Python packaging on Debian systems is that pure Python modules are byte compiled for all supported versions of Python that are available. There are several utilities available to aid in this task (python-central and python-support, for example). However, these utilities need information about modules in the package (for example, whether a pure Python module supports all versions of Python or not). -------------------------------------------------------------------------- 2.1.1. Python versions supported by the source The XS-Python-Version field in debian/control specifies the versions of Python supported by the package. While this is a requirement only if using the utility package python-central (python-support, for example, prefers debian/pyversions), setting this is "appreciated" in any case, according to the [31]new policy wiki[32][1]. This is used to track packages during Python transitions. This can be a single version, or one or more of a list of non-overlapping ranges. The lowest range may optionally omit a low end, and the highest range may optionally omit an upper end. In other words, the overall range may be open ended. The ranges are often matched to the set of all known Python version that have existed, and the supported set is the intersection of the known versions of python and the range specification. 1. For packages with private modules or private extensions compiled for the current python version and for applications using /usr/bin/python, this should be set to the string "current", or if the module doesn't work with all Python version, one may specify something like "current, >= 2.4" or even "current, >= 2.2, << 2.5" - as long as the current version is supported. 2. For packages with public modules, this should be set to the string "all", unless not all versions of Python are supported (in which case the setting should specify the versions or range of versions actually supported, like ">= 2.4" or ">= 2.2, << 2.y" - as long as the current version is supported). 3. If the current version is not supported, or if the application uses /usr/bin/pythonX.Y explicitly, then this should be set to the (list of) version(s) supported. -------------------------------------------------------------------------- 2.1.2. Supported versions in the binary package python-central uses XB-Python-Version field in the control file, which is substituted in the binary package using the values from the XS-Python-Version field, filtered according to various rules detailed in the rest of this document. Again, this field is not mandatory unless the utility python-central is used, but is recommended nevertheless. python-support uses either the list of public extension versions i /usr/lib/python-support/$package or the /usr/share/python-support/$package/.version. -------------------------------------------------------------------------- 2.1.3. Depends: Packaged modules available for the default Python version (or many versions including the default) must depend on python (>= X.Y). If they require other modules to work, they must depend on the corresponding python-foo. They must not depend on any pythonX.Y-foo. Packaged modules available for only one particular version of Python (even if it is the default), on the other hand, must depend on the corresponding pythonX.Y package instead. If they need other modules (say, "bar"), they must depend on the corresponding pythonX.Y-bar packages, and must not depend on any python-bar. For consistency, if the package ("foo") provides several pythonX.Y-foo packages, and it needs the module "bar", it must also depend on pythonX.Y-bar corresponding to each version "X.Y" for the virtual packages pythonX.Y-foo that it provides. -------------------------------------------------------------------------- 2.1.4. Provides Packages with public modules and extensions should be named, or should provide, python-foo, if the package contains an extension for more than one python version. Also, for every version of python supported the package should provide pythonX.Y-foo packages. This assumes that the package correctly depends on all the appropriate versions of any version specific module that it itself requires. -------------------------------------------------------------------------- 2.1.5. Build-Depends: If the package provides public extension modules, then build depending on "python-all-dev (>= 2.3.5-11)" shall ensure that all the >pythonX.Y-dev packages are available during building. If you are using helper packages (CDBS, yada, debhelper, etc) you should build depend on the appropriate version. This also applies if your build process uses either python-central or python-support. -------------------------------------------------------------------------- 2.2. Script These are executable scripts which start with the magic string #!. -------------------------------------------------------------------------- 2.2.1. Supported versions If a script invokes /usr/bin/pythonX.Y, then the version supported by the source package (XS-Python-Version or debian/pyversions) should be restricted to X.Y, assuming that the field is being provided. Or else, it should be set to the list of python versions that the script can support, or "all". If there is no other value for the versions (set by any modules packaged), then the versions spported by the binary package (XB-Python-Version or the file .versions) is the same as above (XS-Python-Version, if XS-Python-Version is not empty, or "all", if it is empty). -------------------------------------------------------------------------- 2.2.2. Depends: If the script invokes a specific version of Python, the package must depend on pythonX.Y. If there are separate scripts that invoke different versions of Python, then all these versions must be in the Depends field -- if you still want to continue packaging instead of just shooting the upstream. If the script just calls /usr/bin/python, and there is no other restriction on the versions of Python supported, then the dependency is an un-versioned dependency on "python". If there is a range of Python versions supported, then the dependency has to be versioned, the process is as follows. 1. If a lower bound to the range exists, then a. If the lower bound is less than or equal to the current version, then add a dependency on python (>= $min_version) b. Or else depend on python (>= $min_version) | python$min_version) 2. If the upper bound to the range exists, then the stop version is the next minor version after the upper bound (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. a. If the stop version is strictly greater than the upper version, then depend on python (<< $stop_version) -------------------------------------------------------------------------- 2.3. Private Pure Python Modules These are python source code files that live in a private directory. If a package contains a private pure Python module, only one Python version may be supported, even if the private module can be recompiled for any new version of Python. -------------------------------------------------------------------------- 2.3.1. Byte compilation Any directory in which private pure Python modules are placed should be remembered, and the modules byte-compiled on installation. This is done automatically if one uses python-central, python-support uses the information in the file /usr/share/python-support/$package.dirs for a list of directories to scan. -------------------------------------------------------------------------- 2.3.2. Supported versions The version supported by the source package (XS-Python-Version or debian/pyversions) is either the specific version of Python supported, or "current", if there are no specific restrictions based on Python version. If the package contains private extention modules, then the rules for setting XB-Python-Version for the private extention modules take precedence. This should be set to "current" if there are no specific restrictions based on Python version. Note that presence of private extension modules in the same package would mean that there are restrictions on the XB-Python-Version field. If only a subset of Python versions are supported by the package, then if the current version is supported, then XB-Python-Version should be set to "current". If the current version is not supported, and if the minimum version supported is strictly higher than the current version, the XB-Python-Version field should be set to the minimum version supported. Failing that, if the current version is not supported, and the max version is lower than the current version, then this package should be deprecated, and is obsolete. -------------------------------------------------------------------------- 2.3.3. Depends: If there is no restriction on the versions of Python supported, then the dependency is an un-versioned dependency on "python". If a specific version of python is supported, the following dependencies are created: 1. If the current version is supported, then add a dependency on python 2. Or else if the minimum version supported is greater than the current versions, then depend on python (>= $min_version) | python$min_version If a upper bound of the supported versions exists, then the stop version is the next minor version after the maximum supported version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. If there is an upper bound on the supported versions, depend on python (<< $stop_version) 3. If the current version is higher than any supported version, then this package is obsolete. However, the following dependencies are still valid, if the corresponding lower and upper bounds of supported versions exist. python (>= $min_version) | python$min_version, python (<< $stop_version) -------------------------------------------------------------------------- 2.4. Private Extension These are compiled files linked to python libraries, and kept in a private directory. Since these files are compiled with one specific version of python, and do not live in versioned directories, only one version of python is supported at any given time. -------------------------------------------------------------------------- 2.4.1. Supported versions The version supported by the source package (XS-Python-Version or debian/pyversions) is either the specific version of Python supported, or "current", if there are no specific restrictions based on Python version. If a single version of Python is supported, then the versions supported by the binary package (XB-Python-Version field or the file .versions) is set to that version (copied from XS-Python-Version). If the current version is not supported, this field it set to the minimum version actually supported by the module. If the current version is supported (or there are no restrictions on the version of python supported), then this field is set to the current version. Note that this is different from the case of the private pure Python modules, where the explicit string "current" was used in similar situations. -------------------------------------------------------------------------- 2.4.2. Depends: Since only one version can be supported at any given time, and the version supported is recorded in the XB-Python-Version, we create stop version to be the next minor version up from the supported version (by incrementing the minor version). The following process is followed. 1. If the supported version is less than or equal to the current version, then depend on python (>= $version) 2. Or else, depend on python (>= $version) | python$version Also, since only one version is supported, depend on: python (<< $stop_version) -------------------------------------------------------------------------- 2.5. Public pure Python Module Public modules should be packaged with a name of python-foo, where foo is the name of the module. Such a package should support the current Debian Python version, and more if possible. There are two kinds of public pure Python modules, the most common being the variety that live in unversioned public module directories, and, in rare cases, pure python modules that live in versioned public module directories. The latter is usually the case when the pure Python module imports an public extension module from the same directory, and thus the public extension and pure python modules must be in the same directory. Otherwise, pure python modules should live in an unversioned public module directory. Depending on the packaging utility used, the modules live in either /usr/share/python-central or in /usr/share/python-support/$package. Official pure Python modules generally live in a different set of directories than unofficial ones, but are otherwise treated exactly like other public pure Python Module which live in unversioned directories as detailed below. -------------------------------------------------------------------------- 2.5.1. Byte compiling In the common case of pure Python modules in unversioned public module directories, tools exist to help byte compile the pure Python modules for all versions of Python installed on the target system. In case of pure Python modules in versioned public module directories, byte compilation is up to the package scripts. -------------------------------------------------------------------------- 2.5.2. Supported versions For the source package, this should be the version of Python supported, or "all", if there are no specific restrictions based on Python version. In case of pure Python modules in versioned public module directories, the set of versions supported usually corresponds to the list of versioned directories in which pure Python modules live. The rules for determining the versions supported by the binary package setting depend on whether the pure python module lives in a versioned directory, or not. 1. In the common case of pure Python modules in unversioned public module directories, any private modules have preference when it comes to determining the value for this field. If there are no private modules packaged with this package, this is set to the value of XB-Python-Version, if present. If XB-Python-Version is null, then this field is set to "all" 2. In case of pure Python modules in versioned public module directories, this field is set to a list of versions corresponding to the list of versioned public module directories in which pure Python modules live. -------------------------------------------------------------------------- 2.5.3. Depends: The rules for this field setting depend on whether the pure python module lives in a versioned directory, or not. 1. In the common case of pure Python modules in unversioned public module directories, we look to XB-Python-Version to see how to proceed. a. If there is no restriction on the version of python supported (there is no lower or upper bound to the range, and the current version is supported), then add a dependency on python. [33]Note If there is no upper or lower bound, but the current version is specifically excluded, it is unclear what to do. Consider the range -2.2,2.4-. b. Or else if the minimum version supported is greater than the current versions, then depend on python (>= $min_version) | python$min_version If a upper bound of the supported versions exists, then the stop version is the next minor version after the maximum supported version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. If there is an upper bound on the supported versions, depend on python (<< $stop_version) c. If the current version is higher than any supported version, then this package is obsolete. However, the following dependencies are still valid, if the corresponding lower and upper bounds of supported versions exist. python (>= $min_version) | python$min_version, python (<< $stop_version) 2. For the case where pure Python modules in versioned public module directories, the set of versions corresponding to the list of versioned public module directories in which pure Python modules live is used in lieu of the value of XS-Python-Version. a. if the minimum version supported is greater than the current versions, then depend on python (>= $min_version) | python$min_version The stop version is the next minor version after the maximum supported version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. Depend on python (<< $stop_version) -------------------------------------------------------------------------- 2.5.4. Provides: Dependencies for the package also differ slightly based on whether the pure Python module is version dependent or not. 1. In the common case of pure Python modules in unversioned public module directories, a. If there is a minimum supported version, i. If the minimum version supported is less than or equal to the current version, then depend on python (>= $min_version) ii. Or else, if the minimum version supported is greater than the current version, then depend on python (>= $min_version) | python$min_version If a upper bound of the supported versions exists, then the stop version is the next minor version after the maximum supported version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. If there is an upper bound on the supported versions, and the upper bround is greater than or equal to the current version, then depend on python (<< $stop_version) b. If the current version is higher than any supported version, then this package is obsolete. However, the following dependencies are still valid, if the corresponding lower and upper bounds of supported versions exist. python (>= $min_version) | python$min_version, python (<< $stop_version) 2. In case of pure Python modules in versioned public module directories, the set of versions corresponding to the list of versioned public module directories in which pure Python modules live is used in lieu of the value of XS-Python-Version. a. If the lowest version in the set is less than or equal to the current version, then depend on python (>= $min_version) b. Or else depend on python (>= $min_version) | python$min_version, c. If the highest version in the set is greater than or equal to the current version, then depend on python (<< $stop_version) where the stop version is the next minor version greater than the highest supported version. d. For every version in the set, also depend on python$version -------------------------------------------------------------------------- 2.6. Public Extension Public extensions should be packaged with a name of python-foo, where foo is the name of the module. Such a package should support the current Debian Python version, and more if possible. Depending on the packaging utility used, the modules live in either /usr/lib/pythonX.Y/site-packages or in /usr/lib/python-support/$package/pythonX.Y, with proper symbolic links in place. -------------------------------------------------------------------------- 2.6.1. Supported versions The versions supported by the source and binary packages should be set to the list of versions for which the compiled extension modules are provided. -------------------------------------------------------------------------- 2.6.2. Depends: The list of Python versions supported is used to determine the dependency relationships as follows. 1. if the minimum version supported is greater than the current versions, then depend on python (>= $min_version) | python$min_version The stop version is the next minor version after the maximum supported version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. Depend on python (<< $stop_version) -------------------------------------------------------------------------- 2.6.3. Provides For every version of Python for which compiled extension modules are provided, provide a package named pythonX.Y-foo (assuming that the package name is python-foo). Notes [34][1] For compatibility, each of the tools (python-central and python-support) are able to use each others data. References Visible links 1. http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html 2. file:///tmp/html-n28320#AEN38 3. file:///tmp/html-n28320#AEN44 4. file:///tmp/html-n28320#AEN104 5. file:///tmp/html-n28320#AEN106 6. file:///tmp/html-n28320#AEN116 7. file:///tmp/html-n28320#AEN146 8. file:///tmp/html-n28320#AEN156 9. file:///tmp/html-n28320#AEN173 10. file:///tmp/html-n28320#AEN178 11. file:///tmp/html-n28320#AEN189 12. file:///tmp/html-n28320#AEN193 13. file:///tmp/html-n28320#AEN206 14. file:///tmp/html-n28320#AEN235 15. file:///tmp/html-n28320#AEN238 16. file:///tmp/html-n28320#AEN244 17. file:///tmp/html-n28320#AEN261 18. file:///tmp/html-n28320#AEN284 19. file:///tmp/html-n28320#AEN287 20. file:///tmp/html-n28320#AEN298 21. file:///tmp/html-n28320#AEN315 22. file:///tmp/html-n28320#AEN323 23. file:///tmp/html-n28320#AEN326 24. file:///tmp/html-n28320#AEN339 25. file:///tmp/html-n28320#AEN380 26. file:///tmp/html-n28320#AEN430 27. file:///tmp/html-n28320#AEN436 28. file:///tmp/html-n28320#AEN439 29. file:///tmp/html-n28320#AEN452 30. file:///tmp/python_policy.txt 31. http://wiki.debian.org/DebianPython/NewPolicy 32. file:///tmp/html-n28320#FTN.AEN126 34. file:///tmp/html-n28320#AEN126
-- A truly thoroughbred man (a Buddha) is hard to find. He is not born anywhere, but where that seer is born, the people prosper. 193 Manoj Srivastava <[EMAIL PROTECTED]> <http://www.debian.org/%7Esrivasta/> 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C