Debian Python Policy -------------------- Neil Schemenauer Matthias Klose Gregor Hoffleit Josselin Mouette Joe Wreschnig version 0.4.1.0 ------------------------------------------------------------------------------- Abstract -------- This document describes the packaging of Python within the Debian GNU/Linux distribution and the policy requirements for packaged Python programs and modules. Copyright Notice ---------------- Copyright (C) 1999, 2001, 2003, 2006 Software in the Public Interest This manual is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. A copy of the GNU General Public License is available as `/usr/share/common-licences/GPL' in the Debian GNU/Linux distribution or on the World Wide Web at The GNU Public Licence (http://www.gnu.org/copyleft/gpl.html). You can also obtain it by writing to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ------------------------------------------------------------------------------- Contents -------- 1. Python Packaging 1.1. Versions 1.2. Main package 1.3. Python Interpreter 1.3.1. Interpreter Name 1.3.2. Interpreter Location 1.4. Module Path 1.5. Documentation 2. Packaged Modules 2.1. Types of Python Modules 2.2. Module Package Names 2.3. Specifying Supported Versions 2.4. Dependencies 2.5. Provides 2.6. Modules Bytecompilation 3. Python Programs 3.1. Programs using the default python 3.1.1. Programs Shipping Private Modules 3.2. Programs Using a Particular Python Version 4. Programs Embedding Python 4.1. Building Embedded Programs 4.2. Embedded Python Dependencies 5. Interaction with Locally Installed Python Versions A. Build Dependencies B. Packaging Tools B.1. python-support B.2. python-central B.3. CDBS C. Upgrade Procedure ------------------------------------------------------------------------------- 1. Python Packaging ------------------- 1.1. Versions ------------- At any given time, the package `python' will represent the current default Debian Python version. The default Debian Python version should alway be the latest stable upstream release that can be integrated in the distribution. Apart from the default version, legacy versions of Python or beta versions of future releases may be included as well in the distribution, as long as they are needed by other packages, or as long as it seems reasonable to provide them. (Note: For the scope of this document, Python versions are synonymous to feature releases, i.e. Python 2.0 and 2.0.1 are subminor versions of the same Python version 2.0, but Python 2.1 and 2.2 are indeed different versions.) For any version, the main package must be called `python.'. The set of currently supported python versions can be found in `/usr/share/python/debian_defaults'. 1.2. Main package ----------------- For every Python version provided in the distribution, the package `python.' shall comprise a complete distribution for _deployment_ of Python scripts and applications. The package includes the binary `/usr/bin/python.' and all modules of the upstream Python distribution. Excluded are any modules that depend on non-_required_ packages, they will be provided in separate packages. Some tools and files for the _development_ of Python modules are split off in a separate package `python.-dev'. Documentation will be provided separately as well. At any time, the `python' package must contain a symlink `/usr/bin/python' to the the appropriate binary `/usr/bin/python.'. The `python' package must also depend on the appropriate `python.' to ensure this binary is installed. The version of the `python' package must be greater than or equal to . and smaller than .. 1.3. Python Interpreter ----------------------- 1.3.1. Interpreter Name ----------------------- Python scripts depending on the default Python version (see Section 1.2, `Main package') or not depending on a specific Python version should use `python' (unversioned) as the interpreter name. Python scripts that only work with a specific Python version must explicitly use the versioned interpreter name (`python.'). 1.3.2. Interpreter Location --------------------------- The preferred specification for the Python interpreter is `/usr/bin/python' or `/usr/bin/python.'. This ensures that a Debian installation of python is used and all dependencies on additional python modules are met. If a maintainer would like to provide the user with the possibility to override the Debian Python interpreter, he may want to use `/usr/bin/env python' or `/usr/bin/env python.'. However this is not advisable as it bypasses Debian's dependency checking and makes the package vulnerable to incomplete local installations of python. 1.4. Module Path ---------------- The module search path for Debian has been amended to include a directory tree in /usr/local at the beginning of the path. By default, sys.path is searched in the following order: /usr/lib/python.zip /usr/lib/python. /usr/lib/python./plat-linux2 /usr/lib/python./lib-tk /usr/lib/python./lib-dynload /usr/local/lib/python./site-packages /usr/lib/python./site-packages /var/lib/python-support/python. /usr/lib/python./site-packages/ /usr/lib/site-python The use of the `/usr/lib/site-python' directory is deprecated. The directory may be dropped from the path in a future version. The /usr/lib/python.zip archive appeared in python2.3; it is not currently used in Debian. Modules should not install directly to the `/var/lib/python-support' directory; it is for use by Section B.1, `python-support'. 1.5. Documentation ------------------ Python documentation is split out in separate packages `python.-doc'. The package `python-doc' will always provide the documentation for the default Debian Python version. TODO: Policy for documentation of third party packages. ------------------------------------------------------------------------------- 2. Packaged Modules ------------------- The goal of these policies is to reduce the work necessary for Python transitions. Python modules are internally very dependent on a specific Python version. However, we want to automate recompiling modules when possible, either during the upgrade itself (re-bytecompiling pyc and pyo files) or shortly thereafter with automated rebuilds (to handle C extensions). These policies encourage automated dependency generation and loose version bounds whenever possible. 2.1. Types of Python Modules ---------------------------- There are two kinds of Python modules, "pure" Python modules, and extension modules. Pure Python modules are Python source code that works across many versions of Python. Extensions are C code compiled and linked against a specific version of the libpython library, and so can only be used by one version of Python. Python packages are directories containing at least a `__init__.py', other modules, extensions and packages (A package in the Python sense is unrelated to a Debian package). Python packages must be packaged into the same directory (as done by upstream). Splitting components of a package across directories changes the import order and may confuse documentation tools and IDEs. There are two ways to distribute Python modules. Public modules are installed in one of the directories listed in Section 1.4, `Module Path'. They are accessible to any program. Private modules are installed in a directory such as `/usr/share/' or `/usr/lib/'. They are generally only accessible to a specific program or suite of programs included in the same package. 2.2. Module Package Names ------------------------- Public modules should be packaged with a name of `python-', where is the name of the module. Such a package should support the current Debian Python version, and more if possible (there are several tools to help implement this, see Appendix B, `Packaging Tools'). For example, if Python 2.3, 2.4, and 2.5 are supported, the Python command import foo should import the module when the user is running any of `/usr/bin/python2.3', `/usr/bin/python2.4', and `/usr/bin/python2.5'. This requirement also applies to extension modules; binaries for all the supported Python versions should be included in a single package. 2.3. Specifying Supported Versions ---------------------------------- The `XS-Python-Version' field in `debian/control' specifies the versions of Python supported by the package. This is used to track packages during Python transitions, and is also used by some packaging scripts to automatically generate appropriate Depends and Provides lines. The format of the field may be one of the following: XS-Python-Version: all XS-Python-Version: current XS-Python-Version: current, >= X.Y XS-Python-Version: >= X.Y XS-Python-Version: >= A.B, << X.Y XS-Python-Version: A.B, X.Y Where "all" means the package supports any Python version available, and "current" means it supports Debian's current Python version. Explicit Versions or version ranges can also be used. Your control file should also have a line: XB-Python-Version: ${python:Versions} The python:Versions is substituted by the supported Python versions of the binary package, based on `XS-Python-Version'. (If you are not using `dh_python' you will need to handle this substitution yourself.) The format of the field `XB-Python-Version' is the same as the `XS-Python-Version' field for packages not containing extensions. Packages with extensions must list the versions explicitely. If your package is used by another module or application that requires a specific Python version, it should also `Provide: python.-foo' for each version it supports. 2.4. Dependencies ----------------- Packaged modules available for the default Python version (or many versions including the default) as described in Section 2.2, `Module Package Names' must depend on "`python (>= .')". If they require other modules to work, they must depend on the corresponding `python-foo'. They must not depend on any `python.-foo'. Packaged modules available for one particular version of Python must depend on the corresponding `python.' package instead. If they need other modules, they must depend on the corresponding `python.-foo' packages, and must not depend on any `python-foo'. 2.5. Provides ------------- Provides in packages of the form `python-' must be specified, if the package contains an extension for more than one python version. Provides should also be added on request of maintainers who depend on a non-default python version. Packaged modules available for one particular version of Python must depend on the corresponding `python.' package instead. If they need other modules, they must depend on the corresponding `python.-foo' packages, and must not depend on any `python-foo'. 2.6. Modules Bytecompilation ---------------------------- If a package provides any binary-independent modules (`foo.py' files), the corresponding bytecompiled modules (`foo.pyc' files) and optimized modules (`foo.pyo' files) must not ship in the package. Instead, they should be generated in the package's postinst, and removed in the package's prerm. The package's prerm has to make sure that both `foo.pyc' and `foo.pyo' are removed. A package should only byte-compile the files which belong to the package. The file `/etc/python/debian_config' allows configuration how modules should be byte-compiled. The postinst scripts should respect these settings. Modules in private installation directories and in `/usr/lib/site-python' should be byte-compiled, when the default python version changes. ------------------------------------------------------------------------------- 3. Python Programs ------------------ 3.1. Programs using the default python -------------------------------------- Programs that can run with any version of Python must begin with `#!/usr/bin/python' or `#!/usr/bin/env python' (the former is preferred). They must also specify a dependency on `python', with a versioned dependency if necessary. If the program needs the python module `foo', it must depend on `python-foo'. 3.1.1. Programs Shipping Private Modules ---------------------------------------- A program using `/usr/bin/python' as interpreter can come up with private Python modules. These modules should be installed in `/usr/share/', or `/usr/lib/' if the modules are architecture-dependent (e.g. extensions). `/usr/lib/site-python' is deprecated and should no longer be used for this purpose. The rules explained in Section 2.6, `Modules Bytecompilation' apply to those private modules: the bytecompiled modules must not be shipped with the package, they should be generated in the package's postinst, using the current default Python version, and removed in the prerm. Modules should be bytecompiled using the current default Python version. Programs that have private compiled extensions must either handle multiple version support themselves, or declare a tight dependency on the current Python version (e.g. `Depends: python (>= 2.4), python (<= 2.5)'. No tools currently exist to alleviate this situation. 3.2. Programs Using a Particular Python Version ----------------------------------------------- A program which requires a specific version of Python must begin with `#!/usr/bin/python.' (or `#!/usr/bin/env python.'). It must also specify a dependency on `python.' and on any `python.-foo' package providing necessary modules. It should not depend on any `python-foo' package, unless it requires a specific version of the package (since virtual packages cannot be versioned). If this is the case, it should depend on both the virtual package and the main package (e.g. `Depends: python2.4-foo, python-foo (>= 1.0)'). The notes on installation directories and bytecompilation for programs that support any version of Python also apply to programs supporting only a single Python version. Modules to be bytecompiled should use the same Python version as the package itself. ------------------------------------------------------------------------------- 4. Programs Embedding Python ---------------------------- 4.1. Building Embedded Programs ------------------------------- Programs which embed a Python interpreter must declare a `Build-Depends' on `python.-dev', where python. is the python version the program builds against. It should be the current default python version unless the program doesn't work correctly with this version. 4.2. Embedded Python Dependencies --------------------------------- Dependencies for programs linking against the shared Python library will be automatically created by `dpkg-shlibdeps'. The `libpython..so.' library the program is built against is provided by the `python.' package. ------------------------------------------------------------------------------- 5. Interaction with Locally Installed Python Versions ----------------------------------------------------- As long as you don't install other versions of Python in your path, Debian's Python versions won't be affected by a new version. If you install a different subrelease of the version of python you've got installed, you'll need to be careful to install all the modules you use for that version of python too. ------------------------------------------------------------------------------- A. Build Dependencies --------------------- Build dependencies for Python dependent packages must be declared for every Python version that the package is built for. The `python-all-dev' should be used when building modules for any or all Python versions. To build for a specific version or versions, Build-Depend on `python.-dev'. Some applications and pure Python modules may be able to depend only on `python' or `python-all' and not require the -dev packages. Build-Depend on at least: Build-Depends: python2.3 (>= 2.3-1) Build-Depends: python2.4 (>= 2.4-1) Build-Depends: python (>= 2.3.5-7) Build-Depends: python-all Build-Depends: python2.3-dev (>= 2.3-1) Build-Depends: python2.4-dev (>= 2.4-1) Build-Depends: python-dev (>= 2.3.5-7) Build-Depends: python-all-dev If you use either `python-support' or `python-central' you must additionally Build-Depend on those. If you are using `dh_python' at all, you must Build-Depend on `python', as `debhelper' does not depend on it. ------------------------------------------------------------------------------- B. Packaging Tools ------------------ This section describes the various tools to help package Python programs and modules for Debian. Although none of these tools are mandatory, their use is strongly encouraged, as the above policy has been designed with them in mind (and vice versa). This appendix is just an overview. If you use these tools, you should read their full documentation. B.1. python-support ------------------- The python-support system provides a simple way to bytecompile pure Python modules and manage dependencies. It integrates with `debhelper'. When using python-support, you should install your modules to `/usr/share/python-support/' rather than the standard Python directories. python-support will then handle compiling the modules and making appropriate symbolic links for installed Python versions to find them, substitute `${python:Depends}', `${python:Versions}', and `${python:Provides}' in your control file, and manage bytecompilation in your postinst/prerm. To use it, call `dh_pysupport' before `dh_python', and make sure you've installed the modules in the right place: PREFIX := debian/python-package/usr ... install: ... ./setup.py install --no-compile \ --install-lib=$(PREFIX)/share/python-support/python-package binary-indep: build install ... dh_pysupport dh_python ... python-support can also manage private modules. To use this feature, pass a list of directories to be managed by python-support to `dh_pysupport' and `dh_python'. python-support cannot handle compiled extensions. B.2. python-central ------------------- python-central provides another way to manage Python modules. It integrates with `debhelper', but can also be used without it. When using python-central, you should install your modules normally. It will then move them to its private directory, and manage the same things python-support does. To use it, call `dh_pycentral' before `dh_python': install: ... ./setup.py install binary-indep: build install ... dh_pycentral dh_python ... python-central can handle compiled extensions for multiple Python versions. If you want python-central to handle private modules, you must pass the list of directories containing them to `dh_python' (but not `dh_pycentral'). If python-central should not move the files to its private directory, use`DH_PYCENTRAL=nomove dh_pycentral' instead. Examples for source packages using python-central are pyenchant, python-imaging (modules and extensions), pyparallel (modules only). B.3. CDBS --------- FIXME: Someone familiar with CDBS should write this part. ------------------------------------------------------------------------------- C. Upgrade Procedure -------------------- This section describes the procedure for the upgrade when the default python version is changed in the `unstable' distribution, requiring recompilation of many python-related packages. 1. Have a long and heated discussion. 2. The Debian Python maintainer decides for the new default Debian Python version and announces the upgrade. 3. Upload of the python core metapackages `python', `python-dev', `python-doc' and several `python-', depending on the new `python.', `python.-dev' and so on. 4. The release team schedules rebuilds for packages that may need it. Packages that require manual work get updated and uploaded. ------------------------------------------------------------------------------- Debian Python Policy Neil Schemenauer Matthias Klose Gregor Hoffleit Josselin Mouette Joe Wreschnig version 0.4.1.0