Re: python packaging infrastructure
Steve Langasek writes: > On Wed, Jan 18, 2006 at 01:06:39PM +0100, Matthias Klose wrote: > > Josselin Mouette writes: > > > Le lundi 16 janvier 2006 à 15:24 +0100, Matthias Klose a écrit : > > > > This is the right direction, and adding support for extensions makes > > > > this complete. Does your proposal allow rebuilding these packages > > > > without actually changing anything (except the changelog). > > > > Being able to rebuild packages for a new python version without changing > > > anything was the purpose of dh_python from the very beginning, for both > > > packaging styles: > > > * for packages with a single python-foo package containing > > > extension foo, build-depending on python-dev, a rebuild will > > > generate a new package built against the new version; > > > * for packages with python2.3-foo and python2.4-foo, a rebuild > > > will make python-foo depend on the new version. The only case > > > that isn't handled is when the package isn't maintained much, > > > and lacks python2.5-foo when the python2.5 transition > > > approaches. > > > This is independent of python-support. > > > the design decision of putting the binary-all python packages in a > > separate directory into /var/lib/python2.x/site-packages has some > > problems when supporting packages with extensions (a proposal beeing > > made on #irc was to keep the extensions in the standard path). > > > suppose you have the following scenario > > > /usr/lib/python2.3/site-packages/foo/ > > __init__.py > > fooext.so (doing a "import foomod") > > foomod.py > > > which is splitted into (by python-support) > > > /usr/lib/python2.3/site-packages/foo/ > > __init__.py > > fooext.so > > > /var/lib/python2.3/site-packages/foo/ > > __init__.py > > foomod.py > > > > Having /var/lib/python2.3/site-packages appended to sys.path let's the > > import of foomod fail (cannot be found). Using just one package > > directory inside /usr/lib/python2.3/site-packages does avoid the > > problem (the way the current python-central works). > > I think this objection is misguided for the following reason. > > If a package includes a python extension, then any .py files it also > includes are either in the same directory (or same tree) with those .so > files, or they are in some other unrelated directory. > > If they are in the same directory, then *that directory has the name of the > python implementation in its path*, because .so's are not shared between > python implementations. In this case, it is not appropriate or reasonable > for these .py files to be managed via a symlink farm, because each minor > version of python has its own separate .py file (which may not vary from > release to release, but that's beside the point). correct, we run into problems if a package (python meaning of package) is hold into two locations. > If they are in different directories, then foomod.py already doesn't enjoy a > privileged location in the current path; so whether they are symlink-farmed > or not, there must already be handling in place to locate foomod.py, so it > doesn't much matter which particular directory it's located in. > > Perhaps you are arguing that a package should be able to ship > /usr/lib/python2.3/site-packages/foo/fooext.so and > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure > make foomod.py (or foomod.pyc?) available in > /usr/lib/python2.3/site-packages/foo. I'm not sure why this would be > advantageous? > > > So how does python-support handle packages, where the extension > > module is split out in it's own binary package? > > If they're split out in separate binary packages, then surely the extension > package doesn't require any special handling, and the module package can be > managed as before? yes, another bunch of packages which then cannot be handled by the packaging infrastructure. > (since obviously it's not sensible to split > /usr/lib/python2.3/site-packages/foo between two separate packages...) obviously? - In the past packagers were encouraged to split packages this way to save disk space (at least on the ftp mirrors), if the size of the arch independent part exceeds the arch dependent part significantly. - packages are split this way, if the package does depend on other libraries or packages, which are not always needed (i.e. have a dependency on X in a separate package, have a separate package for each database adaptor, built from the same package source). - packages are distributed upstream this way, so you have different sources sounds rather sensible to split these. Coming back to: > Perhaps you are arguing that a package should be able to ship > /usr/lib/python2.3/site-packages/foo/fooext.so and > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure > make foomod.py (or foomod.pyc?) available in > /usr/lib/python2.3/site-packages/foo. I'm not sure why this
Re: when and why did python(-minimal) become essential?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 19, 2006 at 05:58:20PM -0500, David Nusinow wrote: > On Thu, Jan 19, 2006 at 01:47:18PM -0800, Matt Zimmerman wrote: > > On Thu, Jan 19, 2006 at 09:23:30PM +, Martin Michlmayr wrote: > > > * Matt Zimmerman <[EMAIL PROTECTED]> [2006-01-19 12:45]: > > > > Please don't do this; it implies that python-minimal would be part > > > > of base, but not full python, and this is something that python > > > > upstream explicitly objects to. > > > > > > Why? Surely having a sub-set of python is better than nothing at all, no? > > > > One of the appealing things about the Python language is their "batteries > > included" philosophy: users can assume that the standard library is > > available, documentation and examples are written to the full API, etc. > > When it's broken into pieces, they get complaints and support requests from > > their user community when things don't work the way they should. > > For what it's worth, we've caught hell from the ruby community for breaking > the standard library in to its component parts and not installing it all by > default. This problem has been largely abrogated as of late, but I'd rather > not see us piss off the python community for making a similar mistake. > > That said, I don't really understand why it's Ok for Ubuntu to do this but > not us. > > - David Nusinow Hi Debian folks, Debian seems to like to support embedded devices and allow folks to install as little as possible in/for a base install. And in that vein, the discussion is on 'essential' packages. I can understand if a whole language community got pissed if when you install a standard-level packages like 'perl' and then it was missing pieces, but aren't Debian devs allowed to design packages for our philosophical/project goals in regards to a 'mininal' install when we design an 'essential' packages? If Ubuntu's goals are to heavily use/promote Python and feel its 'essential' to include the whole shebang and not part, then that's their goals and its fine by me. Giving away code (GPL or otherwise) to the world is done for many reasons. Aparently some folks are more concerned about how their work is used. As with the attribution in .debs, folks want the users to not associate possible (as judged by them) 'bad'/'unofficial'/'off project'/'different' work with their projects. But the perl folks don't seem to have that objection! x-) (at least none have spoken yet!) cheers, Kev - -- counter.li.org #238656 -- goto counter.li.org and be counted! `$' $' $ $ _ ,d$$$g$ ,d$$$b. $,d$$$b`$' g$b $,d$$b ,$P' `$ ,$P' `Y$ $$' `$ $ "' `$ $$' `$ $$ $ $$g$ $ $ $ ,$P"" $ $$ `$g. ,$$ `$$._ _. $ _,g$P $ `$b. ,$$ $$ `Y$$P'$. `YP $$$P"' ,$. `Y$$P'$ $. ,$. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFD0JFJv8UcC1qRZVMRAmvJAJ9AVNJDhdpKRYO2bcCj80u084hPVQCfXnn3 qCUzzKlIHj8v4eieYXR1JvE= =lEuL -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
Joe Wreschnig writes: > On Thu, 2006-01-19 at 12:12 +1000, Anthony Towns wrote: > > I don't know what's actually in (or more importantly not in) > > python2.4-minimal though. > > I'm eyeballing right now. Things that jump out at me: > * No character encoding, translation, or locale handling. > * A little oddly, loss of shutil. > * No sockets. > > The first one seems like it would be a show-stopper to me, unless we > expect programs in the base system to only deal with ASCII. This is a > fairly large addition to package, too. > > The second can easily be fixed; possibly just oversight. It's a small > module and gives Python equivalents of cp -r, rm -r, and mv. > > The third seems like something software in base may want to do; I > mention it specifically because perl-base include socket support. Colin already mentioned that the socket modules are in -minimal. shutil looks reasonable. encodings and locale handling come with a prize: a size of about 3MB, which would more than double the size of -minimal (2.4 ships with the cjk codecs). Matthias -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
Joey Hess writes: > Colin Watson wrote: > > FWIW the relevant design docs from when this was done in Ubuntu are > > here: > > > > https://wiki.ubuntu.com/EssentialPython (requirements) > > https://wiki.ubuntu.com/PythonInEssential (details) > > > > The rationale for the set of included modules is in the latter, and was > > basically done by taking each module in perl-base and mapping it to its > > Python equivalent. > > FWIW, that's a fairly strange way to do it, since modules are > added/removed from perl-base as needed by the perl-using programs in the > base system. No, if you do look at https://wiki.ubuntu.com/PythonInEssential, you will notice: Do not include: * _ssl, pickle, cPickle, pickle ends up as a dependency of subprocess. > For example, perl-base includes Data::Dumper because debconf > (used to) use it, not because there's any other particular reason to > include that module in base, and I've just asked that Data::Dumper be > removed, so including its equivilant (pickle) in python-base on that > rationalle is decidely strange. Ubuntu did use perl-base just as a starting point. > If we followed the same method for python-base, then we would > > a) instroduce python-base iff we had some package(s) written in python >that we wanted in the base system (apt-listchanges comes to mind) > b) include only the modules needed by the package(s). We once had a python-base package and got complaints about the name being misleading. Besides that, I got questions from Debian only developers and Debian users to have the minimal package in Debian as well. That does not look misleading, as long as the name implies that you cannot expect a complete python installation. Matthias -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
On Thu, Jan 19, 2006 at 05:58:20PM -0500, David Nusinow wrote: > For what it's worth, we've caught hell from the ruby community for > breaking the standard library in to its component parts and not > installing it all by default. This problem has been largely abrogated > as of late, but I'd rather not see us piss off the python community > for making a similar mistake. I believe the problem with the ruby situation wasn't that the monolithic ruby distribution was split up; but that there was no clear way to install the lot in one go, without prior knowledge of what the whole distribution was: a simple meta-package with the correct dependencies was all that was missing. -- Jon Dowland http://alcopop.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: python packaging infrastructure
On Fri, Jan 20, 2006 at 10:47:19AM +0100, Matthias Klose wrote: > Steve Langasek writes: > > On Wed, Jan 18, 2006 at 01:06:39PM +0100, Matthias Klose wrote: > > > the design decision of putting the binary-all python packages in a > > > separate directory into /var/lib/python2.x/site-packages has some > > > problems when supporting packages with extensions (a proposal beeing > > > made on #irc was to keep the extensions in the standard path). > > If they are in different directories, then foomod.py already doesn't enjoy a > > privileged location in the current path; so whether they are symlink-farmed > > or not, there must already be handling in place to locate foomod.py, so it > > doesn't much matter which particular directory it's located in. > > Perhaps you are arguing that a package should be able to ship > > /usr/lib/python2.3/site-packages/foo/fooext.so and > > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure > > make foomod.py (or foomod.pyc?) available in > > /usr/lib/python2.3/site-packages/foo. I'm not sure why this would be > > advantageous? > > > So how does python-support handle packages, where the extension > > > module is split out in it's own binary package? > > If they're split out in separate binary packages, then surely the extension > > package doesn't require any special handling, and the module package can be > > managed as before? > yes, another bunch of packages which then cannot be handled by the > packaging infrastructure. Uh, no, I meant "can be managed as before [by python-support]". > > (since obviously it's not sensible to split > > /usr/lib/python2.3/site-packages/foo between two separate packages...) > obviously? > - In the past packagers were encouraged to split packages this > way to save disk space (at least on the ftp mirrors), if the size of > the arch independent part exceeds the arch dependent part significantly. > - packages are split this way, if the package does depend on other > libraries or packages, which are not always needed (i.e. have a > dependency on X in a separate package, have a separate package for > each database adaptor, built from the same package source). > - packages are distributed upstream this way, so you have different > sources > sounds rather sensible to split these. Ah. So this happens in practice? These do all seem to be reasonable; apparently my "obvious" answer was wrong. :) > Coming back to: > > Perhaps you are arguing that a package should be able to ship > > /usr/lib/python2.3/site-packages/foo/fooext.so and > > /usr/lib/python/site-packages/foo/foomod.py, and have the infrastructure > > make foomod.py (or foomod.pyc?) available in > > /usr/lib/python2.3/site-packages/foo. I'm not sure why this would be > > advantageous? > I.e. in version 1, a package is implemented as a pure python module, > the package maintainer uses the new packaging infrastructure, version > 2 implements some functionality in an extension module to speedup > things. Why does a package maintainer have to change the packaging > infrastructure? Er... because the nature of the package has changed? There are going to be changes required here anyway; adding a .so to the package means it can no longer be arch: all, can no longer be invariant wrt multiple python versions, has to build the module once for each flavor of python supported, etc. I believe we should provide the best possible packaging tools to simplify this for maintainers, but I think that expecting a maintainer not to have to do any work to convert from a pure-python package to a package with an extension is unrealistic. BTW, perl has had split arch:any/arch:all paths for modules for years, and I haven't heard anyone complaining about it; I'm really not sure why python should be different. > Make it worse, if a pure python module offering some kind of plugin > structure uses the infrastructure, another package (maybe even > maintained by somebody else) chooses to implement the plugin as an > extension enforces the other maintainer to drop using the packaging > infrastructure. This is because the plugin search path will be relative to the directory the python module is installed in? Can you give me some example python code that would fail in this scenario? > > Also, if you really put all of these files in > > /usr/lib/python2.3/site-packages, doesn't that make it unnecessarily > > difficult to distinguish between symlinks managed by python-support, and > > symlinks managed by the packaging system? > That's another point, but unrelated to the design limitation to not > support arch dependent packages. From a user's point of view it > looks like a normal installation, every step taken to make it better > to distinguish exposes the implementaion to the user. > How do you currently distinguish these symlinks? Is it difficult to > look at the name to which the symlink points? Current Debian policy > doesn't disallow them, is there something planned
python-sqlobject v0.7 package?
Hi all, I was wondering on the status of a version 0.7 package for python-sqlobject. It's current package is still at version 0.6, while version 0.7 has been released in October 2005. It seems the package maintainer has been notified of the new version in November 2005 through the bug report system, though there is still no new package available or reply from the maintainer for that matter. I would like to use some functionality from the new version and would love an update or status on it. Kind regards, - Ramon. -- There are really only three types of people: Those who make things happen, those who watch things happen, and those who say, "What happened?" --- ing. R. Bastiaans HPC - Systems Programmer SARA - Computing and Networking Services Kruislaan 415 PO Box 194613 1098 SJ Amsterdam 1090 GP Amsterdam -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
On Thu, Jan 19, 2006 at 10:38:08PM -0800, Thomas Bushnell BSG wrote: > Ok, but now I'm confused: why is python-minimal needed in Essential? > Why not simply depend on it straightforwardly? Because there are parts of the packaging system where there is no way to express such a dependency relationship, and so only Essential packages can be relied upon to be available. One example is .config maintainer scripts, some of which are quite complex and worth writing in a higher-level language than shell. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
On Fri, Jan 20, 2006 at 09:22:53AM -0800, Matt Zimmerman wrote: > On Thu, Jan 19, 2006 at 10:38:08PM -0800, Thomas Bushnell BSG wrote: > > Ok, but now I'm confused: why is python-minimal needed in Essential? > > Why not simply depend on it straightforwardly? > Because there are parts of the packaging system where there is no way to > express such a dependency relationship, and so only Essential packages can > be relied upon to be available. > One example is .config maintainer scripts, some of which are quite complex > and worth writing in a higher-level language than shell. I asked this question earlier, and no one answered. Are there .config scripts being written in python today in Ubuntu? (Hmm, where are the python bindings for debconf, and what ensures that they're installed?) -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ signature.asc Description: Digital signature
Re: python-sqlobject v0.7 package?
Il giorno ven, 20/01/2006 alle 18.10 +0100, Ramon Bastiaans ha scritto: > Hi all, > > I was wondering on the status of a version 0.7 package for python-sqlobject. > It's current package is still at version 0.6, while version 0.7 has been > released in October 2005. > > It seems the package maintainer has been notified of the new version in > November 2005 through the bug report system, though there is still no > new package available or reply from the maintainer for that matter. > > I would like to use some functionality from the new version and would > love an update or status on it. > > Kind regards, > - Ramon. I'm working on this, and I'll adopt the package. Expect an upload of 0.7-1 in a few days. -- Fabio Tranchitella <[EMAIL PROTECTED]>.''`. Proud Debian GNU/Linux developer, admin and user.: :' : `. `'` http://people.debian.org/~kobold/ `- _ 1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564 signature.asc Description: This is a digitally signed message part
Re: when and why did python(-minimal) become essential?
On Fri, Jan 20, 2006 at 09:52:09AM -0800, Matt Zimmerman wrote: > On Fri, Jan 20, 2006 at 09:40:55AM -0800, Steve Langasek wrote: > > I asked this question earlier, and no one answered. Are there .config > > scripts being written in python today in Ubuntu? (Hmm, where are the python > > bindings for debconf, and what ensures that they're installed?) > No, not yet. The promotion to Essential needed to happen prior to writing > any such scripts. Well, technically a .config script that fails because /usr/bin/python doesn't exist would get a second chance when the postinst runs, just like any other config script failure... so you could get away with this just using a Depends:, you just lose pre-configuration support ;) > The python bindings for debconf are, of all places, in the 'debconf' > package. Hurray! :) -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ signature.asc Description: Digital signature
Re: when and why did python(-minimal) become essential?
On Fri, Jan 20, 2006 at 09:40:55AM -0800, Steve Langasek wrote: > I asked this question earlier, and no one answered. Are there .config > scripts being written in python today in Ubuntu? (Hmm, where are the python > bindings for debconf, and what ensures that they're installed?) No, not yet. The promotion to Essential needed to happen prior to writing any such scripts. The python bindings for debconf are, of all places, in the 'debconf' package. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
Matt Zimmerman wrote: > On Thu, Jan 19, 2006 at 03:34:58PM -0500, Joey Hess wrote: > > If we followed the same method for python-base, then we would > > > > a) instroduce python-base iff we had some package(s) written in python > >that we wanted in the base system (apt-listchanges comes to mind) > > b) include only the modules needed by the package(s). > > Please don't do this; it implies that python-minimal would be part of base, > but not full python, and this is something that python upstream explicitly > objects to. It implies no such thing. -- see shy jo signature.asc Description: Digital signature
Re: when and why did python(-minimal) become essential?
Kevin Mark wrote: > Giving away code (GPL or otherwise) to the world is done for many > reasons. Aparently some folks are more concerned about how their work > is used. As with the attribution in .debs, folks want the users to not > associate possible (as judged by them) 'bad'/'unofficial'/'off > project'/'different' work with their projects. But the perl folks don't > seem to have that objection! x-) (at least none have spoken yet!) perl is priority standard and so it is part of default Debian installs unless the user explcitly asks aptitude not to install it. -- see shy jo signature.asc Description: Digital signature
Re: python-sqlobject v0.7 package?
Cool, great. Thanks for the reply and I'll await it's release. ;) Fabio Tranchitella wrote: Il giorno ven, 20/01/2006 alle 18.10 +0100, Ramon Bastiaans ha scritto: Hi all, I was wondering on the status of a version 0.7 package for python-sqlobject. It's current package is still at version 0.6, while version 0.7 has been released in October 2005. It seems the package maintainer has been notified of the new version in November 2005 through the bug report system, though there is still no new package available or reply from the maintainer for that matter. I would like to use some functionality from the new version and would love an update or status on it. Kind regards, - Ramon. I'm working on this, and I'll adopt the package. Expect an upload of 0.7-1 in a few days. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: when and why did python(-minimal) become essential?
On Fri, Jan 20, 2006 at 02:05:40PM -0500, Joey Hess wrote: > Matt Zimmerman wrote: > > On Thu, Jan 19, 2006 at 03:34:58PM -0500, Joey Hess wrote: > > > If we followed the same method for python-base, then we would > > > > > > a) instroduce python-base iff we had some package(s) written in python > > >that we wanted in the base system (apt-listchanges comes to mind) > > > b) include only the modules needed by the package(s). > > > > Please don't do this; it implies that python-minimal would be part of base, > > but not full python, and this is something that python upstream explicitly > > objects to. > > It implies no such thing. If you won't acknowledge that, then know that upstream also object to the name "python-base" for something which has a stripped-down standard library. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
