On Tue, Oct 9, 2012 at 5:45 PM, Vinay Sajip <vinay_sa...@yahoo.co.uk> wrote: > PJ Eby <pje <at> telecommunity.com> writes: > >> >> In pkg_resources, entry points are references to any importable >> object. They don't have to be callables. >> >> Note, however, that if you load an entry point which is a module, then >> of course the module code will be executed as a side-effect, so >> technically, a module still meets the "start of some executable code" >> standard. > > Yes, I acknowledged this in my response to Lennart's post. > >> In any case, the name was chosen for the primary use case of >> designating executable entry points in Python code, often callables. >> However, given the nature of Python, there was absolutely no reason to >> restrict them to referring *only* to callables. (In fact, the >> feature's name and implementation was inspired by Eclipse's "extension >> points" architecture, where the advertised objects usually implement >> interfaces with more than one method.) > > Right, they're just objects of one sort or another. > >> >> If the name *must* be changed, I'd suggest using "exports", for >> symmetry with "imports". Entry points are always the name of an >> importable object, and they are basically being advertised so that >> other code knows what is available to be imported, and the loading API >> allows one to handle other details such as updating sys.path as >> necessary, and doing the actual import. > > As you say, entry points don't have to be actual entry points, and I didn't > want to slavishly follow pkg_resources nomenclature; though it reflects the > most common use case, it's not the only one. Your suggestion of "exports" > seems very reasonable as a replacement for "registry", which I agree is too > generic. > >> I understand that right now you're not supporting those other features >> in distlib, but I feel I should mention as a reminder that many of >> these pkg_resources features were created to support application >> platforms with plugins, such as Trac, Zope, Chandler, etc., and to >> prevent the need for wheel-reinvention between different Python plugin >> systems. For this type of platform, it's often an important core >> feature to be able to dynamically add plugins and their requirements >> to sys.path on an as-needed basis when the entry points are actually >> *used*, because early loading or additions to sys.path can >> dramatically affect application startup times. > > I understand, and appreciate your pointing these things out. As I see it, > the specific way that some of those features were implemented is what some > people object to. In distlib, importing is still lazy, but sys.path is not > updated before attempting imports. Clearly, distlib can't do things exactly > the way they're done in pkg_resources and setuptools, since those are not > considered suitable to go into the stdlib, whereas distlib hopes to provide > (some of) the useful features provided in those packages in a (hopefully) > non-contentious way, with perhaps part of it going into the stdlib at some > future date if there's a consensus about that. > > ISTM that the "exports" feature in distlib allows applications to cooperate in > the same way as pkg_resources' entry points do, except for the fact that > sys.path is not manipulated at all, meaning that all such cooperating > applications are assumed to be installed on the path. Actual importing is > delayed until needed; scanning of distribution metadata for exports also > happens on demand. Do you feel that this is insufficient to meet the > performance concerns you mentioned?
As far as my performance concerns are concerned, I just want to be able to avoid iterating through all the installed distributions twice for code that is using both APIs, which besides the porting hassle is why I might implement one API on top of the other implementation. I would also like to be able to store my installed package database in sqlite3 by implementing an appropriate distlib/pkg_resources backend and defining a standard post-(un)install "index this new package" hook but I doubt I will have the (unfunded) time. I'm sure sys.path manipulation could be implemented on top of a basic Distributions-from-path API to search for the entry points, without having to be integrated with it. FYI Ruby's gem seems to do this with the equivalent of a Python import hook (our sys.meta_path) and they do index the available gems in an actual not-just-a-bunch-of-files database. There is an interesting but long discussion at http://python.6.n6.nabble.com/Re-Python-Dev-PEP-365-Adding-the-pkg-resources-module-td1995421.html GvR starts off by wishing it wasn't 1700 lines long. It seems "easy to review" was a more important factor than the specific name of any particular feature, except for "easy_install" which was seen as too much of a setuptools brand name. Vinay wrote: I don't believe that wheels are meant to be directly importable, but I could be wrong about that. I am not sure why callee dependencies need to be transparently met by distributions not on sys.path; Broadly speaking, and leaving eggs out of the picture, I would expect a distribution to declare its dependencies, and for these to be installed in the appropriate site-packages by the installer. > > Do you mean that it is not possible to identify the entry points of an > > arbitrary distribution? What about obtaining information about the > At the moment, not for a completely arbitrary distribution, as we don't > have the metadata in non-installed distributions. Of course, an arbitrary > distribution could be unpacked to a temporary location and have the equivalent > of egg-info run to get the information; but for this to work ideally, such > information would need to be present in the archive metadata and also need to > be in the index. There are a number of ways this could happen - it's not > clear to me what the best way is, but I probably haven't thought about it > enough. At this point of the runtime, the "arbitrary non-installed distribution" is just a dist that has been installed outside of sys.path. So the static metadata is there. Wheels are designed to be valid sys.path entries in the simple case. We don't encourage you to do so mostly because we don't want to include .pyc files for all the Pythons you might use, but it is handy for bootstrapping. python wheel-0.12.0-py2.py3-none-any.whl/wheel install wheel-0.12.0-py2.py3-none-any.whl installs wheel's own wheel with itself, even when pkg_resources.py is not installed. (You could do the same thing with PYTHONPATH=... python -m wheel install). You could imagine including with Python a similar bootstrap installer as a runnable .zip file (invent an extension .pya / .pyp?). Since the functionality is hidden in the .zip file, you would only have to commit to and argue about the command-line API instead of the much larger package management API. The bootstrap installer would install the real installer or build systems such as distutils, setuptools, Bento, distlib, providing that those projects were willing to publish themselves to pypi as pre-built wheel archives. If you want to play around with renaming the egg-info files, check out Bento's egg_utils.py line 106: https://github.com/cournape/Bento/blob/master/bento/commands/egg_utils.py#L106 which controls what each metadata file is named. file_table = { "pkg_info": "PKG-INFO", "sources": "SOURCES.txt", "install_requires": "requires.txt", "top_levels": "top_level.txt", "not_zip_safe": "not-zip-safe", "dependency_links": "dependency_links.txt", "entry_points": "entry_points.txt", "ipkg_info": "ipkg.info", } _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
