I would advise against using or even reading about our egg extensions, as the implementation is full of legacy (we've been doing this many years :) ): http://enstaller.readthedocs.org/en/master/reference/egg_format.html
This is what we use on top of setuptools egg: - ability to add dependencies which are not python packages (I think most of it is already handled in metadata 2.0/PEP 426, but I would have to re-read the PEP carefully). - ability to run post/pre install/remove scripts - support for all the of the autotools directories, with "sensible" mapping on windows - a few extensions to the actual binary format (adding support for symlinks is the only one I can think of ATM). Everything else is legacy you really don't want to know (see here if you still want to http://enstaller.readthedocs.org/en/master/reference/egg_format.html) David On Mon, Apr 13, 2015 at 3:55 PM, Daniel Holth <dho...@gmail.com> wrote: > On Mon, Apr 13, 2015 at 3:46 PM, David Cournapeau <courn...@gmail.com> > wrote: > > > > > > On Mon, Apr 13, 2015 at 12:56 PM, Chris Barker <chris.bar...@noaa.gov> > > wrote: > >> > >> NOTE: I don't work for any of the companies involved -- just a somewhat > >> frustrated user... And someone that has been trying for years to make > things > >> easier for OS-X users. > >> > >>>> I’m not sure what (3) means exactly. What is a “normal” Python, do you > >>>> modify Python in a way that breaks the ABI but which isn’t reflected > in the > >>>> standard ABI tag? > >>> > >>> > >>> It could be multiple things. The most obvious one is that generally. > >>> cross-platforms python distributions will try to be "relocatable" > (i.e. the > >>> whole installation can be moved and still work). This means they > require > >>> python itself to be built a special way. Strictly speaking, it is not > an ABI > >>> issue, but the result is the same though: you can't use libraries from > >>> anaconda or canopy on top of a normal python > >> > >> > >> But why not? -- at least for Anaconda, it's because those libraries > likely > >> have non-python dependencies, which are expected to be installed in a > >> particular way. And really, this is not particular to Anaconda/Canopy at > >> all. Python itself has no answer for this issue, and eggs and wheels > don't > >> help. Well, maybe kinda sorta they do, but in a clunky/ugly way: in > order to > >> build a binary wheel with non-python dependencies (let's say something > like > >> libjpeg, for instance), you need to either: > >> - assume that libjpeg is installed in a "standard" place -- really no > >> solution at all (at least outside of linux) > >> - statically link it > >> - ship the dynamic lib with the package > >> > >> For the most part, the accepted solution for OS-X has been to statically > >> link, but: > >> > >> - it's a pain to do. The gnu toolchain really likes to use dynamic > >> linking, and building a static lib that will run on a > >> maybe-older-than-the-build-system machine is pretty tricky. > >> > >> - now we end up with multiple copies of the same lib in the python > >> install. There are a handful of libs that are used a LOT. Maybe there > is no > >> real downside -- disk space and memory are cheap these days, but it sure > >> feels ugly. And I have yet to feel comfortable with having multiple > versions > >> of the same lib linked into one python instance -- I can't say I've > seen a > >> problem, but it makes me nervous. > >> > >> On Windows, the choices are the same, except that: It is so much harder > to > >> build many of the "standard" open source libs that package authors are > more > >> likely to do it for folks, and you do get the occasional "dll hell" > issues. > >> > >> I had a plan to make some binary wheels for OS-X that were not really > >> python packages, but actually just bundled up libs, so that other wheels > >> could depend on them. OS-X does allow linking to relative paths, so this > >> should have been doable, but I never got anyone else to agree this was a > >> good idea, and I never found the roundtoits anyway. And it doesn't > really > >> fit into the PyPi, pip, wheel, etc. philosphy to have dependencies that > are > >> platform dependent and even worse, build-dependent. > >> > >> Meanwhile, conda was chugging along and getting a lot of momentum in the > >> Scientific community. And the core thing here is that conda was designed > >> from the ground up to support essentially anything, This means is > supports > >> python packages that depend on non-python packages, but also supports > >> packages that have nothing to do with python (Perl, command line tools, > what > >> have you...) > >> > >> So I have been focusing on conda lately. > > > > > > The whole reason I started this discussion is to make sure wheel has a > > standard way to do what is needed for those usecases. > > > > conda, rpm, deb, or eggs as used in enthought are all essentially the > same: > > an archive with a bunch of metadata. The real issue is standardising on > the > > exact formats. As you noticed, there is not much missing in the wheel > *spec* > > to get most of what's needed. We've used eggs for that purpose for > almost 10 > > years at Enthought, and we did not need that many extensions on top of > the > > egg format after all. > > > > > >> > >> Which brings me back to the question: should the python tools (i.e. > wheel) > >> be extended to support more use-cases, specifically non-python > dependencies? > >> Or do we just figure that that's a problem better solved by projects > with a > >> larger scope (i.e. rpm, deb, conda, canopy). > > > > > > IMO, given that wheels do most of what's needed, it is worth supporting > most > > simple usecases (compiled libraries required by well known extensions). > > Right now, such packages (pyzmq, numpy, cryptography, lxml) resort to > quite > > horrible custom hacks to support those cases. > > > > Hope that clarifies the intent, > > > > David > > Then it sounds like I should read about the Enthought egg extensions. > It's something else than just defining a separate pypi name for "just > the libxml.so without the python bits"? >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
