Seems like you could extend wheel to do that easily. On Apr 13, 2015 4:19 PM, "David Cournapeau" <courn...@gmail.com> wrote:
> I would advise against using or even reading about our egg extensions, as > the implementation is full of legacy (we've been doing this many years :) > ): http://enstaller.readthedocs.org/en/master/reference/egg_format.html > > This is what we use on top of setuptools egg: > > - ability to add dependencies which are not python packages (I think most > of it is already handled in metadata 2.0/PEP 426, but I would have to > re-read the PEP carefully). > - ability to run post/pre install/remove scripts > - support for all the of the autotools directories, with "sensible" > mapping on windows > - a few extensions to the actual binary format (adding support for > symlinks is the only one I can think of ATM). > > Everything else is legacy you really don't want to know (see here if you > still want to > http://enstaller.readthedocs.org/en/master/reference/egg_format.html) > > David > > On Mon, Apr 13, 2015 at 3:55 PM, Daniel Holth <dho...@gmail.com> wrote: > >> On Mon, Apr 13, 2015 at 3:46 PM, David Cournapeau <courn...@gmail.com> >> wrote: >> > >> > >> > On Mon, Apr 13, 2015 at 12:56 PM, Chris Barker <chris.bar...@noaa.gov> >> > wrote: >> >> >> >> NOTE: I don't work for any of the companies involved -- just a somewhat >> >> frustrated user... And someone that has been trying for years to make >> things >> >> easier for OS-X users. >> >> >> >>>> I’m not sure what (3) means exactly. What is a “normal” Python, do >> you >> >>>> modify Python in a way that breaks the ABI but which isn’t reflected >> in the >> >>>> standard ABI tag? >> >>> >> >>> >> >>> It could be multiple things. The most obvious one is that generally. >> >>> cross-platforms python distributions will try to be "relocatable" >> (i.e. the >> >>> whole installation can be moved and still work). This means they >> require >> >>> python itself to be built a special way. Strictly speaking, it is not >> an ABI >> >>> issue, but the result is the same though: you can't use libraries from >> >>> anaconda or canopy on top of a normal python >> >> >> >> >> >> But why not? -- at least for Anaconda, it's because those libraries >> likely >> >> have non-python dependencies, which are expected to be installed in a >> >> particular way. And really, this is not particular to Anaconda/Canopy >> at >> >> all. Python itself has no answer for this issue, and eggs and wheels >> don't >> >> help. Well, maybe kinda sorta they do, but in a clunky/ugly way: in >> order to >> >> build a binary wheel with non-python dependencies (let's say something >> like >> >> libjpeg, for instance), you need to either: >> >> - assume that libjpeg is installed in a "standard" place -- really no >> >> solution at all (at least outside of linux) >> >> - statically link it >> >> - ship the dynamic lib with the package >> >> >> >> For the most part, the accepted solution for OS-X has been to >> statically >> >> link, but: >> >> >> >> - it's a pain to do. The gnu toolchain really likes to use dynamic >> >> linking, and building a static lib that will run on a >> >> maybe-older-than-the-build-system machine is pretty tricky. >> >> >> >> - now we end up with multiple copies of the same lib in the python >> >> install. There are a handful of libs that are used a LOT. Maybe there >> is no >> >> real downside -- disk space and memory are cheap these days, but it >> sure >> >> feels ugly. And I have yet to feel comfortable with having multiple >> versions >> >> of the same lib linked into one python instance -- I can't say I've >> seen a >> >> problem, but it makes me nervous. >> >> >> >> On Windows, the choices are the same, except that: It is so much >> harder to >> >> build many of the "standard" open source libs that package authors are >> more >> >> likely to do it for folks, and you do get the occasional "dll hell" >> issues. >> >> >> >> I had a plan to make some binary wheels for OS-X that were not really >> >> python packages, but actually just bundled up libs, so that other >> wheels >> >> could depend on them. OS-X does allow linking to relative paths, so >> this >> >> should have been doable, but I never got anyone else to agree this was >> a >> >> good idea, and I never found the roundtoits anyway. And it doesn't >> really >> >> fit into the PyPi, pip, wheel, etc. philosphy to have dependencies >> that are >> >> platform dependent and even worse, build-dependent. >> >> >> >> Meanwhile, conda was chugging along and getting a lot of momentum in >> the >> >> Scientific community. And the core thing here is that conda was >> designed >> >> from the ground up to support essentially anything, This means is >> supports >> >> python packages that depend on non-python packages, but also supports >> >> packages that have nothing to do with python (Perl, command line >> tools, what >> >> have you...) >> >> >> >> So I have been focusing on conda lately. >> > >> > >> > The whole reason I started this discussion is to make sure wheel has a >> > standard way to do what is needed for those usecases. >> > >> > conda, rpm, deb, or eggs as used in enthought are all essentially the >> same: >> > an archive with a bunch of metadata. The real issue is standardising on >> the >> > exact formats. As you noticed, there is not much missing in the wheel >> *spec* >> > to get most of what's needed. We've used eggs for that purpose for >> almost 10 >> > years at Enthought, and we did not need that many extensions on top of >> the >> > egg format after all. >> > >> > >> >> >> >> Which brings me back to the question: should the python tools (i.e. >> wheel) >> >> be extended to support more use-cases, specifically non-python >> dependencies? >> >> Or do we just figure that that's a problem better solved by projects >> with a >> >> larger scope (i.e. rpm, deb, conda, canopy). >> > >> > >> > IMO, given that wheels do most of what's needed, it is worth supporting >> most >> > simple usecases (compiled libraries required by well known extensions). >> > Right now, such packages (pyzmq, numpy, cryptography, lxml) resort to >> quite >> > horrible custom hacks to support those cases. >> > >> > Hope that clarifies the intent, >> > >> > David >> >> Then it sounds like I should read about the Enthought egg extensions. >> It's something else than just defining a separate pypi name for "just >> the libxml.so without the python bits"? >> > >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig