On Mon, Apr 13, 2015 at 3:46 PM, David Cournapeau <courn...@gmail.com> wrote: > > > On Mon, Apr 13, 2015 at 12:56 PM, Chris Barker <chris.bar...@noaa.gov> > wrote: >> >> NOTE: I don't work for any of the companies involved -- just a somewhat >> frustrated user... And someone that has been trying for years to make things >> easier for OS-X users. >> >>>> I’m not sure what (3) means exactly. What is a “normal” Python, do you >>>> modify Python in a way that breaks the ABI but which isn’t reflected in the >>>> standard ABI tag? >>> >>> >>> It could be multiple things. The most obvious one is that generally. >>> cross-platforms python distributions will try to be "relocatable" (i.e. the >>> whole installation can be moved and still work). This means they require >>> python itself to be built a special way. Strictly speaking, it is not an ABI >>> issue, but the result is the same though: you can't use libraries from >>> anaconda or canopy on top of a normal python >> >> >> But why not? -- at least for Anaconda, it's because those libraries likely >> have non-python dependencies, which are expected to be installed in a >> particular way. And really, this is not particular to Anaconda/Canopy at >> all. Python itself has no answer for this issue, and eggs and wheels don't >> help. Well, maybe kinda sorta they do, but in a clunky/ugly way: in order to >> build a binary wheel with non-python dependencies (let's say something like >> libjpeg, for instance), you need to either: >> - assume that libjpeg is installed in a "standard" place -- really no >> solution at all (at least outside of linux) >> - statically link it >> - ship the dynamic lib with the package >> >> For the most part, the accepted solution for OS-X has been to statically >> link, but: >> >> - it's a pain to do. The gnu toolchain really likes to use dynamic >> linking, and building a static lib that will run on a >> maybe-older-than-the-build-system machine is pretty tricky. >> >> - now we end up with multiple copies of the same lib in the python >> install. There are a handful of libs that are used a LOT. Maybe there is no >> real downside -- disk space and memory are cheap these days, but it sure >> feels ugly. And I have yet to feel comfortable with having multiple versions >> of the same lib linked into one python instance -- I can't say I've seen a >> problem, but it makes me nervous. >> >> On Windows, the choices are the same, except that: It is so much harder to >> build many of the "standard" open source libs that package authors are more >> likely to do it for folks, and you do get the occasional "dll hell" issues. >> >> I had a plan to make some binary wheels for OS-X that were not really >> python packages, but actually just bundled up libs, so that other wheels >> could depend on them. OS-X does allow linking to relative paths, so this >> should have been doable, but I never got anyone else to agree this was a >> good idea, and I never found the roundtoits anyway. And it doesn't really >> fit into the PyPi, pip, wheel, etc. philosphy to have dependencies that are >> platform dependent and even worse, build-dependent. >> >> Meanwhile, conda was chugging along and getting a lot of momentum in the >> Scientific community. And the core thing here is that conda was designed >> from the ground up to support essentially anything, This means is supports >> python packages that depend on non-python packages, but also supports >> packages that have nothing to do with python (Perl, command line tools, what >> have you...) >> >> So I have been focusing on conda lately. > > > The whole reason I started this discussion is to make sure wheel has a > standard way to do what is needed for those usecases. > > conda, rpm, deb, or eggs as used in enthought are all essentially the same: > an archive with a bunch of metadata. The real issue is standardising on the > exact formats. As you noticed, there is not much missing in the wheel *spec* > to get most of what's needed. We've used eggs for that purpose for almost 10 > years at Enthought, and we did not need that many extensions on top of the > egg format after all. > > >> >> Which brings me back to the question: should the python tools (i.e. wheel) >> be extended to support more use-cases, specifically non-python dependencies? >> Or do we just figure that that's a problem better solved by projects with a >> larger scope (i.e. rpm, deb, conda, canopy). > > > IMO, given that wheels do most of what's needed, it is worth supporting most > simple usecases (compiled libraries required by well known extensions). > Right now, such packages (pyzmq, numpy, cryptography, lxml) resort to quite > horrible custom hacks to support those cases. > > Hope that clarifies the intent, > > David
Then it sounds like I should read about the Enthought egg extensions. It's something else than just defining a separate pypi name for "just the libxml.so without the python bits"? _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig