On 3 November 2016 at 19:10, Nathaniel Smith <n...@pobox.com> wrote: > On Nov 3, 2016 1:40 AM, "Nick Coghlan" <ncogh...@gmail.com> wrote: >> The approach Tennessee and Robert Collins came up with (which still >> sounds sensible to me) is that instead of dependencies on particular >> external components, what we want to be able express is instead a >> range of *actions* that the software is going to take: >> >> - "I am going to dynamically load a library named <X>" >> - "I am going to execute a subprocess for a command named <Y>" > > And then it segfaults because it turns out that your library named <X> is > not abi compatible with my library named <X>. Or it would have been if you > had the right version, but distros don't agree on how to express version > numbers either. (Just think about epochs.) Or we're on Windows, so it's > interesting to know that we need a library named <X>, but what are we > supposed to do with that information exactly?
I don't think there's much chance of any of this ever working on Windows - conda will rule there, and rightly so. Mac OS X seems likely to go the same way, although there's an outside chance brew may pick up some of the otherwise Linux-only capabilities if they prove successful. Linux distros have larger package management communities than conda though, and excellent integration testing infrastructure on the commercial side of things, so we "just" need to get some of their software integration pipelines arranged in better configurations, and responding to the right software publication events :) > Again, I don't want to just be throwing around stop energy -- if people want > to tackle these problems then I wish them luck. But I don't think we should > be hand waving this as a basically solved problem that just needs a bit of > coding, because that also can create stop energy that blocks an honest > evaluation of alternative approaches. There's a reason I'm not volunteering to work on this in my own time, and am instead letting the Fedora Modularity and Python maintenance folks decide when it's a major blocker to further process improvement efforts on their side of things, while I work on other non-Python-specific aspects of Red Hat's software supply chain management. What I'm mainly reacting to here is Chris's apparent position that he sees the entire effort as pointless. It's not pointless, just hard, since it involves working towards process improvements that span *multiple* open source communities, rather than being able to work with just the Python community or just the conda community. Even if automation can only cover 80-90% of legacy cases and 95% of future cases (numbers plucked out of thin air), that's still a huge improvement since folks generally aren't adding *new* huge-and-gnarly C/C++/FORTRAN dependencies to the mix of things pip users need to worry about, while Go and Rust have been paying much closer attention to supporting distributed automated builds from the start of their existence. >> When it comes to things that stand in the way of fully automating the >> PyPI -> RPM pipeline, there are still a lot of *much* lower hanging >> fruit out there than this. However, the immediate incentives of folks >> working on package management line up in such a way that I'm >> reasonably confident that the lack of these capabilities is a matter >> of economic incentives rather than insurmountable technical barriers - >> it's a hard, tedious problem to automate, and manual repackaging into >> a platform specific format has long been a pretty easy thing for >> commercial open source redistributors to sell. > > The idea does seem much more plausible as a form of metadata hints attached > to sdists that are used as inputs to a semi-automated-but-human-supervised > distro package building system. What I'm skeptical about is specifically the > idea that someday I'll be able to build a wheel, distribute it to end users, > and then pip will do some negotiation with > dnf/apt/pacman/chocolatey/whatever and make my wheel work everywhere -- and > that this will be an viable alternative to conda. Not in the general case it won't, no, but it should be entirely feasible for distro-specific Linux wheels and manylinux1 wheels to be as user-friendly as conda in cases where folks on the distro side are willing to put in the effort to make the integration work properly. It may also be feasible to define an ABI for "linuxconda" that's broader than the manylinux1 ABI, so folks can publish conda wheels direct to PyPI, and then pip could define a way for distros to indicate their ABI is "conda compatible" somehow. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
