On 2 October 2015 at 21:19, Nathaniel Smith <[email protected]> wrote: >> One of the problems with the current system, is that we have no mechanism by >> which to determine dependencies of a source distribution without downloading >> the file and executing some potentially untrusted code. This makes dependency >> resolution harder and much much slower than if we could read that information >> statically from a source distribution. This PEP doesn't offer anything in the >> way of solving this problem. > > What are the "dependencies of a source distribution"? Do you mean the > runtime dependencies of the wheels that will be built from a source > distribution? > > If you need that metadata to be statically in the sdist, then you > might as well give up now because it's simply impossible. > > As the very simplest example, every package that uses the numpy C API > gets a runtime dependency on "numpy >= [whatever version happened to > be installed on the *build* machine]". There are plenty of more > complex examples too (e.g. ones that involve build/configure-time > decisions about whether to rely on particular system libraries, or > build/configure-time decisions about whether particular packages > should even be built).
I'm really not at all clear what you're saying here. It's quite possible that those of us who don't understand the complexities of the scientific/numpy world are missing something important, but if so it would be useful if you could spell out the problems in detail. >From my point of view, it's not a source distribution or a binary distribution that depends on something (numpy or whatever) - it's the *project*. If project foo needs numpy to work, it depends on numpy. If it depends on features in numpy 1.9, it depends on numpy>=1.9. Optional dependencies are covered by extras, and environment specific dependencies are covered by environment markers.[1] That remains true for all wheels that are built from that project, for whatever platform using whatever tools. It should also be true for the source distribution, precisely *because* it's independent of the build process. I can understand that a binary wheel may need a certain set of libraries installed - but that's about the platform tags that are part of the wheel definition, not about dependencies. Platform tags are an ongoing discussion, and a good example of a partial solution that needs to be extended, certainly, but they aren't really relevant in any way that I can see to how the build chain works. You seem to be saying that wheels need a dependency on "the version of numpy they were built against". That sounds to me like a binary compatibility requirement that platform tags are intended to cover. It may well be a requirement that platform tags need significant enhancement (maybe even redesign) to cover, but it's not a dependency in the sense that pip and the packaging PEPs use the term. And if my understanding is correct, I'm against trying to fit that information into a dependency simply to work around the current limitations of the platform tag mechanism. I'm all in favour of new initiatives to make progress in areas that are currently stalled (we definitely need people willing to contribute) but we really don't have the resources to throw away the progress we've already made. Even if some of the packaging PEPs are still works in progress, what is there represents an investment we need to build on, not bypass. Paul [1] If extras and environment markers don't cover the needs of scientific modules, we need some input into their design from the scientific community. But again, let's not throw away the work that's already done. _______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
