On 18 Jan 2014 04:45, "Hannes Schmidt" <han...@ucsc.edu> wrote: > > Thank you for your suggestions, Donald. They are both feasible but they feel like workarounds to me. > > Manually listing transitive dependencies seems like a step backwards to me. Isn't the whole point of setuptools/distutils that each component manages its own list of dependencies? How will this scale to dozens of transitive dependencies and busy development teams where dependencies are revved frequently. I really liked what Maven did for Java and therefore liked what pip is trying to do for Python.
Hannes, this is the role we consider to be best served by running a private index server and using the full dependency resolution mechanism. You may want to review PEP 426 to see the intended mode of operation. However, the question at this point is whether there is a legitimate model of use where dependency links (or an equivalent) are used to create a more concrete dependency tree in a closed environment where the software developer/publisher and integrator/user are the *same* organisation. PEP 426 focuses on the case where those are different organisations (since that case poses the greatest security risk, allowing publishers to compromise the installations of integrators and users, beyond just running the software they provide). While it does include the "meta requirements" field to handle cases of (effective) dependency bundling rather than normal versioning, that model is deliberately more restrictive than dependency links. It seems to me that overall, it's much, much safer to simply avoid supporting this degree of transitive trust directly in the core installation toolchain. A more appropriate solution to handling the trusted internal distribution use case may be a dependency scanner that processes the dependency links metadata to generate a suitable requirements.txt file. >From pip's perspective, trusting that generated file is the same as trusting any other explicitly specified set of packages, while it also provides an opportunity for the resolved dependency links to be audited before trusting them. Unfortunately, we can't actually do that easily in a secure fashion, because obtaining the metadata currently requires executing an untrusted setup.py. So trusting dependency links "just" to generate a requirements.txt file is almost as bad as trusting them for installing software. The main advantage is that you can always generate the dependency list as an unprivileged user or in a sandbox, even if operating in an untrusted environment. Installation, by contrast, is often going to involve elevated privileges. Regardless, perhaps there's a case to be made that such a requirements.txt generator, with opt-in dependency links support, should remain available, with the feature being removed entirely only for actual package installation. That approach is still far more secure by default than the status quo, strongly encourages the use of a private index to better align with the upstream distribution model, but restores support for users like Hannes that are currently relying on dependency links as a dependency resolution solution that doesn't require additional server components in a trusted environment. > I haven't used --find-links yet so I may be wrong but tarring up packages and serving them by a web server is additional work that I'd rather avoid. It just creates yet another copy of the package and requires maintaining an additional server component. > > I loved the fact that I could point pip directly at my source repos for both my top-level projects as well as their dependencies. That seemed like a perfect fit for an interpreted language like Python where packages are distributed in source form. In a trusted environment, it can be, but on the internet, it is intolerably insecure. However, making it easy to generate a transitive requirements.txt for a trusted environment may be a usage model we can support without leaving upstream distribution and usage open to compromise. Regards, Nick.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
