Re: [Distutils] Idea: Positive/negative extras in general and as replacement for features
On Thu, Mar 3, 2016 at 8:30 PM, Ronny Pfannschmidt wrote: > > > Am 09.02.2016 um 22:24 schrieb Robert Collins: >> >> Sorry for not replying for so long. >> >> On 16 December 2015 at 07:54, Ronny Pfannschmidt >> wrote: >>> >>> Hello everyone, >>> >>> the talk about the sqlalchemy feature extra got me thinking >>> >>> what if i could specify extras that where installed by default, but >>> users could opt out >>> >>> a concrete example i have in mind is the default set of pytest plugins >>> i would like to be able to externalize legacy support details of pytest >>> without needing people to suddenly depend on pytest[recommended] instead >>> of pytest to have what they know function as is >>> >>> instead i would like to be able to do something like a dependency on >>> pytest[-nose,-unittest,-cache] to subtract the items >> >> >> So the challenge here will be defining good, useful, and predictable >> behaviour when the dependency graph is non-trivial. >> >> Using your example, what should pip do when told >> >> pip install pytest[nose, -unittest] proj2 >> >> and proj2 depends on pytest[unittest] >> >> ? > > those should be additive, > negative extras should be used purely to leave out dependencies for a single > requirement > if something else needs them, then they should of course be included > > even a dependency on pytest itself should include the unittest dependency > > negative requirements should be absolutely local and only effect the > dependency graph of a single direct dependency > > so if there is another dependency without that negation, then it should > consider the full graph in any case I only just saw this thread because Ronny's recent(-ish) reply, and I agree with him. In my mind the use case for something like this is pretty independent of "more complex" interactions like Robert's example. I think the idea here is that if I run `pip install pytest[-unittest]` I'm not saying the `pytest[-unittest]` must never ever be installed on the system. Rather, it's useful in a far more limited cases where I need more fine-grained control over what's installed--think for example application bundles or running on embedded systems. A case that would be useful to me would be to have an `astropy[tests]` extra that is installed by default. The astropy test suite is a bit weighty, especially due to the size of test files. I've proposed making the tests an extra before but that was shot down (I think rightfully) since it would be preferable to have it installed by default. However, for cases where the tests take up too much space a `pip install astropy[-tests]` would be nice--there's not likely to be an additional package that depends on `astropy[tests]` and even if there were I'd know in building my system whether or not I really need it. The alternative is post-install stripping down of packages, which can be unpredictable and error-prone in my experience. Erik ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Idea: Positive/negative extras in general and as replacement for features
Am 09.02.2016 um 22:24 schrieb Robert Collins: Sorry for not replying for so long. On 16 December 2015 at 07:54, Ronny Pfannschmidt wrote: Hello everyone, the talk about the sqlalchemy feature extra got me thinking what if i could specify extras that where installed by default, but users could opt out a concrete example i have in mind is the default set of pytest plugins i would like to be able to externalize legacy support details of pytest without needing people to suddenly depend on pytest[recommended] instead of pytest to have what they know function as is instead i would like to be able to do something like a dependency on pytest[-nose,-unittest,-cache] to subtract the items So the challenge here will be defining good, useful, and predictable behaviour when the dependency graph is non-trivial. Using your example, what should pip do when told pip install pytest[nose, -unittest] proj2 and proj2 depends on pytest[unittest] ? those should be additive, negative extras should be used purely to leave out dependencies for a single requirement if something else needs them, then they should of course be included even a dependency on pytest itself should include the unittest dependency negative requirements should be absolutely local and only effect the dependency graph of a single direct dependency so if there is another dependency without that negation, then it should consider the full graph in any case If it installs pytest[unittest], then the first pytest dependency was no honoured. If it does not install pytest[unittest], then the proj2 dependencies were not honoured. So it must error. -> Which means that using a [-THING] clause anywhere is going to be super fragile, as indeed 'never install with X' things are in distributions - its much better to find ways to express things purely additively IMO. There are many more complex interactions possible with the + / - DSL you've sketched - and I don't think they are reducible - that is, its not the DSL per se, but the basic feature of allowing cuts in the graph traversal that lead to that complexity. If we can come up with good, consistent, predictable answers, even considering three-party interactions, then I've no objection per se: but I think that will be very hard to do. I certainly don't have time at the moment to help - sorry :(. -Rob ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Idea: Positive/negative extras in general and as replacement for features
On Tue, Dec 15, 2015 at 10:54 AM, Ronny Pfannschmidt wrote: > Hello everyone, > > the talk about the sqlalchemy feature extra got me thinking > > what if i could specify extras that where installed by default, but > users could opt out If we reified extras (https://mail.python.org/pipermail/distutils-sig/2015-October/027364.html), and added support for weak "Recommends/Suggests" dependencies (https://mail.python.org/pipermail/distutils-sig/2016-February/028258.html), then those two features together would give you what you want (as well as solving a variety of other problems). -n -- Nathaniel J. Smith -- https://vorpus.org ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Idea: Positive/negative extras in general and as replacement for features
Sorry for not replying for so long. On 16 December 2015 at 07:54, Ronny Pfannschmidt wrote: > Hello everyone, > > the talk about the sqlalchemy feature extra got me thinking > > what if i could specify extras that where installed by default, but > users could opt out > > a concrete example i have in mind is the default set of pytest plugins > i would like to be able to externalize legacy support details of pytest > without needing people to suddenly depend on pytest[recommended] instead > of pytest to have what they know function as is > > instead i would like to be able to do something like a dependency on > pytest[-nose,-unittest,-cache] to subtract the items So the challenge here will be defining good, useful, and predictable behaviour when the dependency graph is non-trivial. Using your example, what should pip do when told pip install pytest[nose, -unittest] proj2 and proj2 depends on pytest[unittest] ? If it installs pytest[unittest], then the first pytest dependency was no honoured. If it does not install pytest[unittest], then the proj2 dependencies were not honoured. So it must error. -> Which means that using a [-THING] clause anywhere is going to be super fragile, as indeed 'never install with X' things are in distributions - its much better to find ways to express things purely additively IMO. There are many more complex interactions possible with the + / - DSL you've sketched - and I don't think they are reducible - that is, its not the DSL per se, but the basic feature of allowing cuts in the graph traversal that lead to that complexity. If we can come up with good, consistent, predictable answers, even considering three-party interactions, then I've no objection per se: but I think that will be very hard to do. I certainly don't have time at the moment to help - sorry :(. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Idea: Positive/negative extras in general and as replacement for features
Hello everyone,
the talk about the sqlalchemy feature extra got me thinking
what if i could specify extras that where installed by default, but
users could opt out
a concrete example i have in mind is the default set of pytest plugins
i would like to be able to externalize legacy support details of pytest
without needing people to suddenly depend on pytest[recommended] instead
of pytest to have what they know function as is
instead i would like to be able to do something like a dependency on
pytest[-nose,-unittest,-cache] to subtract the items
the extras declaration in setup.py would just include a * sign at the
beginning
an elaborate concrete example could be
extras = {
+cache: pytest-cache
+lastfailed: pyytest[+cache], pytest-lastfailed
+looponfail: pytest[+lastfailed], pytest-looponfail
+unittest: pytest-unittest
+nose: pytest[+unittest], pytest-nose
}
also a dependency declaration using the + sign in the extras should not
imply the default extras of the package, while usage of the - should
so depending on pytest[+unittest] would only imply the unittest support
but depending on pytest[-nose] would include all positive extras except
for nose
please note in particular the dependencies on other postive extras,
those are put in,
so a negative for unittest can imply that nose can't be sensibly used as
well
if +nose did instead depend directly on pytest-unittest, then excluding
it would be a unreasonably tricky resolving algorithm with potential for
lots of mistakes
instead spelling the direct dependency on positives/negatives can
resolve inside of a package and still leave room for more outside of it
this is in particular relevant if a package with extras is depended on
twice in different packages,
because in that case each of the dependence's requirements should add up
to the combined set
there is also room for fleshing out algorithms for interacting the
positive/negative dependence sets,
but i'll leave that for later
as an addition to that later on there could be support for partial wheel
wheels, so features could be materealiyed as wheel package with a
special name
and build tools could fall back to making them from a sdist
as a example, there would be a sqlalchemy package as source wheel,
a sqlalchemy*cext package as windows wheels, and pip would have to find
a source distribution to compile the wheel package
-- Ronny
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
