[Distutils] Idea: Using Trove classifiers for platform compatibility warnings
PyPI already has a reasonably extensive component tagging system in https://pypi.python.org/pypi?%3Aaction=list_classifiers but we don't really *use* it all that much for programmatic purposes. That means the incentives for setting tags correctly are weak, since there isn't much pay-off in the form of tooling-intermediated communication of constraints and testing limitations to end users. What I'm starting to wonder is whether or not it may make sense to start recommending that installation tools emit warnings in the following cases: 1. At least one "Operating System" tag is set, but the tags don't include any that cover the *current* operating system 2. At least one "Programming Language :: Python" tag is set, but the tags don't include any that cover the *current* Python version The "at least one relevant tag is set" pre-requisite would be to avoid emitting false positives for projects that don't provide any platform compatibility guidance at all. Instead, the goal would be to eliminate the cases where *incorrect* guidance is currently provided - no guidance at all would be fine, correct guidance would be fine, but incorrect guidance would result in install time warnings on nominally unsupported platforms. Checking for applicable tags at run time would then be a matter of defining two things: - for a given platform, figure out the list of applicable tags that indicate compatibility - for a given platform, figure out the list of applicable tags that indicate *in*compatibility I'm bringing this idea up now as it came up twice for me this week: - in my current work project, where even though the project itself is pure Python, we're manipulating Linux containers and local VM hypervisors, so I put a "Operating System :: POSIX :: Linux" tag on it - in a discussion of using absolute paths in "data_files" where it can be an entirely reasonable thing to do, as long as you're OK with making the affected library or application platform specific: https://github.com/pypa/python-packaging-user-guide/pull/212#issuecomment-292686566 It's also quite applicable to some of the enhancements Daniel would like to make to the wheel format to support more of the GNU autotools paths (https://www.gnu.org/prep/standards/html_node/Directory-Variables.html), which become a lot more palatable if there's a defined machine readable way of saying "Hey, this project assumes POSIX directory layout conventions" (which would be to set "Operating System :: POSIX" or one of it's more specific variants, given the definitions above). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Idea: Using Trove classifiers for platform compatibility warnings
On 8 April 2017 at 03:17, Nick Coghlan wrote: > PyPI already has a reasonably extensive component tagging system in > https://pypi.python.org/pypi?%3Aaction=list_classifiers but we don't > really *use* it all that much for programmatic purposes. > > That means the incentives for setting tags correctly are weak, since > there isn't much pay-off in the form of tooling-intermediated > communication of constraints and testing limitations to end users. > > What I'm starting to wonder is whether or not it may make sense to > start recommending that installation tools emit warnings in the > following cases: > > 1. At least one "Operating System" tag is set, but the tags don't > include any that cover the *current* operating system > 2. At least one "Programming Language :: Python" tag is set, but the > tags don't include any that cover the *current* Python version > > The "at least one relevant tag is set" pre-requisite would be to avoid > emitting false positives for projects that don't provide any platform > compatibility guidance at all. I agree that there's little incentive at the moment to get classifiers right. So my concern with this proposal would be that it issues the warnings to end users, who don't have any direct means of resolving the issue (they can of course raise bugs on the projects they find with incorrect classifiers). Furthermore, there's a potential risk that projects might see classifiers as implying a level of support they are not happy with, and so are reluctant to add classifiers "just" to suppress the warning. But without data, the above is just FUD, so I'd suggest we do some analysis. I did some spot checks, and it seems that projects might typically not set the OS classifier, which alleviates my biggest concern (projects stating "POSIX" because that's what they develop on, when they actually work fine on Windows) - but propoer data would be better. Two things I'd like to see: 1. A breakdown of how many projects actually use the various OS and Language classifiers. 2. Where projects ship wheels, do the wheels they ship match the classifiers they declare? That should give a good idea of the immediate impact of this proposal. (There's not much we can say about source-only distributions, but that's OK). The data needed to answer those questions should be available - the only way I have of getting it is via the JSON interface to PyPI, so I can write a script to collect the information, but it might be some time before I can collate it. Is this something the BigQuery data we have (which I haven't even looked at myself) could answer? Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Idea: Using Trove classifiers for platform compatibility warnings
On 8 April 2017 at 19:29, Paul Moore wrote: > On 8 April 2017 at 03:17, Nick Coghlan wrote: >> The "at least one relevant tag is set" pre-requisite would be to avoid >> emitting false positives for projects that don't provide any platform >> compatibility guidance at all. > > I agree that there's little incentive at the moment to get classifiers > right. I'll also explicitly note that I think this idea counts as a "nice to have" - in cases where there are real compatibility problems, those are going to show up at runtime anyway, so what this idea really provides is a debugging hint that says "Hey, you know that weird behaviour you're seeing in ? How sure are you that all of your dependencies actually support that configuration?" That said, if folks agree that this idea at least seems plausible, one outcome is that I would abandon the draft "Supported Environments" section for the "python.constraints" extension in PEP 459: https://www.python.org/dev/peps/pep-0459/#supported-environments While programmatic expressions like that are handy for publishers, they don't convey the difference between "We expect future Python versions to work" and "We have tested this particular Python version, and it does appear to work", and they're also fairly hostile to automated data analysis, since you need to evaluate expressions in a mini-language rather than just filtering on an appropriately defined set of metadata tags. When it comes to the "Programming Language :: Python" classifiers though, we already give folks quite a bit of flexibility there: - no tag or the generic unversioned tag to say "No guidance provided" - the "PL :: Python :: X" tags to say "definitely supports Python X" without saying which X.Y versions - the "PL :: Python :: X.Y" tags to say "definitely supports Python X.Y" And that flexibility provides an opportunity to let publishers make a trade-off between precision of information provided (down to just major version, or specifying both major and minor version) and the level of maintenance effort (with the more precise approach meaning always having to make a new release to update the compatibility metadata for new Python feature releases, even when the existing code works without any changes, but also meaning you get a way to affirmatively say "Yes, we tested this with the new version, and it still works"). We also have the "PL :: Python :: X :: Only" tags, but I think that may be a misguided approach and we'd be better off with a general notion of tag negation: "Not :: PL :: Python :: X" (so you'd add a "Not :: Programming Language :: Python :: 2" tag instead of adding a "Programming Language :: Python :: 3 :: Only" tag) > So my concern with this proposal would be that it issues the > warnings to end users, who don't have any direct means of resolving > the issue (they can of course raise bugs on the projects they find > with incorrect classifiers). We need to be clear about the kinds of end users we're considering here, though: folks using pip (or similar) tools to do their own install-time software integration, *not* folks consuming pre-built and pre-integrated components through conda/apt/dnf/msi/etc. In the latter cases, the redistributor is taking on the task of making sure their particular combinations work well together, but when we use pip (et al) directly, that task falls directly on us as useful, and it's useful when debugging to know whether what we're doing is a combination that upstream has already thought about (and is hopefully covering in their CI setup if they have one), or whether we may be doing something unusual that most other people haven't tried yet. While this is also useful info for redistributors to know, I was thinking in PyPI publisher & pip user terms when the idea occurred to me. The concept is based at least in part on my experience as a World of Warcraft player, where there are two main pieces to their compatibility handling model for UI Add-ons: 1. Add-on authors tag the add-on itself with the most recent version of the client API that they've tested it against 2. To avoid having your UI break completely every time the client API changes, he main game client has a simple "Load Out of Date Addons" check box to let you opt-in to continue to use add-ons that may not have been updated for the latest changes to the game's runtime API (while also clearly saying "Don't complain to Blizzard about any UI bugs you encounter in this unsupported configuration") Assuming we do pursue this idea (which is still a big assumption at this point, due to the "potentially nice to have for debugging in some situations" motivation being a fairly weak one for volunteer efforts), I think a sensible way to go would be to have the classifier checking be opt-in initially (e.g. through a "--check-classifiers" option), and only consider making it the default behaviour if having it available as a debugging option seems insufficient. > Furthermore, there's a potential risk > that proj
Re: [Distutils] Idea: Using Trove classifiers for platform compatibility warnings
On Sat, Apr 8, 2017, at 03:17 AM, Nick Coghlan wrote: > 2. At least one "Programming Language :: Python" tag is set, but the > tags don't include any that cover the *current* Python version We've just enabled setuptools, PyPI and pip to use the python-requires metadata field, so that (for instance), IPython 6.0 was uploaded with python-requires: >=3.3 . Pip 9 respects that, so users running 'pip install ipython' on Python 2.7 will get the latest IPython 5.x release. So we have a relatively elegant way to tell pip about supported Python versions - although it's designed for the use case "this won't work on X.Y, so don't even try to install it", rather than "this isn't explicitly marked as supporting X.Y, but it might work anyway." Personally I try to avoid the 'Python :: X.Y' classifiers because I don't want to be updating a dozen packages every time a new Python comes out. I've used the 'Python :: 2'/'Python :: 3' classifiers before - but even those could cause a lot of inaccurate metadata if Python 4 happens and is mostly compatible with Python 3. Thomas ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
