On 1 November 2017 at 00:53, Guido van Rossum <gu...@python.org> wrote:
> On Tue, Oct 31, 2017 at 4:42 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > >> On 31 October 2017 at 02:29, Guido van Rossum <gu...@python.org> wrote: >> >>> What's your proposed process to arrive at the list of recommended >>> packages? >>> >> >> I'm thinking it makes the most sense to treat inclusion in the >> recommended packages list as a possible outcome of proposals for standard >> library inclusion, rather than being something we'd provide a way to >> propose specifically. >> > > I don't think that gets you off the hook for a process proposal. We need > some criteria to explain why a module should be on the recommended list -- > not just a ruling as to why it shouldn't be in the stdlib. > The developer guide already has couple of sections on this aspect: * https://devguide.python.org/stdlibchanges/#acceptable-types-of-modules * https://devguide.python.org/stdlibchanges/#requirements I don't think either of those sections is actually quite right (since we've approved new modules that wouldn't meet them), but they're not terrible as a starting point in general, and they're accurate for the recommended packages use case. > We'd only use it in cases where a proposal would otherwise meet the >> criteria for stdlib inclusion, but the logistics of actually doing so don't >> work for some reason. >> > > But that would exclude most of the modules you mention below, since one of > the criteria is that their development speed be matched with Python's > release cycle. I think there must be some form of "popularity" combined > with "best of breed". In particular I'd like to have a rule that explains > why flask and Django would never make the list. (I don't know what that > rule is, or I would tell you -- my gut tells me it's something to do with > having their own community *and* competing for the same spot.) > The developer guide words this as "The module needs to be considered best-of-breed.". In some categories (like WSGI frameworks), there are inherent trade-offs that mean there will *never* be a single best-of-breed solution, since projects like Django, Flask, and Pyramid occupy deliberately different points in the space of available design decisions. Running the initial 5 proposals through that filter: >> >> * six: a cross-version compatibility layer clearly needs to be outside >> the standard library >> > > Hm... Does six still change regularly? If not I think it *would* be a > candidate for actual stdlib inclusion. Just like we added u"..." literals > to Python 3.4. > It still changes as folks porting new projects discover additional discrepancies between the 2.x and 3.x standard library layouts and behaviour (e.g. we found recently that the 3.x subprocess module's emulation of the old commands module APIs actually bit shifts the status codes relative to the 2.7 versions). The rate of change has definitely slowed down a lot since the early days, but it isn't zero. In addition, the only folks that need it are those that already care about older versions of Python - if you can start with whatever the latest version of Python is, and don't have any reason to support users still running older version, you can happily pretend those older versions don't exist, and hence don't need a compatibility library like six. As a separate library, it can just gracefully fade away as folks stop depending on it as they drop Python 2.7 support. By contrast, if we were to bring it into the 3.x standard library, then we'd eventually have to figure out when we could deprecate and remove it again. > * setuptools: we want to update this in line with the PyPA interop specs, >> not the Python language version >> > > But does that exclude stdlib inclusion? Why would those specs change, and > why couldn't they wait for a new Python release? > The specs mainly change when we want to offer publishers new capabilities while still maintaining compatibility with older installation clients (and vice-versa: we want folks still running Python 2.7 to be able to publish wheel files and use recently added metadata fields like Description-Content-Type). The reason we can't wait for new Python releases is because when we add such things, we need them to work on *all* supported Python releases (including 2.7 and security-release-only 3.x versions). There are also other drivers for setuptools updates, which include: - operating system build toolchain changes (e.g. finding new versions of Visual Studio or XCode) - changes to PyPI's operations (e.g. the legacy upload API getting turned off due to persistent service stability problems, switching to HTTPS only access) With setuptools as a separate project, a whole lot of package publication problems can be solved via "pip install --upgrade setuptools wheel" in a virtual environment, which is a luxury we don't have with plain distutils. > * cffi: updates may be needed for PyPA interop specs, Python > implementation updates or C language definition updates > > Hm, again, I don't recall that this was debated -- I think it's a failure > that it's not in the stdlib. > A couple of years ago I would have agreed with you, but I've spent enough time on packaging problems now to realise that cffi actually qualifies as a build tool due to the way it generates extension module wrappers when used in "out-of-line" mode. Being outside the standard library means that cffi still has significant flexibility to evolve how it separates its buildtime functionality from its runtime functionality, and may eventually be adjusted so that only "_cffi_backend" needs to be installed at runtime for the out-of-line compilation mode, without the full C header parsing and inline extension module compilation capabilities of CFFI itself (see https://cffi.readthedocs.io/en/latest/cdef.html#preparing-and-distributing-modules for the details). Being separate also means that cffi can be updated to generate more efficient code even for existing Python versions. > * requests: updates are more likely to be driven by changes in network >> protocols and client platform APIs than Python language changes >> > > Here I agree. There's no alternative (except aiohttp, but that's > asyncio-based) and it can't be in the stdlib because it's actively being > developed. > > >> * regex: we don't want two regex engines in the stdlib, transparently >> replacing _sre would be difficult, and _sre is still good enough for most >> purposes >> > > I think this needn't be recommended at all. For 99.9% of regular > expression uses, re is just fine. Let's just work on a strategy for > introducing regex into the stdlib. > Given your informational PEP suggestion below, I'd probably still include it, but in a separate section from the others (e.g. the others might be listed as "Misaligned Feature Release Cycles", which is an inherent logistical problem that no amount of coding can fix, while regex would instead be categorised as "Technical Challenges"). > Of the 5, I'd suggest that regex is the only one that could potentially >> still make its way into the standard library some day - it would just >> require someone with both the time and inclination to create a CPython >> variant that used _regex instead of _sre as the default regex engine, and >> then gathered evidence to show that it was "compatible enough" with _sre to >> serve as the default engine for CPython. >> >> For the first four, there are compelling arguments that their drivers for >> new feature additions are such that their release cycles shouldn't ever be >> tied to the rate at which we update the Python language definition. >> > > As you can tell from my arguing, the reasons need to be written up in more > detail. > > >> And is it really just going to be a list of names, or is there going to >>> be some documentation (about the vetting, not about the contents of the >>> packages) for each name? >>> >> >> I'm thinking a new subsection in https://docs.python.org/devgui >> de/stdlibchanges.html for "Recommended Third Party Packages" would make >> sense, covering what I wrote above. >> > > That's too well hidden for my taste. > > >> It also occurred to me that since the recommendations are independent of >> the Python version, they don't really belong in the version specific >> documentation. >> > > But that doesn't mean they can't (also) be listed there. (And each > probably has its version dependencies.) > > >> While the Developer's Guide isn't really the right place for the list >> either (except as an easier way to answer "Why isn't <X> in the standard >> library?" questions), it could be a good interim option until I get around >> to actually writing a first draft of https://github.com/python/redi >> stributor-guide/ (which I was talking to Barry about at the dev sprint, >> but didn't end up actually creating any content for since I went down a >> signal handling rabbit hole instead). >> > > Hm, let's not put more arbitrary check boxes in the way of progress. Maybe > it can be an informational PEP that's occasionally updated? > If I'm correctly reading that as "Could the list of Recommended Third Party Packages be an informational PEP?", then I agree that's probably a good way to tackle it, since it will cover both the developer-centric "Why isn't this in the standard library yet?" aspect *and* the "Redistributors should probably provide this" aspect. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/