... Just had this discussion in regards to easy_install, Ubuntu 14.04 LTS, and the ReadTheDocs Docker images (as well as: ~why should I have to wget/curl get-pip.py) https://github.com/rtfd/readthedocs-docker-images/pull/3 On Nov 23, 2015 2:47 PM, "Wes Turner" <wes.tur...@gmail.com> wrote:
> 1. Does this affect easy_install? > 2. If/because this affects easy_install, > should the guidance / suggested package installation tool be [pip]; > because pip install_requires backports.ssl_match_hostname > > https://pypi.python.org/pypi/backports.ssl_match_hostname > On Nov 10, 2015 6:48 PM, "Nick Coghlan" <ncogh...@gmail.com> wrote: > >> Hi folks, >> >> I have a confession to make - I dropped the ball on the HTTPS >> verification backport proposals in PEP 493, and let the upstream and >> downstream approval processes get out of sequence. >> >> As a result, the RHEL 7.2 beta released back in September incorporates >> the HTTPS verification feature backport based on the current PEP 493 >> draft, even though that hasn't formally been pronounced as an Active >> recommendation by python-dev yet. >> >> Accordingly, I'm belatedly submitting it for pronouncement now: >> https://www.python.org/dev/peps/pep-0493/ >> >> There's currently no BDFL-Delegate assigned, so if Guido doesn't want >> to handle it, we'll need to address that question first. >> >> Our last discussion back in July seemed to show that folks either >> didn't care about the question (because they're using unmodified >> upstream versions so the PEP didn't affect them), or else thought the >> approach described in the PEP was reasonable, so I'm hoping the >> consequences of my mistake won't be too severe. >> >> Regards, >> Nick. >> >> P.S. I'm aware that this looks like presenting a fait accompli at a >> point where it's too late to realistically say "No", but the truth is >> that preparation for the Python in Education miniconf at PyCon >> Australia ramped up immediately after the July discussion, and then I >> personally got confused as to the scope of what was being included in >> 7.2 (I mistakenly thought it was just PEP 466 for now, with 476+493 >> being deferred to a later release, but it's actually the whole package >> of 466+476+493). That's my fault for trying to keep track of too many >> things at once (and thus failing at some of them), not anyone else's. >> >> ================================ >> >> PEP: 493 >> Title: HTTPS verification recommendations for Python 2.7 redistributors >> Version: $Revision$ >> Last-Modified: $Date$ >> Author: Nick Coghlan <ncogh...@gmail.com>, >> Robert Kuska <rku...@redhat.com>, >> Marc-André Lemburg <m...@lemburg.com> >> Status: Draft >> Type: Informational >> Content-Type: text/x-rst >> Created: 10-May-2015 >> Post-History: 06-Jul-2015 >> >> >> Abstract >> ======== >> >> PEP 476 updated Python's default handling of HTTPS certificates to be >> appropriate for communication over the public internet. The Python 2.7 >> long >> term maintenance series was judged to be in scope for this change, with >> the >> new behaviour introduced in the Python 2.7.9 maintenance release. >> >> This PEP provides recommendations to downstream redistributors wishing to >> provide a smoother migration experience when helping their users to manage >> this change in Python's default behaviour. >> >> >> Rationale >> ========= >> >> PEP 476 changed Python's default behaviour to better match the needs and >> expectations of developers operating over the public internet, a category >> which appears to include most new Python developers. It is the position of >> the authors of this PEP that this was a correct decision. >> >> However, it is also the case that this change *does* cause problems for >> infrastructure administrators operating private intranets that rely on >> self-signed certificates, or otherwise encounter problems with the new >> default >> certificate verification settings. >> >> The long term answer for such environments is to update their internal >> certificate management to at least match the standards set by the public >> internet, but in the meantime, it is desirable to offer these >> administrators >> a way to continue receiving maintenance updates to the Python 2.7 series, >> without having to gate that on upgrades to their certificate management >> infrastructure. >> >> PEP 476 did attempt to address this question, by covering how to revert >> the >> new settings process wide by monkeypatching the ``ssl`` module to restore >> the >> old behaviour. Unfortunately, the ``sitecustomize.py`` based technique >> proposed >> to allow system administrators to disable the feature by default in their >> Standard Operating Environment definition has been determined to be >> insufficient in at least some cases. The specific case of interest to the >> authors of this PEP is the one where a Linux distributor aims to provide >> their users with a >> `smoother migration path >> <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__ >> than the standard one provided by consuming upstream CPython 2.7 releases >> directly, but other potential challenges have also been pointed out with >> updating embedded Python runtimes and other user level installations of >> Python. >> >> Rather than allowing a plethora of mutually incompatibile migration >> techniques >> to bloom, this PEP proposes two alternative approaches that redistributors >> may take when addressing these problems. Redistributors may choose to >> implement >> one, both, or neither of these approaches based on their assessment of the >> needs of their particular userbase. >> >> These designs are being proposed as a recommendation for redistributors, >> rather >> than as new upstream features, as they are needed purely to support legacy >> environments migrating from older versions of Python 2.7. Neither approach >> is being proposed as an upstream Python 2.7 feature, nor as a feature in >> any >> version of Python 3 (whether published directly by the Python Software >> Foundation or by a redistributor). >> >> >> Requirements for capability detection >> ===================================== >> >> As these recommendations are intended to cover backports to earlier Python >> versions, the Python version number cannot be used as a reliable means for >> detecting them. Instead, the recommendations are defined to allow the >> presence >> or absence of the feature to be determined using the following technique:: >> >> python -c "import ssl; ssl._relevant_attribute" >> >> This will fail with `AttributeError` (and hence a non-zero return code) >> if the >> relevant capability is not available. >> >> The marker attributes are prefixed with an underscore to indicate the >> implementation dependent nature of these capabilities - not all Python >> distributions will offer them, only those that are providing a multi-stage >> migration process from the legacy HTTPS handling to the new default >> behaviour. >> >> >> Recommendation for an environment variable based security downgrade >> =================================================================== >> >> Some redistributors may wish to provide a per-application option to >> disable >> certificate verification in selected applications that run on or embed >> CPython >> without needing to modify the application itself. >> >> In these cases, a configuration mechanism is needed that provides: >> >> * an opt-out model that allows certificate verification to be selectively >> turned off for particular applications after upgrading to a version of >> Python that verifies certificates by default >> * the ability for all users to configure this setting on a per-application >> basis, rather than on a per-system, or per-Python-installation basis >> >> This approach may be used for any redistributor provided version of >> Python 2.7, >> including those that advertise themselves as providing Python 2.7.9 or >> later. >> >> >> Required marker attribute >> ------------------------- >> >> The required marker attribute on the ``ssl`` module when implementing this >> recommendation is:: >> >> _https_verify_envvar = 'PYTHONHTTPSVERIFY' >> >> This not only makes it straightforward to detect the presence (or >> absence) of >> the capability, it also makes it possible to programmatically determine >> the >> relevant environment variable name. >> >> >> Recommended modifications to the Python standard library >> -------------------------------------------------------- >> >> The recommended approach to providing a per-application configuration >> setting >> for HTTPS certificate verification that doesn't require modifications to >> the >> application itself is to: >> >> * modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment >> variable when the module is first imported into a Python process >> * set the ``ssl._create_default_https_context`` function to be an alias >> for >> ``ssl._create_unverified_context`` if this environment variable is >> present >> and set to ``'0'`` >> * otherwise, set the ``ssl._create_default_https_context`` function to be >> an >> alias for ``ssl.create_default_context`` as usual >> >> >> Example implementation >> ---------------------- >> >> :: >> >> _https_verify_envvar = 'PYTHONHTTPSVERIFY' >> >> def _get_https_context_factory(): >> config_setting = os.environ.get(_https_verify_envvar) >> if config_setting == '0': >> return _create_unverified_context >> return create_default_context >> >> _create_default_https_context = _get_https_context_factory() >> >> >> Security Considerations >> ----------------------- >> >> Relative to an unmodified version of CPython 2.7.9 or later, this approach >> does introduce a new downgrade attack against the default security >> settings >> that potentially allows a sufficiently determined attacker to revert >> Python >> to the vulnerable configuration used in CPython 2.7.8 and earlier >> releases. >> However, such an attack requires the ability to modify the execution >> environment of a Python process prior to the import of the ``ssl`` module, >> and any attacker with such access would already be able to modify the >> behaviour of the underlying OpenSSL implementation. >> >> >> Recommendation for backporting to earlier Python versions >> ========================================================= >> >> Some redistributors, most notably Linux distributions, may choose to >> backport >> the PEP 476 HTTPS verification changes to modified Python versions based >> on >> earlier Python 2 maintenance releases. In these cases, a configuration >> mechanism is needed that provides: >> >> * an opt-in model that allows the decision to enable HTTPS certificate >> verification to be made independently of the decision to upgrade to the >> Python version where the feature was first backported >> * the ability for system administrators to set the default behaviour of >> Python >> applications and scripts run directly in the system Python installation >> * the ability for the redistributor to consider changing the default >> behaviour >> of *new* installations at some point in the future without impacting >> existing >> installations that have been explicitly configured to skip verifying >> HTTPS >> certificates by default >> >> This approach should not be used for any Python installation that >> advertises >> itself as providing Python 2.7.9 or later, as most Python users will have >> the >> reasonable expectation that all such environments will validate HTTPS >> certificates by default. >> >> >> Required marker attribute >> ------------------------- >> >> The required marker attribute on the ``ssl`` module when implementing this >> recommendation is:: >> >> _cert_verification_config = '<path to configuration file>' >> >> This not only makes it straightforward to detect the presence (or >> absence) of >> the capability, it also makes it possible to programmatically determine >> the >> relevant configuration file name. >> >> >> Recommended modifications to the Python standard library >> -------------------------------------------------------- >> >> The recommended approach to backporting the PEP 476 modifications to an >> earlier >> point release is to implement the following changes relative to the >> default >> PEP 476 behaviour implemented in Python 2.7.9+: >> >> * modify the ``ssl`` module to read a system wide configuration file when >> the >> module is first imported into a Python process >> * define a platform default behaviour (either verifying or not verifying >> HTTPS >> certificates) to be used if this configuration file is not present >> * support selection between the following three modes of operation: >> >> * ensure HTTPS certificate verification is enabled >> * ensure HTTPS certificate verification is disabled >> * delegate the decision to the redistributor providing this Python >> version >> >> * set the ``ssl._create_default_https_context`` function to be an alias >> for >> either ``ssl.create_default_context`` or >> ``ssl._create_unverified_context`` >> based on the given configuration setting. >> >> >> Recommended file location >> ------------------------- >> >> This approach is currently only defined for \*nix system Python >> installations. >> >> The recommended configuration file name is >> ``/etc/python/cert-verification.cfg``. >> >> The ``.cfg`` filename extension is recommended for consistency with the >> ``pyvenv.cfg`` used by the ``venv`` module in Python 3's standard library. >> >> >> Recommended file format >> ----------------------- >> >> The configuration file should use a ConfigParser ini-style format with a >> single section named ``[https]`` containing one required setting >> ``verify``. >> >> Permitted values for ``verify`` are: >> >> * ``enable``: ensure HTTPS certificate verification is enabled by default >> * ``disable``: ensure HTTPS certificate verification is disabled by >> default >> * ``platform_default``: delegate the decision to the redistributor >> providing >> this particular Python version >> >> If the ``[https]`` section or the ``verify`` setting are missing, or if >> the >> ``verify`` setting is set to an unknown value, it should be treated as if >> the >> configuration file is not present. >> >> >> Example implementation >> ---------------------- >> >> :: >> >> _cert_verification_config = '/etc/python/cert-verification.cfg' >> >> def _get_https_context_factory(): >> # Check for a system-wide override of the default behaviour >> context_factories = { >> 'enable': create_default_context, >> 'disable': _create_unverified_context, >> 'platform_default': _create_unverified_context, # For now :) >> } >> import ConfigParser >> config = ConfigParser.RawConfigParser() >> config.read(_cert_verification_config) >> try: >> verify_mode = config.get('https', 'verify') >> except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): >> verify_mode = 'platform_default' >> default_factory = context_factories.get('platform_default') >> return context_factories.get(verify_mode, default_factory) >> >> _create_default_https_context = _get_https_context_factory() >> >> >> Security Considerations >> ----------------------- >> >> The specific recommendations for the backporting case are designed to >> work for >> privileged, security sensitive processes, even those being run in the >> following >> locked down configuration: >> >> * run from a locked down administrator controlled directory rather than a >> normal >> user directory (preventing ``sys.path[0]`` based privilege escalation >> attacks) >> * run using the ``-E`` switch (preventing ``PYTHON*`` environment >> variable based >> privilege escalation attacks) >> * run using the ``-s`` switch (preventing user site directory based >> privilege >> escalation attacks) >> * run using the ``-S`` switch (preventing ``sitecustomize`` based >> privilege >> escalation attacks) >> >> The intent is that the *only* reason HTTPS verification should be getting >> turned off system wide when using this approach is because: >> >> * an end user is running a redistributor provided version of CPython >> rather >> than running upstream CPython directly >> * that redistributor has decided to provide a smoother migration path to >> verifying HTTPS certificates by default than that being provided by the >> upstream project >> * either the redistributor or the local infrastructure administrator has >> determined that it is appropriate to override the default upstream >> behaviour >> (at least for the time being) >> >> Using an administrator controlled configuration file rather than an >> environment >> variable has the essential feature of providing a smoother migration >> path, even >> for applications being run with the ``-E`` switch. >> >> >> Combining the recommendations >> ============================= >> >> If a redistributor chooses to implement both recommendations, then the >> environment variable should take precedence over the system-wide >> configuration >> setting. This allows the setting to be changed for a given user, virtual >> environment or application, regardless of the system-wide default >> behaviour. >> >> In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is >> defined, and >> set to anything *other* than ``'0'``, then HTTPS certificate verification >> should be enabled. >> >> Example implementation >> ---------------------- >> >> :: >> >> _https_verify_envvar = 'PYTHONHTTPSVERIFY' >> _cert_verification_config = '/etc/python/cert-verification.cfg' >> >> def _get_https_context_factory(): >> # Check for am environmental override of the default behaviour >> config_setting = os.environ.get(_https_verify_envvar) >> if config_setting is not None: >> if config_setting == '0': >> return _create_unverified_context >> return create_default_context >> >> # Check for a system-wide override of the default behaviour >> context_factories = { >> 'enable': create_default_context, >> 'disable': _create_unverified_context, >> 'platform_default': _create_unverified_context, # For now :) >> } >> import ConfigParser >> config = ConfigParser.RawConfigParser() >> config.read(_cert_verification_config) >> try: >> verify_mode = config.get('https', 'verify') >> except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): >> verify_mode = 'platform_default' >> default_factory = context_factories.get('platform_default') >> return context_factories.get(verify_mode, default_factory) >> >> _create_default_https_context = _get_https_context_factory() >> >> >> Copyright >> ========= >> >> This document has been placed into the public domain. >> >> >> -- >> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com >> >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com